Setting Implementation Research Priorities to Reduce Preterm Births and Stillbirths at the Community Level

Summary

article has not abstract

Introduction

It is estimated that 3.2 million stillbirths occur each year globally, 1 million of which happen during birth [1]. In addition, complications from preterm birth (before 37 completed weeks of gestation) are the leading cause of death for newborns, contributing an additional 1 million or 12% of child deaths [2],[3]. In 2009, more than 200 stakeholders attended the International Conference on Prematurity and Stillbirth convened by the Global Alliance to Prevent Prematurity and Stillbirth (GAPPS, http://www.gapps.org/). The community expert group at the conference included 15 members drawn from technical and funding organizations in addition to program implementers and researchers from around the world (see Acknowledgments section for specific names). In their discussions, the group framed efforts to address preterm and stillbirths within the broader context of maternal–newborn interventions. As most of the evidence supporting these interventions emanates from research projects in controlled settings in specific contexts, the group identified the main challenge being implementing interventions at scale in different contexts. Based on these discussions, the group began a research prioritization exercise for implementation research on community-based maternal-newborn interventions that address prematurity and stillbirths at scale in different contexts. In this paper, we present the results of this exercise.

Methods

A number of research prioritization efforts have recently been applied to various health topics and health system themes [4]–[7]. The GAPPS community expert group chose the methodology proposed by the Child Health and Nutrition Initiative (CHNRI) to systematically list and score research questions. The CHNRI methodology was selected because its conceptual framework [8]–[10] has been used in numerous areas by different national and international organizations [11]–[16] (further information on CHNRI methodology, validity, and potential limitations are discussed in Table S1). The group followed three main stages to derive research priorities (detailed in Box 1). Briefly, guided by the CHNRI methodology the group evaluated 55 research questions against five main criteria:

• Is the research question answerable in an ethical way?

• Does the research question have the potential to reduce the disease burden (due to prematurity and stillbirths)?

• Is it likely that the proposed research would address obstacles to scaling up?

• Would the proposed research attract funding support and national policy attention?

• Would the research results be owned by local actors, including political authorities and elected representatives, health workers, district managers, and communities?

Respondents were 39% women and diverse in terms of regional representation (26% sub-Saharan Africa, 16% Asia, 16% Latin America, 10% Europe, 32% North America). While a substantial number of respondents were based in North America, they all work full-time in developing country contexts. Half of the respondents were based in research institutions, whereas the other half were in charge of implementing programs whether through nongovernmental organizations, UNICEF country offices, or USAID headquarters. Nonrespondents were not significantly different from respondents (Table S2).

Results

The research question that was highlighted as the most important out of all 55 reviewed was “Evaluate ways to reduce the financial barriers to facility births at the community level—e.g., user fee exemptions, emergency loans, conditional cash transfers, transportation vouchers, etc.” Other research questions among the top five prioritized also addressed equity issues (reaching the poor and marginalized), but also behavioral practices and skills (engaging with social norms, identifying prematurity) and service delivery (measuring and maintaining quality of care provided by community health workers). The remaining top ten research questions (Table 1) include other behavioral skills and practices (thermal care and feeding for preterm babies, birth planning), concerns about how to best motivate and compensate community health workers and their supervisors, and different dimensions of making referrals more effective. Congruent with the priority need to measure and maintain quality of care by community health workers, rational drug use by community health workers and community engagement with regard to audits was also listed among the top 25 research questions that received an overall research priority score (RPS) of 0.75 or greater (Table 2).

Table 3 shows the ten research questions that were assigned the lowest RPSs. Several broad policy questions (human resource planning, gender profiles, budget flows, accountability, and monitoring systems) are listed here, along with some questions related to the sequencing of community interventions and one specific question regarding private provider practice (delayed cord clamping). Questions from almost all research avenues were found among the bottom ten research questions, suggesting that no one area was completely discriminated against by the scoring. Furthermore, even these lower-ranked research questions received relatively high RPSs compared to those arising from other CHNRI exercises. The RPS for all 55 questions ranged from 0.86 to 0.56, in contrast to other CHNRI exercises, which have generated RPS ranges from 0.90 to 0.25 [12]–[16]. This suggests that respondents collectively considered all implementation research questions as fairly important.

Research questions did vary in specificity. For example, broad questions such as “evaluate community-based strategies to reach the poor and marginalized” were scored alongside very specific questions like “evaluate ways to provide thermal care and feeding the preterm baby.” Both broad and specific questions were ranked in the top and bottom ten implementation research questions, suggesting that no bias existed against the kind of question asked.

The CHNRI methodology evaluates certain dimensions of each research question according to defined criteria. For example, “Evaluate methods and levels of accountability that can be ensured” was not considered to affect disease burden and “Evaluate ways to ensure delayed cord clamping in deliveries assisted by private providers” was not scored as likely to attract funding support or national policy attention. Among the five criteria, the most discriminative was the one related to disease burden reduction, while the least discriminative was the one regarding ethical answerability.

As mentioned, the relatively high mean scores assigned to questions across all criteria (apart from disease burden reduction) indicate that most of the respondents were fairly optimistic about the value of implementation research questions. Average expert agreement (AEA) ranged from 0.82 to 0.49. Similar to other CHNRI exercises, AEA showed a direct positive association with RPSs, indicating that there was more agreement among experts about what were the priority research questions. This is a property that is inherent to the way AEA is measured: very high or very low RPS scores require high levels of expert agreement, while substantial disagreement among experts will lead to RPS moving closer to a mean value [12]-[16].

To determine whether any systematic bias existed against certain questions due to the profile of the respondent, we analyzed scores for researchers and implementers separately. We found at least a 10% difference in the scoring assigned for 20% of the research questions (Table 4). The 11 questions for which there was a significant difference between researchers and implementers are spread across each research avenue, suggesting no one particular research area was affected by this difference of opinion. In ten out of these 11 questions, implementers ranked the implementation research question as being of higher value than researchers.

Discussion

The top 25 research questions that have been prioritized span a broad range of issues (Table 2). These implementation research priorities address fostering and sustaining specific behavioral skills and practices at the community level, engaging communities in monitoring service delivery through audits, and improving referral. With regard to service delivery, a host of implementation research questions about the management of community health workers, along with the health system supports they require to function, were stressed. Finally, issues of equity, financing, and referral were highlighted, reflecting how community-based approaches cannot be dealt with in isolation from broader health system concerns.

While many of the implementation research priorities identified can be generalized across community-based maternal, newborn, and child health areas, a few distinctions may be particular to this specific exercise. Issues related to referral were present three times within the top 25 research questions. There is little implementation research on linking families from homes to facilities, or referral more broadly, in low-income countries [17]–[19]. While important gains have been made with task-shifting, effective and equitable referral remains vital, because the most serious cases of prematurity and other birth complications cannot be handled at the community level.

Implementation research questions related to community engagement and some other broader policy concerns central to managing health systems, such as human resource planning and monitoring systems, were overall not given high priority by respondents. Nonetheless, even the bottom ten research questions received high RPSs relative to other CHNRI exercises. This could be because the other exercises had more discriminatory criteria or because previous exercises compared different kinds of research (basic science versus implementation research). It may be easier for experts to discern between very different research areas (basic science versus implementation research) than to discern between areas of implementation research, which they may consider to be of relatively similar importance.

In addition, many of the implementation research questions do not by themselves contribute to improved maternal newborn outcomes. Their value comes forth when combined with other implementation issues that together make a more comprehensive and coherent community-based response with linkages to primary health care service delivery. It might therefore be difficult for respondents to think about specific implementation research questions in isolation from their broader social and health systems contexts.

The partiality toward some areas of implementation research could reflect the profile of respondents. A comparison of scoring by implementers and researchers did show some differences—not across any particular kind of research question, but in the direction of the bias, with implementers ranking implementation research questions higher than did researchers. The reasons for this difference among 20% of the questions are not known, but seem to indicate that implementers perceive the results of implementation research to be more powerful if effectively implemented.

While the CHNRI methodology provides a systematic and transparent way to rank research questions that purposefully avoids biases introduced by group dynamics dominated by powerful individuals, it still is a very lengthy process to undertake. Respondents must score 55 research questions according to five criteria that have three subcomponents each, resulting in 825 dimensions to respond to in the spreadsheet. This makes it a complex spreadsheet and likely does not help response rates. Eliciting participation via e-mail alone was not successful—only 42 out of 85 experts responded to the preliminary e-mail. The 42 experts that did express interest reflected a group that was more familiar with the GAPPS conference and had current working relationships with the lead authors who managed the exercise.

Despite these drawbacks, this exercise represents an important collaboration between researchers and program implementers to jointly identify the key implementation research questions vital to improving community-based maternal and newborn interventions that address preterm and stillbirths. The exercise also developed new criteria deemed more appropriate to implementation research, which require further testing and refinement to improve their discriminatory power.

Success in reducing stillbirth and prematurity rates, and in increasing the survival of preterm infants in low-income countries, is strongly dependent on achieving high and equitable coverage with existing cost-effective interventions [20],[21]. Yet coverage of such interventions remains unacceptably low in most countries. For example, across 68 countries with the highest mortality, only 54% of women deliver with a skilled birth attendant and 38% receive a postnatal visit [22]. Furthermore, coverage levels are particularly low among poor and rural families in these countries. Community-based interventions are therefore essential to reach population subgroups whose current access to health facilities is severely limited. The effect of expanding coverage of family and community care to 90% can by itself lead to a 15%–32% reduction in neonatal mortality [22]. Nonetheless, the knowledge gaps around how to sustain these programs at scale in different contexts remain significant.

Conclusion

While important reviews [23]–[28] have helped to spur attention to community-based maternal newborn issues, with intriguing results for specific interventions [29],[30], the implementation research priorities identified in this article will, we hope, help to secure further research attention and financing for this important area. Priority research areas identified include equity concerns (such as removal of financial barriers and responsiveness to the poor and marginalized), specific behavioral skills and practices, and the management of community health workers including referral care. The challenge is now raised; will communities, governments, donors, research institutions, and international organizations respond?

Supporting Information

Attachment 1

Attachment 2

Attachment 3