Predictive value of pulse oximetry for mortality in infants and children presenting to primary care with clinical pneumonia in rural Malawi: A data linkage study

Summary

Background

The mortality impact of pulse oximetry use during infant and childhood pneumonia management at the primary healthcare level in low-income countries is unknown. We sought to determine mortality outcomes of infants and children diagnosed and referred using clinical guidelines with or without pulse oximetry in Malawi.

Methods and findings

We conducted a data linkage study of prospective health facility and community case and mortality data. We matched prospectively collected community health worker (CHW) and health centre (HC) outpatient data to prospectively collected hospital and community-based mortality surveillance outcome data, including episodes followed up to and deaths within 30 days of pneumonia diagnosis amongst children 0–59 months old. All data were collected in Lilongwe and Mchinji districts, Malawi, from January 2012 to June 2014. We determined differences in mortality rates using <90% and <93% oxygen saturation (SpO₂) thresholds and World Health Organization (WHO) and Malawi clinical guidelines for referral. We used unadjusted and adjusted (for age, sex, respiratory rate, and, in analyses of HC data only, Weight for Age Z-score [WAZ]) regression to account for interaction between SpO₂ threshold (pulse oximetry) and clinical guidelines, clustering by child, and CHW or HC catchment area. We matched CHW and HC outpatient data to hospital inpatient records to explore roles of pulse oximetry and clinical guidelines on hospital attendance after referral. From 7,358 CHW and 6,546 HC pneumonia episodes, we linked 417 CHW and 695 HC pneumonia episodes to 30-day mortality outcomes: 16 (3.8%) CHW and 13 (1.9%) HC patients died. SpO₂ thresholds of <90% and <93% identified 1 (6%) of the 16 CHW deaths that were unidentified by integrated community case management (iCCM) WHO referral protocol and 3 (23%) and 4 (31%) of the 13 HC deaths, respectively, that were unidentified by the integrated management of childhood illness (IMCI) WHO protocol. Malawi IMCI referral protocol, which differs from WHO protocol at the HC level and includes chest indrawing, identified all but one of these deaths. SpO₂ < 90% predicted death independently of WHO danger signs compared with SpO₂ ≥ 90%: HC Risk Ratio (RR), 9.37 (95% CI: 2.17–40.4, p = 0.003); CHW RR, 6.85 (1.15–40.9, p = 0.035). SpO₂ < 93% was also predictive versus SpO₂ ≥ 93% at HC level: RR, 6.68 (1.52–29.4, p = 0.012). Hospital referrals and outpatient episodes with referral decision indications were associated with mortality. A substantial proportion of those referred were not found admitted in the inpatients within 7 days of referral advice. All 12 deaths in 73 hospitalised children occurred within 24 hours of arrival in the hospital, which highlights delay in appropriate care seeking. The main limitation of our study was our ability to only match 6% of CHW episodes and 11% of HC episodes to mortality outcome data.

Conclusions

Pulse oximetry identified fatal pneumonia episodes at HCs in Malawi that would otherwise have been missed by WHO referral guidelines alone. Our findings suggest that pulse oximetry could be beneficial in supplementing clinical signs to identify children with pneumonia at high risk of mortality in the outpatient setting in health centres for referral to a hospital for appropriate management.

Keywords:

Death rates – Outpatients – Medical risk factors – Inpatients – pneumonia – Oxygen – Malawi – Global health

Introduction

Pneumonia remains a leading cause of death in children under 5, especially in low-income and middle-income countries (LMICs), with around 800,000 pneumonia-related deaths a year globally [1]. Incidence of clinical pneumonia is estimated to be as high as 500 episodes per 1,000 child-years in some regions, with an average of 122 episodes per 1,000 child-years in Africa in 2017 [2]. Although hospitalisation rates are increasing and hospital case fatality rates are decreasing, case fatality rates are still typically around 3%–5% in LMICs [3]. Many deaths still occur at home after care seeking [4].

Early identification and action is required to prevent more pneumonia-related deaths that currently occur in hospital, often because of late presentation [5], and at home, often because their illness severity is unrecognised by primary care healthcare providers or there are barriers to accessing secondary care.

As an objective measurement of physiological illness severity, noninvasive peripheral oxygen saturation (SpO₂) measurement by pulse oximeters at outpatient primary care and first level health facilities has a potential role to aid early recognition and referral of severe pneumonia episodes for oxygen and injectable antibiotic treatment [6, 7]. The current 2014 World Health Organization (WHO) Integrated Management of Childhood Illness (IMCI) chart booklet includes pulse oximetry as optional rather than mandatory and stipulates a threshold of <90% to indicate hypoxaemia requiring immediate referral to hospital [8]. However, SpO₂ < 93% has also been shown to predict fatality in high-mortality settings [9, 10].

Evidence linking outpatient primary care SpO₂ measurement to hospital referral and fatality is lacking [11, 12]. We aim to address this gap by linking data from previously described community and hospital based morbidity and mortality surveillance studies [9, 13–16] with a concurrent study in the same district involving outpatient pulse oximeter use at the community health worker (CHW) and health centre (HC) levels [6]. We assess the potential added value of outpatient pulse oximetry to hospital referrals and mortality outcomes using the current <90% SpO₂ threshold and a <93% threshold, in conjunction with pre-2018 Malawi guidelines in use at the time of data collection (hereafter ‘Malawi guidelines’) [17]. Additionally, we explored the theory that the inability to obtain a pulse oximetry measurement may be associated with mortality [18]. Malawi guidelines for HC cases, unlike the WHO IMCI 2014 chart booklet, included mandatory hospital referral for 2- to 59-month–old children with chest indrawing (i.e., bilateral inward pulling of the lower anterior subcostal tissue during inspiration). For CHW cases, both WHO integrated community case management (iCCM) [19] and Malawi guidelines [17] indicate referral of infants and children with chest indrawing or danger signs (Table 1). This study was agreed on after an exploratory meeting by WHO on IMCI danger signs for pneumonia [12].

We hypothesised that pulse oximetry use for outpatients could identify children at risk of dying who would not be identified by clinical signs alone.

Methods

Our objectives were to determine whether pulse oximetry at outpatient CHW and HC primary care levels identifies infant and child pneumonia patients for referral to hospital independently of clinical signs included in the Malawi and WHO guidelines for CHW and HC patients (Table 1) and the fatality outcomes at 30 days postdiagnosis. To determine fatality, we linked CHW and HC outpatient pneumonia data sets of 0- to 59-month–olds in Lilongwe and Mchinji districts, Malawi (Fig 1) from 1st Jan 2012 to 30th June 2014 [6, 13] to hospital [13] data for the same time period. We also linked the outpatient data to community-surveillance mortality data available for the Mchinji district only [14]. We developed and then followed our prespecified analysis plan (S1 Appendix) as far as we were able given the limitations of the final matched data set. We added the sensitivity and specificity analyses at the request of the statistical reviewer. This study is reported as per the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline (S1 Checklist).

Setting

In the Malawi healthcare system, children are intended to access care at either village clinics or HCs. At the village clinic level, if the child is found to be referral eligible, then the child is expected to be referred by the CHW to either the HC or hospital. At the HC level, children meeting referral criteria are referred to hospital.

Sample characteristics

Children were aged 0–59 months with a clinical pneumonia diagnosis according to routine data prospectively collected by 38 CHWs and providers from 18 HCs in rural Lilongwe and Mchinji districts, Malawi (Table 1) [6].

Pulse oximetry measurements

Healthcare providers underwent a 1-day training in pulse oximetry, medical record keeping, and the definition of pneumonia at the start of the study period; they had continued support through monthly mentorship visits. SpO₂ measurements were taken using the Lifebox device (Acare Technology, Xinzhuang, Taiwan, China), with a universal adult clip probe applied to the child’s big toe if less than 2 years of age or below 10 kg. Otherwise, for older or heavier children, providers were instructed to use either the big toe or an appropriately sized finger. A paediatric probe was not available during this time period. CHW and HC workers were trained and retrained in the use of pulse oximetry by a paediatric pulmonologist (EDM) as described by McCollum and colleagues [6]. Providers were trained to record measurements that demonstrated consistent plethysmography waveforms along with a stable, nondrifting SpO₂ and age-appropriate pulse rate. Given this work was conducted within a routine clinical context, providers were not required to repeat measurements meeting these quality criteria. This study showed moderate agreement in sequentially obtained SpO₂ readings between EDM and each cadre of health workers [6].

Matching

Matching of the CHW and HC data sets with the hospital and population mortality surveillance outcome data was done using the following parameters: child name, caregiver or parent name, age at known date or date of birth, address, and PCV13 vaccination dates, using a probabilistic algorithm (S2 Appendix, pp. 1–3). We compared outpatient pneumonia episodes successfully matched to 30-day fatality data to those remaining unmatched to assess the representativeness of the matched sample.

Analysis

We constructed sets of 6 mutually exclusive and complete groupings of episodes according to whether they met SpO₂ thresholds for referral or failed attempted SpO₂ measurement (defined as no stable reading after 5 minutes of measurement [6]), clinical referral criteria, both, or neither (Table 2).

We described the distribution of deaths in the matched data set and crude differences in fatality for each of the 6 groupings in the SpO₂ and clinical guidelines exposure sets (Table 2). We determined the independent associations of SpO₂ and danger sign exposures on fatality, using generalised linear models (GLMs) of the binomially distributed binary outcome, with a log link (Eq 1); these are analogous to logistic regression but produce Risk Ratios (RRs), which are easier to interpret than the odds ratios produced by logistic regression [20]. We ran GLMs for the matched CHW and HC data separately, i.e., for each of the 6 exposure sets. The base-case unadjusted model using <90% SpO₂ and Malawi guidelines clinical referral criteria thresholds (Model M90, see Table 3), is

where μ_i is the probability of death for individual i, Y₁ is the outcome, death, for individual i, and the exponent of β₁ is the modelled parameter of interest: the relative risk of death when SpO₂ is measured at <90% (X_{1_1} = 1) compared to when it is measured at ≥90% (X_{1_1} = 0). Children whose SpO₂ reading failed are separately categorised (X_{1_1} = 2), not shown for simplicity), controlling for presence of Malawi guidelines clinical signs (X_{2_1}). We constructed separate models for SpO₂ < 93% (X_{1_2}) (Model M93) and, for HC data for which WHO guidelines are different from Malawi guidelines, for WHO (X_{2_2}) guidelines (Models W90 and W93; see Table 3).

We adjusted for confounding by age, sex, respiratory rate, and, in analyses of HC data only, Weight for Age Z-score (WAZ). Missing data prevented us from including maternal age, education, marital status, and wealth quintile as potential confounders. Too few deaths in each exposure group prevented assessment of effect modification by age group, sex, or CHW or HC level. Although we adjusted for clustering of illness episodes by child and CHW and HC catchment area, these models were unstable and not presented. Our unadjusted models were robust to clustering, with similar headline results following 100 iterations for models that did not converge.

The extent of missing data on outcomes due to the majority of outpatient episodes remaining unmatched to 30-day postdiagnosis survival (mortality) outcomes (Table 1) meant that multiple imputation of the missing outcome data was not feasible. The small numbers of deaths and episodes with low SpO₂ also precluded our planned regression discontinuity analyses of the effect of changing the SpO₂ threshold on mortality and referral outcomes.

Separately to the fatality outcome, we determined the association between our SpO₂/clinical sign exposures and hospital referral as the outcome (Y₂) using the same logistic regression Eq (1) except substituting the fatality outcome (Y₁) with Y₂. Because not all severely ill children referred to hospital actually arrive, as a sensitivity analysis, we repeated this analysis with referral decision from the outpatient exposure data set regardless of actual hospital arrival as the outcome (Y₃).

We calculated the sensitivity, specificity, and diagnostic odds ratio (DOR, with 95%CI) [21] of clinical and SpO₂ eligibility on the mortality outcome and compared it to the sensitivity, specificity, and DOR of clinical eligibility only for SpO₂ eligibility thresholds of <90% and <93% SpO₂ (WHO and Malawi eligibilities) at CHW and HC levels. We included SpO₂ failed measurements as well as SpO₂ below threshold as SpO₂ eligible in these analyses, given failed SpO₂ measurements are associated with mortality.

Ethics statement

Because data were deidentified and analysed anonymously, no authorisation or waiver of authorisation by patients for the release of individually identifiable protected health information was required. This study is a data linkage study, and the data it links together are from studies approved by the ethics boards of University College London (protocol 2006/002), the Malawi National Health Sciences Research Committee (protocols 941 and 837), and the London School of Hygiene & Tropical Medicine (protocol 6047), as detailed in the published research articles from the original studies [6, 9, 13–16].

Results

We successfully matched 417/7,357 (5.7%) CHW diagnosed pneumonia episodes and 695/6,456 (10.8%) HC pneumonia episodes to outcome data (Fig 2). Sixteen (3.8%) of the 417 CHW patients and 13 (1.9%) of the 695 HC patients died within 7 days of diagnosis. There were no deaths between 8 and 29 days after the initial assessment. A further 9 CHW and 17 HC children were recorded as dying between 30 and 1,537 days after outpatient diagnosis and are included in the survival at 30 days groups. There were up to 7 pneumonia episodes in the same child in both CHW and HC data sets combined (mean: 1.2, SD: 0.6), with 1,112 episodes amongst 931 children, though only 36 children had multiple care-seeking episodes, and 32 of these had multiple episodes within 30 days of each other (S2 Appendix Table A). These episodes are all retained in the analysis given the relevance of each opportunity for case management. SpO₂ < 90% hypoxemia was more common in HC pneumonia episodes (608/6,459 = 9.4%) than CHW episodes (88/7,358 = 1.2%, Table 1).

There were some differences in exposures (SpO₂ measurements, chest indrawing, danger signs) and potential confounders (age, sex, respiratory rate, WAZ) between matched and unmatched episodes (Table 1). CHW patients with matched outcome data were less likely to have SpO₂ recorded and less likely to have saturations below 93% when recorded (though there was no significant difference in the proportions with saturations below 90%; Table 1). CHW patients with outcome data were less likely to have been recorded as vomiting everything and having convulsions, and consequently as having danger signs eligible for referral according to WHO and Malawi guidelines (which are the same at CHW level), than CHW patients not matched to outcome data (Table 1), though they were more likely to have been recorded as having severe acute malnutrition. CHW patients with matched outcome data were also more likely to be male (53%) and younger (mean age 10 months old) than those not matched to outcome data (48% male, mean age 24 months old).

HC patients with matched outcome data were not found to have any significant differences in SpO₂ categories compared with those without matched outcome data (Table 1). HC patients with outcome data were more likely to have been recorded as HIV exposed/infected and of being apnoeic, and consequently (though there were no other significant differences across the danger signs) as having danger signs eligible for referral according to WHO guidelines (16%), than HC patients not matched to outcome data (11%; Table 1). HC patients with matched outcome data were more likely to have chest indrawing recorded (35%) than those unmatched to outcome data (25%) and consequently more likely to have Malawi guideline danger signs eligible for referral (40%) than those unmatched to outcome data (29%; Table 1). HC patients with matched outcome data were also more likely to be male (56%), were younger (mean age 8 months old) than those not matched to outcome data (52% male, mean age 16 months old), and were less likely to have low WAZs, on average (Table 1).

Table 2 CHW data show the distribution of the 16 deaths at CHW level and risk of death for each of the 2 SpO₂ and danger sign exposure group sets. Those who were not clinically or SpO₂ eligible for referral had lower mortality rates than those SpO₂-eligible only (3.6%–3.7% versus 11%–25%).

Table 2 HC data show the distribution of the 13 deaths at HC level and risk of death for each of the 4 SpO₂ and danger sign exposure group sets. In all 4 scenarios, those who are not clinically or SpO₂ eligible for referral had lower mortality rates than those both clinically and SpO₂ eligible (0.5%–0.8% versus 7.9%–12.5%). SpO₂-eligible only episodes had higher mortality than clinically eligible only episodes when applying WHO IMCI chart booklet criteria, especially using the <90% SpO₂ threshold (7.3% versus 3.0%).

For HC episodes, SpO₂ < 90% and <93%, respectively, identified 3 (23%) and 4 (31%) of the 13 deaths that would not have been identified using WHO IMCI guidelines alone (Table 2). Malawi guidelines, which include chest indrawing, identified all of these deaths, and 10 (77%) of the 13 HC deaths in total compared with 5 (38%) of the 13 HC deaths identified by WHO IMCI guidelines (Table 2).

For CHW episodes, SpO₂ < 90% and <93% both identified 1 (6%) of the 16 deaths that would not have been identified using iCCM/Malawi guidelines alone (Table 2). Twelve (75%) of the 16 deaths amongst CHW episodes were not identified by iCCM guidelines, which only identified 2 (12.5%) of the 16 deaths (Table 2). These 12 deaths also had SpO₂ ≥ 93%. Failed SpO₂ measurement independently identified 1 (6%) of the 16 CHW deaths and 1 (8%) of the 13 HC deaths (Table 2).

The unadjusted GLM analyses of the associations of SpO₂ and danger sign exposures with death predict the observed mortality rates for each exposure group. In CHW episodes (Table 3), there is an increased risk of death for those with SpO₂ < 90% than those with SpO₂ ≥ 90%, independent of the presence of danger signs (RR: 6.85, 95% CI: 1.15–40.9, p = 0.035). The SpO₂ < 93% threshold was not associated with an increased risk of death compared to those with SpO₂ ≥ 93% (RR: 3.00, 95% CI: 0.44–20.7, p = 0.264)

In HC episodes, only 2–4 of 13 deaths occurred in the ‘reference’ SpO₂ ≥ 90%/93% no-danger–signs groups (Table 2), and both the SpO₂ < 90% (RR: 9.37 [95% CI: 2.17–40.4]) and <93% (RR: 6.68 [95% CI: 1.52–29.4]) thresholds are significantly associated with mortality in both the Malawi and WHO IMCI guideline models (Table 3).

Adjusting the models for age in months, sex, respiratory rate, and, for HC data, WAZ produced broadly similar results to the unadjusted models in Table 3 (S2 Appendix Table B and associated text).

Table 4 shows the outpatient referral decision indication by healthcare providers and linked hospital inpatient records for each of the SpO₂ and danger sign exposure sets. For CHWs, only 9.3% (39) of the 417 episodes had outpatient referral decisions, and these were more common in patients who were clinically or SpO₂ eligible (33.3%–100%) than those who were not (3.6%–7.0%). Only 0.7% (3) of the CHW episodes were found to be hospital inpatients within 7 days of outpatient diagnosis (Table 4), precluding regression analysis of this outcome in CHW episodes. In the HC data (Table 4), 30.4% (211) of the 695 patients had an outpatient referral decision. Notably, for Malawi guidelines, including <90% SpO₂ threshold (i.e., what was used by the healthcare workers in practice), healthcare workers referred 90.6% who were eligible by both criteria, 62.3% who were clinically eligible only, and only 20.0% of those eligible because of SpO₂ < 90% only. Seventy (10.1%) of the 695 HC episodes were hospital inpatients within 7 days; 60 of these followed an outpatient referral decision (28% of the 211 referral decisions were admitted as inpatients within 7 days).

Table 5 CHW data show both low SpO₂ and danger signs are associated with outpatient referral decisions for CHWs. Table 5 HC data show the same for HC episodes and that failed SpO₂ readings were also associated with referral decisions in WHO guideline models. Referrals were also associated with low SpO₂ under SpO₂ < 90% and <93% thresholds and danger signs according to both Malawi and WHO IMCI protocols in HC episodes (Table 5). The interaction terms between the SpO₂ and danger sign parameters were below 1 and statistically significant for the following HC referral models (Table 5): M90 and M93, outpatient referral decision indication outcome, SpO₂ < 90% × danger signs; hospitalisation outcome, failed SpO₂ × danger signs; and W90, outpatient referral decision indication outcome, SpO₂ < 90% × danger signs. These results indicate that the association between the SpO₂ parameter and the outcome was attenuated by the presence of danger signs. Hospital referrals and outpatient episodes with referral decision indications were associated with mortality (see S2 Appendix, p. 11 for further details).

Table 6 shows the sensitivity, specificity, and DOR of pulse oximetry with clinical signs versus clinical signs only in identifying deaths. We focus on sensitivity because the objective is to identify as many of the deaths as possible. At the CHW level, using both SpO₂ thresholds, the sensitivity was 25% compared with a sensitivity of 12.5% for Malawi clinical iCCM criteria only. At the HC level, both <90% and <93% SpO₂ eligibility thresholds give a sensitivity of 85% compared with a sensitivity of 77% for Malawi clinical IMCI criteria only. At the HC level, in scenarios using WHO clinical criteria, 9 of the 13 deaths are identified by the <90% SpO₂ eligibility threshold, and 10 of the 13 deaths are identified by the <93% SpO₂ eligibility thresholds. These sensitivities of 69% and 77% compare with a sensitivity of 38% for WHO clinical IMCI criteria only. Given the small numbers of deaths, the differences in sensitivity between the scenarios are not statistically significant. The DORs show that at the CHW level, neither pulse oximetry with clinical signs (Table 6: <90% SpO₂ threshold, DOR 1.26 [95% CI: 0.40–4.00] and <93% SpO₂ threshold, DOR 1.17 [95% CI: 0.37–3.71]) nor iCCM clinical signs alone (DOR: 1.33 [95% CI: 0.29–6.05]) accurately identified those who died. At the HC level, pulse oximetry was able to accurately identify those who die with both Malawi (<90% SpO₂ threshold, DOR: 7.13 [95% CI: 1.57–32.4]; <93% SpO₂ threshold, DOR: 6.19 [95% CI: 1.36–28.1]) and WHO (<90% SpO₂ threshold, DOR: 6.57 [95% CI: 2.00–21.6]; <93% SpO₂ threshold, DOR: 6.82 [95% CI: 1.86–25.0]) IMCI clinical signs. Although the point estimates for these DORs are higher than those for when Malawi and WHO IMCI clinical signs alone are used without pulse oximetry (Malawi: DOR 5.25 [95% CI: 1.43–19.2), WHO: DOR 3.43 [95% CI: 1.10–10.7]), the confidence intervals are wide and overlapping, indicating these differences are not statistically significant.

Discussion

We show the important role of confirmed hypoxaemia in identifying otherwise unrecognised fatal childhood pneumonia episodes accessing primary healthcare centres by applying the 2014 WHO IMCI clinical signs, which do not include chest indrawing as a referral sign, in children aged 2–59 months old. Pulse oximeter use will identify hypoxaemia in up to 30% of episodes that are not identified by clinical signs alone [22]. The 2014 WHO IMCI recommends use of pulse oximetry when available and referral to the hospital if SpO₂ is less than 90%; however, pulse oximetry is not frequently available at the outpatient level. With low-cost portable pulse oximeters becoming much more available compared to 2014, especially in the context of the ongoing COVID pandemic [23], it is likely that they will be available more widely and used more often at the outpatient level than before. Even so, it will further facilitate pulse oximetry use if WHO were to recommend them without any conditions to evaluate pneumonia episodes.

Pre-2018 Malawi guidelines for community and outpatient case management [17] were consistent with the previous WHO guidelines [19, 24], i.e., chest indrawing is considered a referral criteria. This criteria was removed as a referral sign in the revised 2014 WHO IMCI chart booklet [8] following controlled low-mortality trials that indicated chest indrawing episodes could be managed at home without increased risk [25, 26], and Malawi made this change too in 2018. Since then, 3 other randomised trials—one each in India, Kenya, and Malawi—have demonstrated that chest indrawing pneumonia can be treated with oral amoxicillin on an outpatient basis [27–29]. However, for CHWs, chest indrawing remains a referral sign. The pre-2018 Malawi IMCI protocol identified 2-fold more (10/13 = 77%) of the HC patients who died within 7 days of outpatient diagnosis than the 2014 WHO IMCI protocol (5/13 = 38%). These data and recent data from hospitalised children in Kenya showed that in routine care settings, presence of chest indrawing may require referral in some high-mortality settings [30]. Because not much is known about other factors that influenced adverse outcomes in these children with chest indrawing, such as care seeking and appropriate and timely treatment, further research is required to examine the beneficial effects of hospitalisation of children with chest indrawing in routine settings.

An abnormal SpO₂ measured by pulse oximetry most commonly indicates ventilation–perfusion mismatch from a pulmonary illness and is considered a more objective measurement than observing chest indrawing, which can be subjective and hard to see [31]. Therefore, although chest indrawing identified all of the deaths from HC episodes and all but one of the deaths from CHW episodes identified by pulse oximetry, pulse oximetry may lead to more reliable decision-making. However, pulse oximetry can also be done poorly, has cost implications, and still needs to be conducted in conjunction with a thorough clinical assessment. Additionally, a ‘normal’ SpO₂ reading should be considered an adjunct to, not in lieu of, recognition of clinical danger signs such as inability to drink when making decisions about hospital referral. Whilst pulse oximetry in conjunction with clinical signs can identify more fatal pneumonia episodes than clinical signs alone, this increased sensitivity of diagnosis comes at the price of reduced specificity (Table 6) and therefore has potential to overwhelm weak hospital systems via identifying additional patients for referral to hospital. Detailed understanding of local hospital system capacities will be fundamental when deciding the appropriate SpO₂ threshold for referring patients. Ultimately, further research is needed to examine these trade-offs, though our research indicates that both pulse oximetry and chest indrawing are important in a high-child–mortality setting.

Notably, 12 (75%) of the 16 deaths in CHW episodes that occurred within 7 days of diagnosis were neither hypoxaemic nor clinically eligible for referral highlighting the importance of follow-up and continuing case management, as well as more specific diagnostic approaches (S2 Appendix, pp. 11–12 has further detail on these episodes). CHWs may also have missed danger signs, meaning support for training, supportive supervision, and mentoring for iCCM may be important in this context [32]. The use of adult probes for pulse oximetry in our study may have contributed to missed hypoxaemic episodes. The increasing availability of paediatric probes may provide greater sensitivity to detect children who may die of pneumonia.

The CHW and HC workers in our study were instructed to follow the Malawi guidelines and <90% SpO₂ threshold for identifying outpatient episodes for referral. Nevertheless, we found that SpO₂ readings of <90% and <93% and, at the HC level, both WHO and Malawi clinical danger signs were associated with a healthcare provider decision to refer the child and hospitalisation within 7 days of outpatient diagnosis. Though most of the hospitalisations followed an outpatient referral decision, these represent only 2 (5%) of 39 episodes with an outpatient referral decision at the CHW level and 60 (28%) out of 211 with an outpatient referral decision at HC level. This is likely to be due to a combination of difficulties with transport, finances or family circumstances precluding travel to hospital, or hospitals not admitting referred patients [33, 34]. It could also be due to incomplete hospital inpatient records and issues with matching outpatient episodes to inpatient records, preventing us from knowing of some successful hospitalisations (e.g., referrals to hospitals outside of our study area). When referral is not feasible, prompt treatment at the CHW and HC level with oral antibiotics is required as minimum care, pending injectable antibiotics and oxygen therapy as required.

Outpatient referral decision, and hospitalisations within 7 days of outpatient diagnosis were associated with mortality. All 12 deaths amongst the 73 hospital referrals occurred on the same day of hospitalisation and were from HC episodes (only 3 of the 73 hospitalisations were from CHW episodes). Nine of the 12 hospital deaths were on the same day as outpatient diagnosis and one the following day, suggesting rapid deterioration, delayed care seeking and presentation at the HC, or a combination of all three [4, 35].

At the CHW level, 13 (81%) of the 16 deaths were in patients for whom outpatient HC referral was not recommended, generally reflecting correct guideline application; i.e., the CHW did not record any clinical signs indicating they should be referred. Of these 13 deaths, none were subsequently hospitalised, and all died within 1–5 days of diagnosis by a CHW. This suggests they died at home whilst receiving oral antibiotic treatment. At the HC level, 4 (31%) of the 13 deaths were in episodes not referred to the hospital, although only one met referral criteria. All of these 4 deaths were eventually hospitalised. These episodes all present missed opportunities for intervention at outpatient primary care.

Our findings on referral decision-making are similar to those we previously reported using the whole CHW and HC data sets [6], suggesting our matched sample is representative of the wider sample in this respect. Our finding that significantly more patients with both clinical and SpO₂ eligibility were referred than those who were either only clinically or SpO₂ eligible suggests healthcare workers at the outpatient level considered both pulse oximetry and clinical signs for referral decision-making rather than one or the other. More research to understand how referral decisions are determined when pulse oximeters are available is needed. Training is also required to re-emphasise the importance of following both clinical guidelines and SpO₂ when making decisions to refer patients. Devices with in-built decision support or that are complemented by mHealth tools could play a role in supporting appropriate case management and should be evaluated for both clinical outcomes and changes in care seeking and referral outcomes.

Our main limitation was our ability to only match 6% of CHW episodes and 11% of HC episodes to mortality outcome data. Given the geographical coverage and age differences of the data sets, we were able to match 29% of the CHW episodes and 24% of the HC episodes for which matching was potentially feasible (S2 Appendix, page 3). Although the samples of CHW and HC episodes matched to mortality outcome data had some differences in characteristics from those unmatched to mortality outcome data, the characteristics of our sample and our findings are generalisable to similar LMIC populations likely to be found in high-mortality settings. The matched patients were younger on average than the unmatched patients; the mortality surveillance data only included children born after October 2011, and therefore, older children were only present in the clinical data set. The lower proportions with danger signs and SpO₂ < 93% in the matched CHW data compared with the unmatched CHW data suggest that the matched CHW patients were less sick on average, and therefore, we may have missed some deaths. On the other hand, HC matched patients were sicker on average than the unmatched HC patients, with greater proportions having danger signs, although there were no significant differences in SpO₂ categories (Table 1).

Given the similarity of the unadjusted and adjusted results, the fact the regression models already lack precision due to the small numbers of deaths, and that we were only able to use age, sex, and respiratory rate as confounders, any propensity score matching analysis to balance such potential measured confounders across exposure groups was deemed futile and abandoned. Additionally, given the low matching rates, multiple imputation of missing data was not included because it produced imputations, and consequent regression coefficient estimates, that were too unstable. Analyses of effect modification were also not possible because of the small number of deaths in each category.

It is estimated that the implementation of IMCI enhanced with pulse oximetry may result in 103,000–145,000 pneumonia deaths averted in the 15 highest-burden countries [36], though this modelling study was based on assumptions about associations with mortality [36]. To help place our results in context and aid policy and decision-making, a cost-effectiveness analysis and economic evaluation of outpatient pulse oximetry—based on our results and cost, benefit (disability adjusted life years averted), and population parameters—is forthcoming. Further research looking at an approach in which chest indrawing episodes without other danger signs are referred or not solely based on SpO₂ is also warranted, as is research to evaluate the optimum SpO₂ threshold for action. Similar analyses evaluating pulse oximetry in low-mortality LMICs are necessary.

Pulse oximetry use by health workers at HCs can identify hypoxaemic pneumonia patients who go on to die who would otherwise be missed by current referral guidelines, especially if they do not include chest indrawing and pulse oximeters are unavailable. With low-cost portable pulse oximeters becoming much more available now compared to 2014, the WHO/UNICEF IMCI protocol could recommend pulse oximeters for pneumonia case management, instead of making a conditional recommendation of its use when available. As pulse oximetry use increases in outpatient settings, critical next steps include addressing oxygen availability at clinics and during transportation to hospitals. Our analysis also indicates that timely and appropriate care seeking by families and prompt management by health workers is key to survival of children with pneumonia.

Supporting information

S1 STROBE Checklist [docx]
STROBE checklist for reporting of observational studies.

S1 Appendix [docx]
Prespecified analysis plan.

S2 Appendix [docx]
Additional supporting information and results referred to in the manuscript.