[Download the PDF report.]

Author: Kaitlyn Hobbs

Review: Morgan Fairless & Mo Putera

Date of publication: December 2023

Research period: Research Training Program 2023

This report was conducted within the pilot for Charity Entrepreneurship’s Research Training Program in the fall of 2023 and took around eighty hours to complete (roughly translating into two weeks of work). Please interpret the confidence of the conclusions of this report with those points in mind.

We are also grateful to the experts who took the time to offer their thoughts on this research.

For questions about the structure of the report, please reach out to leonie@charityentrepreneurship.com. For questions about the content of this research, please contact Kaitlyn Hobbs at kaitlynjhobbs@gmail.com.

Executive summary

With a large burden of preventable and treatable diseases in Sub-Saharan Africa (SSA), mass media offers a means to relieve burdens by increasing awareness of specialised topics and encouraging health-improving behaviours (Roser et al, 2021). Types of media that can be leveraged for this purpose may include radio broadcasting, television shows and series, internet videos, or a combination of several means to deliver widespread messaging. 

In this report, I propose a theory of change for using mass media to increase awareness on and, in turn, behaviours addressing preventable or treatable diseases. I also assess the evidentiary support for this intervention method. I found that while there is ample evidence for reaching large audiences with widespread media, robust support falls short on impacting health outcomes.

Using a geographic weighted factor model, I suggest Tanzania as a potential country for a new intervention, with a focus on increasing health-seeking behaviours that ameliorate neonatal morbidities and mortalities. That said, the application of this intervention method by nonprofits and non-government organisations seems to be over-saturated. Instead, I propose that funding an existing organisation would be more effective than starting a new one.

Abbreviations

MMI: Mass Media Intervention

NGO: Non-government organisation

DMI: Development Media International

FEM: Family Empowerment Media

GW: GiveWell

FP: Founders Pledge

CEA: Cost-effectiveness analysis

BOTEC: Back-of-the-envelope-calculation

CE: Charity Entrepreneurship

NCD: Non-Communicable Disease

DALY: Disability-Adjusted Life Years

WELLBY: Well-Being-Year

SSA: Sub-Saharan Africa

COM-B: Capability, Opportunity, and Motivation for Behavior change

GBD: Global Burden of Disease

FSI: Fragile State Index

RCT: Randomised Control Trial

DHS: Demographic and Health Surveys

1     Background

1.1    Cause area 

With intention to elicit a health-seeking or health-improving behaviour change, mass media messaging is often aimed at individuals who are capable of taking action for themselves or others, such as a caregiver. A caregiver may be formal, like a nurse, or informal, like a family member or guardian. In the context of family planning and child survival, media messaging has typically been aimed at women of reproductive ages (15-49 years of age). Equipping caregivers with accurate health information may increase timely intervention and ameliorate rates of morbidity and mortality. Several programs of this nature already exist and some have reported successful changes in behaviour outcomes.

The purpose of this report is first, to map a generalised landscape of using mass media interventions (MMIs) targeting informal caregivers for relieving health burdens and second, to investigate the feasibility of starting a new charity that implements mass media awareness campaigns in a neglected area. Specifically, Section 3 scopes the application of a new charity and proposes a theory of change; Section 4 summarises evidentiary support for key uncertainties in effective implementation; Section 5 details the geographic weighted-factor model designed for identifying regions in which a new charity may operate; Section 6 highlights where analogous organisations already exist; Section 7 evaluates the cost-effectiveness of a new charity and compares external estimates of current organisations; Section 8 and Section 9 discuss considerations for starting a new charity or funding existing ones, respectively; and Section 10 and Section 11 draw conclusions based on the content detailed throughout the report.

1.2 Previous investigations

Founders Pledge, Charity Entrepreneurship (CE), and another researcher of the CE Research Training Program (RTP) have conducted shallow dives that estimate the promise of MMIs. In brief,

• The Founders Pledge report comments on the variability of MMI effectiveness in study results, among other study design components.
• The CE shallow dive also considers MMIs to be a promising idea and lists neglectedness and assessing mortality with intermediate outcomes as key uncertainties. This sentiment is echoed by the RTP shallow dive, which adds uncertainties in scoping, particularly in identifying a cause-focus for media content. 
• Authors of the CE shallow dive propose addressing under 5 mortality in low SDI countries with a focus on the following diseases (according to 2019 GBD rates):
  • Lower respiratory infections - 350k+ 
  • Diarrheal diseases - 330k+
  • Malaria - 250k+ 
  • Meningitis - 70k+

The CE research team also has thought through behavioural frameworks for MMIs and their expected influence, limitations, and designs. This becomes an important consideration for two assumptions in the theory of change proposed in this report: Assumption 9: Action is taken as a result of being informed/would not be taken without media information and Assumption 10: No barriers inhibit access to health care.

Finally, Givewell (GW) has produced a report reviewing three MMI programs: Population Media Center, PCI Media Impact, and Development Media International. Based on one quasi-experimental design and one randomised control trial (RCT), GW did not find enough evidence to support cost-effectiveness of the investigated interventions. GW points to confounding variables in a quasi-experimental report (Rogers et al, 1999), including two similar programs being aired, and has since received responses from Population Media Center that cites challenges in conducting RCTs for mass media interventions and concerns on overlooking other pertinent studies and strengths in the reviewed study by GW.

2     Theory of change

There are two approaches to developing a theory of change around using mass media to increase caregiver awareness: first, current efforts could be scaled-up in other regions. This approach would involve focussing on age groups currently addressed (mainly children under 5 years of age) but varies geographically. Second, new media content addressing neglected causes contributing to a high burden of disability-adjusted life years (DALYs) could be crafted and either target similar geographies or regions with the greatest need. This approach may target ages less commonly addressed, such as children aged 5-14 years. An algorithm used for scoping the intervention in this report is detailed below.

2.1   Scope of the intervention

Although many alternative approaches to scoping down the focus of MMIs exist, I adopted following algorithm to identify geography, age, sex, and intervention:

In simpler terms, the selected MMI should focus on the disease that has the greatest burden in a particular age group of a country that scored highly in the Geographic Weighted Factor Model (GWFM). I chose this course of steps as it prioritises the scale of health burdens first, then caters a target behaviour to a region’s greatest need.

2.1.1 Geography

To identify where in the world MMIs may be most suitable, I assessed the global scale of disease according to the 2019 Global Burden of Disease database. Africa bears the greatest disease burden (highest disability-adjusted life years in counts and rates per 100,000 people) and generally, low to mid-range health expenditure per capita (Roser et al, 2021). As such, the remainder of this report focuses on assessing countries for implementation within Sub-Saharan Africa (SSA). 

2.1.2 Target population: age and sex

In line with the two approaches highlighted in Section 3 Theories of change, the following age groups were assessed using data from the 2019 Global Burden of Disease database: 0-4 years (both sexes), 5-14 years (both sexes) and females aged 15-49 (see Section 6.1.1 Burden). 

2.1.3 Media content

Type of disease

Mass media for caregiver awareness typically target diseases affecting youth; that said, other burdens resulting in disability-adjusted life years (DALYs) may also be ameliorated through mass media. Worldwide, the total disease burden is shifting from communicable, maternal, neonatal, and nutritional diseases to non-communicable diseases (NCDs) and is noted to be attributed to rising incomes and standards of living. This trend is observed in Africa as well; however, whether or not increased awareness would continue this trend would provide a more comprehensive understanding of importance as this category still comprises over two-thirds of the DALYs per 100,000 people in SSA. For this report, the age group bearing the greatest burden of all-cause DALYs within SSA informed the most suitable focus for media content. 

Type of health advice

Within the context of health, one consideration that influences which disease to focus media content on is whether increased awareness is more effective at prevention or treatment of illnesses. The type of disease would best inform whether prevention or seeking treatment is more suitable. Further research on comparing effects on population health of increased awareness and early identification to disease prevention, family planning programs, maternal health, child development, and nutrition could also provide a more complete understanding of which diseases could have the greatest reduction in DALYs through addressing behaviour change. 

2.2   Causal chain

Colour-coding of certainty level are as follows:

(Green) >85%  Quite certain, evidence review would provide additional strength in support.

(Orange) 50%-85% Fairly uncertain, evidence review would be helpful.

(Red) <50% Quite uncertain, evidence review is necessary.

Solid lines represent a causal chain of primary activities that focus on creating direct change. Dashed lines represent secondary activities, focusing more on monitoring and evaluation of causal chain effects.

2.3   Outlining assumptions and levels of uncertainty 

1. Assumption 1: a burden exists for treatable illnesses in children that can be addressed by improved awareness (that leads to early intervention).
   • This assumes that health outcomes targeted are preventable or treatable, which can be controlled for by selecting a health concern of this nature.
2. Assumption 2: There are child illnesses that would be treatable with early intervention and are not currently being addressed.
   • This seems quite likely given the wide-range of ailments but further research into current players would paint a more complete picture. 
   • If an illness itself seems to be covered and effective in one area, scaling may be the appropriate next step (as opposed to identifying new illnesses to treat).
3. Assumption 3: a region exists that provides the necessary means (media distributors, infrastructure for networking, presence of health practitioners, other yet-to-be-identified needs) to operate.
   • Given Development Media International’s (DMI’s) extensive portfolio of similar interventions, it is highly likely that regions exist with necessary and sufficient conditions to operate.
4. Assumption 4: local health experts and media distributors are willing to collaborate.
   • This is likely country/region-dependent, but again, DMI has a large base and success in Sub-saharan Africa so it is reasonable to expect this is feasible and not of concern. 
5. Assumption 5: It is relatively cheap to design and produce media content.
   • Exact costing will be discussed in Section 7 Cost-effectiveness analysis but initial literature review indicates media broadcasting to have low costs and elicit effective change (Kasteng et al, 2018).
   • Generally, short spot campaigns are more cost-effective and easier to scale-up than longform media (Murray et al, 2015).
6. Assumption 6: Health attendance monitoring system already exists or is easy to develop.
   • Again, this is likely dependent on the country/region of intervention. In urban areas of countries with established health-related infrastructure, this is not expected to be a concern. That said, if the intention is to focus on rural areas, presence of health monitoring systems may be lacking. 
   • Availability of community health centres and workers is also crucial for mass media awareness to posit change in health outcomes.
7. Assumption 7: No barriers inhibit access to media content.
   • The mode of message delivery likely will not reach the entire target population if not everyone owns a radio, mobile phone or other means of communication used to spread awareness. While 100% coverage may not be realistic, maximising coverage is important for efficacy.
   • The region of intervention may inform the means of delivering messaging; for example, in some rural villages, radios may be more widespread than mobile devices. In this case, radio broadcasting would have greater reach.
8. Assumption 8: No barriers impact delivery and understanding of media content.
   • While barriers exist, they can likely be mitigated with forethought while developing and delivering media content. Possible barriers may include but are not limited to: language, simplicity of messaging, holding the attention of listeners, among others.
   • DMI uses a Saturation+ model (e.g. 1 min ads broadcasted 10x per day) to ensure content is received.
9. Assumption 9: Behavioural change occurs as a result of being informed/would not be taken without media information.
   • Engagement can be encouraged by including follow-up radio station phone-ins to allow listeners to discuss and clarify uncertainties from broadcast messages (Sarrassat et al, 2018).
   • Potential cultural, social or religious contexts may pose barriers to seeking medical aid. 
   • Presence of multi-family or multi-generational households versus nuclear households may impact efficacy; it could be easier to reach caregivers or views on targeted behaviour change or media may be more easily influenced by other household members.
10. Assumption 10: No barriers inhibit access to health care.
    • For example, physical access to health care (living close to facilities, being able to take time off work, cost of treatment), health staff and availability of appropriate medications (ie. no drug or staff shortages).
    • Some factors such as proximity to healthcare can be mitigated through regional selection for intervention (for example, choosing to implement in urban areas may likely mitigate risk of uncertainty) while other factors such as quality of care provided may be more difficult to control for. This concern would likely largely reduce the efficacy of the intervention.
11. Assumption 11: Medical intervention and/or treatments are successful.
    • To mitigate uncertainty with this assumption, only illnesses that are treatable by a behaviour change should be considered.
    • If the intended behaviour change involves seeking medical care, it is expected that treatment is provided such that the ailment is nearly fully relieved.
    • Several factors may influence successful intervention: medications are effective, whether the illness is treatable at time of seeking health care, among other considerations. 
12. Assumption 12: Successful medical intervention leads to greater school attendance.
    • There’s evidence of this in initiatives led by Deworm the World.
13. Assumption 13: Greater school attendance increases quality-adjusted life years.
    • Given this assumption is slightly out of scope, it is not prioritised for further evidence review during the time constraint.
14. Assumption 14: No negative unintended (flow-through) effects occur.
    • For example, stigmatisation of individuals exhibiting symptoms described in media, strain on hospitals/healthcare workers. 

3    Quality of evidence

#RTP23H10: (ER) Mass Media for Caregiver Awareness

Overall, evidence on MMIs efficacy in increasing knowledge, changing behaviours, and improving health outcomes of a target population is somewhat promising but could be further substantiated with more statistically powered evaluations. A 2014 systematic review on MMI’s for increasing child survival in low and middle income countries (LMICs) observed a publication bias towards successful interventions, which remains accurate for my review of publications since (Naugle & Hornik, 2014). 

This evidence review is primarily informed by a meta-analysis of contraceptive use in SSA, a systematic review of MMIs targeting family planning and child survival, and a recent RCT conducted on an MMI in Burkina Faso (Babalola et al, 2017; Naugle & Hornik, 2014; Sarrassat et al, 2018). Additional evidence was referenced for comprehensivity. That said, the review conducted for this report is not extensive and likely overlooks other pieces of plausible evidence. As such, it should be kept in mind that the review likely suffers from publication bias and that quantitative methods used vary. 

3.1   Evidence that a charity can make change in this space

Counterfactuals of delivering mass media by a charity 

Value over replacement. Little evidence was found to suggest that a government would invest in mass media for caregiver awareness on its own, although mass-media for health awareness has long since captured the attention of large NGOs like United States Agency for International Development (USAID) who continue to support mass communications for increasing health awareness in LMICs. 

Alternative methods to encouraging health-related behaviours, such as in-person advocacy and knowledge-sharing sessions, are typically more costly and have a lessened expected reach of the target population. MMIs also have the advantage of being available in multiple languages and having control over content.

Reach of mass media

Evidence on target population reach varies depending on country of intervention, but we can generally expect a moderate to high exposure prevalence. A meta-analysis on mass media communication of contraception use in SSA estimated a 44% exposure prevalence, though this ranged drastically depending on country (Babalola et al, 2017). Of the publications reviewed by Naugle & Hornik. (2014), around one-third of the studies included in their review on MMIs addressing child survival reached between 61-100% of the target population. In studies not included in the systematic review, Sarrassat et al. (2018) detected a high proportion of both the target population being reached by radio broadcasts and recognition of media broadcasts by mothers interviewed in intervention groups, suggesting large media uptake. 

Enhancement of knowledge about media content

Mass media has potential to be effective at increasing knowledge of a target population across multiple health topics, countries and intervention strategies but the most efficacious strategy is unclear. The meta-analysis report on contraceptive use in SSA, and systematic review on child survival report a positive effect of mass media on knowledge, attitudes and beliefs (Babalol et al, 2017; Naugle & Hornik, 2014). The meta-analysis reports a large effect size of contraceptive knowledge. In some studies, no effect was found on knowledge of media content in target populations while others did not evaluate this outcome. Studies on child survival MMI’s reviewed by Naugle & Hornik. (2014) included additional interpersonal efforts for knowledge-sharing to complement mass media communication, which would lead to confounding results. The RCT in Burkina Faso did not report on increased knowledge of health practices.

Whether absence of knowledge is the key driver to a disease burden is also dependent on the presence or absence in other health care barriers (see Assumption 10.). 

Eliciting behaviour change

Generally, evidence suggests that mass-media can successfully elicit behaviour changes, although estimations of success rates are subject to a social desirability bias of self-reports and barriers such as the type of behaviour change and physical proximity to health care is expected to change efficacy. Around 81% of the publications on mass media for increased child-survival evaluated by Naugle & Hornik. (2014) reported a positive effect in eliciting the intended behaviour change, according to self-reports. The authors of Sarassat et al. (2018) also note an increase in self-reports of care-seeking and treatment, although this may be subject to socially desirable bias. The RCT in Burkina Faso did not detect a change in self-reported habitual behaviours but did note an increase in health-seeking behaviour (Sarrassat et al, 2018). Still, positive results were limited to households living with 5 km of a health facility. Further, a positive association between knowledge of pneumonia symptoms and care-seeking was observed by Glennerster et al. (2021). Bowen et al. (2013) estimates a 6.6% increase in mosquito bed-net use as a result of mass media (music video, public service announcements, and celebrity spokespeople) in Cameroon according to their propensity score matching model. Though it is intriguing that a habitual change (as opposed to a one-time change) in behaviour was reported, there is concern of social desirability bias of the in-person survey conducted for the analysis and the low power of quasi-experimental design. 

Media uptake and consequential changes in behaviour are further detailed in  Assumption 9. 

3.2  Evidence that the change has the expected effects

Evidence supporting effectiveness of behaviour changes as a result of MMIs is mixed. This is largely attributed to a lack of high-powered studies demonstrating an effect of behaviour change on health outcomes (Naugle & Hornik, 2014). While randomised-control trials (RCTs) provide high-powered insights to causal relations, authors of the 2014 systematic review on MMIs propose their inapplicability for mass-media due to desirability of large scale and long-term campaigns (Naugle & Hornik, 2014). The authors instead suggest combining several methods to evaluate the impact of intervention on behaviour change. Researchers attribute challenges in conducting RCTs on MMIs to difficulty isolating geographies from the reach of radio broadcasts (Murray et al, 2018). Of those that exist, generalising results across causes and interventions remains largely uncertain. 

Expected impact on health outcome

Evidence on the impact of MMIs on child survival is mixed but the most powered study shows no reduction in child mortality and is supported by other studies (Sarrassat et al, 2018). Kasteng et al. (2018) also reports no effect on reduction of child mortality by single-cause MMIs. In contrast, Murray et al. (2018) used the Life-saving Statistical Tool (LiST) to model child and maternal mortality reduction as a result of the clustered randomised trial conducted in Burkina Faso by Sarrassat et al. (2018). Their model suggests a 7.1% and 3% reduction of child and maternal mortality rate, respectively. DMI’s involvement in the study, several listed assumptions in the model inputs, and overall theoretical approach (as opposed to relying on empirical data) reduces the reliability of the study in informing my stance on MMI efficacy. My lack of understanding of LiST’s modelling approach also reduces my ability to accurately assess these results. A quasi-experimental study found that knowledge of pneumonia symptoms did not necessarily lead to increased care-seeking in four out of six Sub-Saharan countries evaluated (Noordam et al, 2017).

Evidence supporting marginally increased contraceptive use appears to be more promising and statistically powered. Glennerster et al. (2021) report a 5.9% increase in contraceptive rates in the treatment group relative to control, according to their RCT. An independent survey conducted in Kano, Nigeria reports a 75% increase in contraceptive use 11 months after Family Empowerment Media’s (FEM) family planning radio campaign began.  Results from the survey are less statistically powered than the RCT results so a marginal increase in contraceptive use holds more evidentiary weight.

Lasting effects

Lasting effects of MMIs on behaviour and health outcomes are inconclusive as their evaluation is neglected in the studies reviewed. MMIs may have lasting behavioural effects beyond the period of intervention that are not evaluated in the reviewed studies. One study from 1990 reported a decrease in knowledge and habitual behaviour change three years after campaigning, but holds weak evidentiary weight given the year of publication (Naugle & Hornik, 2014). 

3.3  Evidence on other key uncertainties

3.3.1 Assumption 9: Behavioural change occurs as a result of being informed/would not be taken without media information. 

In attempts to maximise uptake of media information and behaviour changes, approaches to delivering information can be modified according to the goal and primary reasons for the desired behaviour not being exhibited. Comparative evidence on the efficacy of such approaches was not explored, although several models can be leveraged. Counterfactual estimates on whether a behaviour change would occur without media intervention are also not explored due to time limitations.

Theoretical approaches to mitigate concern

Theoretical models exist that assist in identifying a wide-range of potential barriers; for example, the COM-B (Capability, Opportunity, and Motivation for Behavior change) model is a categorization of constructs that affect behaviour and is detailed in Assumption 10: No barriers inhibit access to health care. Other theoretical models for behaviour change exist but were not explored.

Study Examples

DMI incorporates storytelling of a protagonist either overcoming obstacles through a behaviour change or revising their goals in order to evoke emotional response to media campaigns (Glennester et al, 2021). Other tactics exist and could be explored in depth once the behaviour targeted for change is determined. The Behavioural Insights Team’s (BIT) EAST framework may guide details of the approach.

FEM approached maximising media uptake by profiling their listeners through qualitative analysis (Family Empowerment Media, 2021). They developed six general personas to characterise their audience for targeted messaging.

3.3.2 Assumption 10: No barriers inhibit access to health care.  

Evidence does not support this assumption to hold in practice instead, it highlights several key considerations impacting efficacy of MMIs. For ease of organisation, barriers to accessing healthcare noted in studies are organised according to the Capability-Opportunity-Motivation-Behavior (COM-B) model. COM-B proposes several considerations that may enable or inhibit accessing appropriate health care. It is noted by Mayne (2017), a working paper on the COM-B Theory of Change Model, that not all capability, opportunity, and motivation assumptions need to be addressed in an intervention. Exact considerations may become apparent after an intervention cause and geography is selected.

The following examples of COM-B model components found in publications about MMIs vary in terms of evidential strength. Most barriers were identified through surveys that are specific to the region in which they were conducted and are not expected to be reliably generalised. The Demographic and Health Survey (DHS) in SSA is likely the strongest survey in terms of generalisability given its large sample size and coverage.

Physical capability

Physical access to health care is likely a crucial consideration for efficacy of improved care-seeking and receiving timely health treatments. 

• Survey data taken from Sarrassat et al (2018) suggest that distance to health care facility matters; care-seeking increased among families living within 5 km from a facility, but no further. 
• An assessment on DHS data from 2010-2018 cites household distance to health facility as an influential consideration for women aged 15-49 to seek health care for their child under 5 years of age (Ahinkorah et al, 2021). 

Psychological capability (e.g. knowledge/understanding the consequences of not taking action and the benefits of taking action)

• BIT reported resistance from adolescent females to partake in their sexual reproductive health program due to lack of information and myths about contraception. 

Social opportunity (e.g. pressures from family or community, fear of being judged, perceived norms and expectations.)

• FEM’s 2021 qualitative analysis on their family planning campaigns airing in Nigeria revealed that women feared discussing family planning with their spouse(s), the survey participant’s religion did not support use of contraception, among other barriers surrounding lack of information and fear of side effects. 
• Receiving permission to seek care and level of community literacy were also reported as barriers to health seeking according to the DHS data analysis.

Physical opportunity (e.g. cost and availability of intervention method(s).)

• Eligibility for health-care coverage was cited by Harris et al. (2011) as a source of inequality in access to healthcare in South Africa.
• Melberg et al (2016) cite costs, limited decision-making power for women, and physical distance to health as several barriers to seeking obstetric care for deliveries (Melberg et al, 2016). 
• Receiving costs for health care was also listed as a significant barrier for women of reproductive ages to seek medical aid (Ahinkorah et al, 2021).

Reflective and automatic motivations (challenges presented by behaviours that involve evaluation and planning, e.g. belief systems; challenges presented by behaviours that involve emotions or impulses, e.g. habits and desires.)

• Whether the behaviour desired by an MMI is one-time, episodic (such as routine vaccinations), or habitual may also impact efficacy of behaviour changes. The DHS results evaluated by Ahinkorah et al. (2021) mentions women not wanting to go to seek medical help alone as a significant factor affecting likelihood of seeking help.

3.3.3 Assumption 15: No negative unintended (flow-through) effects occur.

Little evidence was found on unintended effects from mass media campaigning within the studies reviewed. Nonetheless, possible negative effects can still be considered. Some considerations include: a heightened sense of anxiety or other emotional dysregulations due to either tone of messaging or increased knowledge of harmful consequences to inaction, misinterpretation of messaging or spread of misinformation, and rise of social judgements as a result of campaigning. Finding quality evidence on this and other potential flow-through effects were not prioritised with the time spent on this report.  It’s possible that these considerations could be mitigated through strategic content design and delivery. With more time, expanding a search for evidence on unintended effects in other subject areas, such as the psychological impacts of widespread health information during COVID-19 (Giri et al, 2021), could serve as proxies for this application.

4     Geographic and cause assessment

#RTP23H10: (GWFM) Mass media awareness for caregivers

Neonatal disorders in Tanzania was the top candidate for mass media intervention according to the geographic and cause assessment. The following weighted geographic assessment evaluates country-level suitability for a MMI in SSA. It considers the scale of disease burden as well as approximations of neglectedness and tractability. Weights of each variable reflect both the quality of data used and uncertainty associated with proxies. 

Section 4.1 overviews the general approach to evaluating countries, including Burden, Tractability, and Neglectedness. Top candidate countries and their disease burdens are reported in 4.1.4 Results for children under 5 (both sexes), children 5-14 years (both sexes) and females ages 15-49 years. The cause of the greatest DALY burden for each country informs which health conditions to target in the intervention. I advise spot-checking top candidate regions by considering whether a country has experienced a coup d’etats as it may impact media control, cooperation and overall tractability.

4.1 Geographic assessment

I assessed 46 countries in SSA using a geographic weighted factor model (GWFM). The total score for a given country was given by:

Where,

For ease of interpretation and comparisons, the total score was normalised using z-scores.

Table 1. GWFM parameters and their associated weights and normalisation method.

4.1.1 Burden

The rate of all-cause DALYs per 100,000 population was assessed to identify burden. DALYs also include years lost due to death, making it a more comprehensive indicator of disease-related burden; however, this analysis could be repeated with mortality rates only, if death-aversion is more heavily valued.

DALYs per 100,000 were assessed for communicable diseases, non-communicable diseases, and nutritional disorders. The category “other non-communicable diseases” was excluded from the comparisons because it was an aggregate of several smaller diseases. Disaggregating this category could be done with more time in the future.

4.1.2 Tractability

I estimated tractability by averaging several indicators of connectivity and adding WHO’s Health centres per 100,000 indicator and the Fragile State Index.

4.1.2.1 Degree of Connectivity

Since the means of delivering media content could vary (for example, using radio, internet, or mobile messaging), several variables encompassing multi-modal delivery were considered: 

1. Percentage of individuals owning a mobile device
2. Percentage of individuals using the internet
3. Percentage of households with a radio

Due to incomplete data on individual and household usage, the maximum value across all three variables was used as a proxy for connectivity. 

A potential spillover effect from living in an intervention region was proxied using the percentage of inhabitants within range of at least a 2G signal. This comprised only 10% of the connectivity estimate (5% of the total tractability estimate) due to uncertainty of the expected effects.

4.1.2.2 Health Centres per 100,000

Is defined as “Number of health centres from the public and private sectors, per 100,000 population” and serves as a proxy for health service availability. Health centres were chosen instead of hospitals as they typically offer services that are non-urgent, although this may vary by country.

It is expected that this is somewhat an underestimate of health service infrastructure availability. With additional time, additional metrics for density of other health infrastructure and services or quality of care could be included to improve representation of access to adequate health care. 

4.1.2.3 Fragile State Index

The Fragile State Index (FSI) represents the general volatility of a nation based on several indicators on social, political, economic, and cohesion.

4.1.2.4 Data Vintages & Quality

Data was taken from the International Telecommunications Union (averaged from 2018-2022), World Bank Development Indicators (2021), WHO's Health centres per 100,000 (2013), and the Fragile State Index (2023). Changes that occurred past the data vintage year may not be captured.

4.1.2.5 Missing data

• Health centre densities were imputed with the median where values were missing.
• Connectivity: If none of Percentage of individuals owning a mobile device, Percentage of individuals using the internet, and Percentage of households with a radio had values, only the spillover variable (percent inhabitants living within range of 2G signal) was used to calculate connectivity but was penalised by a factor of 0.5.
• Spillover: missing data were imputed with 0 (if there is no data, we assume that there is no unexpected reach).  

4.1.2.6 Improvements

Data Sources

• WHO’s 2022 Harmonized Health Facility Assessment (HHFA) indicators would offer more recent and precise indications of health service availability, quality and readiness. The HHFA analysis platform and indicator inventory require permission to access and a brief search did not find the data available elsewhere. It seems as though participation in a 2.5 hour course may provide access; however, this was not explored.
• Data representing women’s household decision-making power would provide some indication on whether a social barrier exists to taking action. From websearch, this seems to be seldomly measured or reported, particularly in LMICs.

Logic/Application

• Health centre density may only be applicable for diseases that require intervention. An MMI promoting prevention of diseases may not need to consider this parameter.
• Assessments could be further broken down by rural-urban divides to better determine geographic area of most need and tractability.
• Apparently, there has been a recent wave of coup d’etats in Africa that doesn’t seem to be captured in the FSI (albeit, the FSI may be an indicator of potential for a coup) but should be considered as it could influence control over media. 

Future Projections

• Population projections could also be incorporated to evaluate where long-term change may occur.

4.1.3 Neglectedness

Neglectedness was inferred by the reduction in DALYs from 2010-2019, reported by GBD. Countries with smaller reductions are considered to be in greater need of support. Neglectedness was normalised by taking the negative log of the absolute value of DALY rate reduced. This ensured that regions with smaller reductions gave a higher total score.

Improvements

• One improvement would be to factor in an estimated number of DALYs reduced by current NGO initiatives:

Reduction in DALYS (from 2010 to 2019) - number of NGOs operating

estimated reduction in DALYs

• A moving-average could also be taken to better capture temporal trends.
• With more time, data could be disaggregated to reflect neglect by cause. WHO offers several indicators that represent coverage such as, Demand for family planning satisfied - modern and traditional methods (%) (Health Inequality Monitor), Children aged < 5 years with diarrhoea receiving oral rehydration salts (%) (Health Inequality Monitor), and Population aged 15-24 years with comprehensive correct knowledge of HIV/AIDS (%).

4.1.4 Results

Table 2. Top 5 countries proposed by GWFM for MMI targeting children under 5 (both sexes). 

Table 3. Top 5 countries proposed by GWFM for MMI targeting youth aged 5-14 years (both sexes). 

Table 4. Top 5 countries proposed by GWFM for MMI targeting women in the reproductive age range (15-49 years). 

Given that the scale of DALYs per 100,000 children under the age of 5 is around 30-fold greater than that of children ages 5-14 and around 10-fold greater than the DALY burden of women aged 15-49, I chose to investigate the greatest cause of the burden of the top country in this category: neonatal disorders in Tanzania.

5 Where existing organisations work

5.1 NGOs

The following list of NGOs operating in LMICs were identified during research but is not expected to be comprehensive:

• Development Media International (DMI) is the key player in this intervention space. Donating to DMI is endorsed on The Life You Can Save, which reports a cost-effectiveness of $196-756 USD per life saved. DMI is recognized as a “stand-out” charity by GiveWell and in 2020, GiveWell granted DMI $200,000 to support COVID-19 broadcast messaging.
• Family Empowerment Media (FEM) founded in 2021, operates similarly to DMI but focusses on radio campaigns relating to family planning (mainly contraceptive use) in Nigeria. Future plans include scaling-up operations and conducting RCTs on their operations. FEM is accepting proposals to expand operations to Chad, Uganda, the Democratic Republic of Congo, and Ethiopia.
• The Behavioral Insights Team (BIT) previously partnered with Children’s Investment Fund Foundation to conduct an RCT on SMS messaging for sexual reproductive health for adolescents in Kenya. It is unclear whether efforts of this nature are a continued area of research.
• Population Media Center shares messages for women in Burundi, Nepal, Nigeria, Uganda, Mexico and other countries. Messages vary from encouraging continued education and family planning to environmental conservation. They use media campaigns, radio, TV shows, and more. Evaluated by conducting local interviews in partnership with stakeholders. 
• PCI Media Impact shares media content focusing on healthy living, environment, and social equality. Their messaging is primarily focused in the Caribbean, South America, South Asia, and Africa but they also include messaging in the USA and Canada, totalling 22 programs in 70 countries. A measure of direct impact of their program is unclear from a brief glance of their website.
• Community Media Trust (CMT) uses media messaging primarily for sexual health and family planning in Africa. Example programs include reducing stigma around HIV in South Africa and medical voluntary male circumcision in Namibia. The organisation has conducted an RCT that reports an increase in antenatal HIV testing but was not reviewed for this report.
• Farm Radio International uses radios to share knowledge and resources for farmers in Sub-Saharan Africa to help improve crop yields and reduce gender inequality in farming. The organisation seems to have large reach and seem  well-funded (based on their extensive list of funding partners).
• BBC Media Action uses messaging in numerous LMICs on a wide-range of concerns including equity, climate change, disaster relief, sexual health and more. Although not explored for this review, BBC Media Action publishes impact reports. Founders Pledge’s report comments that they may be well-funded by the UK government.
• United States Agency for International Development (USAID) has engaged with mass media communications to encourage social and behaviour changes in LMICs for around 40 years, with a particular focus on improving child survival. It seems that USAID implements their own media communication programs, partners with local private companies for campaigns, and supports specialised programs through funding like GirlEngage Niger.

5.2 Local Initiatives

Other local initiatives likely exist and should be investigated depending on region of interest.

5.3 Governments

National Ministries of Health: Health ministries in sub-Saharan African countries may use mass media to raise awareness about various health concerns, but the extent and quality of their efforts was not investigated.

6 Implementation of a new idea: Averting neonatal deaths in Tanzania

According to a Cochrane Review on pregnancy interventions to avoid preterm births (PTBs), a principal cause of neonatal morbidity and mortality, encouraging pregnant women to take folic acid vitamins, zinc, antibiotics for infections, and attending health appointments can reduce PTB (Institute of Medicine (US) Committee on Improving Birth Outcomes, 2003; Medley et al 2018). These examples of health-promoting behaviours are not extensive but may be relatively simple and low-cost to elicit through mass media.

6.1 What does working on this idea look like?

Start-up Activities

First, the charity should identify a region within Tanzania to pilot a mass media intervention. The capital city of Dodoma may be a suitable start as it likely has health and broadcasting infrastructure as well as a high burden but I recommend investigating its suitability further. I also recommend connecting with Nutrition International to discuss their efforts and challenges in transitioning from iron and folic supplements to multimicronutrient supplements and provision to pregnant women in Tanzania.

Next, the charity should identify potential partners and build relationships through contacting the local health networks, governments, and radio broadcasters within a selected region of Tanzania. 

Media content design

The media could either be designed and delivered by radio show hosts in collaboration with health experts while the charity facilitates the collaboration and other operations, or the charity could design the content with heavy input and validation from health experts.

Monitoring and Evaluation

The charity would implement monitoring and evaluation efforts, as overviewed in Feedback loops.

Before mass media is shared, baseline rates of health knowledge, health-seeking and health-improving behaviours should be determined by leveraging current health monitoring or implementing community surveys. Through a community survey, it’s also important to ask about current physical barriers, social beliefs, and other reasons that inhibit engaging in health-improving behaviours. This survey can be completed again at several time points throughout the intervention and be corroborated by rates of health-seeking behaviour recorded by health professionals. Finally, the overall associative effect of mass media on decreasing neonatal morbidities and mortalities can be determined through hospital records on the prevalence of preterm births.

6.2 Key factors 

This section summarises our concerns (or lack thereof) about different aspects of a new charity putting this idea into practice.

Table 6: Implementation concerns.

Talent

Several attributes of charity founders would be helpful, but I do not expect them to be particularly challenging to find. A founder should be personable, open-minded, and possess acumen in managing stakeholder relations and project management would also be valuable.  It would be helpful if new charity founders were generalists with some background in health and/or media communications. Quantitative skills and experience in monitoring and evaluation may also be additional qualities of strong candidates. 

Access

Information

Access to health care monitoring systems is imperative for evaluating impact. This is expected to be of moderate concern as it depends on the following conditions: 

1. Health facilities must agree to data sharing
2. Health facilities must have a reliable monitoring system in place or,
3. It must be relatively easy to improve or establish an adequate monitoring system

Relevant stakeholders

Given the variety of partners (radio broadcasters, health facilities, local organisations, as well as local and/or federal governments), managing stakeholder relations is of high concern and would be a valuable attribute of charity founders.

Feedback loops

Intermediate outcomes like reach, behaviour change and increased knowledge can be investigated by surveys. This is expected to give relatively quick and reliable feedback. Since radio listenership is already monitored by stations, detecting the reach of MMIs is not expected to be a concern. Ensuring that shows are airing may also utilise similar monitoring systems. A cross-sectional survey may also be conducted to detect whether knowledge, attitudes or beliefs towards the health concern have been influenced by the MMIs. 

Measuring causality in health outcomes with confidence may be on the timescale of 1-3 years. While behavioural change may occur shortly after media is delivered, feedback loops for health outcomes may take a year before becoming significantly detectable in studies. The RCT run by Sarrassat et al. (2018) was 3 years long, and another RCT to be run by FEM will take 3 years. That said, the charity does not necessarily need to run an RCT to estimate impact and may instead rely on data provided by health systems on hospital visits and rates on mortality and morbidity.

Funding

According to the RTP CEA on starting a new charity,

• Operational costs would be covered by philanthropic funding 
• Societal costs include:
  • Household (beneficiary) costs to seek treatment/intervention.
  • Costs to the government for increased use of health facilities.

The existence of a grantmaking organisation for MMIs leads me to think receiving fundraising in this field is feasible yet competitive. Media Impact Funders is a grantmaking organisation that maintains a database for funders and MMI programs that includes impact assessment designs and toolkits. They also house a curated collection of impact reports and host a media impact festival. Better insights on grantmaking in MMI is included in the organisation's report on Global Media Philanthropy.

The Bill & Melinda Gates Foundation also may be interested in supporting funding based on their support for Media Impact Funders’ research.

The high estimated cost-effectiveness of DMI and FEM may attract funding from organisations affiliated with the Effective Altruism community, such as GiveWell. For scaling DMI, the primary concern is supporting the operational costs estimated, which are around 13X more than the CE suggested annual operational cost of $250,000.

Scale of the problem

The scale of the problem is of high concern as the burden of disease for various age ranges in Sub-Saharan Africa is apparent and unlikely to be resolved without intervention (recall 2.1 Cause area).

Neglectedness

Given the large number of organisations using mass media for impact, the methods do not seem neglected; still, specific causes and geographic areas are not addressed. While there are many organisations implementing MMIs in SSA, certain diseases remain overlooked and could be further examined.

Tractability

Tractability is a moderate concern and is region-dependent. As mentioned in 6.1.2 Tractability, recent political events that could influence success of MMIs (such as government takeovers that lead to banning specific news sources) may not be captured. It’s advised to investigate the latest conditions in a selected intervention region.

Externalities

Negative externalities: Given the evidence available, I do not expect negative flow-through effects to outweigh the benefits of implementation. Potential consequences may include:

• Increased burden on health care system
• Circulation of misinformation 
• Development of social expectations, pressure, or shaming
• Development of anxiety around health 

Positive externalities: Assuming health interventions are adequate and available, additional indirect returns (aside from death aversion) may include:

• Reduced prevalence of morbidities
• Reduced long-term strain on healthcare systems by reducing morbidity
• Increased educational opportunities 
• Increased social well-being of individuals and communities

Remaining uncertainties

The following uncertainties remain after evidence review:

• Evidential gaps on the efficacy of MMIs persists.
• Exact operational costs are uncertain and seem to vary by CEA and causal chain. 
• It is also unclear how long effects of media campaigns may last, with or without continued broadcasting. For example, are there diminishing returns in behaviour change over time or does health-seeking behaviour arrests when media messaging does? 

7   Cost-effectiveness analysis

Initially, I had attempted to compare cost-effectiveness between current interventions, scaling-up current interventions, and starting a new intervention. This quickly became nuanced (as highlighted in Section 7.1.2 Fickleness of CEAs for comparative purposes) so I redirected my focus to comparing published cost-effectiveness analyses (CEAs) on FEM’s family planning campaigns and DMI’s campaigns addressing child mortality. After, I conducted my own CEA on starting a new charity in Tanzania that estimates cost-effectiveness of increasing awareness for causes of neonatal mortality. My initial efforts to evaluate DMI and FEM are preserved in the CEA workbook but are not commented on in this report.

7.1 Existing CEAs

Previous CEAs on DMI and FEM have been presented by Founders Pledge, DMI and FEM and are summarised in the table below. Rethink Priorities has published a review on FEM and Founders Pledge’s cost-effectiveness analyses that makes several suggestions for improving estimates and is ultimately in agreement that FEM is cost-effective.

Table 5. Summary of CEAs and back-of-the-envelope calculations (BOTECs). Dollar evaluations are reported in USD 2023 unless indicated otherwise. Bottom cells highlighted in blue represent conversions to $/DALY using Founders Pledge’s conversion (provided by RTP), where applicable.

* Societal costs included, adjusted for inflation since 2015.

** For 5 West African Countries.

§ Unclear whether values are reported in 2023 USD

7.1.2 Fickleness of CEAs for comparative purposes

Caution should be taken when comparing CEAs reported by different organisations as inputs, outcomes measured, discounts, and other considerations may vary, adjusting valuations. A few examples of discrepancies I observed in cost-effectiveness reports for MMIs include: 

• Cost inclusions/exclusions: whether or not several subcosts, such as monitoring and evaluation, research efforts, are included in costing may vary by evaluator and exact break-downs are not always detailed.
  • DMI lists several exclusions in their costs, such as long-format radio show and research costs on the basis of not contributing to impact (Kasteng et al, 2018). 
  • I could not find a detailed break-down of operational costs for FEM.
• Outcomes evaluated: selection of health outcomes affected by a mass media campaign will adjust the cost per year of life with disability averted. Moreover, inclusion of indirect benefits may over-inflate the estimated positive effects of an intervention (I did not observe this but am noting it as a potential concern).
  • FEM focuses on maternal and neonatal anaemia averted.
  • DMI estimates effect for several potential causes of child mortality targeted directly by campaigns, including diarrheoa, nutrition during pregnancy, and other causes (Kasteng et al, 2018).
• Adjusting for inflation: Cost-effectiveness estimates tend to reflect static estimations made at the time of evaluation or reference dates and may pose different valuations if made consistent or adjusted for inflation. 
  • DMI references costs from 2012-2015
  • FEM references costs from 2012-2020
  • Founders Pledge relies on the same costs reported by DMI and FEM
• Reporting: The Life You Can Save Website reports slightly different estimated cost per life saved than the DMI Website, which proposes different costs per life saved than DMI’s Report (or at least, it’s unclear to me how the values were derived/what they represent).

Taken together, variability in absolute estimates proposed between different organisations are challenging to directly compare with confidence. That said, evaluations made by the same organisation across several hypotheticals may be most reliable as they apply the same structure to all scenarios, potentially controlling for some of these inconsistencies. As such, I chose to rely on FEM’s BOTEC and Founders Pledge’s BOTEC, both of which have arrived at the same relative conclusion; FEM is reported to be more cost-effective than DMI. RethinkPriorities also proposes similar relative estimates despite listing uncertainties. 

7.2 New MMI charity in Tanzania addressing neonatal disorders

#RTP23H10: Mass Media for Caregiver Awareness - Cost-effectiveness Estimate

Although not directly comparable to cost-effective estimates of FEM and DMI, the remainder of this section details the CEA I’ve developed for starting a charity in Tanzania and highlights comparisons between other CEAs. 

The RTP BOTEC suggests a geometric mean of $378.10 USD 2023 per DALY averted in Tanzania with a best case estimation of $89.54 and worst case of $2,197 per DALY averted. The wide range can be explained by the compounding uncertainties in the worst-case scenario, which were kept conservative. Estimations heavily rely on details of the theory of change and, although they consider societal costs incurred, do not include long-term societal costs saved. 

7.2.1   Effects

A new mass media awareness charity in Tanzania addressing neonatal deaths would affect women aged 15-49 years who own a radio, are expected to be reached by radio broadcasting, and are likely to exhibit a change in behaviour as a result of the media messaging.

 [See PDF report for image]

The effects of increased health-seeking behaviour are expected to reduce neonatal deaths annually.

7.2.2   Costs

The following costs were included in the CEA:

I was not able to find costs or estimates for airing a radio show in Tanzania so the CEA insteads assigns a budget within which 40% (worst case) to 80% (best case) of women ages 15-49 years who own a radio are reached by messaging. Societal costs were proxied from Kaleng et al. (2018) on DMI’s campaigns in Burkina Faso and are likely to change estimations in the context of Tanzania.

8 Directing funding towards an existing entity

On Friday, November 24, 2023, I shared this research in a 5-minute presentation to six other participants of CE’s Research Training Program during a decision-making meeting. The average of all votes concluded that this intervention idea seems promising but funding should be directed towards current organisations in this space as opposed to starting a new one.

There is value in supporting FEM’s programs as the organisation intends to run additional RCTs using their new radio technology. This would enable more evidence-backed insights on the effectiveness of MMIs to be developed.

Funding and scale-up directions

Family Empowerment Media

According to a thread in the EA Forum, in March 2023, GiveWell recommended granting $500,000 to FEM. FEM states a financial gap of $342,000 for 2023. 

Development Media International

Kaleng et al. (2018) propose scale up scenarios for DMI ranging from $7 per DALY averted in Malawi to $27 per DALY averted in Burundi. Costs are reported in USD 2015.

9     Decision

A supported theory of change, sufficient evidence for impact, promising cost-effective evaluations, and saturation of this type of intervention in LMICs ultimately informs the conclusion to direct funding towards existing organisations.

10 Conclusion

Overall, using mass media interventions to increase caregiver awareness in Sub-Saharan Africa seems promising given the burden, high potential for reach, sufficient supporting evidence (despite some evidentiary gaps on impact), and cross-validated cost-effectiveness.

Given the high concentration of current organisations using mass media to raise awareness for health concerns and the expressed need for further evidence on impacts of MMIs, introducing a new organisation may not be the best use of funds. Instead, investments could be made to (1) fill evidential gaps through RCTs, (2) improve efficacy of program delivery through innovative technology and approaches, or (3) scale-up effective programs. 

As mentioned in 9 Directing funding towards an existing entity, there seems to be a comparative advantage to funding FEM, which aims to address evidential gaps through conducting studies while continuing to provide aid cost-effectively. Additional secondary research could help advise the direction of program expansion to address neglected diseases in areas with high burdens or to replicate current cost-effective programs in neglected regions. 

11 Further review with additional time

With more time, the following concepts and publications would be investigated:

Concepts:

• Funding requests by and partners of DMI and FEM.
• Fungibility of directing funding towards existing organisations.
• The feasibility of alternative technologies for mass media messaging, such as using social media, developing ehealth, or other modernisation approaches.
• The current quality and accessibility of healthcare in Tanzania, and other top candidate countries for intervention.
• Update the CEA to include cost per beneficiary estimates.
• Reports and data published by Media Impact Funders.
• Using LiST to compare MMI efficacy across cause areas.
• Efficacy of implementing mass media in rural regions.

Publications:

• Higgs ES, Goldberg AB, Labrique AB, Cook SH, Schmid C, Cole CF, Obregón RA. Understanding the role of mHealth and other media interventions for behavior change to enhance child survival and development in low- and middle-income countries: an evidence review. J Health Commun. 2014;19 Suppl 1(sup1):164-89. doi: 10.1080/10810730.2014.929763. PMID: 25207452; PMCID: PMC4255285.

References

1. Ahinkorah, B.O., Budu, E., Seidu, A-A., Agbaglo, E., Adu, C., Ameyaw, E.K., et al. (2021). "Barriers to healthcare access and healthcare seeking for childhood illnesses among childbearing women in sub-Saharan Africa: A multilevel modelling of Demographic and Health Surveys." PLoS ONE, 16(2), e0244395. https://doi.org/10.1371/journal.pone.0244395.
2. Babalola, S., Figueroa, M.E., & Krenn, S. (2017). "Association of Mass Media Communication with Contraceptive Use in Sub-Saharan Africa: A MetaAnalysis of Demographic and Health Surveys." Journal of Health Communication, DOI: 10.1080/10810730.2017.1373874.
3. Bowen, H.L. Impact of a mass media campaign on bed net use in Cameroon. Malar J 12, 36 (2013). https://doi.org/10.1186/1475-2875-12-36
4. Family Empowerment Media. "FEM’s Research: What we learned about our audience and how best to reach them." 2021. https://www.familyempowermentmedia.org/post/fem-s-research-what-we-learned-about-our-audience-and-how-best-to-reach-them.
5. Giri SP, Maurya AK. "A neglected reality of mass media during COVID-19: Effect of pandemic news on individual's positive and negative emotion and psychological resilience." Pers Individ Dif. 2021 Oct;180:110962. doi: 10.1016/j.paid.2021.110962. Epub 2021 May 4. PMID: 34629579; PMCID: PMC8487296.
6. Glennerster, R., Murray, J., Pouliquen V. (2021). "The Media or the Message? Experimental Evidence on Mass Media and Modern Contraception Uptake in Burkina Faso". CSAE.
7. Harris, B., Goudge, J., Ataguba, J. et al. (2011). "Inequities in access to health care in South Africa." J Public Health Pol, 32 (Suppl 1), S102–S123. https://doi.org/10.1057/jphp.2011.35.
8. Institute of Medicine (US) Committee on Improving Birth Outcomes; Bale, J.R., Stoll, B.J., Lucas, A.O., editors. (2013). “Improving Birth Outcomes: Meeting the Challenge in the Developing World.” Washington (DC): National Academies Press (US). Reducing Neonatal Mortality and Morbidity. Available from: https://www.ncbi.nlm.nih.gov/books/NBK222108/
9. Kasteng F, Murray J, Cousens S, et al. (2018). "Cost-effectiveness and economies of scale of a mass radio campaign to promote household life-saving practices in Burkina Faso." BMJ Global Health, 3, e000809.
10. Mayne, J. (2017). "The COM-B Theory of Change Model (V3)." https://www.researchgate.net/publication/314086441_The_COM-B_Theory_of_Change_Model_V3.
11. Medley, N; Vogel, JP; Care, A; Alfirevic, Z. (2018). "Interventions during pregnancy to prevent preterm birth: an overview of Cochrane systematic reviews". Cochrane Database of Systematic Reviews. https://doi.org//10.1002/14651858.CD012505.pub2
12. Melberg, A., Diallo, A.H., Tylleskär, T., Moland, K.M. (2016). "'We saw she was in danger, but couldn't do anything': Missed opportunities and health worker disempowerment during birth care in rural Burkina Faso." BMC Pregnancy Childbirth, 16(1), 292. doi: 10.1186/s12884-016-1089-3. PMID: 27687500; PMCID: PMC5043633.
13. Murray J, Remes P, Ilboudo R, Belem M, Salouka S, Snell W, Wood C, Lavoie M, Deboise L, Head R. (2015). "The Saturation+ Approach to Behavior Change: Case Study of a Child Survival Radio Campaign in Burkina Faso." Glob Health Sci Pract, 3(4), 544–556. doi: 10.9745/GHSP-D-15-00049. PMID: 26681703; PMCID: PMC4682581.
14. Noordam, A.C., Sharkey, A.B., Hinssen, P. et al. (2017). "Association between caregivers’ knowledge and care seeking behaviour for children with symptoms of pneumonia in six sub-Saharan African Countries." BMC Health Serv Res, 17, 107. https://doi.org/10.1186/s12913-017-2060-3.
15. Rogers et al. (1999). "Effects of an Entertainment-Education Radio Soap Opera on Family Planning Behavior in Tanzania." Studies in Family Planning, 30, 193–211.
16. Roser, M., Ritchie, H., Spooner, F. (2021). "Burden of Disease". Published online at OurWorldInData. https://ourworldindata.org/burden-of-disease.
17. Sarrassat, S., Meda, N., Badolo, H., Ouedraogo, M., Some, H., Bambara R., Murray, J., Remes, P., Lavoie, M., Cousens, S., Head, R. (2018). "Effect of a mass radio campaign on family behaviours and child survival in Burkina Faso: a repeated cross-sectional, cluster-randomised trial." Lancet Glob Health, (3), e330–e341. doi: 10.1016/S2214-109X(18)30004-4. PMID: 29433668; PMCID: PMC5817351.