Epistemic status: Not a SME and not a data analyst. If mistakes are brought to my attention, I will likely make running changes and pretend it was like that all along.

Summary

I'm splitting this into two parts. 

Part A is mostly descriptive, in which I share information on next-generation antimalarial long-lasting insecticide treated bed nets (LLINs):

• Of the LLINs produced to counter insecticide resistance, WHO and GiveWell agree that those that include the active ingredient chlorfenapyr are the most effective.
• Effectiveness analysis conducted by Mosha et al. (2022) estimated that LLINs with chlorfenapyr more than half the incidence of malaria in children in the first year, and reduce it by over a third in the second year, compared to traditional LLINs.
• GiveWell's current estimate is that LLINs that include chlorfenapyr have precisely negated the effects of insecticide resistance accrued since approximately the year 2000.

Part B is my attempted analysis of the effectiveness of LLINs with chlorfenapyr:

• Via my model, I essentially agree with the Mosha et al. (2022) estimates, and also that LLINs with chlorfenapyr reduce malaria illness in children by a fifth in the third year, compared to traditional LLINs.
• I suggest an update to the GiveWell model, that:
  • Interceptor G2 LLINs (a specific LLIN with chlorfenapyr), reduce the rate of malaria illness in children by 50% when compared to recipients of traditional LLINs per "coverage year".
  • For other LLINs with chlorfenapyr, the same estimation is used but with a 20% downward adjustment on the total reduction in malaria illness, pending assessments of how well these LLINs have survived under field-use conditions (and a similar policy could be applied to LLINs of other classes)

************************************

Part A

Introduction

Traditional LLINs have used a pyrethroid as their sole insecticidal ingredient, against which resistance has become a well known concern. The most promising next-generation LLINs^[1] engineered to combat this resistance continue to utilise a pyrethroid, together with one of three additional ingredients:

1. Piperonyl Butoxide (PBO): A synergist that blocks the production of an enzyme that detoxifies the pyrethroid. This enzyme is a common resistance mechanism, but there are others. As such, it potentiates the pyrethroid against (and likely only against) this specific resistance mechanism.
2. Pyriproxyfen: A pesticide that inhibits growth and fertility.
3. Chlorfenapyr: A pyrrole (type of insecticide) which disrupts cell functionality to kill susceptible mosquitos over the course of 2-3 days. In contrast, pyrethroids attack the nervous system to kill susceptible mosquitos immediately. Having different modes of action has the upshot of very limited physiological cross-resistance between these types of insecticides^[2].

As of November 2024^[3], WHO still recommends pyrethroid-only LLINs generally, and in areas with noticeable pyrethroid resistance^[4] they:

• strongly recommend pyrethroid+chlorfenapyr LLINs
• conditionally recommended pyrethroid+PBO and pyrethroid+pyriproxyfen LLINs (in that order)

In "Part A", I will share my understanding of:

• The next-gen LLIN field RCTs that would have driven WHO's recommendations
• A cost-effectiveness analysis that was attached to one of these RCTs
• GiveWell's estimates of the relative effectiveness of next-gen LLINs

Results of next-gen LLIN field RCTs

Overview

Field RCTs compare performance of next-generation LLINs against pyrethroid-only LLINs. In the table below, I summarise the effects found on two primary outcome measures.

Prevalence is the % of the sample population that has any degree of parasitaemia (detected by rapid diagnostic test and/or microscopy), whether they are symptomatic or not. I believe Prevalence RR (Risk Ratio) is essentially the expected probability of having parasitaemia in that group, divided by the expected probability of having parasitaemia in the reference group (pyrethroid-only LLINs)^[5]. 

Incidence is the number of cases of fever plus parasitaemia per person-year. Incidence RR (Rate Ratio) is the expected malaria cases per person-year in that group, relative to the reference group^[6]. It is one minus the reduction in malaria episodes. Going forward, when I describe "effectiveness" it will primarily be by this metric^[7].

Age ranges of sampled participants vary, even within the same study for the different outcome measures. However, they are all young children or adolescents, with one exception^[8]. The different age ranges are likely relevant to the probability of using LLINs, as well as effectiveness of LLINs given use, but probably not substantially. Relevantly for what follows, both trials involving pyrethroid+chlorfenapyr LLINs enrol 6m - 10 year olds into cohorts for tracking malaria incidence ^[9].

Distribution performance: Next-generation LLINs versus pyrethroid-only LLINs

Table notes.^[10] I have highlighted the large incidence reductions found for pyrethroid+chlorfenapyr LLINs.

All of these trials were cluster RCTs. A significant, and likely majority, of the reduction in malaria illness caused by LLIN use comes from the suppression of infection prevalence in both mosquitos and people^[11]. To ensure these community effects are properly captured, and the community effects of surrounding clusters is mitigated, measurements are only taken from participants relatively central in clusters (i.e. inside a buffer zone).

Trials test product effectiveness, not active ingredient effectiveness per se

I think this is something potentially crucial to keep in mind. The LLIN products used for each trial were:

The different manufacturers (i.e. behind Permanet, Olyset and Interceptor) use different types of pyrethroids, that have known differences in their lethality and excito-repellency^[12]. Furthermore, products from the same manufacturer differ in more ways than the chemicals used. As such, RCT results can be due to characteristics of the LLINs other than their "product class^[13]". A few examples:

1. PermaNet: PermaNet 2.0 contains 1.8g/kg of deltamethrin distributed evenly throughout the net. PermaNet 3.0 contains 4.0g/kg in the roof panel and 2.8g/kg in the side panels. Furthermore, the addition of PBO is within the roof panel only^[14]. Benefits of PermaNet 3.0 could be due to higher active ingredient concentrations on the LLIN surface^[15].
2. Olyset: Olyset Net and Olyset Plus have similar concentrations of permethrin. However, Olyset Plus has a smaller mesh size^[16] and releases the permethrin at a faster rate. WHO has suggested that this faster insecticide release is a likely reason for its better performance^[17][18].
3. Interceptor: Interceptor “G1^[19]” is coated with 200 mg/m² of alpha-cypermethrin. Interceptor G2 is coated with 100 mg/m² alpha-cypermethrin plus 200 mg/m² of chlorfenapyr. The manufacturer markets the latest material of Interceptor G2 as new and unique^[20]. 

Testing does indeed show significant differences in longevity between products^[21].

While I think this is an important thing to keep in mind when generalising results across LLIN products, GiveWell equate "non-generic" LLINs of the same product class. I do not refute this as the pragmatically best approach - Mosha et al. (2022) do the same in the following CEA - for a rough guess and parsimonious model. However, I ultimately suggest some differentiation in my analysis in Part B.

Effectiveness Analyses by Pros

Mosha et al.

Mosha et al. (2022)^[22] conducted CEA for all LLIN types, using results from the first two years of their Tanzania trial. They ran Monte Carlo simulations using estimated distributions of parameters (both for benefits and costs). All total benefits and costs below are for a hypothetical cohort of 10,000 people covered (1.7 per LLIN).

On the effects side of the equation:

• Main outcome measure is disability-adjusted life years (DALYs).
• Necessarily for DALY calculations, this included estimations for malaria illness and deaths.
• They use their trial to estimate absolute malaria incidence rates for pyrethroid-only LLINs, and relative incidence rates for next-generation LLINs, for children under 10 (CU10).
• They use GBD and population data to model expected malaria cases and deaths for CU10 and persons >10y separately.

Comparisons of the estimated effects^[23]:

These are expected cases, deaths and DALYs for a community of 10,000 people having received each type of LLIN. So lower values are better. YLL = Years of life lost; YLD = Years of life lived with disability ^[24].

On the costs side of the equation:

• They consider costs primarily as the societal costs which equals costs to public providers plus households. The majority component of public provider costs come from the subcategory of donors, with the remainder being the Tanzanian government.
• They consider estimated costs of the different LLINs and costs incurred through diagnosis and treatment. For example, they estimated a household cost of $1.54 per day spent ill with malaria.
• They presume all LLIN types incur the same distribution costs, so it is not included in estimates.

Comparisons of the estimated costs^[23]:

And combining effects and costs over the two year period ^[25]:

By advanced analysis techniques^[26], I have broken down the ~1750 cases avoided thusly:

As expected, 1 minus the % reduction in cases for children under 10 is practically the same as Incidence RR sizes from the trial for both years.

Regarding the model estimations:

• Distributing pyrethroid+chlorfenapyr LLINs will reduce malaria illness (for all ages) over the 2 years by roughly:
  • 38% of the total illness that would occur under an pyrethroid-only LLIN distribution
  • 32% of the total illness that would occur under a pyrethroid+PBO LLIN distribution^[27]
  • 39% of the total illness that would occur under a pyrethroid+pyriproxyfen LLIN distribution^[28]
• This comes to roughly 0.3 cases avoided for each pyrethroid+chlorfenapyr LLIN distributed instead of a pyrethroid-only LLIN^[29].
• For public providers (and donors) pyrethroid+chlorfenapyr LLIN distributions are more costly than pyrethroid-only LLIN distributions, even after including savings to public and donor health care. However, pyrethroid+chlorfenapyr LLIN distributions are the most affordable option if we include financial costs to beneficiaries.

On the model itself:

• They do not include deaths indirectly caused by malaria (something GiveWell roughly estimates as 0.75 per malaria-attributed death^[30]).
• The three validity adjustments I attempt in my modelling below are not considered (although the net result on case reduction is similar).
• (I cannot see how this is of value but I find it interesting....) The GiveWell and Mosha et al. models both predict greater effectiveness for children compared to adults, but for different reasons. In the GiveWell model they attribute this to differences in net use behaviours^[31]. In the Mosha et al. model it is due to GBD models indicating that the proportion of the burden of malaria that is on children (compared to adults) decreases as the overall transmission intensity decreases.

I am aware of another modelling study into the effectiveness of pyrethroid+chlorfenapyr LLINs that I will be ignoring in this post^[32].

GiveWell's estimates

GiveWell of course publicise how their model works, if you want it direct from the source I suggest to start by reading this section. The following is a summary, focussed on details relevant for this post.

A ~45% base effectiveness

A meta-analysis found that across 5 field trials from circa 1990s, distributing conventionally treated nets (CTNs) reduced malaria episodes by an average of 45% (compared to control groups not receiving any nets, and with very low existing use^[33]). GiveWell take this as the estimated effect on malaria episodes, for under 5s in communities receiving a present day LLIN distribution^[34], but with 3 main downward adjustments^[35]:

1. A -10% adjustment for "net usage in program contexts", representing the fall in estimated net usage from these field trials (70%) to modern day distributions (63%)^[36].
2. Over the 3 years an LLIN is intended to last, premium LLINs get effectively 2.27 CTN-year's worth of coverage^[37]. A "CTN-year" can be understood as a year where the average ITN use rate is 70%, and the ITNs are in comparable condition to the meta-analysis CTNs in terms of physical integrity and pyrethroid bioavailability.
3. Diminished effectiveness caused by pyrethroid resistance accrued since the 1990s.

How much has pyrethroid resistance diminished LLIN effectiveness?

Pyrethroid-only LLINs: The estimates of diminished effectiveness are dictated by averaging results of susceptibility bioassays^[38] at a country-level.

Pyrethroid+PBO LLINs: This is largely dictated by the pyrethroid-only LLIN estimates, combined with results from synergist-insecticide bioassays^[38]. Basic idea is to estimate what proportion of the accrued pyrethroid resistance is overcome by adding PBO. 

Pyrethroid+pyriproxyfen LLINs: Estimates are the pyrethroid-only LLIN estimates due to having "not yet seen compelling evidence"^[39] for a significant improvement in effectiveness.

Pyrethroid+chlorfenapyr LLINs: All pyrethroid resistance accrued since the 1990s has been negated. 

Bioassays are highly variable, and as such so are effectiveness estimates for LLINs (except those with chlorfenapyr) between countries^[40]. GiveWell states that their estimates for +chlorfenapyr (and +pyriproxyfen) LLINs are particularly rough guesses. From my impression, I would describe them as placeholder estimates. 

A summary of the GiveWell estimates^[41]:

I made graphical representations of these estimations for Chad and Nigeria (sans pyrethroid+pyriproxyfen LLINs^[46]). I have found it handy to think in terms of LLIN antimalarial features multiplying together to push down malaria illness rates:

Of these next-gen LLINs, those with PBO were the first on the scene in any quantity worthy of being modelled^[47]. As such, the GiveWell model is currently structured to estimate the potentiation-of-the-pyrethroid effects of PBO, and not the hit-them-with-something-else effects of chlorfenapyr. 

The current GiveWell estimate for the effectiveness of pyrethroid+chlorfenapyr LLINs is unaffected by levels of pyrethroid resistance. This means that the benefits of chlorfenapyr are inversely proportional to the benefits of pyrethroid (i.e. peach + green arrows are the same length in Chad and Nigeria). 

************************************

Part B

My estimate for Interceptor G2 LLIN effectiveness

I think a malaria reduction effect caused by the addition of chlorfenapyr can be multiplied on top of the estimated effect for pyrethroid-only LLINs, independently of the effectiveness of the pyrethroid-only LLINs. The logic here is that the addition of chlorfenapyr does not reduce the effect of pyrethroid resistance, but is an additional active component on top of the barrier and pyrethroid.

I describe and give estimates specifically regarding Interceptor G2 LLINs against Interceptor G1 LLINs. I comment on generalising to alternative pyrethroid+chlorfenapyr LLINs at the end.

In short, I have estimated that per CTN-year, distributing Interceptor G2 LLINs instead of Interceptor G1 LLINs reduces malaria illness by 50%. 

For Chad and Nigeria, this would look like:

A "CTN-year" is rather abstract^[37]. In more concrete terms, I think a distribution of Interceptor G2 LLINs, compared to a distribution of Interceptor G1 LLINs (where no other ITNs exist), can expect a reduction in child malaria illness of roughly:

• 50% in the first year
• 35% in the second year
• 20% in the third year

This means I go through a whole modelling process below to essentially end up with the effect sizes in the relevant trials (and for first 2 years, estimates of the Mosha et al. CEA). As such, you might like to stop reading and move on with your life. Below is some details on the adjustments I made, and why.

Modelling to get my estimate

Accompanying Google Doc (and sheets) I will reference at times.

I made an adorable^[48] model to address a few validity concerns regarding the relevant trials, and arrive at an estimate for the effectiveness of Interceptor G2 LLINs. Beware of false precision, although at the end I subjectively adjust/round down my estimate.

Nearly all effects are described in terms of incidence rate ratios (Incidence RRs). Keep in mind that 1 minus this rate ratio is the expected reduction in malaria illness, and the smaller the rate ratio the greater the effect size.

The following is a summary table that I will attempt to explain in stepwise fashion.

Effect Size Estimates (Incidence RRs):

TZ = Tanzania; BJ = Benin

Effect size inputs

What is the assumed Interceptor G1 effect at 70% use rates?

Taking the GiveWell methodology^[49], I used susceptibility bioassays conducted as part of the respective trials to estimate the effectiveness of Interceptor G1 LLINs^[50].

In summary:

LLIN use rates are held constant for all years post-distribution to separate the loss in effectiveness via LLIN deterioration from the loss due to LLIN attrition. It is held constant at 70% specifically as that is the presumed use rates behind these GiveWell estimates.

The estimated pyrethroid resistance in Benin is very high. For reference, the theoretical maximum reduction in effectiveness is 73%^[51]. In contrast, the estimated pyrethroid resistance in Tanzania seems typical to slightly low (for the regions where AMF operates in sub-Saharan Africa). The upshot is that, by my estimates, Interceptor G1 LLINs are nearly twice as effective in Tanzania as they are in Benin. 

What was the effect for cohort children in the trials?

These are the effect sizes directly taken from the Tanzania and Benin trials, with one exception.

Hypothetical Incidence RR in 3rd year of Tanzania trial

For the Tanzania trial, incidence was not an outcome measurement for the third year following the distribution. For the model, I assumed an Incidence RR of 0.75, which I think is a slightly conservative estimate. It is loosely based off the prevalence risk ratio and entomological inoculation rate density ratio^[52] in the third year, relative to the third year of the Benin trial and previous years of both trials.

Other model inputs underlying adjustments

Community versus personal protection

Adjustments to the reported trial effect sizes required estimating how to partition effect sizes between community and personal protection, as a function of LLIN use rates. While this is a very rough guess and the model has known flaws^[53], the important outputs (on net) are not significantly affected by what I think are plausible guesses for this partitioning^[54]. There is a logical reason for this parameter to shift depending on the Interceptor G1 effectiveness estimation, but I kept it constant to keep the modelling simpler.

LLIN use rates

All adjustments are centred around estimated use rates of study and non-study LLINs. Generally, I gave the self-reported use rates a downward adjustment of 14%, similar to what has been presumed by GiveWell in their model^[55]. More specific details are explained below.

Internal validity adjustments

1. What was the effect for typical children in the trials?

The children being monitored for malaria incidence ("cohort children") are not typical participants. First, self-reported measures indicate they would use study LLINs much more frequently than typical trial participants of a similar age^[56]. I suspect to a degree this difference is real. Second, they maintain high use rates in the second year, against the typical trend and what seems to be plausible from results of imbedded net attrition and functional survival studies^[57]. These particularly high use rates would overestimate the true effect size, which is what I tried to address through my model. No other adjustments were made for my significant uncertainty regarding use rate estimates, nor likely other unknowns regarding atypical behaviours of cohort children^[58].

2. What would have been this effect if clusters were "pure"?

The frequent use of non-study LLINs in the population would cause the trial effect size to underestimate the true effect size^[59], due to the community benefits^[60] for the cohort children arising from a blend of LLIN types. I've referred to this as "cluster impurity". I attempt to model this using a basic (and not accurate) assumption that any non-study LLINs used are of equivalent efficacy to Interceptor G1 LLINs.

True Incidence RRs under trial context

My best estimate is that the trial effect sizes, averaged together^[61], underestimated the true effect sizes by 0.06 points in the first year (0.46 to 0.40), 0.07 in the second year (0.59 to 0.52), and 0.08 in the third year^[52] (0.78 to 0.70). As is predictable, these estimates were not materially affected by the Interceptor G1 effectiveness estimates^[50]. 

External validity adjustment

These trial effect sizes likely overestimate effect sizes for distributions under program contexts. Primarily, this would be because the non-cohort trial participants (let alone the cohort participants) probably had higher LLIN use rates than recipients of LLINs under program contexts. At the very least, LLIN use rates are higher than GiveWell's estimates which I am accepting as accurate^[62].  

3. What would have been this effect at 70% use rates?

I can input my estimate for the effectiveness of Interceptor G2 LLINs into the GiveWell model and allow it to address the downward adjustment needed for the lower use rates under program contexts^[63], as well as all other adjustments. I take my estimated effectiveness of Interceptor G2 at 70% use rates, and essentially reverse the process I took to get the by-year effectiveness of Interceptor G1 LLINs (at 70% use rates) out of the GiveWell model. This arrives at my estimates for Interceptor G2 LLIN per CTN-year effectiveness:

Final Estimate

These Incidence RRs average to 0.46, representing a 54% reduction in malaria illness against a distribution of Interceptor G1 LLINs, per CTN-year. 

I round this off/down to a 50% reduction in malaria illness, which represents an 8% downward adjustment. 

Extra Notes on Important Things

My Uncertainty

Aside from this final rounding off, I am generally not making downward adjustments for uncertainty^[64].  A large reason for this is that it is a recommendation for the GiveWell model, pre-adjustments for uncertainty built into it (most clearly, ~-10% for external and internal validity).

I am ironically very uncertain about this decision. I can see the Bayesian logic behind adjusting downwards. Effect sizes for these Interceptor LLIN arms are remarkably large relative to other modern trials and the performance of CTNs versus no nets in the 1990s. While I do not know what my priors would have been, it would have been much less than a 54% reduction (yet my priors would be given significantly less weighting than the results of these trials). A 1-year cRCT in Côte d’Ivoire that is set to conclude next year may impact on my estimate and certainty. 

A few general observations:

• I am ignoring the chance I have made a serious blunder in modelling. This is not because I find that unlikely, I just struggle estimating it plus my logic ought to be checked by others before the estimations affect any decision making.
• For how much Interceptor G2 reduces malaria overall (i.e. versus no nets), I think the majority of the uncertainty lies in estimating the effectiveness of Interceptor G1 (it is just a much more difficult thing to do).
  • To use this to highlight the boldness of my estimate, even if Interceptor G1 had zero effect on malaria illness my estimate for Interceptor G2 LLIN effectiveness would still be greater than GiveWell's (50% reduction vs 45% reduction per CTN-year)
• Of the validity concerns I have attempted to address, I am most unsure about the impact from atypicality of cohort children/caregiver behaviours.

Costs

I have not attempted thorough cost effectiveness analysis, because there is relevant data I am not privy to. 

Currently, it seems as if pyrethroid+chlorfenapyr LLINs cost roughly 30-40% more to acquire than pyrethroid-only LLINs^[65]. However, the costs of distribution should be similar, if not identical (as presumed in the Mosha et al. model) and we can expect significant changes in demand and competition, that I suspect will drive the acquisition costs down, never to the level of pyrethroid-only LLINs but potentially to the level of pyrethroid+PBO LLINs.

I have used the GiveWell model to test counterfactual situations, where I set:

• nets distributed to entirely of one class^[66]
• "total cost per insecticide-treated net (ITN) distributed" to either $6 (for pyrethroid-only scenario) or $8 (for pyrethroid+PBO or pyrethroid+chlorfenapyr scenarios). This represents a 33% price increase for the next-gen LLINs, and a slightly conservative (i.e. high) estimate on the absolute costs^[67].

For clarity, here I am equating my Interceptor G2 LLIN estimate with all pyrethroid+chlorfenapyr LLINs.

% improvements on distributing pyrethroid-only LLINs

Generalising to other pyrethroid+chlorfenapyr LLINs

Disclaimer: Knowledge here is especially piecemeal and superficial. I have given it similarly little thought. I will potentially dig deeper, mostly because I find these trials and ITN quality assurance/longevity interesting.

TLDR: For pyrethroid+chlorfenapyr LLINs that are not Interceptor G2 LLINs, reduce the overall estimated effect on malaria incidence by 20% until there is a published assessment on the quality of field-aged samples.

The Interceptor G2 LLIN was the "first-in-class" for pyrethroid+chlorfenapyr LLINs^[13]. Future LLINs of the same class do not need to undergo field trials with epidemiological outcomes, in order to be listed as acceptable by WHO for mass distributions^[68]. Instead they need to pass "non-inferiority" experimental hut trials (EHTs)^[69]. To demonstrate "non-inferiority" the second-in-class product needs to show with statistical significance that they are better than a certain minimum standard relative (and worse) to the first-in-class product (Odds Ratio of 0.7), primarily for mosquito mortality and secondarily for blood feeding inhibition^[70]. Trial arms include new LLINs, and LLINs that are washed 20 times to mimic active ingredient/synergist loss with ageing (and nets in all trial arms are holed to mimic wear-and-tear).

The only pyrethroid+chlorfenapyr LLINs that have passed non-inferiority trials and been accepted by WHO are PermaNet Dual (March, 2003) and Yorkool G5 (September, 2024)^[71]. 

The non-inferiority trials for PermaNet Dual were Ogutu et al. (2024) and Syme et al. (2023). Some high-level observations:

• PermaNet Dual and Interceptor G2 arms are more similar than they are different. That is, they are similarly superior to PermaNet 2.0 and 3.0 (pyrethroid-only and pyrethroid+PBO LLINs, respectively).
• In the Syme et al. study, PermaNet Dual scrapes through as being non-inferior on the primary outcome (mortality), and found to be "not non-inferior" on the secondary outcome (blood feeding inhibition). Yet in the Ogutu et al. study PermaNet Dual is superior (with statistical significance) on the primary outcome. However, effect sizes are small. Differences might come down to the local mosquitos available and chosen, and other environmental factors.
• What might be the most alarming is that across both studies PermaNet Dual retained roughly half as much active ingredient content after 20 washes, compared to Interceptor G2. I am not making strong inferences from this^[72], but it would be something to look out for.

I have not found the two Yorkool G5 non-inferiority trials, but from what I can tell from a skim-read of the supporting documents on the WHO website:

• Again, the two pyrethroid+chlorfenapyr LLINs were significantly superior to those without (this time, two pyrethroid-only LLINs).
• This time, the two LLIN products performed quite similarly in both trials on entomological outcomes.
• Additionally, they were similar on active ingredient retention after 20 washes.

This might point towards a suggestion that Yorkool G5 should be estimated as on par with Interceptor G2, and a downward adjustment needed for PermaNet Dual. But I think it is going too far to make distinctions based off inferences from these trials. The entomological outcome measures are far removed from epidemiological outcomes, and so is the mimicked ageing from real ageing.

I believe it is required that EHTs are conducted once actual aged LLINs are available^[73]. Provided this includes trial arms of established LLINs, this would provide important data on actual LLIN survivorship (being retained), wear-and-tear and active ingredient retention. It might be kicking the can down the road a bit, but at the same time it seems unreasonable to generalise results of Interceptor G2 LLINs onto these LLINs that have yet to "pass" in the real world. For any LLIN product, I think pending entomological^[74] and quality data on field-aged samples, an effectiveness analysis is likely best off:

• taking the overall incidence rate reduction estimated for same-class LLINs that underwent field trials.
• making a downward adjustment to this rate reduction (and as a rough guess, I think 20% is reasonable)

At the same time, I acknowledge that this might not be a plausible approach where the goal is to get just rough effectiveness estimates.

************************************

Conclusion

Looks like human ingenuity got one over evolution.

Acknowledgments 

This is independent and has not been reviewed for accuracy, so naturally all mistakes are my own. However, I learnt much from attempting related analysis while volunteering with SoGive. I thank the organisation and especially Isobel Phillips and Sanjay Joshi for their mentoring and time, which helped me acquire background knowledge. 

Thank you for reading (or at least scrolling) to the end.

1. ^{^}

   As GiveWell identified, WHO will not label an insecticide-treated net (ITN) as an LLIN until they have passed certain examinations. GiveWell and WHO thus label these next-generation nets as ITNs (which encompass conventionally treated nets [CTNs] and LLINs). I however will stick to "LLIN" as it describes their intended functionality, and I think a priori expectations. For the purpose of modelling, GiveWell presumes these next-generation nets last as long as the pyrethroid-only nets of the same manufacturers (that have passed the required testing). My labelling is consistent with the language chosen in much of the research (e.g. Accrombessi et al., 2023).

2. ^{^}

   Bajunirwe & Nantongo, 2024: "Chlorfenapyr acts through disruption of the oxidative phosphorylation in the mitochondria, rather than through the neural receptors, and also carries the advantage of no cross-resistance to PYs [pyrethroids] and other insecticides." I added in "physiological" to distinguish between behavioural resistance mechanisms that are unlikely to be overcome be adding additional active ingredients. Furthermore, even considering just physiological resistance, the claim here of "no cross-resistance" is likely not completely accurate, as cuticular thickening and salivary gland proteins may provide non-specific insecticide resistance (Messenger et al., 2021).

3. ^{^}

   WHO guidelines for malaria - 30 November 2024 pp. 5 - 6

4. ^{^}

   This would include all of Sub-Saharan Africa

5. ^{^}

   Potentially important is that nearly all trials (Uganda the exception) reported effects on prevalence in terms of an “Odds Ratio” (OR), which is the relative odds instead of probability. I have converted these ORs into RRs in a way that is rudimentary and likely not completely accurate (especially where the analysis controlled for effects of baseline prevalence), but I think accurate enough for this purpose as I primarily will focus on Incidence RR. The formula I used was:

   Prevalence RR = OR * p(negative | study group) / p(negative | reference group). See Tamhane et al. (2016) p.10 

   For understanding the difference between Odds, Risk, Rate and Hazard Ratios (see next footnote), I found short explanatory articles by Knol et al. (2011) and Bland & Altman (2000) helpful.

6. ^{^}

   The Benin trial uses a “Hazard Ratio” instead, which I am treating as being equivalent to the Rate Ratio. Unlike rate ratio, the hazard ratio does not require the assumption that the rate is consistent over time, but they can be interpreted the same way (Knol et al., 2011).

7. ^{^}

   And by GiveWell's presumption, this equates to the effect on death (at least when it comes to children under 5). In linked Google docs I might discuss effectiveness on a different outcome (but hopefully this will be clear)

8. ^{^}

   For prevalence, Accrombessi et al (2023) samples from any age.

9. ^{^}

   It is likely that children under 5 are more likely to use LLINs than the population average (e.g. Olapeju et al. 2018, World Malaria Report 2022; c.f. Rek et al. 2020). However older children and young adults are less likely than the population average (e.g. Olapeju et al. 2018, Rek et al. 2020). I treat the effect on 6m - 10 year olds the same as the effect on under 5 year olds (for whom it is likely more effective, but my best guess is that it is in the realm of 1-2 percentage points).

10. ^{^}

    italics = average of 2 point-in-time measurements.  * = statistically significant @ p < .05 (when an average of 2 point-in-time measurements, this is when either of the individual measurements were statistically significant; when reported with OR^[5], this is regarding the OR). Year given is approximate year of the distribution (sometimes distributions are in batches). Data collated here.

11. ^{^}

     Hawley et al. (2003), Killeen et al. (2007), Unwin et al. (2023)

12. ^{^}

    OlySet (and the pyrethroid it contains) has been shown to have significantly higher repellency effects than Permanet (and the pyrethroid it contains), and this might be because they are Type I and Type II pyrethroids respectively (Kawada et al., 2014). If this is true, Interceptor nets likely have low repellency, since alpha-cypermethrin is a Type II pyrethroid (and chlorfenapyr has very low repellency). I think low repellency might be a beneficial trait, but this is speculative on my part as I have no source and in contrast I have seen repellency typically considered to be a good thing.

13. ^{^}

    Determining non-inferiority of insecticide-treated nets and indoor residual spray products within an established product class (WHO, 2018)

14. ^{^}

    Maiteki-Sebuguz et al. (2022)

15. ^{^}

    Lindsay et al. (2021) 

16. ^{^}

    GiveWell conversation with Hilary Ranson

17. ^{^}

     Protopopoff et al. (2018)

18. ^{^}

    However, Martin et al (2024) provided empirical evidence that Olyset Plus physically deteriorates much more quickly than what is expected of an LLIN. That said, Lukole et al (2022) shows Olyset Plus to be comparable to pyrethroid-only LLINs, although the paper raised longevity concerns for LLINs generally.

19. ^{^}

    I add “G1” to distinguish them from G2. Elsewhere, you will likely see them called Interceptor LLIN.

20. ^{^}

    https://agriculture.basf.com/global/en/business-areas/public-health/products/interceptor-g2.html. (I have not found how exactly the material differs from G1 to G2.)

21. ^{^}

    E.g. Martin et al (2024) 

22. ^{^}

    Mosha et al. (2022). Everything referred to in this section can be found in the paper's Supplementary Appendix (pp. 23 - 30). But output and more information on main Monte Carlo analysis is to be found in the main paper.

23. ^{^}

    Mosha et al. (2022). Supplementary Appendix. S23

24. ^{^}

    I am quite sure both YLD bars are there, and just hard to make out because they are so thin

25. ^{^}

    Mosha et al. (2022). Supplementary Appendix. S24b

26. ^{^}

    I've eyeballed the "cases" bar graph

27. ^{^}

    (4600 - 400) / 2850. Benefits over Olyset Plus (that is the branding of the pyrethroid+PBO LLINs used) are nearly entirely due to outperformance in the second year. Martin et al (2024) is a fabric integrity study nested in this trial, which demonstrated that Olyset Plus deteriorated at a significantly faster rate (and what is expected of an LLIN) than the alternative study LLINs.

28. ^{^}

    (4600 + 100) / 2850

29. ^{^}

    1,759 / (10,000/1.7)

30. ^{^}

    Cell H136

31. ^{^}

    Model entry, and write-up

32. ^{^}

    ..aside from this footnote. Many LLIN products are only assessed for performance in "experimental hut trials" (EHTs) on entomological outcomes (e.g. mosquito mortality and blood-feeding success). Performance under realistic field conditions on epidemiological outcomes (e.g. malaria prevalence and incidence) is only needed for the first-in-class products. I give more details behind this under "Generalising to other pyrethroid+chlorfenapyr LLINs".

    Sherrard-Smith et al. (2022) created a model to predict epidemiological outcomes in real-world settings given entomological results from EHTs, which Churcher et al. (2024) used to predict the epidemiological outcomes for both the Tanzania and Benin trials given entomological data on pyrethroid+chlorfenapyr LLINs . In short, the model did well predicting the benefit of pyrethroid+chlorfenapyr LLINs over pyrethroid-only LLINs at reducing incidence in the Tanzania trial, but predicted a much greater effect in the Benin trial than was observed (74% reduction versus 36% reduction).

    I know this modelling has many limitations and uncertainties, but it also addresses some major limitations of field RCTs. So for CEA, it could plausibly be very valuable (as far as I know, GiveWell only uses similar modelling to validate their pyrethroid resistance modelling). However, I do not understand EHTs nor this modelling enough to say anything more at this stage.

33. ^{^}

    Pryce et al. (2018). And GiveWell's breakdown of the trials.

34. ^{^}

    In a hypothetical scenario where alternatively the recipients do not currently have, nor will the receive, any bed nets.

35. ^{^}

    Adjustments I am ignoring are internal validity adjustment (-5%), external validity adjustment (-5%) and "adjustments affecting cost per life saved". To simplify, and because I do not bring them up again in this post.

36. ^{^}

    Cell H22. And Section 3.3 of write-up.

37. ^{^}

    A "CTN-year" can be understood as a year where the average ITN use rate is 70%, and the ITNs are in comparable condition to the meta-analysis CTNs in terms of physical integrity and pyrethroid bioavailability.

    GiveWell typically call this a "coverage year" in their model and any explanation. Not sure if I picked up "CTN-year" from any GiveWell document, or I made it up, but I like it.

    LLINs give less than 3 CTN-years due higher wear-and-tear and attrition rates. Diminishment in efficacy from insecticidal decay over the 3 years of an LLIN is estimated to average out with the more rapid insecticidal decay, but 6-monthly replenishment, of a CTN.

38. ^{^}

    In short, mosquitos are exposed to twice the dose needed to kill susceptible mosquitos, and mortality rate is counted. My summary of types of bioassays, which do not cover innovations for testing pyriproxyfen and chlorfenapyr.

39. ^{^}

    Cell note in CEA

40. ^{^}

    GiveWell highlights this as an issue in this forum post

41. ^{^}

    Essentially just this section of the GiveWell CEA

42. ^{^}

    Due to GiveWell attributing 27% of benefits to the physical barrier of the nets, the theoretical maximum this can be is 73%.

43. ^{^}

    A simple average across the 7 countries in the overall CEA

44. ^{^}

    Chad is in the GiveWell model due to previously considering funding campaigns, but they decided against it

45. ^{^}

    Effective estimates are for areas where PBO is determined to provide an advantage (which is where there is pyrethroid resistance and the resistance mechanism PBO combats has been detected)

46. ^{^}

    I will not be referring to these much. I do not expect them to play a big role going forward, and I predict WHO to downgrade their recommendation for them due to higher costs without any clear benefit. 

47. ^{^}

    I believe comfortably surpassing pyrethroid-only distributions via AMF

48. ^{^}

    Potentially lousy, but it's mine and I gave it some pretty colours.

49. ^{^}

    Although personal judgements were needed to adapt the estimates for this purpose.

50. ^{^}

    I also took a look with Interceptor G1 effectiveness based on GiveWell's estimates for Togo and Nigeria. You likely see them if you take a look through any Google doc/sheet I link. They are largely being ignored within this post.

51. ^{^}

    Due to 27% of the benefits of circa 1990s CTNs presuming to be due to the physical barrier of the net.

52. ^{^}

    Entomological inoculation rate is the rate of infectious bites. See my Google doc for more details.

53. ^{^}

    I use some old modelling of personal vs community effects on entomological inoculation rates (EIR) as a rough proxy, but the relationship between EIR and malaria incidence is not linear. Also, I presume the additional benefit from personal protection is constant at all coverage levels (as a % reduction), but it probably decreases as coverage increases.

54. ^{^}

    Very roughly, I would say that there was 3 percentage point differences between the spans of subjective 80% confidence interval.

55. ^{^}

    For the circa 1990s CTN studies,  GiveWell estimated approximately 80-82% average self-reported use rates, and essentially downgraded this to 70%.

56. ^{^}

    Especially Benin, where they reported essentially 100% use rates.

57. ^{^}

    In the Tanzania trial, a nested study by Martin et al. (2024) found 21.1% of Interceptor G2 had been discarded or were too torn for practical use by the end up the 1st year, and this grew to 43.2% by end of the 2nd year . For the Benin trial, a similar study by Ngufor et al. (2024) seems to have found similar results, but I am a little confused over how these figures have been measured and have not invested the time to understand.

58. ^{^}

    Of the adjustments I make, I am least confident about my interpretation here. I contacted the POCs on the papers about this uncertainty but did not hear back.

59. ^{^}

    Churcher et al. (2024) shares this observation with me in their modelling study using entomological data to predict the results of these field RCTs^[32]. At first glance, their adjustment is similar but slightly greater than mine.

60. ^{^}

    Not the personal benefits, as the model assumed cohort children were never using non-study LLINs.

61. ^{^}

    Go to the spreadsheet if you want to view the trials individually.

62. ^{^}

    GiveWell has empirical support for their estimate, using actual program distribution data, that I have not looked into. I am accepting their estimations. They are likely accurate and I want to focus on comparison of LLIN types. I think these next-gen RCTs are giving some indication that use rates have increased since the 1990s, but they are also probably elevated (in the Benin trial, hang-up campaigns were run 1 and 7 months post-distribution). There may be other support for an increase in net use rates since the 1990s: Bertozzi-Villa et al. (2021) stated (the trend is likely important, but ignore absolute value of figures due to differences in definitions and measurement): “The use rate has increased with small fluctuations across the time series examined, from a low of 71.3% (67.6–75.1) in 2004 to a 2020 estimate of 87.1% (83.1–90.3).

63. ^{^}

    If you look into my model, you will see I alternatively make this adjustment by directly reading the Incidence RR from the model (using the use rates for the year indicated by GiveWell, rounded to nearest 5%) 

64. ^{^}

    Most I do is err towards conservatism on small things, such as treating these effects for children under 10 as being applicable to children under 5^[9], and being perhaps naively trusting of the very high study LLIN use rates reported by cohort children.

65. ^{^}

    https://www.theglobalfund.org/media/5861/psm_llinreferenceprices_table_en.pdf. 

    2.56/1.95 = 131%; 2.65/1.95 =  136%

    2.84/2.08 = 137%; 2.94/2.08 = 141%

66. ^{^}

    For CEA reasons only. I am sure that AMF will not be distributing 100% pyrethroid+chlorfenapyr LLINs anytime soon and likely never (researchers often warn about flooding the ecosystem with these LLINs in order to manage chlorfenapyr resistance).

67. ^{^}

    Absolute costs do not matter for the pre-leverage/funging percentage changes for under 5 mortality. 

    They do for the post-leverage/funging percentage changes for units of unconditional cash transfers. The percentage change is negatively associated with the absolute price. Hence the reason for realistic but slightly conservatively-high estimated costs. I have not attempted to understand leverage and funging calculations.

68. ^{^}

    Known as being "prequalified".

69. ^{^}

    More accurately, they need to pass non-inferiority "semi-field trials", all of which I have seen are EHTs. However, I think I have seen implications of semi-field trials that are not EHTs.

70. ^{^}

    these are entomological outcome measures

71. ^{^}

    https://extranet.who.int/prequal/vector-control-products/prequalified-product-list

72. ^{^}

    I believe this artificial ageing technique has questionable validity to real world, retention does not necessarily equate to bioavailability on the surface, and does not track linearly with effectiveness.

73. ^{^}

    The product’s inclusion within the policy class should be re-examined once experimental hut trial data become available for products aged for 3 years under user conditions  (page 7).

74. ^{^}

    I believe you cannot expect field trials with epidemiological data