Inferring the epidemiological benefit of indoor vector control interventions against malaria from mosquito data
Systematic reviewA systematic review (PROSPERO Registered: CRD42020165355) of all cluster-randomised control trials currently published on ITNs [including conventional nets (CTNs), pyrethroid-only long-lasting nets (pyrethroid-nets), and pyrethroid-piperonyl butoxide synergist nets (pyrethroid-PBO ITNs)], IRS or a combination of both interventions was completed to validate an established transmission model for Plasmodium falciparum malaria parameterised using entomological assessment of the interventions. Three search platforms, Web of Knowledge, PubMed and Google Scholar were used and further studies were included from three recent Cochrane reviews that have focused on individual- or cluster- randomised control trials testing either ITNs, IRS or both26,27,28. Our search criteria focused on studies within Africa, and those reporting an epidemiological outcome such as parasite prevalence or clinical incidence in a defined age-cohort. A total of 138 studies were initially identified for further assessment (Supplementary Fig. S2).Those papers identified through the systematic review went through another round of screening to ensure they fell within the scope of the work and were compatible with existing modelling parameterisation. These criteria included (i) the intervention falls within an existing World Health Organization recommendation (so trials, or arms of trials, investigating pyrethroid-pyriproxyfen ITNs29 or insecticide-treated curtains30 were excluded), (ii) the entomological impact of the product had been previously statistically characterised as part of the modelling framework (trials investigating DDT31 or propoxur IRS32 were excluded), (iii) the study was within the Africa continent, (iv) the study randomised interventions in the intervention arm across the community (i.e., interventions were not targeted to individuals or risk groups within the community)33,34,35, and (v) the study was not reporting a cluster-randomised design36. A full description of why studies and arms were excluded is provided in Data S1.1.RCTs can assess the public health impact of interventions using different epidemiological endpoints. The two most common metrics used in malaria RCTs is infection prevalence (generally assessing parasitemia in a particular age group using microscopy or rapid diagnostic tests) or clinical incidence (typically assessed using active case detection in a cohort, which had previously been cleared of infection). These metrics are both equally valid though may give different results. For example, it may be harder to change malaria parasite prevalence with a partially effective intervention in a high-transmission setting (where people have a high chance of being reinfected) compared to a low-transmission setting (where reinfection is less common). Similarly, estimates of clinical incidence will vary depending on the study design and regularity of follow-up. For example, there are practical constraints on the number of times people within an active cohort can be tested. In areas of higher transmission incidence estimates will be greater the more regularly the cohort is tested as people infected multiple times between screening will be less common. This information on the regularity of screening is not always reported making it difficult to adjust models accordingly. It is also important to account for cluster-level effects when interpreting trial results, and this cluster-level data is also mostly unavailable37. The systematic review identified more studies that evaluated interventions in their ability to change malaria prevalence, with 13 out of 14 RCTs showing how the intervention changed parasite prevalence between the study arms compared with 8 RCTs, which reported changes in clinical incidence. Therefore, we focus on prevalence as our metric for epidemiology impact in this framework though note this should be repeated with clinical incidence estimates should more data become available. The final dataset had 73 cross-sectional surveys of prevalence in a defined age-cohort, 37 trial arms from 13 different RCTs.Characterising the entomological impact of ITNs and IRSExperimental hut trials (EHTs) measure the outcome of wild, free-flying, mosquito attempting to feed on volunteers resting indoors in the presence of an indoor intervention38. This includes (i) whether or not a mosquito is deterred away from a hut, which has the intervention (calculated by the number of mosquitoes found in the control hut relative to the intervention hut), (ii) whether the mosquito exits without feeding (repellence, measured as the percentage of alive unfed mosquitoes inside the intervention hut), (iii) the percentage entering the hut that successfully blood-feed, or (iv) the percentage of mosquitoes which die. Intervention efficacy is typically summarised for the intervention huts relative to a no-intervention (or untreated net) control huts, be it induced mortality (the increase in the percentage of mosquitoes dying over a 24-h period) or blood-feeding inhibition (the reduction in the percentage of mosquitoes receiving a blood-meal).EHTs use specially built structures that follow a defined floor-plan and set of specifications. There are multiple designs of experimental hut as they were originally intended to replicate the predominant type of housing found in the local area. We recently conducted a systematic review to capture the average behaviours of mosquitoes across different hut designs19. The two most used huts in Africa are the West African design and East Africa hut39 (a third hut—the Ifakara hut—is not considered here39). The meta-analyses showed that the associations describing the probable outcome of a mosquito feeding attempt (deterrence, repellence, successful feeding, or death) varies according to hut design. It is unclear that hut design best predicts epidemiological impact.Meta-analyses of EHT data have shown how the entomological efficacy of pyrethroid-nets has diminished over time, probably due to the rise of pyrethroid-resistant mosquitoes16,19,40, though there may be some manufacturing changes41. EHTs are conducted throughout Africa but are limited to the sites where the huts are built and cannot directly inform estimates of ITN efficacy outside of these areas. The most widely used quantitative measure for approximating the phenotypic level of resistance in the local mosquito population is the discriminating-dose bioassay. There are two main types of discriminating assays, the WHO susceptibility bioassay and the CDC bottle bioassay42,43. Both these assays measure the proportion of local Anopheline mosquitoes that survive 24-h following exposure to a discriminatory dose of pyrethroid for 60 min. Results from these bioassays are highly variable44 though collating data from multiple tests has shown clear trends over time45. The relationship between the level of resistance in the local mosquito population (as measured in a discriminating-dose bioassay) and the mortality induced by ITNs in EHTs can be used to extrapolate the results from hut trials to other geographical regions16.Modelling rationaleThe two main metrics recorded in EHTs do not capture all entomological impacts of ITNs and IRS. Though useful, induced mortality does not consider the sub-lethal impact of interventions whilst blood-feeding inhibition fails to differentiate between preventing blood-meals and killing mosquitoes, which are likely to have very different epidemiological impacts. Killing mosquitoes reduces the force of infection for users and non-users (through a community effect) so the overall effectiveness of treated nets and IRS will vary according to how abundantly and regularly they are used by the local human population. In addition, the impact of ITNs and IRS is likely to vary between sites because of factors such as the disease endemicity itself driven by societal behaviours, seasonality of transmission and the use of other malaria control interventions, amongst others. This means that raw EHT data is unlikely to directly correlate with the results of RCTs.EHTs are widely used to parameterise malaria transmission dynamics mathematical models46,47,48. These models rigorously quantify the outcome of each mosquito feeding attempt and, by making a limited number of assumptions, can estimate an overall entomological efficacy by combining the impact of the level of personal protection elicited by the intervention to the user and the indirect community effect provided to both users and non-users. Transmission dynamics mathematical models are designed to mechanistically capture the underlying processes governing malaria transmission and so can account for known non-linear processes such as the acquisition of human immunity49,50,51. This enables these models to translate the entomological efficacy quantified in an EHT into predictions of epidemiological impact given the characteristics of the site. Unfortunately, to date, there are no published EHTs that have been conducted alongside RCT evaluation of ITNs or IRS products (and therefore evaluated against the same mosquito population). To overcome this issue we parameterise the models using a meta-analyses of 136 EHT results16,19 collated from across Africa, which quantifies how mosquito deterrence, repellence, successful feeding, or death varies with time since the intervention is deployed and according to the level of pyrethroid resistance in the local mosquito population (as measured by the discriminating-dose bioassay). This approach has been able to recreate the epidemiological impact observed in RCTs evaluating a small number of ITNs15 or IRS products9, but this is the first attempt at using this method to validate the modelling framework against all trials evaluating nets and IRS.There is considerable uncertainty in how the entomological efficacy of treated ITNs varies with the level of resistance in the local population. This is a key relationship determining how field discriminating-dose bioassay data should be interpreted yet it is highly uncertain, with a recent meta-analyses indicating that it is equally well explained by two different functional forms (the logistic or log-logistic functions)19. Similarly, it is unclear whether the epidemiological impact of ITNs or IRS is best captured by all experimental hut data combined (Supplementary Fig. S14C, D)19 or if the meta-analyses should be restricted to just West or East African hut design data alone. To rigorously differentiate between these options six different models are run for each trial arm (n = 37), varying both the relationship between discriminating-dose bioassay and EHT mosquito mortality (either the logistic or log-logistic function) and the data used in the EHT meta-analyses (all data, East or West African design huts). The ability of these models to recreate the observed results is statistically compared and the most accurate selected for the main analyses.Transmission dynamics modelThe malaria transmission model that we use here incorporates the transmission dynamics of Plasmodium falciparum between human hosts and Anopheles mosquito vectors. The differential equations and associated assumptions of the original transmission model52 have been comprehensively reported in the Supplementary Material from Griffin et al.53, Walker et al.54 and Winskill et al.55. The model has been extensively fitted to data on the relationship between vector density, entomological inoculation rate, parasite prevalence, uncomplicated malaria, severe disease and death49,52,53,56,57. Model equations and assumptions are provided in the Supplementary Methods and https://github.com/jamiegriffin/Malaria_simulation. Unless stated (Supplementary Data S1), default parameters are taken from these papers.Data requirements for model simulationThe transmission model can be parameterised to describe the specific ecology of each RCT location using data on the mosquito bionomics, seasonal transmission patterns, historic use of various interventions—principally insecticide-treated ITNs or the residual spraying of insecticides (IRS)—and baseline endemicity. These data are recorded within the research articles reporting the trials at the trial arm level (Supplementary Data S1.2 notes where data are available and which resources were used; Supplementary Data S1.3 lists the key data identified for model parameterisation) and Supplementary Fig. S1 provides a diagram of how they are combined to inform the model.Briefly, the Anopheles mosquito species composition at baseline is used to determine the proportion of mosquitoes with bespoke behaviours that could alter exposure risk to mosquito bites and thus transmission risk. Species-specific mosquito behaviours are parameterised from systematic reviews on anthropophagy, using the human blood index47,58,59, and the proportion of mosquito bites that are received indoors or in bed because this impacts the efficacy estimate for indoor interventions60.Other information that are specific to each trial also help interpret our success at predicting, or not, the observed results of an intervention tested in an RCT; the diagnostic used to measure prevalence or incidence is useful because different tests have different sensitivities61, which can be included in the model framework54. The baseline burden of infection is particularly important to enable the model to be calibrated to the endemicity of the study site by varying the number of mosquitoes per person (the human:mosquito ratio). This is determined by a cross-sectional estimate of parasite prevalence in a defined age-cohort at a particular time of year of the baseline survey.For any location, the current level of endemicity is determined by the historic interventions already operating at the site. Therefore, wherever possible, ITN use and the historic use of sprayed insecticides, as well as the estimated proportion of clinical cases that are drug-treated, are included as baseline parameters.In addition to the waning potency of insecticide active ingredient outlined above, the impact of nets can also wane because of changes in the proportion of people using them. This can be driven by the quality of the product, seasonal patterns in humidity or other social patterns of use62,63,64. Where data are available, this waning adherence to net use is captured by fitting an exponential decay function to the proportion of people using nets measured at cross-sectional surveys throughout the trials:$${{{{{{{mathrm{U}}}}}}{{{{{mathrm{sage}}}}}}}}_{i}={e}^{-{sigma }_{i}t}$$
(1)
where σ is a parameter determining how rapidly people stop using nets in an intervention arm i of the trial and t is time in years. Parameter estimates for pyrethroid-only and pyrethroid-PBO ITNs are provided for different levels of resistance for the 6 potential methods of associating bioassays and using data (Supplementary Data S1.4).The IRS product used is equally important as the entomological impact of different products vary, particularly for pyrethroid-based IRS in the presence of resistant mosquitoes9. Supplementary Data S1.5 show the parameter estimates for products included in the analysis.The seasonality of transmission has been defined previously for each RCT site (at the administration subunit 1 level) using normalised rainfall patterns obtained from the US Climate Prediction Center65. The daily time series are aggregated to 64 points per year for years 2002 to 2009. A Fourier function is fitted to these data to capture seasonality by reconstructing annual rainfall patterns54,66. We deliberately do not match rainfall data from the respective RCTs, which would likely improve the model estimates because we are ultimately testing whether this framework has predictive power across future years or alternative ecologies, where we will not know how rainfall will exactly impact mosquito densities and hence malaria transmission.Statistical analysisThe mean simulated malaria prevalence (matching the age-cohort of the trial) is recorded for all RCT surveys timepoints. This equates to a total of 73 cross-sectional surveys post-implementation. The process was repeated using the 6 different entomological parameter sets (the relationship between bioassay and hut trial mortality and the hut design used to summarise treated net entomological impact). An illustration of the different models and their fit to data is demonstrated in Supplementary Fig. S17 for a recent study trialling pyrethroid-only nets, pyrethroid-PBO ITNs alone or in combination with a long-lasting IRS product in Tanzania5. The difference between the observed and predicted prevalence at each timepoint is shown for all RCTs in Supplementary Fig. S18. A simple linear regression is conducted comparing observed and predicted results are summarised in Supplementary Table 3. Let Xi denote the malaria prevalence predicted by the model at timepoint i while Yi is the observed prevalence. The regression,$${Y}_{i}=m{X}_{i}$$
(2)
for i = 1,…,c + n, where m is the gradient between the observed and predicted result (consistent across studies), c is the number of post-intervention datapoints in the control arms and n is the number of post-intervention datapoints in the intervention arms (c + n = 73 for analyses of all RCTs). Better fitting models have a higher adjusted R2 (adjusted R2 values of one indicate the model is perfectly predicting the trial result) whilst the gradient of the regression m indicates any bias (with value of one reporting the model can predict prevalence equally well across the endemicity range). Results are presented for all ITNs and IRS RCTs and separately for RCTs of different types of (pyrethroid-only ITNs, pyrethroid-PBO ITNs and IRS, Supplementary Table 3). The log-logistic model (results 4–6 in Supplementary Table 3) describing the relationship between bioassay and hut trial mortality consistently fits the data better, with models fit using either all hut trial data or East African design huts having a similar accuracy (adjusted R2 = 0.95). This parameter combination also had the least bias, with the best fit regression line being closer to one.The average efficacy of the different ITNs and IRS combinations was calculated by comparing malaria prevalence for the different trial arms to the respective control arms at matched timepoints following the introduction of interventions. Let ({E}_{{jk}}^{l}) be the relative reduction in the malaria prevalence between the control (k = 0) to intervention (k = 1) arms at matched timepoint j in the same trial for either the predicted (l = Xjk) or observed (l = Yjk) malaria prevalence,$${E}_{j}^{X}=({{X}_{j0}-{X}}_{j1})/{X}_{j0},{{{{{rm{ and }}}}}},{E}_{j}^{Y}=({Y}_{j0}-{Y}_{j1})/{Y}_{j0}$$
(3)
for j = 1,…,n. The goodness of fit for the efficacy estimates is calculated in a similar manner to the prevalence estimates by substituting in ({E}_{j}^{X}) and ({E}_{j}^{Y}) into Xi and Yi in E2, respectively. Models are on average able to estimate the efficacy of the interventions at different timepoints (Supplementary Table 3). Estimates for some timepoints diverge substantially (for example, the study testing conventional nets in the Gambia relative to untreated nets67 measured negative effect in one setting; the treated net arm having more infected children whereas the model predicted a 12.5% reduction due to the CTN (with parameters derived from all EHT data and the log-logistic function, 4 in Supplementary Table 3), Supplementary Data S1.8), but in most studies the trial average (averaged across all timepoints) is remarkably consistent. Accuracy is lower than estimates of absolute prevalence, in part because the difference between the percentage of people slide positive in low-endemicity settings may be relatively modest in absolute terms but might represent a substantial difference as a percentage. It is also important to note that when the models do systematically miss some timepoints, this is consistent across the control and treated arms. For example, in the Protopopoff et al. study in Tanzania5 (Figs. S14 and S17) efficacy is over-estimated in all arms 18 months after the start of the trial, but the relative difference between the arms (in terms of ordering, and the efficacy estimate) is relatively consistent. This indicates that unmeasured factors, such as differences in the timing and duration of the rainy season, may have occurred across all trial arms. As previously, the log-logistic functional form describing the relationship between bioassay and hut trial mortality consistently fits the data better (Supplementary Table 3, options 4 to 6). The models fit describing the entomological efficacy of any net using all EHT data predicts efficacy data better with East African design hut data providing similar accuracy (adjusted R2 = 0.64 vs. 0.62, respectively). Following this we select the log-logistic functional form to describe the relationship between mortality in the discriminating-dose bioassay and EHT and characterise the entomological efficacy of treated ITNs using data from both East and West African design huts for the main analyses (Fig. 2B, C).The ability of the best-performing model (Supplementary Table 3, column 4: log-logistic function and all EHT data) to capture the relative drop in prevalence over time compared to the baseline (pre-intervention) estimate is shown in Supplementary Fig. S19. This value is denoted as ({dot{E}}_{t}^{l}) and is calculated as,$${dot{E}}_{t}^{X}=({X}_{0}-{X}_{t}),{{{{{rm{and}}}}}},{dot{E}}_{t}^{Y}=({X}_{0}-{Y}_{t})$$
(4)
where ({X}_{0}) is the malaria prevalence at baseline (prior to intervention deployment with the exception of Chaccour et al.68) observed from the RCT and the model is calibrated to this endemicity. Xt is then the subsequent cross-sectional survey observed for each study, and RCTs have different numbers of surveys ranging from 1 to 4 in the published literature. The corresponding model estimate is represented by Yt. Estimates are calculated for all post-intervention timepoints in both control and intervention arms and are shown in Fig. S19A. The difference between ({dot{E}}_{t}^{X}) and ({dot{E}}_{t}^{Y}) can be used to explore how closely the model is able to predict this absolute difference observed in the trials (a value of 0 indicates exact match, high predictive ability). The model overestimates the performance of IRS only, deployed in 1995 using the pyrethroid IRS ICON CS 10% (Syngenta), but otherwise there is no difference in the models’ ability to estimate different ITN interventions or combination net and IRS interventions, be it the absence of an intervention, conventional dipped-nets, pyrethroid-only nets, pyrethroid-PBO ITNs with or without IRS (Fig. S19B). All code is available69.Reporting summaryFurther information on research design is available in the Nature Research Reporting Summary linked to this article. More
