Published on in Vol 5, No 10 (2021): October

Preprints (earlier versions) of this paper are available at, first published .
The Potential of Digital Data Collection Tools for Long-lasting Insecticide-Treated Net Mass Campaigns in Nigeria: Formative Study

The Potential of Digital Data Collection Tools for Long-lasting Insecticide-Treated Net Mass Campaigns in Nigeria: Formative Study

The Potential of Digital Data Collection Tools for Long-lasting Insecticide-Treated Net Mass Campaigns in Nigeria: Formative Study

Original Paper

1Boston Medical Center, Boston, MA, United States

2Catholic Relief Services, Baltimore, MD, United States

3Graduate Institute of International Development Studies, Geneva, Switzerland

4Catholic Relief Services, Abuja, Nigeria

5Red Rose Ödeme Sistemleri AŞ, İstanbul, Turkey

Corresponding Author:

Youngji Jo, PhD

Boston Medical Center

One Boston Medical Center Pl

Boston, MA, 02118

United States

Phone: 1 4438001626


Related ArticleThis is a corrected version. See correction statement in:

Background: Nigeria has the world’s largest malaria burden, accounting for 27% of the world’s malaria cases and 23% of malaria mortality globally. This formative study describes the operational process of the mass distribution of long-lasting insecticide-treated nets (LLINs) during a campaign program in Nigeria.

Objective: This study aims to assess whether and how digital data collection and management tools can change current practices and help resolve major implementation issues.

Methods: Qualitative data on the technical features and operational processes of paper-based and information and communication technology (ICT)–based systems in the Edo and Kwara states from June 2 to 30, 2017, were collected on the basis of documented operation manuals, field observations, and informant interviews. During the LLIN campaign in Edo State, we recruited 6 local government area focal persons and monitors and documented daily review meetings during household mobilization (9 days) and net distribution (5 days) to understand the major program implementation issues associated with the following three aspects: logistic issues, technical issues, and demand creation. Each issue was categorized according to the expected degree (low, mid, and high) of change by the ICT system.

Results: The net campaign started with microplanning and training, followed by a month-long implementation process, which included household mobilization, net movement, net distribution, and end process monitoring. The ICT system can improve management and oversight issues related to data reporting and processes through user-centered interface design, built-in data quality control logic flow or algorithms, and workflow automation. These often require more than 50% of staff time and effort in the current paper-based practice. Compared with the current paper-based system, the real-time system is expected to reduce the time to payment compensation for health workers by about 20 days and produce summary campaign statistics for at least 20 to 30 days.

Conclusions: The ICT system can facilitate the measurement of population coverage beyond program coverage during an LLIN campaign with greater data reliability and timeliness, which are often compromised due to the limited workforce capacity in a paper-based system.

JMIR Form Res 2021;5(10):e23648




Nigeria has the world’s largest burden of malaria, accounting for 27% of the world’s malaria cases and 23% of malaria mortality globally [1]. The disease continues to be a pressing public health challenge, responsible for 60% of the country’s outpatient visits and 30% of hospitalizations [1]. Long-lasting insecticide-treated nets (LLINs) have played an important role in the remarkable reduction in the number of malaria cases and deaths worldwide over the past decade. Sleeping under a net has been shown to reduce all-cause mortality in children younger than 5 years by about 20% and malarial illnesses among children younger than 5 years and in pregnant women by up to 50% [2]. Studies demonstrating the effectiveness [3-6] and cost-effectiveness [7,8] have contributed to substantial success in scaling up this intervention in malaria-endemic African countries over the past decades [9]. However, more progress is required to improve sustained coverage [10] as well as access and use from an equity perspective [11].

Mobile phone technology coupled with electronic geographic information systems (GISs) has made data collection processes easier and more accurate and has become a powerful tool for managing data in the context of malaria prevention and control [12-15]. Modernized high-resolution mapping technologies and satellite imagery can help locate target populations and provide reliable estimates of populations at risk. This can also facilitate the identification of issues pertinent to health service delivery and produce data that can guide better intervention strategies so that scarce resources can be allocated in a cost-effective manner, especially in low-income countries [16]. Such systems allow government health officials in capital cities to better plan interventions in remote rural areas. This, in turn, promises a better assessment approach for trends and correlations to improve the planning, monitoring, and surveillance of public health programs in resource-limited countries.


The use of digital data collection and management tools based on information and communication technology (ICT) can promote supply chain management [17-19] and sustainable and effective use of LLINs [20,21]; however, there is not enough evidence or guidelines that describe the specific features, processes, and consequences of how the ICT system could change the current practice. From the perspective of management decision-making, the goal of this study is to describe the detailed process and system of mass distribution LLIN campaign programs and identify major challenges and resolution approaches in the paper- and ICT-based systems. The study findings will discuss whether and how a digital data collection and management tool can potentially improve the management, monitoring, and evaluation of the program.

Study Setting

In Nigeria, Catholic Relief Services (CRS) is working in partnership with the National Malaria Elimination Program (NMEP) to distribute LLINs to prevent malaria through a program funded by the Global Fund to Fight AIDS, Tuberculosis, and Malaria [22]. Together with NMEP and state ministries of health, CRS implemented malaria LLIN replacement campaigns between April and November 2017 in 6 Southeast Nigerian states: Kwara, Ondo, Imo, Edo, Osun, and Adamawa. Among the 6 states, we collected data from Edo State—in all 18 local government areas (LGAs)—using the traditional paper-based intervention, and from Kwara State with a pilot ICT-based intervention in 1 LGA (Oyun). According to the 2016 census, Edo State consists of approximately 4 million people [23], and Kwara State consists of approximately 3 million people [24]. One LGA generally consists of approximately 16 to 30 wards, and the size of a ward varies, consisting of 500 to 8000 households per ward [25].

Research Participants

One researcher (YJ) and 2 technical staff members (OA and EE) conducted structured in-depth interviews with 10 CRS program managers and 26 local stakeholders (eg, local partner organizations, government officers from NMEP, CRS local staff) working in Edo and Kwara states. We conducted field observations of the paper-based system in Edo State. We recruited 1 LGA focal person and 1 monitor from 6 LGAs and documented daily review meetings during the household mobilization (9 days) and net distribution (5 days) campaign from June 2 to June 30, 2017. We could not conduct field observations of the ICT system in the Kwara State, as the campaign schedule had to be changed owing to delayed mobile phone procurement. Thus, we conducted informant interviews with 2 field managers in Kwara and 2 technical staff who developed the Red Rose system to understand the system characteristics, operational procedure, challenges, and resolution approaches of the ICT-based system.

Data Collection and Analyses

We reviewed the documented operation manuals and conducted informant interviews to map out operational processes and identify the technical features of paper- and ICT-based systems (Figure 1). On the basis of the documented contents during LGA- and state-level review meetings, we categorized the major program implementation issues discussed during the meetings into the following three campaign activity components: (1) logistic issues, (2) technical issues, and (3) demand creation activity and identified typical resolution approaches in paper-based campaigns. Each issue was then reviewed and categorized into low, middle, and high levels based on the expected degree of change by the ICT system (Multimedia Appendix 1).

LLIN Campaign Structure and Processes

The campaign consisted of multiple levels of staff and a set of activities, including planning, LLIN logistics, training, demand creation, household mobilization, net distribution, and in or end process monitoring. First, during the macroquantification stage, the required number of LLINs for each LGA was determined on the basis of the working assumption of “a 1.8 person-per-net ratio” [26], a set of simple formulas for calculating administrative coverage, and data from a population-based survey or census. On the basis of the required numbers of LLINs, a logistic plan, including net procurement for each LGA, was developed. Microplanning at the state level determined the number and location of each settlement in the LGA.

Next, the training took place at the state, LGA, ward, and community levels as a cascade or training of trainers model. Through existing social and community networks, media jingles, community sensitization, and demand creation, activities were implemented to increase community awareness of the LLIN campaign and improve demand for and use of nets. Then, household mobilization took place through door-to-door visits, in advance of the net distribution, to encourage active participation in the campaigns, register households, issue net cards based on the number of household members (ie, 1 net card to be issued per 2 people but a maximum of 4 net cards to be issued per household), and provide information about collecting and using the LLINs. Household registration teams wrote information about the LGA or ward names of the household, the designated location of distribution point (DP), and the pick-up date on the net cards when they issued them to the households. Each registration team registered 24 to 35 households per day. On the scheduled dates, beneficiaries brought their net cards to their designated DP. At the DP, their net cards were exchanged for a net. Monitors randomly visited 8 to 10 households per day during the household mobilization period and 2 to 4 DPs during the net distribution period to check the net card and net distribution status during the campaign.

In parallel, the review meetings were conducted during the household mobilization activities (9 days) and net distribution activities (5-7 days) to monitor daily performance and address any challenges. The meetings were held at LGA and state levels. At the LGA level, ward or DP supervisors gathered at a meeting and reported campaign progress in their catchment areas to an LGA focal person. After the meeting, LGA focal persons and monitors gathered for a state-level meeting to report the progress and discuss major issues with state supervisors and a committee that consisted of government officers from NMEP, CRS program staff, and local partner organizations. An end process evaluation (also known as a rapid assessment) was carried out to check the net distribution status by the independent monitors 2 days after distribution in all LGAs using the two-stage cluster sampling (ie, 4-4-10 principle: a convenient sampling of 4 wards in each LGA, 4 communities in each ward, and 10 households in each community, respectively). All the data from the process were submitted directly to the state data manager for validation and compilation. The information retrieved was used to prepare the state debriefing presentation, state campaign report, and postcampaign behavior change communication priorities.

Data Flow of the LLIN Campaign

The net campaign started with microplanning and training followed by a month-long implementation, including household mobilization, net movement, net distribution, and end process monitoring (Figures 1 and 2). In paper-based systems, training attendance was checked by training attendance sheets filled by participants, and payments were manually processed based on the records (Figure 1). For paper-based registration, the household mobilization team consisted of 2 people who visited households for registration. At each household visit, one household mobilizer filled out the Household Registration Sheets (Form I-3, as presented in Figure 1) and another household mobilizer completed the Net Cards Distribution Daily Summary Sheet (Form I-4a). Once registered, households were given net cards based on the number of household members (1 net card per 2 household members, but a maximum of 4 net cards per household). At the end of the day, ward supervisors collated the net card distribution summary sheet (Form I-4b). Then, LGA supervisors completed the LGA Nets cards distribution summary sheets (Form I-4c) at the LGA and collated them into the LGA LLIN Summary Sheet (Form I-8c) at the state level at the end of the campaign.

Figure 1. Operation process and data flow of the paper-based system of the long-lasting insecticide-treated net campaign in Edo State. DP: distribution point; HH: household; ICC: inventory control card; LGA: local government area.
View this figure
Figure 2. Operation process and data flow of the information and communication technology–based system of the long-lasting insecticide-treated net campaign in Kwara State. DP: distribution point; HH: household; ICC: inventory control card; LGA: local government area.
View this figure

On the basis of the summary data at the end of the household mobilization activities, net positioning was planned for each designated DP (which takes an additional 5-7 days). In terms of net logistics, inventory control card forms were used to track the net inventory stock status by DP supervisors and were submitted to the LGA supervisor. Waybill forms were used to record net movement information such as types of vehicle, amount of nets, date of delivery, and sender or recipient of the nets. During net distribution, distributors filled the Daily Net Collection Tally Sheets (Form I-7) and DP supervisors filled out the DP net collection summary sheets (Form 1-8a) at the end of the day. Then, LGA supervisors filled out the LLIN and Net Card Aggregation Summary Sheets (Form I-8b) at the end of the net distribution and LGA LLIN summary sheets (Form I-8c) based on I-4c and I-8b forms from all LGAs.

At the end of the distribution, the DP supervisor calculated the total number of LLINs distributed over the 5-day distribution period and the 2 additional days, as well as the number of LLINs remaining in the DP storage, to the ward supervisor using the inventory control card form. Net distributors recorded these forms, and ward supervisors and LGA supervisors collated the data. Data clerks at the state level then finally entered the data into computers. In addition, independent monitors filled in in-process monitoring forms during household mobilization (Form I-9a) and net distribution (Form I-9b) periods. At the end of the net distribution, the independent monitors filled the end-process monitoring forms. Overall, these processes took approximately 20 to 30 days from the initiation of household registration activities in the current paper-based practice.

The ICT system included the use of a technology platform—Red Rose software loaded onto Android phones that can function in real time whether it is connected or disconnected to the internet network (Figure 2). The overall campaign operation processes and staffing structures were similar to those of the paper systems. The Red Rose platform was used throughout the campaign, including for tracking attendees at different training events, mobile money payments, household mobilization, net distribution, in-process monitoring, and end-process monitoring surveys. Portable power banks or wireless network adapters were provided for network connectivity. During training, biometric sensors tracked the participants’ check-in or check-out times, and the participation records were automatically processed for a payment schedule. Similar to the paper system, the household mobilization team consisted of 2 people who visited households for registration. Once registered, households were given identifiable net cards based on the number of household members. The net cards were associated with a set of unique information, including household ID, household register ID, household phone number, mobilization date, designated DP, geocoordinate, and time data at the event of household registration and net distribution. Each net card was preprinted with an individual machine-readable quick response (QR) code, scanned using Android phones to ease tracking and prevent errors that may arise from manual entry. These QR codes were scanned by mobile phones in every transaction of the net card distribution and collection during household registration and net distribution. In this way, each net card distributed was entered into and tracked by the Red Rose platform. The number of LLINs that a household receives was automatically calculated when the number of people in the household was entered into the Android device. At each DP, a DP supervisor recorded the net stock position in the Red Rose platform. When LLINs were handed to beneficiaries, each net card was scanned, and the stock position was automatically updated. The progress of net card distribution and collection, as well as stock positions, was viewed regularly on customized and interactive dashboards. The application had built-in anomaly detection algorithms for inactive net cards if the net cards were not registered or assigned to different DPs. Once these data were entered by household mobilizers and net cards were scanned by net distributors, each data point was automatically synchronized based on a unique net card ID (QR code) and calculated for summary data for DP, ward, LGA, and state levels in real time.

Program Implementation Issues

As described in Table S1 in Multimedia Appendix 1, the characteristics of paper-based systems have a high dependence on individual capacity (ie, how an individual performs his or her task and the capacity of supervisors to effectively monitor activities across a wide geographic area), which are inherently exposed to several potential risks [19]. For example, during microplanning, there was a lack of reliable, up-to-date population figures of census units or village-level population, which resulted in gaps and discrepancies in household registration or net distribution planning [27]. During the training, participants’ attendance checks and bank verification for payment took a long time (at least a month), which might reduce staff motivation. During the LLIN campaign, most issues were identified through staff reporting during review meetings or through oversight visits by supervisors. The reporting system also had multiple levels of reporting, with exposure to errors in data entry and requiring additional manual scrutiny. Although the system required a large volume of hard copies of reports and forms, which often require more than 50% of staff time and effort in data entry, the reporting and processing of information flow was one-way and not actionable. As a result, effective monitoring was complicated and laborious, often done merely by occasional checks (spot checks) by supervisors to monitor attendance and performance of staff; even if these were identified, there were limited mechanisms to ensure that the issues were correctly resolved. Although campaign staff were generally highly committed to their work, willing to serve the community, and had good knowledge of national guidelines, these attributes were not enough to overcome various implementation challenges, coupled with shortages of qualified workforce in resource-limited settings.

Of the identified operational issues, the ICT system mostly influenced the technical issues, in particular household mobilization and net distribution activities compared with logistic issues and demand creation issues. The ICT system has the potential to improve management and oversight through user-centered interface design, built-in data quality control logic flow or algorithms, workflow automation, and visualized interactive dashboards. The data linked to GIS information and an automated alert system could help supervisors detect issues and take immediate actions to resolve them. For example, when supervisors reviewed the satellite maps and discovered households that were not provided with net cards, they could guide them to visit the missing household. This allowed us to identify the target population more closely, without missing settlements or households. Real-time workforce performance data improved management and facilitated monitoring and supervision during household mobilization and net distribution.

Data Outputs

At the end of each phase of activities, each level’s supervisor reported, collated, and produced a summary of the data in the respective catchment areas indicating the total number of households (or individuals) reached, total number of LLINs needed, and total net cards distributed. The coverage estimates were generally high, ranging between 69% (estimated program coverage of 142,635/206,717 based on population size in Orhionmwon) and 99% (152,841/154,385 in Owan east) across 18 LGAs in Edo State (Figure S1 in Multimedia Appendix 1). The supervisors also provided DP data on the total number of net cards received, the total number of LLINs distributed, and the total number of LLINs remaining at the end of the distribution phase. These coverage measures (ie, household registration rate and net redemption rate) were compared across LGAs. In addition, end-process monitoring produced various snapshots of the overall campaign performance, based on the sampled household, in terms of net use rate, net retention rate, and net redemption rate across LGAs.

In the ICT-based system, various visual dashboards (diagrams and maps) generated from the collected data provided a comprehensive projection of the performance and coverage data according to the geographical location in close to real time (Figures 3-5). The Red Rose interactive visualization dashboard system allowed data to be tabulated, analyzed, merged, sorted, and filtered interactively to assist exploratory analyses and examine performance progress. Depending on the choice of parameters, users could decide the various visual outputs based on their interests, such as workforce performance or geographical coverage (Figure 4). The data could be accessed in real time so that any anomalies or gaps could be addressed during the campaign, not at the end of the campaign. These included workforce performance level by showing the number, location, and time of household mobilization or net distribution of health workers, percentages of household mobilization or net distribution by geographical level or location, and the service coverage level on the satellite maps (Figures 4 and 5). It also allowed additional functions, such as presenting the names of high-performance workers based on the number of household registrations (Figure S2 in Multimedia Appendix 1).

Figure 3. Interactive visualized dashboards of the information and communication technology system in Kwara State (mobilization or distribution by time and cluster).
View this figure
Figure 4. Interactive visualized dashboards of the information and communication technology system in Kwara State (mobilization density on the map for Oyun local government area, Kwara State).
View this figure
Figure 5. Interactive visualized dashboards of the information and communication technology system in Kwara State (distribution points on the map).
View this figure

Principal Findings

The LLIN campaign with paper-based systems revealed several challenges in terms of data reliability, validity, and timeliness coupled with limited workforce capacity. Although the ICT-based system would have implementation issues similar to those in the current paper-based system, it can better manage the issues related to the data reporting system and processes, which comprise significant staff time and effort in the current paper-based practice. Consequently, staff could spend more resources and time addressing issues related to LLIN supply chain management and demand creation. The information collected by the ICT system is accurate and verifiable through personal identification, geocode, and time data, which allows real-time monitoring and effective management decision-making. It can reduce any missed settlements and households based on satellite maps during the household mobilization period and promote net card redemption during the net distribution period. Accordingly, ICT enables a system that provides a measure of accurate targets and ensures population coverage (ie, the denominator of the service coverage measure is likely the actual number of households identified by the GIS-based system and registered by staff) beyond program coverage (ie, the denominator of the service coverage measure is limited by the number of households registered by the staff) with greater confidence and transparency.


Many previous studies have demonstrated the effectiveness and cost-effectiveness of LLIN in preventing malaria infection and reducing associated morbidity and mortality [28-30]. Mass campaigns of LLIN have thus been the primary vector control strategy for malaria-endemic countries over the past decades, with universal campaigns every 3 years, which has been the norm since 2009 [31]. Although LLIN coverage dramatically increased with mass distribution campaigns, the coverage gaps started to appear almost immediately after the campaign through net deterioration or loss of nets [32,33]. Studies also demonstrated remaining gaps in geographical heterogeneity, household net access or ownership [34], and equity gaps in terms of LLIN coverage as well as prevention effectiveness of LLIN [35,36]. The ICT system allows a verifiable transaction system through the net card that connects information of people (ie, households), place (ie, geographical location), and products (ie, nets); this integrated system can be used as push and pull mechanisms to identify and address the gaps (ie, missed households or net redemption and ensuring appropriate net use) in the service delivery process [37]. Moreover, the systematic platform and information can be also used to guide targeted strategies and improve performance for other complementary continuous distribution channels (eg, antenatal care clinics and the expanded program on immunization) even after the mass campaign as well as the next cycle of mass campaigns in 3 years, which can also benefit other population-based surveillance and intervention programs, in addition to malaria [38]. Our study described the detailed process and system of the LLIN campaign and addressed contextual barriers and facilitators to implement the campaign based on the paper- and ICT-based systems. Because establishing effectiveness or innovation itself does not guarantee its scale-up and routine use, our study provides important insights and potential research agenda of implementation science not only to the traditional paper-based practice but also to the future ICT-based program design, implementation, and evaluation [39].


When the system functions well at scale, we expect the ICT-based system to promote various supply- and demand-side factors in health systems in resource-limited settings. From the supply side, it can enhance workforce capacity and productivity by improving attendance at training, providing supervisors with better tools to monitor the activities of field staff, reducing the time required for tabulating paper forms, and automating certain processes to minimize human errors. It can also improve oversight and promote data-driven management decision-making, which allows implementation challenges to be resolved in a timely manner. On the basis of the automation of data entry and reporting, resources can be invested more in behavior change communication campaigns and to deliver nets to hard-to-reach areas for effective, sustainable, and equitable net use [17,32]. The accurate household mapping and village-level population estimates can be important data at a local level that provide accurate denominator estimates for other public health services [40]. From the demand side, spatial analysis can enable campaign staff to select accessible DPs, so people spend less time traveling, and with more efficient operations to scan net cards quickly, people can also spend less time waiting to receive the nets. Furthermore, once clients’ mobile phone numbers are registered during the household registration, server systems can send SMS text messages to clients about the proper and sustained use of LLINs [20,32]. Some previous studies in sub-Saharan African countries also introduced digital health applications in LLIN campaigns for demand creation strategies by using SMS to raise awareness and eVoucher systems to monitor net distributions and reduce fraud [41]. Overall, we expect ICT to contribute to improving health outcomes through more efficient and accountable processes to ensure that all nets reach the intended users.


As an ICT system holistically changes the overall program process and dynamics, rather than simply expediting the current process, it requires new roles and practices of the major stakeholders [42]. First, implementing organizations need to develop new operations and training manuals and policies that define new roles and management responsibilities of staff. These may include identifying who are going to be responsible for tracking and analyzing all the data that came in through the ICT system, what was the daily set of tasks or standard checks to review the data, and what specific actions were taken as a result of reviewing these data, and in what time frame. It may also require new process or performance indicators beyond net coverage to ensure better quality and equity. Second, international and national stakeholders should foster enabling conditions and collaborations for microplanning, supply chain management, and workforce capacity based on a shared data platform [18,19]. For effective supervision using visual dashboards, investment in and installation of monitoring screens for the LGA-level meetings and computer system (or tablets) for the DP managers and improving electricity and network connectivity would be important in local centers. Furthermore, integration of the data collected from mobile phones into the national health information system is important for evaluating the progress and impact of the program at scale [40,43,44]. Finally, the technical organization should continuously promote the improvement of optimized workflow automation, a well-functioning logic flow, user-centered interface design, and interactive data dashboard. Ultimately, it is critical to ensure and foster effective use of data at each level of management capacity and policy making [45].


This study has several limitations. The study sites (6 LGAs) in Edo State were selected on the basis of operational convenience. As we could not conduct a similar field observation in the Kwara State due to an operational delay, alternative approaches, such as informant interviews and documentation reviews, were used to understand the ICT system by staff involved in the ICT pilot, based on their knowledge of the Red Rose platform and experience testing the system in Kwara State. Thus, although this formative study illustrates how the ICT system can potentially and positively change current practices and processes, there may be new challenges in adopting and scaling up this new system. For example, intensive learning processes may be required for staff regarding new data entry methods, workflow, and new management responsibilities. In this regard, future studies can consider observation and qualitative studies during implementation and identify challenges or unintended consequences (eg, additional training or resource requirements, system integration, stakeholder coordination) through field observations during the adaptation and scaling-up process. Future studies may also use implementation research methods, such as the normalization process theory [46] or dynamic sustainability framework [47], to understand the factors and conditions that contribute to the adoption, scale-up, and sustainability of ICT-based systems. Future studies may measure explicit quantitative or qualitative comparisons of the performance between paper and ICT systems from the aspects of the organization (ie, efficiency: time or cost savings in program implementation), health workers (ie, satisfaction or accountability: timely compensation, productivity), and community (ie, equity: equitable net distribution).


To conclude, our study shows that high program coverage estimates in a paper-based system may not be a sufficient measure to ensure effective and sustainable universal coverage of LLINs, given the existing gaps in data reliability and timeliness, coupled with limited workforce capacity. The systematic functions and various technical features available in the ICT system can improve the impact and quality of LLIN campaigns by reducing staff time and resources in the data reporting system and processes. Accordingly, it can facilitate the measurement of true population coverage beyond program coverage with greater confidence and transparency. The ICT system can facilitate effective service delivery, promote workforce capacity and performance, and enhance the transparency and accountability of public health systems in resource-constrained settings. Continued efforts are necessary to develop new operations and training manuals and policies and to foster enabling conditions and collaborations for supply chain management and workforce capacity. Our findings and lessons may provide useful insights for developing, implementing, and evaluating future ICT systems for malaria vector control and other public health campaigns.


The long-lasting insecticide-treated net campaign program and this research were supported by grants from the Global Fund to Fight AIDS, Tuberculosis and Malaria. The funding organizations did not participate in the design of the study; the collection, analysis, and interpretation of data; or the writing of the manuscript. The findings and conclusions in this study are those of the authors and do not necessarily represent the official position of the Catholic Relief Services, the Catholic Church, or the Global Fund. The authors are thankful for the coordination and support of the Catholic Relief Services Nigeria team (Imran Chishtie, Kingsley Godson, Gbenga Jokodola, Uchechukwu Ikhimioya, Orezi Ajugba, Selbo Gambo, Vyvyenne Chigboh, and Adebayo Adeshola) and local government area focal persons (Esther Enabulele Ebose, Pat Aigbovia, Queen N Omorogbe, Emma-Esekhile Pat, Amu Rose Ireoba, and Edith Esohe Osula) and monitors (Ochoche Grace, Ufort Anietie John, John Oteikwu Lucia, Babavwemu Sweet, Bienose Julia, and Ogunkoya Abiola) who helped to document daily review meetings in Edo State. The authors also gratefully acknowledge Issa Kawu, John Hembling, Stephen Helen, and Scott LeFevre for their review and comments on the manuscript and study findings. Finally, we would also like to thank the partner organizations in this campaign for their support —National Malaria Elimination Program in Nigeria, Red Rose team (Hakan Buyubayrak), and Society for Family Health (John Ocholi, Akimlalu Titilope, and Dangana Christopher)—who shared their experience and insights for this study.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Major program implementation issues and resolution approaches of the paper-based system and information and communication technology–based system in Nigeria.

DOCX File , 967 KB

  1. World Health Organization. 2020 Nov 30.   URL: [accessed 2021-06-23]
  2. The Alliance for Malaria Prevention: A toolkit for mass distribution campaigns to increase coverage and use of long-lasting insecticide-treated nets, 2012. The Alliance for Malaria Prevention. 2012.   URL: [accessed 2021-06-23]
  3. Okell LC, Griffin JT, Kleinschmidt I, Hollingsworth TD, Churcher TS, White MJ, et al. The potential contribution of mass treatment to the control of Plasmodium falciparum malaria. PLoS One 2011 May 24;6(5):e20179 [FREE Full text] [CrossRef] [Medline]
  4. Stuckey EM, Stevenson J, Galactionova K, Baidjoe AY, Bousema T, Odongo W, et al. Modeling the cost effectiveness of malaria control interventions in the highlands of western Kenya. PLoS One 2014 Oct 07;9(10):e107700 [FREE Full text] [CrossRef] [Medline]
  5. Richards JS, Arumugam TU, Reiling L, Healer J, Hodder AN, Fowkes FJI, et al. Identification and prioritization of merozoite antigens as targets of protective human immunity to Plasmodium falciparum malaria for vaccine and biomarker development. J Immunol 2013 Jul 15;191(2):795-809 [FREE Full text] [CrossRef] [Medline]
  6. Teklehaimanot A, Sachs JD, Curtis C. Malaria control needs mass distribution of insecticidal bednets. Lancet 2007 Jun 30;369(9580):2143-2146. [CrossRef] [Medline]
  7. Yukich JO, Lengeler C, Tediosi F, Brown N, Mulligan J, Chavasse D, et al. Costs and consequences of large-scale vector control for malaria. Malar J 2008 Dec 17;7:258 [FREE Full text] [CrossRef] [Medline]
  8. Value for Money in LLIN Specifications Guidance: Ensuring highest value for money in LLIN procurement by balancing cost with end-user and program benefits. USAID - Result for Development. 2012.   URL: [accessed 2021-06-23]
  9. Teklehaimanot A, McCord GC, Sachs JD. Scaling up malaria control in Africa: an economic and epidemiological assessment. Am J Trop Med Hyg 2007 Dec;77(6 Suppl):138-144. [Medline]
  10. Paintain LS, Kolaczinski J, Renshaw M, Filler S, Kilian A, Webster J, et al. Sustaining fragile gains: the need to maintain coverage with long-lasting insecticidal nets for malaria control and likely implications of not doing so. PLoS One 2013 Apr 10;8(12):e83816 [FREE Full text] [CrossRef] [Medline]
  11. Aderibigbe SA, Olatona FA, Sogunro O, Alawode G, Babatunde OA, Onipe AI, et al. Ownership and utilisation of long lasting insecticide treated nets following free distribution campaign in South West Nigeria. Pan Afr Med J 2014 Apr 11;17:263. [CrossRef] [Medline]
  12. Sipe NG, Dale P. Challenges in using geographic information systems (GIS) to understand and control malaria in Indonesia. Malar J 2003 Nov 04;2(1):36 [FREE Full text] [CrossRef] [Medline]
  13. Nhavoto JA, Grönlund A. Mobile technologies and geographic information systems to improve health care systems: a literature review. JMIR Mhealth Uhealth 2014 Aug 05;2(2):e21 [FREE Full text] [CrossRef] [Medline]
  14. Clements AC, Reid HL, Kelly GC, Hay SI. Further shrinking the malaria map: how can geospatial science help to achieve malaria elimination? Lancet Infect Dis 2013 Aug 13;13(8):709-718. [CrossRef] [Medline]
  15. Kelly GC, Hale E, Donald W, Batarii W, Bugoro H, Nausien J, et al. A high-resolution geospatial surveillance-response system for malaria elimination in Solomon Islands and Vanuatu. Malar J 2013 Mar 21;12(1):108 [FREE Full text] [CrossRef] [Medline]
  16. Wangdi K, Banwell C, Gatton ML, Kelly GC, Namgay R, Clements AC. Development and evaluation of a spatial decision support system for malaria elimination in Bhutan. Malar J 2016 Mar 22;15:180 [FREE Full text] [CrossRef] [Medline]
  17. Zöllner C, De Allegri M, Louis VR, Yé M, Sié A, Tiendrebéogo J, et al. Insecticide-treated mosquito nets in rural Burkina Faso: assessment of coverage and equity in the wake of a universal distribution campaign. Health Policy Plan 2015 Mar 30;30(2):171-180. [CrossRef] [Medline]
  18. Masaninga F, Mukumbuta N, Ndhlovu K, Hamainza B, Wamulume P, Chanda E, et al. Insecticide-treated nets mass distribution campaign: benefits and lessons in Zambia. Malar J 2018 Apr 24;17(1):173 [FREE Full text] [CrossRef] [Medline]
  19. Arroz JA, Candrinho B, Pedro S, Mathe G, da Silva M, Tsabete S, et al. Planning and implementation of a countrywide campaign to deliver over 16 million long-lasting insecticidal nets in Mozambique. Malar J 2018 Jul 09;17(1):254 [FREE Full text] [CrossRef] [Medline]
  20. Janko MM, Churcher TS, Emch ME, Meshnick SR. Strengthening long-lasting insecticidal nets effectiveness monitoring using retrospective analysis of cross-sectional, population-based surveys across sub-Saharan Africa. Sci Rep 2018 Nov 20;8(1):17110 [FREE Full text] [CrossRef] [Medline]
  21. Dolan CB, BenYishay A, Grépin KA, Tanner JC, Kimmel AD, Wheeler DC, et al. The impact of an insecticide treated bednet campaign on all-cause child mortality: A geospatial impact evaluation from the Democratic Republic of Congo. PLoS One 2019 Feb 22;14(2):e0212890 [FREE Full text] [CrossRef] [Medline]
  22. Nigeria Malaria Brief: Working with the global fund to reduce the burden of malaria in Nigeria. Catholic Relief Services.   URL: [accessed 2021-06-23]
  23. Edo - State in Nigeria. CityPopulation.   URL: [accessed 2021-06-23]
  24. Kwara - State in Nigeria. CityPopulation.   URL: [accessed 2021-06-23]
  25. Nigeria National Population Commission.   URL: [accessed 2021-06-23]
  26. Kilian A, Koenker H, Paintain L. Estimating population access to insecticide-treated nets from administrative data: correction factor is needed. Malar J 2013 Jul 26;12:259 [FREE Full text] [CrossRef] [Medline]
  27. Augustine Uchenna A. Quality micro-planning processes - Key strategy for achieving high quality LLINs Campaign: Pilot report from Cross-River State Nigeria. Euro J Prev Med 2019 May 10;7(2):32. [CrossRef]
  28. Staedke SG, Gonahasa S, Dorsey G, Kamya MR, Maiteki-Sebuguzi C, Lynd A, et al. Effect of long-lasting insecticidal nets with and without piperonyl butoxide on malaria indicators in Uganda (LLINEUP): a pragmatic, cluster-randomised trial embedded in a national LLIN distribution campaign. Lancet 2020 Apr 18;395(10232):1292-1303 [FREE Full text] [CrossRef] [Medline]
  29. Gama E, Were V, Ouma P, Desai M, Niessen L, Buff AM, et al. Large-scale implementation of disease control programmes: a cost-effectiveness analysis of long-lasting insecticide-treated bed net distribution channels in a malaria-endemic area of western Kenya-a study protocol. BMJ Open 2016 Nov 21;6(11):e012776 [FREE Full text] [CrossRef] [Medline]
  30. Worrall E, Were V, Matope A, Gama E, Olewe J, Mwambi D, et al. Coverage outcomes (effects), costs, cost-effectiveness, and equity of two combinations of long-lasting insecticidal net (LLIN) distribution channels in Kenya: a two-arm study under operational conditions. BMC Public Health 2020 Dec 07;20(1):1870 [FREE Full text] [CrossRef] [Medline]
  31. Koenker H. More is more: Are we delivering enough LLINs? EClinicalMedicine 2018 Jul;1:5-6 [FREE Full text] [CrossRef] [Medline]
  32. Girond F, Madec Y, Kesteman T, Randrianarivelojosia M, Randremanana R, Randriamampionona L, et al. Evaluating effectiveness of mass and continuous long-lasting insecticidal net distributions over time in madagascar: a sentinel surveillance based epidemiological study. EClinicalMedicine 2018 Jul 18;1:62-69 [FREE Full text] [CrossRef] [Medline]
  33. Musa JJ, Moore SJ, Moore J, Mbuba E, Mbeyela E, Kobe D, et al. Long-lasting insecticidal nets retain bio-efficacy after 5 years of storage: implications for malaria control programmes. Malar J 2020 Mar 14;19(1):110 [FREE Full text] [CrossRef] [Medline]
  34. Koenker H, Arnold F, Ba F, Cisse M, Diouf L, Eckert E, et al. Assessing whether universal coverage with insecticide-treated nets has been achieved: is the right indicator being used? Malar J 2018 Oct 11;17(1):355 [FREE Full text] [CrossRef] [Medline]
  35. Khanam F, Hossain MB, Chowdhury TR, Rahman MS, Kabir M, Naher S, et al. Exploring the gap between coverage, access, and utilization of long-lasting insecticide-treated nets (LLINs) among the households of malaria endemic districts in Bangladesh. Malar J 2018 Dec 6;17(1):455. [CrossRef]
  36. Galactionova K, Smith TA, de Savigny D, Penny MA. State of inequality in malaria intervention coverage in sub-Saharan African countries. BMC Med 2017 Oct 18;15(1):185 [FREE Full text] [CrossRef] [Medline]
  37. Guidelines for managing the malaria supply chain: a companion to the logistics handbook. USAID Global Health Supply Chain Program. 2017.   URL: [accessed 2021-06-23]
  38. Malaria Online: The digital surveillance system for Thailand malaria elimination. Division of Vector Borne Diseases, Department of Disease Control, Ministry of Public Health, Thailand. 2019.   URL: https:/​/publicadministration.​​unpsa/​Portals/​0/​UNPSA_Submitted_Docs/​2019/​3fe4c1ba-e00b-4250-8816-f513c3b209c6/​2020%20UNPSA_Malaria%20online_full%20report_27112019_111848_f62725d9-bd75-4846-a44c-8524838f4e87.​pdf?ver=1441-03-30-111848-927 [accessed 2021-06-23]
  39. Bauer MS, Kirchner J. Implementation science: What is it and why should I care? Psychiatry Res 2020 Jan 22;283:112376 [FREE Full text] [CrossRef] [Medline]
  40. Rosewell A, Makita L, Muscatello D, John LN, Bieb S, Hutton R, et al. Health information system strengthening and malaria elimination in Papua New Guinea. Malar J 2017 Jul 05;16(1):278 [FREE Full text] [CrossRef] [Medline]
  41. Krezanoski PJ. Delivering insecticide-treated nets for malaria prevention: innovative strategies. Res Rep Trop Med 2016 May 07;7:39-47 [FREE Full text] [CrossRef] [Medline]
  42. Aidenagbon G, Adeusi O, Akilah J, Umar A. Improving quality through process assessments during mass distribution campaigns of Long Lasting Insecticidal Nets (LLINs) in Nigeria: Studies from four states. IJID Online.   URL: [accessed 2021-06-23]
  43. Ashton RA, Bennett A, Al-Mafazy A, Abass AK, Msellem MI, McElroy P, et al. Use of routine health information system data to evaluate impact of malaria control interventions in Zanzibar, Tanzania from 2000 to 2015. EClinicalMedicine 2019 Jun 21;12:11-19 [FREE Full text] [CrossRef] [Medline]
  44. Ashton RA, Bennett A, Yukich J, Bhattarai A, Keating J, Eisele TP. Methodological considerations for use of routine health information system data to evaluate malaria program impact in an era of declining malaria transmission. Am J Trop Med Hyg 2017 Sep 01;97(3_Suppl):46-57 [FREE Full text] [CrossRef] [Medline]
  45. Kim S, Piccinini D, Mensah E, Lynch M. Using a human-centered design approach to determine consumer preferences for long-lasting insecticidal nets in Ghana. Glob Health Sci Pract 2019 Jun 24;7(2):160-170 [FREE Full text] [CrossRef] [Medline]
  46. May CR, Cummings A, Girling M, Bracher M, Mair FS, May CM, et al. Using Normalization Process Theory in feasibility studies and process evaluations of complex healthcare interventions: a systematic review. Implement Sci 2018 Dec 07;13(1):80 [FREE Full text] [CrossRef] [Medline]
  47. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci 2013 Oct 02;8:117 [FREE Full text] [CrossRef] [Medline]

CRS: Catholic Relief Services
DP: distribution point
GIS: geographic information system
ICT: information and communication technology
LGA: local government area
LLIN: long-lasting insecticide-treated net
NMEP: National Malaria Elimination Program
QR: quick response

Edited by G Eysenbach; submitted 19.08.20; peer-reviewed by L Jordan, T Turk; comments to author 11.03.21; revised version received 29.06.21; accepted 27.07.21; published 08.10.21


©Youngji Jo, Nathan Barthel, Elizabeth Stierman, Kathryn Clifton, Esther Semee Pak, Sonachi Ezeiru, Diwe Ekweremadu, Nnaemeka Onugu, Zainab Ali, Elijah Egwu, Ochayi Akoh, Orkan Uzunyayla, Suzanne Van Hulle. Originally published in JMIR Formative Research (, 08.10.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.