The HIV epidemic in the Philippines is one of the fastest growing epidemics globally, with 207% increase in new HIV infections and 388% increase in AIDS deaths from 2010 to 2020. In 2020, an estimated 73% of people living with HIV in the Philippines knew their status and 44% of people living with HIV were on antiretroviral therapy (ART) [1-4]. In 2 studies of cohorts of patients with HIV in Manila, 84% to 90% of patients who started ART had achieved viral suppression [4,5]. Most new and existing HIV infections occur among men who have sex with men (MSM) [3]. Improving treatment coverage, retention, adherence, and viral suppression are key to slowing the spread of HIV in the Philippines. Unfortunately, widespread stigma, lack of knowledge, and barriers to accessing care pose a challenge to engaging patients in testing and ensuring high levels of adherence to ART and retention in care [6-8]. High rates of first-line treatment failure, loss to follow-up, and suboptimal treatment adherence lead to poor outcomes in many patients with HIV in the Philippines [9,10].

This paper describes the process evaluation of a mobile phone technology for health (mobile health [mHealth]) intervention for people living with HIV in Metro Manila, Philippines. To support ART adherence, the intervention, Connect for Life, provided patients with HIV with individualized voice calls and SMS text messages, pill reminders, appointment reminders, symptom reporting, health tips, and adherence feedback.

The Connect for Life platform was developed by Janssen Global Public Health, and before adaptation for the Philippines, its versions were piloted in India and Uganda. The mMitra (mobile friend) project in India aimed to improve maternal health outcomes through health messages to pregnant women [11,12]. The Treatment Advice using Mobile Alerts project in India [13,14] and Call for Life Uganda [15,16] supported ART adherence among people living with HIV.

As mHealth technologies have become widespread in low-income and middle-income countries, mobile phone interventions have become increasingly popular in the global health and development sectors as an inexpensive and efficient way to communicate and deliver services. Several trials have shown that mHealth approaches show potential for improving self-management of chronic diseases, including adherence to HIV medications [17-21], whereas systematic reviews show mixed outcomes of mHealth interventions and highlight the need for more rigorous evaluation methods and longer follow-up periods in mHealth studies [22-31].

Trials assessing mHealth adherence interventions for HIV often do not include process evaluations to examine the fidelity and quality of the intervention delivery, causal mechanisms for the health outcomes, contextual factors affecting the delivery, and costs to implement [29,32,33]. For mHealth interventions, current guidance suggests that practitioners should also include a minimum set of information about the content, context, and technical features of the intervention, including aspects such as ease of use, content quality, privacy and security, service quality, personalization, and perceived enjoyment [34-37].

Process evaluations of SMS text messages and interactive voice response systems (IVRSs) have examined fidelity, reach, dose delivered, and user satisfaction for projects ranging from water and sanitation to prevent diarrheal disease [38]; airline pilot fatigue [39]; and prevention of weight gain, smoking, or HIV among young people [40-42]. A systematic review of mHealth projects in Africa found that in projects where acceptability and usability of mHealth technology among participants was measured, it was generally high. However, infrastructure issues (unreliable network and internet and electricity access) were frequently cited as key challenges in delivery [24].

The success of mHealth projects in achieving the intended health outcomes is almost entirely dependent on the adaptation and delivery of the intervention in local contexts. Having a complete understanding of the implementation process of an mHealth intervention can enable practitioners to interpret the outcomes and replicate the intervention in other contexts. Therefore, we performed a process evaluation alongside the Connect for Life Philippines prospective cohort study. The process evaluation examined the fidelity, dose delivered and received, reach, usability, acceptability, and cost of the Connect for Life Philippines intervention.

The study was conducted at the Sustained Health Initiatives of the Philippines (SHIP) Clinic, a low-cost, private facility in Metro Manila, a city with approximately 13 million people in the predominantly Catholic country of the Philippines.

SHIP Clinic provides HIV primary care and wraparound services to approximately 900 people living with HIV. Approximately 98% of SHIP’s clients are MSM, with an average age of 30 years at initial consultation. Most are full-time or part-time employees. The clients come from all regions of Metro Manila, and some live in other provinces.

Recruitment into the Connect for Life study occurred in person at the study site between October 2016 and December 2017. As patients checked in for their routine clinic visits, the study coordinator approached all patients seated in the clinic waiting room, briefly introduced the study following a recruitment script, elicited their interest in participating, screened them for eligibility, completed the informed consent process, and provided a brief orientation to the intervention.

The study team worked with IT specialists and public health professionals from Jannsen Global Public Health, University of the Philippines, and local IT companies to develop the content and functionality of the Connect for Life mHealth platform (Figure 1). Connect for Life is a technology built on the Mobile Technology for Community Health (MOTECH; Grameen Foundation) open-source software platform [43]. It enables health facilities to connect to patients via their mobile phones through IVRS call flows or SMS text messages. As Connect for Life works through phone calls and SMS text messages, it does not require the user to have a smartphone, install an app, or have mobile internet connection. This makes it accessible to a wide range of users in the Philippines, where, in 2015, mobile phone penetration was high, but smart phone coverage and internet access were low (with 113 mobile subscriptions per 100 people, 99% of the population reached by network coverage, and 22% of the population owning a smart phone) [44-46].

The study team tailored the Connect for Life platform for the Philippine context. Some existing features were retained, such as reminders sent on the recipient’s preferred days and times, health tips, and symptom screening. New features were developed, such as medical record functionality and adherence feedback scores. Clinicians at the study site developed new content for the voice and SMS text messages, which were recorded by a local voice talent agency. During the formative study and intervention development stage, a series of focus groups were conducted to engage with patients at the clinic about their adherence behaviors and preferences for configuration and content, and their feedback was incorporated to ensure that the intervention was tailored to the target population [47-49].

The Connect for Life system was installed in a secure cloud server environment and linked to a local telecom provider through application programming interface integration to execute calls and SMS text messages. A local IT service provider was contracted to monitor server functionality, install software updates, and troubleshoot technical issues. The Connect for Life software developers provided in-depth technical training and software documentation to the local IT provider and training for the clinical staff on how to use the Connect for Life web-based platform.

The intervention development process was guided by the Behavior Change Wheel and the Capability Opportunity Motivation–Behavior model developed by Michie, Atkins, and West [50-52]. Behavior change techniques related specifically to ART adherence were informed by the information-motivation-behavioral skills model of ART adherence [53]. Each service in the intervention package was designed to address ≥1 of the 3 main components that drive behavior in the Capability Opportunity Motivation–Behavior model, as outlined in Figure 2 [47,48].

A mixed methods approach was used with qualitative data embedded in the experimental design of the 48-week prospective cohort study [54]. The design allowed us to assess participants’ use of and experience with the system and use quantitative and qualitative analyses to generate complementary data about acceptability, usability, and the impact of contextual factors on the intervention.

The process evaluation measures were based on the framework proposed by Linnan and Steckler [55], which defines the approach to adequately describe the context, reach, dose (delivered and received), acceptability, and fidelity of the intervention. Additional aspects related to reporting on mHealth technology were included based on guidance from the mHealth Evidence Reporting and Assessment checklist [36].

The process evaluation questions, tools, methods, and data sources are described in Multimedia Appendix 1.

To measure the fidelity and dose of intervention delivery, records from the mHealth platform detailing the services received by each participant were exported. To understand the usability and acceptability of the intervention, participants completed self-administered paper-based questionnaires at 3 time points during the study. Where questionnaires had blank or missing fields, all available data points were included in the analysis. Data distributions were explored to categorize the responses to the questionnaires. Associations between acceptability of the intervention and independent variables (time point, treatment experience, and reminder frequency) were calculated using chi-square tests. Data analysis was conducted using Stata 15.

Qualitative feedback was collected in several ways: routine monthly process reports from clinicians to document implementation successes and challenges, comments recorded on the acceptability questionnaires, and a series of focus group discussions (FGDs). The study team conducted 2 FGDs with a total of 12 participants during the intervention development phase in 2016. In early 2017, a total of 2 additional FGDs were conducted with 5 participants after an 8-week pilot phase. Finally, in 2018, during the final 2 months of the study, 3 FGDs were conducted with 15 participants. The FGDs were transcribed, transcripts were manually coded using a deductive coding methodology to group responses by topic areas in the FGD guide, subtopics were assigned through line-by-line coding, and data were consolidated in a structured template that enabled identification of salient themes. Results from the FGDs in the formative and pilot phases informed the content and structure of the intervention and helped to identify implementation issues early in the project [47].

Ethics approval for the study was obtained from the University of the Philippines Manila research ethics board (protocol number 2016-265-01) and the London School of Hygiene and Tropical Medicine (reference number 11631). All participants provided written consent before inclusion in the study.

During the study, >31,000 IVRS calls and 8000 SMS text messages were sent to 462 study participants. The Connect for Life system was acceptable to both participants and providers. Participants liked that the intervention was highly personalizable, enabling them to select the frequency and time of calls or SMS text messages and the topics of health tips. Feedback on the pill reminders, visit reminders, and health tips was very positive. Participants appreciated that health tips covered a variety of topics beyond HIV basics. The FGDs revealed that acceptability of the weekly adherence scores and symptom reporting functionalities of the intervention was low, as these 2 functions required lengthy navigation of the IVRS menu.

Owing to technical issues, the intervention was not implemented as originally intended, with only 22.1% (102/462) of the participants receiving the IVRS pill reminder intervention and others receiving a scaled-back SMS text message intervention. When the technical issues were first identified, enrollment in the study was paused for 3 months, while the study team assessed the cause of the issue. Ultimately, the issue of DTMF malfunction was attributed to issues in the telecommunications system that neither the telecommunications provider nor the Connect for Life developers could resolve. When enrollment was resumed, participants were provided SMS text messages rather than IVRS services. Despite the technical challenges, acceptability remained high, and only 0.6% (3/462) of the participants withdrew from the study. Following the study, the frequency of technical issues has decreased significantly, and the study site has continued to provide the service. Currently, pill reminder calls are a routine service for all new patients undergoing ART.

Notably, the technical challenges experienced in delivering the intervention were related to navigating the IVRS menu and made it difficult to distinguish whether the issues raised with ease of use or overall satisfaction were related to the technical challenges (ie, the dial tones were not recognized when keyed in) or to the product design (ie, IVRS menus were very complicated). The accuracy of the adherence scores in the weekly feedback SMS text messages was dependent on successful navigation of the IVRS process. This type of feedback may have been better delivered via a smart phone app rather than an IVRS setup. The interactive component of the IVRS system was an important aspect of the study design, which was not effectively evaluated in this study owing to the low number of participants who received this part of the service, warranting ongoing monitoring and future studies.

The scaled-back intervention provided everyone with visit reminders, which addressed part of the theory of change by improving medication accessibility through timely refills, but did little to prompt pill-taking, habit forming, and improvements in health knowledge. Individuals who received a high dose of the intervention (daily or weekly pill reminders) were more likely to recommend the intervention to others, suggesting that the planned intervention was more acceptable than the scaled-back version.

Our analysis of dose received shows that the call answer rate was low, with only 26.24% (8119/30,940) of outgoing calls answered, which is reflective of a preference for SMS text messages and chat services among the target population. The requirement of a PIN reduced exposure to the intervention, which was mostly owing to technological challenges. After experiencing technical difficulties several times, many participants stopped answering the calls. However, some mentioned that the phone ringing at the set time each day served as an effective adherence reminder.

Privacy considerations were paramount, with 11% (51/462) of the participants reporting that they had concerns about the potential for disclosure of their HIV status. Therefore, in situations where entering a PIN is a barrier to intervention exposure, practitioners can consider adapting the content to eliminate potentially sensitive health information and delivering the service with no PIN requirement.

Ultimately, the study showed the importance of choosing technologies that can function in local contexts. In low-resource settings, it may take time to scale-up technologies that will be quick to roll out in high-resource settings. Practitioners must identify service providers with appropriate capacity and ensure that patients have the skills and motivation to use the intervention. Conducting an iterative process with several pilot stages is advantageous, as it enables practitioners to identify the problems with functionality and adapt the intervention before scaling up.

An important aspect of the intervention was that, through this regular contact from the clinic, participants felt cared for and felt that their health care provider was concerned about their well-being. This social support was a key motivator for adherence. Participants requested to be able to speak to someone about side effects or for social support, suggesting that an intervention that links calls to counselors more effectively may be an area for future evaluation.

The Connect for Life Philippines intervention was adapted from the same platform used for Call for Life Uganda and mMitra and Treatment Advice using Mobile Alerts in India. Acceptability was high in all 3 settings [11,16]. However, there were differences in the preferences and use patterns of the participants in the Philippines setting compared with those in Uganda and India. The Philippines had a high preference for SMS text messages over voice calls and a low call answer rate. The Connect for Life Philippines and Call for Life Uganda projects experienced similar challenges with network instability issues in the early stages [16].

Similar to Connect for Life and Call for Life, other mHealth interventions for people living with HIV have shown improvements in ART adherence, even where participant response rates (ie, dose received) are low [29,31,56]. For example, the PositiveLinks app used by people living with HIV in Virginia, United States, had response rates of <40% to most app prompts, but participant retention in care, CD4 results, and viral suppression improved significantly [57]. There is an important distinction between adherence to the intervention (ie, calls, app prompts, and device use) and adherence to medication.

A strength of the Connect for Life platform is its scalability; the project can easily be expanded to cover a large number of sites and patients with great cost efficiency, if those facilities have access to computers and internet connectivity. To deliver the project at scale, creation of content in regional languages will be an important consideration. The platform is adaptable, as the local IT provider can add and remove new data fields and update the SMS text message content, voice files, and call flows. However, changes to the functionality of the software or interoperability with other systems will require support from the software developers at Johnson and Johnson Global Public Health. The Philippines Department of Health has plans to implement electronic reporting systems for HIV services at an aggregated level. If the department is ever to implement a patient-level electronic medical record, interoperability with Connect for Life will be an important consideration to ensure delivery at scale.

A strength of this process evaluation study is the mixed methods and participatory approach. The study used prospectively collected quantitative data on participants’ responses to the intervention and qualitative feedback from questionnaires, monthly process reports, and FGDs. The evaluation included the users of the intervention, clinical service providers, and developers of the technology platform.

The methodology addressed all key components in process evaluation for public health interventions and studies (context, reach, dose delivered, dose received, fidelity, implementation, and recruitment) [55]. Furthermore, the study included information on the technology platform, infrastructure, security, and cost, as guided by the mHealth Evidence Reporting and Assessment checklist developed by the World Health Organization mHealth Technical Evidence Review Group [35].

A limitation of our approach was that the evaluation was conducted by the same study team responsible for planning and implementing the intervention, rather than by independent evaluators. Other limitations included the convenience sampling strategy for participants in the FGDs and the low participation in the focus groups. Although the study team approached many individuals to participate, it was a challenge to identify those who were willing owing to reluctance to disclose their HIV status in a group. Furthermore, owing to transportation challenges, there was low attendance among those who confirmed their intention to participate in the groups.

Incomplete data may have affected the interpretation of the results. Of the 462 participants in the study, 440 (95.2%) attended the final study visit at week 48, and 89.1% (392/440) of them completed the questionnaire during the final visit. There may be differences in the experiences of participants who transferred out, died, withdrew from the study or were lost to follow-up, attended but did not complete the questionnaire, and completed the questionnaire.

This study focused on MSM in Metro Manila, and the study population was urban and highly educated. Participants may have had alternative adherence reminders, including self-set phone alarms and email alerts. Therefore, the results are not broadly generalizable to other contexts.

mHealth interventions are useful to support adherence, as they have low replication costs and are highly adaptable to specific cultural contexts. On the basis of the findings of this process evaluation, we can guide practitioners implementing mHealth interventions to support medication adherence to consider the following recommendations:

The Connect for Life mHealth intervention to support adherence to ART had high participant satisfaction and acceptability. However, the feasibility of the intervention was dependent on the reliability of local telecommunications networks, and poor reliability of the local mobile networks had a large impact on the intervention’s usability, fidelity, and dose received.

The process evaluation allowed us to better understand the preferences and use patterns of mHealth services by MSM in the Philippines. This will enable the effective scale-up of mHealth services for this key population, which is essential in the context of the dual HIV and COVID-19 pandemics, where more services must be delivered virtually.