In recent years, there has been a notable increase in the incidence of chronic disease, especially among the younger population [1]. These chronic diseases include obesity and type 2 diabetes [1]. In terms of diabetes prevalence, Singapore, with 600,000 adults living with diabetes, ranks second among high-income countries, and obesity levels are also on the rise [2]. In addition, 34% of the men and 39% of the women in Singapore do not reach the weekly target of 150 minutes of moderate-intensity activity per week, leaving people at higher risk of developing diseases such as diabetes [3]. People with diabetes live shorter lives—by at least 10 years—and have a lower quality of life (QoL) than those without diabetes [4,5]. Prediabetes, a precursor to diabetes, affected 15.5% of the Singaporean adults in 2010 (ie, 1 in 7), and this figure is estimated to increase to 24.9% by 2035 [6]. Prediabetes increases the risk of heart disease, and if untreated, over time, most people with prediabetes transition to diabetes [7]. Research has shown that lifestyle interventions (such as increasing physical activity and eating a healthy diet) delivered by trained health care professionals can help to promote healthy weight loss [8-10]. In addition, lifestyle change through high-risk and population-based approaches were also directly associated with a reduction in the incidence of type 2 diabetes, accentuating the efficacy of healthy lifestyle behavior change for diabetes prevention [8].

Achieving healthy lifestyle behavior changes independently can be challenging, and support from experts such as dietitians or exercise physiologists has been shown to be more effective [8]. However, access to experts at a population level may not be feasible or affordable. A potentially more accessible alternative to in-person support and supervision could be novel digital health interventions such as conversational agents. Conversational agents, or chatbots, are computer programs designed to mimic human-to-human conversations in the form of either text messaging or verbal discourse [11]. The heightened accessibility, personalization, and efficiency that conversational agents offer highlight the potential for conversational agents to improve patient care [11-13]. Conversational agents enable 2-way communication, and their text- or speech-based method of communication makes them suitable for a variety of target populations, ranging from young children to older people. The application of conversational agents in health care is gaining traction in a number of medical fields, including health care service provision, chronic disease management, and patient education [14]. They can be delivered through a variety of means: messaging apps, individual apps, or even standalone devices [14].

Singapore is a technologically savvy country, and citizens avidly use messaging apps. In addition, Singapore’s ministry of health has proposed the increasing use of conversational agents in health care in tackling issues such as the rising chronic disease burden and the aging population [15] that can lead to more primary care appointments. The ministry envisions a near future where a conversational agent can collect a patient’s history from them before their consultation, streamlining the primary care visit and thus cutting down waiting times. This makes Singapore an ideal place for the evaluation of novel mobile health interventions such as conversational agents. Moreover, health programs delivered over the internet have shown success, as exemplified by web-based interventions for smoking [16], alcohol intake [17], sexual health [18], cancer screening [19], physical activity [20], and diet modifications [21]. The ubiquity of the internet makes these programs easily accessible to a diverse group.

The evidence for the use of conversational agents for healthy lifestyle behavior change from trials is limited. The feasibility and acceptability of implementing and evaluating the use of novel interventions are essential for informing potential future trials. Correspondingly, we aim to assess the feasibility, acceptability, and preliminary efficacy of the use of conversational agents for healthy lifestyle behavior changes in the general population in Singapore.

This web-based single-arm feasibility study was approved by the Nanyang Technological University Ethics Committee (IRB-2018-11-032). All participants signed their digital informed consent before embarking on the study.

Participants were eligible if they were aged above 21 years, were Singapore citizens or permanent residents, owned a smartphone, and had a Facebook Messenger (Facebook, Inc) account. Prospective participants were excluded if they were pregnant or had any of the following conditions: cancer, chronic liver disease, chronic kidney disease, a neurodegenerative condition, heart disease, stroke, a physical disability, hypertension, or a condition that does not allow for regular physical activity. Eligibility was confirmed by having participants complete an eligibility questionnaire, after which they were asked to provide informed consent on a digital form sent to them through email.

The participants were recruited on the web through Facebook in August-September 2019. A digital poster listing the study aims and eligibility criteria was uploaded on relevant Facebook pages focused on healthy living, such as Singapore fitness and health community and Singapore healthy cooking. In addition, we used snowballing in our recruitment; therefore, participants were also procured through redistribution of our study poster through messaging apps or through word of mouth.

The conversational agent was designed to be used on Facebook Messenger using a free web-based tool, Chatfuel [22]. A research associate (AS) and a PhD student (DAD) developed the script for the program and performed the input. The intervention focused on diabetes and prediabetes knowledge, diet, physical activity, sleep, and stress management. These were the topics of focus identified in other diabetes prevention programs targeting lifestyle change [23-25].

The content was informed by existing evidence-based sources of information, including clinical guidelines and systematic reviews. Advice on improving sleep quality was generated from published evidence reporting on techniques and successful interventions for sleep disorders [26]. The domain on stress was informed by distance learning–based stress management techniques identified from a review of existing studies that described methods to reduce stress and improve health [27]. Pertinent nutritional advice for individuals with prediabetes was obtained from authenticated government health portals and other validated health and nutrition webpages [28]. The collated advice was then compartmentalized into themes, which translated into the topic of focus for each interaction between conversational agent and user (Multimedia Appendix 1 [8,26,27,29-53]). The content for the section on physical activity was informed by advice on the recommended duration and intensity of exercise from Singapore’s Health Promotion Board [28]. Validated fitness routines were then presented as part of the conversation as methods to achieve the necessary level of fitness [29]. Examples of conversational exchanges between the conversational agent and users are presented in Figure 1.

We also followed the Capability, Opportunity, Motivation, Behavior model of behavior to guide the development of the intervention [54]. This model posits that to exhibit a particular behavior (B), the participant must be physically and psychologically capable (C), have sufficient social and physical opportunities (O) to perform the behavior, and must have the desire or need to do so—motivation (M). This was important in determining the inclusion criteria for the intervention (eg, physically fit with no comorbidities) and in conversation designing.

The content was mostly presented in the form of text, supplemented with some images to make the conversational agent more engaging and to enhance the user experience. These images were obtained from free-to-use sources or photographs taken by our study team members. To further contribute to a positive user experience that mimicked human interaction, the conversational agent was given a name, Precilla, and it displayed human-like characteristics such as in the tone of speech, profile picture, and through using the typing function for messages. From here onward, we use Precilla to refer to the conversational agent used in our pilot study.

The intervention was scheduled to last for 4 weeks, whereby participants would receive 4 messages per week, (every other day), 1 for each topic (diet, exercise, sleep, and stress). A sample of the dialog tree is presented in Multimedia Appendix 2.

An overview of the study workflow is demonstrated in Figure 2. Interested individuals were first required to complete an eligibility questionnaire. The eligible participants then provided informed consent, after which they were requested to complete a baseline questionnaire on a web-based tool, REDCap (Research Electronic Data Capture; Vanderbilt University) [55]. The questionnaire obtained general information regarding demographics as well as data on diet, physical activity, sleep, and stress.

The actual pilot study lasted for 4 weeks. Upon completion of the piloting period, the participants were required to complete a follow-up questionnaire that contained all the details outlined in the baseline questionnaire, with some additional questions on conversational agent usability and their overall satisfaction with the study. Interested participants also had an opportunity to take part in a follow-up interview to share their views on the conversational agent, their experience while taking part in this web-based study, as well as their thoughts on points of improvement.

Data analysis was conducted on the data collected at baseline and follow-up (Table 1). The participant responses on satisfaction and usability were presented as percentages of each option on a Likert scale (eg, strongly agree, disagree, much better than before, or neither better nor worse). The participant outcomes for QoL, knowledge, physical activity, sleep, and stress were all presented as pre- and postscores using the individualized scoring system of each questionnaire.

We performed descriptive analyses of the data. The data were presented using percentages, means, and SDs. As this was a feasibility study, no hypothesis testing was performed to assess the efficacy of the intervention [62]. Mean differences between baseline and follow-up were presented, accompanied by 95% CIs [62].

A total of 136 individuals expressed initial interest in participating in the study; however, 9 (6.6%) were ineligible because of comorbidities or because they had not installed Facebook Messenger on their smartphone, and a further 52 (38.2%) completed the screening questionnaire but did not proceed to provide informed consent, leaving 75 (55.1%) participants eligible for participation. Of these 75 participants, 60 (80%) completed baseline assessments, and of these 60, 56 (93%) completed the follow-up questionnaires (Figure 3).

Of the enrolled participants, 38% (23/60) were men. The mean age was 33.7 years (SD 9.3), the mean BMI was 22.3 kg/m² (SD 3.8; Table 2), and the average technical competency on a scale of 1 to 10 was 8.07.

Of the 56 participants who followed the study through till completion, 52 (92%) were moderately or very satisfied with Precilla, 30 (54%) thought that their likelihood of recommending Precilla to others would be somewhat likely, 32 (57%) felt that the likelihood of their opting to use Precilla again for personal use would also be somewhat likely, and, finally, 29 (51%) posited that their health was somewhat better or much better than before.

The agree and strongly agree responses exceeded 50% (28/56) for all the questions relating to Precilla’s acceptability, except for question 5, “Chatting with Precilla motivated me to change unhealthy sleeping habits,” where the collective response in agreement (agree and strongly agree) was only 46% (26/56) (Figure 4). The smallest percentage of disagreement was for questions 1 and 8, indicating that most of the participants (55/56, 98%) found the chat easy to use and thought that Precilla used simple language and was easy to talk to. The neutral response was a common selection by participants for questions 2-7.

Of the 56 participants, 37 (67%) agreed that Facebook Messenger was an appropriate medium to deliver the messages sent by Precilla. The remaining participants were either neutral or disagreed. Telegram and WhatsApp were proposed as potential alternative delivery media. Of the 56 participants, 9 (15%) thought that the number of messages exchanged with Precilla was too high, and they would have preferred the intervention to deliver fewer messages; 12 (21%) disagreed that 12 PM was the most appropriate time to receive messages, whereas the remaining 44 (79%) either agreed, strongly agreed, or were neutral (Figure 4) and indicated a preference for interacting with Precilla after work, in the evening or before bedtime. Some indicated specific timings such as 5 PM, 8 PM, or 10 PM, and 1 participant suggested 9 AM or during the morning commute to work. There were no disagreements to statement 13 (“I liked images and videos in Precilla’s messages”) and minimal disagreement for statements 14 (5/56, 9%) and 15 (4/56, 7%) on the inclusion of more visual components and Precilla’s personality, respectively. Most participants did not find Precilla to be very human-like and could clearly tell that they were communicating with a conversational agent. Of the 56 participants, 3 (5%) suggested some degree of personalization, whereby Precilla’s messages should be initiated when the user prompted the conversational agent and not the other way around.

The number of complete, incomplete, and absent interactions were noted (Figure 5). In addition, the number of immediate and delayed interactions were also counted (Figure 6). Some interactions were started but not completed. The reasons for not completing interactions are presented in Multimedia Appendix 4.

All the conversations were completed by at least 50% (>30/56) of the participants consistently over the period of the study. The number of participants who completed interactions from start to finish exceeded 71% (40/56) for 56% (9/16) of the interactions, denoting high engagement with the conversational agent. Similarly, at least 40% (<22/56) of the participants responded immediately to the conversational agent 75% (12/16 interactions) of the time, even in the last week of the study, denoting consistently high conversational agent engagement.

The participants were restricted to a predetermined list of options when providing responses to the questions asked or information provided. A frequent cause of disruption to the conversation flow was participants typing free text, which led them off track. This resulted in their being redirected to the main menu, which they potentially found confusing, or they were reluctant to go through the whole conversation flow again and, hence, failed to complete that interaction. The average duration of each interaction was in the 2- to 5-minute range. More data on conversational agent engagement are presented in Multimedia Appendix 4.

The 14-item QoL score was 54 (SD 6.90) at baseline and 53 (SD 6.46) at follow-up (mean difference 0.95; 95% CI −1.51 to 3.41). Of the participants taking medication (10 at baseline, 17 at follow up), participants rated adherence as fair (baseline 5/10, 50%; follow-up 6/17, 35%), good (baseline 5/10, 50%; follow-up 9/17, 53%), and very good (baseline 0/10, 0%; follow-up 2/17, 12%). The participants rated overall life satisfaction as fair (baseline 16/60, 27%; follow-up 12/56, 21%), good (baseline 35/60, 58%; follow-up 33/56, 59%), or very good (baseline 9/60, 15%; follow-up 10/56, 18%), and 2% (1/56) of participants reported poor overall life satisfaction at follow-up.

The scoring system for the questionnaires meant that lower scores denoted better knowledge outcomes (sections 1 and 3) and lower perceived risk of diabetes (section 2). In section 1, the scores were 7 (SD 1.37) at baseline and 6 (SD 0.93) at follow-up (mean difference 0.52, 95% CI 0.09-0.95); the possible score range was 6-18. In section 2, the scores were 28 (SD 2.7) at baseline and 27 (SD 3.5) at follow-up (mean difference 0.9, 95% CI −0.25 to 2.05); the possible score range was 12-48. In section 3, the scores were 10 (SD 4.11) at baseline and 9 (SD 2.2) at follow-up (mean difference 1.0, 95% CI −0.23 to 2.23); the possible score range was 8-32.

The number of individuals consuming vegetables at least once a day was 27% (16/60) at baseline and 29% (16/56) at follow-up. Similarly, the percentage of participants having three portions of fruit per day was 3% (2/60) at baseline and 7% (4/56) at follow-up. With regard to intake of unhealthy food comprising sweetened beverages and fried food or snacks, the almost never consumption category for sweetened beverages was 38% (23/60) at baseline and 45% (25/56) at follow-up, whereas for fried food and snacks, the almost never consumption category was 25% (15/60) at baseline and 30% (17/56) at follow-up.

The median METs-per-week score was 857 at baseline (IQR 902) and 765 (IQR 882) at follow-up. The average categorical score remained constant from baseline to follow-up where the average exercise intensity was moderate. The mean time spent sitting was 439 minutes at baseline and 406 minutes at follow-up. In addition, the median amount of moderate and vigorous physical activity was the same, 30 min per week (IQR 90) at baseline and 50 min per week (IQR 90) at follow-up.

According to the scoring method, a lower score indicated better sleep. The mean PSQI score was 4.38 (SD 2.36) at baseline and 4.43 (SD 2.45) at follow-up (mean difference −0.05, 95% CI −0.93 to 0.83). The prevalence of poor sleep quality was 28% (16/60) at baseline and 30% (16/56) at follow-up. The participants who reported sleeping for less than 7 hours were classified as short sleepers; 82% (49/60) were short sleepers at baseline, whereas 88% (49/56) were short sleepers at follow-up.

The scoring for stress meant that lower scores denoted lower stress levels and higher scores denoted higher stress levels. The mean score at baseline was 17 (SD 5.13), denoting a moderate stress level, which was maintained even at follow-up, mean 16 (SD 5.1; mean difference 0.73; 95% CI −1.15 to 2.61).

To our knowledge, this is the first web-based pilot study to test the feasibility and acceptability of a healthy lifestyle behavior change conversational agent in the Singaporean population. In this web-based study, which used Facebook Messenger as a delivery medium for a conversational agent targeting healthy lifestyle behavior change in the Singaporean population, we managed to recruit 37.5% (75/200) of the target sample size in 1 month. Retention was high at 93% (56/60); conversational agent engagement was also high, with all the conversations being completed by at least 50% (28/56) of the participants consistently. Furthermore, at least 40% (22/56) of the participants responded almost immediately 75% (12/16) of the time. Acceptability, usability, and satisfaction were also generally high. In general, we were able to conduct the study with high fidelity, and each phase ensued as planned. Any definitive signs of improvement in QoL, knowledge, diet, physical activity, sleep, and stress would have to be studied in more detail in a future study of effectiveness.

The secondary outcomes in the study were measured using validated scales, namely, the short form version of the QoL Enjoyment and Satisfaction Questionnaire, PSQI, Perceived Stress Scale, Food Frequency Questionnaire, and IPAQ. We took note that although these questionnaires have not necessarily been optimized for the Singaporean population, these well-established scales were chosen to ensure a certain level of validity and reliability in our results.

Conversational agents have been used for healthy behavior change; however, they have been used mainly in niche areas such as smoking cessation, alcohol misuse treatment, and the promotion of physical activity in sedentary populations [63-65]. The application presented in this study is very comprehensive, novel, and relevant to the general population in Singapore.

The outcome measures were similar to those in other studies looking at the acceptability of conversational agents in health care, and acceptability was high in other studies as well as in this feasibility trial. Previous studies noted high response rates (ie, conversational agent engagement) and strongly agree or agree scores for user-friendliness, appropriateness, consistency, and speed of response, as in our study [66]. The other measures in prior studies that denoted acceptability were high perceived ease of use, usefulness, and intention to use, similar to our measures of ease and enjoyment of use, motivating change in unhealthy habits, and intention to use Precilla again or recommend it to others, respectively [67,68].

In another study using a health behavior change conversational agent, high compliance was attributed to a rewarding game system [68]. Considering the slight decrease in weekly compliance in our feasibility study (43/56, 77% completed interactions in week 1 vs 39/56, 69% in week 4), it may be worth exploring the possibility of introducing a gaming or reward component in future iterations. Personalization has previously been met with positive user reception, for example, by providing personalized advice to maintain target blood glucose levels or personalized reminders for taking medication [69]. Similarly, personalized content and message timing delivery should be implemented in future versions of Precilla, considering the preference indicated by users.

It has previously been indicated that users tend to prefer interacting with female conversational agents, as evidenced by Siri, Cortana, and Alexa, for example [70]. In addition, the study by Brahnam et al [71] has explained that in the field of human-computer interaction, “the standard of believability has become inextricably linked to gender personification, especially female personification.” As such, we chose to use Precilla and reinforced her character with a profile picture.

Views on conversational agent personality have been mixed, depending on the function of the agent. For example, participants appreciated an empathetic demeanor from conversational agents for e-therapy, in contrast to the participants’ indifference to Precilla’s humanity in this pilot study, where she played more of an impersonal, informative or educational role [72]. It may be that the conversational agent’s purpose determines how important the degree of humanity is.

Another aspect of this study was the effectiveness of participant recruitment through social media. We chose a goal of 75% (150/200) recruitment rate based on other studies involving mobile health trials having achieved an application rate of 86% (70/81) for enrollment through Facebook or an 85.1% (605/711) response rate to Facebook advertisements in 45 days [73,74]. Although our study only yielded a recruitment rate of 37.5% (75/200) in the given period, it aligns quite appropriately with the recruitment rates of 33%, 30%, or 37.7% in other web-based mobile apps or Facebook trials [75-77]. Hence, our 37.5% (75/200) recruitment rate seems to indicate feasibility regarding the recruitment of participants in the general population in Singapore for a web-based conversational agent intervention delivered through Facebook.

A systematic review accentuated the effectiveness of Facebook as a recruitment tool [78]. It was suggested that on average, studies allowed a 3-month recruitment period, and Facebook was found to be more efficient than traditional methods (print, radio, television, email, or word of mouth) because of the reduced costs, shorter recruitment times, and ability to connect with harder-to-reach populations. In future studies, we may need to consider complementing our current no-fee Facebook approach with advertisements or other recruitment methods, as well as potentially a longer recruitment period.

In terms of the recruitment, we noticed some attrition among the eligible participants just before provision of informed consent. As the only communication was through email from an institutional email address, the uncertainty involved in remote participation, such as never meeting the study team in person, could have been a cause for the participants’ apprehension. This lack of direct engagement may have led to some participants not feeling comfortable sharing their digital signature with us because of data protection and privacy concerns. In future research studies, a mix of a digital recruitment approach and face-to-face exchange (possibly in the form of recruitment or debriefing) could provide participants with an opportunity to ask questions and validate the legitimacy of the study.

Regarding the preliminary efficacy, minimal improvements were observed in the participants’ knowledge, stress, and diet, whereas there was a lack of improvement in QoL and sleep and no change in physical activity. The dearth of significant improvement in these areas could be attributed to the content and delivery not being adequately designed to target effectiveness and, possibly, the short study length (4 weeks), which may not have been sufficient for any noteworthy changes to have been observed. Furthermore, this study was not designed to test the effectiveness of the intervention, and this should be explored in more detail in future iterations.

The participants indicated that they could clearly tell that Precilla was indeed a conversational agent. This was beneficial in confirming Precilla’s identity and in reducing expectations from users that the conversational agent should think, function, and react like a human being. It was interesting to note that although the participants did not find Precilla to be very human-like, they still rated the conversational agent highly in terms of content, usability, general acceptability, ease of use, visual components, and so on. This shows that, for this study population and for a conversational agent with Precilla’s functions and capabilities, acceptability was not dependent on the anthropomorphism of the conversational agent.

The conversational agent engagement data revealed that there was a slight decrease in the number of interactions that the participants completed fully with each successive week. It is possible that the content and delivery methods could have come across as repetitive because we provided a review of take-home points from the previous interaction from week 2 onwards. Alternatively, perhaps the routine that Precilla followed made the intervention very regimented, not leaving much room for spontaneity in the form of varied message timings or even variations in the ways that users could respond to Precilla, for example, using free text. These factors may have contributed to the decrease in interest over time. Future iterations of Precilla could explore introducing more novelty and personalization, as recommended by the participants in the telephone interviews. The participants shared some points for improvement, including alternative modes of delivery such as WhatsApp and Telegram, a possibility of shortening the conversation lengths, and more personalized timings for message delivery.

One of the main strengths of the study is the high fidelity in the delivery of this low-cost, fully web-based feasibility study. We did not experience any software malfunctions and were able to implement the intervention as per protocol. The content of the intervention was evidence based, and it was co-designed with members of the target population to make the intervention as relevant to them as possible. We managed to reach a diverse range of age groups—from 23 to 60 years—using Facebook without the need to pay for advertising. Although the recruited sample—recruited over 4 weeks using solely Facebook—consisted of fewer participants than our target size, it is comparable with those reported in other studies that used paid Facebook advertisements within the same time frame. However, the participants were recruited through healthy living–focused Facebook groups and pages; therefore, these individuals were already very much invested in healthy lifestyle behavior change and more likely to use apps and social media. Furthermore, the participants were Facebook users, capable of navigating Facebook Messenger, had high technological competency, and a very high level of tertiary education. As such, our findings may not be generalizable to other demographic groups such as older people; individuals with lower education levels; or those not active on, or familiar with, social media. However, this study did largely cover the general characteristics of the average Singaporean, making the findings very much valuable and relevant. A further limitation is that although we used validated outcome measurement tools, all outcomes were self-reported by the participants. Furthermore, although these measurement scales are well reputed and established, they were not necessarily optimized for use in the Singaporean population. Finally, we used a single-arm pre-post study design, which is acceptable for a feasibility study but does not allow for assessment of the effectiveness of the intervention.

This web-based feasibility study showed that the delivery of a conversational agent for healthy lifestyle behavior change using Facebook Messenger is, to a large extent, feasible in Singapore. Precilla is a low-cost intervention that was popular among the participants and was well received, with most participants being satisfied with the intervention and prepared to recommend it to friends and family. This study demonstrated the ability to conduct a web-based trial to assess the impact of a novel intervention. Our preliminary data on the acceptability of the intervention showed the need for further enhancement of this conversational agent intervention, potentially through humanization of the agent and personalization of the messaging. Such an intervention needs to be evaluated with a rigorous study design and larger sample size.