Embodied conversational agents (ECAs) have been proposed as a promising interaction modality for the delivery of programs focused on promoting lifestyle changes [1], such as physical activity [2-4] and nutrition [5,6], and preconception care [7,8]. ECAs often have a human-like appearance and communicate via prewritten dialogue. They also have the ability to establish and maintain an empathic relationship by using empathic behavior, both verbal, via text or speech, and nonverbal, via facial and gaze expressions and hand and body gestures [9]. These behaviors may make them more engaging than traditional eHealth interventions [10,11]. Results are promising, as ECA interventions have been found to be easier to use [5] and used more frequently [5,11-13] than interventions without an ECA. Nonetheless, ECA use does decline over time, limiting long-term health effects [1,14-17]. Moreover, it is unknown what factors influence use of an ECA. When designing an ECA, designers are advised to select the right role for the ECA, combine the most important personality characteristics, and use informational, nonjudgmental language [18]. In addition, a scoping review identified which use-related factors were assessed when evaluating the effect of ECAs on promoting healthy lifestyles. These factors included usability and user satisfaction, further specified as factors including liking and trusting the ECA and the desire to continue using the ECA. However, evidence for the effect of these factors on ECA use is limited. Furthermore, there is scarce and inconclusive evidence for the health effects of ECAs and the pathways to these effects [1].

In order to assess the pathways to effects and understand ECA use when evaluating an ECA, conceptual models can be used (as shown in Figure 1; further details were reported in the research protocol for this study [16]). The conceptual model explaining ECA use is based on existing human-computer interaction literature, including the technology acceptance model (TAM) [19]. The key variables in TAM are perceived usefulness and perceived ease of use. Systematic reviews have shown that these 2 variables typically explain about 40 percent of an individual’s intention to use a technology in a variety of contexts [20-22]. However, there is mixed evidence regarding whether intention predicts actual use [23,24]. Since actual use, rather than intention to use, is deemed necessary to achieve any health benefits, use is at the center of the conceptual model. Increased use is expected to improve the intensity of the relationship with the ECA, because of the capacity of ECAs to establish and maintain an empathic relationship. Usability and perceived usefulness are hypothesized to act as antecedents for use, whereas increased usability is expected to result in increased perceived usefulness.

The conceptual model explaining health effects occurring after the use of an ECA starts with behavioral change techniques (see Multimedia Appendix 1), which are expected to lead to an improvement in the 3 basic psychological needs: autonomy, competence, and relatedness. [25]. Ultimately, improved health behaviors will lead to a better quality of life. This model is primarily based on self-determination theory [25] and has an explorative character. By contrast, the classification system of Teixeira et al [26,27] is used to form hypotheses to explain which techniques improve which needs. Hence, the objectives of this study were to (1) identify whether ECAs could persuade community-dwelling older adults to change their dietary behavior and decrease their loneliness, (2) assess the pathways to these effects, and (3) understand ECA use.

This study used a randomized controlled trial design. Participants in the first cohort received access to the 8-week intervention immediately, while participants in the second cohort served as a control group, receiving access to the intervention after being placed on a 4-week waiting list.

We aimed to include a total of 60 participants with a 1:1 ratio of participants per cohort. Participants were deemed eligible if they were aged 65 years or older, not in paid employment, and lived alone and independently at home. In addition, participants needed to speak Dutch, be able to use a tablet or computer by themselves, and have a wireless internet connection. The project members recruited participants via research panels, flyers, newspapers, and social media. After providing informed consent, the participants were invited to complete the intake questionnaire. They were asked to report their demographics (gender, age, educational level, health conditions, risk of malnutrition [28], and eHealth literacy [29]), their possession of a device to use for the study, and their motivation to participate. All participants were asked to complete the baseline questionnaire (T0) after creating an online account and complete another questionnaire after 4 (T1) and 8 (T2) weeks of use. Participants in cohort 2 were asked to complete an additional waitlist questionnaire (Tw) 4 weeks before T0. In the last questionnaire, participants were asked whether they were open to an interview by phone.

The intervention, PACO, is a web-based eHealth service in which 2 ECAs engage in dialogue with an older adult to provide motivation for improving eating behavior and decreasing loneliness. The service consists of 5 modules, each one applying a different behavioral change technique (Figure 2, Multimedia Appendix 1). The user can engage in dialogue with Herman (the cook, who provides nutritional advice) and Ellen (the peer, who provides social advice). The ECAs are represented as 2D humans in cartoon style, are not animated, and use text as the means of communication. During the onboarding process, the ECAs introduce themselves and explain the PACO program.

The primary outcomes include use of the service, eating behavior, and loneliness. Use was assessed via log data collected on the PACO back end. Eating behavior was self-assessed by 3 open questions about the previous day’s fruit, vegetable, and liquid intake and loneliness was assessed via a validated questionnaire (Table 1 shows further details). The experience and the willingness to pay for PACO were measured via a self-compiled scale. All other outcomes were measured via validated online questionnaires. In an interview of approximately 30 minutes, participants were asked further questions about their experiences with PACO and any behavioral changes.

We created a single score for each scale and checked the test assumptions. Due to the violation of the linearity assumption, we deviated from the original protocol by using nonparametric tests. Relationships between demographics and the main study outcomes were calculated using Spearman ρ and, for gender, Mann-Whitney U. Differences between Tw and T0, and in health-related outcomes over time, were compared using the Friedman test. Differences in the strength of the relationship with the ECA over time were compared with a repeated-measures ANOVA. Spearman ρ was used to calculate the correlations between use- and health-related outcomes. Linear regression analysis was used to calculate the multivariate relationships between use, eating behavior, loneliness, and significant outcomes. The statistical significance level was P<.05. Recordings of the interviews were transcribed and thematically analyzed by LLK and BCM.

This study was preregistered at ClinicalTrials.gov (NCT04510883) and approved by the medical ethics committee of Wageningen University (number NL73121.081.20). We refer to the study protocol article for all details on the protocol, the development process of the intervention, and the conceptual models [16].

In total, 51 participants met the inclusion criteria. Nineteen participants did not use the PACO service for 14 consecutive days and were treated as dropouts. Among participants who dropped out, 7 did not respond to emails or telephone calls, 3 dropped out due to illness, 3 due to lack of time, 2 due to lack of motivation, 2 due to difficulties with the service, and 1 due to internet issues. Eight participants who dropped out had created an account, of whom 4 had completed T0. The mean age of the 32 participants was 73.00 years (SD 5.33, range 65-85); 18 (56%) were women. In total, 12 (38%) had completed high school or an associate degree and 19 (59%) had completed college or university. The mean eHealth literacy score was 29.25 (SD 4.36, range 15-34), and the risk of malnutrition was 9.69 (SD 1.35, range 7-11). None of the demographic characteristics were significantly associated with use, eating behavior, or loneliness, and there were no significant differences in health-related outcomes between Tw and T0. During intake, participants stated that they were mainly motivated to participate because they were interested in research and in new developments and thought it was important to contribute. Some participated because they were interested in nutrition and wanted to stay healthy or improve their habits.

The ECAs were not able to persuade users to change their fruit, vegetable, or liquid intake (χ²₂=0.34, P=.85) or decrease loneliness (χ²₂=0.02, P=.99). There were also no significant differences over time in quality of life (χ²₂=2.99, P=.22), autonomy (χ²₂=0.34, P=.85), competence (χ²₂=2.32, P=.31), or relatedness (χ²₂=2.46, P=.29). Table 2 shows all descriptive health outcomes.

During the interviews, most participants indicated that they thought they had a healthy diet. Nonetheless, a majority mentioned that the food diary helped them to become aware of their food intake. Some people were even shocked by the observation that they had such a fixed eating pattern and described PACO as a wake-up call. About half of the participants mentioned that they did introduce changes into their diet, such as cooking with more fresh ingredients, baking bread, eating more fruits and vegetables, and eating less meat. With respect to loneliness, most participants mentioned that they already had ample social contacts, even though some stated that they were feeling rather lonely. Apart from the unfortunate timing of the pandemic, 4 participants mentioned making changes in their social network because of PACO. For example, 1 participant created a list of everyone he knew and contacted them occasionally. Also, the chat connected a few people with each other and resulted in one-on-one contacts.

Following our conceptual model for health, we expected to find a significant correlation between minutes spent on the different modules and eating behavior. However, this was not the case (P>.05, see Table 3 for all correlations). With respect to the other pathways, we found that competence correlated with eating behavior (r=–0.38, P=.03) and that it predicted eating behavior over time (F_1,30=4.30, P=.047, R²=0.13). Quality of life (r=–0.60, P<.001), autonomy (r=–0.38, P=.03), relatedness (r=–0.59, P<.01), and number of chat messages (r=0.72, P=.03) correlated with loneliness, but did not predict loneliness (F_4,8=1.32, P=.40, R²=0.14).