This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2410) Dementia and Cognitive Decline (481) Mobile Devices and Apps for Seniors and Healthy Aging (167) Supporting Informal Care and Caregivers (293)

Published on in Vol 3, No 3 (2019): Jul-Sep

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14530, first published .
Using Computer Tablets to Improve Moods for Older Adults With Dementia and Interactions With Their Caregivers: Pilot Intervention Study

Using Computer Tablets to Improve Moods for Older Adults With Dementia and Interactions With Their Caregivers: Pilot Intervention Study

Using Computer Tablets to Improve Moods for Older Adults With Dementia and Interactions With Their Caregivers: Pilot Intervention Study

Authors of this article:

Aaron Gilson1 Author Orcid Image ;   Debby Dodds2 Author Orcid Image ;   Arveen Kaur1 Author Orcid Image ;   Michael Potteiger2 Author Orcid Image ;   James H Ford II1 Author Orcid Image
Article Authors Cited by (23) Tweetations (17) Metrics

    Original Paper

    1University of Wisconsin-Madison, School of Pharmacy, Madison, WI, United States

    2Generation Connect, York, PA, United States

    Corresponding Author:

    James H Ford II, PhD

    University of Wisconsin-Madison

    School of Pharmacy

    777 Highland Ave

    Madison, WI, 53705

    United States

    Phone: 1 608 262 4748

    Email: jhfordii@wisc.edu


    Background: Persons living with dementia represent a significant and growing segment of the older adult (aged 65 years and older) population. They are often challenged expressively and may experience difficulties with sharing their feelings or moods. Availability of, and easy access to, tablets facilitates the use of information and communication technologies (ICTs) as a delivery mechanism for nonpharmacological interventions, especially for persons living with dementia. Evidence of the impact of ICTs in different community settings on mood with older adults and the impact of engagement on their caregivers is needed to promote broader adoption and sustainment of these technologies in the United States.

    Objective: This study aimed to determine the extent of the effects of tablets on positive mood change and examine the effects of study variables on care recipients’ mood changes and caregivers’ daily interactions.

    Methods: The tablet intervention was developed and evaluated in five programs. The primary outcome was caregivers’ assessment of care recipients’ mood (n=1089) before and after a tablet engagement session using an eight-point mood visual analog scale. Session influence on caregivers’ daily activities was captured for a subsample of participants (n=542). Frequency distributions were computed for each study variables. Chi-square tests of association were calculated to determine the association of the variables on mood changes for all care recipients, as well as those being treated in skilled nursing facilities and in-home, and then for those that affected caregivers’ daily activities.

    Results: The study sample comprised 1089 care recipient and caregiver engagement sessions. Cumulatively, 50.78% (553/1089) of care recipients showed a transition from negative to positive moods, whereas another 41.78% (455/1089) maintained an already-positive mood after the caregiver engagement session. Chi-square analyses demonstrated that positive mood changes resulted from using music (χ210=72.9; P<.001), using YouTube as the sole app (χ212=64.5; P<.001), using multiple engagement strategies (χ22=42.8; P<.001), and when cared for in a skilled nursing facility (χ24=236.8; P<.001) across the entire care recipient sample. In addition, although many features of the engagement session positively influenced the caregivers’ day, the largest effect was observed when care recipients’ mood was considered to have improved following the session (χ24=234.7; P<.001).

    Conclusions: The study is one of the first in the United States to explore the impact of ICTs, in particular managed tablets and Web-based video services that can be used on a tablet through an app, on improving mood in persons living with dementia, and enhancing caregivers’ perceptions about their care recipient interactions. Importantly, these pilot data substantiate ICTs as part of a personalized engagement approach, as beneficial alternatives to pharmaceutical interventions for mood enhancement. However, a more comprehensive study that explores the ICT’s impact on additional clinical outcomes is needed to confirm these preliminary findings.

    JMIR Form Res 2019;3(3):e14530

    doi:10.2196/14530

    Keywords

    mood change; caregiver interactions; older adults; Alzheimer disease; dementia; computer tablets; person-centered care 


    Background

    The number of people living with Alzheimer and related dementia is projected to triple by 2050 [1-3]. Persons living with dementia can have complex physical health concerns, comorbid medical conditions, and exhibit behavioral and psychological symptoms of dementia (BPSD) [4-11]. In fact, professional caregivers of persons living with dementia already note that responding to BPSD is one of the greatest care challenges and can negatively affect the health of caregivers [12-14] especially in long-term health care facilities, where two-thirds of residents have dementia [9,13,15,16]. Family caregivers of persons living with dementia are also are at risk for higher rates of depression, health and sleep issues, social isolation, and mortality [14,17].

    Pharmacological or nonpharmacological interventions can help manage BPSD in older adults with moderate-to-severe cognitive impairment. Medications may control the physical aspects of BPSD but can have many side effects [18]. Nonpharmacological approaches, such as reminiscence therapy, music therapy, or behavior management techniques, are preferred because they sustain cognitive function, improve quality of life, and mitigate BPSD [19-25]. The growing number of persons living with dementia, overall care challenges, and the negative effects of pharmacological interventions combine to highlight the need, as well as the opportunity, for additional nonpharmacological interventions.

    Availability of tablets and mobile phones has changed how nonpharmacological interventions can be delivered to older adults. Tablets can enhance residents’ emotions through the use of multisensory activities. Individuals can now obtain informational content, interact socially (including participating in Web-based games), listen to music, or reminisce about the past or a recent special event. Easy access to information and communication technologies (ICTs) such as tablets facilitates the delivery of nonpharmacological interventions especially for persons living with dementia and their caregivers [26-28]. For persons living with dementia, ICTs such as YouTube, a Web-based video service, can be used to deliver reminiscence therapy interventions on a tablet through an app [29]. For these individuals, the use of ICTs mitigates motor and sensory impairments; compensates for memory deficits; or enhances latent skills and abilities for sensory awareness, musical responsiveness, and emotional memory [30,31]. Overall, the use of ICTs to deliver nonpharmacological interventions for persons living with dementia benefits care recipients’ well-being and mood; communication and interactions; and caregivers’ mental health (eg, depression and anxiety), self-efficacy, and relationship with the care recipient [26,28,29,32-39]. An additional benefit is the potential for persons living with dementia to remain in the home for as long as possible. This outcome is a widespread aspiration, with “Nearly 90 percent of people over age 65 [wanting] to stay in their homes for as long as possible” [40]. Therapeutic approaches that help persons living with dementia live at home longer also reduces overall health care costs [41].

    Development of a Tablet Intervention

    Although successful use of tablets originated and empirically documented about a decade ago in Great Britain, with investigations continuing into the present both in Europe and beyond [42-47], its adoption in the United States has been slow [34,48-55]. Two Great Britain studies offer a progressive look at how tablets can be applied to improve communications between persons living with dementia and their caregivers [56,57]. The first study compared a traditional reminiscence programs (noncomputer) with computerized touchscreen, one-on-one, reminiscence sessions and tested the hypothesis that caregivers offer greater choices and engage in more conversational activities during the touchscreen sessions than during the traditional one-on-one reminiscence sessions with physical properties [56]. After comparing the 2 types of sessions across both verbal and nonverbal parameters, such as laughter, singing, pointing, and eye gaze, the touchscreen system was shown to provide greater benefits for both caregivers and persons living with dementia. The second study was based on the central concept that “external memory aids have demonstrated a reduction in resistance to care...” [57], and using the iPad as the focus of joint attention not only relieves some of the burden of communication for caregivers but also provides the opportunity for the caregiver to support greater independence in persons living with dementia through scaffolding behavior.

    The project described in this study extends this seminal international work to implement a tablet intervention using multiple device management and software apps in different community settings and document the effect on mood with older adults, as well as the impact of engagement on their caregivers. This manuscript reports aggregate findings from a pilot study evaluating the impact of personalized tablet engagement sessions. Four project aims represent the purpose of this study to examine study variables effects on mood changes in (1) the overall care recipient sample, (2) the subsample of care recipients being treated in skilled nursing facilities (SNFs), (3) care recipients in home care only, and (4) caregivers’ daily activities.


    Personalized Tablet Engagement

    One of the primary advantages in using a tablet for engagement activities for persons living with dementia is the ability to personalize activities because every person has a unique lifetime of experiences. For this program, the term personalized tablet engagement reflects a tablet engagement session that is designed by a caregiver based on the personal history of the care recipient. The session is presented to the care recipient, who is given the option of participating. In this way, tablet engagement is both based on an individual’s interests, abilities, and preferences and allows the individual to exercise autonomy.

    Study Settings

    Since 2015, Generation Connect initiated partnerships to evaluate the effectiveness of tablet interventions in dyads of caregivers and persons living with dementia. The Music & Memory Foundation used tablet computers that were preconfigured with recommended apps, deployed with management software, and implemented with training resources created and facilitated by Generation Connect. These tablets allowed tailored interactions to residents’ interests or hobbies to expedite engagement and enhance emotions through multisensory activities.

    This study describes a naturalistic study in which the Music & Memory Foundation and a Visiting Angels in-home care agency franchise collaborated with Generation Connect to develop and conduct an ongoing tablet engagement pilot study involving 5 different programs across time (2017-2018). Multimedia Appendix 1 describes the 5 programs contributing the tablet engagement session data analyzed for this study. 

    Participants’ Attributes

    The intervention was designed to be useful for persons living with dementia, and all participating communities opted into the pilot study and self-selected the residents that participated, no standardized test was used to record non-normative memory loss. The sessions were not limited restricted to a dementia population and contained a variety of other conditions as well. However, participants’ demographics beyond these primary challenges were not collected in the pilot programs.

    Data Collection

    Caregiver Training for Tablet Engagement Sessions

    Caregivers were trained to gather personal history and take personal lifestyle into consideration when selecting the activities for care recipients. As a result, caregivers were better able to provide multisensory tablet engagement for specific and prior positive life events, such as reminiscing or listening to personal music. All training was conducted by members of Generation Connect in a series of live webinars, Web-based courses, and printable handouts. Instructors demonstrated apps and talked through best practices for meeting the engagement training elements of discovery, engagement, and planning, emphasizing that the engagement process is guided by care recipient needs (see Multimedia Appendix 2). The training process was dynamic and changed over the course of time as new information was gathered.

    Engagement
    Personalized Tablet Engagement

    Many older adults are hesitant to interact with a device or computer but are comfortable watching videos. The caregiver’s goal for engagement is to make the tablet disappear and provide an enjoyable personalized experience. The most successful engagement sessions reflect a best friend approach, and engagement is based on mentoring not care recipient mastery. This study did not provide official tracking of session frequencies or lengths.

    Selection of Apps

    This project was conceptualized over a number of years, based on supporting research at that time. As a result, apps available in the public domain (see Multimedia Appendix 3) were chosen because they represented at least 1 of 4 basic types of traditional activity-based engagement activities that evidence suggests can be beneficial for people with dementia: reminiscing, music, images, and games [58-63].

    Engagement Session Feedback Survey

    Data collection in the programs was to improve residents’ socialization, personalized care, and mood management through nonpharmacological interventions and make the day better for caregivers. The short, 5-question survey was available in a Web app on the tablets’ homepage for easy caregiver access (see Multimedia Appendix 4). Instructors demonstrated how to use the app survey to record session feedback information.

    Mood Rating

    Persons living with dementia are often challenged expressively and may experience difficulties with sharing their feelings or moods. A visual analog mood scale (VAMS) represents a valid tool to assess baseline and changes in mood, especially in those with expressive disabilities [48,52,55] and for dementia [64]. VAMS consists of a sequence of 8 faces (Figure 1) and was incorporated into the tablets used by Generation Connect. Caregivers reported changes in care recipient mood before and immediately after their tablet engagement activity.

    For this analysis, individual moods were categorized into 2 domains: (1) negative moods (ie, angry, anxious, sad, confused, and indifferent) and (2) positive moods (ie, relaxed, happy, and joyful). Indifference was considered a negative mood because of its potential to adversely affect other aspects of the care recipients’ functioning throughout the day, as well as influence caregivers’ activities [65]. Mood changes were then categorized according to the mood assessed before the care recipient and caregiver engagement compared with the mood observed after engagement, resulting in 4 categories: worsening of mood (ie, transitions from positive to negative mood domains), maintaining negative mood (ie, changes remain within the negative mood domain), maintaining positive mood (ie, changes remain within the positive mood domain), or improvement of mood (ie, transitions from negative to positive mood domains).

    ‎
    Figure 1. Participants’ mood rating scale.
    View this figure
    Caregivers’ Daily Activities

    The impact of the sessions with older adults was captured for a subsample of caregivers across the 5 programs. Caregivers responded to the following question: how did the engagement session impact your day? Impact was rated using a 5-point scale anchored at 0 (made it worse), 3 (no impact), and 5 (made it better). Scale values for 2 and 4 had no assigned anchor label but were, in this analysis, assigned slight worsening and slight improvement, respectively.

    Data Cleaning

    All data were reviewed for accuracy and completeness. In addition, data cleaning and label standardization ensured response uniformity before analysis. Our team then reviewed and converted the variables into the following categories:

    • Care recipients’ primary physical challenge—memory loss, nonambulatory, movement disorder, communicative or expressive disorders, or other
    • Method used for care recipient and caregiver engagement (either singularly or combined with other methods)—music, stories video, games, communication, photos, or other
    • Type of strategy of care recipient and caregiver engagement (either singularly or combined with other strategies)—music, reminiscing, socialization, relaxation, or achievement
    • Number of strategies used—either 1 or 2 or more
    • Type of app used to facilitate strategies—YouTube only, Personal playlist only, Personal photos or videos only, Google only, Puzzles only, combination of apps, or other
    • Number of apps used to facilitate strategies—either 1 or 2 or more

    Facility type was self-reported as skilled nursing, home care, or other (which consist of Continuing Care Center, Assisted Living Facility, Long-Term Care Facility, and Veterans Hospital). Facility locations were classified as rural or urban using the US Department of Agriculture Rural-Urban Continuum Code, and for Canadian facilities, rural or urban status was determined through website search.

    Statistical Analysis

    Frequency distributions were computed for each of the study variables, including the type of care recipients’ mood changes. All variables were collected as categorical and ranged from dichotomous, trichotomous, up to 6 categories. Chi-square tests of association were calculated to determine the influence of the variables on mood changes for all care recipients, as well as those being treated in SNFs and in-home, and then those that affected caregivers’ daily activities.

    Ethics Consideration

    The project was determined to constitute quality improvement or program evaluation and in accordance with federal regulations, the project did not constitute research as defined under 45 Code of Federal Regulations 46.102(d).


    Posthoc Power Analysis

    Posthoc power analysis for the chi-square tests demonstrated superior power for all analyses. The power levels calculated for all 4 project aims conformed to Cohen’s accepted standard of a four-to-one weighting between type II and type I error risks [66]. In addition, no significant chi-square results were associated with more than 33% of contingency table cells having expected counts of less than 5.

    Descriptive Statistics

    Frequencies were computed for each study variable and each of the analyzed samples (ie, full sample, SNFs only, and home care only), as shown in Multimedia Appendix 5, which also demonstrates a relatively unequal distribution of resident mood changes after use of the tablet.

    Table 1 represents overall mood improvement after the use of the app technology among individuals with various disabilities and is further illustrated when examining frequencies within each of the mood change categories.

    Table 1. Mood transitions for individuals.
    Mood before sessionMood after session, n (%)

    		AngryAnxiousSadConfusedIndifferentRelaxedHappyJoyful
    Angry (n=33)2 (6)a—b———2 (6)c11 (33)c18 (55)c
    Anxious (n=52)—4 (8)a2 (4)a2 (4)a4 (8)a24 (46)c11 (21)c5 (10)c
    Sad (n=64)——2 (3)a1 (2)a—3 (5)c30 (47)c28 (44)c
    Confused (n=70)—1 (1)a—3 (4)a6 (9)a29 (41)c23 (33)c8 (11.4)c
    Indifferent (n=407)1 (0.2)a1 (0.2)a—7 (1.7)a36 (8.9)a91 (22.5)c176 (43.2)c95 (23.3)c
    Relaxed (n=119)———1 (0.8)d6 (5.0)d28 (23.5)e70 (58.8)e14 (11.8)e
    Happy (n=326)—————7 (2.1)e257 (78.8)e62 (19.0)e
    Joyful (n=18)——1 (6)d———5 (28)e12 (67)e

    aNot applicable as no care recipients transitioned to this mood from their beginning mood.

    bMaintaining negative mood.

    cTransition from negative to positive mood.

    dWorsening of mood.

    eMaintaining positive mood.

    Worsening of Mood (Transition From Positive to Negative Mood Domains; n=8)

    Any worsening of mood represented the mood change category with the fewest number of care recipients. One care recipient began an engagement session in a joyful mood but ended up confused after the tablet intervention, whereas another care recipient transitioned from relaxed to confused. The remaining 6 cases involved a transition from relaxed to indifferent.

    Maintaining Negative Mood (n=72)

    Within this category, 65% (47/72) of care recipients showed no mood change, and most (n=36) of these moods were viewed as indifference. The most frequent mood changes were found for transitions from indifferent to confused (n=7) and confused to indifferent (n=6).

    Maintaining Positive Mood (n=455)

    Within this category, 65.3% (297/455) of care recipients showed no mood change, and most (257/297, 86.5%) of these moods were viewed as happy. The most frequent mood transitions within the positive mood domain represented changes from relaxed to happy (n=70) and then from happy to joyful (n=62), and the most notable improvement was found for transitions from relaxed to joyful (n=14).

    Improvement of Mood (Transition From Negative to Positive Mood Domains; n=553)

    Two-thirds (362/553) of all care recipients began the sessions in an indifferent mood and then increased to relaxed (n=91), happy (n=176), or joyful (n=95). Furthermore, the proportion of care recipients achieving joyful mood after the session was noteworthy. Cumulatively, 21% of care recipients who began the session as either anxious (5/52) or confused (8/70) became joyful as a result of the session, whereas 44% (28/64) of care recipients who were sad ended the session in a joyful mood. Importantly, 18 of the 33 care recipients (58%) transitioned from being angry to being joyful, demonstrating the greatest movement possible within this mood category.

    Chi-Square Analyses

    Table 2 contains the frequencies for each study variable within the 4 separate mood change categories, which provides insight into the category of each variable that contributed to the mood effect.

    Table 2. Chi-square comparisons of variables to mood change in full sample.
    VariablesMood change

    		Worse or maintaining negative mood (n=80), n (%)Maintaining positive mood (n=455), n (%)Mood improvement (n=553), n (%)
    Primary challengea

    		Memory loss61 (5.60)424 (38.93)300 (27.55)

    		Nonambulatory6 (0.55)6 (0.55)93 (8.54)

    		Movement disorder10 (0.92)13 (1.19)105 (9.64)

    		Communicative or expressive disorders2 (0.18)9 (0.83)45 (4.13)

    		Other1 (0.09)4 (0.37)10 (0.92)
    Engagement method use (singularly or one of multiple)b

    		Music22 (2.3)127 (13.4)228 (24.1)

    		Stories video12 (1.3)101 (10.7)86 (9.1)

    		Games13 (1.4)67 (7.1)33 (3.5)

    		Communication1 (0.1)47 (5.0)48 (5.1)

    		Photos3 (0.3)48 (5.1)24 (2.5)

    		Other6 (0.6)21 (2.259 (6.2)
    Type of strategies usedc

    		Music


    		Singularly15 (2.9)89 (17.3)95 (18.5)


    		One or more26 (5.1)81 (15.8)207 (40.4)

    		Reminiscing


    		Singularly2 (0.7)5 (1.6)36 (11.8)


    		One or more7 (2.3)93 (30.6)161 (53.0)

    		Socialization


    		Singularly9 (1.7)119 (22.2)58 (10.8)


    		One or more15 (2.8)129 (24.07)206 (38.4)

    		Relaxation


    		Singularly2 (0.8)4 (1.7)7 (3.0)


    		One or more21 (8.9)60 (25.4)142 (60.2)

    		Achievement


    		Singularly10 (4.0)60 (23.9)33 (13.1)


    		One or more6 (2.4)62 (24.7)80 (31.9)
    Number of strategies usedd

    		140 (3.75)276 (25.89)225 (21.11)

    		≥233 (3.10)170 (15.95)322 (30.21)
    Type of app used for strategiesd

    		YouTube only10 (0.94)117 (10.98)178 (16.70)

    		Personal Playlist only36 (3.38)115 (10.798)157 (14.73)

    		Personal photos or videos only0 (0.00)35 (3.28)41 (3.85)

    		Google only3 (0.28)53 (4.97)35 (3.28)

    		Puzzles only6 (0.56)48 (4.50)24 (2.25)

    		Combinations of apps10 (0.94)52 (4.88)45 (4.22)

    		Other11 (1.03)27 (2.53)63 (5.91)
    Number of apps used for strategiese

    		166 (6.20)394 (37.00)498 (46.76)

    		≥210 (0.94)52 (4.88)45 (4.23)
    Urban or rural statusa

    		Rural39 (3.58)64 (5.88)306 (28.10)

    		Urban41 (3.76)392 (36.00)247 (22.68)
    Facility typea

    		Skilled nursing28 (2.57)42 (3.86)303 (27.82)

    		Home care46 (4.22)393 (36.09)227 (20.84)

    		Other6 (0.55)21 (1.93)23 (2.11)

    aPercentages for primary challenge in the chi-square analysis are based on a sample of 1089 care recipients. Differences as compared with the total sample is based on missing data, which were excluded from the chi-square analysis.

    bPercentages for engagement method use in the chi-square analysis is based on a sample of 946 care recipients.

    cPercentages for type of strategy used in the chi-square analysis varied based on the strategy. The sample sizes are music (n=513), reminiscing (n=304), socialization (n=506), relaxation (n=236), and achievement (n=251), respectively.

    dPercentages for number of strategies used and the type of app used for the strategies in the chi-square analysis is based on a sample of 1066 care recipients.

    ePercentages for number of apps used for strategies in the chi-square analysis is based on a sample of 1065 care recipients.

    Project Aim 1: Analysis of Full Sample

    Chi-square analyses of all care recipients were conducted to determine the extent that significant mood changes occurred for those receiving the intervention (n=1089). When examining the effects of the care recipients’ primary health care challenges on their observed mood, memory loss was associated with maintaining positive mood (χ28=191.5; P<.001). Engagement methods demonstrated some effect on care recipients’ mood, with music being more likely involved in care recipients’ improvements from negative to positive moods (χ210=72.9; P<.001). Characteristics of the apps used to engage also influenced mood. When using YouTube as the sole app, care recipients’ mood transitioned from negative to positive after the engagement sessions (χ212=64.5; P<.001). Results also suggested that some strategies implemented to evoke mood changes were effective. Specifically, using music as a single method (χ22=20.1; P<.001) or reminiscing (χ22=9.9; P=.007) as a component of additional strategies was associated with maintenance of positive mood; socialization (χ22=38.3; P<.001) and achievement (χ22=12.9; P=.002), when used as a component, were related to care recipients having improved mood. These findings support the separate analysis showing that care recipients using multiple strategies were more likely to be associated with negative to positive mood transitions (χ22=42.8; P<.001). Examining the influences of facility characteristics revealed that care recipient placement in an SNF was linked to changes from negative to positive moods following engagement (χ24=236.8; P<.001), whereas placement in an urban setting was associated with maintaining positive mood (χ22=186.4; P<.001).

    Project Aim 2: Analysis of Care Recipients in Skilled Nursing Facilities

    Subsample analyses were designed to reveal the effects of the study variables on mood changes for care recipients from SNFs only (n=373). Notably, fewer significant results emerged from this sample compared with either the full sample or the home care subsample analyses; however, these results demonstrated a consistent effect on mood. Engagement sessions involving reminiscing were more likely to facilitate care recipients’ going from negative to positive moods as a result of the engagement sessions (χ24=12.0; P=.02). Socialization, when used as a component of additional strategies, also was related to care recipients having positive mood changes (χ22=11.0; P=.004). Overall, care recipients using multiple strategies were associated with improvements from negative to positive moods (χ22=8.5; P=.014). This trend substantiates the finding derived from the full sample, employing 2 or more strategies is an effective approach for positive moods. Finally, engagement sessions occurring in SNFs operating in an urban setting was associated with maintaining positive mood (χ22=18.4; P<.001).

    Project Aim 3: Analysis of Care Recipients in Home Care

    A subsample analysis also was conducted for care recipients involved only in in-home care (n=666). Care recipients with a movement disorder as their primary challenge were associated with negative to positive mood changes as a result of the intervention (χ22=115.7; P<.001). Engagement sessions involving music were more likely linked to care recipients’ improvements from negative to positive moods (χ210=112.7; P<.001). Furthermore, using combinations of strategies involving music was associated with mood improvement (χ22=34.4; P<.001). The only other strategy achieving significance was the sole use of socialization, which was shown to maintain positive mood (χ22=45.5; P<.001). Overall, although, any strategy used in combination with others was related to sustaining positive moods (χ21=75.8; P<.001). When considering specific apps, Personal Playlist, as the sole app, was more likely to be associated with care recipients transitioning from negative to positive mood (χ212=135.5; P<.001). Finally, home care occurring in urban settings was associated with care recipients sustaining positive mood as a result of the intervention (χ22=57.5; P<.001).

    Project Aim 4: Analysis of Caregivers’ Daily Activities

    A subsample of caregivers also rated the impact of the intervention on their day (n=542). When care recipients had a movement disorder, the engagement session made the day better for caregivers (χ22=10.1; P=.006). Music, when used singularly, slightly improved the caregiver’s day (χ22=17.2; P<.001), whereas socialization as a component of other interventions made the day better for the caregiver (χ22=37.2; P<.001). Using only 1 strategy (χ22=39.4; P<.001) or using apps multiple times (χ22=12.7; P=.002) slightly improved the caregiver’s day, but using only Personal Playlist apps made the caregiver’s day much better (χ212=91.9; P<.001). In urban settings, the positive impact of the engagement session was only slight (χ22=119.8; P<.001). When engagement occurred in other than SNFs or home care, the session made the day better for caregivers (χ24=11.0; P=.03). Importantly, when care recipients’ mood was observed to improve, the engagement session was more likely to make the day better for staff (χ24=234.7; P<.001).


    Principal Findings

    This study is one of the first in the United States to focus on the implementation of tablets across institutional (eg, SNFs) and in-home settings for persons living with dementia. The most apparent implication from this study was the substantial benefit of the tablet interventions to most care recipients. Less than 8% of the entire sample evidenced either the transition of positive to negative mood (8/1089, 0.73%) or the sustainment of negative moods (72/1089, 6.61%) after caregiver engagement sessions. That is, for the vast majority of the entire sample, positive care recipient moods were either maintained or enhanced or were achieved after beginning the session in a negative mood. The unequivocal therapeutic benefit of personalized tablet engagement with music and other approaches seems a function not only of the effectiveness of the particular intervention but also of caregiver commitment to and compassion for the care recipient.

    Caregiver supports that maintain interest and participation, as well as promote feelings of achievement and mastery, create a positive experience for participants and caregivers with tablets [33]. Successful interactions should provide achievable goals facilitating feelings of self-worth, which require the caregiver to be in the moment and select apps based on resident mood to promote engagement [33]. Study results extend this concept further, indicating that app selection should target multiple strategies (eg, reminiscing or socialization) to facilitate improvements in resident moods.

    Our findings also offer guidance around the care recipient and caregiver engagement process. Specifically, the use of a single app, especially YouTube, resulted in moderate to substantial improvements in resident mood across the entire sample. This influence of YouTube supports additional research, indicating that exposure to YouTube videos improves participants’ well-being and mood, as well as their communication, interaction, and engagement [29]. In our study, YouTube was primarily used to share music or stories through video and strategies related to reminiscing, socialization, or music. One resident encounter convincingly illustrates tablet impact on resident mood:

    Listened and watched a YouTube video with headphones and closed caption in Navajo. Elder was happy and smiling. He sang along in Navajo and tried [to] teach me the words in Navajo.
    [From one resident encounter]

    This session successfully targeted all 3 strategies, and the resident’s mood improved the greatest extent possible, from sad to joyful. Years of experience has shown that YouTube holds great potential for personalization because of the nearly unlimited number of topics available. When given the choice of activity, caregivers are choosing to work with YouTube.

    Interestingly, using gaming apps (eg, Puzzable or Solitaire) to engage care recipients did not statistically improve mood, especially compared with other engagement strategies. For these care recipients, the complexity of playing the game may have undermined mood improvement. For example, one resident reported the following:

    the worst part [of the game] was there were too many questions - wanted too much information before getting to anything interesting.
    [One resident]

    Although an unfortunate reaction, this finding supports research suggesting that gaming apps are not as useful for promoting engagement as apps targeting relaxation or telling an individual’s life story in this population [34]. Conversely, other research demonstrates that interactive computer games (eg, a Nintendo Wii) affected participant cognition [67]. These contradictory findings warrant further research to examine the specific gaming features and the interaction of these features with care recipient characteristics and cognitive conditions that prompt mood changes.

    Mood changes differed by location. Overall, 82.9% (339/409) of care recipients in rural areas reported more negative moods (sad, confused, angry, anxious, or indifferent) at baseline versus those in urban areas (286/680, 42.1%). Similarly, 88.2% (329/373) of care recipients in SNFs began sessions with a negative mood compared with 40.1% (267/666) of individuals in home care settings. However, after tablet engagement, only 7.5% (28/373) of individuals were assessed with a negative mood. A potential explanation for this finding is a greater continuity of care existing between caregivers and care recipients in home care sessions, including more frequent and sustained interactions. How facility-setting characteristics influence care recipient or caregiver mood and mood changes, along with their feelings of isolation or loneliness, remains crucial research topics when implementing tablets or other ICTs.

    Limitations

    Although illustrating numerous areas for investigation, this study had limitations. First, to ease data collection burden on caregivers, no resident or caregiver attributes (such as gender, age, or duration of the primary challenges) or years of caregiving service were documented. Second, although efforts were made to systematize training and data collection, there may have been variability in these processes either across or within programs. For example, the key strategy (eg, music or reminiscing) may have been recorded inconsistently. Third, as caregiver observations determined mood baseline and changes, knowledge of study objectives or varying assessments of residents’ mood could, even unintentionally, introduce bias. Fourth, the time allotted for the care recipient and caregiver interaction may have affected mood changes but was not collected. Finally, findings related to the caregivers’ day suggest that the intervention was beneficial overall, but the effect may differ for larger samples or other care recipient or caregiver samples.

    Conclusions

    This study documents further evidence that ICTs, in particular, tablets such as iPads and apps such as YouTube or using a Personal Playlist, improve mood in older adults and enhance caregiver perceptions about their care recipient interactions, especially with persons living with dementia. Importantly, these pilot data substantiate ICTs, as part of a personalized engagement approach, as beneficial alternatives to pharmaceutical interventions for mood enhancement. As was done for this study, matching engagement methods and types of strategies and apps, based on care recipient preferences or other features to maximize their mood effects, is critical for strengthening the utility of this approach. Accumulating evidence and the additional research questions emerging from this nascent examination offer exciting prospects for continued empirical inquiry and bettering care recipient well-being.

    Acknowledgments

    The data for the project were provided by the staff at Generation Connect, and the analysis was supported by internal funding available to JF from the University of Wisconsin–Madison. The pilot projects were supported by grants from the Consumer Technology Association and Verizon. The Visiting Angels pilot implemented through a paid business contract with Visiting Angels. Without the participation of older adults and their caregivers, this project would not have been possible.

    Conflicts of Interest

    DD and MP work for Generation Connect, which is a gerontology-driven technology company with an intergenerational team that works together with forward-thinking care organizations to build engagement solutions that promote person-centered care through technology innovation. All other authors report no conflict of interest.

    ‎
    Multimedia Appendix 1

    Description of tablet engagement programs.

    PDF File (Adobe PDF File), 247KB
    ‎
    Multimedia Appendix 2

    Caregiver training process.

    PDF File (Adobe PDF File), 221KB
    ‎
    Multimedia Appendix 3

    Description of the apps included on the tablet.

    PDF File (Adobe PDF File), 43KB
    ‎
    Multimedia Appendix 4

    Feedback survey questions.

    PDF File (Adobe PDF File), 131KB
    ‎
    Multimedia Appendix 5

    Frequency of study variables.

    PDF File (Adobe PDF File), 288KB
    1. Alzheimer's Association. 2013 Alzheimer's disease facts and figures. Alzheimers Dement 2013 Mar;9(2):208-245 [FREE Full text] [CrossRef] [Medline]
    2. Alzheimer's Association. 2014 Alzheimer's disease facts and figures. Alzheimers Dement 2014 Mar;10(2):e47-e92. [CrossRef] [Medline]
    3. Hebert LE, Weuve J, Scherr PA, Evans DA. Alzheimer disease in the United States (2010-2050) estimated using the 2010 census. Neurology 2013 May 7;80(19):1778-1783 [FREE Full text] [CrossRef] [Medline]
    4. Cummings JL, Mega M, Gray K, Rosenberg-Thompson S, Carusi DA, Gornbein J. The neuropsychiatric inventory: comprehensive assessment of psychopathology in dementia. Neurology 1994 Dec;44(12):2308-2314. [CrossRef] [Medline]
    5. Dewing J. Sundowning in older people with dementia: evidence base, nursing assessment and interventions. Nurs Older People 2003 Nov;15(8):24-31; quiz 32. [CrossRef] [Medline]
    6. Ishii S, Streim JE, Saliba D. A conceptual framework for rejection of care behaviors: review of literature and analysis of role of dementia severity. J Am Med Dir Assoc 2012 Jan;13(1):11-23.e1. [CrossRef] [Medline]
    7. Khachiyants N, Trinkle D, Son SJ, Kim KY. Sundown syndrome in persons with dementia: an update. Psychiatry Investig 2011 Dec;8(4):275-287 [FREE Full text] [CrossRef] [Medline]
    8. Konno R, Kang HS, Makimoto K. A best-evidence review of intervention studies for minimizing resistance-to-care behaviours for older adults with dementia in nursing homes. J Adv Nurs 2014 Oct;70(10):2167-2180. [CrossRef] [Medline]
    9. McKenzie G, Teri L, Pike K, LaFazia D, van Leynseele J. Reactions of assisted living staff to behavioral and psychological symptoms of dementia. Geriatr Nurs 2012;33(2):96-104. [CrossRef] [Medline]
    10. Mega MS, Cummings JL, Fiorello T, Gornbein J. The spectrum of behavioral changes in Alzheimer's disease. Neurology 1996 Jan;46(1):130-135. [CrossRef] [Medline]
    11. Spector A, Orrell M, Goyder J. A systematic review of staff training interventions to reduce the behavioural and psychological symptoms of dementia. Ageing Res Rev 2013 Jan;12(1):354-364. [CrossRef] [Medline]
    12. de Vera MA. Institutional Repository (IR) at the University of San Francisco. 2015. Improving Care of Assisted Living Residents with Dementia Through a Competency-Based Behavioral Training Curriculum for Assisted Living Caregivers   URL: https://repository.usfca.edu/cgi/viewcontent.cgi?article=1061&context=dnp [accessed 2019-06-25]
    13. Park-Lee E, Sengupta M, Harris-Kojetin LD. Dementia special care units in residential care communities: United States, 2010. NCHS Data Brief 2013 Nov(134):1-8 [FREE Full text] [Medline]
    14. Reinhard SC, Feinberg LF, Choula R, Houser A. AARP. 2015. Valuing the Invaluable: 2015 Update: Undeniable Progress, but Big Gaps Remain   URL: https://www.aarp.org/content/dam/aarp/ppi/2015/valuing-the-invaluable-2015-update-new.pdf [accessed 2019-06-19]
    15. Alzheimer's Association. 2019 Alzheimer's disease facts and figures. Alzheimers Dement 2019 Mar;15(3):321-387. [CrossRef]
    16. Zimmerman S, Sloane PD, Reed D. Dementia prevalence and care in assisted living. Health Aff (Millwood) 2014 Apr;33(4):658-666. [CrossRef] [Medline]
    17. Kasper JD, Freedman VA, Spillman BC, Wolff JL. The disproportionate impact of dementia on family and unpaid caregiving to older adults. Health Aff (Millwood) 2015 Oct;34(10):1642-1649 [FREE Full text] [CrossRef] [Medline]
    18. Olazarán J, Reisberg B, Clare L, Cruz I, Peña-Casanova J, Del Ser T, et al. Nonpharmacological therapies in Alzheimer's disease: a systematic review of efficacy. Dement Geriatr Cogn Disord 2010;30(2):161-178 [FREE Full text] [CrossRef] [Medline]
    19. Caspar S, Davis ED, Douziech A, Scott DR. Nonpharmacological management of behavioral and psychological symptoms of dementia: what works, in what circumstances, and why? Innov Aging 2018 Jan;2(1):igy001 [FREE Full text] [CrossRef] [Medline]
    20. Na R, Yang JH, Yeom Y, Kim YJ, Byun S, Kim K, et al. A systematic review and meta-analysis of nonpharmacological interventions for moderate to severe dementia. Psychiatry Investig 2019 May;16(5):325-335 [FREE Full text] [CrossRef] [Medline]
    21. Wang G, Albayrak A, van der Cammen TJ. A systematic review of non-pharmacological interventions for BPSD in nursing home residents with dementia: from a perspective of ergonomics. Int Psychogeriatr 2018 Oct 18:1-13 (epub ahead of print)(forthcoming). [CrossRef] [Medline]
    22. World Health Organization. First WHO Ministerial Conference on Global Action Against Dementia. Geneva, Switzerland: World Health Organization; 2015.
    23. Cabrera E, Sutcliffe C, Verbeek H, Saks K, Soto-Martin M, Meyer G, et al. Non-pharmacological interventions as a best practice strategy in people with dementia living in nursing homes. A systematic review. Eur Geriatr Med 2015 Apr;6(2):134-150. [CrossRef]
    24. Kulibert D, Ebert A, Preman S, McFadden SH. In-home use of personalized music for persons with dementia. Dementia (London) 2018 Jan 1:1471301218763185 (epub ahead of print)(forthcoming). [CrossRef] [Medline]
    25. O'Neil ME, Freeman M, Christensen V, Telerant R, Addleman A, Kansagara D. A Systematic Evidence Review of Non-Pharmacological Interventions for Behavioral Symptoms of Dementia. Washington, DC: Department of Veterans Affairs; 2011.
    26. Lewis V, Bauer M, Winbolt M, Chenco C, Hanley F. A study of the effectiveness of MP3 players to support family carers of people living with dementia at home. Int Psychogeriatr 2015 Mar;27(3):471-479. [CrossRef] [Medline]
    27. Lorenz K, Freddolino PP, Comas-Herrera A, Knapp M, Damant J. Technology-based tools and services for people with dementia and carers: mapping technology onto the dementia care pathway. Dementia (London) 2019 Feb;18(2):725-741. [CrossRef] [Medline]
    28. Nijhof N, van Gemert-Pijnen JE, Seydel E. A personal assistant for dementia to stay at home safe at reduced cost. Gerontechnology 2013;11(3):469-479. [CrossRef]
    29. O'Rourke J, Tobin F, O'Callaghan S, Sowman R, Collins DR. 'YouTube': a useful tool for reminiscence therapy in dementia? Age Ageing 2011 Nov;40(6):742-744. [CrossRef] [Medline]
    30. Lazar A, Thompson H, Demiris G. A systematic review of the use of technology for reminiscence therapy. Health Educ Behav 2014 Oct;41(1 Suppl):51S-61S [FREE Full text] [CrossRef] [Medline]
    31. Ryan AA, McCauley CO, Laird EA, Gibson A, Mulvenna MD, Bond R, et al. 'There is still so much inside': the impact of personalised reminiscence, facilitated by a tablet device, on people living with mild to moderate dementia and their family carers. Dementia (London) 2018 Sep 7:1471301218795242 (epub ahead of print)(forthcoming). [CrossRef] [Medline]
    32. Kerssens C, Kumar R, Adams AE, Knott CC, Matalenas L, Sanford JA, et al. Personalized technology to support older adults with and without cognitive impairment living at home. Am J Alzheimers Dis Other Demen 2015 Feb;30(1):85-97 [FREE Full text] [CrossRef] [Medline]
    33. Smith SK, Mountain GA, Hawkins RJ. Qualitatively exploring the suitability of tablet computers to encourage participation with activities by people with moderate stage dementia. Dementia (London) 2018 Sep 30:1471301218802897 (epub ahead of print)(forthcoming). [CrossRef] [Medline]
    34. Ford II JH, Dodds D, Hyland J, Potteiger M. Evaluating the impact of music & memory’s personalized music and tablet engagement program in Wisconsin assisted living communities: pilot study. JMIR Aging 2019 Mar 14;2(1):e11599. [CrossRef]
    35. Boots LM, de Vugt ME, Kempen GI, Verhey FR. Effectiveness of a blended care self-management program for caregivers of people with early-stage dementia (partner in balance): randomized controlled trial. J Med Internet Res 2018 Jul 13;20(7):e10017 [FREE Full text] [CrossRef] [Medline]
    36. Hopwood J, Walker N, McDonagh L, Rait G, Walters K, Iliffe S, et al. Internet-based interventions aimed at supporting family caregivers of people with dementia: systematic review. J Med Internet Res 2018 Jun 12;20(6):e216 [FREE Full text] [CrossRef] [Medline]
    37. Laird EA, Ryan A, McCauley C, Bond RB, Mulvenna MD, Curran KJ, et al. Using mobile technology to provide personalized reminiscence for people living with dementia and their carers: appraisal of outcomes from a quasi-experimental study. JMIR Ment Health 2018 Sep 11;5(3):e57 [FREE Full text] [CrossRef] [Medline]
    38. Zhao Y, Feng H, Hu M, Hu H, Li H, Ning H, et al. Web-based interventions to improve mental health in home caregivers of people with dementia: meta-analysis. J Med Internet Res 2019 May 6;21(5):e13415 [FREE Full text] [CrossRef] [Medline]
    39. Delello JA, McWhorter RR. Reducing the digital divide: connecting older adults to iPad technology. J Appl Gerontol 2017 Jan;36(1):3-28. [CrossRef] [Medline]
    40. Farber N, Shinkle D, Lynott J, Fox-Grage W, Harrell R. Activities for People with Alzheimer's disease - AARP. 2011. Aging in Place: A State Survey of Livability Policies and Practices   URL: https://assets.aarp.org/rgcenter/ppi/liv-com/aging-in-place-2011-full.pdf [accessed 2019-06-24]
    41. Deb A, Thornton JD, Sambamoorthi U, Innes K. Direct and indirect cost of managing alzheimer's disease and related dementias in the United States. Expert Rev Pharmacoecon Outcomes Res 2017 Apr;17(2):189-202 [FREE Full text] [CrossRef] [Medline]
    42. Astell AJ, Smith SK, Potter S, Preston-Jones E. Computer interactive reminiscence and conversation aid groups-delivering cognitive stimulation with technology. Alzheimers Dement (N Y) 2018;4:481-487 [FREE Full text] [CrossRef] [Medline]
    43. Joddrell P, Astell AJ. Studies involving people with dementia and touchscreen technology: a literature review. JMIR Rehabil Assist Technol 2016 Nov 4;3(2):e10 [FREE Full text] [CrossRef] [Medline]
    44. Joddrell P, Astell AJ. Implementing accessibility settings in touchscreen apps for people living with dementia. Gerontology 2019 Apr 24:1-11 (epub ahead of print)(forthcoming). [CrossRef] [Medline]
    45. Purves BA, Phinney A, Hulko W, Puurveen G, Astell AJ. Developing CIRCA-BC and exploring the role of the computer as a third participant in conversation. Am J Alzheimers Dis Other Demen 2015 Feb;30(1):101-107. [CrossRef] [Medline]
    46. Schikhof Y, Goumans M, Joddrell P, Astell A. Comparing recent reviews about touchscreens for dementia with lessons learnt from the field. Stud Health Technol Inform 2017;242:1-4. [CrossRef] [Medline]
    47. Xiong C, Astell A, Mihailidis A, Colantonio A. Needs and preferences for technology among Chinese family caregivers of persons with dementia: a pilot study. J Rehabil Assist Technol Eng 2018;5:2055668318775315 [FREE Full text] [CrossRef] [Medline]
    48. Arruda J, Stern R, Hooper C, Wolfner G, Somerville J, Bishop D. Visual analogue mood scales to measure internal mood state in aphasic patients: description and initial validity evidence with normal and neurologically impaired subjects. Arch Clin Neuropsych 1996 Jan 1;11(5):364. [CrossRef]
    49. Elefant AB, Contreras O, Muñoz RF, Bunge EL, Leykin Y. Microinterventions produce immediate but not lasting benefits in mood and distress. Internet Interv 2017 Dec;10:17-22 [FREE Full text] [CrossRef] [Medline]
    50. Gallagher LM, Lagman R, Bates D, Edsall M, Eden P, Janaitis J, et al. Perceptions of family members of palliative medicine and hospice patients who experienced music therapy. Support Care Cancer 2017 Jun;25(6):1769-1778. [CrossRef] [Medline]
    51. Gallagher LM, Lagman R, Rybicki L. Outcomes of music therapy interventions on symptom management in palliative medicine patients. Am J Hosp Palliat Care 2018 Feb;35(2):250-257. [CrossRef] [Medline]
    52. Lorish CD, Maisiak R. The Face Scale: a brief, nonverbal method for assessing patient mood. Arthritis Rheum 1986 Jul;29(7):906-909. [CrossRef] [Medline]
    53. Reilly N, Turner G, Taouk J, Austin MP. 'Singing with your baby': an evaluation of group singing sessions for women admitted to a specialist mother-baby unit. Arch Womens Ment Health 2019 Feb;22(1):123-127. [CrossRef] [Medline]
    54. Shella TA. Art therapy improves mood, and reduces pain and anxiety when offered at bedside during acute hospital treatment. Arts Psychother 2018 Feb;57:59-64. [CrossRef]
    55. Stern RA, Arruda JE, Hooper CR, Wolfner GD, Morey CE. Visual analogue mood scales to measure internal mood state in neurologically impaired patients: description and initial validity evidence. Aphasiology 1997 Jan;11(1):59-71. [CrossRef]
    56. Astell AJ, Ellis MP, Bernardi L, Alm N, Dye R, Gowans G, et al. Using a touch screen computer to support relationships between people with dementia and caregivers. Interact Comput 2010 Jul;22(4):267-275. [CrossRef]
    57. Upton D, Upton P, Jones T, Jutlla K, Brooker D, Grove H. CORE – Aggregating the world's open access research papers. 2011. Evaluation of the Impact of Touch Screen Technology on People With Dementia and Their Carers Within Care Home Settings   URL: https://core.ac.uk/download/pdf/51151130.pdf [accessed 2019-06-16]
    58. Dartigues JF, Foubert-Samier A, le Goff M, Viltard M, Amieva H, Orgogozo JM, et al. Playing board games, cognitive decline and dementia: a French population-based cohort study. BMJ Open 2013 Aug 29;3(8):e002998. [CrossRef]
    59. Särkämö T, Tervaniemi M, Laitinen S, Numminen A, Kurki M, Johnson JK, et al. Cognitive, emotional, and social benefits of regular musical activities in early dementia: randomized controlled study. Gerontologist 2014 Aug;54(4):634-650. [CrossRef] [Medline]
    60. Subramaniam P, Woods B. The impact of individual reminiscence therapy for people with dementia: systematic review. Expert Rev Neurother 2012 May;12(5):545-555. [CrossRef] [Medline]
    61. The Spokesman-Review. 2015. Memory 2.0 Café Uses Tablet, Smartphone Technology to Help Those With Memory Loss   URL: http://www.spokesman.com/stories/2015/oct/19/memory-20-cafe-uses-tablet-smartphone-technology-t/ [accessed 2018-12-05] [WebCite Cache]
    62. York JA. Santa Cruz Sentinel. 2015 May 18. Scotts Valley Workshop Engaging Memories in Seniors   URL: https://www.santacruzsentinel.com/2015/05/18/scotts-valley-workshop-engaging-memories-in-seniors/ [accessed 2018-12-08]
    63. Nuffield Bioethics. 2009. Dementia: Ethical Issues   URL: http://nuffieldbioethics.org/wp-content/uploads/2014/07/Dementia-report-Oct-09.pdf [accessed 2019-06-10]
    64. Temple RO, Stern RA, Latham J, Ruffolo JS, Arruda JE, Tremont G. Assessment of mood state in dementia by use of the visual analog mood scales (VAMS). Am J Geriatr Psychiatry 2004;12(5):527-530. [CrossRef] [Medline]
    65. Landes AM, Sperry SD, Strauss ME, Geldmacher DS. Apathy in Alzheimer's disease. J Am Geriatr Soc 2001 Dec;49(12):1700-1707. [CrossRef] [Medline]
    66. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
    67. Bleakley CM, Charles D, Porter-Armstrong A, McNeill MD, McDonough SM, McCormack B. Gaming for health: a systematic review of the physical and cognitive effects of interactive computer games in older adults. J Appl Gerontol 2015 Apr;34(3):NP166-NP189. [CrossRef] [Medline]

    ‎
    BPSD: behavioral and psychological symptoms of dementia
    ICT: information and communication technology
    SNF: skilled nursing facility
    VAMS: visual analog mood scale

    Edited by G Eysenbach; submitted 30.04.19; peer-reviewed by J Delello, E Dove, C Kunze; comments to author 15.05.19; revised version received 09.07.19; accepted 28.07.19; published 03.09.19

    Copyright

    ©Aaron Gilson, Debby Dodds, Arveen Kaur, Michael Potteiger, James H Ford II. Originally published in JMIR Formative Research (http://formative.jmir.org), 03.09.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.