Background: Anxiety and mood problems in adolescents often go unnoticed and may therefore remain untreated. Identifying and preventing the development of emotional problems requires monitoring and effective tools to strengthen adolescents' resilience, for example, by enhancing coping skills.
Objective: This study describes the developmental process, feasibility, and acceptance of Grow It!, a multiplayer serious game app for adolescents aged 12-25 years. The app consists of the experience sampling method (ESM) to monitor thoughts, behaviors, and emotions in daily life to enhance self-insight and daily cognitive behavioral therapy–based challenges to promote adaptive coping.
Methods: Our approach entails an iterative game design process combined with an agile method to develop the smartphone app. The incorporated game features (ie, challenges, chat functionality, and visual representation) in the Grow It! app were co-designed with adolescent end users to increase participant engagement and adherence.
Results: The Grow It! app was delivered for Android and iOS in May 2020. Grow It! was offered to adolescents during the COVID-19 crisis between May and December 2020. Participants of the Grow It! COVID-19 study (sample 1: N=685; mean age 16.19, SD 3.11 years; 193/685, 28.2% boys; sample 2: N=1035; mean age 18.78, SD 3.51 years; 193/1035, 18.64% boys) completed 31.5% (13.2/42) to 49.5% (10.4/21) of challenges. Compliance of ESM was suboptimal (35.1/210, 16.7% to 32.5/105, 30.9%). Follow-up questionnaires indicated an overall score of the app of 7.1 out of 10. Moreover, 72.6% (278/383) to 75.6% (487/644) would recommend the app to friends.
Conclusions: To our knowledge, Grow It! is the first gamified ESM app that both measures individual differences in emotional dynamics and offers an integrated cognitive behavioral therapy–based intervention. Our findings support the feasibility and acceptance, and therefore applicability, of the Grow It! app in adolescents. Further iterations of this serious game app will focus on the increase of compliance and on providing participants feedback through their personal mood profiles.
Internalizing problems, such as anxiety and mood problems, have a substantial impact on young people’s lives. These internalizing problems are often associated with school dropout, reduced social functioning, loneliness, unemployment, and reduced quality of life [- ]. Anxiety and mood disorders usually begin in adolescence [ , ], and unfortunately, they often go unnoticed or untreated [ ]. When persistent, internalizing problems often result in emerging psychiatric disorders affecting young people’s daily lives, their future, and society [ - , ]. Therefore, early identification and timely intervention are crucial to prevent further deterioration, improve prognosis, and reduce the burden on health care systems and society in general [ , , ].
Mobile health (mHealth) can play an important role in accurate recognition of symptoms and timely treatment [- ]. mHealth is defined as wireless technologies, such as smartphone apps, to support or achieve health objectives. In terms of its advantages, first of all, mHealth is scalable, accessible, and maybe less stigmatizing than traditional treatment for youths because of the level of anonymity and privacy [ ]. Furthermore, mHealth offers the possibility of incorporating motivational elements such as playfulness and gamification, which is advantageous because humans supposedly learn best by playing [ - ]. Finally, mHealth offered in an attractive and fun way through adolescents’ own devices fits very well with their daily life and activities [ , ]. Use is flexible, as it is independent of time and place and can be at a self-determined pace, which is thought to enhance self-efficacy [ ]. Indeed, most adolescents indicated that they would use an app to screen for emotional problems and treatment if available [ ]. Even though there are already several mHealth apps [ ], preventive mHealth apps that integrate novel methods for early identification and preventive intervention are still lacking.
A promising method for the early identification of emotional problems is the experience sampling method (ESM). The ESM is a structured diary method in which participants obtain multiple random notifications on their phone during the day. When a notification pops up, they fill out a microquestionnaire in which they report on their behaviors, thoughts, and feelings in real time (eg, how are you doing right now?). The strength of the ESM is the high ecological validity. Moment-to-moment assessments in real-world settings address the problem of recall bias [, ]. The promise of ESM data for early identification of adolescents at risk for the development of psychopathology has been demonstrated in a research setting with adults [ ] and adolescents [ ]. Moreover, self-management may be enhanced by obtaining insights into everyday functioning dynamics, based on ESM data [ , , ].
The first choice of treatment of anxiety and mood problems in adolescents is psychological therapy, especially cognitive behavioral therapy (CBT) [- ]. It is known that the way adolescents cope with stress or handle negative emotions in daily life may increase or buffer against the development of anxiety and depressive symptoms [ - ]. That is why CBT is one of the effective interventions aimed at improving coping. Recently, results of an internet-based CBT intervention revealed that positive effects occur already after 4 weeks of CBT [ ]. CBT is mostly used in clinical practice but increasingly also applied for preventive purposes [ , ].
Therefore, we cocreated the Grow It! app (Android and iOS) for adolescents aged 12 to 25 years. Grow It! is a multiplayer serious game, a game that is designed for a primary purpose other than entertainment . Incorporated in the Grow It! app are ESM, to enhance self-management and identify mood problems earlier on, and gamified CBT-based challenges, to increase coping. Initially, the app was developed for high-risk adolescent populations, such as adolescents with chronic somatic conditions, offspring of parents with psychiatric disorders, or adolescents experiencing extreme stressful societal circumstances, for example, the COVID-19 pandemic. However, the app may also serve a broader purpose of prevention for adolescents from a general population.
The first aim of this study is to give an elaborate description of the developmental process of the multiplayer serious game app Grow It! Second, we aim to study the feasibility and acceptance of Grow It! among end users.
The Grow It! app was developed, with intermittent periods, from March 2016 to February 2020 (). During its development, we cocreated the app with a large multidisciplinary team of child and adolescent psychiatrists, developmental and clinical psychologists, data analysts, game designers, and multiple test panels (adolescents aged 12-25 years) [ , ]. The initial concept was developed with University of the Arts Utrecht (test 1a-1b). With IJsfontein BV and consultancy company Game Architect Studio, an agile process (defined as a software development methodology including iterative development, where requirements and solutions evolve in multidisciplinary teams [ ]) then was used to develop and evaluate the minimal viable product (MVP; test 2a-2e). For all tests, informed consent was obtained.
In March 2016, the developmental process of Grow It! started with a prepilot (test 1a, N=10), which resulted in a paper-based prototype and wireframes. Later on, in the pilot (test 1b), adolescents from the general population (N=21) tested the app’s beta version during a 6-week trial. Weeks 1 and 6 of the test consisted of an ESM-only study with 8 mood assessments per day. During weeks 2 to 5, the mood assessments were given twice a day and combined with CBT-based challenges. In all weeks, users received feedback through push messages complimenting them. Lessons learned were that adolescents were motivated by the game mechanics of Grow It! and liked completing the ESM questionnaires and daily challenges.
Minimal Viable Product
In June 2019, an MVP was built. We aimed to improve the app's content, visual design, interaction design, and reliability of assessments and ran a technical test. The app was developed using agile development and user-centered design methods, including different tests and collaboration with focus groups. Different groups of adolescents (n=6 and n=9) received instructions and were invited to design CBT-based challenges aiming at adaptive coping . Thereafter, all ideas were formulated into specific challenges and were rated in terms of their clinical appropriateness and coping effectiveness by 11 child and youth psychologists and psychiatrists. As a result of these focus groups (test 2a), 126 challenges were formulated, which were later used as the challenges in the Grow It! app. Furthermore, a survey among 107 adolescents (test 2b) resulted in the choice for a visual design that (1) was accepted by a broad range of ages and by both boys and girls and (2) was low cost in maintenance. In interviews (N=4, test 2c) with regard to the navigation of the app, special attention was given to the answer-scale development (Likert and visual analog scale) to increase the assessments' reliability. In empirical studies on ESM data [ ], visual analog scales had demonstrated the anchor-influenced results, and therefore, different approaches were tested (eg, add a cursor above the anchor line). Interviews with adolescents indicated that Likert scales are more intuitive if they run from high to low on the screen instead of low to high. In December 2019 and February 2020, user experience tests of the Grow It! app were run (test 2d). Adolescents (N=23) played the Grow It! app for 1 week. On the basis of interviews, we improved and extended the content of the app by (1) adding a limited chat function with predesigned stickers to motivate team members, (2) allowing users to choose from 3 challenges a day for 6 weeks, and (3) adding a tour in which game mechanics are explained. Finally, a handbook for errors arose from our technical test and quality assessment, which was performed by the research team (N=10, test 2e).
Feasibility and Acceptance Test
In May 2020 and in December 2020, at the first 2 peaks of the COVID-19 pandemic, the Grow It! app was launched to assess the game mechanics and user acceptance of the MVP. Owing to government restrictions, all adolescents had to follow social distancing measures (eg, staying at home because schools were closed). Through (social) media, the app was made available to Dutch-speaking adolescents living in the Netherlands, aged 12-25 years, who owned a smartphone. Participants were consecutively enrolled in a Grow It! team after completion of the baseline questionnaire that was linked to the web-based informed consent procedure on a secure website. This way, participants started with the app as soon as possible.
In total, 685 adolescents (sample 1: mean age 16.19, SD 3.11 years; 193/685, 28.2% boys) played the Grow It! app for 6 weeks, and in the second sample, another 1035 adolescents (sample 2: mean age 18.78, SD 3.51 years; 193/1035, 18.64% boys) played the Grow It! app for 3 weeks. A follow-up questionnaire was filled out by 383 and 644 adolescents for samples 1 and 2, respectively (seefor demographics). In the Grow It! app, participants were given 5 ESM notifications per day and daily challenges. Users who did not show activity in the app (0 or 1 activity in ESM or challenges) were excluded from the sample, because to evaluate the user experience of the Grow It! app we were interested in participants who were involved in playing the app.
A complete overview of all (ESM) instruments and questionnaires is provided in our internet-based codebook . The outcomes of our feasibility and acceptance test can be found in the Results section. Statistical analyses are descriptive (means, SDs, and frequencies) and performed using SPSS (version 25; IBM Corp) [ ].
|Sample 1||Sample 2|
|App engagement (Grow It! activity; N=685)||Follow-up questionnaire (n=383; 55.9% retention)||App engagement (Grow It! activity; N=1035)||Follow-up questionnaire (n=644; 62.2% retention)|
|Age (years), mean (SD)||16.19 (3.11)||16.26 (3.07)||18.78 (3.51)||18.48 (3.43)|
|Gender, n (% boys)||193 (28.2)||100 (26)||193 (18.6)||120 (18.7)|
|Educationlevela, n (%)|
|Primary school||30 (4.4)||9 (2.3)||9 (0.9)||6 (1)|
|Low||104 (15.2)||54 (14.1)||167 (16.1)||98 (15.2)|
|Medium||152 (22.1)||92 (24.1)||337 (32.6)||201 (31.2)|
|High||399 (58.2)||228 (59.5)||438 (42.3)||293 (45.5)|
|Other||N/Ab||N/A||84 (8.1)||46 (7.1)|
|Cultural identity, n (%)|
|Dutch||622 (90.8)||348 (90.7)||1013 (97.9)||631 (98.1)|
|Mixed||57 (8.3)||4 (1.1)||17 (1.6)||11 (1.7)|
|Other||6 (0.9)||31 (8.1)||5 (0.6)||2 (0.3)|
aLow: (preparatory school for) technical and vocational training; medium: (preparatory school for) professional education; and high: (preparatory school for) university.
bN/A: not applicable.
End Product of Developmental Process: The Grow It! App
The user journey first entails a phase of enrollment, during which participants can personalize their account. After receiving a 6-digit code (letters and numbers) from the research team via SMS text messaging, they log in to Grow It! and choose their nickname based on 2 turntables. The first turntable shows an adjective (eg, Adorable, Dangerous, Lucky, Creative, and Romantic), and the second turntable shows an animal name (eg, Alpaca, Snake, Iguana, Rabbit, and Crocodile). Participants can rotate the turntables as often as they want to personalize their nickname. For example, one participant nickname could be Lucky Rabbit or Adorable Alpaca (). The game mechanics (ie, personalization, collaboration, competition, and feedback) are explained in the mandatory tour of the Grow It! app.
As adolescents are sensitive to peer influence and can be motivated by interactions with peers [- ], each participant collaborates anonymously in a team with 3 to 7 other players. Adolescents are allocated to a team by the researchers. To support team members, participants can chat by sending and receiving positive stickers ( ). Via this chat system, participants can motivate each other, while the system minimizes the possibilities of bullying and negative peer pressure.
Competition is encouraged at the team level, where teams play versus each other. Each team has a virtual tree with a name (eg, Oaks, Pines, and Palms), which allows participants to compare their team performance with that of other teams (). At the start of the game, the tree is empty. Teams grow their tree when participants receive points by reporting their feelings and behaviors (ESM) and doing daily challenges. These reports are personal and shielded. Team members only see the amount of points their teammates have collected. After a team collects a specified amount of points, they achieve a spurt (ie, level-up), which means that the tree grows in height, and every team member receives a gift to embellish the tree ( and ). As game mechanics, these provide a positive feedback loop and a progress update and establish the reward scheme or the behavioral conditioning that increases retention. Upon earning a gift (ie, loot box) from a growth spurt, a participant can then select his or her choice from 3 gifts. The gifts are wrapped so that there is no indication of what is inside. As a game mechanic, selecting a random gift creates surprise and moments of anticipation essential to maintaining a state of play [ ]. When teams have just started using the app, it does not take many points to achieve the first spurt and earn a gift. As the game progresses, however, and teams move to higher levels, more and more points are needed to earn a spurt and gifts. In this way, adolescents are stimulated to keep playing and remain engaged with the app. The difficulty level is scaffolded by incrementally increasing difficulty, which supports retention by continuously challenging the participants as they progress through the game.
The game mechanics of Grow It! provide feedback at different levels. Whereas users can see their own and their team members’ scores in the score overview screen (), on their profile page, they obtain an overview of how many times they reported their feelings, behaviors, and challenges that day ( ). Finally, the Grow It! app has a contact button in case of technical issues or urgent psychological problems. Whenever a participant pushes the contact button, a phone number is displayed through which the research team can be reached by telephone or texting on working days during office hours. On the study website, information can be found on how to reach help in acute situations or outside working hours, referring to professional and free services.
The ESM is an integral part of the app, which can be used for early identification of emotional problems and enhancing self-management in adolescents . To prompt adolescents to report on their feelings, they receive several notifications per day, which are randomized to prevent structural answering patterns (eg, always in math class at 11 AM) [ ]. In the first studies, adolescents answered 5 microquestionnaires per day (taking approximately 1-2 minutes) regarding their sleep, activity, affective well-being (eg, I feel happy or sad), coping strategies, pain, fatigue, social behavior, loneliness, stress, and coping ( ).
Our ESM approach's novelty is that it is gamified to increase motivation, which intends to result in a higher rate of compliance (percentage completed self-evaluations) as well as improved data quality . Compliance is one of the critical quality markers for ESM studies [ ].
To teach adolescents how to cope with setbacks and to promote emotional resilience, the Grow It! app contains daily challenges aimed at strengthening adaptive coping, supporting physical activation, and preventing emotional problems. Coping styles incorporated in the challenges promote distraction, problem solving, social support, and acceptance [, ]. Participants can choose 1 out of 3 challenges per day ( ). Challenges are divided into three categories: photo challenges, quizzes, and assignments. Examples of challenges are as follows: make a picture of something you hold dear (photo challenge; aimed at distraction), ask someone what they like about you and write it down (assignment; aimed at social support), or answer a multiple-choice question such as What is sushi usually rolled in? (quiz; aimed at problem solving). An additional randomly available assignment is the photo check. Participants are shown a matrix of 9 photos and assess which photos fit a particular theme. In this way, they act as the photo challenge jury and can award points to participants and earn extra points themselves for this task.
Ethical Considerations and Privacy
Risks related to privacy were mitigated by making all participants pseudonymous and only identifiable in the app by participation codes and pseudonyms. Participants determine their pseudonym (ie, nickname) from a preselected set of words provided by the app. Participants cannot be identified, and only the research team has insight into the private data of the participants. Data collected with the app pertain to the user's game data (eg, game-specific actions) and responses to the ESM. The app also accesses the mobile device's camera, but only when a participant takes a photo for a challenge (not at other times), and the app does not access other functionalities (eg, Google, GPS, or health apps). All user data are encrypted and sent directly to a secure server at the researcher’s institute. The privacy and security of the Grow It! app is approved by the privacy and security office of Erasmus Medical Center, and the app complies with the Dutch General Data Protection Regulation (Algemene Verordening Gegevensbescherming) and NEN-norm 7510:2017 (Dutch standard of information security management systems in health care). The app is available for research purposes in the Google Play store  and Apple store [ ]. This study has been approved by the Medical Ethics Committee of the Erasmus Medical Center (MEC-2020-0287).
As shown in, participants of sample 1 (N=685; mean age 16.2, SD 3.1 years) were somewhat younger than adolescents in sample 2 (N=1035; mean age 18.8, SD 3.5 years). In general, participants of both samples were relatively highly educated and mostly of Dutch ethnicity. The follow-up questionnaire was filled out by 55.9% (383/685) and 62.2% (644/1035) of the users in sample 1 and sample 2, respectively.
Feasibility and Acceptance
An overview of app engagement (ESM compliance and challenges) and answers of the user evaluation questionnaire carried out in the follow-up can be found inand .
Regarding the ESM component, overall compliance was 16.7% (35.1/210 notifications, sample 1) and 30.9% (32.5/105 notifications, sample 2). About 56.77% (583/1027) of the participants indicated that they thought the number of questions per day was too high, whereas about 40.7% (418/1027) indicated the number of questions as sufficient. Some participants also reported that they did not understand why the same questions were asked repeatedly (sample 1: N=20, sample 2: N=16; ie, at each notification, the same ESM questions were asked to monitor their feelings and behavior over the study weeks). Moreover, 15% (97/644) to 22.9% (88/383) reported no effect of the ESM. However, 66.8% (256/383) to 72.4% (466/644) reported reflecting more on their feelings as a result of the ESM.
With regard to the daily CBT-based challenges, participants completed 31.5% (216/685) to 49.47% (512/1035) of all challenges. Whereas in sample 1, a total of 1.2% (8/685) participants completed all 42 challenges (100%), 6.57% (68/1035) participants completed all 21 challenges (21/21, 100%) in sample 2. The self-reported effect of the challenges showed that 20.6% (79/383) to 44.2% (285/644) of the participants became more active as a result of the Grow It! challenges.
The overall user evaluation of Grow It! was positive. The average app evaluation score was 7.1 of 10 (SD 1.5) in sample 1 and 7.2 of 10 (SD 1.3) in sample 2. Moreover, the app’s design was evaluated with a score of 7.7 to 8.0 of 10. Finally, 72.6% (278/383) to 75.6% (487/644) would recommend the app to their friends.
|Total||Aged 12-17 years||Aged 18-25 years||Difference test (aged 12-17 vs 18-25 years)|
|t test (df)||Chi-square (df)||P value|
|App engagement (Grow It! activity)||N/Aa|
|Number of users||685||500||185||N/A||N/A|
|Compliance of ESMb (n=210), number of notifications (%)||35.1 (16.7)||34.4 (16.4)||41.8 (19.9)||1.89 (683)||.06|
|Challenges (n=42), n (%)||13.2 (31.5)||13.3 (31.6)||13.1 (31.2)||0.19 (683)||.85|
|User evaluation (follow-up questionnaire)||N/A|
|Number of users||383||273||110||N/A||N/A|
|Evaluation of the app (1-10), mean (SD)||7.1 (1.5)||7.4 (1.3)||6.6 (1.7)||4.70 (381)||<.001|
|Evaluation of the design (1-10), mean (SD)||7.7 (1.5)||7.8 (1.5)||7.5 (1.5)||2.21 (381)||.03|
|Self-reported effect of ESM, n (%)||N/A||1.99 (1)||.58|
|I got to know myself better||20 (5.3)||17 (6.3)||3 (2.9)|
|It made me feel better||19 (5)||13 (4.7)||6 (5.7)|
|It made me think about how I feel more||256 (66.8)||181 (66.4)||75 (67.6)|
|No effect||88 (22.9)||62 (22.5)||26 (23.8)|
|Evaluation amount of ESM per day, n (%)||N/A||9.18 (1)||.002|
|Few||15 (4)||8 (3)||7 (6.5)|
|Sufficient||155 (40.3)||122 (44.8)||32 (29)|
|A lot||213 (55.7)||143 (52.2)||71 (64.5)|
|Self-reported effect of challenges, n (%)||N/A||5.26 (1)||.26|
|I have had more contact with others||10 (2.5)||6 (2.3)||3 (2.9)|
|I am better at solving problems||8 (2.2)||4 (1.5)||4 (3.8)|
|I am better at accepting situations||32 (8.3)||22 (8.1)||10 (8.7)|
|I have become more active||79 (20.7)||64 (23.6)||15 (13.5)|
|I have started to feel less lonely||28 (7.4)||20 (7.3)||8 (7.7)|
|Evaluation chat function, n (%)||N/A||5.79 (1)||.22|
|Not nice at all||79 (20.7)||51 (18.7)||28 (25.5)|
|A little bit nice||175 (45.7)||122 (44.7)||53 (48.1)|
|Quite nice||54 (14.1)||41 (14.9)||14 (13.3)|
|Nice||45 (11.7)||33 (12.2)||11 (10.4)|
|Very nice||30 (7.9)||26 (9.5)||4 (3.8)|
|Would recommend Grow It! to friends, n (%)||278 (72.6)||212 (77.7)||66 (60)||N/A||12.28 (1)||<.001|
|Want to help with future development of the app, n (%)||133 (34.7)||93 (33.9)||41 (36.9)||N/A||0.35 (1)||.55|
aN/A: not applicable.
bESM: experience sampling method.
|Total||Aged 12-17 years||Aged 18-25 years||Difference test (aged 12-17 vs 18-25 years)|
|t test (df)||Chi-square (df)||P value|
|App engagement (Grow It! activity)||N/Aa|
|Number of users||1035||405||630||N/A||N/A|
|Compliance of ESMb (n=105), number of notifications (%)||32.5 (30.9)||30 (28.6)||33.9 (32.3)||2.17 (1033)||.03|
|Challenges (n=21), n (%)||10.4 (49.5)||10.5 (50.1)||10.3 (49.2)||0.45 (1033)||.65|
|User evaluation (follow-up questionnaire)||N/A|
|Number of users||644||256||388||N/A||N/A|
|Evaluation of the app (1-10), mean (SD)||7.2 (1.3)||7.5 (1.3)||6.9 (1.3)||1.59 (624)||.11|
|Evaluation of the design (1-10), mean (SD)||8.0 (1.3)||8.2 (1.3)||7.8 (1.3)||4.48 (624)||<.001|
|Self-reported effect of ESM, n (%)||N/A||8.54 (1)||.003|
|I got to know myself better||62 (9.6)||27 (10.4)||35 (9)|
|It made me feel better||19 (3)||13 (5.2)||6 (1.5)|
|It made me think about how I feel more||466 (72.4)||174 (68.0)||292 (75.3)|
|No effect||97 (15)||42 (16.4)||55 (14.2)|
|Evaluation amount of ESM per day, n (%)||N/A||5.77 (1)||.06|
|Few||11 (1.7)||5 (1.8)||7 (1.7)|
|Sufficient||263 (40.9)||119 (46.5)||144 (37.2)|
|A lot||370 (57.4)||132 (51.6)||237 (61.2)|
|Self-reported effect of challenges, n (%)||N/A||10.47 (1)||.001|
|I have had more contact with others||106 (16.5)||31 (12)||78 (20)|
|I am better at solving problems||29 (4.5)||16 (6.3)||12 (3)|
|I am better at accepting situations||120 (18.7)||47 (18.3)||74 (19)|
|I have become more active||285 (44.2)||121 (47.2)||162 (41.7)|
|I have started to feel less lonely||104 (16.1)||41 (16.2)||62 (16.1)|
|Evaluation chat function, n (%)||N/A||28.11 (1)||<.001|
|Not nice at all||142 (22)||35 (13.5)||107 (27.6)|
|A little bit nice||288 (44.7)||109 (42.7)||179 (46)|
|Quite nice||106 (16.4)||54 (21.2)||51 (13.2)|
|Nice||77 (12)||41 (16.1)||36 (9.4)|
|Very nice||31 (4.9)||17 (6.6)||15 (3.8)|
|Would recommend Grow It! To friends, n (%)||487 (75.6)||215 (84)||272 (70)||N/A||16.12 (1)||<.001|
|Want to help with future development of the app, n (%)||263 (40.9)||117 (45.7)||146 (37.7)||N/A||0.74 (1)||.39|
aN/A: not applicable.
bESM: experience sampling method.
Results for Adolescents Compared With Emerging Adults
Given the broad age range in which adolescents participated (12-25 years), we also reported outcomes separately for adolescents (12-17 years) and emerging adults (18-25 years) inand . Higher user evaluations of the Grow It! app were found in the adolescent group in comparison with the emerging adult group (ie, samples 1 and 2: adolescents rated the design of the app higher, and more adolescents would recommend the app to their friends in samples 1 and 2; sample 1: adolescents’ evaluation of the app was higher, and number of ESM notifications per day was evaluated better; sample 2: evaluation of the chat function was better in adolescents). Moreover, no differences between age groups were found with regard to app engagement with the exception of slightly higher compliance of ESM in emerging adults than in adolescents (only in sample 1).
The Grow It! app emerged from a lack of and need for preventive interventions for emotional problems and promoting adaptive coping for adolescents; in addition, these interventions need to be low key, nonstigmatizing, fun, attractive, private, and secure. On the basis of the developmental process and acceptance and feasibility of Grow It!, key lessons learned and directions for future research are formulated and shared. Our approach entails an iterative game design process combined with an agile method to develop the smartphone app. The incorporated game features in the Grow It! app were co-designed with adolescent end users to increase participant engagement and adherence. With regard to the app engagement and user evaluation filled out at follow-up, we indicated that we have some evidence that supports the feasibility and acceptance, and therefore applicability, of Grow It! in adolescents.
Earlier studies suggest that adolescents are open to using mHealth . Indeed, the large interest as well as the positive user evaluation provide reason to believe that adolescents, at least a large group, are positive and open to using mHealth. Grow It was co-designed with youths, which was reflected in age-adequate daily challenges and an overall positive rating of the Grow It! app. Moreover, 66.8% (256/383) to 72.4% (466/644) of the participants felt they reflected more on their emotions.
Although the results of the developmental process, acceptance, and feasibility are informative and promising for mHealth, several limitations should be mentioned. First, the majority of the study sample consisted of girls. The design of the app seems more appealing to girls, despite the long process of cocreation with both boys and girls. One of the explanations might be that girls are more inclined to seek help  and therefore are also more inclined to participate in our study.
Second, the COVID-19 pandemic and related governmental restrictions might have influenced the results. Owing to remote working during the COVID-19 pandemic, participants had no or minimal personal contact and received minimal instructions on how to participate in an ESM study, although this is stated as an important factor to obtain reliable data . Third, concerning self-monitoring, it is notoriously difficult to motivate adolescents. In this study, compliance of ESM was lower than in a typical research design [ ]; however, participants received no financial incentive and were entirely motivated by the game structure to answer questionnaires. We, therefore, expect that with more targeted use (eg, in blended care when the app is explained by a professional), the compliance and user satisfaction might be more favorable. Furthermore, the dropout rates were comparable with other web-based studies [ ]. In order to increase app engagement, improvements are needed; for instance, designs may need to be tailored to the individual (personalization) and (in-person) feedback is needed.
With regard to the early identification of emotional problems, ESM data of the Grow It! app provides an opportunity to develop algorithms in future research for the early detection of emotional problems, which often go unnoticed in adolescents . Identifying emotional problems early on would require capitalizing on novel developments in clinical psychology [ ] combined with the motivational game architecture codeveloped with youths [ ]. The ESM has already shown to be a reliable method to investigate variation in thoughts, feelings, and symptoms over time and context in research settings [ , ]. Dietvorst et al [ ] have demonstrated that ESM data helps to identify the onset of depressive feelings among adolescents 3 months ahead. Specifically, this was done by differentiating typical adolescents (eg, grumpy at home), from early depressive feelings. In future work, analyzing highly rich ESM data with more powerful analytical techniques, such as machine learning, could potentially improve this early identification.
In clinical practice, self-management and self-insight may be enhanced by obtaining insights into one's emotion dynamics [, , ]. A feature such as providing participants with feedback through a daily life emotion chart (eg, mood profile) could provide participants with better insight and feedback into their well-being [ , ]. It may also serve as a therapeutic function, as integrating real time mood profiles in the app could encourage adolescents to reflect more on their emotions, coping, and behavior in different contexts [ ] and could also be used as a routine outcome measure during therapy. To test the effects of the app upon adolescent well-being and resilience, an additional in-depth evaluation is required. Research questions and hypotheses that are beyond the scope of the developmental process focusing on the main effect of the app are preregistered [ , ] and will be executed in the future accordingly, including a randomized controlled trial study to test the effectiveness of the Grow It! app.
The Grow It! app has been developed and improved through iterations in collaboration with a large multidisciplinary team. It is innovative, age-attuned, easily accessible, fun, and visually appealing and, most importantly, serves the needs of adolescents. The app was well received by adolescents, and the first findings presented here indicate that adolescents were motivated by the game mechanics of Grow It! and liked completing the ESM questionnaires and daily challenges. Initially, the app was developed for high-risk adolescent populations, such as adolescents with chronic somatic conditions, offspring of parents with psychiatric disorders, or adolescents experiencing extreme stressful societal circumstances, for example, the COVID-19 pandemic. However, the app may also serve a broader purpose of prevention for adolescents from a general population. Our findings support the feasibility and acceptance, and therefore applicability, of the Grow It! app in adolescents.
The ambition is to further improve the app after each research study by including new features and resolving usability issues. The next step will be to focus on the increase of compliance and providing participants feedback through their personal mood profiles.
This research was funded by the Netherlands Organization for Health Research and Development Juiste Zorg Op De Juiste Plek (ZonMw JZOJP) together with the Netherlands Organisation for Scientific Research (Nederlandse Organisatie voor Wetenschappelijk Onderzoek; project 440.20.006), the Stichting Vrienden van het Sophia (project B18-05), Gemeente Rotterdam (projects SUB.20.03.00149.SBSA and 40258362), and the Dutch National Research Agenda (Nationale WetenschapsAgenda/eHealth Junior consortium; project 1292.19.226).
Conflicts of Interest
- Thapar A, Collishaw S, Pine DS, Thapar AK. Depression in adolescence. The Lancet 2012 Mar;379(9820):1056-1067. [CrossRef]
- Cummings CM, Caporino NE, Kendall PC. Comorbidity of anxiety and depression in children and adolescents: 20 years after. Psychol Bull 2014 May;140(3):816-845 [FREE Full text] [CrossRef] [Medline]
- Adolescent mental health. World Health Organization. URL: https://www.who.int/news-room/fact-sheets/detail/adolescent-mental-health [accessed 2022-01-27]
- Garber J, Keiley MK, Martin NC. Developmental trajectories of adolescents' depressive symptoms: predictors of change. J Consult Clin Psychol 2002 Feb;70(1):79-95. [CrossRef] [Medline]
- Ormel J, Raven D, van Oort F, Hartman CA, Reijneveld SA, Veenstra R, et al. Mental health in Dutch adolescents: a TRAILS report on prevalence, severity, age of onset, continuity and co-morbidity of DSM disorders. Psychol Med 2015 Jan;45(2):345-360. [CrossRef] [Medline]
- Kessler RC, Angermeyer M, Anthony JC, Graaf RDE, Demyttenaere K, Gasquet I, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry 2007 Oct;6(3):168-176. [Medline]
- Leijdesdorff SM, Huijs CE, Klaassen RM, Popma A, van Amelsvoort TA, Evers SM. Burden of mental health problems: quality of life and cost-of-illness in youth consulting Dutch walk-in youth health centres. J Ment Health 2020 Oct 22:1-8. [CrossRef] [Medline]
- Raven D, Jörg F, Visser E, Oldehinkel AJ, Schoevers RA. Time-to-treatment of mental disorders in a community sample of Dutch adolescents. A TRAILS study. Epidemiol Psychiatr Sci 2017 Apr;26(2):177-188 [FREE Full text] [CrossRef] [Medline]
- Eijlers R, Legerstee JS, Dierckx B, Staals LM, Berghmans J, van der Schroeff MP, et al. Development of a virtual reality exposure tool as psychological preparation for elective pediatric day care surgery: methodological approach for a randomized controlled trial. JMIR Res Protoc 2017 Sep 11;6(9):e174 [FREE Full text] [CrossRef] [Medline]
- Granic I, Lobel A, Engels RC. The benefits of playing video games. Am Psychol 2014 Jan;69(1):66-78. [CrossRef] [Medline]
- Watts SE, Andrews G. Internet access is NOT restricted globally to high income countries: so why are evidenced based prevention and treatment programs for mental disorders so rare? Asian J Psychiatr 2014 Aug;10:71-74. [CrossRef] [Medline]
- Schröder J, Berger T, Westermann S, Klein JP, Moritz S. Internet interventions for depression: new developments. Dialogues Clin Neurosci 2016 Jun;18(2):203-212. [Medline]
- Ehrmann J, Lewis C, Lewis P. Homo ludens revisited. Yale French Stud 1968(41):31. [CrossRef]
- What can we learn from playing interactive games. In: Playing Video Games Motives, Responses, and Consequences. New York: Routledge; 2006.
- van Berkel N, Goncalves J, Hosio S, Kostakos V. Gamification of mobile experience sampling improves data quality and quantity. Proc ACM Interact Mob Wearable Ubiquitous Technol 2017 Sep 11;1(3):1-21. [CrossRef]
- Brown S, Vaughan C. Play: How it Shapes the Brain, Opens the Imagination, and Invigorates the Soul. New York: Avery; 2010.
- McGonigal J. Reality is Broken Why Games Make Us Better and How They Can Change the World. New York: Random House; 2011.
- Visch VT, Vegt NJ, Anderiesen H, van der Kooij K. Persuasive game design: a model and its definitions. In: Proceedings of the CHI2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences. 2013 Presented at: CHI2013 Workshop on Designing Gamification: Creating Gameful and Playful Experiences; Apr 27-May 2, 2013; Paris, France.
- Keijsers L, Van Roekel E. Longitudinal methods in adolescent psychology. In: Reframing Adolescent Research. Oxfordshire, England, UK: Routledge; 2018.
- Proudfoot J, Parker G, Hadzi Pavlovic D, Manicavasagar V, Adler E, Whitton A. Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress. J Med Internet Res 2010 Dec 19;12(5):e64 [FREE Full text] [CrossRef] [Medline]
- Gindidis S, Stewart S, Roodenburg J. A systematic scoping review of adolescent mental health treatment using mobile apps. Adv Ment Health 2018 Sep 28;17(2):161-177. [CrossRef]
- van Roekel E, Keijsers L, Chung JM. A review of current ambulatory assessment studies in adolescent samples and practical recommendations. J Res Adolesc 2019 Sep 17;29(3):560-577 [FREE Full text] [CrossRef] [Medline]
- Larson R, Csikszentmihalyi M. The experience sampling method. In: Flow and the Foundations of Positive Psychology. Dordrecht: Springer; 2014.
- Wichers M, Smit AC, Snippe E. Early warning signals based on momentary affect dynamics can expose nearby transitions in depression: a confirmatory single-subject time-series study. J Pers Oriented Res 2020;6(1):1-15 [FREE Full text] [CrossRef] [Medline]
- Dietvorst E, Hiemstra M, Maciejewski D, van Roekel E, Bogt TF, Hillegers MH, et al. Grumpy or depressed? Disentangling typically developing adolescent mood from prodromal depression using experience sampling methods. J Adolesc 2021 Apr;88:25-35 [FREE Full text] [CrossRef] [Medline]
- van Os J, Verhagen S, Marsman A, Peeters F, Bak M, Marcelis M, ESM-MERGE Investigators PhD, et al. The experience sampling method as an mHealth tool to support self-monitoring, self-insight, and personalized health care in clinical practice. Depress Anxiety 2017 Jun;34(6):481-493. [CrossRef] [Medline]
- Widdershoven RL, Wichers M, Kuppens P, Hartmann JA, Menne-Lothmann C, Simons CJ, et al. Effect of self-monitoring through experience sampling on emotion differentiation in depression. J Affect Disord 2019 Feb 01;244:71-77. [CrossRef] [Medline]
- Havinga PJ, Maciejewski DF, Hartman CA, Hillegers MH, Schoevers RA, Penninx BW. Prevention programmes for children of parents with a mood/anxiety disorder: systematic review of existing programmes and meta-analysis of their efficacy. Br J Clin Psychol 2021 Jun;60(2):212-251 [FREE Full text] [CrossRef] [Medline]
- National Collaborating Centre for Mental Health. Depression: The NICE Guideline on the Treatment and Management of Depression in Adults. London, England: The British Psychological Society & The Royal College of Psychiatrists; 2010.
- Djaouti D, Alvarez J, Jessel JP. Classifying serious games: the G/P/S model. In: Handbook of Research on Improving Learning and Motivation Through Educational Games: Multidisciplinary Approaches. Pennsylvania, USA: IGI Global; 2011.
- Cohen S, Wills TA. Stress, social support, and the buffering hypothesis. Psychol Bull 1985 Sep;98(2):310-357. [Medline]
- Cohen S. Social relationships and health. Am Psychol 2004 Nov;59(8):676-684. [CrossRef]
- Compas BE, Connor-Smith J, Jaser SS. Temperament, stress reactivity, and coping: implications for depression in childhood and adolescence. J Clin Child Adolescent Psychol 2004 Feb;33(1):21-31. [CrossRef]
- Fu Z, Burger H, Arjadi R, Nauta MH, Bockting CL. Explaining the efficacy of an internet-based behavioral activation intervention for major depression: a mechanistic study of a randomized-controlled trial. Clin Psychol Eur 2021 Sep 30;3(3):1-24 [FREE Full text] [CrossRef]
- Eberle SG. The elements of play: toward a philosophy and a definition of play. Am J Play 2014;6(2):214-233 https://www.journalofplay.org/sites/www.journalofplay.org/files/pdf-articles/6-2-article-elements-of-play.pdf.
- Scholten H, Granic I. Use of the principles of design thinking to address limitations of digital mental health interventions for youth: viewpoint. J Med Internet Res 2019 Jan 14;21(1):e11528 [FREE Full text] [CrossRef] [Medline]
- Cohen D, Lindvall M, Costa P. An introduction to agile methods. Adv Comput 2004;62:1-66. [CrossRef]
- Grow it! Corona project. OSF HOME. URL: https://osf.io/2at58/?view_only=b691104ecc3d45ad8b48e1bd60ad7125 [accessed 2022-02-01]
- IBM Corp. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp; 2017.
- Radomski AD, Wozney L, McGrath P, Huguet A, Hartling L, Dyson MP, et al. Design and delivery features that may improve the use of internet-based cognitive behavioral therapy for children and adolescents with anxiety: a realist literature synthesis with a persuasive systems design perspective. J Med Internet Res 2019 Feb 05;21(2):e11128 [FREE Full text] [CrossRef] [Medline]
- Ryan AM. The peer group as a context for the development of young adolescent motivation and achievement. Child Dev 2001;72(4):1135-1150. [CrossRef] [Medline]
- van Hoorn J, van Dijk E, Meuwese R, Rieffe C, Crone EA. Peer influence on prosocial behavior in adolescence. J Res Adolesc 2014 Sep 27;26(1):90-100. [CrossRef]
- Telzer EH, van Hoorn J, Rogers CR, Do KT. Social influence on positive youth development: a developmental neuroscience perspective. Adv Child Dev Behav 2018;54:215-258 [FREE Full text] [CrossRef] [Medline]
- Legerstee JS, Garnefski N, Jellesma FC, Verhulst FC, Utens EM. Cognitive coping and childhood anxiety disorders. Eur Child Adolesc Psychiatry 2010 Feb;19(2):143-150. [CrossRef] [Medline]
- Connor-Smith JK, Compas BE, Wadsworth ME, Thomsen AH, Saltzman H. Responses to stress in adolescence: measurement of coping and involuntary stress responses. J Consult Clin Psychol 2000 Dec;68(6):976-992. [Medline]
- Grow it!. Google Play. URL: https://play.google.com/store/apps/details?id=nl.erasmusmc.growit [accessed 2022-02-24]
- Grow it!. App Store. URL: https://apps.apple.com/nl/app/grow-it/id1496767455 [accessed 2022-02-24]
- Bergin AD, Vallejos EP, Davies EB, Daley D, Ford T, Harold G, et al. Preventive digital mental health interventions for children and young people: a review of the design and reporting of research. NPJ Digit Med 2020;3:133 [FREE Full text] [CrossRef] [Medline]
- Marcell AV, Klein JD, Fischer I, Allan MJ, Kokotailo PK. Male adolescent use of health care services: where are the boys? J Adolescent Health 2002 Jan;30(1):35-43. [CrossRef]
- Linardon J, Fuller-Tyszkiewicz M. Attrition and adherence in smartphone-delivered interventions for mental health problems: a systematic and meta-analytic review. J Consult Clin Psychol 2020 Jan;88(1):1-13. [CrossRef] [Medline]
- Schreuder MJ, Hartman CA, George SV, Menne-Lothmann C, Decoster J, van Winkel R, et al. Early warning signals in psychopathology: what do they tell? BMC Med 2020 Oct 14;18(1):269 [FREE Full text] [CrossRef] [Medline]
- Thabrew H, Fleming T, Hetrick S, Merry S. Co-design of eHealth interventions with children and young people. Front Psychiatry 2018 Oct 18;9:481 [FREE Full text] [CrossRef] [Medline]
- Riese H, von Klipstein L, Schoevers RA, van der Veen DC, Servaas MN. Personalized ESM monitoring and feedback to support psychological treatment for depression: a pragmatic randomized controlled trial (Therap-i). BMC Psychiatry 2021 Mar 10;21(1):143 [FREE Full text] [CrossRef] [Medline]
- Dietvorst E, Legerstee JS, Vreeker A, Koval S, Mens M, Keijsers L, et al. The Grow It! Serious game: longitudinal changes in adolescent well-being during COVID-19. Open Science Framework 2021 preregistered(forthcoming) [FREE Full text]
- Mens M, Keijsers L, Dietvorst E, Koval S, Legerstee JS, Hillegers MH. Mapping Mood during the COVID-10 pandemic in Dutch adolescents and youngsters. Open Science Framework 2021 preregistered (forthcoming) [FREE Full text]
|CBT: cognitive behavioral therapy|
|ESM: experience sampling method|
|mHealth: mobile health|
|MVP: minimal viable product|
Edited by A Mavragani; submitted 30.04.21; peer-reviewed by J Berghmans, A Knapp, A Daros; comments to author 10.08.21; revised version received 26.10.21; accepted 21.12.21; published 03.03.22Copyright
©Evelien Dietvorst, Michelle A Aukes, Jeroen S Legerstee, Annabel Vreeker, Micah M Hrehovcsik, Loes Keijsers, Manon H J Hillegers. Originally published in JMIR Formative Research (https://formative.jmir.org), 03.03.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.