Web-based interventions are typically self-guided programs attempting to create positive change through an interactive presentation of health-related material [1]. Our study investigates the design and development of a web-based intervention relying on positive psychology [2] to achieve beneficial change in young unemployed adults. The specific objective of the intervention is to increase the users’ likelihood for engaging successfully with training, work, or employment. The successful engagement of young adults in the labor market plays a vital role in promoting personal well-being, driving economic growth, and fostering social cohesion [3]. Research suggests the importance of exploring further how to build psychological well-being and resilience [4,5] to cope with setbacks and challenges, which will occur as part of a job-seeking process [6-8]. The research idea of this study was thus that unemployed young adults could benefit from building confidence in the possibility to learn and improve and through normalizing and providing strategies for overcoming common challenges and stress, which occur in job-seeking process and in a young adult’s life in general.

To provide young unemployed people with alternative interpretations of their situation and a mindset oriented toward learning and change, we draw upon the concept of wise interventions, as established in prior research [9]. Wise interventions are psychological interventions aimed at addressing social and personal challenges [10]. This field suggests that altering a maladaptive interpretation of a situation or an ability may change how the individual engages with their contexts in ways that lead to positive changes [11]. More specifically, our approach is guided by the previous research demonstrating that a time-limited and web-based wise intervention can improve stress management and enhance motivation and learning among young adults [12-14].

In the United States, the National Study of Learning Mindsets (NSLM) [13,15] developed and tested a web-based growth-mindset intervention. The NSLM intervention has evolved through years of iterative development originating from early paper-based interventions designed as newspaper articles distributed to students [16]. Over time, it has progressed toward web-based interventions (refer to the study by Blackwell et al [17]) and has also been translated and adapted to other institutional contexts (refer to the studies by Rege et al [12] and Dweck and Yeager [16]). A Norwegian adaptation, U-SAY, has been studied in experiments in a Norwegian educational context [12,18]. Both U-SAY and NSLM use reading and writing exercises to communicate a “growth mindset”—the belief that human capacities are not fixed but can be developed and increased in response to efforts, good strategies, and help from others [16] and a “stress-can-be-enhancing mindset” [14,19], which offers a more positive interpretation of psychological stress responses [19,20]. The content is carefully crafted to be relevant and meaningful to school students with clear and consistent messaging.

The NSLM and U-SAY interventions are designed for use in controlled contexts such as classrooms [15,16], administered through PCs with headphones [21]. Thus, researchers can control when and how the intervention content is being consumed and make sure there are no distractions. Such contexts provide structures to piggyback on [16] to deliver the intervention and social incentives to comply with the use of the intervention (eg, established routine, doing what the teacher says, etc). Despite falling under the umbrella term of web-based interventions, the NSLM and U-SAY interventions represent structured textual content digitized and iterated from the original paper-based intervention. Both interventions consist of 2 short 30-minute reading lessons in combination with writing reflection tasks. The interventions are not designed to be particularly engaging as interactive experiences; they are rather static and linear but then the intervention is only competing with what would normally take place in the classroom [16].

By contrast, fully self-administered web-based interventions, lack these structures and suffer from low user adherence as a consequence [22]. In uncontrolled environments, it is up to 3 times more likely that young adults will drop out before the intervention has been completed [23]. This challenge is accentuated when the users are not interested in, or do not enjoy engaging with the intervention [24,25]. This underscores the importance of focusing on the interventions themselves to find a better fit with the preferences of young adults [26,27].

Use of design methods has been suggested as an approach to increase adherence and reach of web-based interventions [15,28]. In a human-centered design process, understanding the context of use and the needs of the user is a central part of the design work [29]. Design methods and participatory approaches which enable integration of user feedback into the designed outcome [30] have been shown to be helpful in enhancing psychological interventions [26]. In addition, Kelders et al [22] argue that the persuasive design features of the user experience (UX) in web-based interventions may predict higher adherence. In persuasive design, behavior is a product of motivation, ability, and prompts [31]. Transcending the “activation threshold” (ie, desired behavior) is possible by working to increase motivation (praise, rewards, fun), reducing barriers (making it easy, thereby lowering needed ability) and adding prompts (triggers) [31]. Thus, human-centered design methods and persuasive design features are central elements in the explorative work described in this development study.

This study aimed to design and develop a web-based intervention application using positive psychology to shape mindsets and engage young adults in the labor market. Due to the challenges posed by low motivation in the user population, we explored how to strengthen intervention usability and motivation through engaging with relevant user participants and other stakeholders. We were particularly interested in understanding how to leverage theoretical underpinnings and established empirical knowledge as part of the design process, while adapting this to the context of young unemployed adults. Our research question was thus: “How to design a self-administered wise intervention for young unemployed adults leveraging evidence-based, relevant wise interventions?” In this paper, we present the process and the resulting intervention—the RØST web app.

This study is part of a broader project, Career Learning App, which will use design methodology to adapt protocols from psychology into an application supporting young adults who are currently not in education, employment, or training. The new labor market requires workers with high competencies who are not afraid of change, challenges, and acquiring new skills. For people with weak beliefs in their capacity to learn, this could be a major risk factor for their exclusion from labor market. The aim of the research project is to design, develop, and test the effectiveness of a scalable, human-centered, and evidence-based growth-mindset intervention for young unemployed adults. The resulting web-based intervention will be used in large-scale randomized controlled trials (RCTs) in Norway [32]. The project will provide new knowledge about young adults who are not in education, employment, or training, and how the public welfare system may support them with psychological tools to improve resilience in entering the labor market or starting with education.

The research question was explored through the design and development of a web app, intended as a self-administered wise intervention for young unemployed adults. The study used a human-centered design methodology [29,33] to ensure that the intervention would be relevant, user-friendly, and motivating to its users. Empathy with the end-user perspective may be particularly relevant in this context where motivation may be low in general, as indicated by the target group’s incomplete schooling [34] and frequent dropping in and out of employment and training [35].

In this process, researchers collaborated and engaged with designers, developers, and stakeholders, including potential end users, from October 2020 until the completion of the RØST application in December 2022. The process occurred in stages following a 3-step iterative design process, as outlined in Figure 1, consisting of both divergent (creative and intuitive) and convergent (logical and analytical) thinking processes [36-38].

The human-centered design process is a broad approach and may include many possible research methods to empathize with the user perspective. Specifically, this study used formative and qualitative usability testing [39] as a core method to evaluate concepts and prototypes and explore end users’ experiences. These usability testing sessions were conducted with the relevant end-user participants over Zoom (Zoom Video Communication, Inc; due to COVID-19 pandemic restrictions) and included a walk-through of the prototypes where users could comment freely what they were thinking about each screen. The sessions lasted 45 to 90 minutes and included semistructured and open interview questions.

The target of the usability sessions was to receive feedback on different prototype designs and ideas or inspiration for improvement. Thus, we tested prototypes of both high and low fidelity to capture participants’ opinions, reasoning, and attitudes regarding an interactive experience in relation to their own situation as unemployed. Participants were asked open questions such as “What do you think about what you see here?” and “Would this be helpful to you—or not?” Thus, the prototypes enabled exploration and functioned as boundary objects [40,41] that framed the conversations with end users.

Results from the usability testing were analyzed through the design process. Members of the core team took on the roles of observer and a moderator in the sessions with the participants. The observer wrote notes that could be used as input to a posttest summary of observations and findings, which represented key findings, highlights, and particularly interesting observations. These were established through a brief discussion between the moderator and the observer, combining the outcome of the discussion with that of visual diagramming in the form of affinity maps (the Jiro Kawakita method; refer to the study by Scupin [42]) and Post-it or notes of interesting quotes from the participants. The findings were shared either orally as a form of usability testing “debrief” in the weekly stand-up meeting or in writing on Slack (Slack Technologies, LLC). The feedback collected was then discussed in the core team and used to improve and iterate the design and content through new sketches and changes to prototypes [43]. This way, sensemaking of the collected data occurred through synthesis [44,45] where findings emerged through intuitive, holistic interpretations [46] and were applied to action through creative problem-solving to iterate on the designs.

In total, 13 young adults (n=7, 54% men and n=6, 46% women) from the target group participated in the study, giving their feedback on the prototypes. In addition to the end users, a multitude of other stakeholders were also involved in some manner, either in cocreating activity to bring forth ideas or contributing to problem-solving, or they were consulted for their input and expertise for the intervention app design, content, or development.

A core team of 3 researchers (including author IJS), 2 UX designers, and 2 developers worked to bring forth the design and the build of the web app. The core team met at least once a week, but usually, at least some of the team members met daily or several times per week and collaborated closely on all aspects of the web app.

End-user participants represented the most central stakeholder to this study and were involved through taking part in usability testing, and gave feedback on the latest prototype designs. Beyond end-user participants, other stakeholders were also involved, consulted, or informed for quality check and expertise. This included other researchers with expertise in fields such as behavioral and wise interventions, psychology, and design methods. These researchers were involved approximately once per month and took part in workshops or were consulted as advisers, often about the contents of the app, via emails or meetings. This advisory group of researchers shared expertise from their respective fields while being outside of the ongoing and day-to-day designing process of the intervention app.

The remaining stakeholders were people that the core team engaged with on an ad hoc basis, either through meetings or emails to get their input or expertise. Some contributed by performing parts of the work, for example, editing or proofreading (communication staff). Others contributed via their knowledge of the user group, such as the caseworkers at the Norwegian Labour and Welfare Administration (NAV) who provided the core team with their experience-based insight into the user group specific to the local context. Furthermore, the caseworkers shared their thoughts on how they believed the user group would react to the different parts of the web app, particularly regarding the wording. Thus, the core team could adjust the prototypes before usability testing with participants. The case workers and the advisory researchers also contributed by recruiting end-user participants.

Finally, some of the stakeholders were important to liaise with as potential “gatekeepers” to the final web app because of their roles. This included data protection (data protection officers and IT and legal advisers at NAV and at the University), technical architecture (university IT staff), and content owners that could grant permissions to use third-party intellectual property for the intervention, including the use of design systems, photos and news media content.

Inclusion criteria for the study were as follows: participants aged between 18 to 25 years and living in Norway or speaking Norwegian (due to in-app language) with experience of being unemployed or dropping out of school. Recruitment was done through partner stakeholders who deal directly with this population. The COVID-19 pandemic made in-person meetings impossible and also made it difficult to reach participants, for instance with the loss of physical premises to distribute flyers and so forth. Thus, caseworkers at NAV sent out invitations to take part in the study through their secure dialogue platform, and users who were interested could express their interest through a web-based form. This way, we recruited 8 young adults, which was a little below our aim. To avoid delays in the ongoing design process, we recruited additional 2 participants through a local individual placement and support program and 3 participants through snowballing. All testing sessions were done on the web via the videoconferencing platform Zoom (the lessons learned from conducting this research remotely are presented in the study by Straand et al [47]). The participants had diverse backgrounds and stated diverse reasons for their current situation, including unfinished schooling, mental illnesses, waiting for support from, for instance, NAV, and lack of training placements following vocational schooling.

Other stakeholders involved in this study were not recruited as research participants but rather contacted directly due to their specific roles or expertise. Some of them also had a defined role in the Career Learning App project.

The study was evaluated and approved by the Norwegian Centre for Research Data (approval number 131074). All end users provided explicit and written consent to participate in the study, and were given the opportunity to opt out at any time. Identifying information is omitted from all publications. End users were rewarded with gift cards of US $30 per session for their participation. Stakeholders were not rewarded for their participation.

Throughout the process of design and development, involving end users as participants in usability testing influenced content revisions, helped tackle usability issues, and supported fine-tuning of strategies to increase user motivation. While the design and functionality of the web app is the central outcome of this study and is elaborated on in detail in the next section, it is important to also highlight some of the key findings that were drawn from the usability testing from across stages in the process and their implications for the ongoing design. The findings represent the interpreted sentiment and mood of the users, that is, what the users expressed directly or could be interpreted from their more indirect comments, gestures, or behavior.

For transparency on the prevalence of reported findings across participants and prototypes, we apply the following use of wording to describe the strength of each finding: All and nearly all (observed for the vast majority of relevant stakeholders and participants), most (observed for the majority of stakeholders), some (observed for a substantial minority of stakeholders), a few (observed for a small minority of stakeholders). These findings are summarized in Textbox 2 and visualized in Figure 3 with the derived design implications.

This study used design methods to create an interactive intervention app based on positive psychologies targeting unemployed or inactive young adults. During an iterative process, with frequent usability testing with end users and input from stakeholders, the app evolved from concept sketches and early prototypes to a finished design ready for development. This paper presented the design and development of RØST, a web-based intervention app designed for mobile and desktop use. Through usability testing of prototypes, we could receive fast feedback and consider user insight in the next design iteration. Not only did the input from users lead to changes in design and functionality, but it also led to changes in content and its presentation. The intervention web app has been fully developed and is ready to be used on any device. With minimum adjustments, the web app may also be scaled up to be released to a broader population after being rigorously tested in planned forthcoming studies.

In this study, we have not only reported on the outcome of the study, which was the web app, but also detailed the process for how the end result came about. Rather than formalized analysis, we relied on action-oriented synthesis [44] and intuitive, abductive discovery [42]. In line with other design-oriented research, it is not always easy to see the direct link between the “research for design” [85,86], which was the feedback from end users and stakeholders, and the design itself [45]. The core team acted as interpreters and facilitators, balancing the opinions of the end users and stakeholders and insight into the context of unemployment in Norway, with past wise interventions, psychological theory, and theory on designing for behavior change and motivation.

A particular challenge to be addressed in the design and development of RØST was the adaption of content from existing interventions, designed for use in structured contexts, to this self-administered intervention app. This challenge was particularly important, given potential issues concerning motivation and structure in a user population of youth outside work and education. This challenge was addressed through a human-centered design process where end users and other stakeholders provided feedback on early prototypes that were iterated on several times. Through this process, we were able to develop more relatable examples and adapt to a new context, which is expected to be more useful for this population. To support the adaptation of existing intervention content, the development team established a set of principles and strategies for the human-centered design of RØST. Furthermore, we included features to ensure engagement and adherence via persuasive and interactive design elements. This research was thus informed both from the existing knowledge base of several evidence-based psychological and wise interventions and from end users and stakeholders involved throughout the development process.

Figure 8 depicts the dual grounding of the RØST application. On one hand it is based on a human-centered design process with participatory UX insights and theory from human-computer interaction, such as persuasive design, However, on the other hand, the app content and the psychologies it contains are developed by combining multiple evidence-based wise interventions.

Crafting effective digital interventions that are ready for use requires interdisciplinary work and skills within knowledge domains such as psychology, technology, and human-centered design [87]. Reflecting on the experiences of taking part in this research, we see that this may at times be a difficult balancing act. By elaborating on a real-world design research and development case in detail, we hope to contribute to a greater understanding of the complexity and many layers of decision-making and considerations that occur throughout such a process. Specifically, there is a potential for interactive technology bringing more to the interventions, moving beyond digitizing [87] and adapting past evidence-based interventions. Losing current restraints could lead to increased user engagement through new uses of interactive technology.

The user’s interests in the intervention content appeared to be minimal. This was seen not only in the evaluation of U-SAY and the early prototypes but also from our past interaction with the user group when exploring a gaming-based intervention [27]. Thus, a more visually pleasing and user-friendly app experience with relatable content will likely help reduce “friction” (lowering barriers to use). However, it will probably not be sufficiently motivating to get our target users to stick to the intervention from start to finish. Persuasive design [31] could be a way forward to boost motivation, as a lack of such features (eg, reminders, praise, and rewards) are potential shortcomings in web-based interventions [22]. Therefore, the completed RØST application includes praise, SMS text messaging triggers, and a scoring system leading to gift cards. The gift cards were pointed to by participants in our study as the biggest motivation factor for completing the program.

Though persuasive design may ensure adherence and get users to form and stick to new habits [88], there are also risks connected with the introduction of such features. They may undermine the intended therapeutic effects by adding more stress to the users [89]. This user group of unemployed young adults may be particularly sensitive to this kind of external pressure, even mild pressure like completing app content, if they already have depression or anxiety leading to avoidance. Furthermore, the provision of external incentives may also undermine intrinsic motivation [90]. The latter is not something we regard as particularly relevant because in this case, the baseline motivation is already low and as such, there is little risk of undermining intrinsic motivation via incentives when there is none to begin with [91]. We also consider the incentives provided in our study to be of a relatively low value so that it will not crowd out intrinsic motivation in those who are already motivated.

The use of persuasive design and gamification may be criticized as leading to engagement with a “flawed product,” as pointed out by Bogost [92]. Ideally, the intervention app should be motivating to use in itself. However, we consider this “superficial engagement” through gamification and persuasion to be ethically reasonable, if they contribute toward making the mundane fun while achieving changes that are beneficial to the users. This has been a successful approach within, for example, education, with the gamification of language learning apps such as Duolingo (Duolingo). However, the fine line between persuasion and “dark patterns” in UX design should be handled with great caution, and further research is needed to evaluate whether and how these persuasive and gamification design features impact the effectiveness of the app and its user motivation.

There were a number of needs and considerations related to forthcoming experimental studies (RCTs) that shaped the design process in this study. The strengths of this approach to development are that the app will be tested for effectiveness in large-scale experiments and is built on evidence-based intervention designs. A weakness of this approach is that controlling variance influences and shapes the UX design in a way that could itself diminish engagement and user satisfaction.

Other weaknesses relate predominately to our methodology of capturing user feedback on prototype experiences. Thus, the user feedback and observation of app use were not based on ready-to-use solutions but on a range of prototypes. The method was chosen specifically to be able to take participant contribution and input into account in the designed outcome, but limits to some degree the reliability of the insights reported. There were also a limited set of participants recruited, and they all received rewards to contribute to the usability testing. We cannot know whether the users we engaged in the study were more prone to see the importance of rewards since they were already being rewarded to take part. Furthermore, most of the prototypes included some form of reward scheme or gamification built-in, either points, lottery, praise, money, or social features. Being aware of this limitation, we have built 2 versions of the app, one version which includes the gift cards and points and another one where we have removed this feature. Both versions of the app were tested and compared in a recent study (not yet published).

Another limitation is the lost opportunity of reporting more specifically on the stakeholder perspectives. In this study, stakeholders were not recruited as research participants, and their opinions or influence on the intervention app were not seen as central to the research question. However, in retrospect, perhaps a more complete documentation of the process, including stakeholder opinions could have contributed to enriching the picture of the research.

Furthermore, we do not venture into a discussion on whether the psychologies and content used in the app are indeed appropriate for the target group of unemployed young adults and for the desired outcome set for the intervention. For example, the MCII intervention, especially the goal-setting aspect, may not be suitable for users who lack specific objectives or set unrealistic goals, potentially leading to stress. Whether or not the selected positive psychologies used are appropriate for the target population will be explored further during larger scale experimental studies to understand its effects.

Involving end users in the development of a self-administered wise intervention enabled relatable content development and resolved potential usability problems in the adaptation of several evidence-based interventions into a web app specifically designed for a population of unemployed young adults. This approach also confirmed the need for increasing user motivation through supportive persuasive functionality, such as notifications, reminders, praise, and rewards. Such persuasive features are likely pivotal for promoting signup and adherence to the intervention. The results of this study have several implications. An essential implication is to enact and cater to end-user feedback and observation through an application-oriented developmental protocol extended by user insights, as well as to be guided by psychologically informed design. Furthermore, we identified a challenge in harmonizing the flexible interactivity that users wanted and needed to feel engaged with an evidence-based research approach. The lack of interactivity and catering for responsive user journeys identified may contribute to a suboptimal UX. Future research should apply design methods to these issues to resolve them and explore whether it is possible to include user perspectives not only in the refinement and development of a solution to make it self-administered but also in the identification of and definition of the problem that the intervention is trying to tackle. Nevertheless, we conclude that the RØST application is ready to be tested and used in further research with users to measure intervention effectiveness and to get a greater understanding of use, reach and adherence, and the effect of incentives introduced in such an app. Although the app is designed specifically with research needs in mind, it may be implemented relatively easily as a ready-to-use app outside of the research domain.