Published on in Vol 5, No 12 (2021): December

Preprints (earlier versions) of this paper are available at, first published .
Development of the Shift Smartphone App to Support the Emotional Well-Being of Junior Physicians: Design of a Prototype and Results of Usability and Acceptability Testing

Development of the Shift Smartphone App to Support the Emotional Well-Being of Junior Physicians: Design of a Prototype and Results of Usability and Acceptability Testing

Development of the Shift Smartphone App to Support the Emotional Well-Being of Junior Physicians: Design of a Prototype and Results of Usability and Acceptability Testing

Original Paper

1Black Dog Institute, Randwick, Australia

2School of Psychiatry, Faculty of Medicine, UNSW Sydney, Kensington, Australia

3Institute for Positive Psychology and Education, Australian Catholic University, North Sydney, Australia

4Brain and Mind Centre, Faculty of Medicine and Health, University of Sydney, Australia

Corresponding Author:

Samineh Sanatkar, PhD

Black Dog Institute

Hospital Road

Randwick, 2031


Phone: 1 02 9382 4368


Background: Junior physicians report higher levels of psychological distress than senior doctors and report several barriers to seeking professional mental health support, including concerns about confidentiality and career progression. Mobile health (mHealth) apps may be utilized to help overcome these barriers to assist the emotional well-being of this population and encourage help-seeking.

Objective: This study describes the development and pilot trial of the Shift mHealth app to provide an unobtrusive avenue for junior physicians to seek information about, and help for, well-being and mental health concerns, which is sensitive to workplace settings.

Methods: A 4-phase iterative development process was undertaken to create the content and features of Shift involving junior physicians using the principles of user-centered design. These 4 phases were—needs assessment, on the basis of interviews with 12 junior physicians; prototype development with user experience feedback from 2 junior physicians; evaluation, consisting of a pilot trial with 22 junior physicians to assess the usability and acceptability of the initial prototype; and redesign, including user experience workshops with 51 junior physicians.

Results: Qualitative results informed the content and design of Shift to ensure that the app was tailored to junior physicians’ needs. The Shift app prototype contained cognitive behavioral, mindfulness, value-based actions, and psychoeducational modules, as well as a tracking function that visualized patterns of daily variations in mood and health behaviors. Pilot-testing revealed possible issues with the organization of the app content, which were addressed through a thorough restructuring and redesign of Shift with the help of junior physicians across 3 user experience workshops.

Conclusions: This study demonstrates the importance of ongoing end user involvement in the creation of a specialized mHealth app for a unique working population experiencing profession-specific stressors and barriers to help-seeking. The development and pilot trial of this novel Shift mHealth app are the first steps in addressing the mental health and support-seeking needs of junior physicians, although further research is required to validate its effectiveness and appropriateness on a larger scale.

JMIR Form Res 2021;5(12):e26370



Junior physicians exhibit levels of psychological distress and emotional exhaustion to a greater degree than their senior counterparts [1,2]. Junior physicians reported feeling impacted by a range of workplace-related stressors, including long working hours, a lack of breaks, and, at times, bullying and harassment [3,4]. A review of several prospective studies showed that individual factors, such as self-criticism, emotional instability, and a family history of psychopathology, are predictive of mental illness in junior physicians [5]. Furthermore, structural and personal barriers to help-seeking for mental health concerns have been noted, such as concerns about being reported to medical regulators and anxiety about reversing roles from being a physician to becoming a patient [6-8]. Australian data indicate that the most commonly reported barriers to professional help-seeking for depression among depressed physicians are privacy and confidentiality concerns [9].

Delays in receiving targeted treatment have the potential to compound and prolong symptoms of poor mental health as well as to increase the likelihood of developing comorbidities such as alcohol dependence [7,10]. Research suggesting a negative association between physicians’ psychopathology and best patient care practices highlights the implications of physicians’ mental health in the broader community [11-13]. Therefore, in addition to the adverse effects on the individual, it is in the wider public interest to support junior physicians in their transition into a demanding work environment and to help them seek and receive effective mental health care.

Although psychological interventions designed specifically to support physicians’ mental health are scarce, a recent meta-analysis found that interventions targeting physicians yielded small but significant reductions in symptoms of common mental disorder and suicidal ideation, particularly when therapeutic components were based on principles of cognitive behavioral therapy and mindfulness [14,15]. Of note, one study found that medical interns randomly assigned to a web-based cognitive behavioral therapy intervention group were 60% less likely to report suicidal ideation during their internship year than the comparator attention-control group [16]. Further research suggests that valued living and present-moment awareness components practiced in acceptance and commitment therapy and psychoeducational programs may be useful in reducing psychological distress in health care professionals and students [17-19].

Although the potential benefits of teaching these skills to junior physicians are clear, the practicality of such interventions can be challenging. Junior physicians are busy and regularly move between roles. New digital tools may be able to assist with these logistical challenges. Although mobile health (mHealth) apps have been shown to improve working populations’ mental well-being and willingness to seek help for mental health concerns [20-23], they have yet to be tested in well-controlled studies among physicians. Many physicians have already used mobile apps in the workplace to guide their management or prescription. A novel mHealth app using the same avenue as these professional development tools may be an acceptable way of delivering mental health support to junior physicians.

Previous research highlights the importance of a user-centered approach to developing mHealth app interventions [24-26] and, if designed for employees, additional factors, such as the workplace environment, should be taken into account [27,28]. As such, mHealth apps involving workplace considerations pose additional constraints on the app development process to ensure that content is effective, adequately delivered, and suits the target working population. User-centered design approaches involve iterative phases of prototyping, ideally employing co-design and end user feedback, and involve consideration of the user at every stage of the design process to ensure that the intervention meets their needs [29,30]. This, in turn, has been reported to maximize users’ engagement with, and adherence to, an mHealth intervention, and hence its impact [31].

This paper describes the process of developing Shift, a self-guided mental health and help-seeking smartphone app for junior physicians located in New South Wales, Australia. To our knowledge, this is the first mHealth app designed specifically to support the mental health of junior physicians. We used a user-driven and iterative development process, employing the principles of user-centered design and a multiphase process. This paper presents the 4-phase Shift app development process and how this process incorporates knowledge and feedback derived from qualitative assessments, pilot-testing, stakeholder and expert consultations, and user experience workshops with the target population.


There were 4 project phases as follows: phase 1, needs assessment through qualitative end user interviews; phase 2, Shift app prototype development; phase 3, pilot-testing of the Shift app prototype; and phase 4, generation of an updated version of the Shift app. In preparation for phase 1, consultations with a range of stakeholders were conducted (including junior physician managers, providers of support services for junior physicians, and professional organizations related to junior physicians in New South Wales) to examine the existing mental health support services for junior physicians, and facilitators of and barriers to engagement with these services. This was to ensure that the app development complemented existing support and provided up-to-date information on available services. Furthermore, our broader research team had previously developed HeadGear, an mHealth app that has been found to be effective in male-dominated working populations [21,32,33]. The intervention component in HeadGear was delivered in a 30-day challenge format, which successively unlocked psychoeducational material, as well as behavioral activation, goal-setting, and mindfulness techniques. Hence, one option available to our team was to utilize the 30-day challenge format of the evidence-based HeadGear app and to modify the clinical content to meet the needs of junior physicians. To this end, the interviews with junior physicians in phase 1 included a question about their attitudes toward a 30-day challenge in an app to support the well-being of junior physicians.

Phase 1: Needs Assessment Through Qualitative Interviews

The objective of the qualitative component of this project was to inform the design and development of clinical content and to ensure that the app is tailored to the specific needs, characteristics, and challenges faced by this unique user group. Specifically, the interviews aimed to (1) identify the main stressors and challenges for junior physicians, both at and outside of work; (2) explore their attitudes toward a mental health app; (3) identify facilitators of and barriers to their use of, interest, and engagement in a mental health app; (4) assess current use of general apps; and, finally, (5) to identify their suggestions, preferences, and dislikes or unwanted features that they felt would support junior physicians’ mental health and well-being. The qualitative component was granted full ethical approval by the South Eastern Sydney Local Health District Human Research Ethics Committee (protocol #: 18/140, HREC/18/POWH/321). A qualitative analysis of the interviews focusing on the experiences of junior physicians is reported in detail elsewhere [34]. This paper presents specific app-related items and findings from interviews.

Participants were recruited from 2 metropolitan hospital sites in the Sydney area between July and September 2018. Advertising took place via email invitations distributed by medical supervisors, through email and social media announcements to professional organizations related to junior physicians, and through on-site hospital visits by members of the research team (JC, KP, and GB); 41 junior physicians expressed interest in the study, of whom 12 provided written informed consent and were recruited for one-on-one face-to-face interviews (conducted by JC). Participants included 9 women and 3 men aged between 24 and 35 years. The sample was evenly spread across the early stages of training and comprised interns (n=3), residents (n=5), and registrars (n=4). The majority had studied medicine in Australia and were generally 3-5 years into their clinical training.

Data on app-related items were analyzed using a thematic analysis approach [35] informed by grounded theory and Massey emergent approach [36]. In an iterative process undertaken by one researcher (KP), all transcripts were reviewed closely to generate an initial first-level coding framework. Through subsequent refinements through discussion with co-authors, broader second-level themes related to the app issues were identified, and subthemes and common suggestions were listed under each of the 4 main issues of interest, summarized below in the Results section.

Phase 2: Prototype Development

On the basis of the recommendations made by junior physicians in the qualitative interviews and consultations with stakeholders in line with evidence from recent literature [14,17,18], new clinical content for the Shift app was written by 1 psychologist, 2 clinical psychologists (JC and RC), 2 psychiatrists (SH and NG), and 1 researcher (GB).

Phase 3: Pilot-Testing

Pilot-testing was conducted in October and November 2019 to examine the usability and acceptability of the prototype version of the Shift app. Trends in depression and anxiety symptom severity, as well as changes in help-seeking intentions before and after using the Shift app prototype over a 4-week period were also examined. This study was approved by the South Eastern Sydney Local Health District Human Research Ethics Committee (protocol # 2019/ETH00318).

Two New South Wales hospital sites, one regional and one metropolitan, issued recruitment calls via email messages on behalf of the research team to junior physicians at the intern, resident, or registrar levels. The eligibility criteria were current employment as junior medical officers in New South Wales and ownership of an internet-enabled smartphone with an Apple or Android operating system. A total of 52 candidates accessed the study website, of which 50% (26/52) consented to participate. A final sample of 22 participants (13/22, 59% women; mean age 29, SD 4.1 years) entered the pilot study and completed a baseline assessment. A diagram of the participant flow is shown in Figure 1.

Figure 1. Flow chart describing how participants progressed through the pilot study phases.
View this figure

A baseline questionnaire assessed basic demographic information (age, gender, level of training, and regional or metropolitan placement) and symptoms of depression (Patient Health Questionnaire 9-item) and anxiety (General Anxiety Disorder scale 7-item) over the past 2 weeks. Participants indicated their depression and anxiety symptoms on a 4-point Likert-type response scale ranging from 0 (not at all) to 3 (nearly every day). In addition, participants indicated their previous and recent help-seeking intentions for mental health problems (eg, “If you were to develop a mental health issue, how likely would you be to seek help from a GP or mental health professional [eg, psychologist/psychiatrist]?”) on an 11-point Likert-type response scale ranging from 0 (not at all likely) to 10 (very likely). After completion of the baseline questionnaire, participants were given instructions on how to download the Shift app onto their mobile phones. The app automatically recorded usage metrics, such as the number of log-ins and challenges or sessions completed.

After 1 month, participants were invited via email to complete a poststudy questionnaire reassessing their current depression and anxiety symptom levels and help-seeking intentions. At the poststudy assessment point, participants were further asked to respond to a battery of purpose-built questions relating to the usability and acceptability of the Shift app prototype (eg, “Was the app interesting/engaging?”) on a 5-point Likert-type scale ranging from 1 (not at all) to 5 (completely) and to the overall satisfaction rating of the app on a 5-point Likert-type scale ranging from 1 (low) to 5 (high). A modified version of the System Usability Scale was also administered to obtain an objective indication of the overall ease of use of the app.

Phase 4: Shift App Redevelopment

It was anticipated that a final phase of design would be required after the initial pilot-testing of the Shift app. A series of 3 user-consultation workshops (N=51) at 2 metropolitan hospitals, driven by a lead user experience designer (VC), helped develop any final changes and responses to insights gained from the pilot-testing. The feedback of junior physicians participating in the workshops was collected using an interactive prototype containing screen mock-ups and questions aimed at testing potential changes and solutions.

Phase 1: Needs Assessment Through Qualitative Interviews

Attitudes Toward a Mental Health App for Junior Physicians

All participants reported that they owned a smartphone and used apps multiple times each day. Half of the sample had already tried at least one mindfulness app, and 2 had previously used a web-based mental health app. Although the use of work-related apps was common, the use of such apps was centered on communication and medical information–seeking and reference material. A total of 83% (10/12) participants reported to be very or somewhat interested in the idea of an app to support the mental health of junior physicians and provided positive endorsement for the idea in principle and for its potential to benefit the population. Most of the participants endorsed use of the app for both prevention of and early intervention in mental illness and suggested the inclusion of directions to support and treatment services for those with more severe mental illness. Although no participants were directly opposed to the idea of an mHealth app to support junior physicians’ mental health, 2 participants questioned whether fellow junior physicians would use an mHealth app or to what extent they would benefit from it, particularly if physicians were not currently experiencing symptoms of mental illness (“no one takes their medicine when they are feeling good”).

App Naming, Content, and Feature Suggestions

Shift was named based on participants’ views that the name of the app should be unobtrusive and not obviously related to mental health. This term refers to both shift work, which is one of the most common stressors in this population, and to shifting cognitions and behaviors to promote better mental health and well-being, in line with cognitive behavioral and mindfulness principles.

Participants made numerous suggestions for clinical app contents, including mindfulness, sleep strategies, mood monitoring, behavioral activation, and cognitive therapy targeting specific situations commonly faced by junior physicians. The provision of strategies to deal with problematic work situations was often seen as having a stress-buffering and destigmatizing effect. A list of recommendations for the broad content areas provided by participants is presented in Table 1.

Table 1. Qualitative interview participants’ overview of main content recommendations for Shift and whether recommendations were adopted in successive versions of the app (N=12).
Recommended content componentsVersion 1Version 2
Cognitive behavioral therapya
Sleep hygiene
Mindfulness and stress management
Pleasant activity scheduling
Practical lifestyle strategiesa
Problematic work situationsa
Hand over tips for changing terms or hospitals

Stories from junior physicians

aIndicates an addition or improvement compared with the previous version.

All participants provided positive feedback about the idea of a 30-day challenge, with some reporting that they were more likely to use this feature as it was time-limited and seemed achievable with a set end point and small regular goal-oriented challenges. Several participants suggested graphical feedback on symptom trackers that compared multiple outcomes over time to appeal to the scientifically minded. A list of desired mHealth app features, as expressed by the participant sample, is presented in Table 2.

Table 2. Qualitative interview participants’ desired features for Shift and whether recommendations were adopted (N=12).
Desired featuresVersion 1Version 2
Logical, clear app structure with clear user flowa
Simple layout, easy to navigate quicklya
Default private option—no linkage to social media
Quick start option, easy access log ina
Skip function; ability to return to modules latera
Provision of both text and audio formats
Symptom tracker function with graphics showing chartsa
At log in, quick tick box of symptom self-assessmenta
Centralized access to many things from one placea
Notifications and reminders should be optional

aIndicates improvement compared with the previous version.

Facilitators of and Barriers to App Use

The main app use barrier reported by the participants was that the app would feel like another chore when participants were already time-poor and working long hours. Although most participants expressed concerns about confidentiality, deidentification, and minimal information-sharing, 92% (11/12) of participants reported that they would still provide their name and email address to register to a mental health app.

The main facilitators of app engagement that the participants reported were the app being quick and easy to use and having added-value features that would distinguish the app over others (that’s targeted to medics, that other apps aren’t going to address). Most participants reported that they would be happy to use the app quite frequently, such as every morning or every few days, but only if the sessions were very brief.

Phase 2: Prototype Development

The Shift app was developed for use in Android and Apple operating systems. The main features of the app are the following: (1) therapeutic and psychoeducational modules, (2) provision of contact details to mental health organizations and workplace resources, (3) mood and habit tracking, and (4) brief symptom assessments. Shift delivers content through a variety of text, audio, video, and graphical displays.

On the basis of previous research on the therapeutic benefits of cognitive behavioral, mindfulness, and value-based action components for medical professionals [14,15] and guided by preferences expressed in phase 1 qualitative interviews, the Shift app was developed employing cognitive behavioral principles of thought evaluation (ie, identification, evaluation, and modification of unhelpful thoughts) and engagement in valued action (ie, values-consistent patterns of action) adopted from acceptance and commitment therapy. Relaxation techniques (eg, progressive muscle relaxation) based on mindfulness and stress management practices were also incorporated to lessen the impact of stressful life events or daily stressors. Cognitive behavioral, value-based, and mindfulness modules were generated and presented in a 30-day challenge format. Each challenge was designed to take approximately 3-4 minutes to complete.

In addition, a suite of psychoeducational modules (sessions) was developed, including informational content on common mental health disorders, avenues through which to seek help for mental health concerns, and suggestions on how to incorporate relevant strategies, such as how to adjust to shift work. Psychoeducational content included mental health, help-seeking, and workplace information, such as depression, anxiety, mandatory reporting, at home and workplace avenues for seeking help, workplace bullying, adjusting to rural and regional placements, exams and interviews, and sleep health.

A tracking tool and symptom screening options were designed to allow users to capture daily snapshots of how they were faring and, in the case of the tracking function, build a visual tool to observe variations in mood and behavioral patterns over time.

The clinical content and design drafts of the app prototype were modified based on user experience feedback. User experience experts and psychologists worked with 2 junior physicians to refine the user pathways, functionality, color palette, and design and to modify the clinical content (eg, examples of scenarios used in cognitive therapy). Tables 3 and 4 outline the contents of the resulting Shift app prototype, which were organized into challenges and sessions.

Table 3. Organization of challenge topics in the Shift app prototype version.
Therapeutic type and challenge day or daysTopic nameType

2Introduction to mindfulnessVideo

3Seeing the horizonAudios

9Grounding anchorAudios

16Calming breathAudios


24Cargo thoughtsAudios

26Breathing windAudios

28Lapping oceanAudios

4Introduction to values and values as a physicianVideo

5Strive for fiveText

6, 12, 17, 21, 27Scheduling meaningful actionsText
Cognitive behavioral

7Introduction to unhelpful thoughtsVideo

8Unhelpful thoughtsText

11Cognitive biasesText

14Introduction to thought challengingVideo

15Thought challengingText

22Worry decision treeText

23Cognitive therapy reviewVideo
Positive psychology


13Getting activeText

18Social supportText

19Help a friendText

25100 enjoyable activitiesText

29Planning for the futureText


30Putting it all togetherVideo
Table 4. Organization of session topics in the Shift app prototype version.
Sleep and fatigue

Sleep healthText

Adjusting to shift workText
Common mental health problems




Posttraumatic stressText

Alcohol and other drugsText
Getting help

Get help nowText

Dealing with intense emotionsText

How to seek help: workplace avenuesText

How to seek help: nonworkplace avenuesText

Mandatory reportingText
Common issues for JMOsa

Exams and interviewsText

Work-life balanceText

Adjusting to rural and regional placementsText

Bullying in the workplaceText

Dealing with the death of a patientText

Calling for a consultText

Feeling inadequateText

aJMO: junior medical officer.

Phase 3: Pilot-Testing

App Acceptability and Usability

As shown in Table 5, the median responses to questions relating to overall app rating, content understandability, appropriateness, and usefulness were all on or above the midrange of the response scales. The overall system usability rating was 84.72 (SD 8.33), which was above the average score of 70 across technological tools more generally [37] and comparable with an average score of 77 reported for common smartphone apps and tablets [38]. Tables 6 and 7 present a breakdown of participants’ usefulness ratings of Shift challenge and session components. Participants rated the mindfulness challenges favorably and rated the general and value-based components least favorably. Among the session topics, the sleep and fatigue information components received the highest usefulness ratings, while the other sessions were rated lower or were not attempted.

Table 5. Pilot trial participants’ responses to the usability and acceptability of the Shift app prototype (N=9)a.
ItemValues, median (range)Values, minimum-maximum
How well did you understand the content of the app?5 (2)3-5
Was the app content appropriate for you?5 (3)2-5
Was the app interesting/engaging?4 (2)3-5
Do you feel that the app has helped you improve your mental health?3 (2)1-3
Would you recommend this app to other junior physicians?4 (2)3-5
What is your overall rating of the app?3 (3)2-5

aResponse scales ranged from 1 to 5.

Table 6. Pilot trial participants’ usefulness ratings of the Shift app prototype 30-day challenge contents (N=8)a.
Challenge type and topic namebValues, median (range)Values, minimum-maximum
Mindfulness4 (5)0-5
Value-based0 (5)0-5
Cognitive behavioral3 (5)0-5
Positive psychology

Gratitude3 (5)0-5

Getting active1.5 (5)0-5

Social support2.5 (5)0-5

Help a friend0 (4)0-4

Enjoyable activities3 (5)0-5

Planning for the future1.5 (5)0-5
General0 (4)0-4

aResponse scales ranged from 0 to 5, where 0 indicates unattempted components, 1 indicates low perceived usefulness, and 5 indicates high perceived usefulness.

bOnly positive psychology challenge topics were assessed individually because of the distinctiveness of each topic in this category.

Table 7. Pilot trial participants’ usefulness ratings of the Shift app prototype session contents (N=8)a.
Session type and topic nameValues, median (range)Values, minimum-maximum
Sleep and fatigue

Sleep health3 (5)0-5

Adjusting to shift work3 (5)0-5
Common mental health problems

Depression1.5 (4)0-4

Anxiety2.5 (5)0-5

Burnout1.5 (5)0-5

Posttraumatic stress0 (4)0-4

Alcohol and other drugs0 (4)0-4
Getting help

Intense emotions0 (4)0-4

Workplace avenues1 (5)0-5

Nonworkplace avenues1.5 (5)0-5

Mandatory reporting0 (5)0-5
Common issues for JMOsb

Exams and interviews0 (3)0-3

Work-life balance1.5 (5)0-5

Remote placements0 (4)0-4

Workplace bullying0 (5)0-5

Death of a patient0 (5)0-5

aResponse scales ranged from 0 to 5, where 0 indicates unattempted components, 1 indicates low perceived usefulness, and 5 indicates high perceived usefulness.

bJMO: junior medical officer.

App Use

Of the 22 participants, 95% (21) downloaded Shift and used the app at least once. The mean number of log-ins was 5.24 (SD 5.93) and participants spent an average of 28 minutes (SD 52.7) on the app, although the large SD indicated that there was considerable variability in use times. The median use time was 11 minutes (range 199.35). Participants completed an average of 5 challenges (SD 7.75) and spent 3.24 minutes working through these challenges (SD 4.53). Sessions were used less frequently (mean 1.81, SD 2.56), and less time was spent on sessions (mean 1.38, SD 1.46 minutes).

Symptoms Change

Wilcoxon signed-rank tests indicated that depression (Z=−1.38; P=.168) and anxiety (Z=−1.05; P=.293) scores slightly decreased, albeit not significantly, and that help-seeking intentions were largely unchanged (Z=−.38; P=.705) over the 1-month period of app use.

Phase 4: Shift App Redevelopment

On the basis of the results of the pilot, a major redesign of the prototype was conducted to create a more user-friendly and user-driven learning experience. The pilot study results indicated that the first version of the app did not engage junior physicians sufficiently well, which may have been an important factor contributing to small effect sizes in symptom change and help-seeking scores. A series of 3 user-consultation workshops (N=51) at 2 metropolitan hospitals helped finalize the proposed changes to the prototype. These were (1) layout and design improvements, (2) increased personalization and ease of access, (3) updates of clinical contents, (4) the inclusion of self-reflection activities, and (5) the adoption of more meaningful and relatable wording [6].

A streamlined login process with the inclusion of a biometric security system (ie, fingerprint or face ID authentication) was incorporated to facilitate use after app download. Personalization enhancements were achieved through significant changes in the presentation of the app. Importantly, the 30-day challenge structure was removed, and challenges and sessions were organized under a general overview of the topics. In this view, users were directed to contents through headings named Mental Health, Getting Help, Lifestyle, and Work. The challenge concept was maintained by incorporating an option for users to set their own weekly targets (ie, number of activities to complete each week) during the app on boarding process. With this new functionality, users could choose achievable goals relating to their app use while still encouraging regular use of the app.

The previously limited line-graph tracking function was completely redesigned to become more interactive and visual, as well as allowing a new option of work/life balance. To accommodate these changes, thorough layout and presentation updates were made, as well as enhancing the interactivity with a complimentary day-by-day view to show which activities were completed on which dates. Consistent with the user experience workshop feedback, new modules on exercise and diet were incorporated into the app. Two additional modules were developed in response to the COVID-19 pandemic. Informational sessions were extended with the inclusion of example stories from junior physicians and adjunct brief empirical evaluations (symptom screeners, for example, the Patient Health Questionnaire–2-item, General Anxiety Disorder scale–2-item). Example stories, based on recommendations put forward by junior physicians in qualitative interviews (Table 1), illustrated mental health challenges and invited users to elaborate on symptoms. The mHealth app contents were finally proofread by 2 clinical psychologists, 3 researchers (IC, AB, and SS), and 1 digital learning designer (JP) to ensure the provision of up-to-date clinical and psychoeducational contents in a way that facilitates learning [39,40]. The structure and design graphical guides of the novel Shift version are provided in Figure 2.

Figure 2. Visual examples of the most recent Shift version Home, Topics, Tracking, and Settings screens.
View this figure

Principal Findings

This paper describes the development of an mHealth app, Shift, designed to support the mental health and help-seeking of junior physicians. In line with gold standard recommendations on the importance of user-centered design principles [26], the 4-phased app development process (ie, semistructured interviews, prototype development, pilot-testing, and app redevelopment) focused on a participatory approach to promote effective engagement and facilitate cognitive, affective, and behavior changes of junior physicians. Junior physicians were involved at every stage of this process through qualitative interviews, user experience workshops, and participation in pilot-testing. The aim of this approach was to create, deliver, and refine content in a way that was acceptable, effective, and engaging to end users.

Pilot-testing revealed several issues with the delivery of in-app content components to junior physicians. Although a successive, day-by-day delivery of therapeutic content has been successfully employed in a previous working population sample [21] and was generally viewed favorably by junior physicians in qualitative assessments, preliminary use data indicated that this format failed to engage junior physicians in practice. Inspection of app use data revealed that discontinuation of the 30-day challenge tended to appear around day 5, which was a generally lower rated, value-based activity. With the 30-day challenge format, users were unable to skip challenge topics or change the order of challenges, possibly facilitating the discontinuation of app use. As a time-poor, well-educated group, junior physicians may be more insistent on being able to choose their own modules from other working populations. In addition, informational sessions were underutilized in comparison to challenge content, possibly due to their less-prominent positioning within the app. Feedback on challenge and session components indicated considerable variability in the favorability ratings of the contents. This observation highlights 2 key aspects. First, even when app components are generated based on qualitative data from focus groups, their use in the real world needs to be tested. Second, simply modifying the modes of content delivery (ie, 30-day challenge structure) from one evidence-based app to suit another working population is not always successful. To meet the needs of a population of junior physicians, a new app structure needs to be developed. Therefore, in phase 4, the app was adjusted with the help of junior physicians across successive user experience workshops to enhance the overall experience and encourage engagement. The main changes included streamlining login and onboarding procedures and categorizing contents by topics, which allowed for the personal selection of modules and for an updated design and learning experience.

Strengths and Limitations

The 4-phase process emphasizes the need for customization for end users. In line with previous research, this project illustrates the role of usability testing in the development of a digital intervention tool [24,41]. Using participatory mixed methods, such as qualitative and quantitative assessments, to involve end users at all stages of the product development process was fundamental in our attempt to create a digital solution that allowed for the pursuit of multiple outcomes, such as cognitive behavioral, psychoeducational, or providing contact details to relevant specialized services [26]. Furthermore, our development process suggests that user experience and learning designers are critical in translating methods proven useful in face-to-face settings in the digital arena. The establishment of a multidisciplinary team including academics, clinicians, and digital experts helped incorporate the suggestions and feedback put forward by junior physicians into an mHealth app intervention environment.

This study had several important limitations. First, the Shift mHealth app has not been evaluated for its effectiveness to date. Future studies are required to establish whether it is indeed a useful tool to reduce or prevent the onset of common mental health symptoms in junior physicians. Similarly, it needs to be established whether help-seeking intentions or actions improve after using Shift. Second, although the increased personalization of the novel Shift version is expected to increase engagement, the freedom to choose modules may unintentionally facilitate avoidance behaviors or choice overload and thus potentially minimize exposure to beneficial content. Thus, use behaviors and outcomes should be thoroughly inspected to predict which use patterns constitute effective engagement if such a pathway indeed exists. Finally, Shift focuses on individual-level change factors. However, to address distress among junior physicians in a comprehensive manner, structural change programs including organization-level solutions are likely to be required alongside the delivery of interventions directed at an individual, such as the app [14].


The integration of new technology in the creation of workplace and personal well-being programs constitutes an easily accessible and cost-effective approach to addressing mental health concerns. Digital mental health programs provide a potential solution to engaging hard-to-reach, time-poor, and potentially help-seeking averse junior physicians in a way that does not exasperate confidentiality concerns around discussing mental health problems face-to-face. To our knowledge, Shift is the first initiative of its kind in that it aims to deliver mental health and support resources to junior physicians through unobtrusive mHealth app technology. This study describes an innovative, multiphase, and multidisciplinary user-driven design process that was undertaken to ensure a match between the app and the needs and barriers faced by junior physicians. Further research is planned to examine whether Shift proves useful for a substantial number of junior physicians.


The authors would like to thank Jennifer Chapman and Deborah Frew from the New South Wales Ministry of Health and the Shift Steering Committee for their continued support and guidance on all aspects related to the development and promotion of Shift. They are also extremely grateful to all their contacts across New South Wales’ Local Health Districts and hospitals for promoting the Shift study across their channels, to all junior physicians who volunteered to participate in the development and testing of the Shift app, and to the Black Dog Institute IT Department for their ongoing information technology and development support.

Conflicts of Interest

The development of Shift was made possible with funding from the New South Wales Ministry of Health and icare Foundation. All researchers have remained independent from the funders in the completion and submission of this work. Intellectual property for the Shift app is owned by the Black Dog Institute; however, the Shift app does not currently produce any income and the authors do not receive any financial gain from this intellectual property.

  1. Markwell AL, Wainer Z. The health and wellbeing of junior doctors: insights from a national survey. Med J Aust 2009 Oct 19;191(8):441-444. [CrossRef] [Medline]
  2. Wu F, Ireland M, Hafekost K, Lawrence D. National mental health survey of doctors and medical students. Beyond Blue.   URL: https:/​/www.​​docs/​default-source/​research-project-files/​bl1132-report---nmhdmss-full-report_web [accessed 2021-11-08]
  3. Forbes MP, Iyengar S, Kay M. Barriers to the psychological well-being of Australian junior doctors: a qualitative analysis. BMJ Open 2019 Jun 12;9(6):e027558 [FREE Full text] [CrossRef] [Medline]
  4. Medical training survey. Medical Board Aphra.   URL: [accessed 2021-11-08]
  5. Tyssen R, Vaglum P. Mental health problems among young doctors: an updated review of prospective studies. Harv Rev Psychiatry 2002;10(3):154-165. [CrossRef] [Medline]
  6. Brooks SK, Del Busso L, Chalder T, Harvey SB, Hatch SL, Hotopf M, et al. 'You feel you've been bad, not ill': sick doctors' experiences of interactions with the General Medical Council. BMJ Open 2014 Jul 17;4(7):e005537 [FREE Full text] [CrossRef] [Medline]
  7. Brooks SK, Gerada C, Chalder T. Review of literature on the mental health of doctors: are specialist services needed? J Ment Health 2011 Apr;20(2):146-156. [CrossRef] [Medline]
  8. Wallace JE. Mental health and stigma in the medical profession. Health (London) 2012 Jan;16(1):3-18. [CrossRef] [Medline]
  9. Muhamad Ramzi NS, Deady M, Petrie K, Crawford J, Harvey S. Help-seeking for depression among Australian doctors. Intern Med J 2020 Aug 24:- (forthcoming). [CrossRef] [Medline]
  10. Brooks SK, Chalder T, Gerada C. Doctors vulnerable to psychological distress and addictions: treatment from the Practitioner Health Programme. J Ment Health 2011 Apr;20(2):157-164. [CrossRef] [Medline]
  11. Fahrenkopf AM, Sectish TC, Barger LK, Sharek PJ, Lewin D, Chiang VW, et al. Rates of medication errors among depressed and burnt out residents: prospective cohort study. BMJ 2008 Feb 07;336(7642):488-491. [CrossRef]
  12. West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA 2009 Sep 23;302(12):1294-1300. [CrossRef] [Medline]
  13. Williams ES, Manwell LB, Konrad TR, Linzer M. The relationship of organizational culture, stress, satisfaction, and burnout with physician-reported error and suboptimal patient care: results from the MEMO study. Health Care Manage Rev 2007;32(3):203-212. [CrossRef] [Medline]
  14. Petrie K, Crawford J, Baker ST, Dean K, Robinson J, Veness BG, et al. Interventions to reduce symptoms of common mental disorders and suicidal ideation in physicians: a systematic review and meta-analysis. Lancet Psychiatry 2019 Mar;6(3):225-234. [CrossRef]
  15. West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med 2018 Jun;283(6):516-529 [FREE Full text] [CrossRef] [Medline]
  16. Guille C, Zhao Z, Krystal J, Nichols B, Brady K, Sen S. Web-based cognitive behavioral therapy intervention for the prevention of suicidal ideation in medical interns: a randomized clinical trial. JAMA Psychiatry 2015 Dec;72(12):1192-1198 [FREE Full text] [CrossRef] [Medline]
  17. Frögéli E, Djordjevic A, Rudman A, Livheim F, Gustavsson P. A randomized controlled pilot trial of acceptance and commitment training (ACT) for preventing stress-related ill health among future nurses. Anxiety Stress Coping 2016;29(2):202-218. [CrossRef] [Medline]
  18. Kushnir T, Malkinson R, Ribak J. Rational thinking and stress management in health workers: a psychoeducational program. Int J Stress Manag 1998;5(3):169-178. [CrossRef]
  19. Reeve A, Tickle A, Moghaddam N. Are acceptance and commitment therapy-based interventions effective for reducing burnout in direct-care staff? A systematic review and meta-analysis. Ment Health Rev J 2018 Jul 04;23(3):131-155. [CrossRef]
  20. de Korte E, Wiezer N, Bakhuys Roozeboom M, Vink P, Kraaij W. Behavior change techniques in mHealth apps for the mental and physical health of employees: systematic assessment. JMIR Mhealth Uhealth 2018 Oct 03;6(10):e167 [FREE Full text] [CrossRef] [Medline]
  21. Deady M, Johnston D, Milne D, Glozier N, Peters D, Calvo R, et al. Preliminary effectiveness of a smartphone app to reduce depressive symptoms in the workplace: feasibility and acceptability study. JMIR Mhealth Uhealth 2018 Dec 04;6(12):e11661 [FREE Full text] [CrossRef] [Medline]
  22. Ganesan AN, Louise J, Horsfall M, Bilsborough SA, Hendriks J, McGavigan AD, et al. International mobile-health intervention on physical activity, sitting, and weight: the stepathlon cardiovascular health study. J Am Coll Cardiol 2016 May 31;67(21):2453-2463 [FREE Full text] [CrossRef] [Medline]
  23. Johnson J, Devdutt J, Mehrotra S, Bhola P, Sudhir P, Sharma A. Barriers to professional help-seeking for distress and potential utility of a mental health app components: stakeholder perspectives. Cureus 2020 Feb 28;12(2):e7128 [FREE Full text] [CrossRef] [Medline]
  24. Hill C, Martin JL, Thomson S, Scott-Ram N, Penfold H, Creswell C. Navigating the challenges of digital health innovation: considerations and solutions in developing online and smartphone-application-based interventions for mental health disorders. Br J Psychiatry 2017 Aug;211(2):65-69. [CrossRef] [Medline]
  25. Inal Y, Wake JD, Guribye F, Nordgreen T. Usability evaluations of mobile mental health technologies: systematic review. J Med Internet Res 2020 Jan 06;22(1):e15337 [FREE Full text] [CrossRef] [Medline]
  26. Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
  27. de Korte EM, Wiezer N, Janssen JH, Vink P, Kraaij W. Evaluating an mHealth app for health and well-being at work: mixed-method qualitative study. JMIR Mhealth Uhealth 2018 Mar 28;6(3):e72 [FREE Full text] [CrossRef] [Medline]
  28. Melzner J, Heinze J, Fritsch T. Mobile health applications in workplace health promotion: an integrated conceptual adoption framework. Proc Technol 2014;16:1374-1382. [CrossRef]
  29. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012;Suppl:49-56. [CrossRef] [Medline]
  30. Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform 2016 Apr;60:243-251 [FREE Full text] [CrossRef] [Medline]
  31. Lyon AR, Koerner K. User-centered design for psychosocial intervention development and implementation. Clin Psychol (New York) 2016 Jun;23(2):180-200 [FREE Full text] [CrossRef] [Medline]
  32. Deady M, Peters D, Lang H, Calvo R, Glozier N, Christensen H, et al. Designing smartphone mental health applications for emergency service workers. Occup Med (Lond) 2017 Aug 01;67(6):425-428. [CrossRef] [Medline]
  33. Peters D, Deady M, Glozier N, Harvey S, Calvo RA. Worker preferences for a mental health app within male-dominated industries: participatory study. JMIR Ment Health 2018 Apr 25;5(2):e30 [FREE Full text] [CrossRef] [Medline]
  34. Petrie K, Crawford J, Deady M, Lupton D, Boydell K, Harvey S. 'The Hardest Job I've Ever Done?: a qualitative exploration of the factors affecting junior doctors' mental health and wellbeing during medical training in Australia. BMC Health; under review; Services Research 2021:- (forthcoming).
  35. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
  36. Massey OT. A proposed model for the analysis and interpretation of focus groups in evaluation research. Eval Program Plann 2011 Feb;34(1):21-28. [CrossRef] [Medline]
  37. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Human Comput Inter 2008 Jul 30;24(6):574-594. [CrossRef]
  38. Kortum P, Sorber M. Measuring the usability of mobile applications for phones and tablets. Int J Human Comput Inter 2015 Jul 31;31(8):518-529. [CrossRef]
  39. Honey P, Mumford A. Manual of Learning Styles. Berkshire: Peter Honey; 1982.
  40. Knowles M. Andragogy: An emerging technology for adult learning. In: The Modern Practice of Adult Education: From Pedagogy to Andragogy. London, UK: Cambridge Book Company; 1970.
  41. Kushniruk A. Evaluation in the design of health information systems: application of approaches emerging from usability engineering. Comput Biol Med 2002 May;32(3):141-149. [CrossRef] [Medline]

mHealth: mobile health

Edited by G Eysenbach; submitted 09.12.20; peer-reviewed by S Six, L Balcombe, A Knapp; comments to author 25.02.21; revised version received 09.04.21; accepted 21.10.21; published 02.12.21


©Isabelle Counson, Alexandra Bartholomew, Joanna Crawford, Katherine Petrie, Geetanjali Basarkod, Victoria Moynihan, Josie Pires, Rachel Cohen, Nicholas Glozier, Samuel Harvey, Samineh Sanatkar. Originally published in JMIR Formative Research (, 02.12.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.