Multiple sclerosis (MS) is a neurological condition affecting the central nervous system. More than 2.5 million people worldwide have MS with over 130,000 in the United Kingdom [1,2] at an estimated cost to the UK economy of £3.3 to £4.2 billion (US $4.5 to $5.7 billion) per annum [3]. Fatigue is one of the most common and debilitating symptoms of MS [4], experienced by over 80% of people with MS [5], and the main reason for stopping work early [6]. FACETS (Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle) is a group-based, face-to-face fatigue management program for people with MS developed by members of our team that has been shown to be effective in a national multicenter randomized controlled trial (RCT) funded by the UK MS Society [7-11]. In response to the COVID-19 pandemic, some health care professionals have been delivering the FACETS program virtually (via video conferencing) with initial participant feedback promising.

The program is delivered in 6 weekly sessions [8]. A key component of the FACETS program is the homework tasks that provide an opportunity for participants to try out what they have learned and put it into practice in their daily lives (Table 1).

Given the favorable results from the national multicenter RCT of FACETS [9-11] and subsequent rollout of the program, team members conducted a consultation on behalf of the MS Society to gather views from stakeholders (people with MS and health care professionals) about potential digital delivery models to enhance reach [12]. Self-guided web-based delivery models for fatigue management have shown promise in MS, although dropout rates have tended to be relatively high [13,14], a common issue encountered in eHealth trials [15]. Findings from the consultation indicated that stakeholders considered an online delivery model of FACETS to be of value [12], although not a direct replacement for the face-to-face version. An online e-learning package was deemed the best way to deliver a minimum viable solution—a product requiring only a limited amount of development time that is implemented with a minimal number of features to provide a basic working model with scope for future expansion and improvement. This has now been launched by the MS Society [16].

During the consultation, it also became apparent there were no high-quality free apps that could support digital completion of the FACETS homework tasks. A separate key recommendation, therefore, was to initiate a project to create a free stand-alone digital toolkit consolidating the structured homework elements of the FACETS program [12]. This project forms the focus of our paper.

Smartphone ownership and use is high in people with MS [17-19]. An app format would have the advantage of permitting on-the-go access and real-time symptom logging and use of reminders, potentially enhancing adherence to the homework tasks [12,20].

The 2018 MS Society/Nuffield Trust data and technology report presents a vision of “personalized, coordinated, and empowering care for people with MS, enabled by effective technology,” noting that “Digital transformation and the possibilities it provides have not yet been realised within the care and support people with MS access” [21,22]. A complementary mobile solution enabling the FACETS homework elements to be made interactive and portable aligns closely with recommendations from the MS Society data and technology report and action plan, particularly in the areas of having more control over care and accessible and coordinated care [21-23]. It would help to meet the aims of the UK MS Society research strategy in relation to self-management and implementation [24] and would address the third (fatigue) and fourth (self-management) James Lind Alliance research priorities for MS [25].

In their 2017 systematic review of MS apps available from US app stores, Giunti et al [26] noted there were few apps available for MS relative to other long-term conditions such as cancer and diabetes [27-29]. While there are mobile apps available for fatigue management in MS [30-33], they did not meet our requirements of being free to use. To date, only a fatigue management app for cancer, which draws upon cognitive behavioral principles, has been evaluated in a full-scale RCT [34-36]. While there are separate apps available to support self-management of MS symptoms (including diaries and symptom loggers), these did not align sufficiently with the FACETS homework elements, and most were not free.

Findings from a 2018 systematic search and scoping review by Giunti et al [37] suggested that most MS-specific apps lack features desired by people with MS. They and others have called for greater involvement of people with MS and health care professionals before digital solutions are implemented [37-40], noting the importance of understanding condition-specific factors when designing mobile health (mHealth) apps [37-39]. Developers need to consider the requirements of people with MS and possible MS symptoms (blurry vision, reduced fine motor control, cognitive impairment, and fatigue) throughout the development, prototyping, testing, and implementation of any digital solution [21-23,38,41,42].

Initial requirements for the FACETS toolkit were categorized using the MoSCoW framework [43], a simple method used across business disciplines to enable project teams and stakeholders to define requirements: must have (a necessity for meeting the specified goal), should have (beneficial but not essential for a successful product), could have (desirable but not important), and won’t have this time (future possibilities but not feasible for immediate implementation). Findings from the consultation [12] informed the initial set of draft requirements, and a stakeholder workshop and interviews with service users were conducted to explore client expectations regarding digitization and begin gathering baseline requirements [44]. An affinity diagram and personas (fictional characters that incorporate composite attributes of target users) were created and used in the early design phases [44]. A card-sorting task (involves ordering, grouping, and naming of objects or concepts) was undertaken with the research team to guide the design of the navigational structure [44]. Paper designs were sketched leading to the creation of low-fidelity wireframes and, subsequently, to the first interactive high-fidelity design prototype.

The toolkit was primarily aimed at complementing the existing FACETS face-to-face program and the MS Society online course. This meant that it needed to remain tethered to the existing materials and the way they are structured to ensure consistency. A secondary consideration was that there might be users downloading the toolkit who had not attended FACETS but who might wish to use elements of the toolkit for recording data relating to their MS and fatigue. A further consideration was that in the future the toolkit might be used to share information with health care providers [45] or expanded to include more content from the FACETS program; the toolkit might also be considered relevant for other long-term conditions [46].

Choosing the type of technology to use for the toolkit was challenging as this is a constantly evolving area. One recommendation from Beatty et al [47] in the context of an evaluation of an intervention for cancer-related distress was that future online programs should be multiplatform to facilitate access across a full range of devices. The FACETS toolkit was initially developed for Android—chosen due to its larger market share and lower anticipated cost of development compared to Apple iOS [48]. Given that a key requirement was access to the toolkit without Wi-Fi and that it would potentially need to align with the MS Society’s e-learning course [16] (in addition to the face-to-face program) [7], mobile web solutions were not considered suitable. The toolkit was developed using an agile approach following Google’s Material Design guidelines [49] and industrial best practices, with reference to the adapted technology acceptance model [50-52]. Google user interface guidelines [53] were followed when creating and using icons and widgets, and core app quality guidelines [54] were followed to ensure a baseline satisfactory user experience. Coding started in October 2018. Several prototypes were developed that incorporated all MoSCoW requirements identified as must have and should have [43]. Initial versions concentrated on implementing basic operational functionality that could then be expanded upon or altered as required.

This was a mixed methods study [55] involving quantitative approaches (MS-specific and demographic questions and the System Usability Scale [SUS] [56]) and qualitative approaches (focus groups, think-aloud protocol, and semistructured interviews). Ethical approval for this research was obtained from Bournemouth University (ref. 17430).

Participants were recruited via a local MS center (individuals who had participated in a previous study and had given permission to be contacted about future research), a local MS support group mailing list, and via an advertisement on the MS Research, Treatment, and Education website. For both the phase 1 focus groups and phase 2 supervised usability testing studies, participants who contacted the research team were emailed participant information sheets with information about the study. Inclusion criteria included being age 18 years or older, having a self-reported diagnosis of MS, experiencing fatigue impacting daily life, and being an active smartphone user (phase 1 only). See Figure 1 for schematic of phases of study.

The focus groups (phase 1) were held at a conference center in Bristol. Supervised usability sessions (phase 2) were held at a conference center in Bristol and on the Bournemouth University campus. Both venues were accessible, and taxi-booking and reimbursement of travel expenses were offered to all participants. To minimize participant burden, we obtained written informed consent via a participant agreement form on the day of the focus groups or usability sessions. In phase 2, we audiorecorded and filmed some parts of the supervised usability testing sessions. If participants did not wish to be audiorecorded or filmed, we offered a one-to-one session with notes taken as an alternative. Two copies of the agreement form were countersigned by the researcher. One copy was given to participants for their records, and one copy was retained by the researcher. The main ethical consideration related to fatigue, which is a major issue for people with MS. Focus groups and usability sessions included regular rest breaks and provision of refreshments, and we emphasized that participants could take a break or stop participating at any time. Duration of sessions was no longer than 90 minutes.

The SUS [56] is a standardized questionnaire for collecting usability evaluations of a system being tested and has been shown to have good validity and reliability [57], including in the evaluation of mobile health apps [58]. Standard scoring is between 0 and 100 [57], which the Sauro-Lewis curved grading scale [59] converts to a normative percentile score and associated grade. These grades can range from A (best imaginable on the adjective rating scale by Bangor et al [60]) to F (worst imaginable). Participant ratings on the SUS were collected during 2019. Quantitative ratings from the questionnaire were summarized using descriptive statistics.

Focus group and interview recordings were transcribed and thematically analyzed using a deductive approach that focused on the domains covered in the topic guide (focusing on design, functionality, and content). A generic qualitative approach to thematic analysis was used [61] with interresearcher interpretation. Following familiarization with the transcripts, a member of the team charted themes in a matrix. Possible enhancements and amendments to the toolkit (logged by the developer on GitHub) and field notes taken during the think-aloud sessions were also considered in the analysis process. Subsequently, another team member familiarized themselves with the transcripts and the matrix of initial themes. They developed an agreed coding scheme using an analytical framework that combined a priori issues from the original topic guide and emerging themes [62].

Focus groups comprised 8 females and 1 male; all participants had previously attended the face-to-face FACETS program. Self-reported descriptives for the usability testing participants (n=11) are presented in Table 3. All study participants consented to being recorded.

Below we summarize the key toolkit requirements suggested by focus group participants:

Below we summarize preferences for the toolkit expressed by focus group participants:

Summary statistics for the SUS scores are presented in Table 3, and frequencies of ratings for each of the 10 SUS items are presented in Figure 3. Overall, the median SUS score for the toolkit was 75 and the mean was 74.3 (SD 16.81; 95% CI 63.2 to 85.6; range 37.5-95.0). This equates to an adjective rating of good [60] and a B (70th to 79th percentile range) on the Sauro-Lewis curved grading scale [59]. The majority of participants (8/11) thought they “would like to use this toolkit frequently” (the version of the prototype they tested; SUS-Q1). Most considered the toolkit easy to use (9/11; SUS-Q3) with 55% (6/11) believing most people would learn to use it quickly (SUS-Q7). The majority (7/11, 64%) considered the toolkit’s functions to be well integrated (SUS-Q5).

Below we organized the feedback from the usability sessions encompassing general comments and those related to design, functionality, and content, mapped to the toolkit elements. We considered suggestions made for additional functionality. We also included relevant feedback elicited during the focus groups.

We implemented a notes section as suggested, text-to-speech functionality (available in recap sections) was added, and expanded help sections were developed. The concept of the personalized, customizable dashboard was developed gradually as more toolkit elements were introduced and tested over time.

In its current form the FACETS toolkit was evaluated as good on the SUS, and qualitative feedback from usability sessions indicated that users felt the toolkit would be useful. Users provided numerous suggestions for improving the toolkit in terms of design, functionality, and content. Some suggestions were implemented immediately following feedback from usability testers. Although the digital format necessitated the simplification of some aspects of the toolkit elements (to reduce cognitive demands and fatigue), it also presented opportunities to create synergies and interactivity between toolkit elements and the dashboard along with visualization and customization possibilities [63].

Findings from previous research suggest that most MS-specific apps lack features desired by people with MS [37,64], resulting in poor uptake [63]. Giunti et al [37] and others have noted the importance of taking into consideration condition-specific factors when designing mHealth apps [39]. The involvement of people with MS throughout the design, prototyping, and usability phases of the toolkit means that such considerations have played a pivotal role. Examples include providing a customizable color scheme (for those with visual difficulties), using icons where possible (to aid memory), providing signposting, and incorporating guidance (eg, buttons to indicate additional scrolled content or the availability of additional information or definitions).

There was some degree of tension between maintaining consistency with the original paper-based FACETS homework tasks and capitalizing on the possibilities afforded by the digital format. The toolkit was developed based on the homework elements of FACETS but did not provide an exact one-to-one mapping. For example, after review of the initial prototype by the team, the thought diary and thought challenge homework elements were combined into one tool as it was felt that the original screen flow was complex to navigate and could lead to frustration or disengagement by users (Table 2) [65].

As noted in the introduction, there are currently few mobile apps for fatigue management in MS. A review of mHealth in MS by Gromisch et al [66] identified 3 mHealth-based apps and 1 web-based platform that promote fatigue self-management through a variety of approaches [66]. These include cognitive behavior therapy principles (MS Energise, which draws upon the FACETS program [30-32]), gamification of energy management via stamina credits (More Stamina [67,68]), and use of validated self-assessments and medication and activity diaries (MSMonitor; web-based platform) [69]. One app (MS Telecoach) focuses on increasing physical activity levels via telemonitoring (accelerometers and self-reported fatigue) and telecoaching (advice, motivational messages, and goal setting) [70].

Similar to findings from the MS Energise [32] and More Stamina [67] usability studies, user feedback for the digital toolkit suggested a need to simplify some aspects. A cross-national qualitative study on facilitators and barriers to using mHealth tools for managing MS highlighted the importance of clear, simple design and features to enhance user accessibility and engagement [40].

Users in the MS Energise usability study suggested text-to-speech functionality would be helpful [32]. We obtained similar feedback in our early usability sessions and incorporated text-to-speech functionality into the recap sections. Users reported finding this useful, describing how it broke up content and aided concentration.

The More Stamina app incorporates gamified elements in the form of stamina credits [67]. In this study, one user said that they would like the digital toolkit to be more game-like. In previous work, we obtained mixed feedback in relation to gamification from both people with MS and health care professionals in the context of fatigue management [12]. Giunti et al [67] found that in their formative work for More Stamina, people with MS reported a preference for collaborative gamified tasks rather than competitive tasks. Untire, a fatigue management app for cancer-related fatigue, incorporates gamified elements (such as progression bars, rewards, and badges) [34]. Gamified elements could be incorporated as optional features in a future version of the digital toolkit. This will be an important area to explore further with users.

The Untire app for cancer-related fatigue allows users to invite a buddy so that they can manage their fatigue with a family member or friend [34]. Feedback from users in this study suggested that the digital toolkit could be a useful tool to help family members and friends understand more about MS fatigue.

In the MS Energise usability study, users felt the app could be particularly helpful for people with MS soon after diagnosis [32]. Similar comments were made in relation to the digital toolkit, and it was seen to offer potential as a tool that could support communication with the clinical team about fatigue and ongoing monitoring, assessment, and treatment. Most testers in this study had been diagnosed for more than 5 years, so further testing of the toolkit is needed with participants who are relatively newly diagnosed.

During field testing of MS Energise, the authors reported that a task involving the identification of unhelpful thoughts in a fictional character gave rise to unhelpful thoughts in some users about their own fatigue [30]. In our study, one user reported that seeing a visual display or heatmap of their own fatigue ratings could be demoralizing. Findings such as these underscore the importance of working closely with users throughout the development lifecycle.

Due to the power and graphical capabilities of smartphones, mobile apps can be overdesigned [42,71] leading to the paradox of choice [65], where users struggle to process an overwhelming amount of content or too many interface options. The use of online interactive prototyping tools for initial design iterations helped the project team envision their ideas more effectively than with paper alone. They also helped address some of the challenges of working in different locations and across digital and health fields. We recommend the use of an interactive prototyping tool to maximize the advantages of participatory design [72,73] in early design stages. Like Giunti et al [38], we found the use of personas (fictional characters that incorporate composite attributes of target users) a helpful way to capture and convey the varied nature of MS during preliminary stages of development [44].

A mobile platform offers scope for greater personalization than the paper-based FACETS program materials. As noted, while most participants liked the idea of being able to visualize their fatigue levels over time, one participant noted that this could be disheartening. While self-monitoring can help people with long-term conditions to feel more in control [74], it can also evoke negative emotional reactions [75,76]. In the longer term, we intend to offer customization of the dashboard enabling users to specify the information they wish to visualize or focus on so that it is relevant and meaningful to them. We also explored the best ways to support users to make sense of their self-monitoring data [76].

Suggested methods of highlighting progress included using tracking to document accessed sections and displaying progress on a central dashboard, also highlighted in our previous consultation [12]. Over a longer period of time, this might provide the user (and potentially their health care team) with the ability to track data relating to fatigue and could help to reinforce the FACETS program principles and support users in making lifestyle changes. In the longer term, the toolkit offers possibilities for longitudinal symptom and self-monitoring, data sharing, and greater integration of self-management strategies into daily life, with potential applicability to other long-term conditions. For example, fatigue is a significant and debilitating symptom in a number of neuromuscular disorders [77] and post-COVID-19 [78]. The FACETS program has recently been piloted with a group of long COVID-19 patients using a videoconference delivery format [79].

Digital tools like wearables and apps are now starting to help users manage the logistics of their long-term conditions, reminding them to take their medication [80] or helping them manage injection sites [81]. In some conditions like diabetes, significant progress has been made in developing and embedding specific technologies to support self-management [82] and empower patients [22], such as flash glucose monitoring [83]. These have yet to make much impact in MS to date [21-23,64], and concerns over the quality, sustainability, and effectiveness of some solutions remain [84]. Global app stores currently have no direct links with official medical organizations like the NHS or obligations to regulate apps on their behalf. This presents potential risks in terms of inaccurate information being made available and subsequently used within a health context [85,86]. Providing hypothetical worked-through examples within the toolkit elements provided a way of conveying key concepts and illustrating potential benefits from their use. In terms of sustainability, due to the number of devices supporting Android, fragmentation within the current market is huge. Most Android devices are currently using versions of the operating system that are more than 2 years old. Considered targeting of a significant portion of the user base is recommended (rather than trying to target every possible Android legacy device). This is especially important given that new versions of Android and Apple operating systems are released yearly to encourage consumers to upgrade their phones regularly.

The process of developing the digital toolkit was highly iterative and agile; where possible we implemented user suggestions for improvements as we went along. For example, in response to user suggestions, we incorporated the capability for real-time weather data in the activity diary (that could be time stamped at time of input or called back retrospectively) and text-to-speech functionality (available in recap sections).

Emergent technological enhancements (such as Google Voice and Google Assistant) offer further opportunities for improved personalized eHealth solutions [87] and increased engagement [88] and adherence [89]. We are currently exploring the use of voice-activated speakers and assistants such as Google Assistant or Amazon Alexa to enable the input of information via voice rather than keyboard, an innovative way to reduce screen fatigue. We have successfully prototyped this in a closed test with further development planned.

Enhancements could also encompass how a digital toolkit could be used intelligently for smart home-based monitoring and assessment via mobile devices [90]. Emerging technologies such as artificial intelligence, machine learning, and remote monitoring of condition markers could provide opportunities for more tailored and personalized care, from services to treatments [21]. For example, personalized advice and support could be offered based on user responses (in the longer term learned by artificial intelligence) to improve engagement.

Areas for further exploration might focus on the use of the toolkit for self-management and monitoring [64] and facilitating communication with health care or other service providers (such as long-term disability benefits assessors [91]) [39,75]. User suggested enhancements made to the prototype included the ability to add notes, which could be used for meetings with health care providers. Other enhancements might include integration of the toolkit with existing data streams. This could include the ability to collect biometric real-time data by using plug-in oximeters and wearable monitoring devices [92] (although improvements in accuracy are required). There is also the possibility of linking recorded data from the toolkit (such as FACETS attendance, activities logged, and goals set) to secure online sources such as the UK MS Register [93]. The toolkit’s database design and security structures allow for this possibility.

A limitation of this research is that the prototype toolkit has only been tested to date in a closed, controlled environment with a limited number of potential users (n=11). It has been argued, however, that 10 or more testers is adequate to identify the majority of usability issues [94]. A further limitation is that currently the toolkit is only available on Android.

Strengths of this study include the multidisciplinary project team, involvement of people with a range of MS types and mobility throughout, the use of mixed methods, and our agile approach to development. Our testing protocols were designed to take fatigue-related issues into account including ensuring testing locations were fully accessible, providing taxis (if required) to the usability testing location, providing refreshments, and incorporating rest breaks.

The keeping-on-track planner (section 5 of the toolkit) is currently still under development. The next phase of the study will involve people with MS remotely testing the prototype using their own Android smartphones and providing feedback via an online semistructured questionnaire (telephone interviews with a subsample). This will enable additional comments, ideas, and development glitches to be identified by a wider range of users across a broader range of mobile devices and actioned prior to release. We will next pilot the toolkit in conjunction with the FACETS program (either face-to-face or virtual) and the e-learning course (similar to the real-world usability testing of MindClimb, an app developed to support skills practice alongside group cognitive behavioral therapy for anxiety in adolescents [20]).

To date, involvement of health care professionals as stakeholders in the development process has been limited. We plan to gather feedback about the toolkit from health care professionals in our next phase of testing. Development of an iOS version has begun, with the toolkit currently in a pre-alpha stage, and this development will continue. Future plans include expanding the app to include more content from the face-to-face program to provide further scaffolding for the tools in the toolkit and self-management support. As the 2018 MS Society/Nuffield Trust data and technology report suggests [21], rather than immediately reaching for data and technology solutions, the future should be constructed within an environment that enables the potential of data and technology to give people with MS the best care possible.

We have described a mixed methods approach to the design, prototyping, and usability testing of a digital toolkit comprising the homework tasks of FACETS. This work highlights the importance of the participation of people with MS in the development cycle, working to a human-centered design methodology. Continued horizon scanning for emergent technological enhancements will enable us to identify opportunities for further improvements prior to launch.