This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Fatigue is one of the most common and debilitating symptoms of multiple sclerosis (MS), experienced by more than 80% of people with MS. FACETS (Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle) is an evidence-based, face-to-face, 6-session group fatigue management program for people with MS. Homework tasks are an integral part of FACETS and are currently undertaken in a paper-based form. Feedback from a consultation undertaken with FACETS attendees and health care professionals with experience in delivering the FACETS program suggested that being able to complete the homework tasks digitally would be desirable, potentially enhancing engagement and adherence and enabling on-the-go access to fit into busy lifestyles. Relative to other long-term conditions, there are few apps specifically for MS and, of those available, many have been developed with little or no input from people with MS.
The purpose of this mixed methods study was to create a digital toolkit comprising the homework tasks (eg, activity diary, goal planner, thought diary) of the FACETS program for people with MS, considering end users’ unique requirements throughout the design, build, prototyping, and testing stages.
Phase 1 involved the elicitation of detailed user requirements for the toolkit via 2 focus groups with previous attendees of FACETS (n=3 and n=6) and wireframing. Phase 2 involved supervised usability testing with people with MS (n=11) with iterative prototyping. The usability sessions involved going through test scenarios using the FACETS toolkit on an Android test phone with video capture and concurrent think-aloud followed by completion of the System Usability Scale (SUS) and a semistructured interview collecting feedback about design, content, and functionality.
The mean SUS score for the digital toolkit was 74.3 (SD 16.8, 95% CI 63.2-85.6; range 37.5-95), which equates to an adjective rating of good and a B grade (70th-79th percentile range) on the Sauro-Lewis curved grading scale. A number of usability and design issues (such as simplifying overall screen flow to better meet users’ needs) and suggestions for improvements (such as using location-based services and displaying personalized information and progress via a central dashboard) were addressed and implemented during the usability testing cycle.
This work highlights the importance of the participation of people with MS across the entire development cycle, working to a human-centered design methodology to enable a considered and MS-centered solution to be developed. Continued horizon scanning for emergent technological enhancements will enable us to identify opportunities for further improvements to the FACETS toolkit prior to launch. The toolkit supports self-monitoring and management of fatigue and has potential applicability to other long-term conditions where fatigue is a significant issue.
Multiple sclerosis (MS) is a neurological condition affecting the central nervous system. More than 2.5 million people worldwide have MS with over 130,000 in the United Kingdom [
The program is delivered in 6 weekly sessions [
Overview of the FACETSa program.
Session no. | Session title | Homework elements |
1 | What is MSb-related fatigue? | Activity and fatigue diary; energy measure |
2 | Opening an energy account | Rest, activity, and sleep planner |
3 | Budgeting energy and smartening up goals | Goal-setting exercise |
4 | Stress response; cognitive behavioral model | Fatigue thought diary |
5 | Putting unhelpful thoughts on trial | Thought challenge sheet |
6 | Recapping and taking the program forward | Keeping-on-track planner |
aFACETS: Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle.
bMS: multiple sclerosis.
Given the favorable results from the national multicenter RCT of FACETS [
During the consultation, it also became apparent there were no high-quality free apps that could support digital completion of the FACETS homework tasks. A separate key recommendation, therefore, was to initiate a project to create a free stand-alone digital toolkit consolidating the structured homework elements of the FACETS program [
Smartphone ownership and use is high in people with MS [
Everywhere I go, I’ve got my phone. If I’ve got a few minutes, I sit and fill it in and if it’s fresh and current, I wouldn’t fill in paperwork. Even on the course, I’d fill in the paperwork the night before, or the morning I was coming to the class. But if I had it on my phone, I’d be more inclined to fill it in.
The 2018 MS Society/Nuffield Trust data and technology report presents a vision of “personalized, coordinated, and empowering care for people with MS, enabled by effective technology,” noting that “Digital transformation and the possibilities it provides have not yet been realised within the care and support people with MS access” [
In their 2017 systematic review of MS apps available from US app stores, Giunti et al [
Findings from a 2018 systematic search and scoping review by Giunti et al [
Initial requirements for the FACETS toolkit were categorized using the MoSCoW framework [
The toolkit was primarily aimed at complementing the existing FACETS face-to-face program and the MS Society online course. This meant that it needed to remain tethered to the existing materials and the way they are structured to ensure consistency. A secondary consideration was that there might be users downloading the toolkit who had not attended FACETS but who might wish to use elements of the toolkit for recording data relating to their MS and fatigue. A further consideration was that in the future the toolkit might be used to share information with health care providers [
Choosing the type of technology to use for the toolkit was challenging as this is a constantly evolving area. One recommendation from Beatty et al [
This was a mixed methods study [
Participants were recruited via a local MS center (individuals who had participated in a previous study and had given permission to be contacted about future research), a local MS support group mailing list, and via an advertisement on the MS Research, Treatment, and Education website. For both the phase 1 focus groups and phase 2 supervised usability testing studies, participants who contacted the research team were emailed participant information sheets with information about the study. Inclusion criteria included being age 18 years or older, having a self-reported diagnosis of MS, experiencing fatigue impacting daily life, and being an active smartphone user (phase 1 only). See
Schematic of study phases. FACETS: Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle; pwMS: people with multiple sclerosis; SUS: System Usability Scale.
The focus groups (phase 1) were held at a conference center in Bristol. Supervised usability sessions (phase 2) were held at a conference center in Bristol and on the Bournemouth University campus. Both venues were accessible, and taxi-booking and reimbursement of travel expenses were offered to all participants. To minimize participant burden, we obtained written informed consent via a participant agreement form on the day of the focus groups or usability sessions. In phase 2, we audiorecorded and filmed some parts of the supervised usability testing sessions. If participants did not wish to be audiorecorded or filmed, we offered a one-to-one session with notes taken as an alternative. Two copies of the agreement form were countersigned by the researcher. One copy was given to participants for their records, and one copy was retained by the researcher. The main ethical consideration related to fatigue, which is a major issue for people with MS. Focus groups and usability sessions included regular rest breaks and provision of refreshments, and we emphasized that participants could take a break or stop participating at any time. Duration of sessions was no longer than 90 minutes.
This phase involved the elicitation of detailed user requirements and wireframing. Two focus groups (n=3 and n=6) with previous attendees of the FACETS face-to-face program were held to gather feedback about requirements and preferences for the toolkit (
Participants (n=11) were asked to use elements of the toolkit and complete up to 2 specific test scenarios (lasting 30 minutes in total) on a supplied Android mobile phone (
Mapping of FACETSa sessions to the toolkit elements and usability test scenarios.
Homework task in face-to-face program | Toolkit element | Test scenarios | Participants (1-11) who tested element |
Week 1: activity and fatigue diary; energy measure | Activity diary | Add one or more activities (eg, vacuuming, swimming, making breakfast, gardening) performed recently. Make a change to the activity just added, and view it to check that the information was amended correctly. | 1-5 |
Week 2: rest, activity, and sleep planner | Rest and sleep routine | Thinking about your wake-sleep routine, add a wind-down alarm (for when typically planning to start getting ready for bed) and a wind-up alarm (for when typically planning to start to wake up in the morning). Then create one or more rest periods during the day, setting the duration (scheduling for times when you might wish to take a rest during a typical day). Test out a few alarm options and see if they work correctly. | 1-3 |
Week 3: goal-setting exercise | Goal planner | Create one or more SMART (specific, measurable, achievable, realistic, time for review) goals (a lifestyle change you would like to make). Some possible ideas for areas for change could be exercise routines, relaxation practice, incorporating rest periods, and establishing sleep-wake routines. | 4-9 |
Week 4: fatigue thought diary | Thought diary | Think about a situation that triggered strong emotions and unhelpful thoughts related to fatigue. Describe the situation in the thought diary along with up to 3 unhelpful thoughts, and select the emotions experienced at the time. Then rate strength of belief for each unhelpful thought and for the accompanying emotions. | 10-11 |
Week 5: thought challenge sheet | Thought diary | Select a thought to challenge. Start off by identifying any unhelpful thinking styles, think of one or more alternative thoughts, and rate strength of belief. Then rerate strength of belief in the original thought, and rerate the strength of the associated emotions. | 10-11 |
Week 6: keeping-on-track planner | Keeping-on-track planner | Help the user maintain momentum: complete a plan that focuses on the next 3 months (this element is still under development). | —b (element not yet developed) |
aFACETS: Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle.
bNot applicable.
Dashboard and homework element screenshots from working prototype v0.0.3.
The SUS [
Focus group and interview recordings were transcribed and thematically analyzed using a deductive approach that focused on the domains covered in the topic guide (focusing on design, functionality, and content). A generic qualitative approach to thematic analysis was used [
Focus groups comprised 8 females and 1 male; all participants had previously attended the face-to-face FACETS program. Self-reported descriptives for the usability testing participants (n=11) are presented in
Self-reported descriptives for usability testing participants (phase 2; n=11).
Characteristic | Value | ||
|
|||
|
Male | 4 (36) | |
|
Female | 7 (64) | |
Age (years), mean (SD), range | 49 (8.41) 34-62 | ||
|
|||
|
Relapsing remitting | 8 (73) | |
|
Secondary progressive | 1 (9) | |
|
Primary progressive | 2 (18) | |
|
6.8 (2.5) 2-9c | ||
|
1: mild symptoms that don’t limit activity | —d | |
|
2: noticeable symptoms with mild, small impact | 2 (18) | |
|
3: limitations on activities of daily living | — | |
|
4: interferes with walking, can walk 300-500 m | — | |
|
5: can walk 100-200 m but often uses a stick | — | |
|
6: needs a stick or single crutch | — | |
|
7: needs 2 canes or walker to walk 20 m | 3 (27) | |
|
8: wheelchair is main form of mobility; can move from wheelchair without help | 2 (18) | |
|
9: wheelchair is main form of mobility; help needed to move with wheelchair | 2 (18) | |
|
|||
|
1-5 | 3 (27) | |
|
6-10 | 2 (18) | |
|
11-15 | 3 (27) | |
|
16-20 | 2 (18) | |
|
>20 | 1 (9) | |
|
|||
|
Working full time (>30 hours per week) | 4 (36) | |
|
Unable to work | 3 (27) | |
|
Retired | 4 (36) | |
|
|||
|
Never use | 1 (9) | |
|
Use a few | 4 (36) | |
|
Use a lot | 6 (55) | |
|
|||
|
Yes | 7 (64) | |
|
No | 4 (36) | |
|
|||
|
Median (range) | 75 (37.5-95) | |
|
Mean (SD) [95% CI] | 74.3 (16.81) [63.2, 85.6] | |
|
|
||
|
|
A/A+ | 5 (45) |
|
|
B | 1 (9) |
|
|
C | 2 (18) |
|
|
D | 2 (18) |
|
|
E | 1 (9) |
aMS: multiple sclerosis.
bAPDDS: Adapted Patient-Determined Disease Steps.
cPossible scores on the APDDS scale range from 0-10 corresponding to 11 ordinal levels of functioning. However, 2 participants gave ratings indicating they perceived their functioning to fall between ordinals (1 participant between 7-8 and another participant between 8-9) and for the summary statistics these were scored as 7.5 and 8.5, respectively.
dNot applicable.
eFACETS: Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle.
fSUS: System Usability Scale.
Below we summarize the key toolkit requirements suggested by focus group participants:
Should be a “tool to help rather than a time-consuming task”
Suitable for those unfamiliar with FACETS
Important to include positive aspects
Include self-monitoring feedback (eg, dashboard and graphs)
Below we summarize preferences for the toolkit expressed by focus group participants:
App should be freely available at app stores
Reminder (eg, reminders to take rests) and note functions would be useful
Would like relaxation module to be included
Linking with Alexa is a good idea
Summary statistics for the SUS scores are presented in
System Usability Scale—item response frequencies.
Below we organized the feedback from the usability sessions encompassing general comments and those related to design, functionality, and content, mapped to the toolkit elements. We considered suggestions made for additional functionality. We also included relevant feedback elicited during the focus groups.
Overall, users found the app relatively easy to navigate, liked its look and feel, and felt that they would quickly get used to the system.
I did like it and I liked the colors because they are bright but they’re not too bright, it was quite, it looked quite vivid I think it is fair to say, which is good. I think it was quite easy to navigate round, I mean once you have got the basic, you know, the three dots and the three lines, from there you can pretty much go anywhere and yes I think it was quite good
I found my way around easy. I think initially I just had to get to grips with it. Just to look where things were but...I like the menu structure, it was easy to follow.
However, sometimes users noted difficulty in keeping track of where they were in the app, and early versions of the recap section were found to be confusing.
I’m quite visual and I couldn’t picture the structure of where I was in the...you know...hierarchy of areas, kind of thing, so that was kind of confusing because I then couldn’t then think “Oh I need to skip that bit.”
Users wanted content and information to be kept brief and easy to understand, without unnecessary repetition and with options to skip content. As one user noted in a focus group in phase 1, it should be “a tool to help rather than a time-consuming task.” Users were in favor of the inclusion of video in addition to audio and text.
You have to be careful not to over...put so much information that it becomes overwhelming, that you can analyze down to so much and you think I can’t think about this anymore and you put too much into it.
I don’t want to be spending too much time doing it so I’m missing out on doing things...
Yes you know I am ok reading text but I don’t want to read too much, so you know it is balancing it between video and text.
Users liked the concept of section-based onboarding and said that prompts and examples were useful, particularly for those with fatigue and cognitive issues. They noted the importance of ensuring reminders are neutral and nonjudgmental in tone.
Yes and you know people with MS we do sometimes get flustered and forget how to do things, so a reminder every now and again, the option to be able to go back and actually see how I do this.
Yes I think...if somebody is struggling a bit and it keeps saying, “You should be doing this.” But if it said, “Would you like to go?” that, that seems a bit more neutral.
One user said they would like the app to “look more like a game” (P2011, Usability sessions). The app was seen to be a helpful self-monitoring tool that would be useful for describing fatigue levels, activity patterns, and symptoms to an individual’s clinical team. It was noted that it could also include content that would help family and friends understand more about MS fatigue and its impact.
As well as having the ability to be able to explain to someone who doesn’t have MS, “Do you know what? Watch this video on this app that I’ve got.”
Users felt that the app could be a useful tool not only alongside the FACETS program but also before or after the program and could be used to gather information prior to a referral by a clinical team for fatigue.
But anyway, it would have been really, really good to have had an app with videos or animations [when newly diagnosed] to talk through the premise of FACETS, almost maybe as a precursor to attending the course.
Yeah, because my stuff at the moment goes into spreadsheets and when I have to phone up to say, “Oh, I think I am having a relapse,” they ask you a whole bunch of questions: “Have you done this?” “Have you done that?” “When was the last episode you had?” “What is your fatigue like?” All these questions it’s all on a little Excel sheet on my phone at the moment but this will be a bit more of a proper user interface to get that kind of stuff....
I think for me this is a really hugely useful parallel tool alongside the course or, kind of, after the course
I mean it would be brilliant for me, I went on the FACETS course a really long time ago, or it feels like a really long time ago now, so to have all of that information would be really good to refresh my memory.
Users noted that the app could be useful for those with other conditions where fatigue is a symptom.
I think it could be useful...I think it could be infinitely useful for all people with MS, but not just MS, all people who have fatigue, who have a condition that is...causes fatigue.
The activity diary was seen to be a helpful way of enabling users to consider activity and fatigue patterns.
Okay, so yes there, that’s quite good for monitoring because actually then you could start to build up a pattern of what makes you feel really fatigued, couldn’t you?
So it’s as if you can put something in so you don’t overbook yourself you know...and my thing is that I am supposed to have two rest days a week you know, it’s like reminding people it’s fine, rest is part of management really.
Enabling the user to choose from a prepopulated activity list (and eventually using predictive text suggestions) rather than manual entry was viewed as a useful time-saving feature. It would also make a breakdown of activity type possible.
I think... I think more is probably, I know it’s difficult to put in more, but in terms of fatigue, it’s often the quite nuanced type of activity that is worse or better... it wouldn’t be particularly helpful to have the broad categories because actually work means doing so many different types of things that um...
The use of location-based services to pull and record relevant information, such as the outside temperature (heat often adversely affects MS fatigue), was suggested by a focus group participant.
P101: ...if it can be linked into the weather, like today it is humid so we have got to think take the shade, keep cool.
Design changes implemented based on usability feedback included implementing swipe to refresh layouts, updating the calendar view originally used to reduce background thread load, fixing a problem whereby activities entered between 12:00 AM and 1:00 AM were not displaying, and revising the layout to improve flow. Location-based suggestions and the addition of predictive text functionality were implemented.
Functionality issues reported included the variable quality of some of the alarm tone sounds included as default on the test phone.
Participant: ...and I wake up to the sound of birds, which is nice, it’s not...
Interviewer 1: Yeah, I was looking at those and they are....they are much more...
Participant: It is nicer than a belting alarm...
Concern was also expressed by users over the initial layout and legibility of the clock display and rest, wind-up, and wind-down periods chosen (
Yeah so if I’m looking at that, right ok, especially with my eyesight problems, I’ve got to read through this whole list to find the thing I want. You’ve got icons next to it, so sleep and wake has got a little cog and half-moon icon next to it but if that...what I mean is if that was bigger and maybe colorful then I could go to it quicker and say right that’s what I want to go back and have a look at my sleep patterns.
We amended the visuals to include the duration of wind-up and wind-down periods (
Participants considered the opportunity to set goals a very useful part of the face-to-face program. Suggestions were made to improve user understanding within a screen-based format by offering the ability to view hypothetical worked examples to help convey the concept of SMART (specific, measurable, achievable, and realistic with time for review) goals.
...if you click, 'yes I understand that', then yes it goes on to the next bit or if 'no' then the health care professional would tell you what SMART means because you don’t need another MS person telling you that.
Users noted that having the ability to select personalized, customizable reminders for entered goals and review and update them could increase engagement with the toolkit.
The option to note a goal quickly and come back to it later to apply SMART criteria was implemented following user feedback. Case study examples were added to enhance understanding of the concept of SMART goals as this can be difficult for some to understand. Other changes made to improve functionality included adding in pop-up logic to provide guidance to the user when 'no' was selected in the 'realistic' field of the SMART criteria and revising the layout to make it more consistent with the activity diary.
As can be seen in
Yes I think this thought summary that’s useful because, you had to think about how you were thinking in reflection to the impact it’s had on you, and I found that quite useful to think, because you do exhaust yourself worrying about everything.
Well, because I’d already rated it once, that was the original frustration of what that incident or whatever was causing me. I don’t think, by the time I’d got to reflecting on the thing and gone back and reviewed what was going on and how I could approach it differently or this, that and the other, I didn’t think it made me any less frustrated or angry from the point that I’d already rated it.
The screen flow and structure was simplified further (eg, by removing the requirement to provide ratings for strength of belief in alternative thoughts).
Adding a notes function (to enable the quick addition of general pieces of information or thoughts for later expansion or reflection) was suggested as a way of reminding users about historical health-related information (eg, about new symptoms that had emerged, ongoing concerns, other scheduled appointments) that could be useful at MS review consultations (sometimes held a year apart).
When you are newly diagnosed you just, there is this barrage of questions and they don’t come to you when you are sat in front of the consultant or even the nurse and you go home and you think, “Oh god I wish I had asked that.” Oh do you know what I’ve just had a thought, maybe having a notes app, a notes section to write down, you know, notes about your [... ] fatigue or notes about anything else MS-related that you need to talk to your nurse or doctor about, that might be nice.
I use the notes in my phone a lot, so, er, information that I am told...almost everything, if it doesn’t go into my phone at the time, it gets lost. For instance, I have got the swimming timetable, which I would then put into my fatigue management diary...
Displaying progress on the dashboard and allowing the user to apply custom queries to data entered (such as their most fatiguing or enjoyable activities) were suggested as ways to enhance engagement. A customizable dashboard could allow the user to create a display containing summary information most relevant and useful to them.
Yes because I would refer back to this and kind of look at what I have done and gauge my fatigue levels.
It would be good to have one that was a weekly... so you could see what days you were particularly, you know, it’s like actually I need...I have got a lot that I am trying to achieve on that day, so I need to know that I need to have a rest on that day or before....
Although most participants liked the idea of being able to visualize their fatigue levels over time, one participant felt that this could be disheartening. A customizable dashboard would mean that this feedback could be turned off if a user did not find it helpful.
Yeah if you could turn it on and off or something so you wouldn’t have to see it if you didn’t want to see it or something, yeah it could, if you had that over several weeks and you’re just looking at red, it could be sort of like a bit demoralizing yes.
Users liked the text-to-speech feature that we included in the recap content.
The fact that you watch or even [with] text you can have somebody saying it at the same time, I find that really helpful for myself, because just reading stuff sometimes it doesn’t, it doesn’t go in properly.
It helps there was a voice saying the words as well because it breaks it up.
The text was really easy to read, but having it read to me was really useful.
Feedback from users in the initial usability sessions suggested they found the synthesized text-to-speech voice overly robotic.
Then as you went on she turned more robotic and more... yes, less human and I think it would be better to be told by somebody sounding a bit more personal. Whether it’s a man or a woman it doesn’t really matter I suppose, it’s just yeah the right person, but yeah.
As the voice was device-dependent, we only had a certain degree of control over it, but we did find slowing it down a little made it sound more natural.
Usability testers considered the possible integration of voice-activated speakers as a positive method of engagement potentially saving time and energy and requiring less dexterity. In some of the sessions, we gave a demo of inputting a diary entry using Google Assistant, and this was well received.
Yeah. That’s exactly what I would like. Something like that I would use.
Ensuring that help and support functions on the toolkit accommodated the needs of those unfamiliar with the FACETS program was seen as important. Different types and levels of information would be required for those using the toolkit in conjunction with the FACETS face-to-face program or the e-learning course versus those using it independently. The need for this information to be structured more logically to improve the user journey was highlighted in early usability sessions where users experienced difficulties navigating the recap menu.
So yeah, if I clicked on the activity diary and it says recap, I think I don’t know what that is.
We implemented a notes section as suggested, text-to-speech functionality (available in recap sections) was added, and expanded help sections were developed. The concept of the personalized, customizable dashboard was developed gradually as more toolkit elements were introduced and tested over time.
In its current form the FACETS toolkit was evaluated as good on the SUS, and qualitative feedback from usability sessions indicated that users felt the toolkit would be useful. Users provided numerous suggestions for improving the toolkit in terms of design, functionality, and content. Some suggestions were implemented immediately following feedback from usability testers. Although the digital format necessitated the simplification of some aspects of the toolkit elements (to reduce cognitive demands and fatigue), it also presented opportunities to create synergies and interactivity between toolkit elements and the dashboard along with visualization and customization possibilities [
Findings from previous research suggest that most MS-specific apps lack features desired by people with MS [
There was some degree of tension between maintaining consistency with the original paper-based FACETS homework tasks and capitalizing on the possibilities afforded by the digital format. The toolkit was developed based on the homework elements of FACETS but did not provide an exact one-to-one mapping. For example, after review of the initial prototype by the team, the thought diary and thought challenge homework elements were combined into one tool as it was felt that the original screen flow was complex to navigate and could lead to frustration or disengagement by users (
As noted in the introduction, there are currently few mobile apps for fatigue management in MS. A review of mHealth in MS by Gromisch et al [
Similar to findings from the MS Energise [
Users in the MS Energise usability study suggested text-to-speech functionality would be helpful [
The More Stamina app incorporates gamified elements in the form of stamina credits [
The Untire app for cancer-related fatigue allows users to invite a buddy so that they can manage their fatigue with a family member or friend [
In the MS Energise usability study, users felt the app could be particularly helpful for people with MS soon after diagnosis [
During field testing of MS Energise, the authors reported that a task involving the identification of unhelpful thoughts in a fictional character gave rise to unhelpful thoughts in some users about their own fatigue [
Due to the power and graphical capabilities of smartphones, mobile apps can be overdesigned [
A mobile platform offers scope for greater personalization than the paper-based FACETS program materials. As noted, while most participants liked the idea of being able to visualize their fatigue levels over time, one participant noted that this could be disheartening. While self-monitoring can help people with long-term conditions to feel more in control [
Suggested methods of highlighting progress included using tracking to document accessed sections and displaying progress on a central dashboard, also highlighted in our previous consultation [
Digital tools like wearables and apps are now starting to help users manage the logistics of their long-term conditions, reminding them to take their medication [
The process of developing the digital toolkit was highly iterative and agile; where possible we implemented user suggestions for improvements as we went along. For example, in response to user suggestions, we incorporated the capability for real-time weather data in the activity diary (that could be time stamped at time of input or called back retrospectively) and text-to-speech functionality (available in recap sections).
Emergent technological enhancements (such as Google Voice and Google Assistant) offer further opportunities for improved personalized eHealth solutions [
Enhancements could also encompass how a digital toolkit could be used intelligently for smart home-based monitoring and assessment via mobile devices [
Areas for further exploration might focus on the use of the toolkit for self-management and monitoring [
A limitation of this research is that the prototype toolkit has only been tested to date in a closed, controlled environment with a limited number of potential users (n=11). It has been argued, however, that 10 or more testers is adequate to identify the majority of usability issues [
Strengths of this study include the multidisciplinary project team, involvement of people with a range of MS types and mobility throughout, the use of mixed methods, and our agile approach to development. Our testing protocols were designed to take fatigue-related issues into account including ensuring testing locations were fully accessible, providing taxis (if required) to the usability testing location, providing refreshments, and incorporating rest breaks.
The keeping-on-track planner (section 5 of the toolkit) is currently still under development. The next phase of the study will involve people with MS remotely testing the prototype using their own Android smartphones and providing feedback via an online semistructured questionnaire (telephone interviews with a subsample). This will enable additional comments, ideas, and development glitches to be identified by a wider range of users across a broader range of mobile devices and actioned prior to release. We will next pilot the toolkit in conjunction with the FACETS program (either face-to-face or virtual) and the e-learning course (similar to the real-world usability testing of MindClimb, an app developed to support skills practice alongside group cognitive behavioral therapy for anxiety in adolescents [
To date, involvement of health care professionals as stakeholders in the development process has been limited. We plan to gather feedback about the toolkit from health care professionals in our next phase of testing. Development of an iOS version has begun, with the toolkit currently in a pre-alpha stage, and this development will continue. Future plans include expanding the app to include more content from the face-to-face program to provide further scaffolding for the tools in the toolkit and self-management support. As the 2018 MS Society/Nuffield Trust data and technology report suggests [
We have described a mixed methods approach to the design, prototyping, and usability testing of a digital toolkit comprising the homework tasks of FACETS. This work highlights the importance of the participation of people with MS in the development cycle, working to a human-centered design methodology. Continued horizon scanning for emergent technological enhancements will enable us to identify opportunities for further improvements prior to launch.
User requirements topic guide (phase 1).
Interview topic guide (phase 2).
Fatigue: Applying Cognitive Behavioral and Energy Effectiveness Techniques to Lifestyle
mobile health
must have, should have, could have, and won’t have
multiple sclerosis
randomized controlled trial
specific, measurable, achievable, and realistic with time for review
System Usability Scale
The authors would like to thank all participants contributing to the participatory design sessions and the MS Research, Treatment, and Education charity for supporting recruitment. This research forms part of a Higher Education and Innovation Funding project.
ST and PWT conceived the study. All authors contributed to the study design. BF and AP performed the data collection and requirements generation in phase 1; AP did the same in phase 2. AP and ST analyzed and interpreted the data and conducted qualitative data analysis. ST and AP led the manuscript preparation. All authors contributed to the manuscript preparation, participated in final approval, and take responsibility for the integrity and accuracy of the data analysis.
None declared.