This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
The use of digital therapeutics (DTx) in the prevention and management of medical conditions has increased through the years, with an estimated 44 million people using one as part of their treatment plan in 2021, nearly double the number from the previous year. DTx are commonly accessed through smartphone apps, but offering these treatments through additional platforms can improve the accessibility of these interventions. Voice apps are an emerging technology in the digital health field; not only do they have the potential to improve DTx adherence, but they can also create a better user experience for some user groups.
This research aimed to identify the acceptability and feasibility of offering a voice app for a chronic disease self-management program. The objective of this project was to design, develop, and evaluate a voice app of an already-existing smartphone-based heart failure self-management program,
A voice app version of
Common themes were identified in the results of the usability study, which involved 8 participants with heart failure. Almost all participants (7/8, 88%) were satisfied with the voice app and felt confident using it, although half of the participants (4/8, 50%) were unsure about using it in the future. Six main themes were identified: changes in physical behavior, preference between voice app and smartphone, importance of music during voice app interaction, lack of privacy concerns, desired reassurances during voice app interaction, and helpful aids during voice app interaction. These findings were triangulated with the quantitative data, and it concluded that the main area for improvement was related to the ease of use; design changes were then implemented to better improve the user experience.
This work offered preliminary insight into the acceptability and feasibility of a
The prevalence of heart failure (HF) continues to be on the rise as more people are surviving cardiovascular disease [
HF creates a burden not only on health care resources and expenditures [
Given these encouraging results, a longitudinal study measuring adherence was also performed on the
Voice apps are an emerging technology in the health care field, ranging from integration with in-clinic registration processes [
Guided by the findings from the literature review and coupled with a user-centered design (UCD) process, we sought to design, develop, and evaluate a voice app for the smartphone-based HF self-management program
The
The
The usability study was performed in Toronto General Hospital, part of UHN, and was approved by UHN Research Ethics Board (19-5051.2). Interested participants were asked to sign a written consent form before being enrolled in the study. They were reminded that (1) they can contact the study coordinator at any point should they have questions, (2) their decision to participate or not has no impact on their clinical care in any way, and (3) they are free to withdraw from the study at any point with no impact on their clinical care. All information collected pertaining to observations, semistructured interviews, and questionnaire responses was identified by study number only. Neither participants’ name nor any identifying information will be used in any publication or presentation. The participants received a CAD $25 (US $18.34) Visa gift card for participating in the study.
The first phase of the UCD process, identifying user needs and ideation of the
The second phase of the UCD process involved using the information gathered in the first phase to develop and deploy a prototype of the
The design process was guided by a user-centered design process; details describing what was done during each phase are included.
Design requirements were created before developing the
The architecture of the
Voice apps consist of both a VUI and app logic. When the user speaks, the VUI invokes the voice app, and a request is created and processed in the context of the voice app’s interaction model using tools such as machine learning, natural language understanding, and automatic speech recognition. Once the interaction model has processed the speech, a request is sent to the app logic, which provides a response back to the user. For the purposes of this project, an Alexa-hosted Skill built using the Amazon Alexa Skills Kit [
Finally, the verification and validation of the
The purpose of the usability study was to investigate the potential of VUI devices as an option for patients to access the
A total of 8 participants were recruited for this study, and all of them used
Participants who volunteered attended a single, 1-hour session with the study coordinator after their scheduled appointment and were asked to complete four main tasks: (1) inputting measurements on the
Both quantitative and qualitative data were gathered during the usability study and triangulated when identifying the main conclusions from the results. The SUS questionnaire was analyzed as per standard protocol [
The mixed methods triangulation design followed a convergence model, with a larger emphasis being placed on the qualitative results. The findings from the SUS questionnaire were used to validate the main conclusions gathered from the interviews and field notes. The rationale for this approach was due to the usability study design; because a small sample size was needed to reach saturation, the interpretations from the quantitative data were used to support the qualitative findings.
A voice app prototype of
A high-level overview of the conversation was first mapped out and created (
An interaction model was then created to implement the conversational flow described earlier, and it consisted of intents, training phrases, and slot types. An intent was created for each piece of required information: weight, blood pressure, heart rate, and symptom-related questions. Within each intent, training phrases were added and consisted of predictable utterances the user may say to record their responses. Within each training phrase, the most important pieces of data were identified and labeled as a slot type (weight, blood pressure, and heart rate were categorized as
A high-level description of the conversational flow for the Medly voice app. “Milestones” are outlined and signify an important event.
A usability study was performed with 8 participants with HF from UHN’s Ted Rogers Center of Excellence for Heart Failure clinic. Saturation was successfully achieved with the sample size because of the common themes that emerged across the 8 sessions. After interacting with the voice app, the participants filled out the SUS questionnaire to describe their experience. The average SUS score was 92, out of 100 (SD 5.2), ranking the voice app in the 98th percentile based on previous studies. According to these data, most participants were satisfied with the design and development of the voice app. Results for the positive and negative statements of the SUS questionnaire are shown in
Looking at the positive attribute statements, almost all participants (7/8, 88%) felt confident using the app and thought it was easy to use. This was further supported by the responses to the negative attribute statements, with the most popular response being “strongly disagree.” However, when asked whether they would use the voice app frequently, there was a greater divide in opinion, with the most popular response (3/8, 38%) being neutral.
(A) Data showcasing the participant results for the questions in the System Usability Scale (SUS) relating to positive attributes of the Medly voice app system. (B) Data showcasing the participant results for the questions in the SUS relating to the negative attributes of the Medly voice app system.
Overall, these findings showed promise when determining user acceptability because of the overall positive responses from the SUS questionnaire. Triangulating these results with the field notes and interview findings indicated areas for improvement regarding the ease of use. Although most of the responses were “strongly agree” or “strongly disagree,” 3 of the 10 statements had more variation: “I think that I would like to use the voice app frequently,” “I found the various functions in the voice app were well integrated,” and “I thought there was too much inconsistency in the voice app.” This can potentially be explained using 2 observations that were commonly seen during the user sessions. First, the voice app had difficulties understanding measurements when users said each digit separately for their weight, blood pressure, and heart rate (Table S1 in
The observational notes and semistructured interview responses were analyzed to (1) identify and describe common themes and (2) create a list of potential errors that could occur while the user is interacting with the app. Both these deliverables provided insights and helped shape the redesign and redevelopment of the voice app. Six main themes were identified: (1) changes in physical behavior, (2) preference between voice app and smartphone, (3) importance of music during voice app interaction, (4) lack of privacy concerns, (5) desired reassurances during voice app interaction, and (6) helpful aids during voice app interaction.
One of the more notable themes was the observed change in behavior when the participants interacted with the voice app; most participants showed signs of nervousness and looked tense. However, as the conversation progressed, there was a noticeable difference in their demeanor, as they became more relaxed once they better understood the conversational flow. Some voice app features helped the users navigate through their session, making for a more pleasant experience, such as having music play in the background so that they did not feel rushed through the interaction and having the voice app reiterate measurements back to them to ensure that the device correctly recorded the values. Because the users were accustomed to seeing visual feedback of their measurements on their smartphones, they preferred having some type of feedback (verbal or visual) to ensure that the voice app heard them correctly. The participants heavily relied on the instruction card as an aid, and those who followed it closely were the most successful in completing the interaction. The use of music, reiteration feature, and instruction card was observed to be helpful during the interactions, and this was confirmed with the participants during the interview sessions. Insights were also provided as to why some patients found the smartphone device difficult to use, and the responses typically revolved around the idea of convenience. Some participants (4/8, 50%) found that inputting their data through a conversation (voice app) was easier for them than using their smartphone device. One of the participants said that they use their smartphone only for the Medly app and otherwise have it turned off. Finally, concerns about privacy were brought up only by 25% (2/8) of participants, indicating it may not be a prominent barrier to use as originally predicted. Further details describing these themes are outlined in Table S2 in
As mentioned previously, a list of potential risks, related to the interaction of the user with the voice app, was identified. The severity of each risk was assessed based on its likelihood and consequence. Mitigation strategies for each risk were developed and implemented in the voice app design during the redevelopment work. These data can be seen in Table S1 in
The prototype built for the usability study was further developed to meet all the design requirements of the
Having a screen display on the device meant that the users would be able to visually see instructions and measurements and also have the option of responding to yes or no questions using the touch screen. Although some demographics struggle to use touch screen, these interaction features are optional for the user. Although we predicted that the visual feedback will be helpful for most users and that having a bigger screen will be easier than the smartphone, the users will still be able to interact with the app using only their voice if they find it to be the easiest and simplest mode of interaction for them. Each screen was designed to only include the information the user needed to avoid any confusion. A consistent design theme, similar to the one used in the
The incorporation of a screen display for the
Screen designs showcased through images A-H, representing different events that occur in the conversation.
The conversational script used in the preliminary design stayed the same during the redevelopment work. Findings from the usability study helped inform which parts of the conversation were most prone to error and needed to be changed. More training phrases and symptom confirmations were new additions to the conversation, and the summary of the interaction is shown in
A high-level description of the conversational flow for the final version of the Medly voice app. A new milestone has been added to describe the Medly voice app, which confirms the responses to the symptom-related questions that were answered with “yes.”.
The software architecture and
This manuscript describes the UCD approach that was used to develop a voice app experience for HF management. Accessing chronic disease management programs through voice apps has the potential to increase the uptake of these programs by making them more accessible, thereby reaching more patients. The evaluation of the voice app demonstrated user acceptability through the SUS questionnaire (92/100 average score), and the qualitative findings provided assurance of deployment feasibility, as the users were able to successfully interact with the voice app. Common themes from the usability sessions (preference for music playing during interaction, providing oral or visual feedback, and the instruction card being a helpful aid) served as feedback for the redevelopment phase of the project.
The impact of the findings from this study is promising, especially given the near-ubiquitous nature of smart speakers, such as Amazon Alexa and Google Home. These devices can be easily integrated within households, and they make it easy and convenient to perform tasks through simple conversations and offer flexibility to do multiple tasks simultaneously. Patients with chronic illness experience a constant need to record and transmit data to better track their health, and voice apps, deployed through smart speakers, can help them accomplish this seamlessly, not only because it uses speech to collect these data but also because it can guide the user through their tasks more explicitly than the smartphone.
As this platform requires less technical knowledge than the smartphone (mHealth apps), those who struggle to use technology have the opportunity to participate in the program. According to the usability study observations, most participants felt that it was easy to interact with the voice app because it only required conversation and no technical background. The platform’s multilingual capability offers another opportunity, which may make some more individuals willing to use it to record measurements. VUIs also allow users to initiate a call to anyone on their list through voice command; this can be useful in cases where they need assistance but cannot reach their phone to call for help. This scenario often occurs with older adults, who are also the most common group to be diagnosed with chronic illnesses and, as a result, are prescribed digital therapeutics.
It is also important to consider the implications of using voice apps for chronic disease self-management. When inputting measurements on an mHealth app, it is typical for the user to see their data on the screen before submitting. With a VUI that solely relies on verbal communication, visual feedback is no longer a possibility. The participants commented on the importance of knowing which measurements were recorded on the voice app and valued the ability to see historical data and trends on the smartphone app. With these findings, we incorporated verbal feedback as a necessary component in the conversational design to help with user confidence. Adding this step in the conversation comes at the cost of prolonging the time it takes to complete the interaction but is not necessarily a drawback because of the lesser focus required when compared with using a smartphone.
Although users only need to use speech when interacting with VUIs, the conversation style may vary significantly when compared with human-to-human interaction. During this study, the users who had never used these types of devices before felt the need to raise their voice in hopes that it heard and understood them better. Expelling this energy repeatedly may make the user feel tired, as one of the participants noted, especially because the demographic that uses these digital therapeutics are older adults. Without knowing how the voice app will respond, a difference in body language was also observed, namely users tensed up, felt the need to sit up straighter, and focused more when interacting with the device. This behavior is in stark contrast to how users interacted with the smartphone, as their physical body language while using the smartphone indicated a much more relaxed behavior. Despite these differences, it is expected that user body language will shift toward a more relaxed behavior as they become more comfortable with using voice apps.
Finally, it is also important to consider the potential barriers to entry that exist as a result of using this technology. For example, most VUI devices require the user to have an account to interact with the device, creating an accessibility issue. Another barrier to entry is the requirement to have a constant, reliable internet connection to use the device. This is in contrast to the
To our knowledge, this study is one of the first to investigate the acceptability and feasibility of a personalized chronic disease self-management voice app, designed to be used by patients with HF in their homes. Our work is similar to other recent studies that have used either an Amazon Alexa or a Google Home device to deploy a self-management voice app for various chronic diseases, including type 2 diabetes [
Although the results from this research project showed promise for future use, it is important to acknowledge the limitations associated with this work.
The main limitation of usability studies is the short interaction time. Although the observations and feedback from the participants were helpful in identifying the potential that voice apps may have for chronic disease management, they only interacted with the technology for a short period. Future work should include a study that requires the users to interact with the device for a longer period to give them more experience with the voice app to reflect more grounded feedback.
A sample size of 8 participants was chosen because the usability study was only focused on gathering user insights and not statistically relevant data. According to the literature, the total participants needed for a usability study is dependent on when saturation is achieved in results. In most cases, this occurs when the sample size is typically between 5 and 8 participants [
Finally, the participants may have had social desirability bias when answering interview questions and filling out the questionnaire, especially because they knew that the study coordinator also had involvement in the design and development of the voice app [
This project involved the design, development, and usability evaluation of a voice app for HF management. A UCD process was followed to systematically create a voice app that would meet user needs and be easy to use. The usability study performed at the PMCC at UHN provided insightful user feedback about the voice app’s design, with the overall response being positive with high user satisfaction. The findings from this usability study impacted the redevelopment of the voice app, which will be used in a future clinical evaluation. As advancements in VUIs progress, we believe they will play an integral role in providing access to chronic disease management programs by (1) helping more patients complete their tasks independently, (2) offering a more convenient experience to record relevant data, and (3) allowing those with limited technical and English skills access to these programs. The findings from this research show promise in using VUIs to help with chronic disease management.
Design requirements (including both user and software) that were considered during the design stage (Table S1). Software architecture diagram of the Medly voice app system (Figure S1). Instruction card provided to users to help them navigate the conversation (Figure S2). An example of the “script” describing the conversation for the Medly voice app (Figure S3). A “script” describing the conversation that will occur if the Medly voice app did not correctly capture the user’s symptom responses (Figure S4).
The System Usability Scale questionnaire that was handed out to the participants at the end of the usability session.
List of potential errors identified from usability study results, along with risk levels and mitigation strategies (Table S1). Main themes derived from the usability sessions (Table S2).
heart failure
mobile health
Peter Munk Cardiac Centre
System Usability Scale
user-centered design
University Health Network
voice user interface
The authors wish to thank Jacqueline Simpson for designing the
The data sets generated and analyzed during this study are not publicly available due to an agreement with the Research Ethics Board of the University Health Network, which describes very specific circumstances that prohibits us from sharing the data on a public platform; however, the data are available from the corresponding author on reasonable request.
JAC and HR are part of the team that founded the Medly system under the intellectual property policies of the University Health Network and may benefit from future commercialization of this technology.