Published on 19.11.20 in Vol 4, No 11 (2020): November
Early-Stage Feasibility of a Mobile Health Intervention (Copilot) to Enhance Exacerbation-Related Self-Management in Patients With Chronic Obstructive Pulmonary Disease: Multimethods Approach
Background: There is an emergence of mobile health (mHealth) interventions to support self-management in patients with chronic obstructive pulmonary disease (COPD). Recently, an evidence-driven mHealth intervention has been developed to support patients with COPD in exacerbation-related self-management: the Copilot app. Health care providers (HCPs) are important stakeholders as they are the ones who have to provide the app to patients, personalize the app, and review the app. It is, therefore, important to investigate at an early stage whether the app is feasible in the daily practice of the HCPs.
Objective: The aim of this study is to evaluate the perceived feasibility of the Copilot app in the daily practice of HCPs.
Methods: A multimethods design was used to investigate how HCPs experience working with the app and how they perceive the feasibility of the app in their daily practice. The feasibility areas described by Bowen et al were used for guidance. HCPs were observed while performing tasks in the app and asked to think aloud. The System Usability Scale was used to investigate the usability of the app, and semistructured interviews were conducted to explore the feasibility of the app. The study was conducted in primary, secondary, and tertiary care settings in the Netherlands from February 2019 to September 2019.
Results: In total, 14 HCPs participated in this study—8 nurses, 5 physicians, and 1 physician assistant. The HCPs found the app acceptable to use. The expected key benefits of the app were an increased insight into patient symptoms, more structured patient conversations, and more tailored self-management support. The app especially fits within the available time and workflow of nurses. The use of the app will be influenced by the autonomy of the professional, the focus of the organization on eHealth, costs associated with the app, and compatibility with the current systems used. Most HCPs expressed that there are conditions that must be met to be able to use the app. The app can be integrated into the existing care paths of primary, secondary, and tertiary health care settings. Individual organizational factors must be taken into account when integrating the app into daily practice.
Conclusions: This early-stage feasibility study shows that the Copilot app is feasible to use in the daily practice of HCPs and can be integrated into primary, secondary, and tertiary health care settings in the Netherlands. The app was considered to best fit the role of the nurses. The app will be less feasible for those organizations in which many conditions need to be met to use the app. This study provides a new approach to evaluate the perceived feasibility of mHealth interventions at an early stage and provides valuable insights for further feasibility testing.
JMIR Form Res 2020;4(11):e21577
Chronic obstructive pulmonary disease (COPD) is a highly prevalent disease and a major cause of mortality worldwide [, ]. Exacerbations are important events during the course of COPD because they accelerate the decline in lung function [ ], negatively affect quality of life [ , ], and lead to increased mortality and high socioeconomic costs [ , ]. Self-management is important to reduce the impact of COPD exacerbations on both patients and society. Self-management is defined as “an individual’s ability to detect and manage symptoms, treatment, physical and psychosocial consequences, and lifestyle changes inherent in living with a chronic condition” [ ]. Over the past years, research has increasingly focused on exacerbation-related self-management interventions, as they have shown to have positive effects on quality of life and hospital admissions [ , ]. In this context, the use of mobile health (mHealth) is considered to be promising to engage patients in their own health and to change health behaviors [ - ].
Current eHealth and mHealth interventions often focus on telemonitoring strategies to reduce the impact of exacerbations [- ]. Although positive outcomes were found for telemonitoring, the results are thus far inconclusive because of the poor quality of the studies and the heterogeneity among the studies [ , , ]. The inconclusive results might also be explained by the lack of focus on enhancing self-management skills, as the decision-making process is mostly professional based. mHealth interventions aimed at facilitating, supporting, and sustaining self-management in patients with COPD have shown to improve quality of life and levels of activity; however, no firm conclusions could be drawn from them [ ]. Recently developed mHealth interventions focusing on self-management have shown variable results. Farias et al [ ] showed that using telehealth technologies to enhance adherence to a COPD action plan resulted in faster exacerbation recovery and decreased number of COPD-related emergency department visits and hospitalizations. Another mHealth tool that supports self-management of exacerbations showed no positive effects on exacerbation-free time, health status, and health care utilization [ ]. However, given the proven effectiveness of self-management interventions in patients with COPD, it could be expected that mHealth interventions supporting patients in self-management can be effective in reducing exacerbation impact.
Recently, an evidence-driven mHealth intervention has been developed in the Netherlands to support patients with COPD (the end users) in exacerbation-related self-management—the Copilot app. The Copilot app is a mobile app that targets the early detection of exacerbations through self-monitoring and performing prompt actions by using individualized action planning. The Copilot app consists of 4 components: (1) a personalized COPD action plan, (2) symptom monitoring, (3) an overview of registered symptoms and undertaken self-management actions, and (4) an information module about COPD and self-management. The Copilot app focuses specifically on developing patient self-management skills over time (learning by doing) without real-time monitoring by health care providers (HCPs). Nevertheless, the HCPs are important stakeholders as the Copilot app requires a case manager role from HCPs. The role of the HCPs is to provide the app to patients, instruct patients on how to use the app, set up a personalized action plan together with a patient, and evaluate the patient’s condition based on registrations in the app during consultations. The app can be provided by HCPs across health care settings. The Copilot app was developed by following a user-centered design that included several phases of usability testing . The usability of the Copilot app for patients was considered to be good [ ]. More information about the Copilot app is provided in .
An important next step is to investigate whether the Copilot app can work within the daily practice of HCPs by evaluating how HCPs perceive the feasibility of the Copilot app . Evaluating feasibility within the daily practice of HCPs at an early stage helps to determine whether the Copilot app is appropriate for further testing and to identify what changes are needed and how they might occur [ ]. Although patients with COPD and HCPs are both crucial stakeholders in feasibility evaluation, adequate personalization of the app and evaluation of the patient’s condition by HCPs is essential for safe and effective self-management by patients using the Copilot app [ , ]. Therefore, early feasibility evaluation in the daily practice of the HCPs should precede further longitudinal feasibility testing among patients. On the basis of this step, essential design input for optimization of the Copilot app can be generated and a substantial part of the feasibility problems in the daily care by HCPs can be eliminated before further testing.
The aim of this study is to evaluate the perceived feasibility of the Copilot app in the daily practice of HCPs.
A feasibility study with a multimethods design was used to investigate how HCPs experience working with the first version of the Copilot app and how they perceive the feasibility of the app in their daily practice. On the basis of the work by Bowen et al , feasibility areas relevant for this early-stage feasibility evaluation were used for guidance. This study evaluates how HCPs react to the Copilot app (acceptability); the extent to which the Copilot app is likely to be used by HCPs (demand); the extent, likelihood, and manner in which the Copilot app can be used by HCPs as planned and proposed (implementation); the extent to which the app can be used by HCPs in their routine daily practice considering the available resources (practicality); and the extent to which the Copilot app can be integrated within the context of Dutch health care settings (integration) [ ]. A one-time interactive session was conducted to observe how HCPs interacted with the app. HCPs were observed while performing tasks with the app and asked to think aloud to verbalize initial perceptions and feelings toward the app, clarify their process of decision making, and express experienced problems [ , ]. Afterward, a standardized usability questionnaire was used to investigate how HCPs perceived the usability of the app, and semistructured interviews were conducted with HCPs to further explore how they perceive the feasibility of the app. The study was conducted in primary, secondary, and tertiary care settings in the Netherlands from February 2019 to September 2019. The study was approved by the Medical Ethics Research Committee of the University Medical Centre Utrecht (18/831).
A purposive sample of HCPs who have a case manager role in COPD care was selected from primary, secondary, and tertiary care settings in the Netherlands. Case management was defined as physicians or nurses who provide ongoing and/or follow-up self-management support during patient consultations . Eligible HCPs were pulmonologists, general practitioners, respiratory nurses, respiratory nurse specialists, physician assistants, and primary care nurses, with a minimum work experience of 1 year and employment at their present organization for at least one year. The 1-year cutoff point was considered relevant in providing meaningful insights into the feasibility areas. The maximum variation in profession and work setting was pursued to select a sample that adequately represents the HCPs providing self-management support in Dutch COPD care and to increase the likelihood of capturing different perspectives in the findings. On the basis of the guidance for evaluating the use of apps in general, a minimum of 10 HCPs were approached [ ]. HCPs were included until saturation was reached for acceptability, demand, and implementation. Practicality and integration vary greatly across various health care organizations; therefore, it was not considered feasible to achieve saturation in these areas of focus using this study design. Therefore, data collected on practicality and integration are only described in this study.
Recruitment and Informed Consent
HCPs were approached by email or telephone to participate in this study by 2 researchers (YK and TH). Four HCPs were approached based on their expressed interest during participation in previous studies related to the development of the app, whereas others were recruited through local COPD networks. The HCPs received an invitation to participate in the study by email. A reminder was sent after 1 week in case of nonresponse. In case of continued absence of a response, the HCPs were contacted within a week by phone to determine their interest to participate in the study. The HCPs willing to participate received further study information and an informed consent form. An appointment was scheduled at the place of employment of the HCPs, and the informed consent form was signed during the appointment. Recruitment started initially with 5 HCPs to initiate data analysis. Further recruitment was determined based on ongoing data analysis and data saturation for acceptability, demand, and implementation.
Data were collected during a single, 1-hour session following a stepwise procedure. The stepwise procedure is described in the following sections.
Step 1: Interactive Session (Observations and the Think Aloud Method)
Before starting the interactive session, participants were informed about the study procedure and received further information about the Copilot app, the scenario for use of the app in daily practice, the intended role of both HCPs and patients, and the developmental stage of the app (). Furthermore, the researcher walked through the 4 components of the app together with the participants. The participants were then asked to read a fictional patient case together with a set of tasks developed by the research team ( ). These tasks focused on personalizing the action plan, adjusting the action plan, and evaluating the overview of registered symptoms and actions in the app. The participants were asked to perform these tasks in the app and were instructed and stimulated to think aloud while performing these tasks [ , ]. Almost all HCPs used the first version of the Copilot app. In the final interview, an updated version of the Copilot app was used, in which minor changes were made to improve usability. No major changes were made to the content and functions of the app; therefore, the influence of these changes on study outcomes was considered to be minor.
Step 2: The System Usability Scale
After working with the app, participants were asked to fill in the validated 10-item System Usability Scale (SUS) to obtain an overall view of the usability of the app for HCPs . Each item was scored on a 5-point Likert scale, and all items were converted to a total score (range 0-100; a score >70 is considered to be acceptable) [ , ]. Although investigating usability was not the objective of this study, usability problems could have been experienced by participants who worked with the first version of the Copilot app. The perceived usability by HCPs was considered to be an important factor that could influence the adoption of the app by HCPs in daily practice and was therefore evaluated roughly in this study as well [ ].
Step 3: Semistructured Interview
A semistructured interview was conducted to investigate perceptions toward the feasibility of the Copilot app in daily practice. A topic list was developed based on the 5 areas of focus and their related outcomes of interest . Questions that were formulated by Bowen et al [ ] to illustrate the areas of focus and the outcomes of interest were used as a basis for formulating interview questions. The topic list is further detailed in . During and directly after the interviews, memos were created to describe observations, reflect on methodological issues, and capture initial thoughts related to theoretical concepts. Insights gained during the interviews were introduced in subsequent interviews, leading to data saturation.
Step 4: Baseline Questionnaire
After finishing the interview, participants were asked to fill in a questionnaire to collect baseline characteristics.
The stepwise procedure of data collection is further detailed inand .
|Method and area of focus||Outcome of interest|
|Step 1: interactive session|
|Implementationa||Degree of execution of tasks and success or failure of execution of tasks|
|Step 2: SUSb|
|Acceptabilityc||Satisfaction with the app|
|Demandd||Intention to use the app|
|Implementation||Degree of execution of tasks|
|Step 3: interview|
|Acceptability||Satisfaction with the app, perceived appropriateness, and fit within the organizational culture|
|Demand||Perceived demand and intention to use the app|
|Implementation||Degree of execution of tasks, success or failure of execution of tasks, and factors affecting implementation ease or difficulty|
|Practicalitye||Expected benefits and burden for end users and ability of HCPsf to carry out tasks in their routine daily practice|
|Integrationg||Perceived fit with local care infrastructure at the patient and organizational level and perceived sustainability at the patient and organizational level|
|Step 4: baseline questionnaire||Profession, age, work experience, size of organization, amount of patient consultations for COPDh in a week, average time available for patient consultations, disease severity of patients with COPD in daily care, current use of written action plan, current use of mobile health|
aImplementation: the extent, likelihood, and manner in which the Copilot app can be used by health care providers as planned and proposed.
bSUS: System Usability Scale.
cAcceptability: how the health care provider reacts to the Copilot app.
dDemand: To what extent is the Copilot app likely to be used by the health care provider.
ePracticality: To what extent the Copilot app can be used by health care providers in their routine daily practice considering the available resources.
fHCP: health care provider.
gIntegration: to what extent can the Copilot app be integrated in Dutch primary, secondary, and tertiary care settings.
hCOPD: chronic obstructive pulmonary disease.
All sessions were conducted by 1 researcher (YK or TH) who guided the interactive sessions and conducted the interview. The whole procedure was video recorded to observe the hand interaction of participants with the app and to audio record verbalizations during the interactive session and the interview. In addition, the researcher made notes on observed user problems and relevant verbalizations of the participants while they were working with the app. Before starting the data collection, 2 pilot sessions with professionals in COPD care were conducted to investigate whether participants understood the procedure and questions, to determine whether the questions resulted in relevant answers, and to determine whether the stepwise data collection procedure fitted in a 1-hour session. Findings of the pilot sessions were used to modify data collection procedures by reducing the set of tasks that HCPs had to perform, by adjusting the information about the app and study procedure, and by merging interview questions that resulted in similar answers. The results of the pilot sessions were not included in this study. Practical issues that arose during the study resulted in iterations in the data collection guideline ().
Data from the interactive sessions and semistructured interviews were analyzed according to a thematic analysis as described by Braun and Clarke . Video recordings were reviewed for usability issues and categorized according to the type of problem. All video recordings, including both the think aloud comments of the interactive sessions and the interviews, were transcribed verbatim. In total, 13 hours of video recordings were transcribed verbatim, resulting in 230 pages of transcription. Data analysis was supported by NVivo 11.0 software (QSR International Pty Ltd Version 11). The analysis took place in a cyclic process, alternating data analysis with data collection. Data were analyzed by 2 researchers (YK and TH) and were discussed with a third researcher (SV).
First, the 2 researchers independently read the transcripts to obtain an overall picture and noted initial ideas on relevant themes. Second, the transcripts were reread in more detail, and initial codes were linked to meaningful paragraphs by both researchers and discussed afterward to reach consensus. Next, identified codes were brought under potential themes and were reviewed for correspondence to the coded paragraphs, generating a thematic map of the analysis. Finally, potential themes were further refined, and clear definitions were generated for each theme. The third researcher was involved from the stage at which potential themes were reviewed, by coding a selection of data and participating in discussions on the final definition of themes. Interpretation of the data was discussed with experts in the fields of nursing science, self-management, and mHealth (JT, MS, and LS), which contributed to the credibility of the study . Memo writing supported the data analysis process. The confirmability of the study was enhanced by peer review of the methodological quality by an external expert on qualitative research (SV) [ ].
Data from the SUS and the baseline questionnaire were analyzed with SPSS 25.0 (IBM Corporation) using descriptive statistics.
Baseline Characteristics of the Participants
A total of 14 HCPs (9 females and 5 males) participated in this study, including 8 nurses, 5 physicians and 1 physician assistant recruited from 5 primary, 7 secondary, and 2 tertiary care settings. The baseline characteristics of the participants are presented in. A total of 11 HCPs currently used written COPD action plans in their daily care to some extent, varying from barely using action plans to integrating action plans in regular care. A total of 10 HCPs had experience with the use of digital technology in COPD care; however, in most cases, technology was only used occasionally or within a study context. Maximum variation was achieved for profession, work setting, age, work experience, patient category most frequently seen by the HCP based on disease severity, number of patient consultations for COPD during a week, and organization size.
|ID||Age range (years)||Profession||Setting||Work experience (years)||Patient categorya based on GOLDb stage||Patient consultations per week||Consultation duration (minutes)||Organization sizec||Use of written action plan||Set up action plan (minutes)||Use of digital technology in COPDd care|
|R02||50-59||Primary care nurse||General practice||30||2-3||15||45||26-50||No||N/Ae||Apps|
|R05||30-39||Pulmonologist||Hospital||7||3-4||15||10||>1000||No||N/A||eHealth and apps|
|R06||50-59||Respiratory nurse specialist||Hospital||20||3-4||70||20||>1000||Yes||10||No|
|R07||40-49||General practitioner||General practice||7||1-2||4||10||101-250||Yes||10||eHealth|
|R10||50-59||Physician Assistant||Rehabilitation clinic||30||3-4||10||30||251-1000||Yes||15||No|
|R11||50-59||Primary care nurse||General practice||12||2-3||2||30||<10||Yes||10||eHealth|
|R12||40-49||General practitioner||General practice||10||1-2||4||10||11-25||Yes||30||eHealth and apps|
|R13||60-69||Respiratory nurse specialist||Hospital||14||2-3||20||30||>1000||Yes||30||Apps|
|R14||30-39||Primary care nurse||General practice||3||1-2||3||60||11-25||No||N/A||eHealth|
aPatient category most frequently seen by the health care provider, based on Global Initiative for Chronic Obstructive Lung Disease stage.
bGOLD: Global Initiative for Chronic Obstructive Lung Disease.
cOrganization size determined by the number of employees.
dCOPD: chronic obstructive pulmonary disease.
eN/A: not applicable.
fThe health care provider uses an action plan during consultations but does not set up an action.
The themes that emerged from this study are described in the following paragraphs and illustrated by quotes of the HCPs. Q references in the text refer to quotes of specific themes that are provided in the textboxes at the end of each paragraph. An overview of the themes is provided in. The themes are categorized based on the areas of focus, and the perceived benefits and risks of using the app in daily practice are described separately.
|Area of focus||Themes|
|Acceptability of the app and perceived demand|
|Perceived benefits and risks of using the app in daily practice|
|Factors that could influence the use of the app in daily practice|
|The extent to which the app can be used in current daily practice|
|Integration of the app across Dutch health care settings|
|How to integrate the app into daily practice|
|The role of health care professionals|
aHCP: health care provider.
bGDPR: General Data Protection Regulation.
cCOPD: chronic obstructive pulmonary disease.
Acceptability of the App and Perceived Demand
When the HCPs were asked about their first impressions of the app, the HCPs spoke about the usability aspects of the app and the relevance of the app for daily practice. Overall, high satisfaction about working with the app was expressed by ratings of 7 or higher on a 10-point numeric scale. These high ratings were attributed to finding the app user friendly because of its ease of use, intuitive navigation, design simplicity, attractive layout, and receiving positive feedback when using the app (Q1). Although HCPs needed time to familiarize themselves with the app, working with the app was perceived to be easy to learn. The high level of satisfaction and the app being perceived as user friendly was also supported by an average score of 83.8 (SD 15.1) on the SUS, indicating good usability of the app.
Furthermore, almost all HCPs believed that the app would be relevant for daily practice (Q2). HCPs recognized the content of the app, and particularly nurses found the app in line with the current self-management support. The calendar function in the app was considered to be most relevant for HCPs as it provided a compact, rapid, and clear overview of registered symptoms and actions. The action plan was largely in line with the current written action plans used by HCPs. The gray zone for decision support in determining the correct color zone () and the focus on personalizing the green zone based on registered symptoms were considered to have added value compared with written action plans. However, 8 HCPs preferred further tailoring of the yellow and orange zones ( ) by adding more personalized signals for symptom deterioration. The information module was considered to be clear and relevant, although information could be presented more attractively and could be extended with more detailed information about self-management actions. Most HCPs were positive about the symptom monitoring module, as indicating symptoms fits well with how patients would describe their symptoms. A total of 3 HCPs found the symptom monitoring module to be more convenient than the currently used questionnaires in COPD care, such as the Medical Research Council dyspnea scale and the Clinical COPD Questionnaire [ , ], whereas 6 HCPs expressed a wish to add those questionnaires to the app.
A total of 11 HCPs expressed that the app fits well within the organizational culture as their organizations are open to digital innovations in health care because of the fit with the current environment in which health care digitalization is rapidly evolving. However, some organizations neither prioritize nor facilitate digital innovations yet. Almost all HCPs had a positive attitude toward using the app in daily practice because of opportunities to improve the quality of self-management support (Q3). When discussing colleagues’ attitudes toward using mHealth, 3 HCPs believed that physicians would be hesitant and resistant toward these innovations on account of time constraints in their profession.
The majority of HCPs expressed a high level of interest in using the app, scoring an 8 or higher on a 10-point numeric scale. HCPs explained these ratings by their personal interest and enthusiasm for innovations and the fit with their needs and demands for health care improvement, such as more structured self-management support for patients with COPD (Q4). All HCPs expressed having the intention to use the app in their own practice, except for 1 general practitioner who believed the app will not fit with the current workflow and related time constraints.
Quotes of HCPs (Q references in the text) related to acceptability of the app and perceived demand are provided in.
Perceived Benefits and Risks of Using the App in Daily Practice
The HCPs believed that there would mainly be benefits of using the app in daily practice at the patient, HCP, and organizational levels and only a few potential risks. Almost all HCPs found the app a useful tool for patients to support self-management behavior as it can help patients to create awareness of their stable symptoms and signals of symptom deterioration and could then support taking prompt and adequate actions. More than half of the HCPs believed that patients being the owner of the app could enhance patient control of the disease by becoming less dependent on their HCP and being in control when receiving support from HCPs across health care settings (Q5). The 4 HCPs explicitly mentioned that the app would be an improvement for patients compared with the use of written action plans as patients carry the app with them all the time and an app stays clean and readable.
The HCPs perceived many benefits of using the app for their own practice. Most importantly, 11 HCPs believed the app could lead to more in-depth and structured patient conversations, as the conversations would be more initiated by patients and, therefore, be more tailored to the specific needs and preferences of the patients (Q6). HCPs experience that patients often have difficulties with recalling experienced symptoms and performed actions and tend to underestimate or exaggerate their symptoms. The calendar in the app would provide more insight into actual experienced symptoms when patients are at home and prevents that HCPs have to dig for information. This could save valuable time and lead to more meaningful contact between HCPs and patients. The calendar could contribute more tailored treatment and self-management support by HCPs as the output could be used by physicians to evaluate medication treatment and by nurses to match patient needs with relevant self-management support (Q7-8). In addition, 5 HCPs believed that the app could increase uniformity in self-management support by HCPs. Nowadays, self-management support by HCPs is often inconsistent between HCPs within and across health care organizations. Some HCPs mentioned that the app could provide more guidance in providing self-management support and could also facilitate making clear agreements on which HCPs are assigned as contact persons for a patient (Q9). Moreover, most HCPs agreed that the app could be used by various HCPs throughout and across health care settings, including those without a case manager role, and could enhance collaboration between HCPs within and across health care settings. Although some HCPs expressed having good contact with HCPs in other health care organizations and clear agreements about their roles and responsibilities, others expressed experiencing limited collaboration between HCPs in primary, secondary, and tertiary care. Some HCPs expressed that the patient being the owner of the app would facilitate the patients themselves being able to show the agreements made about their treatment with an HCP in one setting to HCPs in other settings. This could result in more continuity of care (Q10). Most HCPs found it difficult to reflect on the potential benefits of the app on an organizational level. A few HCPs mentioned that the app could potentially reduce health care costs by preventing hospital admissions or reducing the duration of hospital admissions.
Most HCPs perceived limited risks associated with the use of the app in daily practice. However, some HCPs expressed concerns about the safety of the app as there is potential for making mistakes during manual registration of medication into the app. In addition, 1 nurse specialist was concerned about the misinterpretation of the color zones by patients. Furthermore, 1 general practitioner expressed that patients substituting HCP contact with the app could be a risk, as patients might be less likely to involve their HCP when they have the app to guide them. Especially during a stable phase, a patient might think that the HCP contact is redundant because their app indicates that all is well. Moreover, 1 pulmonologist felt that using the app could be distracting from interacting with patients, which could form an obstacle for the HCPs role (Q11). Finally, 1 HCP from tertiary care expressed that the app could contribute to an increase in treatment burden for patients as they are already treated in multidisciplinary teams with a variety of (digital) interventions (Q12).
Quotes of HCPs (Q references in the text) related to the benefits and risks of using the app in daily practice are provided in.
Use of the App by HCPs and Factors That Could Influence the Use of the App in Daily Practice
Observed Use of the App by HCPs
During the interactive session, all HCPs clearly understood the tasks they had to perform in the app and they were able to perform those tasks well. HCPs who had experience with written action plans expressed that setting up the action plan in the app corresponds with the current workflow of setting up a written action plan and could even improve this workflow. All HCPs were able to set up and adjust the action plan and review registered symptoms and actions in the calendar function of the app. Support from the researcher was needed when usability issues were observed. These issues were mostly related to the registration of medication, setting up the yellow zone of the action plan, and saving registrations.
Factors That Could Influence the Use of the App in Daily Practice
By asking HCPs about their perceptions toward using the app in their daily practice, HCPs reflected on factors that could facilitate or hinder using the app as intended at the level of patients, HCPs, and the organization. All HCPs believed patient skills, opportunity, and motivation influence the use of the app (Q13-14). The proliferation of mobile device use and improvement in the digital skills of patients were believed to facilitate the use of the app in a large patient population. However, low health literacy, avoiding confrontation with illness, limited digital skills, lack of access to the internet, and loss of interest in the app on the long term were considered to be threats for continued use.
The fit with the available time and workflow of HCPs was perceived to be an important factor that would influence the use of the app in daily practice. HCPs’ perceptions toward this fit were influenced by the traditional division between the roles of nurses and physicians. Most HCPs mentioned that the app fits within the available time and workflow of nurses as they already have an important role in providing self-management support (Q15). A total of 6 HCPs believed that the app does not fit within the available time and workflow of physicians as they have a main focus on medical treatment in the limited time they have available for patients (Q16). Two physicians even felt that using the app could result in more work as they would be obligated to focus on issues that normally would not come to light. For physicians, it was important to determine whether the app would improve their work efficiency (Q17). Furthermore, the autonomy of the professional in implementing innovations and scheduling extra time for consultations, if needed, was perceived to facilitate use of the app in daily practice (Q18). Moreover, 2 HCPs in primary care mentioned that primary care nurses might not feel comfortable with adding medication prescriptions to the app, which could hinder the use of the app as intended.
The focus of the organization on eHealth and self-management was considered to be an important facilitator for implementation of the app, as this would also facilitate the existence of innovation teams within organizations that could support HCPs in the use of digital innovations (Q19). Furthermore, some HCPs mentioned that costs associated with the use of the app would influence the use of the app in an organization, as innovations should not be too expensive and should ideally lead to a reduction in health care costs. Finally, some HCPs expressed that the compatibility of the app with current systems and eHealth initiatives in their organization is important, as a mismatch with current systems could hinder the use of the app in an organization (Q20).
Quotes of HCPs (Q references in the text) related to performance of tasks in the app and factors that could influence the use of the app in daily practice are provided in.
The Extent to Which the App Can be Used in Current Daily Practice
By asking HCPs how the app would fit within the current daily practice, 3 HCPs explained that the app could already be used in daily practice considering the available conditions, time, and resources. However, the majority of HCPs mentioned conditions that should be met before being able to use the app in daily practice. An important condition that should be met is the app being in line with the General Data Protection Regulation (GDPR) rules, as HCPs asked questions about privacy issues (Q21). In addition, some HCPs explained that the app should function technically well on various devices so that patients can use the device they prefer.
On patient and HCP levels, most HCPs expressed a need for clear instruction about the app for both patients and HCPs, such as written information or a demo about the app, to have sufficient knowledge about how the app should be used and to show patients how the app works (Q22). Especially for HCPs without the experience of working with written action plans, instruction on how to set up an action plan that includes medication prescription is important. Furthermore, for HCPs specifically, there should be sufficient time during consultations to be able to work with the app. On the basis of this study, HCPs expected to need approximately 30 min to install the app and to personalize the action plan for the patient. They believed this would probably take longer in daily practice as they would have to communicate with patients at the same time. Therefore, some HCPs explained that their consultation time should be extended. Most of these HCPs expressed that they would have the opportunity to schedule some extra time for consultations. Two HCPs emphasized the importance of embedding the app into current workflows to realize sustained use over time.
Many HCPs insisted that approval of using the app within the organization was considered to be an important condition that must be met before being able to use the app in daily practice (Q23). Having an organizational mandate to implement the app was considered to contribute to the allocation of time and resources. Overall, HCPs working in larger organizations believed acquiring management support to be more difficult and time consuming than HCPs working in smaller organizations. Moreover, 2 HCPs working in larger organizations expressed that having a plan for implementation of the app would be important to mobilize people in an organization to start working with the app. Furthermore, a practical issue mentioned was that organizations need to have access to a fast Wi-Fi connection to be able to quickly download and use the app in daily practice. Finally, HCPs emphasized the importance of good coordination between HCPs in collaborating organizations with regard to their roles and responsibilities in the use of the app. A total of 7 HCPs indicated that training and instructional material for HCPs across health care settings would help to create awareness about the app among HCPs and about their role in using the app (Q24).
On the basis of the question whether HCPs would prefer to have access to a separate HCP portal to set up an action plan and review patients’ registrations on their own computer, 8 HCPs expressed that a separate HCP portal would have added value for their daily practice. These HCPs, mostly physicians, believed that a separate portal would be more user friendly and efficient, would be helpful in preparing patient consultations, and would enable HCPs to review the app during consultations by telephone. Furthermore, it could reduce the administrative burden and could support collaboration between HCPs in an organization when the HCP portal is integrated into local information technology systems. Concerns about the privacy sensitivity of an HCP portal influenced the perceptions of the HCPs on having a separate portal. On one hand, 2 HCPs expressed that they would not feel comfortable if they had to work on the patient’s device itself, as this is a private device. On the other hand, some HCPs expressed that a connection with a separate portal or patient system could entail privacy risks as well. A separate HCP portal was less important for nurses. Three nurses argued that HCPs having access to the app could undermine or diminish self-management behavior of the patient. It could give patients the feeling of being monitored by their HCP, which emphasizes an external locus of control (Q25). Nonetheless, most HCPs perceived a separate HCP portal as an important condition that must be met to stimulate use of the app in daily practice.
Quotes of HCPs (Q references in the text) related to the extent to which the app can be used in current daily practice are provided in.
Integration of the App Across Dutch Health Care Settings
How to Integrate the App Into Daily Practice
Overall, the HCPs from primary, secondary, and tertiary care settings felt that the app is feasible to integrate into their daily practice and in already existing paths of care as the app could be easily adapted to the specific context of health care organizations. The HCPs believed that the app could be introduced in annual COPD checkups in primary care; clinical care paths, outpatient follow-up care, and pulmonary rehabilitation programs in secondary care; and pulmonary rehabilitation programs in tertiary care (Q26). Most HCPs emphasized the importance of flexibility in the moment of introducing the app to patients and evaluating the app to be able to integrate the app into their workflow. Individual organizational factors, such as a specific path of care or division between HCP roles, would determine the specifics of how the app is to be integrated. The HCPs explained that the app could be used as guidance during patient consultations. HCPs currently using written action plans intended to replace the use of written action plans for the app. Nonetheless, for them, patient preferences and skills would determine whether a written action plan could be replaced by the app. Most HCPs felt that the use of the app would be sustainable in Dutch COPD care because of the fit within national COPD guidelines and current paths of care and their wish for more structured self-management support (Q27). However, 3 HCPs mentioned that the need for the app could decrease over time when the self-management skills of patients have been improved.
The Role of Health Care Professionals Toward Using the App
Although all HCPs indicated that working with the app best fits the role of the nurses because of their current role in providing self-management support, most HCPs considered the use of the app to be a shared responsibility between nurses and physicians (Q28). As nurses and physicians have a shared responsibility in the care for patients with COPD, this also applies to the use of the app in daily practice. Overall, introducing the app to patients and initiating and evaluating the action plan was considered to best fit the role of the nurses. However, some HCPs explained that physicians could have a role in prescribing and evaluating medication treatment in the action plan, which depends on the autonomy of the nurses with regard to medication prescription. HCPs expressed that reviewing the calendar of the app could be integrated in the consultations of both the nurses and physicians.
Most HCPs felt that the selection of eligible patients for the app should be the responsibility of HCPs. Although 4 HCPs expressed the intention to provide the app to all of their patients, most HCPs believed that not all patients will be eligible for the app. Therefore, they would select patients based on their assumptions about patient skills and motivation to use the app. A few HCPs explained that the motivation to use the app could be related to the severity of the disease. HCPs expected that patients with severe COPD and frequent exacerbations would be more motivated for exacerbation-related self-management compared with patients with an early stage of COPD who prefer to avoid confrontation with the disease. Therefore, most HCPs mentioned that they would provide the app to patients who frequently have exacerbations (Q29). Furthermore, most HCPs believed they would have an important role in installing and personalizing the app together with a patient, although 2 HCPs believed patients could do this initially by themselves. Finally, 6 HCPs mentioned that they would evaluate patient skills and use of the app so that they could provide support in using the app when needed, thereby guaranteeing safe and effective use by patients over time (Q30).
Quotes of HCPs (Q references in the text) related to integration of the app across Dutch health care settings are provided in.
This early-stage feasibility study provides insight into the perceptions of Dutch HCPs with a case manager role in COPD care regarding the use of the Copilot app in daily practice. Overall, the HCPs were able to work with the app and found the app acceptable to use in daily practice. The app could be used as guidance during patient consultations and could replace the use of written action plans in COPD care. Many benefits and only a few risks were expected regarding the use of the app in daily practice at the patient, HCP, and organizational levels. The app was considered to best fit the role of the nurses. Physicians were expected to have a marginal role in working with the app because of time constraints and misfit with their workflow. Other key factors that could influence the use of the app were the autonomy of the professional, the focus of the organization on eHealth, costs associated with the app, and compatibility with the current systems used. There are various conditions that must be met to be able to use the app in daily practice. The level of importance of these conditions varied between professions and contexts and may be attributed to organizational factors or fundamental differences in needs between physicians and nurses. The app was considered to be feasible to integrate into existing care paths of primary, secondary, and tertiary health care settings. Individual organizational factors must be taken into account when integrating the app in daily practice.
Some of the findings of this study are in line with those of other studies. Two recent studies focusing on the adoption of mHealth by HCPs also identified usefulness, ease of use, perceived benefits, autonomy of the professional, and integration with other systems as facilitators for the adoption of mHealth by HCPs [, ]. Similar to our results, these studies considered disruption to workflow, lack of time, increased workload, cost issues, and privacy and security issues as key adoption barriers [ , ]. Gagnon et al [ ] pointed out that the use of mHealth could be disruptive during visits as it could influence the interaction between patients and health care professionals; this was identified in our study as well and was perceived as a risk for the use of the app in daily practice. Furthermore, a study focusing on the adoption of new technology by physicians found that high initial physician time costs, uncertain financial benefits, and lack of electronic exchange between systems were key physician-related barriers [ ]. These studies indicate that nurses may hold the key to successful implementation of the Copilot app because of their role, the fit with their workflow and available time, and numerous advantages for their daily practice. Moreover, these studies strengthen our findings on the importance of meeting specific conditions to use the app in daily practice. According to our results, a separate portal for HCPs and integration with current systems could potentially facilitate the use of the app, especially for physicians. However, HCPs’ perspectives toward system integration differ, which was also observed in conversations with HCPs during the development of the Copilot app [ ]. On the basis of the literature, it could be expected that interoperability is important for integration of the app across health care settings [ ].
The findings of this study show that factors influencing future implementation and integration of the app into health care organizations are context dependent. Recently, much emphasis has been placed on the importance of taking into account the context in intervention research aiming at changing behaviors, to increase the likelihood of developing appropriate, implementable, effective, and sustainable interventions . On the basis of HCPs’ perceptions that the app is feasible to implement and integrate into Dutch health care organizations, taking context into account in the development of the app seemed to have resulted in sufficient flexibility in the design of the app to work across a range of contexts [ , ].
Strengths and Limitations
A strength of this study was the maximum variation in settings and HCPs resulting in a broad range of perspectives, thereby increasing the transferability of our findings to similar settings in the Netherlands . Furthermore, the credibility and confirmability of this study were enhanced by using data and researcher triangulation [ ]. The feasibility framework described by Bowen et al [ ] ensured that feasibility was evaluated by considering several important areas of focus to determine if the app can work within the constraints of daily practice. Although not the focus of data saturation, data collected on integration and practicality gave a general impression of contextual differences on how to integrate the app and which HCP role is perceived to be most suited.
A limitation of this study was the variation in the course of the interactive sessions and interviews because of time constraints and unforeseen circumstances within the HCPs’ workflow. In some cases, this resulted in limited in-depth interviews and underexposure of some topics. However, systematic reflection on these methodological issues and subsequently adapting the guideline of the session resulted in more in-depth data collection as the study proceeded. Furthermore, a relatively large part of the study population had experience with digital technology to some extent. This may have resulted in a more positive perception toward the use of technology as familiarity with mHealth and technologies in general is considered to facilitate the adoption of mHealth . Finally, this early-stage feasibility study evaluated the Copilot app within an artificial context, consisting of 1 interactive session with a fictional patients’ case. It could be discussed whether perceptions of feasibility would be different in the case of actual implementation of the app in the daily practice of the HCPs. Nonetheless, HCPs have experienced working with the app by simulating the use of the app in daily practice.
Implications for Practice and Future Research
The findings of this study are important for HCPs in COPD care and for researchers focusing on the development and evaluation of mHealth interventions. The study shows that the Copilot app is considered to be relevant and acceptable to use in the daily practice of the HCPs. The app could result in various benefits for patients, HCPs, and health care organizations and has high potential for successful implementation and integration across Dutch health care settings. Important lessons can be learned from this study with regard to practicality, which we described in this study as conditions that have to be met to use the app in daily practice. To use the app in daily practice, it is important that clear instruction about the use of the app is provided to both patients and HCPs, that there is sufficient time during consultations, and that approval to use the app within organizations is realized. In addition, good coordination about the use of the app between HCPs in collaborating organizations is needed. Adequate training and support for HCPs regarding the use of the app is important for implementation and integration of the app in daily practice, as using the app requires behavior change from HCPs. Essential in changing HCPs’ behaviors is that they have the capability, motivation, and opportunity to use the app in daily practice . Training and support should therefore focus on motivating HCPs to use the app and enhancing HCPs’ knowledge and skills needed to use the app, with a specific focus on the use of action plans [ ]. Finally, a separate portal for HCPs is an important condition that must be met in some organizations to stimulate the use of the app in daily practice. Contextual factors across health care settings will determine the specific conditions that should be met to be able to use the app in daily practice.
For researchers and developers focusing on the development and evaluation of mHealth interventions, this study provides insight into a new approach to evaluate the feasibility of mHealth interventions at an early stage. This approach has been shown to be a thorough and relatively quick way to investigate perceptions toward feasibility. The methods used in this study provided rapid insight into influencing factors and conditions regarding feasibility, thereby allowing researchers and developers to adapt mHealth interventions by moving backward or forward quickly. Evaluating feasibility at an early stage helps to determine whether mHealth interventions are appropriate for further feasibility testing with end users over a longer time period.
Further research on the Copilot app should focus on longitudinal feasibility testing of the Copilot app with both patients and HCPs to investigate the delivery and acceptability of the intervention, compliance with the intervention, and recruitment of patients and to investigate limited efficacy. In a next phase, the effect of the Copilot app on relevant patient outcomes and health care use should be evaluated. This evaluation should include an assessment of how context influences the effectiveness of the app . Understanding how the app relates to context is critical to understand how the app works and for whom, what influences implementation success and failure, whether the app can be successfully adapted or scaled-up from one context to another, and to what extent effects could be generalized to other contexts [ ]. To achieve this, a thorough process evaluation using qualitative and quantitative methods from a system lens is recommended [ , , ].
This early-stage feasibility study shows that the Copilot app is feasible to use in the daily practice of Dutch HCPs and is considered to best fit the role of the nurses. The app is perceived to be acceptable to use and relevant for the daily practice of HCPs. The app can be used as guidance during patient consultations and could replace the use of written action plans in COPD care. Many benefits and only a few risks were expected regarding the use of the app in daily practice at the patient, HCP, and organizational levels. The app will be less feasible in organizations where relatively many conditions need to be met. The app is considered to be feasible to be integrated into primary, secondary, and tertiary health care settings in the Netherlands. Individual organizational factors must be taken into account when integrating the app in daily practice. This study provides a new approach to evaluate the perceived feasibility of mHealth interventions at an early stage and provides valuable insights for further feasibility testing. Future research should focus on longitudinal feasibility testing of the Copilot app by both patients and HCPs.
The authors would like to thank all the HCPs who participated in this study for their time and sharing perceptions toward the feasibility of the mHealth intervention.
Conflicts of Interest
Stepwise procedure of data collection.PDF File (Adobe PDF File), 261 KB
Fictional patient case and assignment for health care providers.PDF File (Adobe PDF File), 181 KB
Topic list for the semistructured interview.PDF File (Adobe PDF File), 163 KB
- World Health Organization. The Global Burden of Disease: Update 2004. URL: https://www.who.int/healthinfo/global_burden_disease/GBD_report_2004update_full.pdf?ua=1 [accessed 2018-10-28]
- Global Initiative for Chronic Obstructive Lung Disease (GOLD). Global Strategy for the Diagnosis, Management, and Prevention of Chronic Obstructive Pulmonary Disease: Report. 2018. URL: https://goldcopd.org/wp-content/uploads/2017/11/GOLD-2018-v6.0-FINAL-revised-20-Nov_WMS.pdf [accessed 2018-06-12]
- Donaldson GC, Seemungal TA, Bhowmik A, Wedzicha JA. Relationship between exacerbation frequency and lung function decline in chronic obstructive pulmonary disease. Thorax 2002 Oct;57(10):847-852 [FREE Full text] [CrossRef] [Medline]
- Seemungal TA, Donaldson GC, Paul EA, Bestall JC, Jeffries DJ, Wedzicha JA. Effect of exacerbation on quality of life in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1998 May;157(5 Pt 1):1418-1422. [CrossRef] [Medline]
- Miravitlles M, Ferrer M, Pont A, Zalacain R, Alvarez-Sala JL, Masa F, IMPAC Study Group. Effect of exacerbations on quality of life in patients with chronic obstructive pulmonary disease: a 2 year follow up study. Thorax 2004 May;59(5):387-395 [FREE Full text] [CrossRef] [Medline]
- Soler-Cataluña JJ, Martínez-García MA, Román Sánchez P, Salcedo E, Navarro M, Ochando R. Severe acute exacerbations and mortality in patients with chronic obstructive pulmonary disease. Thorax 2005 Nov;60(11):925-931 [FREE Full text] [CrossRef] [Medline]
- Toy EL, Gallagher KF, Stanley EL, Swensen AR, Duh MS. The economic impact of exacerbations of chronic obstructive pulmonary disease and exacerbation definition: a review. COPD 2010 Jun;7(3):214-228. [CrossRef] [Medline]
- Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns 2002;48(2):177-187. [CrossRef] [Medline]
- Howcroft M, Walters EH, Wood-Baker R, Walters JA. Action plans with brief patient education for exacerbations in chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2016 Dec 19;12:CD005074 [FREE Full text] [CrossRef] [Medline]
- Lenferink A, Brusse-Keizer M, van der Valk PD, Frith PA, Zwerink M, Monninkhof EM, et al. Self-management interventions including action plans for exacerbations versus usual care in patients with chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2017 Aug 4;8:CD011682 [FREE Full text] [CrossRef] [Medline]
- Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, et al. A user-centered model for designing consumer mobile health (mHealth) applications (apps). J Biomed Inform 2016 Apr;60:243-251 [FREE Full text] [CrossRef] [Medline]
- World Health Organization. Mhealth. New Horizons for Health Through Mobile Technologies. Second Global Survey on Ehealth. 2011. URL: https://www.who.int/goe/publications/goe_mhealth_web.pdf [accessed 2018-08-17]
- McCabe C, McCann M, Brady AM. Computer and mobile technology interventions for self-management in chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2017 May 23;5:CD011425 [FREE Full text] [CrossRef] [Medline]
- Sul A, Lyu D, Park D. Effectiveness of telemonitoring versus usual care for chronic obstructive pulmonary disease: A systematic review and meta-analysis. J Telemed Telecare 2020 May;26(4):189-199. [CrossRef] [Medline]
- Goldstein RS, O'Hoski S. Telemedicine in COPD: time to pause. Chest 2014 May;145(5):945-949. [CrossRef] [Medline]
- McKinstry B. The use of remote monitoring technologies in managing chronic obstructive pulmonary disease. QJM 2013 Oct;106(10):883-885. [CrossRef] [Medline]
- Alwashmi M, Hawboldt J, Davis E, Marra C, Gamble J, Abu Ashour W. The effect of smartphone interventions on patients with chronic obstructive pulmonary disease exacerbations: a systematic review and meta-analysis. JMIR Mhealth Uhealth 2016 Sep 1;4(3):e105 [FREE Full text] [CrossRef] [Medline]
- Bolton CE, Waters CS, Peirce S, Elwyn G, EPSRCMRC Grand Challenge Team. Insufficient evidence of benefit: a systematic review of home telemonitoring for COPD. J Eval Clin Pract 2011 Dec;17(6):1216-1222. [CrossRef] [Medline]
- McLean S, Nurmatov U, Liu JL, Pagliari C, Car J, Sheikh A. Telehealthcare for chronic obstructive pulmonary disease: cochrane review and meta-analysis. Br J Gen Pract 2012 Nov;62(604):e739-e749 [FREE Full text] [CrossRef] [Medline]
- Farias R, Sedeno M, Beaucage D, Drouin I, Ouellet I, Joubert A, et al. Innovating the treatment of COPD exacerbations: a phone interactive telesystem to increase COPD Action Plan adherence. BMJ Open Respir Res 2019;6(1):e000379 [FREE Full text] [CrossRef] [Medline]
- Boer L, Bischoff E, van der Heijden M, Lucas P, Akkermans R, Vercoulen J, et al. A smart mobile health tool versus a paper action plan to support self-management of chronic obstructive pulmonary disease exacerbations: randomized controlled trial. JMIR Mhealth Uhealth 2019 Oct 9;7(10):e14408 [FREE Full text] [CrossRef] [Medline]
- Korpershoek YJ, Hermsen S, Schoonhoven L, Schuurmans MJ, Trappenburg JC. User-centered design of a mobile health intervention to enhance exacerbation-related self-management in patients with chronic obstructive pulmonary disease (copilot): mixed methods study. J Med Internet Res 2020 Jun 15;22(6):e15449 [FREE Full text] [CrossRef] [Medline]
- Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
- Walters JA, Turnock AC, Walters EH, Wood-Baker R. Action plans with limited patient education only for exacerbations of chronic obstructive pulmonary disease. Cochrane Database Syst Rev 2010 May 12(5):CD005074. [CrossRef] [Medline]
- Effing T. Action plans and case manager support may hasten recovery of symptoms following an acute exacerbation in patients with chronic obstructive pulmonary disease (COPD). J Physiother 2012;58(1):60 [FREE Full text] [CrossRef] [Medline]
- Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004 Feb;37(1):56-76 [FREE Full text] [CrossRef] [Medline]
- Nielsen J. Usability Engineering. Boston, MA: Academic Press Inc; 1993.
- Long Alliantie Nederland (LAN). Zorgstandaard COPD [COPD Care Standard]. 2016. URL: http://www.longalliantie.nl/files/9014/5578/9160/LAN_Zorgstandaard_COPD-2016-2.pdf [accessed 2017-06-01]
- Brooke J. System Usability Scale (SUS). 1986. URL: https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html [accessed 2018-09-02]
- Science Direct. System Usability Scale. 2019. URL: https://www.sciencedirect.com/topics/computer-science/system-usability-scale [accessed 2019-05-23]
- Gagnon M, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc 2016 Jan;23(1):212-220. [CrossRef] [Medline]
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
- Lincoln YS, Guba EG. Naturalistic Inquiry. Newbury Park, CA: Sage Publications; 1985.
- Bestall JC, Paul EA, Garrod R, Garnham R, Jones PW, Wedzicha JA. Usefulness of the Medical Research Council (MRC) dyspnoea scale as a measure of disability in patients with chronic obstructive pulmonary disease. Thorax 1999 Jul;54(7):581-586 [FREE Full text] [CrossRef] [Medline]
- van der Molen R, Willemse BW, Schokker S, ten Hacken NH, Postma DS, Juniper EF. Development, validity and responsiveness of the clinical COPD questionnaire. Health Qual Life Outcomes 2003 Apr 28;1:13 [FREE Full text] [CrossRef] [Medline]
- Alwashmi MF, Fitzpatrick B, Davis E, Gamble J, Farrell J, Hawboldt J. Perceptions of health care providers regarding a mobile health intervention to manage chronic obstructive pulmonary disease: qualitative study. JMIR Mhealth Uhealth 2019 Jun 10;7(6):e13950 [FREE Full text] [CrossRef] [Medline]
- Miller RH, Sim I. Physicians' use of electronic medical records: barriers and solutions. Health Aff (Millwood) 2004;23(2):116-126. [CrossRef] [Medline]
- Craig P, Di Ruggiero E, Frohlich KL, Mykhalovskiy E, White M. Taking Account of Context in Population Health Intervention Research: Guidance for Producers, Users and Funders of Research. Southampton, UK: NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.
- Michie S, Atkins L, West R. The Behaviour Change Wheel. A Guide to Designing Interventions. Great Britain, UK: Silverback Publishing; 2014.
- Feiring E, Friis T. Facilitators and barriers to clinicians' use of COPD action plans in self-management support: A qualitative study. Patient Educ Couns 2020 Apr;103(4):693-701 [FREE Full text] [CrossRef] [Medline]
- Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, et al. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation (Lond) 2019 Jan;25(1):23-45 [FREE Full text] [CrossRef] [Medline]
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance. Developing and evaluating complex interventions: the new medical research council guidance. Br Med J 2008 Sep 29;337:a1655 [FREE Full text] [CrossRef] [Medline]
|COPD: chronic obstructive pulmonary disease|
|GDPR: General Data Protection Regulation|
|GOLD: Global Initiative for Chronic Obstructive Lung Disease|
|HCP: health care provider|
|mHealth: mobile health|
|SUS: System Usability Scale|
Edited by G Eysenbach; submitted 18.06.20; peer-reviewed by MF Alwashmi, M Peeples; comments to author 24.08.20; revised version received 17.09.20; accepted 21.09.20; published 19.11.20
©Yvonne JG Korpershoek, Tjitske Holtrop, Sigrid CJM Vervoort, Lisette Schoonhoven, Marieke J Schuurmans, Jaap CA Trappenburg. Originally published in JMIR Formative Research (http://formative.jmir.org), 19.11.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.