Published on in Vol 5, No 3 (2021): March

Preprints (earlier versions) of this paper are available at, first published .
A Mobile Patient-Facing App for Tracking Patient-Reported Outcomes in Head and Neck Cancer Survivors: Single-Arm Feasibility Study

A Mobile Patient-Facing App for Tracking Patient-Reported Outcomes in Head and Neck Cancer Survivors: Single-Arm Feasibility Study

A Mobile Patient-Facing App for Tracking Patient-Reported Outcomes in Head and Neck Cancer Survivors: Single-Arm Feasibility Study

Original Paper

1Academic Department of Radiation Medicine, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, New York, NY, United States

2Department of Radiation Medicine, Northwell Health Cancer Institute, Lake Success, NY, United States

3Center for Health Innovations and Outcomes Research, Department of Medicine, Northwell Health, Manhasset, NY, United States

4Center for Research Informatics & Innovation, Feinstein Institutes for Medical Research, Northwell Health, Manhasset, NY, United States

5Department of Otolaryngology, Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States

6Department of Otolaryngology, Northwell Health Cancer Institute, Lake Success, NY, United States

Corresponding Author:

Sewit Teckie, MD

Academic Department of Radiation Medicine

Donald and Barbara Zucker School of Medicine at Hofstra/Northwell

130 East 77th Street

New York, NY, 10010

United States

Phone: 1 212 434 2919


Background: Patients with head and neck cancer (HNC) frequently experience disease-related symptoms and treatment adverse effects that impact their overall quality of life. Cancer-specific mobile health apps for patient-related outcomes allow patients to communicate with their clinicians and proactively track their symptoms, which have been shown to improve clinical management and disease outcomes.

Objective: The purpose of this study was to evaluate the feasibility of LogPAL, a novel iPhone-based mobile health app designed to help HNC survivors track and manage their posttreatment symptoms.

Methods: Patients who completed curative treatment for HNC in the preceding 24 months were recruited from 2 clinical sites within a single institution. Upon enrollment, participants completed a brief sociodemographic survey, downloaded the app onto their iPhone devices, and were asked to complete a series of biweekly questionnaires (based on the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events) via the app for an 8-week study period. The primary feasibility endpoints included retention (retaining >80% of the enrolled participants for the duration of the study period), adherence (>50% of the participants completing 100% of the questionnaires over the study period), and usability (a mean system usability scale [SUS] score >68). Additional postintervention questions were collected to assess perceived usefulness, acceptance, and overall satisfaction.

Results: Between January and October 2019, 38 participants were enrolled in the study. Three participants dropped out, and 3 were classified as nonusers. The remaining 32 (87%) were eligible for analysis. Their mean age was 57.8 (SD 12.3) years (range 24-77 years, 81% [26/32] male). Overall, 375 of 512 (73.2%) questionnaires were completed, with 17 (53%) of the 32 participants adherent. Participant-reported usability was acceptable; the mean SUS score was 71.9 (95% CI 64.3-79.5) with high satisfaction of LogPAL usefulness and likelihood to recommend to other cancer survivors.

Conclusions: This single-arm prospective pilot study showed that LogPAL is a feasible, regularly used, accepted app for HNC survivors, justifying a full-scale pilot. Based on the findings from this study, future iterations will aim to improve usability and test intervention efficacy.

JMIR Form Res 2021;5(3):e24667




The demographics of head and neck cancer (HNC) are changing [1-4], with an increase in the incidence of HNC in younger patients without significant smoking or alcohol use history. HNC survival rates are increasing as a result of these changing demographics and more effective multidisciplinary treatment options [5-8]. Despite these advances in disease outcomes, HNC survivors often experience significant toxicities and unique functional impairments such as dysphagia, mucositis, xerostomia, and dysphonia, which are distinctive from those reported in other cancer survivors [9-11]. These can lead to debilitating and lifelong consequences due to adverse effects during and after treatment, which impair the quality of life. Nevertheless, patients commonly underreport symptoms or delay reporting of symptoms, which, in turn, lead to delay in the best clinical management [12,13]. Mobile health (mHealth) interventions such as smartphone apps have been advocated as promising strategies in patient self-management [14]. These tools have the potential to increase accessibility of patient’s health information, provide real-time reporting of the concerning symptoms to providers, and improve overall patient satisfaction via proactive self-management care [15-19]. The proliferation of mHealth apps, with over 325,000 mHealth apps developed [20], is changing how patients interact with the health care system. Yet, there remains a dearth in knowledge regarding the feasibility of using mHealth electronic patient-reported outcomes (ePROs) to assess and address the vulnerable population of HNC survivors. To date, only few studies have evaluated a smartphone-based self-management system during HNC treatment [16,21]. To fill this knowledge gap, our study aimed to evaluate the feasibility of LogPAL, a novel patient-facing mHealth app specific for HNC survivors.

Objectives of This Study

The primary objectives of this study were to (1) assess the feasibility of LogPAL and (2) explore the perceived usefulness, acceptance, and overall satisfaction through validated questionnaires and participant feedback. We hypothesized that there would be an >80% retention rate (proportion of enrolled participants who completed at least one questionnaire during the study period), >50% adherence rate (proportion of participants who will complete 100% of scheduled questionnaires), and a mean system usability scale (SUS) score >68. Secondary exploratory objectives reviewed additional engagement metrics and preliminary associations between outcome measures and participant characteristics (covariates). As there is no consensus on how best to evaluate mHealth pilot studies, criteria selection and determination were based on insights from the one of the co-principal investigators (MAD) who is experienced in evaluating web-based and app-based health tools for patients with cancer.

Recruitment and Enrollment

All HNC survivors were screened for eligibility from 2 Northwell Health otolaryngology surgical oncology clinics. Potential participants were identified through a review of electronic medical records to determine if the following inclusion criteria were met prior to their upcoming in-clinic appointment: HNC survivors ≥18 years old who completed all curative treatments for primary HNC (lip/oral cavity, pharynx, larynx, salivary gland, paranasal sinus) within the preceding 24 months, who had no serious self-reported cognitive impairment, who were able to read and speak English fluently, and who had access to a smartphone that operates the iPhone operating system software (iPhone/iPad). Eligible participants were contacted by phone to discuss meetings about enrollment at the clinic during an upcoming appointment. Upon meeting, additional details about the study were reviewed and participants were informed that participation was voluntary. If informed consent was provided, participants underwent a training session with a study team member on how to use the app, with additional time for any questions prior to the first log-in.

Study Design

A nonrandomized, prospective, single-arm pilot study was carried out from January to October 2019. The study period consisted of an 8-week intervention during which participants were instructed to answer and complete ePROs twice weekly for 16 sessions. The frequency and duration of the ePROs were determined by a team of HNC specialists to reflect clinically relevant timepoints that encouraged self-monitoring, but did not compromise HNC patient safety during their recovery. This study was reviewed and approved by the Institutional Review Board of Northwell Health.

LogPAL App

LogPAL Development

The LogPAL app was developed using an iterative user-centered design approach [22,23], which systematically engaged end users to identify requirements and app core functionality to enhance the effectiveness, efficiency, and usability [24,25]. The goal was to develop a patient-facing mHealth app that could provide HNC survivors with a tool to better manage and track their symptoms by self-reporting any adverse effects and offer immediate informational resources to help develop and practice health self-management skills.

LogPAL Overview

For this pilot study, a version of LogPAL, currently available in the Apple App Store, that consisted of the following 4 core features was used: (1) Start Tracking, (2) View My Progress, (3) Self Care Tips, and (4) Resources (Figure 1). Users were not specifically required to access other features other than Start Tracking during the intervention period.

Figure 1. Snapshots and overview of the LogPAL app features.
View this figure

Structure of the PROs

A preliminary list of PROs was selected from the existing validated National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events library [26]. Individual items were selected by the study team for their relevance to symptoms and complications experienced by patients with HNC. To allow for iterative refinement, 2-week usability testing sessions were conducted using a purposive cohort of patients with HNC. At the end of the 2-week period, participants engaged in focus group interviews, which used the think-aloud method to elicit feedback regarding the perceived understanding of PRO questions and overall experience using the app. PROs were rated on a 5-tier scale ranging from “none” to “very severe.” Questions were categorized into 10 common disease-specific symptoms, which included difficulty swallowing and chewing, dry mouth, loss of appetite, changes in taste, impaired speech, sores/pain in the mouth, overall pain, cough, nausea, and fatigue (Figure 2). Through this careful selection process, 2 versions were created. A weekly questionnaire (consisting of a series of 26 symptom-based questions) and a monthly questionnaire (weekly questionnaire with an additional 16 questions) were asked at the end of every month.

Figure 2. Examples of patient-reported outcome questions and illustrations of the relevant adverse effects. (Figure created on BioRender).
View this figure

Data Collection

Questionnaires and app-generated data analytics were used to record participant sociodemographic characteristics at baseline, to measure engagement throughout the study period, and to perform postintervention analysis. Questionnaire data were entered into the Research Electronic Data Capture (REDCap) software and encrypted and stored in a secure server, which was selected and approved by the Office of the Chief Information Officer. REDCap’s web-based app uses secure 2-factor web authentication, data logging, and encryption that ensures the security and confidentiality of private information for obtaining informed consent [27].


Patient Demographics

Participants completed an 18-item survey at enrollment. The survey included demographic characteristics about the participant’s sex, racial and ethnic background, highest level of education, and marital status. Health-related information about the treatment type received, year of diagnosis, number of years since last treatment, smoking history, self-reported physical health and physical activity relative to peers, comorbidities, and clinic site of enrollment were also obtained. Self-reported physical health and physical activity relative to peers were determined using a 5-point Likert scale (excellent, very good, good, fair, and poor) and a 3-point Likert scale (more, about the same, less), respectively.

Follow-up Survey

After 8 weeks of using the app, participants were invited to complete a postintervention survey via email. The survey comprised of (1) the validated SUS, a standardized questionnaire commonly used to assess participants’ perceptions of usability of an electronic system or device [28,29]; (2) a 5-point Likert response scale (1=strongly disagree to 5=strongly agree) acceptability questions; and (3) open-ended questions to prompt ideas for app improvement.

Outcome Measures

For this feasibility study, a priori criteria were defined as ≥80% retention rate (defined as the number of participants who enrolled and completed at least one questionnaire during the study period), ≥50% adherence rate (defined as the percentage of participants who completed all scheduled sessions), and mean SUS>68. This criterion was based on prior studies that assessed feasibility in applied intervention research [30-32] and self-management apps among cancer survivors [33]. To assess implementation outcomes such as recruitment and retention, rates were tabulated based on data collected from the research team tracking logs and summarized using a CONSORT diagram. Secondary exploratory outcomes tracked additional engagement metrics such as number of log-ins and frequency of participant interactions with other features [34]. Insights on different types of engagement indicators could provide opportunities for designing more engaging and clinically effective mHealth interventions [35].

Statistical Analysis

For this feasibility study, 50 participants were sought to review the app. This estimate was drawn on prior experiences accruing participants [36]. Taking into consideration a retention rate of 80% of recruited patients, it is estimated that 41 patients will remain in the study. All statistical analyses were conducted using Prism 8.4.0 (GraphPad Software) and SPSS version 25.0 (IBM Corp). Descriptive statistics was utilized to summarize participant characteristics and rates of engagement. Categorical variables were reported as frequencies (n) and percentages (%); continuous variables were reported as mean (SD) or median (IQR), that is, 25th-75th percentile, as needed. Nonparametric Mann-Whitney U test for 2 independent groups or the Kruskal-Wallis test for more than 2 groups were used for exploratory analyses of adherence scores (out of 16). Qualitative responses to open-ended questions were categorized using a thematic analysis, and relevant quotations were included to illustrate those themes. Results with P values less than .05 were considered to be statistically significant.

Recruitment and Enrollment

A total of 315 patients were screened, of which 90 (28.6%) met the eligibility criteria. Of the 90 patients, 38 (42%) enrolled in the study. The primary reason for ineligibility was the lack of an iPhone device (133/315, 42.2%). The primary reason that eligible patients declined to participate was limited time and disinterest (50/90, 56%). Attrition rate was 16% (6/38) consisting of 3 dropouts and 3 classifieds as nonusers (participants who did not complete at least one PRO questionnaire during the study period) (Figure 3).

Figure 3. CONSORT flowchart for this study.
View this figure

Participant Characteristics

Participants were predominantly males (26/32, 81%), Whites (25/32, 78%), and married (21/32, 66%), reflecting the regional demographics of HNC. The mean age of the participants was 57.8 (SD 12.3) years (range 24-77 years), and the median time from treatment completion to study enrollment was 10.8 months (range 1-23 months). The majority were college educated, including some college (7/32, 22%), bachelor’s degree (9/32, 28%), or higher (9/32, 28%) (Table 1).

Table 1. Sociodemographic and clinical characteristics of the participants (N=32).
Characteristicsa,bValues, n (%)
Female6 (19)
Male26 (81)
White25 (78)
Black3 (9)
Asian2 (6)
Other2 (6)
Marital status
Single5 (16)
Married21 (66)
Divorced5 (16)
Widow1 (3)
Highest level of education
High school/GED7 (22)
Some college7 (22)
Bachelors or equivalent9 (28)
> Bachelor’s9 (28)
Type of treatment
Radiation6 (19)
Chemotherapy14 (44)
Combination12 (38)
Self-reported physical activitycompared to others in their age group
More6 (19)
Less6 (19)
The same20 (63)
Self-reported physical health
Poor/Fair8 (25)
Good14 (44)
Very Good/Excellent10 (31)
Site 126 (81)
Site 26 (19)

aMean (SD) age in years = 57.8 (12.3) years.

bMean (SD) time since last treatment = 10.8 (8.0) months.

App Use Tracking

Of the 32 participants eligible for analysis, 17 (53%) completed all of the scheduled sessions, 20 (63%) completed 75% or more of the sessions, and 25 (78%) completed at least 50% of the scheduled sessions by the end of the study period. Overall, 73.2% (375/512) of the questionnaires were completed (range 6.25%-100%) with participants opening the app 693 times over the course of 8 weeks.

Postintervention Survey


At the end of the study, 17 of the 32 participants (53%) reconsented to complete the SUS. The mean SUS score (95% CI) was 71.9 (64.3-79.5), which was an “acceptable” rating based on the standard SUS [28]. Further analysis of the subscales showed that the mean SUS (95% CI) learnability domain was 78.7 (71.2-86.1) and the mean (95% CI) usability domain was 70.2 (61.8-78.7). In the responses to the SUS questionnaire, 88% (15/17) found LogPAL “easy to use,” 94% (16/17) felt that “most people could learn to use LogPAL very quickly,” and 82% (14/17) felt “very confident using the system.”


Among the participants, 76% (13/17) agreed that LogPAL was useful, with 59% (10/17) and 71% (12/17) agreeing with the frequency and length of the PROs, respectively. Additionally, 76% (13/17) of the participants agreed that they would recommend LogPAL to other cancer survivors. A total of 5 (29%) of the 17 participants gave additional feedback on their experience using LogPAL. In terms of what they liked, participants used the words “informative,” “helpful,” and “valuable.” One patient stated, “I thought it was definitely helpful, it really let you understand what was going on with your body and it just wasn’t you experiencing these symptoms.” In terms of areas of improvements, participants mentioned the need for the app to be available to patients during or immediately after treatment, an alert tone as a reminder to update, and the ability to provide additional comments on changes. As one patient mentioned, the app “...does not allow for comments when an answer changes due to new variable in my situation, to explain why I changed my answer from previous weeks.”

Exploratory Analyses

Additional App Features

Engagement with additional app features were reviewed among eligible participants. Findings showed that 32 (100%) used View My Progress, 27 (84%) used the Self-Care Tips, and 5 (16%) used the Resource feature. During the study period, the View My Progress feature was clicked 3445 times and the Self-Care Tips was clicked 79 times.

Relationship Among Covariates

Analysis of mean (SD) adherence scores and participant characteristics showed higher scores among those who self-reported conducting “more” physical activity (16.0 [SD 0.0]) than among those who self-reported “about the same” (12.2 [SD 4.9]) as others their age (P=.04). Similarly, the scores were higher among participants who self-reported their physical health as “very good”/“excellent” (14.5 [SD 4.4]) than among those who self-reported their physical health as “fair”/“poor” (7.8 [SD 2.1]) (P=.05). Lastly, the mean (SD) scores of the participants at clinic site 1 (13.2 [SD 4.5]) were higher than those of the participants at clinic site 2 (5.8 [SD 5.7]) (P=.003).

Principal Results

To the best of our knowledge, this is the first study to evaluate the feasibility of a patient-facing mobile app that collects ePROs specifically for HNC survivors. Our results indicate that participants considered LogPAL as a feasible approach by meeting the a priori criteria. Our findings highlight the potential that mHealth apps have to improve symptom control and promote self-management of symptoms to improve health outcomes and quality of life. These findings are encouraging now more than ever, as the COVID-19 pandemic has placed greater importance on remote technology-enabled monitoring of high-risk patients through a digital platform. We believe that there is a greater need to develop an approach that allows HNC survivors to feel that their unique symptom needs are met and to have the ability to access straightforward information and valued material that address relevant issues.

Comparison With Prior Work

With the ubiquity of mobile phones in our society, there is a growing interest in and use of mobiles apps for patients with cancer to self-manage symptoms during treatment and those that persist into survivorship [37]. These apps have the potential to provide individually tailored self-management advice for different participants during their survivorship at home [38]. Furthermore, mHealth apps can increase patient engagement in their own recovery, provide better patient-provider communication, and flag patients at risk for readmission, thereby facilitating potential early interventions. Results from this study demonstrate that participant adherence (17/32, 53%) was congruent with that reported in previous studies that defined usage as over 50% of the participants that actually use an eHealth PRO intervention as intended. Other recent single-arm pilot trials have shown higher adherence rates; 1 mHealth ePRO study among patients with prostate cancer found that 86% (25/29) of the participants satisfactorily completed 60% of the weekly questions over a 3-month period [39]. Another pilot study of 10 patients with gynecological cancers who received palliative chemotherapy showed >70% adherence to daily smartphone surveys >4 days per week [40]. One explanatory factor for this inconsistency is the lack of clarity regarding evaluation methodologies, leading to substantial heterogeneity in the reported outcomes [41]. Recent attempts to meaningfully summarize indicators used in pilot studies have led to the creation of a lexicon of the most commonly used terms to identify effective app components [34,42]; however, a conceptually coherent framework is yet to be adapted. Another possibility to explain the lower engagement is the relationship between usage and descriptive variables such as age, marital status, years of education, and socioeconomic status [43]. Concomitantly, we did not find a significant relationship between ePRO usage among patients with HNC and the common descriptive mediating factors [44]. However, additional variables such as physical health, physical activity, and site of recruitment were not previously investigated. As such, these findings were unexpected and suggest that those who have self-perceived better physical health and those who are more physically active than those of their age may lead to higher engagement. It is important to note that there is strong evidence to show that usage of mobile phones and wearable devices increases physical activity and health, which significantly reduces cancer-related symptoms/side effects, leads to greater quality of life [45,46], as well as good retention rates and adherence. However, further investigation is needed to determine the relationships.

Additionally, clinic site 2 had notably lower retention and adherence rates compared to clinic site 1. Clinic site 2 is located in an ambulatory clinic within an urban hospital with 1 surgeon seeing those patients. In contrast, clinic site 1 is located in an outpatient ambulatory clinic in a suburban hospital campus, with 3 different surgeons seeing patients. The recruitment staff at site 1 were different from those at site 2. Lastly, our results corroborated the need to develop cancer-supportive digital interventions that are interactive and tailored [47]. Integrating relevant information such as Self Care Tips as participants progress has been suggested as a way to maintain and enhance patients’ experience of personal relevance [42]. In the future iterations, it might be beneficial to integrate an option for participants to include or supplant symptoms or concerns in PROs to make it more person-facing.


This study is not without limitations. First, the app is currently only available on iPhone operating system (Apple iPhone and iPad) devices, which potentially caused selection bias (during enrollment) for age, ethnicity/race, education, and socioeconomic factors. Second, although the demographic characteristics of our cohort were slightly more heterogenous than those in most cancer pilot studies [48], this study was conducted in a single multi-site health system, thereby limiting its generalizability and not typifying those of the wider population. Third, patients who were enrolled in the study may be healthier than a general HNC follow-up population, thereby potentially biasing the engagement and usage figures to appear higher than they would be in a generalized population. Lastly, more rigorous recruitment protocols are needed to ascertain equitable retention rates. Additional updates and development of an Android version will help reduce these biases.


This study has provided preliminary evidence to suggest that the LogPAL app is a feasible and acceptable mHealth intervention that collects PROs to improve symptom management and proactively detect serious downstream complications among HNC survivors. The success of this feasibility study presents support for conducting a larger, multisite, randomized clinical trial to assess the efficacy of LogPAL with an active control and more heterogenous sample size.


The authors would like to thank all those at the Center for Health Innovations and Outcomes Research who helped with the development of the app as well as those who provided additional support with the recruitment of participants for this study.

Conflicts of Interest

None declared.

  1. Ferlay J, Colombet M, Soerjomataram I, Mathers C, Parkin D, Piñeros M, et al. Estimating the global cancer incidence and mortality in 2018: GLOBOCAN sources and methods. Int J Cancer 2019 Apr 15;144(8):1941-1953 [FREE Full text] [CrossRef] [Medline]
  2. GBD 2016 Disease Injury Incidence Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 2017 Sep 16;390(10100):1211-1259 [FREE Full text] [CrossRef] [Medline]
  3. Murphy C, Galloway T, Handorf E, Egleston B, Wang L, Mehra R, et al. Survival Impact of Increasing Time to Treatment Initiation for Patients With Head and Neck Cancer in the United States. JCO 2016 Jan 10;34(2):169-178 [FREE Full text] [CrossRef]
  4. Miller M, Shuman A, American HeadNeck Society’s Committee on Survivorship. Survivorship in Head and Neck Cancer: A Primer. JAMA Otolaryngol Head Neck Surg 2016 Oct 01;142(10):1002-1008. [CrossRef] [Medline]
  5. So W, Wong C, Choi K, Chan C, Chan J, Law B, et al. A Mixed-Methods Study of Unmet Supportive Care Needs Among Head and Neck Cancer Survivors. Cancer Nurs 2019;42(1):67-78 [FREE Full text] [CrossRef]
  6. Lo Nigro C, Denaro N, Merlotti A, Merlano M. Head and neck cancer: improving outcomes with a multidisciplinary approach. CMAR 2017 Aug;Volume 9:363-371 [FREE Full text] [CrossRef]
  7. Cramer J, Burtness B, Le Q, Ferris R. The changing therapeutic landscape of head and neck cancer. Nat Rev Clin Oncol 2019 Nov;16(11):669-683. [CrossRef] [Medline]
  8. Jackson L, Ridner S, Deng J, Bartow C, Mannion K, Niermann K, et al. Internal Lymphedema Correlates with Subjective and Objective Measures of Dysphagia in Head and Neck Cancer Patients. J Palliat Med 2016 Sep;19(9):949-956 [FREE Full text] [CrossRef] [Medline]
  9. Cramer J, Johnson J, Nilsen M. Pain in Head and Neck Cancer Survivors: Prevalence, Predictors, and Quality-of-Life Impact. Otolaryngol Head Neck Surg 2018 Nov;159(5):853-858. [CrossRef] [Medline]
  10. Rogers L, Courneya K, Robbins K, Malone J, Seiz A, Koch L, et al. Physical activity and quality of life in head and neck cancer survivors. Support Care Cancer 2006 Oct;14(10):1012-1019. [CrossRef] [Medline]
  11. Murphy B, Gilbert J, Ridner S. Systemic and global toxicities of head and neck treatment. Expert Rev Anticancer Ther 2007 Jul;7(7):1043-1053. [CrossRef] [Medline]
  12. Pakhomov SV, Jacobsen SJ, Chute CG, Roger VL. Agreement between patient-reported symptoms and their documentation in the medical record. Am J Manag Care 2008 Aug;14(8):530-539 [FREE Full text] [Medline]
  13. Hansen D, Larsen P, Holm L, Rottmann N, Bergholdt S, Søndergaard J. Association between unmet needs and quality of life of cancer patients: A population-based study. Acta Oncologica 2012 Dec 17;52(2):391-399 [FREE Full text] [CrossRef]
  14. Faiola A, Papautsky E, Isola M. Empowering the Aging with Mobile Health: A mHealth Framework for Supporting Sustainable Healthy Lifestyle Behavior. Curr Probl Cardiol 2019 Aug;44(8):232-266 [FREE Full text] [CrossRef] [Medline]
  15. Basch E, Deal A, Dueck A, Scher HI, Kris MG, Hudis C, et al. Overall Survival Results of a Trial Assessing Patient-Reported Outcomes for Symptom Monitoring During Routine Cancer Treatment. JAMA 2017 Jul 11;318(2):197-198 [FREE Full text] [CrossRef] [Medline]
  16. Falchook A, Tracton G, Stravers L, Fleming ME, Snavely AC, Noe JF, et al. Use of mobile device technology to continuously collect patient-reported symptoms during radiation therapy for head and neck cancer: A prospective feasibility study. Adv Radiat Oncol 2016;1(2):115-121 [FREE Full text] [CrossRef] [Medline]
  17. Benze G, Nauck F, Alt-Epping B, Gianni G, Bauknecht T, Ettl J, et al. PROutine: a feasibility study assessing surveillance of electronic patient reported outcomes and adherence via smartphone app in advanced cancer. Ann Palliat Med 2019 Apr;8(2):104-111 [FREE Full text] [CrossRef] [Medline]
  18. Childes JM, Palmer AD, Fried-Oken MB. Communication Support Before, During, and After Treatment for Head and Neck Cancer. In: Doyle P, editor. Clinical Care and Rehabilitation in Head and Neck Cancer. Cham, Switzerland: Springer Nature; 2019:247-264.
  19. Aapro M, Bossi P, Dasari A, Fallowfield L, Gascón P, Geller M, et al. Digital health for optimal supportive care in oncology: benefits, limits, and future perspectives. Support Care Cancer 2020 Oct;28(10):4589-4612 [FREE Full text] [CrossRef] [Medline]
  20. 325,000 mobile health apps available in 2017? Android now the leading mHealth platform. Research2Guidance. 2017.   URL: [accessed 2020-09-29]
  21. Zini E, Lanzola G, Quaglini S, Bossi P, Licitra L, Resteghini C. A pilot study of a smartphone-based monitoring intervention on head and neck cancer patients undergoing concurrent chemo-radiotherapy. Int J Med Inform 2019 Sep;129:404-412. [CrossRef] [Medline]
  22. De Vito Dabbs A, Myers B, Mc Curry KR, Dunbar-Jacob J, Hawkins RP, Begey A, et al. User-centered design and interactive health technologies for patients. Comput Inform Nurs 2009;27(3):175-183 [FREE Full text] [CrossRef] [Medline]
  23. Matthew-Maich N, Harris L, Ploeg J, Markle-Reid M, Valaitis R, Ibrahim S, et al. Designing, Implementing, and Evaluating Mobile Health Technologies for Managing Chronic Conditions in Older Adults: A Scoping Review. JMIR Mhealth Uhealth 2016 Jun 09;4(2):e29 [FREE Full text] [CrossRef] [Medline]
  24. Nacheva R. Standardization Issues of Mobile Usability. Int J Interact Mob Technol 2020 May 06;14(07):149 [FREE Full text] [CrossRef]
  25. Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
  26. Dueck A, Mendoza T, Mitchell S, Reeve BB, Castro KM, Rogak LJ, National Cancer Institute PRO-CTCAE Study Group. Validity and Reliability of the US National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol 2015 Nov;1(8):1051-1059 [FREE Full text] [CrossRef] [Medline]
  27. Harris P, Taylor R, Thielke R, Payne J, Gonzalez N, Conde J. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
  28. Bangor A, Kortum P, Miller J. An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 2008 Jul 30;24(6):574-594 [FREE Full text] [CrossRef]
  29. Lewis JR, Sauro J. The Factor Structure of the System Usability Scale. In: Kurosu M, editor. Human Centered Design. San Diego, CA, USA: Springer; 2009:94-103.
  30. Kazak A, Simms S, Alderfer M, Rourke MT, Crump T, McClure K, et al. Feasibility and preliminary outcomes from a pilot study of a brief psychological intervention for families of children newly diagnosed with cancer. J Pediatr Psychol 2005 Dec;30(8):644-655. [CrossRef] [Medline]
  31. Diefenbach M, Mohamed N, Butz B, Bar-Chama N, Stock R, Cesaretti J, et al. Acceptability and preliminary feasibility of an internet/CD-ROM-based education and decision program for early-stage prostate cancer patients: randomized pilot study. J Med Internet Res 2012 Jan 13;14(1):e6 [FREE Full text] [CrossRef] [Medline]
  32. Zhao J, Freeman B, Li M. Can Mobile Phone Apps Influence People's Health Behavior Change? An Evidence Review. J Med Internet Res 2016 Oct 31;18(11):e287 [FREE Full text] [CrossRef] [Medline]
  33. Melissant H, Verdonck-de Leeuw I, Lissenberg-Witte B, Konings I, Cuijpers P, Van Uden-Kraan C. ‘Oncokompas’, a web-based self-management application to support patient activation and optimal supportive care: a feasibility study among breast cancer survivors. Acta Oncologica 2018 Feb 16;57(7):924-934 [FREE Full text] [CrossRef]
  34. Pham Q, Graham G, Carrion C, Morita PP, Seto E, Stinson JN, et al. A Library of Analytic Indicators to Evaluate Effective Engagement with Consumer mHealth Apps for Chronic Conditions: Scoping Review. JMIR Mhealth Uhealth 2019 Jan 18;7(1):e11941 [FREE Full text] [CrossRef] [Medline]
  35. Zhang R, Nicholas J, Knapp A, Graham AK, Gray E, Kwasny MJ, et al. Clinically Meaningful Use of Mental Health Apps and its Effects on Depression: Mixed Methods Study. J Med Internet Res 2019 Dec 20;21(12):e15644 [FREE Full text] [CrossRef] [Medline]
  36. Leach C, Diefenbach M, Fleszar S, Alfano, Stephens RL, Riehman K, et al. A user centered design approach to development of an online self-management program for cancer survivors: Springboard Beyond Cancer. Psychooncology 2019 Oct;28(10):2060-2067. [CrossRef] [Medline]
  37. Osborn J, Ajakaiye A, Cooksley T, Subbe C. Do mHealth applications improve clinical outcomes of patients with cancer? A critical appraisal of the peer-reviewed literature. Support Care Cancer 2020 Mar;28(3):1469-1479 [FREE Full text] [CrossRef] [Medline]
  38. Ruland C, Andersen T, Jeneson A, Moore S, Grimsbø GH, Børøsund E, et al. Effects of an internet support system to assist cancer patients in reducing symptom distress: a randomized controlled trial. Cancer Nurs 2013;36(1):6-17. [CrossRef] [Medline]
  39. Tran C, Dicker A, Leiby B, Gressen E, Williams N, Jim H. Utilizing Digital Health to Collect Electronic Patient-Reported Outcomes in Prostate Cancer: Single-Arm Pilot Trial. J Med Internet Res 2020 Mar 25;22(3):e12689 [FREE Full text] [CrossRef] [Medline]
  40. Wright A, Raman N, Staples P, Schonholz S, Cronin A, Carlson K, et al. The HOPE Pilot Study: Harnessing Patient-Reported Outcomes and Biometric Data to Enhance Cancer Care. JCO Clinical Cancer Informatics 2018 Dec(2):1-12 [FREE Full text] [CrossRef]
  41. Warrington L, Absolom K, Conner M, Kellar I, Clayton B, Ayres M, et al. Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review. J Med Internet Res 2019 Jan 24;21(1):e10875 [FREE Full text] [CrossRef] [Medline]
  42. Short C, DeSmet A, Woods C, Williams SL, Maher C, Middelweerd A, et al. Measuring Engagement in eHealth and mHealth Behavior Change Interventions: Viewpoint of Methodologies. J Med Internet Res 2018 Nov 16;20(11):e292 [FREE Full text] [CrossRef] [Medline]
  43. Jiang Y, West BT, Barton DL, Harris MR. Acceptance and Use of eHealth/mHealth Applications for Self-Management Among Cancer Survivors. Stud Health Technol Inform 2017;245:131-135 [FREE Full text] [Medline]
  44. Head B, Keeney C, Studts J, Khayat M, Bumpous J, Pfeifer M. Feasibility and Acceptance of a Telehealth Intervention to Promote Symptom Management during Treatment for Head and Neck Cancer. J Support Oncol 2011 Jan 01;9(1):e1-e11 [FREE Full text] [CrossRef] [Medline]
  45. Granger C, Connolly B, Denehy L, Hart N, Antippa P, Lin KY, et al. Understanding factors influencing physical activity and exercise in lung cancer: a systematic review. Support Care Cancer 2017 Mar;25(3):983-999. [CrossRef] [Medline]
  46. Schaffer K, Panneerselvam N, Loh K, Herrmann R, Kleckner IR, Dunne RF, et al. Systematic Review of Randomized Controlled Trials of Exercise Interventions Using Digital Activity Trackers in Patients With Cancer. J Natl Compr Canc Netw 2019 Jan;17(1):57-63 [FREE Full text] [CrossRef] [Medline]
  47. Moradian S, Voelker N, Brown C, Liu G, Howell D. Effectiveness of Internet-based interventions in managing chemotherapy-related symptoms in patients with cancer: a systematic literature review. Support Care Cancer 2018 Feb;26(2):361-374. [CrossRef] [Medline]
  48. Unger J, Cook E, Tai E, Bleyer A. The Role of Clinical Trial Participation in Cancer Research: Barriers, Evidence, and Strategies. Am Soc Clin Oncol Educ Book 2016;35:185-198 [FREE Full text] [CrossRef] [Medline]

ePRO: electronic patient-reported outcome
HNC: head and neck cancer
mHealth: mobile health
PRO: patient-reported outcome
REDCap: Research Electronic Data Capture
SUS: system usability scale

Edited by G Eysenbach; submitted 30.09.20; peer-reviewed by D Di Stasio, J Jabson; comments to author 18.11.20; revised version received 02.12.20; accepted 17.01.21; published 19.03.21


©Sewit Teckie, Jeffrey Solomon, Karthik Kadapa, Keisy Sanchez, David Orner, Dennis Kraus, Dev P Kamdar, Lucio Pereira, Douglas Frank, Michael Diefenbach. Originally published in JMIR Formative Research (, 19.03.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.