To adapt COMPASS-CP to colorectal cancer survivors who completed treatment, we adopted the Integrate, Design, Assess, and Share (IDEAS) framework, which consists of the following 10 stages: (1) empathize with target users; (2) specify the target behavior; (3) ground in behavioral theory (self-determination theory to inform future work [25]); (4) ideate implementation strategies; (5) prototype potential products; (6) gather user feedback; (7) build a minimum viable product, defined as a fully functional initial intervention with only the most essential features; (8) pilot test to assess potential efficacy and usability (current and future studies); (9) evaluate efficacy (future studies); and (10) share the intervention and findings (future studies) [26]. Stages 1-3 are part of the Integrate aspect, stages 4-7 are part of the Design aspect, stages 8 and 9 are part of the Assess aspect, and stage 10 is part of the Share aspect of the framework. This framework is implemented iteratively until a refined viable product is developed [27,28]. We also used a mixed methods approach to assess implementation outcomes first in focus groups and then after building a minimum viable product as part of usability testing.

Study procedures involving the participation of human subjects received institutional review board (IRB) approval (phase 1: IRB00059620; phase 3: IRB00064181). Participants were provided an information sheet that described study involvement. This sheet stated, “By continuing, I agree to take part in this study,” with a waiver for signed consent. Study data were deidentified. Patients and caregivers received US $50 for participation in phase 1.

Patients who completed treatment of colorectal cancer and health care professionals involved with colorectal cancer care were targeted for this application development. The study workflow is shown in Figure 1. The first component involved patients filling out measures before a clinic visit to inform provider sessions. Completion of the provider decision support tool (component 2) then facilitated shared decision-making and resulted in real-time creation of a personalized follow-up CP (component 3).

We iteratively compiled feedback to clarify the most important data elements to incorporate in creating follow-up CPs (IDEAS stages 2 and 7). The core set of data elements in relevant clinical guidelines for follow-up care of colorectal cancer included the following [14,30-32]: (1) schedule of clinic visits; (2) cancer surveillance or other recommended tests (eg, surveillance colonoscopy, history, and physical screening for other primary cancers); (3) assessment and management of physical and psychosocial long-term and late effects of colorectal cancer and its treatment (eg, bowel/gastrointestinal issues, numbness/tingling, and ostomy/stoma); (4) other late and long-term effects (depression, anxiety, fatigue, sleep disturbance, sexual function, fertility, financial toxicity, insurance, memory/cognitive function, social functioning, physical functioning, and pain); (5) health behaviors (alcohol use, diet/nutrition, obesity/weight changes, physical activity, smoking, and sunscreen use); and (6) care coordination and practice implications (communication among specialists, management of medications, and management of other illnesses). The development team evaluated the availability of the identified data elements in Epic.

COMPASS-CP currently uses SMART on FHIR, which is a data exchange standard to enhance interoperability with EHRs and other systems. The FHIR Patient Resource pushes the patient medical record number and other demographics into COMPASS-CP, which creates the patient record. It is also possible through additional FHIR resources (eg, Encounter) for COMPASS-CP to be able to communicate back the CPs and provider reports to Epic.

We identified appropriate patient-completed measures for assessing other essential data elements. To streamline the integration of the patient-completed data in this study, we integrated the REDCap electronic data capture system. Therefore, patients were able to complete assessments independently (prior work was conducted using a phone interview). REDCap has a comprehensive and publicly accessible library of PRO measures. COMPASS-CP uses REDCap’s extensive APIs, which enable bidirectional data exchange in a secure and scalable manner, to incorporate PRO data and other patient-completed data for CP generation.

We created algorithms in COMPASS-CP CFC informed by clinical guidelines [32] to integrate patient-completed measures with provider-facilitated decisions and immediately generate personalized CPs. We also updated the existing COMPASS-CP resource database to be more specific to cancer care [32]. The minimal viable product of COMPASS-CP CFC generated CPs based on the needs identified by elevated health concerns raised in the patient-completed measures and EHR data elements, which were then matched with relevant resources that could be selected by zip code and preference to support self-management. Patient zip code from the FHIR drives the filtering and recommendation of proper local community resources that are geographically close to the patient.

We individually implemented the minimal viable version of COMPASS-CP CFC and elicited qualitative feedback about the potential efficacy and usability (IDEAS stage 8). We conducted 2 rounds of usability tests and expected to reach saturation with 5 participants per round [33,34]. The first round of testing was with health care professionals (informatics professionals and health care providers) using mock patient scenarios. For the second round, a health care professional met with patients who were colorectal cancer survivors. Both rounds of feedback used a concurrent “think aloud” protocol and retrospective probing questions regarding acceptability [35]. We used the System Usability Scale and aimed to achieve an average score ≥70 (considered “good”) [34,36,37]. We further assessed quantitative measures of acceptability, appropriateness, and feasibility (Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure, respectively) [38].

We also incorporated qualitative assessments in a brief semistructured interview and questionnaire after the think-aloud testing in both groups [39]. With the relevant components of the CFIR Qualitative Interview guide, we systematically considered select intervention characteristics (design quality and packaging) and individual characteristics (knowledge and beliefs about the intervention) to inform the future content, skills training, and support needed by those who will be using the application.

Detailed field notes and audio recordings were reviewed to summarize qualitative data regarding participant recommendations that were iteratively incorporated into the application adaptation. For the CFIR analyses to inform future implementation, we reviewed all deidentified transcripts by listening to the audio while reading the transcripts to ensure accuracy. Transcripts were imported into ATLAS.ti (version 7.5; ATLAS.ti Scientific Software Development GmbH) to manage the data. Two research team members independently coded each transcript using the CFIR codebook. After the data were coded, the data within each category were abstracted and synthesized into themes. Themes were determined by their prevalence and salience in the data, according to the principles of thematic analysis [40]. Participant characteristics and usability data were summarized using descriptive statistics.

The characteristics of health care professionals who provided feedback are described in Table 1, and the characteristics of patients and caregivers are described in Table 2. The first focus group included 3 health care professionals, and this was followed up by 7 individual interviews to represent feedback from 2 nurses, 2 nurse practitioners, 1 physician, 1 physician assistant, and 1 allied health professional who was in an administrative role. The second focus group included 5 patients and 2 caregivers.

The demographic characteristics of health care professionals (informatics professionals and health care providers; n=5) and patients (n=5) who engaged in usability testing for COMPASS-CP CFC are described in Tables 1 and 2. The average System Usability Scale score was above our target score of 70 for both health care professionals and patients (Table 3), indicating an overall high ease of use. Yet, the range of scores reported by health care professionals included values lower than the target score, indicating the additional need for improved usability. Participants also quantitatively reported high levels of acceptability, appropriateness, and feasibility for COMPASS-CP CFC (Table 3). The following qualitative feedback guided by the CFIR elaborates on intervention and individual characteristics to consider for improving usability and for other aspects of future implementation planning.

The adaptation of COMPASS-CP CFC was rated as usable by health care professionals and patients who were colorectal cancer survivors. COMPASS-CP provides flexibility to compile assessments from different locations (eg, PROs and EHR data) and provides tools to help with interpretation of the data collected. COMPASS-CP also links identified concerns with personalized community resources. Incorporating validated PROMIS measures is an important strength of COMPASS-CP CFC since this measurement system has many advantages, including its ability to function as both a screening tool and a tool for assessing clinical outcomes [51-53]. We also enhanced the original COMPASS-CP tool by successfully integrating REDCap as the patient-facing interface, and this is particularly important because REDCap streamlines our ability to send surveys to patients, which they can complete independently (as opposed to a facilitated telephone interview), and allows investigators to easily make modifications. It is also possible for patients to complete the assessments on paper and for responses to be entered by clinical staff.

The COMPASS-CP CFC tool facilitates integrating patient-completed assessments in clinical care, which is relevant to improving patient engagement and patient-provider communication [54,55]. Patient feedback in this study and other studies indicated that providers are expected to discuss the concerns raised in measures completed by patients and that this would influence the selection of the personalized recommended actions in the CP. COMPASS-CP CFC has the potential to reduce the time it takes providers to comprehensively assess and prioritize patient concerns during a clinic visit, thus facilitating more time for coaching to empower patients’ self-management. Supporting patients to increasingly engage in self-management and connecting them with relevant resources are important components of proposed systematic changes that involve risk stratifying the delivery of follow-up cancer care [56,57]. COMPASS-CP CFC could facilitate access and referral to recommended supportive interventions that are often not integrated into standard care.

Health care professionals indicated that the ability of COMPASS-CP CFC to link patients’ concerns with resources near their place of residence was particularly valuable since it was an unmet need that could potentially save them time. Yet, health care professionals were also concerned about fulfilling the responsibility to provide adequate and timely resources for the concerns identified (eg, a patient who scores high on concerns such as depression). Thus, we considered that it was particularly important for each patient-completed measure included in COMPASS-CP CFC to be linked to an actionable resource. The next steps may include integrating COMPASS-CP CFC with an existing national database of community resources that is increasingly used for addressing social determinants of health to support the feasibility of maintaining current and up-to-date resources [58]. This integration would further enhance scalability.

Incorporating stakeholder feedback in this adaptation process is a critical step to ensure that COMPASS-CP CFC will be successfully implemented in future work [59]. Data collected using the CFIR will particularly be applied to inform the integration of COMPASS-CP CFC into clinical care. As currently designed, for the use of COMPASS-CP CFC, it will be important to include a shared decision-making discussion with a health care professional when selecting and providing resources to ensure that patients’ concerns are normalized and appropriately prioritized [60]. Future research will explore the appropriate professional for this role. As suggested by a stakeholder, this professional may be a health coach or social worker who is integrated with clinical care. The appropriate professional may also vary based on the local clinical context. The next steps for this line of work include pilot testing COMPASS-CP CFC when integrated within clinical care to assess further usability and effectiveness. It will be important to assess the appropriate timing to implement COMPASS-CP CFC in the follow-up care process. Moreover, it will be important to consider how to follow-up with patients and encourage ongoing adherence to CPs.

A limitation of this study includes the range of usability scores reported by health care professionals, with some scores being below the target score. This indicates that further work is needed to improve usability. Future studies may incorporate additional usability feedback from health care professionals to further optimize COMPASS-CP CFC. In addition, COMPASS-CP CFC has not yet been tested in the flow of clinical care, which will be an imperative next step for tool refinement. Prior research on electronic care planning support interventions suggests that adopting COMPASS-CP CFC at multiple time points as part of a dynamic care planning process will result in the largest impact [15]. Furthermore, although this study was focused on follow-up care specific to colorectal cancer, we designed COMPASS-CP CFC to be easily adapted to other cancer types (by modifying the colorectal cancer–specific items). We chose to begin with colorectal cancer because these patients commonly have a good prognosis with a high need for follow-up care and this cancer type is prevalent in both men and women. Therefore, the next steps include broadening stakeholder involvement and considering patients diagnosed with other types of cancers.

We gathered stakeholder feedback to inform our iterative adaptation of COMPASS-CP CFC to facilitate patient engagement in their follow-up cancer care. This electronic tool is integrated with the EHR and provides flexibility to comprehensively compile assessments from different locations (including PROs), add assistance with interpretation of the data collected, and link concerns identified with personalized resources. This study supports the initial usability of COMPASS-CP CFC and will inform the next steps for implementation in clinical care. COMPASS-CP has the potential to be a widely applicable and scalable tool for personalizing and streamlining comprehensive follow-up cancer care. The implementation of this tool in coordination with clinical care may ultimately facilitate sustainable improvements in patient health outcomes.