This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2491) Emerging Technologies for Rehabilitation (270) Portable and Mobile Technologies for Rehabilitation (156) Usability of Apps and User Perceptions of mHealth (993) mHealth in a Clinical Setting (624) Web-based and Mobile Health Interventions (3357)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/57579, first published .
Feasibility of a Web-Based and Mobile-Supported Follow-Up Treatment Pathway for Adult Patients With Orthopedic Trauma in the Netherlands: Concurrent Mixed Methods Study

Feasibility of a Web-Based and Mobile-Supported Follow-Up Treatment Pathway for Adult Patients With Orthopedic Trauma in the Netherlands: Concurrent Mixed Methods Study

Feasibility of a Web-Based and Mobile-Supported Follow-Up Treatment Pathway for Adult Patients With Orthopedic Trauma in the Netherlands: Concurrent Mixed Methods Study

Authors of this article:

Gijs J A Willinge1 Author Orcid Image ;   Jelle F Spierings2 Author Orcid Image ;   Kim A G J Romijnders3 Author Orcid Image ;   Elke G E Mathijssen4 Author Orcid Image ;   Bas A Twigt1 Author Orcid Image ;   J Carel Goslings1 Author Orcid Image ;   Ruben N van Veen1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    1Department of Trauma Surgery, OLVG, , Jan tooropstraat 164, Amsterdam, , the Netherlands

    2Department of Trauma Surgery, St. Antonius Ziekenhuis, , Utrecht, , the Netherlands

    3Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, , Utrecht, , the Netherlands

    4The Healthcare Innovation Center, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, , Utrecht, , the Netherlands

    Corresponding Author:

    Gijs J A Willinge, MD


    Background: Orthopedic trauma care encounters challenges in follow-up treatment due to limited patient information provision, treatment variation, and the chaotic settings in which it is provided. Additionally, pressure on health care resources is rising worldwide. In response, digital follow-up treatment pathways were implemented for patients with orthopedic trauma, aiming to optimize health care resource use and enhance patient experiences.

    Objective: We aim to assess digital follow-up treatment pathway feasibility from the patient’s perspective and its impact on health care resource use.

    Methods: A concurrent mixed methods study was conducted parallel to implementation of digital follow-up treatment pathways in an urban level-2 trauma center. Inclusion criteria were (1) minimum age of 18 years, (2) an active web-based patient portal account, (3) ability to read and write in Dutch, and (4) no cognitive or preexisting motor impairment. Data were collected via electronic patient records, and surveys at three time points: day 1‐3, 4‐6 weeks, and 10‐12 weeks after an initial emergency department visit. Semistructured interviews were performed at 10‐12 weeks post injury. Anonymous data from a pre-existing database were used to compare health care resource use between the digital treatment pathways and traditional treatment. Quantitative data were reported descriptively. A thematic analysis was used for qualitative data. All outcomes were categorized according to the Bowen feasibility parameters: acceptability, demand, implementation, integration, and limited efficacy.

    Results: Sixty-six patients were included for quantitative data collection. Survey response rates were 100% (66/66) at day 1‐3, 92% (61/66) at 4‐6 weeks, and 79% (52/66) at 10‐12 weeks. For qualitative data collection, 15 semistructured interviews were performed. Patients reported median satisfaction scores of 7 (IQR 6‐8) with digital treatment pathways and 8 (IQR 7‐9) for overall treatment, reflecting positive experiences regarding functionality, actual and intended use, and treatment safety. Digital treatment pathways reduced secondary health care use, with fewer follow-up appointments by phone (median 0, IQR 0‐0) versus the control group (median 1, IQR 0‐1; P<.001). Consequently, fewer physicians were involved in follow-up treatment for the intervention group (median 2, IQR 1‐2) than for the control group (median 2, IQR 1‐3; P<.001). Fewer radiographs were performed for the intervention group (median 1, IQR 0-1) than for the control group (P=.01). Qualitative data highlighted positive experiences with functionalities, intended use, and safety, but also identified areas for improvement, including managing patient expectations, platform usability, and protocol adherence.

    Conclusions: Use of digital follow-up treatment pathways is feasible, yielding satisfactory patient experiences and reducing health care resource use. Recommendations for improvement include early stakeholder involvement, integration of specialized digital tools within electronic health record systems, and hands-on training for health care professionals. These insights can guide clinicians and policy makers in effectively integrating similar tools into clinical practice.

    JMIR Form Res 2024;8:e57579

    doi:10.2196/57579

    Keywords

    musculoskeletal extremity injury; patient portal; follow-up treatment; healthcare utilization; patient experience; feasibility; orthopedics; trauma; Netherlands; mixed methods; resource utilization; electronic patient records; thematic analysis; qualitative data; digital treatment; mobile phone 


    Orthopedic trauma care faces significant challenges when it comes to adequate follow-up treatment and optimization of patient journeys [1,2]. These challenges include the often hectic situation surrounding patients’ initial visit in the emergency department (ED), the restricted provision and recall of medical information by patients, and variety in follow-up treatment pathways [3,4]. These challenges may impede patients’ understanding of their injuries and supplementary treatments, leading to inaccurate expectations and potentially hindering the recovery process and perceived engagement during follow-up treatment [3,5].

    To address these challenges, the use of digital tools to support and empower patients during their patient journeys has significantly increased in recent years [2,6-10]. In the context of orthopedics, this has led to improvements of patient information provision, patient engagement, and overall patient journeys [2,6,11,12]. However, these tools have mainly been applied in chronic orthopedic care [9,13]. Their application in acute orthopedic trauma care remains limited. Moreover, existing digital tools lack interactive features that allow patients to actively shape their treatment, which could enhance care efficiency [14]. Therefore, we developed digital follow-up treatment pathways for patients with orthopedic trauma, comprising timed patient information, injury-specific patient-reported outcome measures (PROMs), and a specific questionnaire through which patients could directly influence scheduling of follow-up appointments. These pathways aimed to assist patients during their recovery, collect real time patient data, and optimize health care resource use by tailoring treatment to patients’ needs.

    As this is a novel approach to trauma follow-up care in the Netherlands, evaluation of feasibility is crucial for sustainable development and implementation, especially considering the patient’s perspective as an end user [15]. Therefore, this study aimed to assess digital treatment pathways feasibility from a patient’s perspective and its effect on health care resource use. We hypothesized that the digital treatment pathways would be considered feasible by patients, while simultaneously reducing secondary health care use.


    Design and Setting

    A monocentric, explorative, prospective concurrent mixed method study was performed to assess the feasibility of newly implemented digital treatment pathways for patients with orthopedic trauma in an urban, level-2 trauma center and teaching hospital. A concurrent mixed methods triangulation design was used, using quantitative and qualitative research methods simultaneously with equal weight [16]. One team (GJAW and JFS) collected and analyzed quantitative data, and another team (KAGJR and EGEM) separately collected and analyzed qualitative data. Using the Bowen feasibility framework, both data types were categorized within the following themes: acceptability, demand, implementation, integration, and limited efficacy [17].

    Our study was conducted among adult patients who received follow-up treatment through a digital treatment pathway at our institution between October 1, 2022, and April 1, 2023. Patients were eligible for a digital treatment pathway if they (1) were aged 18 years or older, (2) had an active account for the web-based patient portal, (3) were able to read and write in Dutch, and (4) had no cognitive impairment or preexisting motor impairment. Exclusion criteria were: multiple injuries, pathological fracture, and initial presentation or follow-up treatment at another institution. Additionally, a control group was formed using previously collected anonymous data from 100 consecutive patients to compare secondary health care use–related pre- and postimplementation outcomes of the digital treatment pathways. These data were collected between October 1, 2021, and February 10, 2022, as part of a clinical audit.

    This study was reported according to the GRAMMS (Good Reporting on a Mixed Methods Study) criteria (Multimedia Appendix 1) and COREQ (Consolidated Criteria for Reporting Qualitative Studies; Multimedia Appendix 2) [18,19].

    Ethical Considerations

    This study was ethically reviewed and approved by the Medical Ethical Committee of Utrecht, the Netherlands (WO 22.049). All included participants provided informed consent before study participation and all participants had the ability to opt out at any given time during the study. Prospectively collected data were deidentified and all retrospective data were used anonymously. Furthermore, all study data were stored on a secure institutional drive, only accessible by 2 researchers (GJAW and KAGJR). Study participants did not receive any compensation for their participation in this study.

    Treatment Specifics

    Patients with orthopedic trauma at our institution are treated according to a virtual fracture care review protocol [20,21]. To optimize this protocol, additional digital treatment pathways were introduced for the 8 most frequent types of hand, wrist, ankle, and foot fractures on October 1, 2022, including both nonoperative and operative treatment (Table 1). Multimedia Appendix 3 offers a detailed description of the virtual fracture care review workflow and the digital treatment pathway.

    The mode of delivery of the digital pathways was the web-based patient portal and its accessory app: “MyChart,” an extension of the electronic patient record (EPR) system: Epic (Epic Systems Corporation). An impression of its layout is provided in Figure 1.

    In selected pathways, patients could directly influence their follow-up treatment through an additional anchor questionnaire sent prior to the final routine function check appointment. Based on the response, the need for an appointment was evaluated and planned accordingly, as shown in Figure S1 in Multimedia Appendix 3. Only patients who indicated stagnation or deterioration of recovery or dissatisfaction with their treatment received a follow-up appointment. Function check appointments that included imaging could not be guided by an anchor questionnaire, as this could not account for imaging results in its current form. An example of the course of a digital treatment pathway is provided in Figure 2.

    Table 1. Injuries and treatment methods included in digital treatment pathways and used questionnaires.
    InjuryTreatmentPROMsaTiming (weeks)Anchor questionnaire
    Triquetrum fractureNonoperativePRWEb6-12-24Yes
    Scaphoid fractureNonoperativePRWE6-12-24Noc
    Distal radius fractureNonoperativeQDASHd6-12-24Yes
    Distal radius fractureOperativeQDASH7-13-24Yes
    Unimalleolar ankle fractureNonoperativeAOFASe6-12-24Noc
    Unimalleolar ankle fractureOperativeAOFAS8-14-24Yes
    Bi or trimalleolar ankle fractureOperativeAOFAS8-14-24Yes
    Metatarsal fractureNonoperativeAOFAS6-12-24Noc

    aPROM: Patient-Reported Outcome Measure.

    bPRWE: Patient Related Wrist Evaluation.

    cThese pathways did not include an anchor questionnaire-based function check appointment, as these function check appointments also included imaging and could therefore not be replaced in the digital treatment pathway.

    dQDASH: Quick Disabilities of the Arm, Shoulder, or Hand.

    eAOFAS: American Orthopedic Foot and Ankle Score.

    ‎
    Figure 1. An impression of the layout of the menu within the patient portal and the accessory app. QUICKDASH: Quick Disabilities of the Arm, Shoulder, or Hand; PROM: Patient-Reported Outcome Measure; VFC: virtual fracture care.
    ‎
    Figure 2. An example of a digital treatment pathway for an operatively treated distal radius fracture. PROM: Patient-Reported Outcome Measure.

    Sampling and Recruitment

    The sample size for quantitative data collection was set at 70, based on general recommendations for the design of feasibility studies [17,22]. For the quantitative part, patients were recruited via convenience sampling. A researcher informed them about our study the day after their ED visit via an information email and by phone, and sent an information letter and informed consent form by email to patients indicating willingness to participate (Multimedia Appendix 4). Study participants provided digital informed consent before participation. Upon providing consent, patients also indicated willingness to partake in an interview. For the qualitative part of this study, patients were included using a purposive maximum variation sampling method to ensure heterogeneity in terms of gender, age, type of injury, and treatment strategy. The qualitative sample size was guided by the principle of data saturation and determined a posteriori [23]. The expected number of interviews was 15 [23].

    Data Collection

    Overview

    Data were collected from system data (EPR), surveys, and semistructured patient interviews.

    System Data

    Quantitative system data were collected 6 months after treatment initiation, including age, sex, smoking status, fracture type and treatment, PROMs and anchor questionnaire results, protocol compliance, follow-up appointments (face-to-face or by phone, with each contact counting as one appointment), health care professionals involved in follow-up treatment (physician or casting technician), follow-up imaging (radiograph, computed tomography scan, and magnetic resonance imaging scan), and ED reattendances and registered complications. The control group data were similar.

    Surveys

    Three web-based surveys were used to prospectively gather quantitative data, comprising a total of 38 questions. Surveys included 4-point Likert Scales (1=totally disagree to 4=totally agree), Visual Analogue Scales (0=extremely dissatisfied to 10=extremely satisfied), and free-text questions. The surveys were developed by 3 researchers (JFS, GJAW, and EGEM) and checked by 3 experts: a (orthopedic) trauma professor (JCG), a (orthopedic) trauma surgeon (RNvV), and an associate professor in process evaluations of health care innovations (JCA Trappenburg, PhD). Patients received surveys at three time points; day 1‐3 (T0), 4‐6 weeks (T1), and 10‐12 weeks (T2) after the ED visit (Multimedia Appendix 5). Patients who did not complete questionnaires received up to two reminder emails, sent after 3 and 5 days. All respondents per time point were included in the final analysis.

    Semistructured Interviews

    Qualitative data were collected through semistructured interviews using Microsoft Teams. The research team developed an interview topic list guide in collaboration with a variety of health care professionals (ED caregivers, casting technician, orthopedic trauma surgeon, and surgical resident) to explore patients’ perspectives (Multimedia Appendix 6). Interviews lasted approximately 45 minutes and were conducted by KAGJR an academic social scientist or EGEM an academic nursing scientist, 3 months after injury (T2). Open-ended questions were used to stimulate participants’ own interpretation and prompts were used to encourage deliberation. Interviews were conducted in Dutch and were audio recorded.

    Data Analysis

    Quantitative Analysis

    Quantitative data were analyzed using IBM SPSS (version 27; IBM Corp). Data were reported using frequencies with percentages for categorical variables, and continuous variables as mean with SD or median with IQR, depending on data distribution. Statistical significance was assessed using a chi-square test for categorical variables and either a paired t test (2-tailed) or Mann-Whitney U test for continuous variables, based on data distribution.

    Qualitative Analysis

    After 5 interviews, posteriori data saturation was assessed. This was reached after completing 14 interviews. The 15th interview planned was still conducted. Audio recordings were transcribed verbatim. Data were systematically analyzed by KAGJR and EGEM following the steps of thematic analysis to reflect data transformation [24,25]. NVivo (version 20; Lumivero) was used for data analysis. The main analysis involved 6 phases to identify recurring patterns and perspectives. Relevant themes and subthemes were deduced from the first 5 interviews (step 1), forming the basis for a coding list, which was used to code an entire transcript independently (step 2). This was then used to check for additional relevant codes during a joint meeting (step 3). Differences in coding were resolved to establish a final coding taxonomy (step 4), which was used to code all transcripts (step 5). Dutch quotes were translated to English using the forward backward method, and were used to illustrate themes (step 6).

    Triangulation

    After separate analyses, quantitative and qualitative data were triangulated through a Pillar Integration Process to highlight distinctive themes within data and identify similarities and differences between data sources [26]. The triangulation session was attended by both research teams and two (orthopedic) trauma surgeons (RNvV and BAT). One researcher (GJAW) presented quantitative results per study parameter, while another (KAGJR) presented qualitative findings.


    Demographics

    In total, 251 patients were eligible for a digital treatment pathway (Figure 3).

    Of these patients, 125/251 (49.8%) were eligible for study inclusion, of whom 66/125 (52.8%) were included. Baseline characteristics are shown in Table 2.

    Similar characteristics were compared between the control and digital treatment pathway groups, and no differences were identified. All included patients (66/66, 100%) completed the T0 study survey, 61/66 (92%) completed the T1 survey and 52/66 (79%) completed the T2 survey. The respondent groups at various time points did not differ in baseline characteristics (Table 2).

    Thirty of 66 (45%) patients consented to interview participation of whom 15/66 (23%) were interviewed.

    The reasons for not participating were not obtained. Nine out of 15 (60%) interviewees were female and 6/15 (40%) were male. Interviewee age ranged between 23 and 77 (median 48, IQR 35-62) years.

    ‎
    Figure 3. Flowchart of patient inclusion.
    Table 2. Baseline characteristics of adult patients with orthopedic trauma included in a mixed method evaluation of digital treatment pathways for follow-up treatment of a musculoskeletal extremity injury.
    VariablesT0aa (n=66)T1b (n=61)T2c (n=52)P value
    T0-T1T0-T2T1-T2
    Sex, n (%).93.83.89
    Male26 (39)23 (38)19 (37)
    Female40 (61)38 (62)33 (63)
    Age (years), median (range)53 (20-77)53 (20-77)57 (22-77).80.09.15
    Smoking, n (%).98.99.99
    No50 (75)46 (75)39 (75)
    Former0 (0)0 (0)0 (0)
    Current3 (5)2 (3)2 (4)
    Unknown13 (20)13 (21)11 (21)
    Type of fracture and treatment, n (%)>.99>.99>.99
    Triquetrum conservative5 (7)5 (8)4 (8)
    Scaphoid conservative0 (0)0 (0)0 (0)
    Distal radius conservative23 (35)23 (38)21 (39)
    Distal radius operative18 (27)16 (26)14 (27)
    Metatarsal shaft conservative3 (5)2 (3)2 (4)
    Unimalleolar ankle conservative5 (7)4 (7)2 (4)
    Unimalleolar ankle operative10 (15)9 (15)7 (14)
    Bi or trimalleolar ankle operative2 (3)2 (3)2 (4)
    Treatment strategy, n (%).82.80.97
    Nonoperative36 (55)35 (57)30 (58)
    Operative30 (45)26 (43)22 (42)

    aT0: 1-3 days after emergency department visit.

    bT1: 4-6 weeks after emergency department visit.

    cT2: 10-12 weeks after emergency department visit.

    Acceptability

    Overview

    Acceptability was assessed through quantitative and qualitative evaluation of patients’ expectations of digital treatment pathways and its anticipated functionalities and advantages. Positive and negative patient experiences, along with influencing factors, were identified. Finally, patient satisfaction was evaluated.

    Expectations

    Participants predominantly expected to be able to successfully use the digital treatment pathway (Figure 4). Additionally, 56/66 (85%) respondents expected that the digital treatment pathway would assist them throughout their treatment and 46/66 (70%) expected this would positively impact communication with their treatment team.

    These expectations were further specified in qualitative data, including easy scheduling of follow-up appointments, insight into patient records (ie, summaries of visits and images results), and information on their fracture and recovery process. Some interviewees emphasized their initial expectations were not met, partly due to insufficient expectation management (Multimedia Appendix 7, quote 1). Additionally, participants acknowledged having lower expectations for this application compared to commercial apps, recognizing the unique context of a health care provider application.

    ‎
    Figure 4. Likert scales acceptability: expectations and experiences. DTP: digital treatment pathway; T0: 1-3 days after emergency department visit; T1: 4-6 weeks after emergency department visit; T2: 10-12 weeks after emergency department visit.
    Experiences

    Survey respondents predominantly reported the digital treatment pathway had helped them during their treatment, with 36/61 (60%) at T1 and 30/52 (58%) at T2 (Figure 4). This was significantly lower compared to T0 with 56/66 (85%; P<.001). A total of 21/61 (35%) patients indicated their expectations were not met at T1, and 13/52 (25%) at T2. Respondents predominantly perceived the digital treatment pathways as safe, with 51/61 at T1 (83%) and 39/52 at T2 (75%).

    These results were supported in qualitative data, as interviewees shared the functionalities of the app supported their follow-up treatment. The possibility to schedule and track appointments was particularly experienced as positive (Multimedia Appendix 7, quote 2). Negative experiences were linked to limited user-friendliness, difficult login and registration processes, and difficulty with finding desired information. Despite unmet expectations, overall experiences with received care were mainly positive (Multimedia Appendix 7, quote 3).

    Satisfaction

    Quantitative data showed a median satisfaction score with digital treatment pathways of 7 (IQR 6‐8) at T1, increasing to 7.5 (IQR 7‐9) at T2. A similar score was observed for satisfaction with overall treatment team communication, with a median of 7 (IQR 6‐8) at both T1 and T2. Additionally, overall treatment scores at T2 were higher than scores reported specifically for the digital treatment pathway, with a median score of 8 (IQR 7‐9) for overall treatment satisfaction and a median score of 8 (IQR 7‐8.5) for overall information provision.

    During interviews, participants were generally satisfied with the digital treatment pathways. This was mainly based on the appointment scheduling and reminder or notification feature. One of the most frequent reasons provided for a relatively low score was the unclear layout and menu of the application, complementing quantitative data. Participants clearly indicated this would need to be improved to increase their satisfaction level (Multimedia Appendix 7, quote 4).

    Demand

    Overview

    Demand was both quantitatively and qualitatively assessed through patients’ perceived capability to use digital treatment pathways effectively and factors facilitating and hindering adequate use. Additionally, patients’ intended use of digital treatment pathways in the future was evaluated, along with their likelihood to recommend this to others.

    Actual Use

    During follow-up treatment, participants predominantly felt able to use the digital treatment pathway with provided instructions, with 52/61 (85%) positively reporting on this at T1 and 42/52 (81%) at T2 (Figure 5).

    The majority of patients found received notifications sufficient throughout follow-up, with 39/61 (64%) at T1 and 32/52 (61%) at T2. Respondents who considered notifications insufficient expressed a need for more information and supportive notifications in the later stages of follow-up (eg information and exercise instructions after the first month of recovery).

    During the interviews, participants shared they were able to use the MyChart app well, as its features were familiar to smartphone users. However, several barriers were also shared; the main barrier being the confusing layout of the application. This hindered patients in finding the right information or functions (Multimedia Appendix 7, quote 5).

    ‎
    Figure 5. Likert scales demand: actual use. DTP: digital treatment pathway T1: 4-6 weeks after emergency department visit; T2: 10-12 weeks after emergency department visit.
    Intended Future Use

    Respondents generally expressed a high likelihood of using digital treatment pathways in the future, with a median score of 9 (IQR 7-10) at T1 and a median score of 8 (IQR 7-10) at T2. Regarding the likelihood of recommending the digital treatment pathways to others, patients reported a median score of 8 (IQR 6-9) at T1 and 8 (IQR 6-9) at T2.

    Interviewees reported similarly positive outlooks. They were convinced that the future of health care would become more digital and regarded digital treatment pathways as a great starting point. From a practical view, interviewees shared they would use it again, if only to keep track of appointments (Multimedia Appendix 7, quote 6).

    Implementation

    Overview

    The success of the implementation was assessed by health care professionals’ compliance with the new protocol and patients’ completion of questionnaires in the digital treatment pathway. Factors perceived by patients as facilitating or hindering implementation were also identified, with results from both data sources included.

    Protocol Compliance and Technical Issues

    During inclusion, 117/251 (46.6%) eligible patients did not receive a digital treatment pathway due to lack of initiation by health care professionals. No technical issues were encountered.

    PROM Completion

    The completion rate of PROM questionnaires was substantial, although it exhibited a decrease as the follow-up duration extended (Table 3).

    Table 3. Questionnaire completion among adult patients with orthopedic trauma and protocol compliance by health care professionals for follow-up treatment of a musculoskeletal extremity injury with a digital treatment pathway.
    OutcomesDigital treatment pathway (n=66), n (%)
    Completion of PROMsa
    6‐8 wk completed60 (91)
    12 wk completed56 (85)
    24 wk completed44 (67)
    Anchor questionnaires sent52 (79)
    Completed48 (92)
    Missed4 (8)
    Anchor questionnaire results
    No appointment40 (77)
    Phone appointmentb11 (21)
    Face-to-face appointment1 (2)
    Anchor questionnaire compliance by HPc
    Compliant33 (63)
    Noncompliant19 (37)
    Type of protocol deviation
    Extra appointment10 (19)
    Extra appointment and radiograph9 (17)
    Reason for protocol deviation
    Unknownd14 (27)
    Patient’s request1 (2)
    Clinical indication4 (8)

    aPROM: Patient Reported Outcome Measure.

    bFour phone appointments due to incomplete or missed anchor questionnaire.

    cHP: health care professional.

    dThis was attributed to unawareness or noncompliance to protocol.

    Anchor Questionnaire Compliance and Results

    Of the 66 patients, 52 (79%) received an anchor questionnaire (Table 3). Eleven out of 66 (17%) patients were ineligible due to the type of injury and 3/66 (4%) were not willing to participate in the anchor questionnaire. These patients were planned for a routine function check follow-up appointment by phone. The 4/66 (8%) patients who did not complete the questionnaire were also planned for function check follow-up by phone. Following the questionnaire results, 40/52 (77%) of patients did not need a follow-up appointment. However, health care professionals deviated from the protocol in 19/52 (37%) of patients and planned a follow-up appointment despite the questionnaire results. For 14/19 (74%) of protocol deviations, no clear indication was documented. These were thus attributed to protocol unawareness by health care professionals.

    Barriers and Facilitators to Implementation

    Barriers shared by interviewees included limited user-friendliness of the patient portal and the MyChart app, the required digital skills, and the linguistic orientation of the platform. Facilitators included patients’ positive attitudes toward digital supportive tools and the flexibility for patients to answer questions, influence their treatment, and access information at their convenience (Multimedia Appendix 7, quote 7). Furthermore, understanding the efficiency it offered health care providers increased interviewees’ willingness to use the digital treatment pathways.

    Integration

    This was assessed though qualitative data provided by interviewees focusing on the ease of integration of digital treatment pathways into patients’ daily activities. Interviewees generally found the MyChart app easy to incorporate into their lives, noting that its mode of delivery was familiar to smartphone users. However, they pointed out that successful integration depended on users’ digital proficiency. Furthermore, participants mentioned that the app’s language settings and layout made it time-consuming to navigate to the desired section, which could be challenging during daily activities such as work, hindering a seamless integration (Multimedia Appendix 7, quote 8).

    Limited Efficacy

    Limited efficacy was assessed by comparing secondary health care resource use before and post implementation of the digital treatment pathways. This was complemented by patients’ perspectives on the necessity of function check appointments and the new scheduling process through the anchor questionnaire.

    Quantitative data showed a reduction of follow-up appointments following the implementation of the digital treatment pathways (P<.001; Table 4).

    Table 4. Secondary health care use and complications for adult patients with orthopedic trauma treated for a musculoskeletal extremity injury before and post implementation of a digital follow-up treatment pathway.
    VariablesControl (n=100)DTPa (n=66)P value
    n (%)Median (IQR)n (%)Median (IQR)
    Follow-up appointments429 (100)4 (2-5)192 (100)3 (2-4)<.001
    Phone appointments128 (29.8)1 (0‐1)39 (20.3)0 (0‐0)<.001
    Face-to-face appointments301 (70.2)2 (1-4)153 (79.7)2 (1-4).07
    Involved caregivers429 (100)4 (2-5)192 (100)3 (2-4)<.001
    Physician274 (63.9)2 (1-3)111 (57.8)2 (1-2)<.001
    Plaster technician149 (34.7)1 (0‐2)79 (41)1 (0‐1).21
    Physician assistant6 (1)0 (0‐0)2 (1)0 (0‐0).37
    Imaging
    Radiograph128 (97)1 (0‐1)56 (93)1 (0‐1).01
    CTb scan4 (3.0)0 (0‐0)3 (5)0 (0‐0).87
    MRIc scan0 (0)0 (0‐0)1 (2)0 (0‐0).22
    EDd reattendances5 (5)—e1 (1)—.24
    Complications
    Persisting pain or stiffness reportedf19 (19)—8 (12)—.24
    Infection0 (0)—1 (2)—.76

    aDTP: digital treatment pathway.

    bCT: computed tomography.

    cMRI: magnetic resonance imaging.

    dED: emergency department.

    eNot applicable.

    fBased on reported persisting pain or stiffness at the last follow-up appointment within the 6 month follow-up period.

    Primarily, this was the result of a reduction of appointments by phone. Control group patients had a median of 1 (IQR 0‐1) follow-up appointment by phone versus 0 (IQR 0‐0) in patients treated with a digital treatment pathway (P<.001). The number of face-to-face appointments remained similar with a median of 2 (IQR 1‐4) in the control group versus 2 (IQR 1‐4) in the digital treatment pathway group (P=.07). Fewer health care professionals, particularly physicians (P<.001), were involved in the follow-up of the intervention group compared to the control group. Patients in the digital treatment pathways had fewer follow-up radiographs (P=.01), while the number of computed tomography and magnetic resonance imaging scans remained unchanged. ED reattendances were rare in both groups, mostly due to cast-brace issues. No differences in complaints or complications were observed within 6 months. One patient in the intervention group developed a wound infection, treated with surgical debridement and antibiotics, leading to full recovery. This patient independently sought care, unaffected by the anchor questionnaire.

    Qualitative data revealed varying outcomes regarding the need for follow-up appointments among participants. Some individuals found follow-up appointments unnecessary, while others deemed them essential (Multimedia Appendix 7, quote 9). Another participant shared an experience around persistent pain and mobility issues that were not picked up by the questionnaire (Multimedia Appendix 7, quote 10). This participant scheduled a follow-up appointment on their own initiative after receiving questionnaire feedback, emphasizing the importance of this option.


    Principal Findings

    Our findings show that using a digital treatment pathway for patients with orthopedic trauma with extremity fractures is feasible, reflected in high usage, satisfaction, positive experiences, and reduced health care use. Participants saw it as a promising step toward digitizing trauma care, expressing willingness for future use. Nevertheless, clear areas for improvement were identified, including patient expectation management, user-friendliness of the patient portal and accompanying application, the implementation process, and protocol compliance by health care professionals.

    Patient Expectations and Experiences

    The positive expectations and readiness among patients toward the digital treatment pathways, as reflected by the high questionnaire completion rates, aligns with other studies on the use of digital tools and home-monitoring within orthopedic patients [27-29]. The decline in completion rates during later stages of follow-up is a known challenge within this population, suggesting this it is not unique to the digital treatment pathways but rather associated with other characteristics of the orthopedic trauma population [30].

    Although results were predominantly positive, approximately one-third of survey respondents expressed unmet expectations, a sentiment reinforced by interview findings. This was primarily due to a lack of user-friendliness, a common barrier in digital health care innovation [31]. In our study, this was attributed specifically to the web-based patient portal and the accessory MyChart app, both an extension of our EPR system. While integration with the EPR enables direct digital feedback, these systems were not designed for home monitoring, introducing usability issues [8,15,32-35]. Furthermore, these systems are generally used hospital-wide, requiring a wide array of different features and functions for different patient types. This could confuse patients, as confirmed by our participants. Specialized digital tools and pathways designed from an end user perspective could enhance usability and improve this mode of care [8,9]. Such digital tools have already demonstrated positive usability outcomes; however, these are generally costly and lack necessary EPR integration [36-38]. Applications and EPRs capable of independent integration would offer an ideal solution, leveraging the strengths of both approaches. However, this requires substantial financial resources and close collaboration among the commercial sector, hospitals, health insurance companies, and government.

    Protocol Compliance

    A key challenge we encountered was the poor protocol compliance by health care professionals following implementation. During this study inclusion phase, 117/251 (46.6%) of eligible patients were excluded due to improper initiation of the digital pathway, and protocol deviations due to unawareness were high with 14/52 (27%). These rates were unexpectedly high compared to similar studies, highlighting flaws in our implementation process [36,39,40]. An essential area for improvement in this context is the early involvement of diverse health care professionals, particularly those whose daily tasks are altered [31,35,41,42]. This could facilitate integration into daily workflows and consequently improve protocol compliance. This principle also holds true for patients. Existing research in this field underscores the significance of a user-centered design strategy and early stakeholder engagement in the development and implementation of digital tools in daily practice, transforming patients from testers to designers [15,40,43]. Furthermore, using hands-on training and on-the-job guidance for involved health care professionals, complementary to an instructional training system, could yield better results [34,44,45].

    Secondary Health Care Use

    Digital treatment pathways effectively reduced secondary health care use by replacing routine function check follow-up appointments with a more patient-centered approach. Notably, the reduction in follow-up appointments exceeded the number of anchor questionnaires sent, suggesting other aspects such as improved patient information and the ability to address concerns via the patient portal and MyChart app also played a role. Our results emphasize the importance of providing patients with an option to schedule an appointment, regardless of questionnaire outcomes as a safety net. Allowing shared decision-making is crucial, as standardized automated systems may not adequately address all concerns and needs [46]. The decline in radiographic imaging likely resulted from fewer routine follow-up appointments, which were previously often linked to routine scheduling of imaging. The added value of follow-up radiographs in extremity injuries has been strongly debated [47]. These findings align with the aims of the digital treatment pathways, in which these radiographs are only performed on clinical indication, rather than routinely. Despite reduced resource use, ED reattendances and complications remained similar, indicating treatment safety was not compromised. These findings are in accordance with current literature; however, future studies with larger sample sizes that focus on the functional outcomes for specific injuries are needed to make definitive statements regarding treatment safety [9].

    While the digital pathways improved resource use, their full potential was limited by user-friendliness issues and poor protocol adherence. Addressing these could enhance their benefits. The strain on health care resources and medical personnel warrant reconsideration and innovation of the trauma care system. In this, digitization of the trauma care chain may prove an important tool. However, these new approaches must be carefully designed and (not hastily) implemented, to ensure alignment with the needs of all users.

    Strengths and Limitations

    This study has several strengths. First, a concurrent mixed methods design was used, enabling a comprehensive assessment of the digital treatment pathways. Quantitative data measured specific effects, while qualitative data enabled direct interpretation. The concurrent data collection prevented one type of data informing the other, thus ensuring independence of data sources [48,49]. Second, separate teams conducted quantitative and qualitative data collection and analysis independently. Subsequently, they collaboratively engaged in a triangulation process, enhancing the reliability and validity of the results [26]. Additionally, the use of quotes enhanced transparency in presenting qualitative findings [50]. Lastly, using a validated framework provided a structured examination of various feasibility parameters [51].

    Several limitations must be addressed. The primary issue is this study’s sample, which consisted of patients with relatively high digital literacy and potentially positive attitudes toward digital tools, due to the convenience sampling method used. This introduces selection bias. Additionally, patients could choose to make an appointment regardless of questionnaire results, meaning those who opted for the digital appointment may have been early adopters. Therefore, our results might not reflect the general patient population accurately, limiting their generalizability, especially to those with lower health literacy or digital skills [44,52]. Future research should validate these findings with a more representative trauma population.

    Furthermore, the high protocol noncompliance among health care professionals resulted in a substantial loss of potential study participants. Nonetheless, the final sample size was still deemed sufficient for the purposes of this feasibility study [17,22]. Again, regarding sample representativity, the number of operatively treated patients in our quantitative sample, 30/65 (45%), was relatively high compared to a general monotrauma population. It is unknown whether this influenced our results, this would require a validation study on a larger scale. Importantly, the other baseline characteristics were deemed representative for the monotrauma population at our institution.

    Finally, we could not account for the perspective of health care professionals in this study. Future qualitative evaluation in this stakeholder group would be valuable to gain a complete understanding digital treatment pathway feasibility and potentially to identify other points of improvement.

    Conclusion

    This study shows that patients with orthopedic trauma consider our digital treatment pathways in follow-up care as feasible and satisfactory, with positive attitudes toward current and future use. This patient-centered approach has the potential to empower patients and reduce secondary health care use. Opportunities to further improve feasibility lie in early stakeholder involvement in design and implementation, integrating specialized tools with existing EPR systems, and providing hands-on training for health care professionals. Clinicians and policy makers can use these insights to better integrate similar digital tools into clinical practice and future guidelines.

    Acknowledgments

    The authors wish to thank Dr JCA Trappenburg and Dr W de Lange from The Healthcare Innovation Center (THINC) for the assistance in development of the surveys, and collecting and analyzing of the qualitative data. We would also like to thank GD Duijzings for her role as project lead in developing and implementing the new digital treatment pathways. Furthermore, we would like to thank B Dronkert for his assistance in creating the control group database.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Good Reporting of a Mixed-Methods Study (GRAMMS) checklist.

    DOCX File, 25 KB
    Multimedia Appendix 2

    Consolidated Criteria for Reporting Qualitative Studies (COREQ).

    DOCX File, 36 KB
    Multimedia Appendix 3

    Full detailed description of the VFC review workflow and the digital treatment pathway workflow. VFC: virtual fracture care.

    DOCX File, 180 KB
    Multimedia Appendix 4

    An anonymized version of the patient information letter and consent form.

    DOCX File, 29 KB
    Multimedia Appendix 5

    Study surveys at T0, T1, and T2. T: time point.

    DOCX File, 28 KB
    Multimedia Appendix 6

    Patient interview topic guide.

    DOCX File, 26 KB
    Multimedia Appendix 7

    Quotes per feasibility domain.

    PDF File, 89 KB
    1. Vardon-Bounes F, Gracia R, Abaziou T, et al. A study of patients’ quality of life more than 5 years after trauma: a prospective follow-up. Health Qual Life Outcomes. Jan 8, 2021;19(1):18. [CrossRef] [Medline]
    2. Timmers T, Janssen L, Kool RB, Kremer JA. Educating patients by providing timely information using smartphone and tablet apps: systematic review. J Med Internet Res. Apr 13, 2020;22(4):e17342. [CrossRef] [Medline]
    3. Engel KG, Heisler M, Smith DM, Robinson CH, Forman JH, Ubel PA. Patient comprehension of emergency department care and instructions: are patients aware of when they do not understand? Ann Emerg Med. Apr 2009;53(4):454-461. [CrossRef] [Medline]
    4. Rowe A, Knox M. The impact of the healthcare environment on patient experience in the emergency department: a systematic review to understand the implications for patient-centered design. HERD. Apr 2023;16(2):310-329. [CrossRef] [Medline]
    5. Khaleel I, Wimmer BC, Peterson GM, et al. Health information overload among health consumers: a scoping review. Pat Educ Couns. Jan 2020;103(1):15-32. [CrossRef] [Medline]
    6. De La Cruz Monroy MFI, Mosahebi A. The use of smartphone applications (apps) for enhancing communication with surgical patients: a systematic review of the literature. Surg Innov. Apr 2019;26(2):244-259. [CrossRef] [Medline]
    7. Kamilu Sulaiman S, Wong AYL, Liangchi Li L, Fordjour Antwi-Afari M, Ou H, Wh Tsang H. The use of mobile health technology in the management of osteoarthritis: a scoping review with scientometric analyses. Int J Med Inform. Feb 2023;170:104937. [CrossRef] [Medline]
    8. Hamilton DF, Giesinger JM, Giesinger K. Technological developments enable measuring and using patient-reported outcomes data in orthopaedic clinical practice. World J Orthop. Dec 18, 2020;11(12):584-594. [CrossRef]
    9. Halm-Pozniak A, Lohmann CH, Zagra L, Braun B, Gordon M, Grimm B. Best practice in digital orthopaedics. EFORT Open Rev. May 9, 2023;8(5):283-290. [CrossRef] [Medline]
    10. Muschol J, Heinrich M, Heiss C, et al. Economic and environmental impact of digital health app video consultations in follow-up care for patients in orthopedic and trauma surgery in Germany: randomized controlled trial. J Med Internet Res. Nov 24, 2022;24(11):e42839. [CrossRef] [Medline]
    11. Elbert NJ, van Os-Medendorp H, van Renselaar W, et al. Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses. J Med Internet Res. Apr 16, 2014;16(4):e110. [CrossRef] [Medline]
    12. Joeris A, Zhu TY, Lambert S, Wood A, Jayakumar P. Real-world patient data: can they support decision making and patient engagement? Injury. May 2023;54 Suppl 3:S51-S56. [CrossRef] [Medline]
    13. van Eijck SC, Janssen DM, van der Steen MC, Delvaux E, Hendriks JGE, Janssen RPA. Digital health applications to establish a remote diagnosis of orthopedic knee disorders: scoping review. J Med Internet Res. Feb 9, 2023;25:e40504. [CrossRef] [Medline]
    14. Slover JD, Karia RJ, Hauer C, Gelber Z, Band PA, Graham J. Feasibility of integrating standardized patient-reported outcomes in orthopedic care. Am J Manag Care. Aug 2015;21(8):e494-e500. [Medline]
    15. Schreiweis B, Pobiruchin M, Strotbaum V, Suleder J, Wiesner M, Bergh B. Barriers and facilitators to the implementation of eHealth services: systematic literature analysis. J Med Internet Res. Nov 22, 2019;21(11):e14197. [CrossRef] [Medline]
    16. Castro FG, Kellison JG, Boyd SJ, Kopak A. A methodology for conducting integrative mixed methods research and data analyses. J Mix Methods Res. Sep 20, 2010;4(4):342-360. [CrossRef] [Medline]
    17. Bowen DJ, Kreuter M, Spring B, et al. How we design feasibility studies. Am J Prev Med. May 2009;36(5):452-457. [CrossRef] [Medline]
    18. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. Dec 2007;19(6):349-357. [CrossRef] [Medline]
    19. O’Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. Apr 2008;13(2):92-98. [CrossRef] [Medline]
    20. Geerdink TH, Salentijn DA, de Vries KA, et al. Optimizing orthopedic trauma care delivery during the COVID-19 pandemic. a closed-loop audit of implementing a virtual fracture clinic and fast-track pathway in a Dutch level 2 trauma center. Trauma Surg Acute Care Open. 2021;6(1):e000691. [CrossRef] [Medline]
    21. Willinge GJA, Spierings JF, Weert T, Twigt BA, Goslings JC, van Veen RN. Efficiency of a virtual fracture care protocol in non-operative treatment of adult patients with a distal radial fracture. J Hand Surg Eur Vol. Mar 2024;49(3):341-349. [CrossRef]
    22. Totton N, Lin J, Julious S, Chowdhury M, Brand A. A review of sample sizes for UK pilot and feasibility studies on the ISRCTN registry from 2013 to 2020. Pilot Feasibility Stud. Nov 21, 2023;9(1):188. [CrossRef] [Medline]
    23. Hennink M, Kaiser BN. Sample sizes for saturation in qualitative research: a systematic review of empirical tests. Soc Sci Med. Jan 2022;292:114523. [CrossRef] [Medline]
    24. Sandelowski M, Barroso J. Classifying the findings in qualitative studies. Qual Health Res. Sep 2003;13(7):905-923. [CrossRef]
    25. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [CrossRef]
    26. Johnson RE, Grove AL, Clarke A. Pillar integration process: a joint display technique to integrate data in mixed methods research. J Mix Methods Res. Jul 2019;13(3):301-320. [CrossRef]
    27. Jonker LT, Haveman ME, de Bock GH, van Leeuwen BL, Lahr MMH. Feasibility of perioperative eHealth interventions for older surgical patients: a systematic review. J Am Med Dir Assoc. Dec 2020;21(12):1844-1851. [CrossRef] [Medline]
    28. Scherer J, Yogarasa V, Rauer T, Pape HC, Heining SM. Perspectives of patients with orthopedic trauma on fully automated digital physical activity measurement at home: cross-sectional survey study. JMIR Form Res. Feb 9, 2023;7:e35312. [CrossRef] [Medline]
    29. Ly S, Tsang R, Ho K. Patient perspectives on the digitization of personal health information in the emergency department: mixed methods study during the COVID-19 pandemic. JMIR Med Inform. Jan 6, 2022;10(1):e28981. [CrossRef] [Medline]
    30. Sleat GKJ, Lefaivre KA, Broekhuyse HM, O’Brien PJ. Predicting completion of follow-up in prospective orthopaedic trauma research. OTA Int. Dec 2019;2(4):e047. [CrossRef] [Medline]
    31. Støme LN, Wilhelmsen CR, Kværner KJ. Enabling guidelines for the adoption of eHealth solutions: scoping review. JMIR Form Res. Apr 30, 2021;5(4):e21357. [CrossRef] [Medline]
    32. Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res. Feb 20, 2015;17(2):e40. [CrossRef] [Medline]
    33. Saeed N, Manzoor M, Khosravi P. An exploration of usability issues in telecare monitoring systems and possible solutions: a systematic literature review. Disabil Rehabil Assist Technol. Apr 2020;15(3):271-281. [CrossRef] [Medline]
    34. Zulman DM, Shah NH, Verghese A. Evolutionary pressures on the electronic health record. JAMA. Sep 6, 2016;316(9):923-924. [CrossRef] [Medline]
    35. Houwen T, Vugts MAP, Lansink KWW, et al. Developing mHealth to the context and valuation of injured patients and professionals in hospital trauma care: qualitative and quantitative formative evaluations. JMIR Hum Factors. Jun 20, 2022;9(2):e35342. [CrossRef] [Medline]
    36. Willems SJ, Coppieters MW, Pronk Y, et al. A clinical journey mobile health app for perioperative patients: cross-sectional study. JMIR Hum Factors. Feb 8, 2021;8(1):e20694. [CrossRef] [Medline]
    37. Fuller-Tyszkiewicz M, Richardson B, Klein B, et al. A mobile app–based intervention for depression: end-user and expert usability testing study. JMIR Ment Health. Aug 23, 2018;5(3):e54. [CrossRef] [Medline]
    38. Vo V, Auroy L, Sarradon-Eck A. Patients’ perceptions of mHealth apps: meta-ethnographic review of qualitative studies. JMIR Mhealth Uhealth. Jul 10, 2019;7(7):e13817. [CrossRef] [Medline]
    39. Lebleu J, Pauwels A, Anract P, Parratte S, Van Overschelde P, Van Onsem S. Digital rehabilitation after knee arthroplasty: a multi-center prospective longitudinal cohort study. J Pers Med. May 13, 2023;13(5):824. [CrossRef] [Medline]
    40. Poncette AS, Mosch LK, Stablo L, et al. A remote patient-monitoring system for intensive care medicine: mixed methods human-centered design and usability evaluation. JMIR Hum Factors. Mar 11, 2022;9(1):e30655. [CrossRef] [Medline]
    41. Desveaux L, Soobiah C, Bhatia RS, Shaw J. Identifying and overcoming policy-level barriers to the implementation of digital health innovation: qualitative study. J Med Internet Res. Dec 20, 2019;21(12):e14994. [CrossRef] [Medline]
    42. Spierings J, Willinge G, Kokke M, et al. Health care professionals’ experiences with a mobile self-care solution for low complex orthopedic injuries: mixed methods study. JMIR Mhealth Uhealth. Feb 2, 2024;12:e51510. [CrossRef] [Medline]
    43. Baines R, Bradwell H, Edwards K, et al. Meaningful patient and public involvement in digital health innovation, implementation and evaluation: a systematic review. Health Expect. Aug 2022;25(4):1232-1245. [CrossRef] [Medline]
    44. Lyles CR, Nelson EC, Frampton S, Dykes PC, Cemballi AG, Sarkar U. Using electronic health record portals to improve patient engagement: research priorities and best practices. Ann Intern Med. Jun 2, 2020;172(11 Suppl):S123-S129. [CrossRef] [Medline]
    45. Palacholla RS, Fischer N, Coleman A, et al. Provider- and patient-related barriers to and facilitators of digital health technology adoption for hypertension management: scoping review. JMIR Cardio. Mar 26, 2019;3(1):e11951. [CrossRef] [Medline]
    46. Madanian S, Nakarada-Kordic I, Reay S, Chetty T. Patients’ perspectives on digital health tools. PEC Innov. Dec 2023;2:100171. [CrossRef] [Medline]
    47. van Gerven P, Rubinstein SM, Nederpelt C, et al. The value of radiography in the follow-up of extremity fractures: a systematic review. Arch Orthop Trauma Surg. Dec 2018;138(12):1659-1669. [CrossRef] [Medline]
    48. Regnault A, Willgoss T, Barbic S, International Society for Quality of Life Research (ISOQOL) Mixed Methods Special Interest Group (SIG). Towards the use of mixed methods inquiry as best practice in health outcomes research. J Patient Rep Outcomes. 2017;2(1):19. [CrossRef] [Medline]
    49. Wasti SP, Simkhada P, van Teijlingen ER, Sathian B, Banerjee I. The growing importance of mixed-methods research in health. Nepal J Epidemiol. Mar 2022;12(1):1175-1178. [CrossRef] [Medline]
    50. Sandelowski M. The use of quotes in qualitative research. Res Nurs Health. Dec 1994;17(6):479-482. [CrossRef] [Medline]
    51. Pearson N, Naylor PJ, Ashe MC, Fernandez M, Yoong SL, Wolfenden L. Guidance for conducting feasibility and pilot studies for implementation trials. Pilot Feasibility Stud. Oct 31, 2020;6(1):167. [CrossRef] [Medline]
    52. Irizarry T, Shoemake J, Nilsen ML, Czaja S, Beach S, DeVito Dabbs A. Patient portals as a tool for health care engagement: a mixed-method study of older adults with varying levels of health literacy and prior patient portal use. J Med Internet Res. Mar 30, 2017;19(3):e99. [CrossRef] [Medline]

    ‎
    COREQ: Consolidated Criteria for Reporting Qualitative Studies
    ED: emergency department
    EPR: electronic patient record
    GRAMMS: Good Reporting on a Mixed Methods Study
    PROM: patient-reported outcome measure
    T0: 1-3 days after emergency department visit
    T1: 4-6 weeks after emergency department visit
    T2: 10-12 weeks after emergency department visit

    Edited by Amaryllis Mavragani; submitted 20.02.24; peer-reviewed by Felix Conrad Öttl, Quirine van der Vliet; final revised version received 15.09.24; accepted 18.09.24; published 26.11.24.

    Copyright

    © Gijs J A Willinge, Jelle F Spierings, Kim A G J Romijnders, Elke G E Mathijssen, Bas A Twigt, J Carel Goslings, Ruben N van Veen. Originally published in JMIR Formative Research (https://formative.jmir.org), 26.11.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.