PRO-based follow-up was implemented for patients with epilepsy at 3 neurological departments in the Central Denmark Region in 2012. In PRO-based follow-up, patients complete a questionnaire at home instead of having prescheduled appointments at the outpatient clinic. To attend PRO-based follow-up, patients are individually referred by a health professional. Individual referral requires that the patient must be 15 years of age or older, have no cognitive impairments, and be capable of reading and writing in Danish [28]. The questionnaire is sent at a predefined time interval of 3, 6, 12, or 24 months either electronically through a secure electronic platform or as a paper version [28].

The epilepsy questionnaire includes 47 items and deals with the frequency and intensity of seizures, medical treatment, and the patient’s well-being. The AmbuFlex system, based on an automated decision algorithm, can assess whether the patient needs clinical attention [8]. To identify patients in need of clinical attention, all response options in the questionnaire are categorized as red, yellow, or green. Red indicates that the patient needs or wants personal contact with the clinic. Yellow indicates that the patient might need personal contact. If all the patients’ answers are categorized as green, the system automatically sends a new questionnaire within the predefined period. All answers that indicate a need for clinical attention are reviewed by a nurse within 14 days. The patients’ questionnaire responses were displayed in a graphical overview available in the patients’ electronic health records. Where clinical attention is needed, the questionnaire data are used to support patient-clinician communication and clinical decision-making [28]. If a patient wants to discontinue or a clinician decides to end a PRO-based follow-up treatment, the clinician must actively register the patient as “deregistered” in the AmbuFlex system. Deceased patients are automatically deregistered in the system. PRO-based follow-ups are offered to approximately 50% of the entire patient population at the 3 neurological departments and have been used since 2012 [24]. In January 2016, approximately 3000 epilepsy outpatients attended the PRO-based follow-up program at the 3 departments.

We conducted a prospective cohort study among patients with epilepsy who attended PRO-based follow-ups at 3 neurology departments in the Central Denmark Region. During the period of January 1st to December 31st, 2016, patients were invited to answer a research questionnaire in addition to their usual scheduled epilepsy questionnaire from the outpatient clinic. The research questionnaire contained information about aspects related to self-management, general and mental health status, and patient satisfaction. Patients could respond to either a web-based or paper version of the questionnaire. Nonresponders received up to 3 reminders [29]. Clinicians assessed the usual epilepsy questionnaire according to their normal routine, but they were blinded to the research questionnaire.

The outcome was the dropout status in remote PRO-based follow-up in February 2021. This information was retrieved from the AmbuFlex system. As a clinician actively has to register the patient as “deregistered” in the AmbuFlex system, we defined dropouts in this study as “patients who, after using PRO-based follow-up, were deregistered by a clinician.” The reason for the dropout was not registered. If the patients were deregistered due to death, this information was shown in the AmbuFlex system. Patients who died during follow-up were excluded from the study.

The domain of patient satisfaction included patient involvement, confidence, comfort, and satisfaction. Information regarding these determinants was collected using questions inspired by the Danish Cancer Society’s Barometer Survey [41] and modified to fit the epilepsy outpatient setting. Information about patient involvement, confidence, and safety was obtained via the following questions: “Do you feel sufficiently involved in your course of treatment?,” “Are you confident that the epilepsy outpatient clinic contributes in the best possible way in your course of treatment?,” and “Are you comfortable with the follow-up you are receiving?.” These questions had five response options: “Yes, to a great extent,” “Yes, to some extent,” “To a minor extent,” “No, not at all,” or “Do not know.” Information about satisfaction was obtained through: “Overall, how would you assess your treatment in the epilepsy outpatient clinic?” with the following 5 response options: “Very good,” “Mostly good,” “Mostly bad,” “Very bad,” or “Do not know,” For all questions, the category “Do not know” was excluded in the analyses. Patient involvement, confidence, and safety were dichotomized into “Yes, to a great extent/Yes, to some extent” and “To a minor extent/not at all.” Satisfaction was dichotomized into “Very good/Mostly good” and “Mostly bad/Very bad.”

A Wilcoxon rank-sum test was utilized to compare age and sex between responders and nonresponders. Characteristics for the study population were presented by outcome. Continuous data were all nonnormally distributed and presented as the median and IQR. Categorical and dichotomized data were presented as numbers (n) and percentage distribution (%). The association between the potential determinants and dropouts in PRO-based follow-up was analyzed through multiple logistic regression analysis. All the odds ratio (OR) estimates were tested at a 5% significance level and reported with 95% CIs. Variables were continuously included if the linearity requirements were met. The assumption on linearity was deemed fulfilled for self-efficacy and well-being. Sex, age, education, and cohabitation were included in the adjusted analyses. The confounder variables were defined a priori based on a systematic literature review [18-27]. For the patient satisfaction domain, most of the determinants were controlled only for sex and cohabitation, as some categories had a limited number of observations. Each potential determinant was examined separately as the determinants were assessed as interdependent. All analyses were performed using the available observations giving varying observations for crude and adjusted estimates. An examination of all missing data was conducted prior to this. Upon inspection, no systematic or significant missing data were identified. For the continuous variables, postestimation analyses were performed to nuance the significance of different scores and dropouts in PRO-based follow-up. The statistical analyses were performed using Stata/IC 16.1 (StataCorp LLC).

The study was approved by the Danish Data Protection Agency (number 1-16-02-691-14). The Ethics Committee of the Central Denmark Region was consulted and decided that the study did not require their approval. According to Danish law, only studies involving human biological material require committee approval. Correspondence with the ethics committee is available from the authors upon request. Informed consent for participation in the study was obtained in accordance with guidelines from the Danish Data Protection Agency. Information was provided to patients in a letter along with the questionnaire. Patients were informed that responding to the questionnaire constituted active consent for their participation and the use of their data in the research project. They were also informed that they could withdraw their consent at any time. All data were stored and handled with confidentiality.

From January 1st to December 31st, 2016, a total of 2975 patients received a paper or web-based questionnaire. Of these, 2464 patients answered the questionnaire, resulting in a response rate of 83%. During the follow-up period, 182 patients died. The final study population included 2282 patients (Figure 1). The median age in the study population was 49 (IQR 29.7) years, 1125 (49.3%) were male, 982 (43%) had a low education, and 1645 (72.1%) were not living alone (Table 1). A total of 770 patients (33.7%) had dropped out of PRO-based follow-up in February 2021. No statistical differences were found between responders and nonresponders in sex distribution (P=.34) or age distribution (P=.70). Missing data ranged from 1.5% (general health) to 7.8% (general self-efficacy) (Table 1).

The ORs for the self-management determinants and dropout in PRO-based follow-up are presented in Table 2. Statistically significant associations were found between all potential self-management determinants and dropout. A lower degree of self-management was associated with dropout. Patients reporting having low social support for health (HLQ4) had higher odds of dropout (OR 2,02, 95% CI 1.38-3.50). Similar associations were identified for the remaining determinants regarding health literacy (Table 2). Patients who had lower confidence in knowing when to get outpatient care (PAM5) had higher odds of dropout (OR 1.94, 95% CI 1.44-2.62). A high level of self-efficacy had decreased odds for dropout per point (OR 0.96, 95% CI 0.95-0.98). When comparing a patient with the lowest GSES score (score of 10) to a patient with the highest possible GSES score (score of 40), the patient with the low self-efficacy had an OR of 3.21 (95% CI 2.05-5.03) for dropout (Table 3).

We found that patients with good general health status had statistically significantly higher odds for dropout (OR 1.28, 95% CI 1.65-2.85) compared to patients who reported excellent/very good health status. Similarly, patients with fair/poor general heath status had statistically significantly higher odds for dropout (OR 2.17, 95% CI 1.65-2.85) compared to patients who reported excellent/very good health status (Table 4). Moreover, patients with lower mental health status had higher odds of dropout. A patient with the lowest possible WHO-5 score (0) had an OR of 3.16 (95% CI 1.94-5.15) for dropout relative to a patient with the highest score (100) (Table 5).

The associations between patient satisfaction and dropout in PRO-based follow-up are presented in Table 6. Patients who, to a minor extent/not at all, felt sufficiently involved in the course of treatment had an OR of 1.93 (95% CI 1.42-2.63) of dropping out compared to patients who felt sufficiently involved. Similar associations were found for the determinants of confidence and comfort. Patients who were not satisfied and found their overall treatment to be mostly bad/very bad had an OR of 2.00 (95% CI: 1.31;3.06) for dropout compared to satisfied patients.

The majority of the patients reported a high level of self-management, good health status, and a high degree of patient satisfaction. A strong association was found between all 9 self-reported potential determinants and dropout in remote PRO-based follow-up for patients with epilepsy in the Central Denmark Region. Patients with reduced self-management, poor health status, and low patient satisfaction had higher odds of dropout in PRO-based follow-up. The association appeared independent, as the correction for sex, age, education, and cohabitation showed only a minor impact on the relation between the potential determinants and dropout.

To the best of our knowledge, previous research has not primarily focused on investigating dropout rates; instead, dropout has typically been included as a secondary finding [18,22]. Furthermore, past research has not given exclusive attention to PRO-based follow-up but has focused on telehealth more broadly, encompassing app-based interventions [18,22]. Using a meta-analysis, Meyerowitz-Katz et al determined that app-based chronic disease management interventions exhibit a dropout rate of 49% (95% CI 27%-70%) [22]. A scoping review of 31 studies on patient reasons for not using digital PRO concepts identified a dropout range of 2.4% to 72.3% [18]. Both studies reported higher dropout rates compared to the dropout rate of 33.7% in this study. The studies included follow-up periods with varying response frequencies, ranging from daily to every 3 months [18,22]. This variability in follow-up schedules may have contributed to the higher dropout rates observed in the comparative studies, along with the contextual differences between dropout rates in digital interventions and dropout rates in continuous chronic disease follow-up. Adherence to technology is another concept that can indicate how patients use digital solutions. A recently published systematic review investigated factors associated with adherence to tele-monitoring using electronic PRO measures in patients with chronic diseases, including heart failure, rheumatoid arthritis, and chronic pain [42]. This review found adherence rates to range between 61% and 96% in the included studies [42], which aligns with the dropout rate of approximately 30% found in our study.

Patient characteristics associated with dropout have been investigated mainly in relation to sociodemographic factors, such as sex, age, education, and cohabitation status in other studies [18,22,42]. Self-reported aspects, including self-management and health literacy, have to our knowledge not been reported in other studies regarding the dropout rate in digital interventions. Thus, our study contributes novel knowledge in this field and underlines the importance of considering different individual needs and competencies when implementing digital solutions in clinical care. Nielsen et al also investigated reasons for not using digital PRO concepts in clinical practice among patients with long-term conditions. They found several reasons for digital nonuse of PRO; among others, the themes covered the ability to use PRO data, engagement, emotional distress, and technical barriers [18]. From the patient’s point of view, these reasons are important to consider before implementing a digital PRO solution in a clinical setting. Reasons for dropout in PRO-based follow-up will be further investigated in future research by interviewing patients as well as clinicians.

The clinicians’ perspective is important to investigate because dropout could be related not only to digital skills or engagement by patients. From the health care provider’s point of view, there may also be concerns that arise during the treatment of patients included in a digital solution. Qualitative research regarding the clinicians’ perspective of using PRO-based follow-up in patients with epilepsy has shown reluctance among clinicians [43]. For example, some of the clinicians found the lack of interpersonal contact as a negative consequence of PRO-based follow-up and felt unsure about some of the patients’ capabilities to participate even though the patients were already included in PRO-based follow-up [43]. There is no standardized guideline regarding referral to PRO-based follow-up in patients with epilepsy in the Central Denmark Region; thus, the personal preferences amongst clinicians may play a role when deciding to refer or exclude patients from PRO-based follow-up. Prior research investigating factors associated with referral to PRO-based follow-up among patients with epilepsy and patients with rheumatoid arthritis supports this statement since patients with a higher socioeconomic background more often were referred to PRO-based follow-up compared to patients with a lower socioeconomic background [24,44]. Similar to our finding, the study in patients with epilepsy also found a positive association between a high level of health literacy and referral to PRO-based follow-up [24]. These findings in addition to our current study support the importance on focusing on the involvement of vulnerable patient groups when designing digital PRO solutions in clinical practice.

We conducted a large Danish prospective cohort study among a representative epilepsy population enrolled in remote PRO-based follow-up. Self-reported data were collected before the outcome of interest; thus, potential misclassification would be nondifferential and not bias the results. The outcome (dropout) was register-based, and missing data were acceptable for both self-reported and register-based data. Thus, the internal and external validity of this study are considered sufficient. However, some limitations should be noted. First, we possessed only baseline information about the potential determinants. It is possible that the patients’ perceptions of self-management, health status, and patient satisfaction changed during the follow-up period.

Second, the initial response rate in this study was high (83%); however, the only data available regarding the nonrespondents were sex and age. No differences were found between responders and nonresponders, but we cannot rule out that nonresponders did differ from the respondents considering other characteristics. In general, we found high levels of self-management, health status, and satisfaction amongst the responders. Nondifferential selection bias could occur if a lower level of these aspects may have been reported by the nonresponders. Third, an issue occurred with the interpretation of the question regarding the confounder variable education. This was evident for some younger patients who reported having a higher educational level than plausible for their age. It was not possible to determine if this issue was present throughout the study, and there is a possibility that this misclassification may have contributed to insufficient correction for education. Furthermore, despite adjusting for the most common sociodemographic confounders, a risk of residual confounding cannot be completely ruled out. Finally, differentiating the reasons for dropout would have enhanced the strength of this study. However, this information was not available and will be explored in future research.

In this study, we found that reduced self-management, poor health status, and patient dissatisfaction were associated with an increased odds of dropout in remote PRO-based follow-up. Thus, findings from this study support that a remote approach in outpatient follow-up may not be suitable to the whole patient population, but a selected group of patients. However, further research is necessary to explore the reasons for patient dropout. The findings underline the importance of involving patients with different needs and competencies when designing and implementing digital PRO solutions in the health care system.