This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2430) Mobile Health (mhealth) (2950) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1407) Breast Cancer (208) Web-based and Mobile Health Interventions (3309) Decision Support for Health Professionals (1192)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/51021, first published .
Support for Chronic Pain Management for Breast Cancer Survivors Through Novel Digital Health Ecosystems: Pilot Usability Study of the PainRELife Mobile App

Support for Chronic Pain Management for Breast Cancer Survivors Through Novel Digital Health Ecosystems: Pilot Usability Study of the PainRELife Mobile App

Support for Chronic Pain Management for Breast Cancer Survivors Through Novel Digital Health Ecosystems: Pilot Usability Study of the PainRELife Mobile App

Authors of this article:

Marianna Masiero1, 2 Author Orcid Image ;   Chiara Filipponi1, 2 Author Orcid Image ;   Elisa Fragale2 Author Orcid Image ;   Silvia Francesca Maria Pizzoli1, 2 Author Orcid Image ;   Elisabetta Munzone3 Author Orcid Image ;   Alessandra Milani4 Author Orcid Image ;   Luca Guido5 Author Orcid Image ;   Vittorio Guardamagna5 Author Orcid Image ;   Sara Marceglia6 Author Orcid Image ;   Roberto Prandin6 Author Orcid Image ;   Marco Prenassi6 Author Orcid Image ;   Annamaria Caruso6, 7 Author Orcid Image ;   Vania Manzelli7 Author Orcid Image ;   Chiara Savino7 Author Orcid Image ;   Costanza Conti8 Author Orcid Image ;   Federica Rizzi8 Author Orcid Image ;   Alice Casalino8 Author Orcid Image ;   Giulia Candiani9 Author Orcid Image ;   Francesca Memini9 Author Orcid Image ;   Luca Chiveri10 Author Orcid Image ;   Andrea Luigi Vitali10 Author Orcid Image ;   Massimo Corbo10 Author Orcid Image ;   Roberto Grasso1, 2 Author Orcid Image ;   Florence Didier2 Author Orcid Image ;   Roberta Ferrucci1 Author Orcid Image ;   Gabriella Pravettoni1, 2 Author Orcid Image
Article Authors Cited by (3) Tweetations (2) Metrics

    Original Paper

    1Department of Oncology and Hemato-Oncology, University of Milan, Milan, Italy

    2Applied Research Division for Cognitive and Psychological Science, European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy

    3Division of Medical Senology, European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy

    4Nursing School, European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy

    5Division of Palliative Care and Pain Therapy, European Institute of Oncology, Istituto di Ricovero e Cura a Carattere Scientifico, Milan, Italy

    6Dipartimento di Ingegneria e Architettura, Università degli Studi di Trieste, Milan, Italy

    7Nuvyta, Società a Responsabilità Limitata, Cologno Monzese, Italy

    8Istituto di Management Sanitario, Milano, Italy

    9Agenzia di comunicazione scientifica Zadig, Società a Responsabilità Limitata, Società benefit, Milan, Italy

    10Dipartimento di Scienze Neuroriabilitative, Casa di Cura del Policlinico, Milan, Italy

    Corresponding Author:

    Marianna Masiero, PhD

    Department of Oncology and Hemato-Oncology

    University of Milan

    Via Festa del Perdono 7

    Milan, 20122

    Italy

    Phone: 39 0294372009

    Email: marianna.masiero@unimi.it


    Background: Chronic pain is one of the most common and critical long-term effects of breast cancer. Digital health technologies enhance the management of chronic pain by monitoring physical and psychological health status and supporting pain self-management and patient treatment decisions throughout the clinical pathway.

    Objective: This pilot study aims to evaluate patients’ experiences, including usability, with a novel digital integrated health ecosystem for chronic pain named PainRELife. The sample included patients with breast cancer during survivorship. The PainRELife ecosystem comprises a cloud technology platform interconnected with electronic health records and patients' devices to gather integrated health care data.

    Methods: We enrolled 25 patients with breast cancer (mean age 47.12 years) experiencing pain. They were instructed to use the PainRELife mobile app for 3 months consecutively. The Mobile Application Rating Scale (MARS) was used to evaluate usability. Furthermore, pain self-efficacy and participation in treatment decisions were evaluated. The study received ethical approval (R1597/21-IEO 1701) from the Ethical Committee of the European Institute of Oncology.

    Results: The MARS subscale scores were medium to high (range: 3.31-4.18), and the total app quality score was 3.90. Patients with breast cancer reported reduced pain intensity at 3 months, from a mean of 5 at T0 to a mean of 3.72 at T2 (P=.04). The total number of times the app was accessed was positively correlated with pain intensity at 3 months (P=.03). The engagement (P=.03), information (P=.04), and subjective quality (P=.007) subscales were positively correlated with shared decision-making. Furthermore, participants with a lower pain self-efficacy at T2 (mean 40.83) used the mobile app more than participants with a higher pain self-efficacy (mean 48.46; P=.057).

    Conclusions: The data collected in this study highlight that digital health technologies, when developed using a patient-driven approach, might be valuable tools for increasing participation in clinical care by patients with breast cancer, permitting them to achieve a series of key clinical outcomes and improving quality of life. Digital integrated health ecosystems might be important tools for improving ongoing monitoring of physical status, psychological burden, and socioeconomic issues during the cancer survivorship trajectory.

    International Registered Report Identifier (IRRID): RR2-10.2196/41216

    JMIR Form Res 2024;8:e51021

    doi:10.2196/51021

    Keywords

    chronic pain; eHealth; cancer; decision-making; survivorship; self-efficacy; pain; oncology; health ecosystem; health ecosystems; breast; survivor; survivors; mHealth; mobile health; app; apps; applications; MARS 


    In 2040, it is expected there will be approximately 26 million new cancer survivors in the United States, underscoring the growing importance of survivorship [1,2]. Mullan [3] defined survivorship as a process characterized by 3 different stages: acute survival, from diagnosis to active treatments; extended survival, from treatments to active surveillance; and permanent survival, in which the probability of recurrence is low. As suggested by Mullan’s definition, cancer survivorship is a complex, multistep, dynamic process characterized by evolving needs and challenges. Pain, fatigue, and psychological distress (eg, anxiety, depression, worry, and fear of cancer recurrence) are typical long-term effects that deleteriously affect survivors’ engagement in work, personal, and social activities [4-6]. In particular, one of the most common and critical long-term effects of cancer in survivors is chronic pain. It has been linked with several physical, psychological, and socioeconomic sequelae. A study by Jiang et al [1] stated that, of 4526 cancer survivors, around 34.6% reported chronic pain, and 16.1% conveyed that it limited daily life and work activities. Notwithstanding, chronic pain throughout the survivorship trajectory is underinvestigated and undertreated [5,7,8]. Since chronic pain affects the quality of life (QoL) of patients with breast cancer [9], it is essential to design, test, and implement patient-driven interventions [10] that enable ongoing monitoring of physical and psychological health status, from the “hospital to the patient’s home,” and support pain self-management and patient treatment decisions throughout the entire clinical pathway. This might be particularly important in extended and permanent survivorship, reducing the risk of cancer survivors exiting the care system [5].

    A growing body of evidence has shown that using digital health technologies integrated into dynamic ecosystems enhances the management of chronic pain by assessing patients' physical health and psychological well-being and providing tailored interventions [11]. Digital health technologies aim to integrate various digital tools and technologies, including patient electronic health records, telemedicine, wearables, and mobile apps, into the health care system [12]. Overall, digital health technology encompasses both eHealth, which involves the use of the internet and related technologies to enhance health care systems through information and communication technology [13,14], and mobile health, which focuses on health practices supported by mobile devices [15]. The widespread use of digital health technologies has opened an innovative “window of opportunity” for managing chronic pain in a more personalized, accessible, and patient-centered way [8]. Evidence has highlighted that digital health technologies are valuable solutions for remotely collecting patient health data (eg, using patient-reported measures or wearable devices), improving symptom management, and decreasing patient appointments and hospitalizations [16]. Jongerius et al [17], in a systematic review, highlighted that digital health technologies are used in cancer clinical practice for the following different reasons: to stimulate the adoption of preventive behaviors, to increase early cancer identification, to manage cancer care, and to provide assistance to cancer survivors. Overall, digital health technology supports patients and the health care system to achieve several critical outcomes for better cancer clinical management [17-19]. For example, Zhu et al [19] reported that an app-based program named “Breast Cancer e-Support” was able to decrease symptomologies related to anticancer treatments and therefore improve self-efficacy and QoL. In addition, Maguire et al [20] designed and tested ASyMS (Advanced Symptom Management System) for the management of chemotherapy toxicity; it not only enables the evaluation and monitoring of patient symptomatology related to anticancer treatments but also provides tailored and evidence-based recommendations to manage symptoms [20].

    Considering the specific case study of chronic pain in cancer survivorship, digital health technologies enhance access to nonpharmacological interventions; address pain-related mobility issues; improve patient networking and connections; foster self-management, self-efficacy, coping, and patient engagement; and facilitate communication among health care professionals of various specialties (eg, physicians, nurses, physiotherapists, psychologists) [18,21,22]. For example, Ranney et al [18] described that 89.5% of patients with chronic pain reported using digital health tools (eg, websites to search for health information, social media, and mobile apps), and such usage is associated with improved chronic pain coping mechanisms.

    Different digital health technologies have been designed to manage cancer and pain in the clinical pathway [23]. For example, an educational digital intervention called the “Pain Education after Cancer Collaborative” (PECAN project) was developed for survivors of breast cancer; in the intervention, a decision tree system is used to provide tailored educational information to cancer survivors based on their answers to specific queries [24]. In addition, a Mobile Pain Coping Skills Training Protocol has been proposed to support patients’ understanding of the pain experience and strategies to cope with the pain [25]. More generally, Hauser-Ulrich et al [26] recently developed a text-based chatbot named “SELMA” (PainSELfMAnagement) aimed to booster self-management of chronic pain in different types of diseases, supporting health care professionals in the delivery of evidence-based interventions. Moreover, digital health technologies could encourage patients to be more involved in their treatment decisions, through the implementation of specific decision aids [16,27-29]. Studies have stressed that the implementation of tailored decision aids in mobile apps increases patients’ awareness about treatment preferences (eg, pharmacological vs nonpharmacological treatments), reduces decisional conflict, and enhances adherence to treatments [30-32].

    Even with the key role of digital health technologies in the cancer clinical pathway, few studies on digital and integrated health ecosystems are currently available [33]. Consequently, in this pilot study, our primary endpoint was the usability of the novel digital integrated health ecosystem, PainRELife, for managing and monitoring chronic pain in patients with breast cancer throughout the survivorship trajectory. Further, we aimed to evaluate the PainRELife ecosystem's effectiveness at enhancing pain self-efficacy, improving shared decision-making, and reducing pain perception as secondary endpoints. The PainRELife ecosystem stands out as the first to seamlessly integrate all the essential components required for comprehensive pain management within a single platform. This includes features such as pain monitoring, physical and psychological assessment, e-diaries, exercises, educational resources, and decision aids. Furthermore, it incorporates a dedicated platform for health care providers and a robust big data cloud infrastructure. This holistic integration sets our ecosystem apart in the realm of pain management solutions.


    Study Design and Patient Recruitment

    PainRELife Ecosystem

    This pilot study is nested in a national project titled “PainRELife, Sustainable and integrated big data ecosystem for continuity of care and decision support for patients with pain“ (ID: 1173269). This national project guided the development and testing of an integrated health ecosystem for the management of chronic pain. Specifically, the PainRELife ecosystem consists of a cloud technology platform interconnected with electronic health records, which is named the Nu Platform, connected to the Fast Healthcare Interoperability Resources (FHIR) server for data analysis related to the patient care pathway. Health care providers use the Nu Platform to collect and store patient clinical data, and it enables the ongoing monitoring of patient health status (eg, pain, psychological well-being, and decision preferences about treatments; see Figures 1A and 1B), from diagnosis and active treatments to follow-ups (see Figures 1C and 1D). In addition, a big data infrastructure linked to the FHIR server enables a series of dynamic dashboards aimed at providing a systematic, intuitive outline of patient population features that might be used by researchers, clinicians, and health care stakeholders. The Nu Platform is associated with a mobile app for patients named PainRELife, which collects health care data. This technological solution permits dual communication between patients and health care professionals. Information collected by the mobile app is saved in the Nu Platform and overseen by health care professionals [7].

    The PainRELife mobile app enables patient education and the collection of patient-reported outcomes. The mobile app is composed of an ”educational section“ that includes educational resources to improve learning about chronic pain throughout the cancer pathway (throughout the different phases of survivorship: acute, extended, and permanent; see Figure 2D) [3] and a “pain and psychological well-being assessment section” that contains a set of validated questionnaires (eg, pain intensity and interference, anxiety, and depression; see Figures 2A and 2B). Furthermore, the mobile app includes an e-diary (see Figure 2C) and exercises for pain and emotion-body mapping (see Figures 2E and 2F), enabling a holistic evaluation of psychological well-being and the pain experience. In addition, the mobile app incorporates a decision aid section, which is structured in 2 modules: profiling patients’ preferences and a decision tree (see Figure 2G). These modules are designed to empower patients with breast cancer by increasing their awareness of treatment preferences and facilitating shared decision-making regarding their care. The “profiling patients’ preferences” module aims to assist patients with evaluating and comprehending essential aspects of both pharmacological and nonpharmacological treatments. This includes understanding how interventions and treatments work to reduce pain or aid in recovery, identifying their advantages, and recognizing potential disadvantages. The decision tree module enables patients with breast cancer to tailor their health care preferences using the subjective expected utility approach [7] (see Figure 2G). This empowers patients with breast cancer to actively participate in the decision-making process, aligning their treatment choices with their unique needs and goals.

    ‎
    Figure 1. Health care professional interface on the Nu Platform: (A) home page displaying all available activities for health care professionals, (B) patient questionnaire list providing the measures used to assess psychological and physical status administered via the PainRELife mobile app, (C) patient list providing a directory of all patients registered on the Nu Platform, (D) clinical evaluation page offering access to detailed information on clinical events and therapeutic suggestions.
    ‎
    Figure 2. Patient interface on the PainRELife mobile app: (A) home page showing an overview of the mobile app sections, (B) pain and psychological well-being assessment section displaying the questionnaires that patients are required to complete, (C) patient’s e-diary, (D) educational section showing some of the educational content within the mobile app, (E) and (F) pain and emotion-body mapping exercises, (G) decision aid section showing an example for preferences for pharmacological and nonpharmacological treatments.
    Participants

    Participants of this pilot usability study included 25 patients with diagnosed breast cancer and pain (mean age 47.12, SD 8.41 years) admitted to the Division of Medical Senology and the Division of Pain Therapy and Palliative Care of the European Institute of Oncology (IEO). Participants were introduced to the mobile app after their clinic visit and instructed to use it for 3 months consecutively. Participants were recruited according to the following established set of inclusion criteria: >18 years old, affected by breast cancer, has undergone surgical intervention for breast carcinoma, experiencing post-surgical pain (≥3 on a numeric rating scale [NRS]), and in possession of internet access and a personal smartphone. We excluded patients with breast cancer who had a previous or ongoing psychiatric or neurological disorder or other disease requiring active analgesic treatments. The inclusion and exclusion criteria were established considering that chronic pain is a common side effect (related to both the surgery and anticancer treatments) for patients with breast cancer (~60% experience it), and persistent acute pain after surgery is considered a risk factor for developing chronic pain during survivorship [34]. A full and detailed description of the research protocol of this pilot study was published previously [7].

    Instruments

    Patient sociodemographic and medical data were gathered through electronic medical records and a set of ad hoc items during the enrollment consultation. For the perceived pain assessment, the NRS was used to evaluate pain using a numerical range from 0 (no pain) to 10 (severe pain) [35]. Further, a set of validated self-measures was used to evaluate the primary and secondary endpoints. In detail, the Mobile Application Rating Scale (MARS) was used to evaluate the eHealth platform usability. MARS is a self-administered questionnaire with 29 items evaluating the following dimensions: engagement; functionality; aesthetics; quality of the information received; subjective perception of the app quality; impact of the mobile app on knowledge, attitudes, and probability of modifying the target individual behaviors (in this specific case, it refers to behaviors related to pain management). The Cronbach α of the NRS is .90, indicating good internal consistency [36-38]. The Pain Self-Efficacy Questionnaire (PSEQ) is a self-administered questionnaire consisting of 10 items that evaluate self-efficacy in patients with pain. The Cronbach α is .94, indicating excellent internal consistency [39,40]. Finally, the 9-item Shared Decision-Making Questionnaire (SDM-Q-9) is a self-administered questionnaire comprising 9 items that evaluate a series of aspects related to the possibility of achieving a shared decision [41]. The Cronbach α is .94, indicating excellent internal consistency [42].

    Statistical Considerations

    A series of descriptive analyses were performed to depict the characteristics of the sample. In order to evaluate the primary endpoint, the mean score and its SD were calculated for each MARS subscale (engagement experienced while using the app; functionality; aesthetics; quality of the information received; subjective perception of the app quality; expected impact on knowledge, attitudes, and probability to change user behaviors) at 3 months; in addition, the total number of times the PainRELife mobile app was accessed by each participant was determined.

    Further, a new variable named total app quality was created using the mean values of the engagement, functionality, aesthetics, and information quality MARS subscales. The final measurement of app quality was the average value of the 4 means [43]. Pearson correlational analysis was performed among all self-reports used (NRS, PSEQ, SDM-Q-9, MARS) and the total number of times the app was accessed during the 3 months of the study. A repeated measures ANOVA was performed to detect variation in pain intensity (NRS) from T0 (baseline) to T2 (3 months). Further, a new dichotomous variable named “frequency of use” was created considering the entire number of times the app was accessed (mean 22.92, SD 15.60; range 2-73) and the lowest number of times the PainRELife mobile app needed to be accessed (21 times) by participants to finalize the study’s tasks. The “frequency of use” variable permitted dividing the participants into groups based on higher and lower frequencies of access. Further, a Student t test was run to evaluate the difference between the “frequency of use” and PSEQ, SDM-Q-9, and MARS scores. Data were analyzed using SPSS version 26.0 (IBM Corp). 

    Ethical Considerations

    This study received ethical approval in December 2021 (R1597/21-IEO 1701) from the Ethical Committee of the IEO and respects the Declaration of Helsinki and Good Clinical Practice Guidelines. All participants read and signed the informed consent form, which encompassed a comprehensive and exhaustive explanation of the primary and secondary endpoints, study procedures, and possible risks and benefits. Participants were not compensated and were able to withdraw their participation at any time during the study. Concerning privacy and confidentiality protection, all data collected were deidentified and anonymized, complied with national data protection legislation, and will be stored in the IEO databases for 10 years.

    The authors affirm that human research participants provided informed consent for publication of their data.


    Participant Characteristics

    The sociodemographic, cancer, and treatment characteristics are listed in Tables 1-3.

    Table 1. Sociodemographic information of participating patients with breast cancer (n=25).
    CharacteristicsResults, n (%)
    Marital status

    		Cohabiting1 (4)

    		Widowed3 (12)

    		Single5 (20)

    		Married16 (64)
    Educational level

    		PhD2 (8)

    		Master’s degree8 (32)

    		High school12 (48)

    		Primary school3 (12)
    Table 2. Diagnosis, cancer type, familiarity, and genetic mutation of participating patients with breast cancer (n=25).
    Cancer characteristicsResults, n (%)
    Diagnosis

    		Lobular carcinoma3 (12)

    		Ductal carcinoma17 (68)

    		Ductal carcinoma in situ3 (12)

    		Mucinous carcinoma1 (4)

    		Occult carcinoma1 (4)
    Cancer type

    		Triple negative2 (8)

    		HER2+a5 (20)

    		Luminal18 (72)
    Familiarity

    		I° breast8 (32)

    		II° breast6 (24)

    		No familiarity11 (44)
    Mutation

    		BRCA12 (8)

    		BRCA12 (8)

    		Negative6 (24)

    		No testing13 (52)

    aHER2: human epidermal growth factor receptor 2.

    Table 3. Surgery, medical treatments, and radiotherapy undergone by participating patients with breast cancer (n=25).
    Treatment characteristicsResults, n (%)
    Surgery

    		Mastectomy23 (92)

    		Axillary dissection1 (4)

    		Quadrantectomy1 (4)
    Medical treatment

    		Chemotherapy + endocrine therapy8 (32)

    		Chemotherapy + immune therapy2 (8)

    		Endocrine therapy12 (48)

    		Immune + endocrine therapy1 (4)

    		No treatment2 (8)
    Radiotherapy

    		Yes8 (32)

    		No17 (68)

    Usability

    The total MARS score (ranging from 1 to 5) provided overall medium-to-high mean values for each subscale (range 3.31-4.18; see Table 4) and a mean total app quality score of 3.90 (SD 0.506), suggesting generally good usability as evaluated by the participants. This was also confirmed by the total number of times the participants accessed the app during the entire study (mean 22.92, SD 15.60; range 2-73). In particular, 3 of 5 subscales had the highest scores: functionality (mean 4.14, SD 0.630), information (mean 4.18, SD 0.608), and behavioral change (mean 4.05, SD 0.666).

    On the functionality subscale, 57% (15/23) of the participants judged that the mobile app is straightforward to use. Moreover, 91% (21/23) of the participants affirmed that the interactions are reliable and intuitive (ease of use: 8/23, 35% agree; 13/23, 57% strongly agree), positively evaluated the design (gestural design: 8/23, 35% agree; 12/23, 52% strongly agree), and evaluated the navigation properties as good (navigation: 12/23, 52% agree; 8/23, 35% strongly agree). However, some slight uncertainties were observed regarding the general performance of the mobile app, specifically moving between pages and sections (performance: 8/23, 35% undecided; see Table 5).

    Concerning the distribution of responses in the information subscale, 78% (18/23) of the participants reported that the information in the mobile app is evidence-based (information: 9/23, 39% agree; 9/23, 39% strongly agree), relevant, focused on chronic pain in breast cancer and its management during the disease clinical pathway (quality of information: 9/23, 39% agree; 11/23, 48% strongly agree), and trustworthy (credibility: 22/23, 96% strongly agree). Further, the amount of information (quantity of information: 7/23, 30% agree; 9/23, 39% strongly agree) and how the information is reported using different setups (visual information: 11/23, 48% agree; 9/23, 39% strongly agree) were considered positive by the participants (see Table 6). Finally, most participants stated that the mobile app's goals are achievable (goals: 11/23, 48% agree; 3/23, 13% strongly agree), even if 30% (7/23) reported some concerns.

    Last, the distribution of responses in the behavioral change subscale revealed that 83% (19/23) of the participants strongly agreed that the mobile app had improved awareness about the issue of chronic pain in the cancer disease pathway, and 70% (16/23) agreed that the app increased chronic pain–related knowledge. Likewise, 70% (16/23) of the participants reported that the mobile app might support attitudes toward chronic pain (attitudes: 9/23, 39% agree; 7/23, 30% strongly agree; see Table 7).

    In addition, most of the participants reported that the mobile app would potentially be helpful to bolster help-seeking behaviors (help seeking: 5/23, 22% agree; 9/23, 39% strongly agree) as well as support an intention to change (intention to change: 5/23, 22% agree; 9/23, 39% strongly agree). Still, 39% (9/23) showed concerns about the capacity to transform intention into a fundamental behavioral change (see Table 7). Overall, participants judged the app to be well-targeted (engagement subscale: mean 3.31, SD 0.617) and the app’s layout to be adequate (aesthetics subscale: mean 3.98, SD 0.849); likewise, the overall subjective quality was adequate (subjective quality subscale: mean 3.50, SD 0.494).

    Table 4. Mobile Application Rating Scale (MARS) subscale scores.
    MARS subscalesResults, mean (SD)
    Engagement3.31 (0.617)
    Functionality4.14 (0.630)
    Aesthetics3.98 (0.850)
    Information4.18 (0.608)
    Subjective quality3.50 (0.494)
    Behavioral change4.05 (0.666)
    Total app quality3.90 (0.506)
    Table 5. Functionality assessment of the PainRELife mobile app using the Mobile Application Rating Scale (MARS) [37,38] (n=23).
    Functionality assessmentResponse, n (%)

    		Strongly disagreeDisagreeUndecidedAgreeStrongly agree
    Ease of usea002 (9)8 (35)13 (57)
    Gestural designb1 (4)02 (9)8 (35)12 (52)
    Navigationc01 (4)2 (9)12 (52)8 (35)
    Performanced03 (13)8 (35)7 (30)5 (22)

    a“How easy is it to learn how to use the app?”; “How clear are the menu labels/icons and instructions?”

    b“Are interactions (taps/swipes/pinches/scrolls) consistent and intuitive across all components/screens?”

    c“Is moving between screens logical/accurate/appropriate/uninterrupted; are all necessary screen links present?”

    d“How accurately/fast do the app features (functions) and components (buttons/menus) work?”

    Table 6. Information assessment of the PainRELife mobile app using the Mobile Application Rating Scale (MARS) [37,38] (n=23).
    Information assessmentResponse, n (%)

    		Strongly disagreeDisagreeUndecidedAgreeStrongly agree
    Informationa005 (22)9 (39)9 (39)
    Credibilityb0001 (4)22 (96)
    Quality of informationc1 (4)02 (9)9 (39)11 (48)
    Quantity of informationd1 (4)06 (26)7 (30)9 (39)
    Visual informatione02 (9)1 (4)11 (48)9 (39)
    Goalsf2 (9)07 (30)11 (48)3 (13)

    a”Contains high-quality information (eg, text, feedback, measures, references) from a credible source.“

    b“Does the app come from a legitimate source (specified in app store description or within the app itself)?”

    c“Is app content correct, well written, and relevant to the goal/topic of the app?”

    d”Is the extent of coverage within the scope of the app and comprehensive but concise?”

    e“Is the visual explanation of concepts—through charts/graphs/images/videos, etc—clear, logical, correct?”

    f”Does app have specific, measurable, and achievable goals (specified in the app store description or within the app itself)?”

    Table 7. Behavior change assessment of the PainRELife mobile app using the Mobile Application Rating Scale (MARS) [37,38] (n=23).
    Behavior change assessmentResponse, n (%)

    		Strongly disagreeDisagreeUndecidedAgreeStrongly agree
    Awarenessa001 (4)3 (13)19 (83)
    Knowledgeb0007 (30)16 (70)
    Attitudesc01 (4)6 (26)9 (39)7 (30)
    Help seekingd03 (13)6 (26)5 (22)9 (39)
    Intention to changee02 (8)7 (30)5 (22)9 (39)
    Behavior changef4 (17)2 (9)9 (39)3 (13)5 (22)

    a“This app is likely to increase awareness of the importance of addressing chronic pain.”

    b“This app is likely to increase knowledge/understanding of chronic pain.”

    c“This app is likely to change attitudes toward improve chronic pain.”

    d“This app is likely to increase intentions/motivation to address chronic pain.“

    e”Use of this app is likely to encourage further help-seeking for chronic pain.”

    f“Use of this app is likely to decrease chronic pain.”

    Frequency of Use and Pain Self-Efficacy

    According to the Student t test, younger participants used the mobile app less (mean 44.15, SD 7.11) than older participants (mean 50.33, SD 8.80; t23=–1.937, P=.03; d=0.77). A difference in pain self-efficacy was observed between participants with higher versus lower frequency use (t23=1.644, P=.057; d=0.65). The latter data indicate that, at T2, participants with a lower pain self-efficacy (mean 40.83, SD 14.58) used the mobile app more than participants with a higher pain self-efficacy (mean 48.46, SD 7.90).

    Pain Intensity and Shared Decision-Making

    The repeated measures ANOVA revealed that participants reported a reduction in pain intensity from T0 (mean 5, SD 1.68) to T2 (mean 3.72, SD 2.59; F2,48=3.407, P=.04). A positive correlation was found between the total number of times the mobile app was accessed and pain intensity at T2 (r=0.458, P=.03).

    No correlations were detected between the MARS subscales and PSEQ or NRS. A negative correlation was observed between the subjective quality subscale and the number of times the mobile app was accessed (r=–0.498, P=.02). Further, the engagement (r=0.445, P=.03), information (r=0.427, P=.04), and subjective quality (r=0.548, P≤.007) subscales were positively correlated with shared decision-making.


    Principal Findings

    Considering the primary endpoint of this pilot study, patients with breast cancer provided a generally positive rating for the usability of the PainRELife mobile app. Patients with breast cancer appreciated both the quality and quantity of the health information provided on chronic pain and how chronic pain should be managed during the survivorship trajectory. Specifically, the information in the different sections and modules were perceived as informative and comprehensible (20/23, 87%) and from credible sources of information (18/23, 78%). Most of the patients with breast cancer reported that habitual use of the mobile app helped increase help-seeking behaviors for chronic pain (14/23, 61%), their general attitudes toward chronic pain, and their willingness to identify preeminent strategies for managing chronic pain. These results are particularly noteworthy considering that many studies have suggested that chronic pain syndrome in patients with breast cancer is commonly undiagnosed and not often considered by oncologists [8]. In addition, many cancer patients report poor knowledge about cancer-related chronic pain, available interventions, and possible health system resources [44].

    Furthermore, the positive correlation between the total number of times the mobile app was accessed and pain intensity (P=.03) might indicate that patients with breast cancer who had a higher pain level might have utilized the mobile app to find evidence-based information and strategies to self-manage their pain. This possible interpretation might be linked to the difference in pain self-efficacy observed between the participants with higher versus lower frequency mobile app use. Specifically, participants with lower pain self-efficacy used the mobile app more than participants with higher pain self-efficacy. Perhaps the participants with lower pain self-efficacy used the mobile app to find a strategy or way to manage their pain. Self-efficacy has a crucial role for patients with cancer, and studies have reported that it improves psychological well-being, reduces fear of cancer recurrence, enhances self-care, and improves management of symptoms such as pain [45,46]. Vinnikova et al [47] observed that individuals might use mobile apps to learn more about their health problems and monitor their physical and psychological status. Furthermore, considering that self-efficacy is an attribute of cancer pain self-management, the prevalent use of the mobile app by participants with a lower self-efficacy could explain the percentage of participants who reported concerns about intention to change and the capacity to transform intention into a fundamental behavioral change [48].

    A second series of results are linked to the secondary endpoints. Participants reported a reduction in pain intensity at 3 months (P=.04). We argue that the use of the mobile app might have relieved the pain experience, disease, and treatment-related symptomatology as observed in other previous studies [16,19]. One noteworthy finding is related to the association between specific features of the mobile app, evaluated with the MARS, and shared decision-making. Indeed, participants who provided higher positive evaluations about engagement (P=.03), information (P=.04), and subjective quality (P=.007) also reported higher perceptions of having shared decisions along their care pathway. We argue that patients with breast cancer who feel involved in their treatment decisions are more engaged with the mobile app. For this type of patient, information retrieved in the mobile app might be used to reinforce and reiterate the ability to achieve a shared decision throughout their care pathway.

    Limitations

    Despite the interesting and challenging results, this pilot usability study had some limitations that must be considered. The primary limitation is the relatively small sample size of patients with breast cancer and the decision to use a single group to test usability. This decision might have caused a loss of pertinent information about the patients’ perceptions of the usefulness of this digital health technology. However, our sampling strategy is consistent with the pilot study design and methodological guidelines [49-51]. Related to this point, we must also mention that the statistical significance reported for pain self-efficacy might be considered borderline (P=.057). However, the effect size is medium-to-large (d=0.65), which supports the statistical difference between the groups. We argue that the P value might be due to the small sample size [52]. We also argue that this could be a significant result that has to be further investigated in successive studies, considering the positive effect of cancer pain self-management on QoL, when compared with pharmacological treatments such as opioid consumption [48].

    In addition, the “frequency of use” variable had a moderately high SD (15.60). However, the distribution of the participants between the low frequency (n=13) and high frequency (n=12) groups was homogeneous and balanced. Furthermore, we hypothesized the presence of selection bias resulting from the inclusion criteria, which required participants to have internet access and a personal smartphone. This criterion may have limited the inclusion of certain vulnerable groups among patients with breast cancer, such as older adults or individuals with lower health literacy and socioeconomic challenges who could be at higher risk of undiagnosed chronic pain. Most of our patients with breast cancer also reported medium-to-low pain during the entire study and were in the acute and extended stages of the survivorship trajectory, which might have affected the frequency of mobile app use. Indeed, even if the total number of times the mobile app was accessed was relatively high and satisfactory, in the last month of the study, some participants decreased their total usage; 2 of 25 participants used the mobile app only at enrollment. The last limitation is related to the previous one and concerns the lack of assessment of the timing of mobile app use. Indeed, only the total number of times the mobile app was accessed was collected and evaluated. These limitations have been intensely discussed in the full protocol published elsewhere, and we plan to address them in future studies [7].

    Conclusions

    The data retrieved from this pilot study evaluating patients’ experiences using a novel and integrated health ecosystem for the management of chronic pain for breast cancer survivors are consistent with other studies highlighting that digital health technologies, when developed using a patient-driven approach, might be considered valuable tools for increasing the participation of patients with breast cancer in clinical care. In addition, these tools permit the achievement of critical clinical outcomes and improvement in QoL [4,8,22]. Moreover, health-integrated ecosystems permit secondary key outcomes such as reducing the burden on health care professionals and optimizing health system resources. Finally, we argue that digital integrated health ecosystems might be important devices for improving the ongoing monitoring of physical status, psychological burden, and socioeconomic issues during the cancer survivorship trajectory.

    Acknowledgments

    This work is supported by a grant from Regione Lombardia: “PainRELife, Sustainable and integrated big data ecosystem for continuity of care and decision support for patients with pain“ (ID: 1173269). The European Institute of Oncology (Istituto Europeo di Oncologia [IEO], Italy) monitors the study's scientific, legal, and ethical aspects. Participants are recruited at IEO.

    Data Availability

    The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request.

    Authors' Contributions

    MM, SFMP, CF, and GP conceived and designed the study. GP and MM coordinated the study, GP and MM acquired legal authorizations, and CF and EF managed the participants. MM handled the drafting and writing of the manuscript. All authors read and approved the final manuscript.

    Conflicts of Interest

    VM is a partner and chief technology officer of Nuvyta, the software house that developed and sells Nuplatform.

    1. Jiang C, Wang H, Wang Q, Luo Y, Sidlow R, Han X. Prevalence of chronic pain and high-impact chronic pain in cancer survivors in the United States. JAMA Oncol. Aug 01, 2019;5(8):1224-1226. [FREE Full text] [CrossRef] [Medline]
    2. Bluethmann SM, Mariotto AB, Rowland JH. Anticipating the "Silver Tsunami": prevalence trajectories and comorbidity burden among older cancer survivors in the United States. Cancer Epidemiol Biomarkers Prev. Jul 30, 2016;25(7):1029-1036. [FREE Full text] [CrossRef] [Medline]
    3. Mullan F. Seasons of survival: reflections of a physician with cancer. N Engl J Med. Jul 25, 1985;313(4):270-273. [CrossRef]
    4. Hernandez Silva E, Lawler S, Langbecker D. The effectiveness of mHealth for self-management in improving pain, psychological distress, fatigue, and sleep in cancer survivors: a systematic review. J Cancer Surviv. Feb 2019;13(1):97-107. [CrossRef] [Medline]
    5. Vaz-Luis I, Masiero M, Cavaletti G, Cervantes A, Chlebowski R, Curigliano G, et al. ESMO Expert Consensus Statements on Cancer Survivorship: promoting high-quality survivorship care and research in Europe. Ann Oncol. Dec 2022;33(11):1119-1133. [FREE Full text] [CrossRef] [Medline]
    6. Munzone E, Bagnardi V, Campennì G, Mazzocco K, Pagan E, Tramacere A, et al. Preventing chemotherapy-induced alopecia: a prospective clinical trial on the efficacy and safety of a scalp-cooling system in early breast cancer patients treated with anthracyclines. Br J Cancer. Aug 17, 2019;121(4):325-331. [FREE Full text] [CrossRef] [Medline]
    7. Masiero M, Filipponi C, Pizzoli SFM, Munzone E, Guido L, Guardamagna VA, et al. Usability testing of a new digital integrated health ecosystem (PainRELife) for the clinical management of chronic pain in patients with early breast cancer: protocol for a pilot study. JMIR Res Protoc. May 12, 2023;12:e41216. [FREE Full text] [CrossRef] [Medline]
    8. De Groef A, Meeus M, Heathcote LC, Wiles L, Catley M, Vogelzang A, et al. Treating persistent pain after breast cancer: practice gaps and future directions. J Cancer Surviv. Dec 11, 2023;17(6):1698-1707. [FREE Full text] [CrossRef] [Medline]
    9. Filipponi C, Masiero M, Pizzoli SFM, Grasso R, Ferrucci R, Pravettoni G. A comprehensive analysis of the cancer chronic pain experience: a narrative review. CMAR. Jul 2022;Volume 14:2173-2184. [CrossRef]
    10. Meints S, Edwards R. Evaluating psychosocial contributions to chronic pain outcomes. Prog Neuropsychopharmacol Biol Psychiatry. Dec 20, 2018;87(Pt B):168-182. [FREE Full text] [CrossRef] [Medline]
    11. Marceglia S, Conti C. A technology ecosystem for chronic pain: promises, challenges, and future research. Mhealth. Feb 21, 2017;3:6-6. [FREE Full text] [CrossRef] [Medline]
    12. Yao R, Zhang W, Evans R, Cao G, Rui T, Shen L. Inequities in health care services caused by the adoption of digital health technologies: scoping review. J Med Internet Res. Mar 21, 2022;24(3):e34144. [FREE Full text] [CrossRef] [Medline]
    13. Boogerd EA, Arts T, Engelen LJ, van de Belt TH. "What Is eHealth": time for an update? JMIR Res Protoc. Mar 12, 2015;4(1):e29. [FREE Full text] [CrossRef] [Medline]
    14. Eysenbach G. What is e-health? J Med Internet Res. 2001;3(2):E20. [FREE Full text] [CrossRef] [Medline]
    15. Bhattarai P, Phillips JL. The role of digital health technologies in management of pain in older people: An integrative review. Arch Gerontol Geriatr. Jan 2017;68:14-24. [CrossRef] [Medline]
    16. Ponder M, Venkatraman V, Charalambous L, Ansah-Yeboah AA, Adil SM, Antezana LA, et al. Mobile health application for patients undergoing breast cancer surgery: feasibility study. JCO Oncol Pract. Oct 2021;17(9):e1344-e1353. [CrossRef] [Medline]
    17. Jongerius C, Russo S, Mazzocco K, Pravettoni G. Research-tested mobile apps for breast cancer care: systematic review. JMIR Mhealth Uhealth. Mar 11, 2019;7(2):e10930. [FREE Full text] [CrossRef] [Medline]
    18. Ranney ML, Duarte C, Baird J, Patry EJ, Green TC. Correlation of digital health use and chronic pain coping strategies. Mhealth. Sep 06, 2016;2:35-35. [FREE Full text] [CrossRef] [Medline]
    19. Zhu J, Ebert L, Liu X, Wei D, Chan SW. Mobile breast cancer e-support program for Chinese women with breast cancer undergoing chemotherapy (part 2): multicenter randomized controlled trial. JMIR Mhealth Uhealth. May 30, 2018;6(4):e104. [FREE Full text] [CrossRef] [Medline]
    20. Maguire R, McCann L, Kotronoulas G, Kearney N, Ream E, Armes J, et al. Real time remote symptom monitoring during chemotherapy for cancer: European multicentre randomised controlled trial (eSMART). BMJ. Jul 21, 2021;374:n1647. [FREE Full text] [CrossRef] [Medline]
    21. Singleton AC, Raeside R, Hyun KK, Partridge SR, Di Tanna GL, Hafiz N, et al. Electronic health interventions for patients with breast cancer: systematic review and meta-analyses. JCO. Jul 10, 2022;40(20):2257-2270. [CrossRef]
    22. Lamper C, Huijnen I, de Mooij M, Köke A, Verbunt J, Kroese M. An ecoach-pain for patients with chronic musculoskeletal pain in interdisciplinary primary care: a feasibility study. Int J Environ Res Public Health. Dec 06, 2021;18(21):11661. [FREE Full text] [CrossRef] [Medline]
    23. Aapro M, Bossi P, Dasari A, Fallowfield L, Gascón P, Geller M, et al. Digital health for optimal supportive care in oncology: benefits, limits, and future perspectives. Support Care Cancer. Oct 12, 2020;28(10):4589-4612. [FREE Full text] [CrossRef] [Medline]
    24. De Groef A, Evenepoel M, Van Dijck S, Dams L, Haenen V, Wiles L, et al. Feasibility and pilot testing of a personalized eHealth intervention for pain science education and self-management for breast cancer survivors with persistent pain: a mixed-method study. Support Care Cancer. Jan 16, 2023;31(2):119. [CrossRef] [Medline]
    25. Somers TJ, Kelleher SA, Dorfman CS, Shelby RA, Fisher HM, Rowe Nichols K, et al. An mHealth pain coping skills training intervention for hematopoietic stem cell transplantation patients: development and pilot randomized controlled trial. JMIR Mhealth Uhealth. Mar 19, 2018;6(3):e66. [FREE Full text] [CrossRef] [Medline]
    26. Hauser-Ulrich S, Künzli H, Meier-Peterhans D, Kowatsch T. A smartphone-based health care chatbot to promote self-management of chronic pain (SELMA): pilot randomized controlled trial. JMIR Mhealth Uhealth. Apr 03, 2020;8(4):e15806. [FREE Full text] [CrossRef] [Medline]
    27. Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. Apr 12, 2017;4(4):CD001431. [FREE Full text] [CrossRef] [Medline]
    28. Gorini A, Masiero M, Pravettoni G. Patient decision aids for prevention and treatment of cancer diseases: are they really personalised tools? Eur J Cancer Care (Engl). Nov 09, 2016;25(6):936-960. [CrossRef] [Medline]
    29. Kondylakis H, Bucur A, Dong F, Renzi C, Manfrinati A, Graf N, et al. iManageCancer: Developing a Platform for Empowering Patients and Strengthening Self-Management in Cancer Diseases. Presented at: IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS); June 22-24, 2017, 2017; Thessaloniki, Greece. [CrossRef]
    30. Col N, Hull S, Springmann V, Ngo L, Merritt E, Gold S, et al. Improving patient-provider communication about chronic pain: development and feasibility testing of a shared decision-making tool. BMC Med Inform Decis Mak. Oct 17, 2020;20(1):267. [FREE Full text] [CrossRef] [Medline]
    31. Yung A, Kay J, Beale P, Gibson KA, Shaw T. Computer-based decision tools for shared therapeutic decision-making in oncology: systematic review. JMIR Cancer. Oct 26, 2021;7(4):e31616. [FREE Full text] [CrossRef] [Medline]
    32. Lucchiari C, Masiero M, Veronesi G, Maisonneuve P, Spina S, Jemos C, et al. Benefits of e-cigarettes among heavy smokers undergoing a lung cancer screening program: randomized controlled trial protocol. JMIR Res Protoc. Feb 03, 2016;5(1):e21. [FREE Full text] [CrossRef] [Medline]
    33. Penedo FJ, Oswald LB, Kronenfeld JP, Garcia SF, Cella D, Yanez B. The increasing value of eHealth in the delivery of patient-centred cancer care. The Lancet Oncology. May 2020;21(5):e240-e251. [CrossRef]
    34. Andersen KG, Kehlet H. Persistent pain after breast cancer treatment: a critical review of risk factors and strategies for prevention. J Pain. Jul 2011;12(7):725-746. [FREE Full text] [CrossRef] [Medline]
    35. Jensen MP, McFarland CA. Increasing the reliability and validity of pain intensity measurement in chronic pain patients. Pain. Dec 1993;55(2):195-203. [CrossRef] [Medline]
    36. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth. Mar 11, 2015;3(1):e27. [FREE Full text] [CrossRef] [Medline]
    37. Domnich A, Arata L, Amicizia D, Signori A, Patrick B, Stoyanov S, et al. Development and validation of the Italian version of the Mobile Application Rating Scale and its generalisability to apps targeting primary prevention. BMC Med Inform Decis Mak. Jul 07, 2016;16(1):83. [FREE Full text] [CrossRef] [Medline]
    38. Terhorst Y, Philippi P, Sander LB, Schultchen D, Paganini S, Bardus M, et al. Validation of the Mobile Application Rating Scale (MARS). PLoS One. Nov 2, 2020;15(11):e0241480. [FREE Full text] [CrossRef] [Medline]
    39. Nicholas MK. Self-efficacy and chronic pain. Presented at: Annual Conference of the British Psychological Society; October 1989, 1989; Bournemouth, England.
    40. Chiarotto A, Vanti C, Ostelo RW, Ferrari S, Tedesco G, Rocca B, et al. The Pain Self-Efficacy Questionnaire: cross-cultural adaptation into Italian and assessment of its measurement properties. Pain Pract. Dec 27, 2015;15(8):738-747. [CrossRef] [Medline]
    41. Kriston L, Scholl I, Hölzel L, Simon D, Loh A, Härter M. The 9-item Shared Decision Making Questionnaire (SDM-Q-9). Development and psychometric properties in a primary care sample. Patient Educ Couns. Jul 2010;80(1):94-99. [CrossRef] [Medline]
    42. Goss C, Ghilardi A, Deledda G, Buizza C, Bottacini A, Del Piccolo L, et al. INvolvement of breast CAncer patients during oncological consultations: a multicentre randomised controlled trial—the INCA study protocol. BMJ Open. May 02, 2013;3(5):e002266. [FREE Full text] [CrossRef] [Medline]
    43. Salazar A, de Sola H, Failde I, Moral-Munoz JA. Measuring the quality of mobile apps for the management of pain: systematic search and evaluation using the Mobile App Rating Scale. JMIR Mhealth Uhealth. Oct 25, 2018;6(10):e10718. [FREE Full text] [CrossRef] [Medline]
    44. Pas R, Leysen L, De Goeij W, Vossebeld L, Van Wilgen P, De Groef A, et al. Pain neuroscience education in cancer survivors with persistent pain: A pilot study. J Bodyw Mov Ther. Oct 2020;24(4):239-244. [CrossRef] [Medline]
    45. Karademas EC, Simos P, Pat-Horenczyk R, Roziner I, Mazzocco K, Sousa B, et al. Cognitive, emotional, and behavioral mediators of the impact of coping self-efficacy on adaptation to breast cancer: An international prospective study. Psychooncology. Oct 24, 2021;30(9):1555-1562. [CrossRef] [Medline]
    46. Karademas EC, Roziner I, Mazzocco K, Pat-Horenczyk R, Sousa B, Oliveira-Maia AJ, et al. The mutual determination of self-efficacy to cope with cancer and cancer-related coping over time: a prospective study in women with breast cancer. Psychol Health. Feb 11, 2023;38(12):1635-1648. [CrossRef] [Medline]
    47. Vinnikova A, Lu L, Wei J, Fang G, Yan J. The use of smartphone fitness applications: the role of self-efficacy and self-regulation. Int J Environ Res Public Health. Oct 20, 2020;17(20):7639. [FREE Full text] [CrossRef] [Medline]
    48. Durosier Mertilus DS, Lengacher CA, Rodriguez CS. A review and conceptual analysis of cancer pain self-management. Pain Manag Nurs. Apr 2022;23(2):168-173. [CrossRef] [Medline]
    49. Pilot Study Sample Size Rules of Thumb. NCSS. 2018. URL: https:/​/www.​ncss.com/​wp-content/​themes/​ncss/​pdf/​Procedures/​PASS/​Pilot_Study_Sample_Size_Rules_of_Thumb.​pdf [accessed 2023-12-31]
    50. Julious SA. Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics. Nov 24, 2005;4(4):287-291. [CrossRef]
    51. Schmettow M. Sample size in usability studies. Commun. ACM. Apr 01, 2012;55(4):64-70. [CrossRef]
    52. Cohen J. Statistical Power for the Behavioral Sciences, 2nd Edition. Abingdon, Oxfordshire, United Kingdom. Routledge; 1988.

    ‎
    ASyMS: Advanced Symptom Management System
    FHIR: Fast Healthcare Interoperability Resources
    IEO: European Institute of Oncology
    MARS: Mobile Application Rating Scale
    NRS: numeric rating scale
    PECAN: Pain Education after Cancer Collaborative
    PSEQ: Pain Self-Efficacy Questionnaire
    QoL: quality of life
    SDM-Q-9: 9-item Shared Decision-Making Questionnaire

    Edited by A Mavragani; submitted 20.07.23; peer-reviewed by T Tabashum, T Wang; comments to author 15.09.23; revised version received 06.10.23; accepted 23.10.23; published 02.02.24.

    Copyright

    ©Marianna Masiero, Chiara Filipponi, Elisa Fragale, Silvia Francesca Maria Pizzoli, Elisabetta Munzone, Alessandra Milani, Luca Guido, Vittorio Guardamagna, Sara Marceglia, Roberto Prandin, Marco Prenassi, Annamaria Caruso, Vania Manzelli, Chiara Savino, Costanza Conti, Federica Rizzi, Alice Casalino, Giulia Candiani, Francesca Memini, Luca Chiveri, Andrea Luigi Vitali, Massimo Corbo, Roberto Grasso, Florence Didier, Roberta Ferrucci, Gabriella Pravettoni. Originally published in JMIR Formative Research (https://formative.jmir.org), 02.02.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.