TY - JOUR AU - Zerrouk, Amine AU - Migchels, Charlotte AU - De Ruysscher, Clara AU - Fernandez, Kim AU - Antoine, Jerome AU - De Meyer, Florian AU - Matthys, Frieda AU - van den Brink, Wim AU - Crunelle, Lina Cleo AU - Vanderplasschen, Wouter PY - 2025/4/30 TI - Incorporating Patient-Reported Outcome Measures and Patient-Reported Experience Measures in Addiction Treatment Services in Belgium: Naturalistic, Longitudinal, Multicenter Cohort Study JO - JMIR Form Res SP - e65686 VL - 9 KW - patient-reported outcome measures KW - patient-reported experience measures KW - substance use disorder KW - recovery KW - ICHOM KW - International Consortium for Health Outcomes Measurement KW - addiction KW - PROMs KW - PREMs KW - SUD KW - treatment KW - protocol KW - substance use KW - inpatient KW - services KW - perspectives KW - treatment outcome N2 - Background: Traditionally, treatment outcomes of service users with a substance use disorder (SUD) are measured using objective and provider-reported indicators. In recent years, there has been a shift toward incorporating patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) to capture service users? perspectives on treatment outcomes and experiences. Objective: The OMER-BE (Outcome Measurement and Evaluation as a Routine Practice in Alcohol and Other Drug Services in Belgium) study evaluates the acceptability and feasibility of PROMs and PREMs in different SUD treatment services, using the recently developed International Consortium for Health Outcomes Measurement Standard Set for Addictions. This paper presents the design and baseline characteristics of the study, indicators of attrition at 45-day follow-up, and the feasibility of the implementation of PROMs and PREMs in residential and outpatient services. Methods: A convenience sample of 189 treatment-seeking individuals with SUD from different inpatient (therapeutic communities and psychiatric centers) and outpatient treatment services was followed for six months. Sociodemographic characteristics; clinical factors; and PROMs including recovery strengths, quality of life, and global health were assessed at baseline and within 3 weeks after starting treatment. Additionally, PROMs and PREMs were measured 45, 90, and 180 days later. Comparisons were made between treatment modalities, and indicators of attrition at the 45-day follow-up were assessed using ANOVA and chi-square tests. Results: Baseline differences were observed between the three treatment modalities regarding education, SUD treatment history, primary substance, and Attention-Deficit/Hyperactivity Disorder Self-Report scores. Overall, patients in psychiatric treatment centers had a higher education level and less polysubstance use, while outpatients had fewer previous SUD treatments but received relatively more often opioid agonist treatment. Inpatients reported more attention-deficit/hyperactivity disorder symptoms and higher SUD severity than outpatients. Additionally, recovery strength scores were significantly lower in the outpatient group compared to the other groups, particularly in the subdomains of ?Substance Use,? ?Self-care,? and ?Outlook on Life.? At the 45-day follow-up assessment, the attrition rate was 36.6%. Comparisons between participants who completed the 45-day follow-up and those who dropped out revealed that completers were significantly older, had a higher level of education, were more likely to live alone, and were more likely to have a mother born in Belgium. They also had higher average scores on the ?Material Resources? domain of the Substance Use Recovery Evaluator, which includes questions about stable housing, a steady income, and effective financial management. Conclusions: Evaluating PROMs and PREMs appears to be feasible in a diverse group of treatment-seeking patients with SUD in Belgium. However, challenges remain for structural implementation in practice, especially in outpatient services. Routine monitoring of PROMs and PREMs has the potential to empower patients, service providers, and policy makers by providing a comprehensive understanding of service users? needs and treatment effectiveness. UR - https://formative.jmir.org/2025/1/e65686 UR - http://dx.doi.org/10.2196/65686 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65686 ER - TY - JOUR AU - Schubert, Cicero Marc AU - Soyka, Stella AU - Wick, Wolfgang AU - Venkataramani, Varun PY - 2025/4/24 TI - Guideline-Incorporated Large Language Model-Driven Evaluation of Medical Records Using MedCheckLLM JO - JMIR Form Res SP - e53335 VL - 9 KW - large language models KW - AI KW - electron medical records KW - checklists KW - LLM KW - language model KW - NLP KW - natural language processing KW - records KW - documentation KW - documents KW - framework KW - conceptual KW - machine learning KW - artificial intelligence KW - evidence KW - evaluate KW - evaluation KW - guideline KW - health care N2 - Abstract: The study introduces MedCheckLLM, a large language model?driven framework that enhances medical record evaluation through a guideline-in-the-loop approach by integrating evidence-based guidelines. UR - https://formative.jmir.org/2025/1/e53335 UR - http://dx.doi.org/10.2196/53335 ID - info:doi/10.2196/53335 ER - TY - JOUR AU - Altweck, Laura AU - Tomczyk, Samuel PY - 2025/4/23 TI - Ecological Momentary Assessment of Parental Well-Being and Time Use: Mixed Methods Compliance and Feasibility Study JO - JMIR Form Res SP - e67451 VL - 9 KW - ecological momentary assessment KW - parents KW - feasibility KW - compliance KW - time use KW - well-being KW - stress KW - EMA KW - mixed methods KW - daily life KW - online questionnaires KW - questionnaires KW - surveys KW - quantitative KW - qualitative KW - sociodemographic KW - mobile phone N2 - Background: Parents often juggle multiple conflicting responsibilities, including work, childcare, and the household, making them a particularly burdened group. However, the impact of daily routines and associated (poor) well-being among parents has received relatively little attention. Ecological Momentary Assessment (EMA) is increasingly being used to capture real-time data and can help address this research gap. Objective: This study aims to examine compliance rates and the feasibility of EMA for measuring daily well-being and time use among parents. Methods: An exploratory mixed-methods study was conducted with 74 German parents (57/74, 77% women, (age: mean 37.6, SD 5.9 years). Participants completed a baseline questionnaire, followed by 4 daily EMA surveys (at 7:30 AM, 12 PM, 16:30 PM, and 21:30 PM) over a 1-week period, and a follow-up questionnaire. A subset of parents was also subsequently interviewed. Sociodemographic background and expected feasibility (open-ended questions) were surveyed at baseline, and feasibility was assessed at follow-up (closed- and open-ended questions) and in the interviews. State well-being (affective and cognitive), state stress, state as well as retrospective time-use were measured in the EMA surveys. Compliance and feasibility were examined using a combination of quantitative (descriptive analyses) and qualitative methodologies. Results: Participants completed an average of 83% (SD 13%) of the daily surveys. Compliance varied by gender and age, where men (90% vs 80%) and older parents showed higher rates. Participants generally found the survey frequency and length manageable, though some suggested adjustments to the study period depending on their individual routines. The 7:30 AM survey was reported as the most challenging due to childcare drop-offs (40%-49%), followed by the 16:30 PM survey for similar reasons (7%-17%). The qualitative analysis further revealed additional points for improvement, for instance, the need for personalization (eg, individual adjustment of the survey timings and intervals), technical support, and the incorporation of gamification elements. Most interviewees (46% vs 23%) found the used measurement of well-being and stress to be appropriate. Regarding time use, they felt that the predefined activity groups (eg, personal care, working) were suitable (46%) but noted challenges assigning less frequent activities (eg, medical appointments) (5%-54%). Reporting the timings of time-use via consecutive questions (ie, specifying the duration or start and end times of an activity) was perceived as confusing (9%-69%), with participants expressing a preference for a visual overview, such as a Gantt chart. Conclusions: The study demonstrates that, when accounting for certain sociodemographic and study design factors, EMA can be a feasible method for data collection regarding daily well-being and time use, even in highly time-constrained populations like parents. This shows great potential for future research, such as exploring work-family conflict or performative gender roles and complementing established methods (eg, retrospective daily diaries). Trial Registration: OSF Registries osf.io/8qj3d; https://osf.io/8qj3d International Registered Report Identifier (IRRID): RR2-10.2196/54728 UR - https://formative.jmir.org/2025/1/e67451 UR - http://dx.doi.org/10.2196/67451 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67451 ER - TY - JOUR AU - Slotkin, Rebecca AU - Kyriakides, C. Tassos AU - Yu, Vinni AU - Chen, Xien AU - Kundu, Anupam AU - Gupta, Shaili PY - 2025/4/22 TI - CoVimmune COVID-19 Immunity Calculator: Web Application Development and Validation Study JO - JMIR Form Res SP - e59467 VL - 9 KW - COVID-19 KW - immunity KW - neutralizing antibody KW - immunoglobulin G KW - vaccine hesitancy KW - vaccine timing KW - patient-centered care KW - web application KW - vaccination KW - SARS-CoV-2 N2 - Background: This study illustrates the development of a simple web-based application, which demonstrates the relationship between serum anti-SARS-CoV-2 S1/receptor-binding domain immunoglobulin G (IgG) and anti-SARS-CoV-2 neutralizing antibody (nAb) half-maximal inhibitory concentration (IC50) titers in a vaccinated US adult population and compares them to prior data on nAb titers at different time points after vaccination. Objective: The objective of this study is to create an easily accessible calculator that uses the results of commercially available anti-SARS-CoV-2 serum IgG to approximate the underlying ability to neutralize SARS-CoV-2. Methods: Our web-based application leveraged two previously published datasets. One dataset demonstrated a robust correlation between nAb and serum IgG. The other dataset measured nAb titers at specific time periods over a year-long interval following a messenger RNA vaccination primary series and booster vaccine dose. Clinical factors that were statistically significant on a forward linear regression model examining the prediction of nAb from serum IgG were incorporated in the application tool. Results: By combining the datasets described above, we developed a publicly available web-based application that allows users to enter a serum IgG value and determine their estimated nAb titer. The application contextualizes the estimated nAb titer with the theoretical distance from the corresponding vaccine-mediated antibody protection. Using the clinical variables that had a significant impact on how well IgG values predict nAb titers, this application allows for a patient-centered, nAb titer prediction. Conclusions: This application offers an example of how we might bring the advances made in scientific research on protective antibodies post-SARS-CoV-2 vaccination into the clinical sphere with practical tools. UR - https://formative.jmir.org/2025/1/e59467 UR - http://dx.doi.org/10.2196/59467 ID - info:doi/10.2196/59467 ER - TY - JOUR AU - O'Laughlin, D. Kristine AU - Cheng, Haugan Britte AU - Volponi, J. Joshua AU - Lorentz, A. John David AU - Obregon, A. Sophia AU - Younger, Wise Jessica AU - Gazzaley, Adam AU - Uncapher, R. Melina AU - Anguera, A. Joaquin PY - 2025/4/21 TI - Validation of an Adaptive Assessment of Executive Functions (Adaptive Cognitive Evaluation-Explorer): Longitudinal and Cross-Sectional Analyses of Cognitive Task Performance JO - J Med Internet Res SP - e60041 VL - 27 KW - executive functions KW - serious games KW - validation KW - computerized assessment KW - cognitive assessment N2 - Background: Executive functions (EFs) predict positive life outcomes and educational attainment. Consequently, it is imperative that our measures of EF constructs are both reliable and valid, with advantages for research tools that offer efficiency and remote capabilities. Objective: The objective of this study was to evaluate reliability and validity evidence for a mobile, adaptive measure of EFs called Adaptive Cognitive Evaluation-Explorer (ACE-X). Methods: We collected data from 2 cohorts of participants: a test-retest sample (N=246, age: mean 35.75, SD 11.74 y) to assess consistency of ACE-X task performance over repeated administrations and a validation sample involving child or adolescent (5436/6052, 89.82%; age: mean 12.78, SD 1.60 years) and adult participants (484/6052, 8%; age: mean 38.11, SD 14.96 years) to examine consistency of metrics, internal structures, and invariance of ACE-X task performance. A subset of participants (132/6052, 2.18%; age: mean 37.04, SD 13.23 years) also completed a similar set of cognitive tasks using the Inquisit platform to assess the concurrent validity of ACE-X. Results: Intraclass correlation coefficients revealed most ACE-X tasks were moderately to very reliable across repeated assessments (intraclass correlation coefficient=0.45-0.79; P<.001). Moreover, in comparisons of internal structures of ACE-X task performance, model fit indices suggested that a network model based on partial correlations was the best fit to the data (?228=40.13; P=.06; comparative fit index=0.99; root mean square error of approximation=0.03, 90% CI 0.00-0.05; Bayesian information criterion=5075.87; Akaike information criterion=4917.71) and that network edge weights are invariant across both younger and older adult participants. A Spinglass community detection algorithm suggested ACE-X task performance can be described by 3 communities (selected in 85% of replications): set reconfiguration, attentional control, and interference resolution. On the other hand, Pearson correlation coefficients indicated mixed results for the concurrent validity comparisons between ACE-X and Inquisit (r=?.05-.62, P<.001-.76). Conclusions: These findings suggest that ACE-X is a reliable and valid research tool for understanding EFs and their relations to outcome measures. UR - https://www.jmir.org/2025/1/e60041 UR - http://dx.doi.org/10.2196/60041 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60041 ER - TY - JOUR AU - Olaussen, Camilla AU - Stojiljkovic, Marko AU - Zlamal, Jaroslav AU - Flølo, Nygaard Tone AU - Nes, Gonçalves Andréa Aparecida PY - 2025/4/21 TI - The Norwegian Version of the Self-Efficacy in Clinical Performance Scale (SECP): Psychometric Validation Study JO - JMIR Form Res SP - e68173 VL - 9 KW - clinical performance KW - self-efficacy KW - instrument validation KW - nursing education KW - psychometric analysis KW - Norway KW - psychometric KW - validation study KW - competence KW - clinical practice KW - translate KW - translation KW - cross-sectional study KW - nursing student KW - reliability N2 - Background: Previous research has demonstrated a correlation between nursing students? self-efficacy and their clinical performance, competence, and behavior during clinical practice placements. Assessing students? self-efficacy in clinical performance could be a valuable method for identifying areas that need reinforcement and for recognizing students who may require additional support during clinical practice placements. Objective: This study aimed to translate the Self-Efficacy in Clinical Performance Scale (SECP) from English into Norwegian and to evaluate the psychometric properties of the Norwegian version. Methods: A cross-sectional study design was used. The SECP was translated into Norwegian following a 6-step process: forward translation, forward translation synthesis, backward translation, backward translation synthesis, cognitive debriefing, and psychometric testing. The validity and reliability of the translated version were assessed using confirmatory factor analysis (CFA), Cronbach ?, McDonald ?, and composite reliability. Results: A total of 399 nursing students completed the Norwegian version of the SECP. The CFA goodness-of-fit indices (?2/df ratio=1.578, comparative fit index=0.98, Tucker-Lewis index=0.98, standardized root mean square residual=0.056, root mean square error of approximation=0.038) indicated an acceptable model fit. Reliability measures, including Cronbach ?, McDonald ?, and composite reliability, were high, with factor-level values ranging from 0.94 to 0.98. Conclusion: The Norwegian version of the SECP demonstrated strong potential as an instrument for assessing self-efficacy in both current and required competencies among nursing students in clinical practice within nursing education. Future research should aim to confirm the factor structure of the SECP and evaluate its test-retest reliability. UR - https://formative.jmir.org/2025/1/e68173 UR - http://dx.doi.org/10.2196/68173 ID - info:doi/10.2196/68173 ER - TY - JOUR AU - Ogunsanmi, Deborah AU - Chambers, Jerica AU - Mahmood, Asos AU - Pakker, Reddy Avinash AU - Kompalli, Anusha AU - Kabir, Umar AU - Surbhi, Satya AU - Gatwood, Justin AU - Mahmud, Sultan Md AU - Bailey, E. James PY - 2025/4/18 TI - Technical Requirements, Design, and Automation Process for a Statewide Registry-Based Tailored Text Messaging System: Protocol for a Longitudinal Observational Study JO - JMIR Res Protoc SP - e62874 VL - 14 KW - text messaging KW - clinical informatics KW - chronic disease KW - telehealth KW - telemedicine KW - electronic health records N2 - Background: Tailored text messaging is a low-cost mobile health intervention approach shown to effectively improve self-care behaviors and clinical outcomes for patients with chronic cardiometabolic conditions. Given the ubiquitous nature of mobile phones, text messages have the potential to reach a large audience. However, automating and disseminating tailored text messages to large populations at low cost presents major logistical challenges that serve as barriers to implementation. Objective: This study aimed to describe the protocol for a longitudinal observational study designed to assess the feasibility of an innovative approach for automating and disseminating personalized and tailored text messages to large populations at risk of cardiovascular events using a low-cost registry-based tailored text messaging system known as the Heart Health Messages (HHM) program. Further, it describes the technical requirements, architectural design, automation process, and challenges associated with program implementation. Methods: Patients at high risk of cardiovascular diseases are identified using a statewide population health registry known as the Tennessee Population Data Network. Tailored invitation messages and enrollment surveys are sent to eligible patients via Twilio. Upon completion of the receipt of consent and enrollment forms, participants receive tailored text messages from a library of generic messages based on participant-selected frequency of message delivery (daily or every other day). In addition, participants receive monthly text-based check-in survey messages designed to assess intervention adherence and improvement in self-care. Participants are also sent quarterly follow-up surveys to update enrollment information and preferences. All enrolled participants will receive tailored text messages for a 12-month intervention period. Results: Since the start of the program, 18,974 patients from 2 major health systems have met the inclusion criteria and were eligible for the HHM program. A total of 3 phases of HHM 1.0 have been implemented so far, reaching 225 eligible patients in phase 1, a total of 5288 patients in phase 2, and 13,461 patients in phase 3, with an enrollment of approximately 2% (n=4/225), 3% (n=137/5228), and 3% (n=350/13461), respectively. Efforts are underway to implement strategies in collaboration with the health systems to enhance the HHM program rollout and patient participation. Conclusions: The HHM program is a low-cost tailored text messaging intervention set for broader dissemination and potential replication. The program has the capacity to improve outcomes for people with chronic medical conditions. International Registered Report Identifier (IRRID): DERR1-10.2196/62874 UR - https://www.researchprotocols.org/2025/1/e62874 UR - http://dx.doi.org/10.2196/62874 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/62874 ER - TY - JOUR AU - Buchan, Claire M. AU - Katapally, Reddy Tarun AU - Bhawra, Jasmin PY - 2025/4/17 TI - Application of an Innovative Methodology to Build Infrastructure for Digital Transformation of Health Systems: Developmental Program Evaluation JO - JMIR Form Res SP - e53339 VL - 9 KW - digital health platform KW - citizen science KW - evaluation KW - health systems KW - digital health KW - app KW - innovative KW - digital transformation KW - public health KW - crises KW - communicable disease KW - coronavirus KW - chronic diseases KW - decision-making KW - assessment KW - thematic analysis KW - self-report survey KW - risk KW - artificial intelligence KW - AI N2 - Background: The current public health crises we face, including communicable disease pandemics such as COVID-19, require cohesive societal efforts to address decision-making gaps in our health systems. Digital health platforms that leverage big data ethically from citizens can transform health systems by enabling real-time data collection, communication, and rapid responses. However, the lack of standardized and evidence-based methods to develop and implement digital health platforms currently limits their application. Objective: This study aims to apply mixed evaluation methods to assess the development of a rapid response COVID-19 digital health platform before public launch by engaging with the development and research team, which consists of interdisciplinary researchers (ie, key stakeholders). Methods: Using a developmental evaluation approach, this study conducted (1) a qualitative survey assessing digital health platform objectives, modifications, and challenges administered to 5 key members of the software development team and (2) a role-play pilot with 7 key stakeholders who simulated 8 real-world users, followed by a self-report survey, to evaluate the utility of the digital health platform for each of its objectives. Survey data were analyzed using an inductive thematic analysis approach. Postpilot test survey data were aggregated and synthesized by participant role. Results: The digital health platform met original objectives and was expanded to accommodate the evolving needs of potential users and COVID-19 pandemic regulations. Key challenges noted by the development team included navigating changing government policies and supporting the data sovereignty of platform users. Strong team cohesion and problem-solving were essential in the overall success of program development. During the pilot test, participants reported positive experiences interacting with the platform and found its features relatively easy to use. Users in the community member role felt that the platform accurately reflected their risk of contracting COVID-19, but reported some challenges interacting with the interface. Those in the decision maker role found the data visualizations helpful for understanding complex information. Both participant groups highlighted the utility of a tutorial for future users. Conclusions: Evaluation of the digital health platform development process informed our decisions to integrate the research team more cohesively with the development team, a practice that is currently uncommon given the use of external technology vendors in health research. In the short term, the developmental evaluation resulted in shorter sprints, and the role-play exercise enabled improvements to the log-in process and user interface ahead of public deployment. In the long term, this exercise informed the decision to include a data scientist as part of both teams going forward to liaise with researchers throughout the development process. More interdisciplinarity was also integrated into the research process by providing health system training to computer programmers, a key factor in human-centered artificial intelligence development. UR - https://formative.jmir.org/2025/1/e53339 UR - http://dx.doi.org/10.2196/53339 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53339 ER - TY - JOUR AU - Draugelis, Sarah AU - Hunnewell, Jessica AU - Bishop, Sam AU - Goswami, Reena AU - Smith, G. Sean AU - Sutherland, Philip AU - Hickman, Justin AU - Donahue, A. Donald AU - Yendewa, A. George AU - Mohareb, M. Amir PY - 2025/4/17 TI - Leveraging Electronic Health Records in International Humanitarian Clinics for Population Health Research: Cross-Sectional Study JO - JMIR Public Health Surveill SP - e66223 VL - 11 KW - refugee KW - population health KW - disaster medicine KW - humanitarian clinic KW - electronic health record KW - Fast Electronic Medical Record KW - fEMR N2 - Background: As more humanitarian relief organizations are beginning to use electronic medical records in their operations, data from clinical encounters can be leveraged for public health planning. Currently, medical data from humanitarian medical workers are infrequently available in a format that can be analyzed, interpreted, and used for public health. Objectives: This study aims to develop and test a methodology by which diagnosis and procedure codes can be derived from free-text medical encounters by medical relief practitioners for the purposes of data analysis. Methods: We conducted a cross-sectional study of clinical encounters from humanitarian clinics for displaced persons in Mexico between August 3, 2021, and December 5, 2022. We developed and tested a method by which free-text encounters were reviewed by medical billing coders and assigned codes from the International Classification of Diseases, Tenth Revision (ICD-10) and the Current Procedural Terminology (CPT). Each encounter was independently reviewed in duplicate and assigned ICD-10 and CPT codes in a blinded manner. Encounters with discordant codes were reviewed and arbitrated by a more experienced medical coder, whose decision was used to determine the final ICD-10 and CPT codes. We used chi-square tests of independence to compare the ICD-10 codes for concordance across single-diagnosis and multidiagnosis encounters and across patient characteristics, such as age, sex, and country of origin. Results: We analyzed 8460 encounters representing 5623 unique patients and 2774 unique diagnosis codes. These free-text encounters had a mean of 20.5 words per encounter in the clinical documentation. There were 58.78% (4973/8460) encounters where both coders assigned 1 diagnosis code, 18.56% (1570/8460) encounters where both coders assigned multiple diagnosis codes, and 22.66% (1917/8460) encounters with a mixed number of codes assigned. Of the 4973 encounters with a single code, only 11.82% (n=588) had a unique diagnosis assigned by the arbitrator that was not assigned by either of the initial 2 coders. Of the 1570 encounters with multiple diagnosis codes, only 3.38% (n=53) had unique diagnosis codes assigned by the arbitrator that were not initially assigned by either coder. The frequency of complete concordance across diagnosis codes was similar across sex categories and ranged from 30.43% to 46.05% across age groups and countries of origin. Conclusions: Free-text electronic medical records from humanitarian relief clinics can be used to develop a database of diagnosis and procedure codes. The method developed in this study, which used multiple independent reviews of clinical encounters, appears to reliably assign diagnosis codes across a diverse patient population in a resource-limited setting. UR - https://publichealth.jmir.org/2025/1/e66223 UR - http://dx.doi.org/10.2196/66223 ID - info:doi/10.2196/66223 ER - TY - JOUR AU - Chevalier, Aline AU - Dosso, Cheyenne PY - 2025/4/17 TI - The Influence of Medical Expertise and Information Search Skills on Medical Information Searching: Comparative Analysis From a Free Data Set JO - JMIR Form Res SP - e62754 VL - 9 KW - information searching KW - credibility KW - internet KW - medicine KW - information search skills N2 - Background: Nowadays, the internet has become the primary source of information for physicians seeking answers to medical questions about their patients before consulting colleagues. However, many websites provide low-quality, unreliable information that lacks scientific validation. Therefore, physicians must develop strong information search skills to locate relevant, accurate, and evidence-based content. However, previous studies have shown that physicians often have poor search skills and struggle to find information on the web, which may have detrimental consequences for patient care. Objective: This study aims to determine how medical students and residents searched for medical information on the internet, the quality of the web resources they used (including their nature and credibility), and how they evaluated the reliability of these resources and the answers they provided. Given the importance of domain knowledge (in this case, medicine) and information search skills in the search process, we compared the search behaviors of medical students and residents with those of computer science students. While medical students and residents possess greater medical-related knowledge, computer science students have stronger information search skills. Methods: A total of 20 students participated in this study: 10 medical students and residents, and 10 computer science students. Data were extracted from a freely accessible data set in accordance with FAIR (Findable, Accessible, Interoperable, and Reusable) principles. All participants searched for medical information online to make a diagnosis, select a treatment, and enhance their knowledge of a medical condition?3 primary activities they commonly perform. We analyzed search performance metrics, including search time, the use of medical-related keywords, and the accuracy of the information found, as well as the nature and credibility of web resources used by medical students and residents compared with computer science students. Results: Medical students and residents provided more accurate answers than computer science students without requiring additional time. Their medical expertise also enabled them to better assess the reliability of resources and select high-quality web sources, primarily from hospital websites. However, it is noteworthy that they made limited use of evidence-based tools such as PubMed. Conclusions: Although medical students and residents generally outperformed computer science students, they did not frequently use evidence-based tools. As previously observed, they may avoid databases due to the risk of encountering too many irrelevant articles and difficulties in applying appropriate filters to locate relevant information. Nevertheless, clinical and practical evidence-based medicine plays a crucial role in updating physicians? knowledge, improving patient care, and enhancing physician-patient relationships. Therefore, information search skills should be an integral part of medical education and continuing professional development for physicians. UR - https://formative.jmir.org/2025/1/e62754 UR - http://dx.doi.org/10.2196/62754 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/62754 ER - TY - JOUR AU - Harry, Christiana AU - Goodday, Sarah AU - Chapman, Carol AU - Karlin, Emma AU - Damian, Joy April AU - Brooks, Alexa AU - Boch, Adrien AU - Lugo, Nelly AU - McMillan, Rebecca AU - Tempero, Jonell AU - Swanson, Ella AU - Peabody, Shannon AU - McKenzie, Diane AU - Friend, Stephen PY - 2025/4/15 TI - Using Social Media to Engage and Enroll Underrepresented Populations: Longitudinal Digital Health Research JO - JMIR Form Res SP - e68093 VL - 9 KW - digital health research KW - digital health technology KW - recruitment KW - research subject KW - participant KW - pregnancy KW - maternal health KW - underrepresented populations KW - health equity KW - diversity KW - marginalized KW - advertisement KW - social media KW - retention KW - attrition KW - dropout N2 - Background: Emerging digital health research poses roadblocks to the inclusion of historically marginalized populations in research. Exclusion of underresourced communities in digital health research is a result of multiple factors (eg, limited technology access, decreased digital literacy, language barriers, and historical mistrust of research and research institutions). Alternative methods of access and engagement may aid in achieving long-term sustainability of diversified participation in digital health research, ensuring that developed technologies and research outcomes are effective and equitable. Objective: This study aims to (1) characterize socioeconomic and demographic differences in individuals who enrolled and engaged with different remote, digital, and traditional recruitment methods in a digital health pregnancy study and (2) determine whether social media outreach is an efficient way of recruiting and retaining specific underrepresented populations (URPs) in digital health research. Methods: The Better Understanding the Metamorphosis of Pregnancy (BUMP) study was used as a case example. This is a prospective, observational, cohort study using digital health technology to increase understanding of pregnancy among 524 women, aged 18-40 years, in the United States. The study used different recruitment strategies: patient portal for genetic testing results, paid/unpaid social media ads, and a community health organization providing care to pregnant women (Moses/Weitzman Health System). Results: Social media as a recruitment tool to engage URPs in a digital health study was overall effective, with a 23.6% (140/594) enrollment rate of those completing study interest forms across 25 weeks. Community-based partnerships were less successful, however, resulting in 53.3% (57/107) engagement with recruitment material and only 8.8% (5/57) ultimately enrolling in the study. Paid social media ads provided access to and enrollment of a diverse potential participant pool of race- or ethnicity-based URPs in comparison to other digital recruitment channels. Of those that engaged with study materials, paid recruitment had the highest percentage of non-White (non-Hispanic) respondents (85/321, 26.5%), in comparison to unpaid ads (Facebook and Reddit; 37/167, 22.2%). Of the enrolled participants, paid ads also had the highest percentage of non-White (non-Hispanic) participants (14/70, 20%), compared to unpaid ads (8/52, 15.4%) and genetic testing service subscribers (72/384, 18.8%). Recruitment completed via paid ads (Instagram) had the highest study retention rate (52/70, 74.3%) across outreach methods, whereas recruitment via community-based partnerships had the lowest (2/5, 40%). Retention of non-White (non-Hispanic) participants was low across recruitment methods: paid (8/52, 15.4%), unpaid (3/35, 14.3%), and genetic testing service subscribers (50/281, 17.8%). Conclusions: Social media recruitment (paid/unpaid) provides access to URPs and facilitates sustained retention similar to other methods, but with varying strengths and weaknesses. URPs showed lower retention rates than their White counterparts across outreach methods. Community-based recruitment showed lower engagement, enrollment, and retention. These findings highlight social media?s potential for URP engagement and enrollment, illuminate potential roadblocks of traditional methods, and underscore the need for tailored research to improve URP enrollment and retention. UR - https://formative.jmir.org/2025/1/e68093 UR - http://dx.doi.org/10.2196/68093 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/68093 ER - TY - JOUR AU - Mahmood, Atiya AU - Rikhtehgaran, Farinaz AU - Nasiri, Rojan AU - Hedayati, Niloofar AU - Pandsheno, Sepehr AU - Sharrock, Aislynn AU - Mora, Juanita Diana AU - Haji Hosseini, Sogol AU - Routhier, François AU - Mortenson, W.Ben PY - 2025/4/10 TI - Adaptation of the Stakeholders? Walkability/Wheelability Audit in Neighborhoods (SWAN) Tool for Individuals With Diverse Disabilities: Protocol for a Mixed Methods Study JO - JMIR Res Protoc SP - e60553 VL - 14 KW - age and accessibility KW - disability experiences KW - community engaged research KW - inclusive urban design KW - user-led built environment audits N2 - Background: The prevalence of sensory, cognitive, and mobility disabilities in Canada underscores the need to address environmental barriers. This study adapts and validates the Stakeholders? Walkability/Wheelability Audit in Neighborhoods (SWAN) tool to assess the challenges the built environment poses for individuals with disabilities, aiming to inform policy changes for accessibility and inclusivity. Objective: This study aims to (1) adapt the SWAN tool for those with hearing, vision, or cognitive disabilities; (2) validate SWAN tool for researching environmental barriers for people with disabilities, including older adults; and (3) offer insights for policy changes in the built environment, contributing to literature and guiding future research. Methods: The study uses a community-based research approach, carried out over 4 phases within an 18-month period in British Columbia. Phase 1 includes adapting and pilot-testing of the SWAN tool. In Phase 2, street intersections are identified for data collection using Geographic Information System tools and consultations with municipal officials. Phase 3 involves recruiting participants across four disability categories. The final phase includes analyzing the data and disseminating findings. Results: Data collection concluded in September 2024, involving 80 eligible participants across four streams in preidentified hotspots. The results are expected to be published in March 2025. To date, data collection is ongoing, and we are currently in the process of data analysis. Conclusions: This study will contribute to the growing body of research on built environment accessibility by adapting the SWAN tool for individuals with diverse disabilities. By identifying key barriers in urban spaces, the study aims to inform policy changes that will lead to more inclusive, accessible, and safe urban environments for all individuals. UR - https://www.researchprotocols.org/2025/1/e60553 UR - http://dx.doi.org/10.2196/60553 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60553 ER - TY - JOUR AU - Coren, A. Morgan AU - Lindhiem, Oliver AU - Angus, R. Abby AU - Toevs, K. Emma AU - Radovic, Ana PY - 2025/4/10 TI - Provider Perspectives on Implementing an Enhanced Digital Screening for Adolescent Depression and Suicidality: Qualitative Study JO - JMIR Form Res SP - e67624 VL - 9 KW - depression KW - suicidality KW - adolescent mental health screening KW - primary care KW - digital tools N2 - Background: With a growing adolescent mental health crisis, pediatric societies are increasingly recommending that primary care providers (PCPs) engage in mental health screening. While symptom-level screens identify symptoms, novel technology interventions can assist PCPs with providing additional point-of-care guidance to increase uptake for behavioral health services. Objective: In this study, we sought community PCP feedback on a web-based, digitally enhanced mental health screening tool for adolescents in primary care previously only evaluated in research studies to inform implementation in community settings. Methods: A total of 10 adolescent providers were recruited to trial the new screening tool and participate in structured interviews based on the Consolidated Framework for Implementation Research domains. Interviews were audio recorded, transcribed, and coded according to a prespecified codebook using a template analysis approach. Results: Providers identified improving mental health screening and treatment in pediatric primary care as a priority and agreed that a web-based digitally enhanced screening tool could help facilitate identification of and management of adolescent depression. Salient barriers identified were lack of electronic health record integration, time to administer screening, implications on clinic workflow, accessibility, and lack of transparency within health care organizations about the process of approving new technologies for clinical use. Providers made multiple suggestions to enhance implementation in community settings, such as incorporating customization options. Conclusions: Technology interventions can help address the need for improved behavioral health support in primary care settings. However, numerous barriers exist, complicating implementation of new technologies in real-world settings. UR - https://formative.jmir.org/2025/1/e67624 UR - http://dx.doi.org/10.2196/67624 ID - info:doi/10.2196/67624 ER - TY - JOUR AU - Suzuki, Takuya AU - Kono, Yuji AU - Ogasawara, Takayuki AU - Mukaino, Masahiko AU - Aoshima, Yasushi AU - Furuzawa, Shotaro AU - Fujita, Yurie AU - Matsuura, Hirotaka AU - Yamaguchi, Masumi AU - Tsukada, Shingo AU - Otaka, Yohei PY - 2025/4/8 TI - Moving Standard Deviation of Trunk Acceleration as a Quantification Index for Physical Activities: Validation Study JO - JMIR Form Res SP - e63064 VL - 9 KW - smart clothing KW - step count KW - moving standard deviation of acceleration KW - MSDA KW - wheelchair KW - activity quantification KW - physical activities KW - validation study KW - accelerometer KW - regular gait patterns KW - older people KW - aging KW - motor impairments KW - step detection KW - stroke KW - hemiparesis KW - measurement system KW - walking KW - mobility KW - rehabilitation N2 - Background: Step count is used to quantify activity in individuals using accelerometers. However, challenges such as difficulty in detecting steps during slow or irregular gait patterns and the inability to apply this method to wheelchair (WC) users limit the broader utility of accelerometers. Alternative device-specific measures of physical activity exist, but their specificity limits cross-applicability between different device sensors. Moving standard deviation of acceleration (MSDA), obtained from truncal acceleration measurements, is proposed as another alternative variable to quantify physical activity in patients. Objective: This study aimed to evaluate the validity and feasibility of MSDA for quantifying physical activity in patients with stroke-induced hemiparesis by comparing it with the traditional step count. Methods: We enrolled 197 consecutive patients with stroke hemiparesis admitted to a convalescent rehabilitation ward. Using the hitoe system, a smart clothing?based physical activity measurement system, we measured the MSDA of trunk movement and step count. The correlation between MSDA and step count was examined in all participants. Based on their daily living mobility levels, measured using the Functional Independence Measure (FIM), participants were categorized into 6 subgroups: FIM1-4, FIM5 (WC), FIM5 (walking), FIM6 (WC), FIM6 (walking), and FIM7 (walking). Intersubgroup differences in MSDA were analyzed. Results: A strong correlation was observed between MSDA and step count (r=0.78; P<.001), with a stronger correlation in the walking group (r=0.79; P<.001) compared with the WC group (r=0.55; P<.001). The Shapiro-Wilk test indicated no significant results for MSDA across all subgroups, supporting a normal distribution within these groups. In contrast, the step count data for the WC subgroups showed significant results, indicating a deviation from a normal distribution. Additionally, 10.2% (20/197) of participants recorded zero steps, demonstrating a floor effect in the step count data. The median MSDA values for the 6 subgroups (FIM1-4, FIM5 WC, FIM5 walking, FIM6 WC, FIM6 walking, and FIM7) were 0.006, 0.007, 0.010, 0.011, 0.011, and 0.014, respectively, reflecting their levels of independence based on the FIM mobility scores. The median step counts for these subgroups were 68, 233, 1386, 367, 2835, and 4462, respectively. FIM5 participants who walked had higher step counts than FIM6 participants using WCs, though the difference was marginally but not statistically significant (P=.07), highlighting the impact of mobility type (walking vs WC). Conclusions: The results suggest the validity of MSDA as a variable for physical activity in patients with stroke, applicable to patients with stroke irrespective of their mobility measures. This finding highlights the potential of MSDA for use in individuals with motor impairments, including WC users, underscoring its broad utility in rehabilitation clinical practice. UR - https://formative.jmir.org/2025/1/e63064 UR - http://dx.doi.org/10.2196/63064 ID - info:doi/10.2196/63064 ER - TY - JOUR AU - Thorup, Brun Charlotte AU - Uitto, Mika AU - Butler-Henderson, Kerryn AU - Wamala-Andersson, Sarah AU - Hoffrén-Mikkola, Merja AU - Schack Thoft, Diana AU - Korsbakke Emtekær Hæsum, Lisa AU - Irrazabal, Gabriela AU - Pruneda González, Laura AU - Valkama, Katja PY - 2025/4/8 TI - Choosing the Best Digital Health Literacy Measure for Research: Mixed Methods Study JO - J Med Internet Res SP - e59807 VL - 27 KW - digital health literacy KW - digital literacy KW - Horizon Europe KW - EU KW - health technology KW - life expectancy KW - health literacy KW - chronic disease KW - digitalization KW - digital health service KW - digital health intervention KW - technology KW - healthcare N2 - Background: The global demographic shift towards longer life expectancy and complex health needs is increasing the number of people with chronic diseases, placing pressure on health and care systems. With the digitalization of healthcare, digital Health Literacy (dHL), or the use of digital skills in health, is gaining importance. It involves navigating digital health information, using digital tools effectively, and making informed health decisions. Measuring dHL can help identify gaps and develop strategies to improve dHL and health, ensuring citizens equal opportunity to participate in a digital healthcare system. The European project ?The Improving Digital Empowerment for Active and Healthy Living (IDEAHL)? with the objective to empower European Union citizens to use digital instruments to take a more active role in managing their health and well-being creates the base for this overview Objective: This paper aims to conduct an overview of existing assessment tools for measuring dHL and recommend strategies for choosing relevant assessment tools. Methods: This study was carried out as a mixed method study initiated by a scoping review (10 scientific databases, 14 databases with grey literature and 14 predefined reports) in addition to three papers published after finalisations the literature search in IDEAHL, followed by a qualitative workshop study and a final analysis combining results. Results: The literature search resulted in 33 papers on dHL instruments, that was analyzed together with three recently published reviews and findings from a workshop with 13 champions (understood as professionals with expertise in HL and dHL) from five countries (Spain, Denmark, Sweden, Australia, and Germany) representing the health sector or health literacy research. Future tools should adapt to the latest trends and technologies, considering attitudes towards digital health and trust in its services. They should identify beneficiaries of digital health services, measure the impact of dHL interventions, and objectively evaluate functional skills. These tools should be evidence-based, validate instruments, interpret dHL results, and capture diverse experiences to reveal health behaviour changes. Conclusions: The eHealth Literacy Scale (eHEALS), despite being the most frequently utilized tool, has limitations in scope and adaptability. Future tools need to reflect digital trends, encompassing individual skills. However, it is important to note that the ?adequacy? of dHL is context-specific and relies on healthcare systems and the technology provided, particularly the user interface. The focus should be on health improvement, not just elevating dHL levels. A comprehensive approach to dHL assessments addressing diversity and relevance is crucial. Ethical considerations in dHL, including privacy and data security, are important due to potential feelings of shame among those with low literacy levels. UR - https://www.jmir.org/2025/1/e59807 UR - http://dx.doi.org/10.2196/59807 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59807 ER - TY - JOUR AU - Lee, Denise AU - Vaid, Akhil AU - Menon, M. Kartikeya AU - Freeman, Robert AU - Matteson, S. David AU - Marin, L. Michael AU - Nadkarni, N. Girish PY - 2025/4/7 TI - Using Large Language Models to Automate Data Extraction From Surgical Pathology Reports: Retrospective Cohort Study JO - JMIR Form Res SP - e64544 VL - 9 KW - natural language processing KW - large language model KW - artificial intelligence KW - thyroid cancer KW - endocrine surgery KW - framework KW - privacy KW - medical KW - surgical pathology KW - report KW - NLP KW - medical question N2 - Background: Popularized by ChatGPT, large language models (LLMs) are poised to transform the scalability of clinical natural language processing (NLP) downstream tasks such as medical question answering (MQA) and automated data extraction from clinical narrative reports. However, the use of LLMs in the health care setting is limited by cost, computing power, and patient privacy concerns. Specifically, as interest in LLM-based clinical applications grows, regulatory safeguards must be established to avoid exposure of patient data through the public domain. The use of open-source LLMs deployed behind institutional firewalls may ensure the protection of private patient data. In this study, we evaluated the extraction performance of a locally deployed LLM for automated MQA from surgical pathology reports. Objective: We compared the performance of human reviewers and a locally deployed LLM tasked with extracting key histologic and staging information from surgical pathology reports. Methods: A total of 84 thyroid cancer surgical pathology reports were assessed by two independent reviewers and the open-source FastChat-T5 3B-parameter LLM using institutional computing resources. Longer text reports were split into 1200-character-long segments, followed by conversion to embeddings. Three segments with the highest similarity scores were integrated to create the final context for the LLM. The context was then made part of the question it was directed to answer. Twelve medical questions for staging and thyroid cancer recurrence risk data extraction were formulated and answered for each report. The time to respond and concordance of answers were evaluated. The concordance rate for each pairwise comparison (human-LLM and human-human) was calculated as the total number of concordant answers divided by the total number of answers for each of the 12 questions. The average concordance rate and associated error of all questions were tabulated for each pairwise comparison and evaluated with two-sided t tests. Results: Out of a total of 1008 questions answered, reviewers 1 and 2 had an average (SD) concordance rate of responses of 99% (1%; 999/1008 responses). The LLM was concordant with reviewers 1 and 2 at an overall average (SD) rate of 89% (7%; 896/1008 responses) and 89% (7.2%; 903/1008 responses). The overall time to review and answer questions for all reports was 170.7, 115, and 19.56 minutes for Reviewers 1, 2, and the LLM, respectively. Conclusions: The locally deployed LLM can be used for MQA with considerable time-saving and acceptable accuracy in responses. Prompt engineering and fine-tuning may further augment automated data extraction from clinical narratives for the provision of real-time, essential clinical insights. UR - https://formative.jmir.org/2025/1/e64544 UR - http://dx.doi.org/10.2196/64544 ID - info:doi/10.2196/64544 ER - TY - JOUR AU - Han, Seunghoon AU - Song, Jihong AU - Han, Sungpil AU - Choi, Suein AU - Lim, Jonghyuk AU - Oh, Yeob Byeong AU - Shin, Dongoh PY - 2025/4/7 TI - Participant Adherence in Repeated-Dose Clinical Studies Using Video-Based Observation: Retrospective Data Analysis JO - JMIR Mhealth Uhealth SP - e65668 VL - 13 KW - adherence KW - mobile health KW - self-administration KW - repeated-dose clinical trials KW - video-based monitoring KW - mobile phone N2 - Background: Maintaining accurate medication records in clinical trials is essential to ensure data validity. Traditional methods such as direct observation, self-reporting, and pill counts have shown limitations that make them inaccurate or impractical. Video-based monitoring systems, available as commercial or proprietary mobile applications for smartphones and tablets, offer a promising solution to these traditional limitations. In Korea, a system applicable to the clinical trial context has been developed and used. Objective: This study aimed to evaluate the usefulness of an asynchronous video-based self-administration of the investigational medicinal product (SAI) monitoring system (VSMS) in ensuring accurate dosing and validating participant adherence to planned dosing times in repeated-dose clinical trials. Methods: A retrospective analysis was conducted using data from 17,619 SAI events in repeated-dose clinical trials using the VSMS between February 2020 and March 2023. The SAI events were classified into four categories: (1) Verified on-time dosing, (2) Verified deviated dosing, (3) Unverified dosing, and (4) Missed dosing. Analysis methods included calculating the success rate for verified SAI events and analyzing trends in difference between planned and actual dosing times (PADEV) over the dosing period and by push notification type. The mean PADEV for each subsequent dosing period was compared with the initial period using either a paired t test or a Wilcoxon signed-rank test to assess any differences. Results: A comprehensive analysis of 17,619 scheduled SAI events across 14 cohorts demonstrated a high success rate of 97% (17,151/17,619), with only 3% (468/17,619) unsuccessful due to issues like unclear video recordings or technical difficulties. Of the successful events, 99% (16,975/17,151) were verified as on-time dosing, confirming that the dosing occurred within the designated SAI time window with appropriate recorded behavior. In addition, over 90% (367/407) of participants consistently reported dosing videos on all analyzed SAI days, with most days showing over 90% objective dosing data, underscoring the system?s effectiveness in supporting accurate SAI. There were cohort differences in the tendency to dose earlier or later, but no associated cohort characteristics were identified. The initial SAI behaviors were generally sustained during the whole period of participation, with only 16% (13/79) of study days showing significant shifts in actual dosing times. Earlier deviations in SAI times were observed when only dosing notifications were used, compared with using reminders together or no notifications. Conclusions: VSMS has proven to be an effective tool for obtaining dosing information with accuracy comparable to direct observation, even in remote settings. The use of various alarm features and appropriate intervention by the investigator or observer was identified as a way to minimize adherence deterioration. It is expected that the usage and usefulness of VSMS will be continuously improved through the accumulation of experience in various medical fields. UR - https://mhealth.jmir.org/2025/1/e65668 UR - http://dx.doi.org/10.2196/65668 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65668 ER - TY - JOUR AU - Castellanos, Arturo AU - Jiang, Haoqiang AU - Gomes, Paulo AU - Vander Meer, Debra AU - Castillo, Alfred PY - 2025/4/4 TI - Large Language Models for Thematic Summarization in Qualitative Health Care Research: Comparative Analysis of Model and Human Performance JO - JMIR AI SP - e64447 VL - 4 KW - artificial intelligence KW - generative AI KW - large language models KW - ChatGPT KW - machine learning KW - health care N2 - Background: The application of large language models (LLMs) in analyzing expert textual online data is a topic of growing importance in computational linguistics and qualitative research within health care settings. Objective: The objective of this study was to understand how LLMs can help analyze expert textual data. Topic modeling enables scaling the thematic analysis of content of a large corpus of data, but it still requires interpretation. We investigate the use of LLMs to help researchers scale this interpretation. Methods: The primary methodological phases of this project were (1) collecting data representing posts to an online nurse forum, as well as cleaning and preprocessing the data; (2) using latent Dirichlet allocation (LDA) to derive topics; (3) using human categorization for topic modeling; and (4) using LLMs to complement and scale the interpretation of thematic analysis. The purpose is to compare the outcomes of human interpretation with those derived from LLMs. Results: There is substantial agreement (247/310, 80%) between LLM and human interpretation. For two-thirds of the topics, human evaluation and LLMs agree on alignment and convergence of themes. Furthermore, LLM subthemes offer depth of analysis within LDA topics, providing detailed explanations that align with and build upon established human themes. Nonetheless, LLMs identify coherence and complementarity where human evaluation does not. Conclusions: LLMs enable the automation of the interpretation task in qualitative research. There are challenges in the use of LLMs for evaluation of the resulting themes. UR - https://ai.jmir.org/2025/1/e64447 UR - http://dx.doi.org/10.2196/64447 ID - info:doi/10.2196/64447 ER - TY - JOUR AU - K?yak, Selim Yavuz AU - Kononowicz, A. Andrzej PY - 2025/4/4 TI - Using a Hybrid of AI and Template-Based Method in Automatic Item Generation to Create Multiple-Choice Questions in Medical Education: Hybrid AIG JO - JMIR Form Res SP - e65726 VL - 9 KW - automatic item generation KW - ChatGPT KW - artificial intelligence KW - large language models KW - medical education KW - AI KW - hybrid KW - template-based method KW - hybrid AIG KW - mixed-method KW - multiple-choice question KW - multiple-choice KW - human-AI collaboration KW - human-AI KW - algorithm KW - expert N2 - Background: Template-based automatic item generation (AIG) is more efficient than traditional item writing but it still heavily relies on expert effort in model development. While nontemplate-based AIG, leveraging artificial intelligence (AI), offers efficiency, it faces accuracy challenges. Medical education, a field that relies heavily on both formative and summative assessments with multiple choice questions, is in dire need of AI-based support for the efficient automatic generation of items. Objective: We aimed to propose a hybrid AIG to demonstrate whether it is possible to generate item templates using AI in the field of medical education. Methods: This is a mixed-methods methodological study with proof-of-concept elements. We propose the hybrid AIG method as a structured series of interactions between a human subject matter expert and AI, designed as a collaborative authoring effort. The method leverages AI to generate item models (templates) and cognitive models to combine the advantages of the two AIG approaches. To demonstrate how to create item models using hybrid AIG, we used 2 medical multiple-choice questions: one on respiratory infections in adults and another on acute allergic reactions in the pediatric population. Results: The hybrid AIG method we propose consists of 7 steps. The first 5 steps are performed by an expert in a customized AI environment. These involve providing a parent item, identifying elements for manipulation, selecting options and assigning values to elements, and generating the cognitive model. After a final expert review (Step 6), the content in the template can be used for item generation through a traditional (non-AI) software (Step 7). We showed that AI is capable of generating item templates for AIG under the control of a human expert in only 10 minutes. Leveraging AI in template development made it less challenging. Conclusions: The hybrid AIG method transcends the traditional template-based approach by marrying the ?art? that comes from AI as a ?black box? with the ?science? of algorithmic generation under the oversight of expert as a ?marriage registrar?. It does not only capitalize on the strengths of both approaches but also mitigates their weaknesses, offering a human-AI collaboration to increase efficiency in medical education. UR - https://formative.jmir.org/2025/1/e65726 UR - http://dx.doi.org/10.2196/65726 ID - info:doi/10.2196/65726 ER - TY - JOUR AU - Kato, Daigo AU - Okuno, Akiko AU - Ishikawa, Tetsuo AU - Itakura, Shoji AU - Oguchi, Shinji AU - Kasahara, Yoshiyuki AU - Kanenishi, Kenji AU - Kitadai, Yuzo AU - Kimura, Yoshitaka AU - Shimojo, Naoki AU - Nakahara, Kazushige AU - Hanai, Akiko AU - Hamada, Hiromichi AU - Mogami, Haruta AU - Morokuma, Seiichi AU - Sakurada, Kazuhiro AU - Konishi, Yukuo AU - Kawakami, Eiryo PY - 2025/4/4 TI - Multilevel Factors and Indicators of Atypical Neurodevelopment During Early Infancy in Japan: Prospective, Longitudinal, Observational Study JO - JMIR Pediatr Parent SP - e58337 VL - 8 KW - early developmental signs KW - neurodevelopmental screening KW - risk factors KW - prediction KW - early intervention KW - longitudinal study N2 - Background: The early identification of developmental concerns requires understanding individual differences that may represent early signs of neurodevelopmental conditions. However, few studies have longitudinally examined how child and maternal factors interact to shape these early developmental characteristics. Objective: We aim to identify factors from the perinatal to infant periods associated with early developmental characteristics that may precede formal diagnoses and propose a method for evaluating individual differences in neurodevelopmental trajectories. Methods: A prospective longitudinal observational study of 147 mother-child pairs was conducted from gestation to 12 months post partum. Assessments included prenatal questionnaires and blood collection, cord blood at delivery, and postpartum questionnaires at 1, 6, and 12 months. The Modified Checklist for Autism in Toddlers (M-CHAT) was used to evaluate developmental characteristics that might indicate early signs of atypical neurodevelopment. Polychoric or polyserial correlation coefficients assessed relationships between M-CHAT scores and longitudinal variables. L2-regularized logistic regression and Shapley Additive Explanations predicted M-CHAT scores and determined feature contributions. Results: Twenty-one factors (4 prenatal, 3 at birth, and 14 postnatal) showed significant associations with M-CHAT scores (adjusted P values<.05). The predictive accuracy for M-CHAT scores demonstrated reasonable predictive accuracy (area under the receiver operating characteristic curve=0.79). Key predictors included infant sleep status after 6 months (nighttime sleep duration, bedtime, and difficulties falling asleep), maternal Kessler Psychological Distress Scale scores, and Mother-to-Infant Bonding Scale scores after late gestation. Conclusion: Maternal psychological distress, mother-infant bonding, and infant sleep patterns were identified as significant predictors of early developmental characteristics that may indicate emerging developmental concerns. This study advances our understanding of early developmental assessment by providing a novel approach to identifying and evaluating early indicators of atypical neurodevelopment. UR - https://pediatrics.jmir.org/2025/1/e58337 UR - http://dx.doi.org/10.2196/58337 ID - info:doi/10.2196/58337 ER - TY - JOUR AU - Ghossein, Jamie AU - Hryciw, N. Brett AU - Ramsay, Tim AU - Kyeremanteng, Kwadwo PY - 2025/3/28 TI - The AI Reviewer: Evaluating AI?s Role in Citation Screening for Streamlined Systematic Reviews JO - JMIR Form Res SP - e58366 VL - 9 KW - article screening KW - artificial intelligence KW - systematic review KW - AI KW - large language model KW - LLM KW - screening KW - analysis KW - reviewer KW - app KW - ChatGPT 3.5 KW - chatbot KW - dataset KW - data KW - adoption UR - https://formative.jmir.org/2025/1/e58366 UR - http://dx.doi.org/10.2196/58366 ID - info:doi/10.2196/58366 ER - TY - JOUR AU - Nasrudin, Nurfarhana AU - Sazlina, Shariff-Ghazali AU - Cheong, Theng Ai AU - Lee, Yein Ping AU - Teo, Soo-Hwang AU - Aneesa, Rashid Abdul AU - Teo, Hai Chin AU - Rokhani, Zaman Fakhrul AU - Haron, Azam Nuzul AU - Harrun, Harzana Noor AU - Ho, Kiau Bee AU - Mohamed Isa, Salbiah PY - 2025/3/28 TI - Increasing the Uptake of Breast and Cervical Cancer Screening Via the MAwar Application: Stakeholder-Driven Web Application Development Study JO - JMIR Form Res SP - e65542 VL - 9 KW - cancer screening KW - stakeholder engagement KW - Quality Function Deployment KW - web health app N2 - Background: Digital health interventions such as web health applications significantly enhance screening accessibility and uptake, particularly for individuals with low literacy and income levels. By involving stakeholders?including health care professionals, patients, and technical experts?an intervention can be tailored to effectively meet the users? needs, ensuring contextual relevance for better acceptance and impact. Objective: The aim of this study is to prioritize the content and user interface appropriate for developing a web health application, known as the MAwar app, to promote breast and cervical cancer screening. Methods: A cross-sectional study for stakeholder engagement was conducted to develop a web-based application known as the MAwar app as part of a larger study entitled ?The Effectiveness of an Interactive Web Application to Motivate and Raise Awareness on Early Detection of Breast and Cervical Cancers (The MAwar study)?. The stakeholder engagement process was conducted in a public health district that oversees 12 public primary care clinics with existing cervical and breast cancer screening programs. We purposively selected the stakeholders for their relevant roles in breast and cervical cancer screening (health care staff, patients, and public representatives), as well as expertise in software and user interface design (technology experts). The Quality Function Deployment method was used to reflect the priorities of diverse stakeholders (health care, technology experts, patients, and public representatives) in its design. The Quality Function Deployment method facilitated the translation of stakeholder perspectives into app features. Stakeholders rated features on a scale from 1 (least important) to 5 (most important), ensuring the app?s design resonated with user needs. The correlations between the ?WHATs? (user requirements) and the ?HOWs? (technical requirements) were scored using a 3-point ordinal scale, with 1 indicating weak correlation, 5 indicating medium correlation, and 9 indicating the strongest correlation. Results: A total of 13 stakeholders participated in the study, including women who had either underwent or never had health screening, a health administrator, a primary care physician, medical officers, nurses, and software designers. Stakeholder evaluations highlighted cost-free access (mean 4.64, SD 0.81), comprehensive cancer information (mean 4.55, SD 0.69), detailed screening benefits (mean 4.45, SD 0.68), detailed screening facilities (mean 4.45, SD 0.68) and personalized risk calculator for breast and cervical cancers (mean 4.45, SD 0.68) as essential priorities of the app. The highest-ranked features include detailed information on screening procedures (weighted score [WS]=367.84), information on treatment options (WS=345.80), benefits of screening (WS=333.75), information about breast and cervical cancers (WS=332.15), and frequently asked questions about the concerns around screening (WS=312.00). Conclusions: The MAwar app, conceived through a collaborative, stakeholder-driven process, represents a significant step in leveraging digital health solutions to tackle cancer screening disparities. By prioritizing accessibility, information quality, and clarity on benefits, the app promises to encourage early cancer detection and management for targeted communities. Trial Registration: ISRCTN Registry ISRCTN10403163; https://www.isrctn.com/ISRCTN10403163 UR - https://formative.jmir.org/2025/1/e65542 UR - http://dx.doi.org/10.2196/65542 ID - info:doi/10.2196/65542 ER - TY - JOUR AU - Roshani, Amin Mohammad AU - Zhou, Xiangyu AU - Qiang, Yao AU - Suresh, Srinivasan AU - Hicks, Steven AU - Sethuraman, Usha AU - Zhu, Dongxiao PY - 2025/3/27 TI - Generative Large Language Model?Powered Conversational AI App for Personalized Risk Assessment: Case Study in COVID-19 JO - JMIR AI SP - e67363 VL - 4 KW - personalized risk assessment KW - large language model KW - conversational AI KW - artificial intelligence KW - COVID-19 N2 - Background: Large language models (LLMs) have demonstrated powerful capabilities in natural language tasks and are increasingly being integrated into health care for tasks like disease risk assessment. Traditional machine learning methods rely on structured data and coding, limiting their flexibility in dynamic clinical environments. This study presents a novel approach to disease risk assessment using generative LLMs through conversational artificial intelligence (AI), eliminating the need for programming. Objective: This study evaluates the use of pretrained generative LLMs, including LLaMA2-7b and Flan-T5-xl, for COVID-19 severity prediction with the goal of enabling a real-time, no-code, risk assessment solution through chatbot-based, question-answering interactions. To contextualize their performance, we compare LLMs with traditional machine learning classifiers, such as logistic regression, extreme gradient boosting (XGBoost), and random forest, which rely on tabular data. Methods: We fine-tuned LLMs using few-shot natural language examples from a dataset of 393 pediatric patients, developing a mobile app that integrates these models to provide real-time, no-code, COVID-19 severity risk assessment through clinician-patient interaction. The LLMs were compared with traditional classifiers across different experimental settings, using the area under the curve (AUC) as the primary evaluation metric. Feature importance derived from LLM attention layers was also analyzed to enhance interpretability. Results: Generative LLMs demonstrated strong performance in low-data settings. In zero-shot scenarios, the T0-3b-T model achieved an AUC of 0.75, while other LLMs, such as T0pp(8bit)-T and Flan-T5-xl-T, reached 0.67 and 0.69, respectively. At 2-shot settings, logistic regression and random forest achieved an AUC of 0.57, while Flan-T5-xl-T and T0-3b-T obtained 0.69 and 0.65, respectively. By 32-shot settings, Flan-T5-xl-T reached 0.70, similar to logistic regression (0.69) and random forest (0.68), while XGBoost improved to 0.65. These results illustrate the differences in how generative LLMs and traditional models handle the increasing data availability. LLMs perform well in low-data scenarios, whereas traditional models rely more on structured tabular data and labeled training examples. Furthermore, the mobile app provides real-time, COVID-19 severity assessments and personalized insights through attention-based feature importance, adding value to the clinical interpretation of the results. Conclusions: Generative LLMs provide a robust alternative to traditional classifiers, particularly in scenarios with limited labeled data. Their ability to handle unstructured inputs and deliver personalized, real-time assessments without coding makes them highly adaptable to clinical settings. This study underscores the potential of LLM-powered conversational artificial intelligence (AI) in health care and encourages further exploration of its use for real-time, disease risk assessment and decision-making support. UR - https://ai.jmir.org/2025/1/e67363 UR - http://dx.doi.org/10.2196/67363 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67363 ER - TY - JOUR AU - Hasegawa, Tatsuki AU - Kizaki, Hayato AU - Ikegami, Keisho AU - Imai, Shungo AU - Yanagisawa, Yuki AU - Yada, Shuntaro AU - Aramaki, Eiji AU - Hori, Satoko PY - 2025/3/27 TI - Improving Systematic Review Updates With Natural Language Processing Through Abstract Component Classification and Selection: Algorithm Development and Validation JO - JMIR Med Inform SP - e65371 VL - 13 KW - systematic review KW - natural language processing KW - guideline updates KW - bidirectional encoder representations from transformer KW - screening model KW - literature KW - efficiency KW - updating systematic reviews KW - language model N2 - Background: A challenge in updating systematic reviews is the workload in screening the articles. Many screening models using natural language processing technology have been implemented to scrutinize articles based on titles and abstracts. While these approaches show promise, traditional models typically treat abstracts as uniform text. We hypothesize that selective training on specific abstract components could enhance model performance for systematic review screening. Objective: We evaluated the efficacy of a novel screening model that selects specific components from abstracts to improve performance and developed an automatic systematic review update model using an abstract component classifier to categorize abstracts based on their components. Methods: A screening model was created based on the included and excluded articles in the existing systematic review and used as the scheme for the automatic update of the systematic review. A prior publication was selected for the systematic review, and articles included or excluded in the articles screening process were used as training data. The titles and abstracts were classified into 5 categories (Title, Introduction, Methods, Results, and Conclusion). Thirty-one component-composition datasets were created by combining 5 component datasets. We implemented 31 screening models using the component-composition datasets and compared their performances. Comparisons were conducted using 3 pretrained models: Bidirectional Encoder Representations from Transformer (BERT), BioLinkBERT, and BioM- Efficiently Learning an Encoder that Classifies Token Replacements Accurately (ELECTRA). Moreover, to automate the component selection of abstracts, we developed the Abstract Component Classifier Model and created component datasets using this classifier model classification. Using the component datasets classified using the Abstract Component Classifier Model, we created 10 component-composition datasets used by the top 10 screening models with the highest performance when implementing screening models using the component datasets that were classified manually. Ten screening models were implemented using these datasets, and their performances were compared with those of models developed using manually classified component-composition datasets. The primary evaluation metric was the F10-Score weighted by the recall. Results: A total of 256 included articles and 1261 excluded articles were extracted from the selected systematic review. In the screening models implemented using manually classified datasets, the performance of some surpassed that of models trained on all components (BERT: 9 models, BioLinkBERT: 6 models, and BioM-ELECTRA: 21 models). In models implemented using datasets classified by the Abstract Component Classifier Model, the performances of some models (BERT: 7 models and BioM-ELECTRA: 9 models) surpassed that of the models trained on all components. These models achieved an 88.6% reduction in manual screening workload while maintaining high recall (0.93). Conclusions: Component selection from the title and abstract can improve the performance of screening models and substantially reduce the manual screening workload in systematic review updates. Future research should focus on validating this approach across different systematic review domains. UR - https://medinform.jmir.org/2025/1/e65371 UR - http://dx.doi.org/10.2196/65371 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65371 ER - TY - JOUR AU - Stone, Chris AU - Adams, Sally AU - Wootton, E. Robyn AU - Skinner, Andy PY - 2025/3/25 TI - Smartwatch-Based Ecological Momentary Assessment for High-Temporal-Density, Longitudinal Measurement of Alcohol Use (AlcoWatch): Feasibility Evaluation JO - JMIR Form Res SP - e63184 VL - 9 KW - smartwatch KW - ecological momentary assessment KW - ?EMA KW - alcohol KW - ALSPAC N2 - Background: Ecological momentary assessment methods have recently been adapted for use on smartwatches. One particular class of these methods, developed to minimize participant burden and maximize engagement and compliance, is referred to as microinteraction-based ecological momentary assessment (?EMA). Objective: This study explores the feasibility of using these smartwatch-based ?EMA methods to capture longitudinal, high-temporal-density self-report data about alcohol consumption in a nonclinical population selected to represent high- and low-socioeconomic position (SEP) groups. Methods: A total of 32 participants from the Avon Longitudinal Study of Parents and Children (13 high and 19 low SEP) wore a smartwatch running a custom-developed ?EMA app for 3 months between October 2019 and June 2020. Every day over a 12-week period, participants were asked 5 times a day about any alcoholic drinks they had consumed in the previous 2 hours, and the context in which they were consumed. They were also asked if they had missed recording any alcoholic drinks the day before. As a comparison, participants also completed fortnightly online diaries of alcohol consumed using the Timeline Followback (TLFB) method. At the end of the study, participants completed a semistructured interview about their experiences. Results: The compliance rate for all participants who started the study for the smartwatch ?EMA method decreased from around 70% in week 1 to 45% in week 12, compared with the online TLFB method which was flatter at around 50% over the 12 weeks. The compliance for all participants still active for the smartwatch ?EMA method was much flatter, around 70% for the whole 12 weeks, while for the online TLFB method, it varied between 50% and 80% over the same period. The completion rate for the smartwatch ?EMA method varied around 80% across the 12 weeks. Within high- and low-SEP groups there was considerable variation in compliance and completion at each week of the study for both methods. However, almost all point estimates for both smartwatch ?EMA and online TLFB indicated lower levels of engagement for low-SEP participants. All participants scored ?experiences of using? the 2 methods equally highly, with ?willingness to use again? slightly higher for smartwatch ?EMA. Conclusions: Our findings demonstrate the acceptability and potential utility of smartwatch ?EMA methods for capturing data on alcohol consumption. These methods have the benefits of capturing higher-temporal-density longitudinal data on alcohol consumption, promoting greater participant engagement with less missing data, and potentially being less susceptible to recall errors than established methods such as TLFB. Future studies should explore the factors impacting participant attrition (the biggest reason for reduced engagement), latency issues, and the validity of alcohol data captured with these methods. The consistent pattern of lower engagement among low-SEP participants than high-SEP participants indicates that further work is warranted to explore the impact and causes of these differences. UR - https://formative.jmir.org/2025/1/e63184 UR - http://dx.doi.org/10.2196/63184 UR - http://www.ncbi.nlm.nih.gov/pubmed/40131326 ID - info:doi/10.2196/63184 ER - TY - JOUR AU - Helgeson, A. Scott AU - Quicksall, S. Zachary AU - Johnson, W. Patrick AU - Lim, G. Kaiser AU - Carter, E. Rickey AU - Lee, S. Augustine PY - 2025/3/24 TI - Estimation of Static Lung Volumes and Capacities From Spirometry Using Machine Learning: Algorithm Development and Validation JO - JMIR AI SP - e65456 VL - 4 KW - artificial intelligence KW - machine learning KW - pulmonary function test KW - spirometry KW - total lung capacity KW - AI KW - ML KW - lung KW - lung volume KW - lung capacity KW - spirometer KW - lung disease KW - database KW - respiratory KW - pulmonary N2 - Background: Spirometry can be performed in an office setting or remotely using portable spirometers. Although basic spirometry is used for diagnosis of obstructive lung disease, clinically relevant information such as restriction, hyperinflation, and air trapping require additional testing, such as body plethysmography, which is not as readily available. We hypothesize that spirometry data contains information that can allow estimation of static lung volumes in certain circumstances by leveraging machine learning techniques. Objective: The aim of the study was to develop artificial intelligence-based algorithms for estimating lung volumes and capacities using spirometry measures. Methods: This study obtained spirometry and lung volume measurements from the Mayo Clinic pulmonary function test database for patient visits between February 19, 2001, and December 16, 2022. Preprocessing was performed, and various machine learning algorithms were applied, including a generalized linear model with regularization, random forests, extremely randomized trees, gradient-boosted trees, and XGBoost for both classification and regression cohorts. Results: A total of 121,498 pulmonary function tests were used in this study, with 85,017 allotted for exploratory data analysis and model development (ie, training dataset) and 36,481 tests reserved for model evaluation (ie, testing dataset). The median age of the cohort was 64.7 years (IQR 18?119.6), with a balanced distribution between genders, consisting 48.2% (n=58,607) female and 51.8% (n=62,889) male patients. The classification models showed a robust performance overall, with relatively low root mean square error and mean absolute error values observed across all predicted lung volumes. Across all lung volume categories, the models demonstrated strong discriminatory capacity, as indicated by the high area under the receiver operating characteristic curve values ranging from 0.85 to 0.99 in the training set and 0.81 to 0.98 in the testing set. Conclusions: Overall, the models demonstrate robust performance across lung volume measurements, underscoring their potential utility in clinical practice for accurate diagnosis and prognosis of respiratory conditions, particularly in settings where access to body plethysmography or other lung volume measurement modalities is limited. UR - https://ai.jmir.org/2025/1/e65456 UR - http://dx.doi.org/10.2196/65456 ID - info:doi/10.2196/65456 ER - TY - JOUR AU - Amagai, Saki AU - Kaat, J. Aaron AU - Fox, S. Rina AU - Ho, H. Emily AU - Pila, Sarah AU - Kallen, A. Michael AU - Schalet, D. Benjamin AU - Nowinski, J. Cindy AU - Gershon, C. Richard PY - 2025/3/21 TI - Customizing Computerized Adaptive Test Stopping Rules for Clinical Settings Using the Negative Affect Subdomain of the NIH Toolbox Emotion Battery: Simulation Study JO - JMIR Form Res SP - e60215 VL - 9 KW - computerized adaptive testing KW - CAT KW - stopping rules KW - NIH Toolbox KW - reliability KW - test burden KW - clinical setting KW - patient-reported outcome KW - clinician N2 - Background: Patient-reported outcome measures are crucial for informed medical decisions and evaluating treatments. However, they can be burdensome for patients and sometimes lack the reliability clinicians need for clear clinical interpretations. Objective: We aimed to assess the extent to which applying alternative stopping rules can increase reliability for clinical use while minimizing the burden of computerized adaptive tests (CATs). Methods: CAT simulations were conducted on 3 adult item banks in the NIH Toolbox for Assessment of Neurological and Behavioral Function Emotion Battery; the item banks were in the Negative Affect subdomain (ie, Anger Affect, Fear Affect, and Sadness) and contained at least 8 items. In the originally applied NIH Toolbox CAT stopping rules, the CAT was stopped if the score SE reached <0.3 before 12 items were administered. We first contrasted this with a SE-change rule in a planned simulation analysis. We then contrasted the original rules with fixed-length CATs (4?12 items), a reduction of the maximum number of items to 8, and other modifications in post hoc analyses. Burden was measured by the number of items administered per simulation, precision by the percentage of assessments yielding reliability cutoffs (0.85, 0.90, and 0.95), and accurate score recovery by the root mean squared error between the generating ? and the CAT-estimated ?expected a posteriori??based ?. Results: In general, relative to the original rules, the alternative stopping rules slightly decreased burden while also increasing the proportion of assessments achieving high reliability for the adult banks; however, the SE-change rule and fixed-length CATs with 8 or fewer items also notably increased assessments yielding reliability <0.85. Among the alternative rules explored, the reduced maximum stopping rule best balanced precision and parsimony, presenting another option beyond the original rules. Conclusions: Our findings demonstrate the challenges in attempting to reduce test burden while also achieving score precision for clinical use. Stopping rules should be modified in accordance with the context of the study population and the purpose of the study. UR - https://formative.jmir.org/2025/1/e60215 UR - http://dx.doi.org/10.2196/60215 ID - info:doi/10.2196/60215 ER - TY - JOUR AU - Carr, L. Alaina AU - Jenkins, M. Angela AU - Jonklaas, Jacqueline AU - Gabriel, Kate AU - Miller, E. Kristen AU - Graves, D. Kristi PY - 2025/3/19 TI - Patient and Provider Perspectives of a Web-Based Intervention to Support Symptom Management After Radioactive Iodine Treatment for Differentiated Thyroid Cancer: Qualitative Study JO - JMIR Form Res SP - e60588 VL - 9 KW - iodine radioisotopes KW - person-based approach KW - self-management KW - Social Cognitive Theory KW - survivorship KW - symptom burden KW - thyroid neoplasms KW - web-based intervention KW - radioactive iodine treatment KW - radiotherapy KW - thyroid cancer KW - qualitative KW - quality of life KW - survivorship care KW - supportive care KW - patient with cancer KW - QoL KW - cancer KW - carcinoma KW - malignancy KW - tumor KW - malignant KW - oncology KW - neoplasm KW - benign KW - neoplasia KW - thyroid N2 - Background: Patients diagnosed with differentiated thyroid cancer (DTC) who receive radioactive iodine (RAI) treatment experience acute, medium, and late treatment effects. The timing and severity of these effects vary by individual; common posttreatment effects include dry mouth, salivary gland swelling, dry eyes, and nose bleeds. The nature of symptoms that patients experience after RAI treatment can significantly and negatively impact health-related quality of life. Adequate information during the postprimary treatment phase remains an unmet need among the population of patients diagnosed with DTC. Objective: This qualitative study aimed to identify and understand self-management strategies for RAI-specific symptom burden from the perspectives of patients and stakeholders (cancer care providers and patient advocates). An additional aim included assessing features and functionalities desirable in the development of a web-based intervention to engage patients in their self-management and thyroid cancer survivorship care. Methods: Following the Social Cognitive Theory framework and person-based principles, we conducted six focus groups with 22 patients diagnosed with DTC who completed RAI treatment and individual interviews with 12 stakeholders in DTC care. The interviews focused on participants? perspectives on current self-management strategies and mockups of a symptom management web-based intervention. Before focus groups and interviews, participants completed a demographics survey. Focus group discussions and interviews were transcribed and coded using content analysis. Interrater reliability was satisfactory (?=.88). Results: A total of 34 individuals (patients and stakeholders) participated in the study; the mean age was 45 (SD 13.4) and 45.3 (SD 13) years, respectively. Three domains emerged from qualitative interviews: (1) difficult-to-manage RAI symptoms: short, medium, and late treatment effects; (2) key intervention structure and content feedback on mockups; and (3) intervention content to promote RAI symptom management and survivorship care. Focus group participants identified the most prevalent RAI symptoms that were difficult to manage as: dry mouth (11/22, 50%), salivary gland swelling (8/22, 36%), and changes in taste (12/22, 55%). Feedback elicited from both groups found education and symptom management mockup videos to be helpful in patient self-management of RAI symptoms, whereas patients and stakeholders provided mixed feedback on the benefits of a draft frequently asked questions page. Across focus groups and stakeholder interviews, nutrition-based symptom management strategies, communication with family members, and practical survivorship follow-up information emerged as helpful content to include in a future web-based supportive care intervention. Conclusions: Results suggest education and symptom management videos can empower patients with DTC to self-manage mild to moderate RAI symptoms on a web-based platform. Findings emphasized the need for additional information for patients related to ongoing care following RAI treatment including social support and thyroid cancer surveillance. The findings provide insights for theoretically informed interventions and recommendations for refinements in thyroid cancer survivorship from patient and provider perspectives. UR - https://formative.jmir.org/2025/1/e60588 UR - http://dx.doi.org/10.2196/60588 ID - info:doi/10.2196/60588 ER - TY - JOUR AU - Bianchini, Edoardo AU - Rinaldi, Domiziana AU - De Carolis, Lanfranco AU - Galli, Silvia AU - Alborghetti, Marika AU - Hansen, Clint AU - Suppa, Antonio AU - Salvetti, Marco AU - Pontieri, Ernesto Francesco AU - Vuillerme, Nicolas PY - 2025/3/18 TI - Reliability of Average Daily Steps Measured Through a Consumer Smartwatch in Parkinson Disease Phenotypes, Stages, and Severities: Cross-Sectional Study JO - JMIR Form Res SP - e63153 VL - 9 KW - gait KW - Parkinson disease KW - phenotype KW - wearable sensors KW - smartwatch KW - step count KW - reliability KW - activity monitor KW - digital health technology KW - digital outcome measures KW - wearable KW - mHealth KW - motor KW - quality of life KW - fall KW - posture KW - mobile health N2 - Background: Average daily steps (avDS) could be a valuable indicator of real-world ambulation in people with Parkinson disease (PD), and previous studies have reported the validity and reliability of this measure. Nonetheless, no study has considered disease phenotype, stage, and severity when assessing the reliability of consumer wrist-worn devices to estimate daily step count in unsupervised, free-living conditions in PD. Objective: This study aims to assess and compare the reliability of a consumer wrist-worn smartwatch (Garmin Vivosmart 4) in counting avDS in people with PD in unsupervised, free-living conditions among disease phenotypes, stages, and severity groups. Methods: A total of 104 people with PD were monitored through Garmin Vivosmart 4 for 5 consecutive days. Total daily steps were recorded and avDS were calculated. Participants were dichotomized into tremor dominant (TD; n=39) or postural instability and gait disorder (PIGD; n=65), presence (n=57) or absence (n=47) of tremor, and mild (n=65) or moderate (n=39) disease severity. Based on the modified Hoehn and Yahr scale (mHY), participants were further dichotomized into earlier (mHY 1?2; n=68) or intermediate (mHY 2.5?3; n=36) disease stages. Intraclass correlation coefficient (ICC; 3,k), standard error of measurement (SEM), and minimal detectable change (MDC) were used to evaluate the reliability of avDS for each subgroup. The threshold for acceptability was set at an ICC ?0.8 with a lower bound of 95% CI ?0.75. The 2-tailed Student t tests for independent groups and analysis of 83.4% CI overlap were used to compare ICC between each group pair. Results: Reliability of avDS measured through Garmin Vivosmart 4 for 5 consecutive days in unsupervised, free-living conditions was acceptable in the overall population with an ICC of 0.89 (95% CI 0.85?0.92), SEM below 10%, and an MDC of 1580 steps per day (27% of criterion). In all investigated subgroups, the reliability of avDS was also acceptable (ICC range 0.84?0.94). However, ICCs were significantly lower in participants with tremor (P=.03), with mild severity (P=.04), and earlier stage (P=.003). Moreover, SEM was below 10% in participants with PIGD phenotype, without tremor, moderate disease severity, and intermediate disease stage, with an MDC ranging from 1148 to 1687 steps per day (18%?25% of criterion). Conversely, in participants with TD phenotype, tremor, mild disease severity, and earlier disease stage, SEM was >10% of the criterion and MDC values ranged from 1401 to 2263 steps per day (30%?33% of the criterion). Conclusions: In mild-to-moderate PD, avDS measured through a consumer smartwatch in unsupervised, free-living conditions for 5 consecutive days are reliable irrespective of disease phenotype, stage, and severity. However, in individuals with TD phenotype, tremor, mild disease severity, and earlier disease stages, reliability could be lower. These findings could facilitate a broader and informed implementation of avDS as an index of ambulatory activity in PD. UR - https://formative.jmir.org/2025/1/e63153 UR - http://dx.doi.org/10.2196/63153 ID - info:doi/10.2196/63153 ER - TY - JOUR AU - Twumasi, Clement AU - Aktas, Mikail AU - Santoni, Nicholas PY - 2025/3/18 TI - Kinetic Pattern Recognition in Home-Based Knee Rehabilitation Using Machine Learning Clustering Methods on the Slider Digital Physiotherapy Device: Prospective Observational Study JO - JMIR Form Res SP - e69150 VL - 9 KW - machine learning KW - cluster analysis KW - force measurement KW - knee replacement KW - musculoskeletal KW - physical therapy KW - Slider device KW - knee osteoarthritis KW - digital health KW - telerehabilitation N2 - Background: Recent advancements in rehabilitation sciences have progressively used computational techniques to improve diagnostic and treatment approaches. However, the analysis of high-dimensional, time-dependent data continues to pose a significant problem. Prior research has used clustering techniques on rehabilitation data to identify movement patterns and forecast recovery outcomes. Nonetheless, these initiatives have not yet used force or motion datasets obtained outside a clinical setting, thereby limiting the capacity for therapeutic decisions. Biomechanical data analysis has demonstrated considerable potential in bridging these gaps and improving clinical decision-making in rehabilitation settings. Objective: This study presents a comprehensive clustering analysis of multidimensional movement datasets captured using a novel home exercise device, the ?Slider?. The aim is to identify clinically relevant movement patterns and provide answers to open research questions for the first time to inform personalized rehabilitation protocols, predict individual recovery trajectories, and assess the risks of potential postoperative complications. Methods: High-dimensional, time-dependent, bilateral knee kinetic datasets were independently analyzed from 32 participants using four unsupervised clustering techniques: k-means, hierarchical clustering, partition around medoids, and CLARA (Clustering Large Applications). The data comprised force, laser-measured distance, and optical tracker coordinates from lower limb activities. The optimal clusters identified through the unsupervised clustering methods were further evaluated and compared using silhouette analysis to quantify their performance. Key determinants of cluster membership were assessed, including demographic factors (eg, gender, BMI, and age) and pain levels, by using a logistic regression model with analysis of covariance adjustment. Results: Three distinct, time-varying movement patterns or clusters were identified for each knee. Hierarchical clustering performed best for the right knee datasets (with an average silhouette score of 0.637), while CLARA was the most effective for the left knee datasets (with an average silhouette score of 0.598). Key predictors of the movement cluster membership were discovered for both knees. BMI was the most influential determinant of cluster membership for the right knee, where higher BMI decreased the odds of cluster-2 membership (odds ratio [OR] 0.95, 95% CI 0.94-0.96; P<.001) but increased the odds for cluster-3 assignment relative to cluster 1 (OR 1.05, 95% CI 1.03-1.06; P<.001). For the left knee, all predictors of cluster-2 membership were significant (.001?P?.008), whereas only BMI (P=.81) could not predict the likelihood of an individual belonging to cluster 3 compared to cluster 1. Gender was the strongest determinant for the left knee, with male participants significantly likely to belong to cluster 3 (OR 3.52, 95% CI 2.91-4.27; P<.001). Conclusions: These kinetic patterns offer significant insights for creating personalized rehabilitation procedures, potentially improving patient outcomes. These findings underscore the efficacy of unsupervised clustering techniques in the analysis of biomechanical data for clinical rehabilitation applications. UR - https://formative.jmir.org/2025/1/e69150 UR - http://dx.doi.org/10.2196/69150 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/69150 ER - TY - JOUR AU - Hao, Jie AU - Chen, Zhenli AU - Peng, Qinglong AU - Zhao, Liang AU - Zhao, Wanqing AU - Cong, Shan AU - Li, Junlian AU - Li, Jiao AU - Qian, Qing AU - Sun, Haixia PY - 2025/3/18 TI - Prompt Framework for Extracting Scale-Related Knowledge Entities from Chinese Medical Literature: Development and Evaluation Study JO - J Med Internet Res SP - e67033 VL - 27 KW - prompt engineering KW - named entity recognition KW - in-context learning KW - large language model KW - Chinese medical literature KW - measurement-based care KW - framework KW - prompt KW - prompt framework KW - scale KW - China KW - medical literature KW - MBC KW - LLM KW - MedScaleNER KW - retrieval KW - information retrieval KW - dataset KW - artificial intelligence KW - AI N2 - Background: Measurement-based care improves patient outcomes by using standardized scales, but its widespread adoption is hindered by the lack of accessible and structured knowledge, particularly in unstructured Chinese medical literature. Extracting scale-related knowledge entities from these texts is challenging due to limited annotated data. While large language models (LLMs) show promise in named entity recognition (NER), specialized prompting strategies are needed to accurately recognize medical scale-related entities, especially in low-resource settings. Objective: This study aims to develop and evaluate MedScaleNER, a task-oriented prompt framework designed to optimize LLM performance in recognizing medical scale-related entities from Chinese medical literature. Methods: MedScaleNER incorporates demonstration retrieval within in-context learning, chain-of-thought prompting, and self-verification strategies to improve performance. The framework dynamically retrieves optimal examples using a k-nearest neighbors approach and decomposes the NER task into two subtasks: entity type identification and entity labeling. Self-verification ensures the reliability of the final output. A dataset of manually annotated Chinese medical journal papers was constructed, focusing on three key entity types: scale names, measurement concepts, and measurement items. Experiments were conducted by varying the number of examples and the proportion of training data to evaluate performance in low-resource settings. Additionally, MedScaleNER?s performance was compared with locally fine-tuned models. Results: The CMedS-NER (Chinese Medical Scale Corpus for Named Entity Recognition) dataset, containing 720 papers with 27,499 manually annotated scale-related knowledge entities, was used for evaluation. Initial experiments identified GLM-4-0520 as the best-performing LLM among six tested models. When applied with GLM-4-0520, MedScaleNER significantly improved NER performance for scale-related entities, achieving a macro F1-score of 59.64% in an exact string match with the full training dataset. The highest performance was achieved with 20-shot demonstrations. Under low-resource scenarios (eg, 1% of the training data), MedScaleNER outperformed all tested locally fine-tuned models. Ablation studies highlighted the importance of demonstration retrieval and self-verification in improving model reliability. Error analysis revealed four main types of mistakes: identification errors, type errors, boundary errors, and missing entities, indicating areas for further improvement. Conclusions: MedScaleNER advances the application of LLMs and prompts engineering for specialized NER tasks in Chinese medical literature. By addressing the challenges of unstructured texts and limited annotated data, MedScaleNER?s adaptability to various biomedical contexts supports more efficient and reliable knowledge extraction, contributing to broader measurement-based care implementation and improved clinical and research outcomes. UR - https://www.jmir.org/2025/1/e67033 UR - http://dx.doi.org/10.2196/67033 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67033 ER - TY - JOUR AU - Thell, Maria AU - Edvardsson, Kerstin AU - Aljeshy, Reem AU - Ibrahim, Kalid AU - Warner, Georgina PY - 2025/3/18 TI - A Trauma Support App for Young People: Co-design and Usability Study JO - JMIR Form Res SP - e57789 VL - 9 KW - co-design KW - young people KW - trauma KW - app development KW - usability testing N2 - Background: One of the most common reasons young people with mental health issues, such as posttraumatic stress disorder, do not seek help is stigma, which digital support tools could help address. However, there is a lack of trauma support apps specifically designed for young people. Involving the target group in such projects has been shown to produce more engaging and effective results. Objective: This study aimed to apply a child rights?based participatory approach to develop a trauma support app with young people. Methods: Seven young people (aged 14-19 years; 3 males and 4 females) with experiences of trauma were recruited as coresearchers. A child rights?based framework guided the working process. The app was developed through a series of Design Studio workshops and home assignments, using the manualized intervention Teaching Recovery Techniques as the foundation for its content. The coresearchers were trained in research methodology and conducted usability testing with other young people (n=11) using the think-aloud method, the System Usability Scale (SUS), and qualitative follow-up questions. Results: A functional app prototype was developed using a no-code platform, incorporating various trauma symptom management techniques. These techniques covered psychoeducation, normalization, relaxation, and cognitive shifting, presented in multiple formats, including text, audio, and video. The contributions of the coresearchers to the design can be categorized into 3 areas: mechanics (rules and interactions shaping the app?s structure), dynamics (user-visible elements, such as the outcome when pressing a button), and aesthetics (the emotional responses the app aimed to evoke in users during interaction). Beyond influencing basic aesthetics, the coresearchers placed significant emphasis on user experience and the emotional responses the app could evoke. SUS scores ranged from 67.5 to 97.5, with the vast majority exceeding 77.5, indicating good usability. However, usability testing revealed several issues, generally of lower severity. For instance, video content required improvements, such as reducing light flickering in some recordings and adding rewind and subtitle selection options. Notably, the feature for listening to others? stories was removed to minimize emotional burden, shifting the focus to text formats with more context. Conclusions: Young people who have experienced trauma can actively participate in the cocreation of a mental health intervention, offering valuable insights into the needs and preferences of their peers. Applying a child rights?based framework to their involvement in a research project supported the fulfillment of the Convention on the Rights of the Child Article 12. UR - https://formative.jmir.org/2025/1/e57789 UR - http://dx.doi.org/10.2196/57789 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57789 ER - TY - JOUR AU - Lau, Jerry AU - Bisht, Shivani AU - Horton, Robert AU - Crisan, Annamaria AU - Jones, John AU - Gantotti, Sandeep AU - Hermes-DeSantis, Evelyn PY - 2025/3/13 TI - Creation of Scientific Response Documents for Addressing Product Medical Information Inquiries: Mixed Method Approach Using Artificial Intelligence JO - JMIR AI SP - e55277 VL - 4 KW - AI KW - LLM KW - GPT KW - biopharmaceutical KW - medical information KW - content generation KW - artificial intelligence KW - pharmaceutical KW - scientific response KW - documentation KW - information KW - clinical data KW - strategy KW - reference KW - feasibility KW - development KW - machine learning KW - large language model KW - accuracy KW - context KW - traceability KW - accountability KW - survey KW - scientific response documentation KW - SRD KW - benefit KW - content generator KW - content analysis KW - Generative Pre-trained Transformer N2 - Background: Pharmaceutical manufacturers address health care professionals? information needs through scientific response documents (SRDs), offering evidence-based answers to medication and disease state questions. Medical information departments, staffed by medical experts, develop SRDs that provide concise summaries consisting of relevant background information, search strategies, clinical data, and balanced references. With an escalating demand for SRDs and the increasing complexity of therapies, medical information departments are exploring advanced technologies and artificial intelligence (AI) tools like large language models (LLMs) to streamline content development. While AI and LLMs show promise in generating draft responses, a synergistic approach combining an LLM with traditional machine learning classifiers in a series of human-supervised and -curated steps could help address limitations, including hallucinations. This will ensure accuracy, context, traceability, and accountability in the development of the concise clinical data summaries of an SRD. Objective: This study aims to quantify the challenges of SRD development and develop a framework exploring the feasibility and value addition of integrating AI capabilities in the process of creating concise summaries for an SRD. Methods: To measure the challenges in SRD development, a survey was conducted by phactMI, a nonprofit consortium of medical information leaders in the pharmaceutical industry, assessing aspects of SRD creation among its member companies. The survey collected data on the time and tediousness of various activities related to SRD development. Another working group, consisting of medical information professionals and data scientists, used AI to aid SRD authoring, focusing on data extraction and abstraction. They used logistic regression on semantic embedding features to train classification models and transformer-based summarization pipelines to generate concise summaries. Results: Of the 33 companies surveyed, 64% (21/33) opened the survey, and 76% (16/21) of those responded. On average, medical information departments generate 614 new documents and update 1352 documents each year. Respondents considered paraphrasing scientific articles to be the most tedious and time-intensive task. In the project?s second phase, sentence classification models showed the ability to accurately distinguish target categories with receiver operating characteristic scores ranging from 0.67 to 0.85 (all P<.001), allowing for accurate data extraction. For data abstraction, the comparison of the bilingual evaluation understudy (BLEU) score and semantic similarity in the paraphrased texts yielded different results among reviewers, with each preferring different trade-offs between these metrics. Conclusions: This study establishes a framework for integrating LLM and machine learning into SRD development, supported by a pharmaceutical company survey emphasizing the challenges of paraphrasing content. While machine learning models show potential for section identification and content usability assessment in data extraction and abstraction, further optimization and research are essential before full-scale industry implementation. The working group?s insights guide an AI-driven content analysis; address limitations; and advance efficient, precise, and responsive frameworks to assist with pharmaceutical SRD development. UR - https://ai.jmir.org/2025/1/e55277 UR - http://dx.doi.org/10.2196/55277 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55277 ER - TY - JOUR AU - Yang, Zhongbao AU - Xu, Shan-Shan AU - Liu, Xiaozhu AU - Xu, Ningyuan AU - Chen, Yuqing AU - Wang, Shuya AU - Miao, Ming-Yue AU - Hou, Mengxue AU - Liu, Shuai AU - Zhou, Yi-Min AU - Zhou, Jian-Xin AU - Zhang, Linlin PY - 2025/3/12 TI - Large Language Model?Based Critical Care Big Data Deployment and Extraction: Descriptive Analysis JO - JMIR Med Inform SP - e63216 VL - 13 KW - big data KW - critical care?related databases KW - database deployment KW - large language model KW - database extraction KW - intensive care unit KW - ICU KW - GPT KW - artificial intelligence KW - AI KW - LLM N2 - Background: Publicly accessible critical care?related databases contain enormous clinical data, but their utilization often requires advanced programming skills. The growing complexity of large databases and unstructured data presents challenges for clinicians who need programming or data analysis expertise to utilize these systems directly. Objective: This study aims to simplify critical care?related database deployment and extraction via large language models. Methods: The development of this platform was a 2-step process. First, we enabled automated database deployment using Docker container technology, with incorporated web-based analytics interfaces Metabase and Superset. Second, we developed the intensive care unit?generative pretrained transformer (ICU-GPT), a large language model fine-tuned on intensive care unit (ICU) data that integrated LangChain and Microsoft AutoGen. Results: The automated deployment platform was designed with user-friendliness in mind, enabling clinicians to deploy 1 or multiple databases in local, cloud, or remote environments without the need for manual setup. After successfully overcoming GPT?s token limit and supporting multischema data, ICU-GPT could generate Structured Query Language (SQL) queries and extract insights from ICU datasets based on request input. A front-end user interface was developed for clinicians to achieve code-free SQL generation on the web-based client. Conclusions: By harnessing the power of our automated deployment platform and ICU-GPT model, clinicians are empowered to easily visualize, extract, and arrange critical care?related databases more efficiently and flexibly than manual methods. Our research could decrease the time and effort spent on complex bioinformatics methods and advance clinical research. UR - https://medinform.jmir.org/2025/1/e63216 UR - http://dx.doi.org/10.2196/63216 ID - info:doi/10.2196/63216 ER - TY - JOUR AU - Sophus, I. Amber AU - Mitchell, W. Jason PY - 2025/3/12 TI - Assessment of Fraud Deterrence and Detection Procedures Used in a Web-Based Survey Study With Adult Black Cisgender Women: Description of Lessons Learned and Recommendations JO - JMIR Form Res SP - e59955 VL - 9 KW - Black women KW - HIV KW - fraud deterrence KW - fraud detection KW - web-based research KW - online research KW - data integrity KW - data collection KW - survey N2 - Background: Online research studies enable engagement with more Black cisgender women in health-related research. However, fraudulent data collection responses in online studies raise important concerns about data integrity, particularly when incentives are involved. Objective: The purpose of this study was to assess the strengths and limitations of fraud deterrence and detection procedures implemented in an incentivized, cross-sectional, online study about HIV prevention and sexual health with Black cisgender women living in Texas. Methods: Data for this study came from a cross-sectional web-based survey that examined factors associated with potential pre-exposure prophylaxis use among a convenience sample of adult Black cisgender women from 3 metropolitan areas in Texas. Each eligibility screener and associated survey entry was evaluated using 4 fraud deterrence features and 7 fraud detection benchmarks with corresponding decision rules. Results: A total of 5862 respondents provided consent and initiated the eligibility screener, of whom 2150 (36.68%) were ineligible for not meeting the inclusion criteria, and 131 (2.23%) completed less than 80% of the survey and were removed from further consideration. Other entries were removed for not passing level 1 fraud deterrent safeguards: duplicate entries with the same IP address (388/5862, 6.62%), same telephone number (69/5862, 1.18%), same email address (114/5862, 1.94%), and same telephone number and email address (17/5862, 0.29%). Of the remaining 2993 entries, 1652 entries were removed for not passing the first 2 items of the level 2 fraud detection benchmarks: screeners and surveys with latitude and longitude coordinates outside of the United States (347/2993, 11.59%) and survey completion time of less than 10 minutes (1305/2993, 43.6%). Of the remaining 1341 entries, 130 (9.69%) passed all 5 of the remaining level 2 data validation benchmarks, and 763 (56.89%) entries were removed due to passing less than 3. An additional 33.4% (423/1341) entries were removed after passing 4 of the 5 remaining validation benchmarks, being contacted to verify survey information, and not providing legitimate contact information or being unable to confirm personal information. The final enrolled sample in this online study consisted of 155 respondents who provided consent, were deemed eligible, and passed fraud deterrence features and fraud detection benchmarks. In this paper, we discuss the lessons learned and provide recommendations for leveraging available features in survey software programs to help deter bots and enhance fraud detection procedures beyond relying on survey software options. Conclusions: Effectively identifying fraudulent responses in online surveys is an ongoing challenge. The data validation approach used in this study establishes a robust protocol for identifying genuine participants, thereby contributing to the removal of false data from study findings. By sharing experiences and implementing thorough fraud deterrence and detection protocols, researchers can maintain data validity and contribute to best practices in web-based research. UR - https://formative.jmir.org/2025/1/e59955 UR - http://dx.doi.org/10.2196/59955 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59955 ER - TY - JOUR AU - Park, Suhyun AU - Marquard, L. Jenna AU - Austin, R. Robin AU - Martin, L. Christie AU - Pieczkiewicz, S. David AU - Delaney, W. Connie PY - 2025/3/11 TI - Exploratory Co-Design on Electronic Health Record Nursing Summaries: Case Study JO - JMIR Form Res SP - e68906 VL - 9 KW - electronic health records KW - interview KW - nurses KW - user-computer interface KW - co-design N2 - Background: Although electronic health record nursing summaries aim to provide a concise overview of patient data, they often fall short of meeting nurses? information needs, leading to underutilization. This gap arises from a lack of involvement of nurses in the design of health information technologies. Objective: The purpose of this exploratory co-design case study was to solicit insights from nurses regarding nursing summary design considerations, including key information types and the preferred design prototype. Methods: We recruited clinical nurses (N=33) from 7 inpatient units at a university hospital in the Midwestern United States using a purposive sampling method. We used images from a simulated nursing summary to generate visual card versions of the 46 information types currently included in an electronic health record vendor?generated nursing summary. Participants selected which cards to include and arranged them in their designs based on their perceived relevance of the information types to the summary and their preferred reading layout. The nurses? perceived relevance of information types to the summary was analyzed by quantifying the frequency of included cards, while the nurses? preferred reading layout was analyzed by quantifying the occurrence of closely paired cards to identify common groupings. After participants evaluated the information type cards, debriefing interviews were conducted and analyzed thematically to explore their rationales for the desired content and its arrangement. Results: The participants demonstrated a high level of engagement in the activities. On average, all 33 participants included 61% (n=28) of the total information types (n=46). The most frequently included cards were ?unit specimen? (results of the analysis of body fluid, tissue, or urine), ?activity,? ?diet,? and ?hospital problems,? each included by 33 participants. Participants most frequently preferred adjacency of the following pairs: ?activity? and ?diet? (paired by 26 participants; 79%) and ?notes to physicians? and ?notes to treatment team? (paired by 25 participants; 76%). Participants preferred arranging the cards to improve information accessibility, focusing on key information types. Conclusions: Involving nurses in the co-design process may result in more useful and usable designs, thereby reducing the time required to navigate nursing summaries. Future work should include refining and evaluating prototypes based on the designs created by the nurses. UR - https://formative.jmir.org/2025/1/e68906 UR - http://dx.doi.org/10.2196/68906 ID - info:doi/10.2196/68906 ER - TY - JOUR AU - Dai, Zhang-Yi AU - Wang, Fu-Qiang AU - Shen, Cheng AU - Ji, Yan-Li AU - Li, Zhi-Yang AU - Wang, Yun AU - Pu, Qiang PY - 2025/3/11 TI - Accuracy of Large Language Models for Literature Screening in Thoracic Surgery: Diagnostic Study JO - J Med Internet Res SP - e67488 VL - 27 KW - accuracy KW - large language models KW - meta-analysis KW - literature screening KW - thoracic surgery N2 - Background: Systematic reviews and meta-analyses rely on labor-intensive literature screening. While machine learning offers potential automation, its accuracy remains suboptimal. This raises the question of whether emerging large language models (LLMs) can provide a more accurate and efficient approach. Objective: This paper evaluates the sensitivity, specificity, and summary receiver operating characteristic (SROC) curve of LLM-assisted literature screening. Methods: We conducted a diagnostic study comparing the accuracy of LLM-assisted screening versus manual literature screening across 6 thoracic surgery meta-analyses. Manual screening by 2 investigators served as the reference standard. LLM-assisted screening was performed using ChatGPT-4o (OpenAI) and Claude-3.5 (Anthropic) sonnet, with discrepancies resolved by Gemini-1.5 pro (Google). In addition, 2 open-source, machine learning?based screening tools, ASReview (Utrecht University) and Abstrackr (Center for Evidence Synthesis in Health, Brown University School of Public Health), were also evaluated. We calculated sensitivity, specificity, and 95% CIs for the title and abstract, as well as full-text screening, generating pooled estimates and SROC curves. LLM prompts were revised based on a post hoc error analysis. Results: LLM-assisted full-text screening demonstrated high pooled sensitivity (0.87, 95% CI 0.77-0.99) and specificity (0.96, 95% CI 0.91-0.98), with the area under the curve (AUC) of 0.96 (95% CI 0.94-0.97). Title and abstract screening achieved a pooled sensitivity of 0.73 (95% CI 0.57-0.85) and specificity of 0.99 (95% CI 0.97-0.99), with an AUC of 0.97 (95% CI 0.96-0.99). Post hoc revisions improved sensitivity to 0.98 (95% CI 0.74-1.00) while maintaining high specificity (0.98, 95% CI 0.94-0.99). In comparison, the pooled sensitivity and specificity of ASReview tool-assisted screening were 0.58 (95% CI 0.53-0.64) and 0.97 (95% CI 0.91-0.99), respectively, with an AUC of 0.66 (95% CI 0.62-0.70). The pooled sensitivity and specificity of Abstrackr tool-assisted screening were 0.48 (95% CI 0.35-0.62) and 0.96 (95% CI 0.88-0.99), respectively, with an AUC of 0.78 (95% CI 0.74-0.82). A post hoc meta-analysis revealed comparable effect sizes between LLM-assisted and conventional screening. Conclusions: LLMs hold significant potential for streamlining literature screening in systematic reviews, reducing workload without sacrificing quality. Importantly, LLMs outperformed traditional machine learning-based tools (ASReview and Abstrackr) in both sensitivity and AUC values, suggesting that LLMs offer a more accurate and efficient approach to literature screening. UR - https://www.jmir.org/2025/1/e67488 UR - http://dx.doi.org/10.2196/67488 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67488 ER - TY - JOUR AU - Frey, Anna-Lena AU - Matei, Diana AU - Phillips, Ben AU - McCabe, Adam AU - Fuller, Rachel AU - Laibarra, Begoña AU - Alonso, Laura AU - de la Hoz, Victor AU - Pratdepadua Bufill, Carme AU - Llebot Casajuana, Berta AU - D'Avenio, Giuseppe AU - Sottile, Angelo Pier AU - Rocchi, Melania Laura AU - Errera, Matteo AU - Laaissaoui, Yasmine AU - Cardinal, Michael AU - Kok, Menno AU - Hoogendoorn, Petra PY - 2025/3/10 TI - Testing and Iterative Improvement of the CEN ISO/TS 82304-2 Health App Quality Assessment: Pilot Interrater Reliability Study JO - JMIR Form Res SP - e64565 VL - 9 KW - health apps KW - mobile health KW - digital health KW - quality evaluation KW - assessment framework KW - interrater reliability N2 - Background: With the increasing use of health apps and ongoing concerns regarding their safety, effectiveness, and data privacy, numerous health app quality assessment frameworks have emerged. However, assessment initiatives experience difficulties scaling, and there is currently no comprehensive, consistent, internationally recognized assessment framework. Therefore, health apps often need to undergo several quality evaluations to enter different markets, leading to duplication of work. The CEN ISO/TS 82304?2 health app assessment seeks to address this issue, aiming to provide an internationally accepted quality evaluation through a network of assessment organizations located in different countries. Objective: This study aimed to develop and evolve the draft CEN ISO/TS 82304-2 assessment handbook and developer guidance by testing them across organizations in several countries. Methods: Assessment organizations from 5 countries were engaged to evaluate 24 health apps using the evolving CEN ISO/TS 82304-2 assessment across 3 evaluation rounds. The information submitted by a given health app developer was evaluated by 2 assessment organizations, and interrater reliability was examined. In addition, app developers and assessors were asked to report how much time they spent on information collation or evaluation and to rate the clarity of the developer guidance or assessor handbook, respectively. The collected data were used to iteratively improve the handbook and guidance between rounds. Results: The interrater reliability between assessment organizations improved from round 1 to round 2 and stayed relatively stable between rounds 2 and 3, with 80% (55/69) of assessment questions demonstrating moderate or better (Gwet AC1>0.41) agreement in round 3. The median time required by developers to prepare the assessment information was 8 hours and 59 minutes (IQR 5.7-27.1 hours) in round 3, whereas assessors reported a median evaluation time of 8 hours and 46 minutes (IQR 7.1-11.0 hours). The draft guidance and handbook were generally perceived as clear, with a median round-3 clarity rating of 1.73 (IQR 1.64-1.90) for developers and 1.78 (IQR 1.71-1.89) for assessors (0=?very unclear?, 1=?somewhat unclear?, and 2=?completely clear?). Conclusions: To our knowledge, this is the first study to examine the consistency of health app evaluations across organizations located in different countries. Given that the CEN ISO/TS 82304-2 guidance and handbook are still under development, the interrater reliability findings observed at this early stage are promising, and this study provided valuable information for further refinement of the assessment. This study marks an important first step toward establishing the CEN ISO/TS 82304-2 assessment as a consistent, cross-national health app evaluation. It is envisioned that the assessment will ultimately help avoid duplication of work, prevent inequities by facilitating access to smaller markets for developers, and build trust among users, thereby increasing the adoption of high-quality health apps. UR - https://formative.jmir.org/2025/1/e64565 UR - http://dx.doi.org/10.2196/64565 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/64565 ER - TY - JOUR AU - Laurent, Maxence AU - Jaccard, Arnaud AU - Suppan, Laurent AU - Erriquez, Elio AU - Good, Xavier AU - Golay, Eric AU - Jaccard, Dominique AU - Suppan, Mélanie PY - 2025/3/7 TI - HUMAn, a Real-Time Evolutive Patient Model for Major Incident Simulation: Development and Validation Study JO - JMIR Form Res SP - e66201 VL - 9 KW - physiological model KW - mathematical model KW - computer simulation KW - major incident management KW - emergency medicine KW - mass casualties KW - healthcare professional education KW - professional education KW - continuing education N2 - Background: Major incidents correspond to any situation where the location, number, severity, or type of casualties requires extraordinary resources. Major incident management must be efficient to save as many lives as possible. As any paramedic or emergency medical technician may unexpectedly have to respond to major incidents, regular training is mandatory. Those trainings usually include simulations. The vast majority of major incident simulations are limited by the fact that simulated patients do not evolve during the simulation, regardless of the time elapsed and treatment decisions. Therefore, most simulations fail to incorporate the critical temporal effect of decision-making. Objective: This study aimed to develop and validate a simplified mathematical model of physiology, capable of plausibly simulating the real-time evolution of several injuries. Methods: A modified version of the user-centered design framework, including a relevance, development, and validation phase, was used to define the development process of the physiological model. A 12-member design and development team was established, including prehospital physicians, paramedics, and computer scientists. To determine whether the developed model was clinically realistic, 15 experienced professionals working in the prehospital field participated in the validation phase. They were asked to rate clinical and physiological parameters according to a 5-point Likert scale ranging from 1 (impossible) to 5 (absolutely realistic). Results: The design and development team led to the development of the HUMAn model (Human is an Uncomplicated Model of Anatomy). During the relevance phase, the team defined the needed features of the model: clinically realistic, able to compute the evolution of prehospital vital signs, yet simple enough to allow real-time computation for several simulated patients on regular computers or tablets. During the development phase, iterations led to the development of a heart-lung-brain interaction model coupled to functional blocks representing the main anatomical body parts. During the validation phase, the evolution of nine simulated patients presenting pathologies devised to test the different systems and their interactions was assessed. Overall, clinical parameters of all patients had a median rating of 5 (absolutely realistic; IQR 4-5). Most (n=52, 96%) individual clinical parameters had a median rating of 5, the remainder (n=2, 4%) being rated 4. Overall physiological parameters of all patients had a median rating of 5 (absolutely realistic; IQR 3-5). The majority of individual physiological parameters (n=43, 79%) had a median rating of 5, with (n=9, 17%) rated 4, and only (n=2 ,4%) rated 3. Conclusions: A simplified model of trauma patient evolution was successfully created and deemed clinically realistic by experienced clinicians. This model should now be included in computer-based simulations and its impact on the teaching of major incident management assessed through randomized trials. UR - https://formative.jmir.org/2025/1/e66201 UR - http://dx.doi.org/10.2196/66201 ID - info:doi/10.2196/66201 ER - TY - JOUR AU - Rajaram, Akshay AU - Judd, Michael AU - Barber, David PY - 2025/3/7 TI - Deep Learning Models to Predict Diagnostic and Billing Codes Following Visits to a Family Medicine Practice: Development and Validation Study JO - JMIR AI SP - e64279 VL - 4 KW - machine learning KW - ML KW - artificial intelligence KW - algorithm KW - predictive model KW - predictive analytics KW - predictive system KW - family medicine KW - primary care KW - family doctor KW - family physician KW - income KW - billing code KW - electronic notes KW - electronic health record KW - electronic medical record KW - EMR KW - patient record KW - health record KW - personal health record N2 - Background: Despite significant time spent on billing, family physicians routinely make errors and miss billing opportunities. In other disciplines, machine learning models have predicted Current Procedural Terminology codes with high accuracy. Objective: Our objective was to derive machine learning models capable of predicting diagnostic and billing codes from notes recorded in the electronic medical record. Methods: We conducted a retrospective algorithm development and validation study involving an academic family medicine practice. Visits between July 1, 2015, and June 30, 2020, containing a physician-authored note and an invoice in the electronic medical record were eligible for inclusion. We trained 2 deep learning models and compared their predictions to codes submitted for reimbursement. We calculated accuracy, recall, precision, F1-score, and area under the receiver operating characteristic curve. Results: Of the 245,045 visits eligible for inclusion, 198,802 (81%) were included in model development. Accuracy was 99.8% and 99.5% for the diagnostic and billing code models, respectively. Recall was 49.4% and 70.3% for the diagnostic and billing code models, respectively. Precision was 55.3% and 76.7% for the diagnostic and billing code models, respectively. The area under the receiver operating characteristic curve was 0.983 for the diagnostic code model and 0.993 for the billing code model. Conclusions: We developed models capable of predicting diagnostic and billing codes from electronic notes following visits to a family medicine practice. The billing code model outperformed the diagnostic code model in terms of recall and precision, likely due to fewer codes being predicted. Work is underway to further enhance model performance and assess the generalizability of these models to other family medicine practices. UR - https://ai.jmir.org/2025/1/e64279 UR - http://dx.doi.org/10.2196/64279 ID - info:doi/10.2196/64279 ER - TY - JOUR AU - Nightingale, L. Chandylen AU - Dressler, V. Emily AU - Kepper, Maura AU - Klepin, D. Heidi AU - Lee, Craddock Simon AU - Smith, Sydney AU - Aguilar, Aylin AU - Wiseman, D. Kimberly AU - Sohl, J. Stephanie AU - Wells, J. Brian AU - DeMari, A. Joseph AU - Throckmorton, Alyssa AU - Kulbacki, W. Lindsey AU - Hanna, Jenny AU - Foraker, E. Randi AU - Weaver, E. Kathryn PY - 2025/3/6 TI - Oncology Provider and Patient Perspectives on a Cardiovascular Health Assessment Tool Used During Posttreatment Survivorship Care in Community Oncology (Results from WF-1804CD): Mixed Methods Observational Study JO - J Med Internet Res SP - e65152 VL - 27 KW - cancer KW - cardiovascular health KW - cancer survivors KW - community oncology KW - electronic health record integration KW - provider acceptability KW - patient-provider KW - assessment tool KW - electronic health records KW - clinical decision support KW - surveys KW - interviews KW - survivors KW - Automated Heart-Health Assessment N2 - Background: Most survivors of cancer have multiple cardiovascular risk factors, increasing their risk of poor cardiovascular and cancer outcomes. The Automated Heart-Health Assessment (AH-HA) tool is a novel electronic health record clinical decision support tool based on the American Heart Association?s Life?s Simple 7 cardiovascular health metrics to promote cardiovascular health assessment and discussion in outpatient oncology. Before proceeding to future implementation trials, it is critical to establish the acceptability of the tool among providers and survivors. Objective: This study aims to assess provider and survivor acceptability of the AH-HA tool and provider training at practices randomized to the AH-HA tool arm within WF-1804CD. Methods: Providers (physicians, nurse practitioners, and physician assistants) completed a survey to assess the acceptability of the AH-HA training, immediately following training. Providers also completed surveys to assess AH-HA tool acceptability and potential sustainability. Tool acceptability was assessed after 30 patients were enrolled at the practice with both a survey developed for the study as well as with domains from the Unified Theory of Acceptance and Use of Technology survey (performance expectancy, effort expectancy, attitude toward using technology, and facilitating conditions). Semistructured interviews at the end of the study captured additional provider perceptions of the AH-HA tool. Posttreatment survivors (breast, prostate, colorectal, endometrial, and lymphomas) completed a survey to assess the acceptability of the AH-HA tool immediately after the designated study appointment. Results: Providers (n=15) reported high overall acceptability of the AH-HA training (mean 5.8, SD 1.0) and tool (mean 5.5, SD 1.4); provider acceptability was also supported by the Unified Theory of Acceptance and Use of Technology scores (eg, effort expectancy: mean 5.6, SD 1.5). Qualitative data also supported provider acceptability of different aspects of the AH-HA tool (eg, ?It helps focus the conversation and give the patient a visual of continuum of progress?). Providers were more favorable about using the AH-HA tool for posttreatment survivorship care. Enrolled survivors (n=245) were an average of 4.4 (SD 3.7) years posttreatment. Most survivors reported that they strongly agreed or agreed that they liked the AH-HA tool (n=231, 94.3%). A larger proportion of survivors with high health literacy strongly agreed or agreed that it was helpful to see their heart health score (n=161, 98.2%) compared to survivors with lower health literacy scores (n=68, 89.5%; P=.005). Conclusions: Quantitative surveys and qualitative interview data both demonstrate high acceptability of the AH-HA tool among both providers and survivors. Although most survivors found it helpful to see their heart health score, there may be room for improving communication with survivors who have lower health literacy. Trial Registration: ClinicalTrials.gov NCT03935282; http://clinicaltrials.gov/ct2/show/NCT03935282 International Registered Report Identifier (IRRID): RR2-https://doi-org.wake.idm.oclc.org/10.1016/j.conctc.2021.100808 UR - https://www.jmir.org/2025/1/e65152 UR - http://dx.doi.org/10.2196/65152 UR - http://www.ncbi.nlm.nih.gov/pubmed/39854647 ID - info:doi/10.2196/65152 ER - TY - JOUR AU - Fekete, Tibor János AU - Gy?rffy, Balázs PY - 2025/3/6 TI - MetaAnalysisOnline.com: Web-Based Tool for the Rapid Meta-Analysis of Clinical and Epidemiological Studies JO - J Med Internet Res SP - e64016 VL - 27 KW - statistics KW - pharmacology KW - treatment KW - epidemiology KW - fixed effect model KW - random effect model KW - hazard rate KW - response rate KW - clinical trial KW - funnel plot KW - z score plot N2 - Background: A meta-analysis is a quantitative, formal study design in epidemiology and clinical medicine that systematically integrates and quantitatively synthesizes findings from multiple independent studies. This approach not only enhances statistical power but also enables the exploration of effects across diverse populations and helps resolve controversies arising from conflicting studies. Objective: This study aims to develop and implement a user-friendly tool for conducting meta-analyses, addressing the need for an accessible platform that simplifies the complex statistical procedures required for evidence synthesis while maintaining methodological rigor. Methods: The platform available at MetaAnalysisOnline.com enables comprehensive meta-analyses through an intuitive web interface, requiring no programming expertise or command-line operations. The system accommodates diverse data types including binary (total and event numbers), continuous (mean and SD), and time-to-event data (hazard rates with CIs), while implementing both fixed-effect and random-effect models using established statistical approaches such as DerSimonian-Laird, Mantel-Haenszel, and inverse variance methods for effect size estimation and heterogeneity assessment. Results: In addition to statistical tests, graphical representations including the forest plot, the funnel plot, and the z score plot can be drawn. A forest plot is highly effective in illustrating heterogeneity and pooled results. The risk of publication bias can be revealed by a funnel plot. A z score plot provides a visual assessment of whether more research is needed to establish a reliable conclusion. All the discussed models and visualization options are integrated into the registration-free web-based portal. Leveraging MetaAnalysisOnline.com's capabilities, we examined treatment-related adverse events in patients with cancer receiving perioperative anti?PD-1 immunotherapy through a systematic review encompassing 10 studies with 8099 total participants. Meta-analysis revealed that anti?PD-1 therapy doubled the risk of adverse events (risk ratio 2.15, 95% CI 1.39-3.32), with significant between-study heterogeneity (I2=95%) and publication bias detected through the Egger test (P=.02). While these findings suggest increased toxicity associated with anti?PD-1 treatment, the z score analysis indicated that additional studies are needed for definitive conclusions. Conclusions: In summary, the web-based tool aims to bridge the void for clinical and life science researchers by offering a user-friendly alternative for the swift and reproducible meta-analysis of clinical and epidemiological trials. UR - https://www.jmir.org/2025/1/e64016 UR - http://dx.doi.org/10.2196/64016 UR - http://www.ncbi.nlm.nih.gov/pubmed/39928123 ID - info:doi/10.2196/64016 ER - TY - JOUR AU - Muscat, M. Danielle AU - Hinton, Rachael AU - Kuruvilla, Shyama AU - Nutbeam, Don PY - 2025/3/6 TI - ?Your Life, Your Health: Tips and Information for Health and Well-Being?: Development of a World Health Organization Digital Resource to Support Universal Access to Trustworthy Health Information JO - JMIR Form Res SP - e57881 VL - 9 KW - health communication KW - health literacy KW - consumer health information KW - digital health KW - universal health care N2 - Background: Access to trustworthy, understandable, and actionable health information is a key determinant of health and is an essential component of universal health coverage and primary health care. The World Health Organization has developed a new digital resource for the general public to improve health and well-being across different life phases and to support people in caring for themselves, their families, and their communities. The goal was to make trustworthy health information accessible, understandable, and actionable for the general public in a digital format and at the global scale. Objective: The aim of this paper was to describe the multistage approach and methodology used to develop the resource Your life, your health: Tips and information for health and well-being (hereafter, Your life, your health). Methods: A 5-step process was used to develop Your life, your health, including (1) reviewing and synthesizing existing World Health Organization technical guidance, member state health and health literacy plans, and international human rights frameworks to identify priority messages; (2) developing messages and graphics that are accessible, understandable, and actionable for the public using health literacy principles; (3) engaging with experts and stakeholders to refine messages and message delivery; (4) presenting priority content in an accessible digital format; and (5) adapting the resource based on feedback and new evidences. Results: The Your life, your health online resource adopts a life-course approach to organize health information based on priority actions and rights that support peoples? health and well-being across different life stages and specific health topics. The resource promotes health literacy by offering advice on asking questions to health workers, making informed decisions about personal and family health, and effectively using digital media to obtain reliable health information. Additionally, it reflects the ambitions of the Sustainable Development Goals by providing essential information on the social determinants of health and clarifies the distinct roles of individuals, frontline workers, governments, and the media in promoting and protecting health. Conclusions: Making health information available?including to the public?is an essential step in strengthening the global health information system. The development process for the Your life, your health online resource outlined in this article offers a structured approach to translate technical health guidelines into accessible, understandable, and actionable health information for the general public. UR - https://formative.jmir.org/2025/1/e57881 UR - http://dx.doi.org/10.2196/57881 ID - info:doi/10.2196/57881 ER - TY - JOUR AU - Zola Matuvanga, Trésor AU - Paviotti, Antea AU - Bikioli Bolombo, Freddy AU - Lemey, Gwen AU - Larivière, Ynke AU - Salloum, Maha AU - Isekah Osang'ir, Bernard AU - Esanga Longomo, Emmanuel AU - Milolo, Solange AU - Matangila, Junior AU - Maketa, Vivi AU - Mitashi, Patrick AU - Van Damme, Pierre AU - Muhindo-Mavoko, Hypolite AU - Van geertruyden, Jean-Pierre PY - 2025/3/6 TI - Long-Term Experiences of Health Care Providers Using Iris Scanning as an Identification Tool in a Vaccine Trial in the Democratic Republic of the Congo: Qualitative Study JO - JMIR Form Res SP - e54921 VL - 9 KW - iris scan KW - vaccine trial KW - iris KW - perception KW - experience KW - views KW - biometric identification KW - Democratic Republic of the Congo N2 - Background: Iris scanning has increasingly been used for biometric identification over the past decade, with continuous advancements and expanding applications. To better understand the acceptability of this technology, we report the long-term experiences of health care providers and frontline worker participants with iris scanning as an identification tool in the EBL2007 Ebola vaccine trial conducted in the Democratic Republic of the Congo. Objective: This study aims to document the long-term experiences of using iris scanning for identity verification throughout the vaccine trial. Methods: Two years after the start of the EBL2007 vaccine trial (February to March 2022), 69 trial participants?including nurses, first aid workers, midwives, and community health workers?were interviewed through focus group discussions. Additionally, 13 in-depth individual interviews were conducted with physicians involved in the trial, iris scan operators, trial staff physicians, and trial participants who declined iris scanning. Qualitative content analysis was used to identify key themes. Results: Initially, interviewees widely accepted the iris scan and viewed it as a distinctive tool for identifying participants in the EBL2007 vaccine trial. However, over time, perceptions became less favorable. Some participants expressed concerns that their vision had diminished shortly after using the tool and continued to decline until the end of the study. Others reported experiencing perceived vision loss long after the trial had concluded. However, no vision impairment was reported as an adverse event or assessed in the trial as being linked to the iris scan, which uses a previously certified safe infrared light for scanning. Conclusions: Our findings highlight the sustained acceptability and perceived high accuracy of the iris scan tool for uniquely identifying adult participants in a vaccine trial over time. Continued efforts to systematically disseminate and reinforce information about the function and safety of this technology are essential. Clearly presenting iris scanning as a safe procedure could help dispel misconceptions, concerns, and perceived risks among potential users in vaccine trials. UR - https://formative.jmir.org/2025/1/e54921 UR - http://dx.doi.org/10.2196/54921 UR - http://www.ncbi.nlm.nih.gov/pubmed/40053756 ID - info:doi/10.2196/54921 ER - TY - JOUR AU - Dewan, Ananya AU - Eifler, M. AU - Hood, Amelia AU - Sanchez, William AU - Gross, Marielle PY - 2025/3/5 TI - Building a Decentralized Biobanking App for Research Transparency and Patient Engagement: Participatory Design Study JO - JMIR Hum Factors SP - e59485 VL - 12 KW - mobile health KW - mHealth KW - application KW - smartphone KW - digital health KW - digital intervention KW - participatory design KW - biobanking KW - research transparency KW - donation KW - patient-derived biospecimens KW - plain language communications KW - patient education N2 - Background: Patient-derived biospecimens are invaluable tools in biomedical research. Currently, there are no mechanisms for patients to follow along and learn about the uses of their donated samples. Incorporating patients as stakeholders and meaningfully engaging them in biomedical research first requires transparency of research activities. Objective: In this paper, we describe the use of participatory design methods to build a decentralized biobanking ?de-bi? mobile app where patients could learn about biobanking, track their specimens, and engage with ongoing research via patient-friendly interfaces overlaying institutional biobank databases, initially developed for a breast cancer use case. Methods: This research occurred in 2 phases. In phase 1, we designed app screens from which patients could learn about ongoing research involving their samples. We embedded these screens in a survey (n=94) to gauge patients? interests regarding types of feedback and engagement opportunities; survey responses were probed during 6 comprehensive follow-up interviews. We then held an immersive participatory design workshop where participants (approximately 50) provided general feedback about our approach, with an embedded codesign workshop where a subset (n=15) provided targeted feedback on screen designs. For phase 2, we refined user interfaces and developed a functional app prototype in consultation with institutional stakeholders to ensure regulatory compliance, workflow compatibility, and composability with local data architectures. We presented the app at a second workshop, where participants (n=25, across 9 groups) shared thoughts on the app?s usability and design. In this phase, we conducted cognitive walkthroughs (n=13) to gain in-depth feedback on in-app task navigation. Results: Most of the survey participants (61/81, 75%) were interested in learning the outcomes of research on their specimens, and 49% (41/83) were interested in connecting with others with the same diagnosis. Participants (47/60, 78%) expressed strong interest in receiving patient-friendly summaries of scientific information from scientists using their biospecimens. The first design workshop identified confusion in terminology and data presentation (eg, 9/15, 60% of co-designers were unclear on the biospecimens ?in use?), though many appreciated the ability to view their personal biospecimens (7/15, 47%), and most were excited about connecting with others (12/15, 80%). In the second workshop, all groups found the app?s information valuable. Moreover, 44% (5/9) noted they did not like the onboarding process, which was echoed in cognitive walkthroughs. Walkthroughs further confirmed interest in biospecimen tracking, and 23% (3/13) had confusion about not finding any of their biospecimens in the app. These findings guided refinements in onboarding, design, and user experience. Conclusions: Designing a patient-facing app that displays information about biobanked specimens can facilitate greater transparency and engagement in biomedical research. Co-designing the app with patient stakeholders confirmed interest in learning about biospecimens and related research, improved presentation of data, and ensured usability of the app in preparation for a pilot study. UR - https://humanfactors.jmir.org/2025/1/e59485 UR - http://dx.doi.org/10.2196/59485 UR - http://www.ncbi.nlm.nih.gov/pubmed/40053747 ID - info:doi/10.2196/59485 ER - TY - JOUR AU - Khundrakpam, Budhachandra AU - Segado, Melanie AU - Pazdera, Jesse AU - Gagnon Shaigetz, Vincent AU - Granek, A. Joshua AU - Choudhury, Nusrat PY - 2025/3/4 TI - An Integrated Platform Combining Immersive Virtual Reality and Physiological Sensors for Systematic and Individualized Assessment of Stress Response (bWell): Design and Implementation Study JO - JMIR Form Res SP - e64492 VL - 9 KW - virtual reality KW - stress KW - physiological response KW - NASA-Task Load Index KW - cognitive demand KW - physical demand KW - vagal tone KW - heart rate variability N2 - Background: Stress is a pervasive issue in modern society, manifesting in various forms such as emotional, physical, and work-related stress, each with distinct impacts on individuals and society. Traditional stress studies often rely on psychological, performance, or social tests; however, recently, immersive virtual reality (VR), which provides a sense of presence and natural interaction, offers the opportunity to simulate real-world tasks and stressors in controlled environments. Despite its potential, the use of VR to investigate the multifaceted manifestations of stress has not been thoroughly explored. Objective: This study aimed to explore the feasibility of using a VR-based platform, bWell, to elicit multifaceted stress responses and measure the resulting behavioral and physiological changes. Specifically, we aimed to design various VR stress exercises based on neurocardiac models to systematically test cardiac functioning within specific contexts of self-regulation (executive functioning, physical efforts, and emotional regulation). Methods: The development process adhered to guidelines for VR clinical trials and complex health interventions, encompassing 3 phases: preparation, development, and verification. The preparation phase involved a comprehensive literature review to establish links between stress, the heart, and the brain, leading to the formulation of a conceptual model based on the Neurovisceral Integration Model (NVIM) and Vagal Tank Theory (VTT). The development phase involved designing VR exercises targeting specific stressors and integrating physiological sensors such as photoplethysmography (PPG) and electromyography (EMG) to capture heart rate variability (HRV) and facial expressions. The verification phase, conducted with a small number of trials, aimed to design a study and implement a workflow for testing the feasibility, acceptability, and tolerability of the VR exercises. In addition, the potential for capturing physiological measures along with subjective ratings of stress for specific dimensions was assessed. Results: Verification trials demonstrated that the VR exercises were well tolerated, with negligible cybersickness and high user engagement. The different VR exercises successfully elicited the intended stress demands, along with the physiological responses. Conclusions: The study presents a novel VR-based experimental setup that allows a systematic and individualized assessment of stress responses, paving the way for future research to identify features that confer stress resilience and help individuals manage stress effectively. While our conceptual model highlights the role of HRV in providing valuable insights into stress responses, future research will involve multivariate and machine learning analyses to predict individual stress responses based on comprehensive sensor data, including EMG and the VR-based behavioral data, ultimately guiding personalized stress management interventions. UR - https://formative.jmir.org/2025/1/e64492 UR - http://dx.doi.org/10.2196/64492 UR - http://www.ncbi.nlm.nih.gov/pubmed/40053709 ID - info:doi/10.2196/64492 ER - TY - JOUR AU - Kim, Kwanho AU - Kim, Soojong PY - 2025/3/4 TI - Large Language Models? Accuracy in Emulating Human Experts? Evaluation of Public Sentiments about Heated Tobacco Products on Social Media: Evaluation Study JO - J Med Internet Res SP - e63631 VL - 27 KW - heated tobacco products KW - artificial intelligence KW - large language models KW - social media KW - sentiment analysis KW - ChatGPT KW - generative pre-trained transformer KW - GPT KW - LLM KW - NLP KW - natural language processing KW - machine learning KW - language model KW - sentiment KW - evaluation KW - tobacco KW - alternative KW - prevention KW - nicotine KW - OpenAI N2 - Background: Sentiment analysis of alternative tobacco products discussed on social media is crucial in tobacco control research. Large language models (LLMs) are artificial intelligence models that were trained on extensive text data to emulate the linguistic patterns of humans. LLMs may hold the potential to streamline the time-consuming and labor-intensive process of human sentiment analysis. Objective: This study aimed to examine the accuracy of LLMs in replicating human sentiment evaluation of social media messages relevant to heated tobacco products (HTPs). Methods: GPT-3.5 and GPT-4 Turbo (OpenAI) were used to classify 500 Facebook (Meta Platforms) and 500 Twitter (subsequently rebranded X) messages. Each set consisted of 200 human-labeled anti-HTPs, 200 pro-HTPs, and 100 neutral messages. The models evaluated each message up to 20 times to generate multiple response instances reporting its classification decisions. The majority of the labels from these responses were assigned as a model?s decision for the message. The models? classification decisions were then compared with those of human evaluators. Results: GPT-3.5 accurately replicated human sentiment evaluation in 61.2% of Facebook messages and 57% of Twitter messages. GPT-4 Turbo demonstrated higher accuracies overall, with 81.7% for Facebook messages and 77% for Twitter messages. GPT-4 Turbo?s accuracy with 3 response instances reached 99% of the accuracy achieved with 20 response instances. GPT-4 Turbo?s accuracy was higher for human-labeled anti- and pro-HTP messages compared with neutral messages. Most of the GPT-3.5 misclassifications occurred when anti- or pro-HTP messages were incorrectly classified as neutral or irrelevant by the model, whereas GPT-4 Turbo showed improvements across all sentiment categories and reduced misclassifications, especially in incorrectly categorized messages as irrelevant. Conclusions: LLMs can be used to analyze sentiment in social media messages about HTPs. Results from GPT-4 Turbo suggest that accuracy can reach approximately 80% compared with the results of human experts, even with a small number of labeling decisions generated by the model. A potential risk of using LLMs is the misrepresentation of the overall sentiment due to the differences in accuracy across sentiment categories. Although this issue could be reduced with the newer language model, future efforts should explore the mechanisms underlying the discrepancies and how to address them systematically. UR - https://www.jmir.org/2025/1/e63631 UR - http://dx.doi.org/10.2196/63631 UR - http://www.ncbi.nlm.nih.gov/pubmed/40053746 ID - info:doi/10.2196/63631 ER - TY - JOUR AU - Jiang, AnHang AU - Li, Shuang AU - Wang, HuaBin AU - Ni, HaoSen AU - Chen, HongAn AU - Dai, JunHong AU - Xu, XueFeng AU - Li, Mei AU - Dong, Guang-Heng PY - 2025/3/4 TI - Assessing Short-Video Dependence for e-Mental Health: Development and Validation Study of the Short-Video Dependence Scale JO - J Med Internet Res SP - e66341 VL - 27 KW - short-video dependence KW - problematic short-video use KW - cutoff point KW - scale development KW - mental health KW - short video KW - internet addiction KW - latent profile analysis KW - exploratory factor analysis KW - confirmatory factor analysis N2 - Background: Short-video dependence (SVD) has become a significant mental health issue around the world. The lack of scientific tools to assess SVD hampers further advancement in this area. Objective: This study aims to develop and validate a scientific tool to measure SVD levels, ensuring a scientifically determined cutoff point. Methods: We initially interviewed 115 highly engaged short-video users aged 15 to 63 years. Based on the summary of the interview and references to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria for behavioral addictions, we proposed the first version of the short-video dependence scale (SVDS). We then screened the items through item analysis (second version) and extracted common factors using exploratory factor analysis (third version) and confirmatory factor analysis (final version). Convergent validity was tested with other scales (Chinese Internet Addiction Scale [CIAS] and DSM-5). Finally, we tested the validity of the final version in 16,038 subjects and set the diagnostic cutoff point through latent profile analysis and receiver operating characteristic curve analysis. Results: The final version of the SVDS contained 20 items and 4 dimensions, which showed strong structural validity (Kaiser-Meyer-Olkin value=0.94) and internal consistency (Cronbach ?=.93), and good convergent validity (rCIAS=0.61 and rDSM-5=0.68), sensitivity (0.77, 0.83, 0.87, and 0.62 for each of the 4 dimensions), and specificity (0.75, 0.87, 0.80, and 0.79 for each of the 4 dimensions). Additionally, an SVDS score of 58 was determined as the best cutoff score, and latent profile analysis identified a 5-class model for SVD. Conclusions: We developed a tool to measure SVD levels and established a threshold to differentiate dependent users from highly engaged nondependent users. The findings provide opportunities for further research on the impacts of short-video use. UR - https://www.jmir.org/2025/1/e66341 UR - http://dx.doi.org/10.2196/66341 UR - http://www.ncbi.nlm.nih.gov/pubmed/40053762 ID - info:doi/10.2196/66341 ER - TY - JOUR AU - Rinaldi, Eugenia AU - Stellmach, Caroline AU - Thun, Sylvia PY - 2025/3/3 TI - How to Design Electronic Case Report Form (eCRF) Questions to Maximize Semantic Interoperability in Clinical Research JO - Interact J Med Res SP - e51598 VL - 14 KW - case report form KW - CRF KW - interoperability KW - standard data model KW - data format KW - metadata KW - core data elements KW - data quality UR - https://www.i-jmr.org/2025/1/e51598 UR - http://dx.doi.org/10.2196/51598 ID - info:doi/10.2196/51598 ER - TY - JOUR AU - Pickard, Abigail AU - Edwards, Katie AU - Farrow, Claire AU - Haycraft, Emma AU - Blissett, Jacqueline PY - 2025/2/27 TI - Capturing Everyday Parental Feeding Practices and Eating Behaviors of 3- to 5-Year-Old Children With Avid Eating Behavior: Ecological Momentary Assessment Feasibility and Acceptability Study JO - JMIR Form Res SP - e66807 VL - 9 KW - pediatric KW - paediatric KW - child KW - child eating KW - parent feeding KW - parent KW - ecological momentary assessment KW - mHealth KW - mobile health KW - mobile app KW - application KW - smartphone KW - digital KW - digital health KW - digital technology KW - digital intervention N2 - Background: The wide use of smartphones offers large-scale opportunities for real-time data collection methods such as ecological momentary assessment (EMA) to assess how fluctuations in contextual and psychosocial factors influence parents? feeding practices and feeding goals, particularly when feeding children with high food approaches. Objective: The main objectives of this study were to (1) assess parents/caregivers? compliance with EMA procedures administered through a smartphone app and (2) estimate the criterion validity of the EMA to capture children?s eating occasions and parents? feeding practices. Participant adherence, technological challenges, and data quality were used to provide an overview of the real-time dynamics of parental mood, feeding goals, and contextual factors during eating occasions. Methods: Parents in the United Kingdom with a child aged 3 to 5 years who exhibit avid eating behavior were invited to participate in a 10-day EMA study using a smartphone app. Of the 312 invited participants, 122 (39%) parents initiated the EMA study, of which 118 (96.7%) completed the full EMA period and the follow-up feasibility and acceptability survey. Results: Of those parents who completed the EMA study, 104 (87.4%) parents provided at least 7 ?full? days of data (2 signal surveys and 1 event survey), despite 51 parents (43.2%) experiencing technical difficulties. The parents received notifications for morning surveys (69.9% response rate), 3 daily mood surveys (78.7% response rate), and an end-of-day survey (84.6% response rate) on each of the 10 days. Over the EMA period, a total of 2524 child eating/food request surveys were self-initiated by the participants on their smartphones, an average of 2.1 times per day per parent (SD 0.18; min=1.7, max=2.3). The majority of parents felt that the surveys made them more aware of their feelings (105/118, 89%) and activities (93/118, 79%). The frequency of daily food requests estimated by parents at baseline was significantly correlated with the frequency of food requests reported daily during the EMA period (r=0.483, P<.001). However, the number of daily food requests per day estimated at baseline (mean 4.5, SD 1.5) was significantly higher than the number of food requests reported per day during the EMA period (mean 3.7, SD 1.1), (t116=18.8, P<.001). Conclusions: This paper demonstrates the feasibility of employing EMA to investigate the intricate interplay between parental mood, feeding goals, contextual factors, and feeding practices with children exhibiting an avid eating behavior profile. However, the use of EMA needs to be carefully developed and tested with parents? involvement to ensure successful data collection. International Registered Report Identifier (IRRID): RR2-10.2196/55193 UR - https://formative.jmir.org/2025/1/e66807 UR - http://dx.doi.org/10.2196/66807 ID - info:doi/10.2196/66807 ER - TY - JOUR AU - Choo, Seungheon AU - Yoo, Suyoung AU - Endo, Kumiko AU - Truong, Bao AU - Son, Hi Meong PY - 2025/2/27 TI - Advancing Clinical Chatbot Validation Using AI-Powered Evaluation With a New 3-Bot Evaluation System: Instrument Validation Study JO - JMIR Nursing SP - e63058 VL - 8 KW - artificial intelligence KW - patient education KW - therapy KW - computer-assisted KW - computer KW - understandable KW - accurate KW - understandability KW - automation KW - chatbots KW - bots KW - conversational agents KW - emotions KW - emotional KW - depression KW - depressive KW - anxiety KW - anxious KW - nervous KW - nervousness KW - empathy KW - empathetic KW - communication KW - interactions KW - frustrated KW - frustration KW - relationships N2 - Background: The health care sector faces a projected shortfall of 10 million workers by 2030. Artificial intelligence (AI) automation in areas such as patient education and initial therapy screening presents a strategic response to mitigate this shortage and reallocate medical staff to higher-priority tasks. However, current methods of evaluating early-stage health care AI chatbots are highly limited due to safety concerns and the amount of time and effort that goes into evaluating them. Objective: This study introduces a novel 3-bot method for efficiently testing and validating early-stage AI health care provider chatbots. To extensively test AI provider chatbots without involving real patients or researchers, various AI patient bots and an evaluator bot were developed. Methods: Provider bots interacted with AI patient bots embodying frustrated, anxious, or depressed personas. An evaluator bot reviewed interaction transcripts based on specific criteria. Human experts then reviewed each interaction transcript, and the evaluator bot?s results were compared to human evaluation results to ensure accuracy. Results: The patient-education bot?s evaluations by the AI evaluator and the human evaluator were nearly identical, with minimal variance, limiting the opportunity for further analysis. The screening bot?s evaluations also yielded similar results between the AI evaluator and human evaluator. Statistical analysis confirmed the reliability and accuracy of the AI evaluations. Conclusions: The innovative evaluation method ensures a safe, adaptable, and effective means to test and refine early versions of health care provider chatbots without risking patient safety or investing excessive researcher time and effort. Our patient-education evaluator bots could have benefitted from larger evaluation criteria, as we had extremely similar results from the AI and human evaluators, which could have arisen because of the small number of evaluation criteria. We were limited in the amount of prompting we could input into each bot due to the practical consideration that response time increases with larger and larger prompts. In the future, using techniques such as retrieval augmented generation will allow the system to receive more information and become more specific and accurate in evaluating the chatbots. This evaluation method will allow for rapid testing and validation of health care chatbots to automate basic medical tasks, freeing providers to address more complex tasks. UR - https://nursing.jmir.org/2025/1/e63058 UR - http://dx.doi.org/10.2196/63058 ID - info:doi/10.2196/63058 ER - TY - JOUR AU - Scherr, Riley AU - Spina, Aidin AU - Dao, Allen AU - Andalib, Saman AU - Halaseh, F. Faris AU - Blair, Sarah AU - Wiechmann, Warren AU - Rivera, Ronald PY - 2025/2/27 TI - Novel Evaluation Metric and Quantified Performance of ChatGPT-4 Patient Management Simulations for Early Clinical Education: Experimental Study JO - JMIR Form Res SP - e66478 VL - 9 KW - medical school simulations KW - AI in medical education KW - preclinical curriculum KW - ChatGPT KW - ChatGPT-4 KW - medical simulation KW - simulation KW - multimedia KW - feedback KW - medical education KW - medical student KW - clinical education KW - pilot study KW - patient management N2 - Background: Case studies have shown ChatGPT can run clinical simulations at the medical student level. However, no data have assessed ChatGPT?s reliability in meeting desired simulation criteria such as medical accuracy, simulation formatting, and robust feedback mechanisms. Objective: This study aims to quantify ChatGPT?s ability to consistently follow formatting instructions and create simulations for preclinical medical student learners according to principles of medical simulation and multimedia educational technology. Methods: Using ChatGPT-4 and a prevalidated starting prompt, the authors ran 360 separate simulations of an acute asthma exacerbation. A total of 180 simulations were given correct answers and 180 simulations were given incorrect answers. ChatGPT was evaluated for its ability to adhere to basic simulation parameters (stepwise progression, free response, interactivity), advanced simulation parameters (autonomous conclusion, delayed feedback, comprehensive feedback), and medical accuracy (vignette, treatment updates, feedback). Significance was determined with ?² analyses using 95% CIs for odds ratios. Results: In total, 100% (n=360) of simulations met basic simulation parameters and were medically accurate. For advanced parameters, 55% (200/360) of all simulations delayed feedback, while the Correct arm (157/180, 87%) delayed feedback was significantly more than the Incorrect arm (43/180, 24%; P<.001). A total of 79% (285/360) of simulations concluded autonomously, and there was no difference between the Correct and Incorrect arms in autonomous conclusion (146/180, 81% and 139/180, 77%; P=.36). Overall, 78% (282/360) of simulations gave comprehensive feedback, and there was no difference between the Correct and Incorrect arms in comprehensive feedback (137/180, 76% and 145/180, 81%; P=.31). ChatGPT-4 was not significantly more likely to conclude simulations autonomously (P=.34) and provide comprehensive feedback (P=.27) when feedback was delayed compared to when feedback was not delayed. Conclusions: These simulations have the potential to be a reliable educational tool for simple simulations and can be evaluated by a novel 9-part metric. Per this metric, ChatGPT simulations performed perfectly on medical accuracy and basic simulation parameters. It performed well on comprehensive feedback and autonomous conclusion. Delayed feedback depended on the accuracy of user inputs. A simulation meeting one advanced parameter was not more likely to meet all advanced parameters. Further work must be done to ensure consistent performance across a broader range of simulation scenarios. UR - https://formative.jmir.org/2025/1/e66478 UR - http://dx.doi.org/10.2196/66478 ID - info:doi/10.2196/66478 ER - TY - JOUR AU - Caruso, Rosario AU - Conte, Gianluca AU - Castelvecchio, Serenella AU - Baroni, Irene AU - Paglione, Giulia AU - De Angeli, Giada AU - Pasek, Malgorzata AU - Magon, Arianna PY - 2025/2/26 TI - Assessing Preparedness for Self-Management of Oral Anticoagulation in Adults With the PERSONAE Scale: Protocol for a Development and Validation Study JO - JMIR Res Protoc SP - e51502 VL - 14 KW - self-monitoring KW - self-management KW - oral anticoagulation KW - vitamin K antagonists KW - preparedness KW - validation N2 - Background: Optimal anticoagulation using vitamin K antagonists prevents strokes associated with atrial fibrillation and heart valve replacements. Preparedness for self-monitoring and self-management could improve outcomes, but this remains a challenge. Objective: This study aimed to outline the methodology for developing and validating the PERSONAE scale, a self-report measure designed to assess the preparedness for self-monitoring and self-management of oral anticoagulation in adult patients. Methods: This study comprises 2 main phases, and it adheres to the ?COnsensus-based Standards for the selection of health Measurement INstruments? (COSMIN) guidelines for instrument development. The first phase involved the conceptualization of the PERSONAE scale, where a comprehensive literature review and a consensus meeting among experts were conducted to draft the initial items. Face and content validity were then established through an expert panel review. In the second phase (ongoing), a detailed sampling methodology will be used, targeting adult Italian patients on long-term oral anticoagulation. According to a performed simulation-based power analysis, the study aims to recruit a sample size of approximately 500 participants by using a combination of convenience and snowball sampling. Data collection will be facilitated through web-based surveys distributed through social media and patient networks, ensuring a wide and representative sample. Analytical procedures will include Mokken scaling analysis for item selection and confirmatory factor analysis to validate the scale?s structure. In addition, internal consistency will be assessed using Molenaar Sijtsma statistics. Results: The scale?s content derived from phase 1 (process completed in December 2023) is grounded in a comprehensive literature review and based on the assessments of a panel of 12 health care expert professionals. The PERSONAE scale derived from phase 1 encompasses 20 items reflecting essential behaviors needed to assess the preparedness for self-monitoring and self-management of oral anticoagulation. Each item obtained a content validity ratio higher than 0.67, which is the critical content validity ratio indicating the minimum level of agreement among the experts for an item to be considered essential beyond the level of chance at a significance level of .05 for a 1-tailed test. From January 2024 to May 2024, we conducted the initial round of data collection and use Mokken scaling analysis to select items. A second round of data collection for confirmatory factor analysis was scheduled from June 2024 to September 2024, which will validate the scale?s unidimensional structure. We expect to achieve robust psychometric properties, including high internal consistency and validated constructs. Conclusions: The PERSONAE scale will be a valuable tool to assess patients? preparedness for self-monitoring and self-management of oral anticoagulation. The study?s insights into technology-assisted learning preferences will inform the design of future educational interventions to enhance preparedness in adult patients. Trial Registration: ClinicalTrials.gov NCT05973240; https://clinicaltrials.gov/study/NCT05973240 International Registered Report Identifier (IRRID): PRR1-10.2196/51502 UR - https://www.researchprotocols.org/2025/1/e51502 UR - http://dx.doi.org/10.2196/51502 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/51502 ER - TY - JOUR AU - Hadar-Shoval, Dorit AU - Lvovsky, Maya AU - Asraf, Kfir AU - Shimoni, Yoav AU - Elyoseph, Zohar PY - 2025/2/24 TI - The Feasibility of Large Language Models in Verbal Comprehension Assessment: Mixed Methods Feasibility Study JO - JMIR Form Res SP - e68347 VL - 9 KW - large language models KW - verbal comprehension assessment KW - artificial intelligence KW - AI in psychodiagnostics KW - personalized intelligence tests KW - verbal comprehension index KW - Wechsler Adult Intelligence Scale KW - WAIS-III KW - psychological test validity KW - ethics in computerized cognitive assessment N2 - Background: Cognitive assessment is an important component of applied psychology, but limited access and high costs make these evaluations challenging. Objective: This study aimed to examine the feasibility of using large language models (LLMs) to create personalized artificial intelligence?based verbal comprehension tests (AI-BVCTs) for assessing verbal intelligence, in contrast with traditional assessment methods based on standardized norms. Methods: We used a within-participants design, comparing scores obtained from AI-BVCTs with those from the Wechsler Adult Intelligence Scale (WAIS-III) verbal comprehension index (VCI). In total, 8 Hebrew-speaking participants completed both the VCI and AI-BVCT, the latter being generated using the LLM Claude. Results: The concordance correlation coefficient (CCC) demonstrated strong agreement between AI-BVCT and VCI scores (Claude: CCC=.75, 90% CI 0.266-0.933; GPT-4: CCC=.73, 90% CI 0.170-0.935). Pearson correlations further supported these findings, showing strong associations between VCI and AI-BVCT scores (Claude: r=.84, P<.001; GPT-4: r=.77, P=.02). No statistically significant differences were found between AI-BVCT and VCI scores (P>.05). Conclusions: These findings support the potential of LLMs to assess verbal intelligence. The study attests to the promise of AI-based cognitive tests in increasing the accessibility and affordability of assessment processes, enabling personalized testing. The research also raises ethical concerns regarding privacy and overreliance on AI in clinical work. Further research with larger and more diverse samples is needed to establish the validity and reliability of this approach and develop more accurate scoring procedures. UR - https://formative.jmir.org/2025/1/e68347 UR - http://dx.doi.org/10.2196/68347 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/68347 ER - TY - JOUR AU - Evans, Kerrie AU - Ko, Jonathan AU - Ceprnja, Dragana AU - Maka, Katherine AU - Beales, Darren AU - Sterling, Michele AU - Bennell, L. Kim AU - Jull, Gwendolen AU - Hodges, W. Paul AU - McKay, J. Marnee AU - Rebbeck, J. Trudy PY - 2025/2/24 TI - Development and Implementation of MyPainHub, a Web-Based Resource for People With Musculoskeletal Conditions and Their Health Care Professionals: Mixed Methods Study JO - JMIR Form Res SP - e63780 VL - 9 KW - clinical pathways KW - allied health KW - self-management KW - health information KW - ehealth KW - co-design N2 - Background: Musculoskeletal conditions, including low back pain (LBP), neck pain, and knee osteoarthritis, are the greatest contributors to years lived with disability worldwide. Resources aiming to aid both patients and health care professionals (HCPs) exist but are poorly implemented and adopted. Objective: We aimed to develop and implement MyPainHub, an evidence-based web-based resource designed to provide comprehensive, credible and accessible information for people with, and HCPs who manage, common musculoskeletal conditions. Methods: This mixed methods study adhered to the New South Wales Translational Research Framework and was evaluated against the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. Consultation with key stakeholders (patients, HCPs, researchers, industry, consumer groups, and website developers) informed content, design, features, and functionality. Development then aimed to meet the identified need for a ?one-stop shop??a central location for information about common musculoskeletal conditions tailored to a person?s condition and risk of poor outcomes. MyPainHub was then developed through an iterative process and implementation strategies were tailored to different health care settings. Quantitative and qualitative evaluation occurred with patients and HCPs. Results: In total, 127 stakeholders participated in the development phase; initial consultation with them led to embedding 2 validated screening tools (the Short Form Örebro Musculoskeletal Pain Screening Questionnaire and the Keele STarT MSK tool) in MyPainHub to guide information tailoring for patients based on risk of poor outcomes. Development occurred in parallel and feedback from stakeholders informed design and content including structure, functionality, and phrasing and images to use to emphasize key points. Consultation resulted in information for patients being categorized using key guideline-based messages (general information, your pathway, exercise, and imaging) while information for clinicians was categorized into assessment, management, and prognosis. Implementation occurred in different health care settings with the most effective strategies being interactive education via webinars and workshops. The evaluation phase involved web-based questionnaires (patients: n=44; HCPs: n=29) and focus groups (patients: n=6; HCPs: n=6). Patients and HCPs found MyPainHub user-friendly, acceptable, credible, and potentially able to support self-management. Patient participants identified areas for improvement such as including more specific information on preventative measures and pain relief options. Despite positive feedback, only 35% (10/29) of HCPs used MyPainHub with their patients. HCP participants identified challenges including insufficient training and lack of familiarity with using web-based resources in existing clinical workflows. Following implementation, the information contained on MyPainHub changed knowledge and practice for some patients and HCPs. Conclusions: Following extensive and iterative stakeholder engagement, MyPainHub was developed as an evidence-based web-based resource and perceived by patients and HCPs as user-friendly, credible, and acceptable. Active implementation strategies are required for adoption and implementation and greater training focusing on strategies to implement MyPainHub into clinical practice may be necessary. Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12619000871145; https://tinyurl.com/438kkyt3 UR - https://formative.jmir.org/2025/1/e63780 UR - http://dx.doi.org/10.2196/63780 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/63780 ER - TY - JOUR AU - Steele, Brian AU - Fairie, Paul AU - Kemp, Kyle AU - D'Souza, G. Adam AU - Wilms, Matthias AU - Santana, Jose Maria PY - 2025/2/24 TI - Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study JO - JMIR Med Inform SP - e63466 VL - 13 KW - natural language processing KW - patient-reported experience KW - topic models KW - inpatient KW - artificial intelligence KW - AI KW - patient reported KW - feedback KW - survey KW - patient experiences KW - bidirectional encoder representations from transformers KW - BERT KW - sentiment analysis KW - pediatric caregivers KW - patient safety KW - safety N2 - Background: Patient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve health care by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but these measures may collect optional free-text comments. These comments can provide insights for health systems but may not be analyzed due to limited resources and the complexity of traditional textual analysis. However, advances in machine learning?based natural language processing provide opportunities to learn from this traditionally underused data source. Objective: This study aimed to apply natural language processing to model topics found in free-text comments of patient-reported experience surveys. Methods: Consumer Assessment of Healthcare Providers and Systems?derived patient experience surveys were collected and linked to administrative inpatient records by the provincial health services organization responsible for inpatient care. Unsupervised topic modeling with automated labeling was performed with BERTopic. Sentiment analysis was performed to further assist in topic description. Results: Between April 2016 and February 2020, 43.4% (43,522/100,272) adult patients and 46.9% (3501/7464) pediatric caregivers included free-text responses on completed patient experience surveys. Topic models identified 86 topics among adult survey responses and 35 topics among pediatric responses that included elements of care not currently surveyed by existing questionnaires. Frequent topics were generally positive. Conclusions: We found that with limited tuning, BERTopic identified care experience topics with interpretable automated labeling. Results are discussed in the context of person-centered care, patient safety, and health care quality improvement. Furthermore, we note the opportunity for the identification of temporal and site-specific trends as a method to identify patient care and safety concerns. As the use of patient experience measurement increases in health care, we discuss how machine learning can be leveraged to provide additional insight on patient experiences. UR - https://medinform.jmir.org/2025/1/e63466 UR - http://dx.doi.org/10.2196/63466 ID - info:doi/10.2196/63466 ER - TY - JOUR AU - Davis-Wilson, Hope AU - Hegarty-Craver, Meghan AU - Gaur, Pooja AU - Boyce, Matthew AU - Holt, R. Jonathan AU - Preble, Edward AU - Eckhoff, Randall AU - Li, Lei AU - Walls, Howard AU - Dausch, David AU - Temple, Dorota PY - 2025/2/24 TI - Effects of Missing Data on Heart Rate Variability Measured From A Smartwatch: Exploratory Observational Study JO - JMIR Form Res SP - e53645 VL - 9 KW - plethysmography KW - electrocardiogram KW - missing data KW - smartwatch KW - wearable KW - ECG KW - photoplethysmography KW - PPG KW - mobile phone KW - heart rate KW - pilot study KW - detection KW - sensor KW - monitoring KW - health metric KW - measure KW - real-world settings KW - rest KW - physical activity KW - remote monitoring KW - medical setting KW - youth KW - adolescent KW - teen KW - teenager N2 - Background: Measuring heart rate variability (HRV) through wearable photoplethysmography sensors from smartwatches is gaining popularity for monitoring many health conditions. However, missing data caused by insufficient wear compliance or signal quality can degrade the performance of health metrics or algorithm calculations. Research is needed on how to best account for missing data and to assess the accuracy of metrics derived from photoplethysmography sensors. Objective: This study aimed to evaluate the influence of missing data on HRV metrics collected from smartwatches both at rest and during activity in real-world settings and to evaluate HRV agreement and consistency between wearable photoplethysmography and gold-standard wearable electrocardiogram (ECG) sensors in real-world settings. Methods: Healthy participants were outfitted with a smartwatch with a photoplethysmography sensor that collected high-resolution interbeat interval (IBI) data to wear continuously (day and night) for up to 6 months. New datasets were created with various amounts of missing data and then compared with the original (reference) datasets. 5-minute windows of each HRV metric (median IBI, SD of IBI values [STDRR], root-mean-square of the difference in successive IBI values [RMSDRR], low-frequency [LF] power, high-frequency [HF] power, and the ratio of LF to HF power [LF/HF]) were compared between the reference and the missing datasets (10%, 20%, 35%, and 60% missing data). HRV metrics calculated from the photoplethysmography sensor were compared with HRV metrics calculated from a chest-worn ECG sensor. Results: At rest, median IBI remained stable until at least 60% of data degradation (P=.24), STDRR remained stable until at least 35% of data degradation (P=.02), and RMSDRR remained stable until at least 35% data degradation (P=.001). During the activity, STDRR remained stable until 20% data degradation (P=.02) while median IBI (P=.01) and RMSDRR P<.001) were unstable at 10% data degradation. LF (rest: P<.001; activity: P<.001), HF (rest: P<.001, activity: P<.001), and LF/HF (rest: P<.001, activity: P<.001) were unstable at 10% data degradation during rest and activity. Median IBI values calculated from photoplethysmography sensors had a moderate agreement (intraclass correlation coefficient [ICC]=0.585) and consistency (ICC=0.589) and LF had moderate consistency (ICC=0.545) with ECG sensors. Other HRV metrics demonstrated poor agreement (ICC=0.071-0.472). Conclusions: This study describes a methodology for the extraction of HRV metrics from photoplethysmography sensor data that resulted in stable and valid metrics while using the least amount of available data. While smartwatches containing photoplethysmography sensors are valuable for remote monitoring of patients, future work is needed to identify best practices for using these sensors to evaluate HRV in medical settings. UR - https://formative.jmir.org/2025/1/e53645 UR - http://dx.doi.org/10.2196/53645 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53645 ER - TY - JOUR AU - Gardner, Leslie Leah AU - Raeisian Parvari, Pezhman AU - Seidman, Mark AU - Holden, J. Richard AU - Fowler, R. Nicole AU - Zarzaur, L. Ben AU - Summanwar, Diana AU - Barboi, Cristina AU - Boustani, Malaz PY - 2025/2/24 TI - Improving the User Interface and Guiding the Development of Effective Training Material for a Clinical Research Recruitment and Retention Dashboard: Usability Testing Study JO - JMIR Form Res SP - e66718 VL - 9 KW - recruitment strategies KW - clinical research KW - research subject recruitment KW - agile science KW - agile implementation KW - human-computer interaction N2 - Background: Participant recruitment and retention are critical to the success of clinical trials, yet challenges such as low enrollment rates and high attrition remain ongoing obstacles. RecruitGPS is a scalable dashboard with integrated control charts to address these issues by providing real-time data monitoring and analysis, enabling researchers to better track and improve recruitment and retention. Objective: This study aims to identify the challenges and inefficiencies users encounter when interacting with the RecruitGPS dashboard. By identifying these issues, the study aims to inform strategies for improving the dashboard?s user interface and create targeted, effective instructional materials that address user needs. Methods: Twelve clinical researchers from the Midwest region of the United States provided feedback through a 10-minute, video-recorded usability test session, during which participants were instructed to explore the various tabs of the dashboard, identify challenges, and note features that worked well while thinking aloud. Following the video session, participants took a survey on which they answered System Usability Scale (SUS) questions, ease of navigation questions, and a Net Promoter Score (NPS) question. Results: A quantitative analysis of survey responses revealed an average SUS score of 61.46 (SD 23.80; median 66.25) points, indicating a need for improvement in the user interface. The NPS was 8, with 4 of 12 (33%) respondents classified as promoters and 3 of 12 (25%) as detractors, indicating a slightly positive satisfaction. When participants compared RecruitGPS to other recruitment and study management tools they had used, 8 of 12 (67%) of participants rated RecruitGPS as better or much better. Only 1 of 12 (8%) participants rated RecruitGPS as worse but not much worse. A qualitative analysis of participants? interactions with the dashboard diagnosed a confusing part of the dashboard that could be eliminated or made optional and provided valuable insight for the development of instructional videos and documentation. Participants liked the dashboard?s data visualization capabilities, including intuitive graphs and trend tracking; progress indicators, such as color-coded status indicators and comparison metrics; and the overall dashboard?s layout and design, which consolidated relevant data on a single page. Users also valued the accuracy and real-time updates of data, especially the integration with external sources like Research Electronic Data Capture (REDCap). Conclusions: RecruitGPS demonstrates significant potential to improve the efficiency of clinical trials by providing researchers with real-time insights into participant recruitment and retention. This study offers valuable recommendations for targeted refinements to enhance the user experience and maximize the dashboard?s effectiveness. Additionally, it highlights navigation challenges that can be addressed through the development of clear and focused instructional videos. UR - https://formative.jmir.org/2025/1/e66718 UR - http://dx.doi.org/10.2196/66718 ID - info:doi/10.2196/66718 ER - TY - JOUR AU - Wyse, Rebecca AU - Forbes, Erin AU - Norton, Grace AU - Viana Da Silva, Priscilla AU - Fakes, Kristy AU - Johnston, Ann Sally AU - Smith, R. Stephen AU - Zucca, Alison PY - 2025/2/21 TI - Effect on Response Rates of Adding a QR Code to Patient Consent Forms for Qualitative Research in Patients With Cancer: Pilot Randomized Controlled Trial JO - JMIR Form Res SP - e66681 VL - 9 KW - QR code KW - qualitative research KW - cancer KW - randomized controlled trial KW - RCT KW - patient recruitment KW - consent forms KW - response rates N2 - Background: The successful conduct of health and medical research is largely dependent on participant recruitment. Effective, yet inexpensive methods of increasing response rates for all types of research are required. QR codes are now commonplace, and despite having been extensively used to recruit study participants, a search of the literature failed to reveal any randomized trial investigating the effect of adding a QR code on qualitative research response rates. Objective: This study aimed to collect data on rates of response, consent, and decline among patients with cancer, and the average time taken to respond following randomization to receive either a QR code or no QR code on the patient consent form for a qualitative research study. Methods: This was a pilot randomized controlled trial (RCT) embedded within a qualitative research study. In total, 40 eligible patients received a recruitment pack for the qualitative study, which included an information statement, a consent form, and an addressed, stamped envelope to return their consent form. Patients were randomized 1:1 to the control (standard recruitment pack only) or intervention group (standard recruitment pack including modified consent form with a QR code). Results: In total, 27 out of 40 patients (age: mean 63.0, SD 14.8 years; 45% female) responded to the consent form. A lower proportion of the QR code group (60%) responded (odd ratio [OR] 0.57, 95% CI 0.14-2.37; P=.44), compared to 75% of the standard recruitment group. However, a higher proportion of the QR group (35%) consented (OR 1.84, 95% CI 0.41-8.29; P=.43), compared to the standard recruitment group (20%). A lower proportion of the QR group (25%) declined (OR 0.34, 95% CI 0.09-1.38; P=.13) relative to the standard recruitment group (55%). The mean response time of the QR code group was 16 days (rate ratio [RR] 0.79, 95% CI 0.47-1.35; P=.39) compared to 19 days for the standard recruitment group. None of the age-adjusted analyses were statistically significant. Conclusions: This underpowered pilot study did not find any evidence that offering an option to respond through a QR code on a patient consent form for a qualitative study increased the overall patient response rate (combined rate of consent and decline). However, there was a nonsignificant trend, indicating that more patients who received the QR code consented compared to those who did not receive the QR code. This study provides useful preliminary data on the potential impact of QR codes on patient response rates to invitations to participate in qualitative research and can be used to inform fully powered RCTs. Trial Registration: OSF Registries 10.17605/OSF.IO/PJ25X; https://doi.org/10.17605/OSF.IO/PJ25X UR - https://formative.jmir.org/2025/1/e66681 UR - http://dx.doi.org/10.2196/66681 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/66681 ER - TY - JOUR AU - Gerdes, John AU - Schooley, Benjamin AU - Sharp, Dakota AU - Miller, Juliana PY - 2025/2/20 TI - The Design and Evaluation of a Simulation Tool for Audiology Screening Education: Design Science Approach JO - JMIR Form Res SP - e47150 VL - 9 KW - design science KW - audiology KW - simulation KW - hearing screening KW - framework KW - speech pathology KW - training N2 - Background: The early identification of hearing loss and ear disorders is important. Regular screening is recommended for all age groups to determine whether a full hearing assessment is necessary and allow for timely treatment of hearing problems. Procedural training is needed for new speech-language pathology students as well as continuing education for those trained to perform this screening procedure. Limited availability and access to physical training locations can make it difficult to receive the needed training. Objective: The aims of this study were to (1) develop a new hearing screening simulation software platform and (2) assess its effectiveness in training a group of graduate-level speech-language pathology students in hearing screening procedures. Methods: An audiology simulator modeled after the commercial Grason-Stadler GSI39 combination audiometer and tympanometer device was developed to serve as a precursor to traditional face-to-face clinical instruction. A description of the simulator development process, guided by a design science approach, is presented. The initiation phase established the initial criteria for the simulator design. This was followed by an iterative process involving prototype development, review, and critique by the clinical faculty. This feedback served as input for the subsequent iteration. The evaluation of the final prototype involved 33 speech-language pathology graduate students as part of an introductory audiology class. These students were randomly assigned to control (receiving in-person instruction) and test (in-person instruction and simulation tool use) groups. Students in both groups were subsequently evaluated as they performed audiology screenings on human participants and completed a 25-item pretest and posttest survey. Nonparametric Mann-Whitney U tests were conducted on the mean differences between pretest and posttest ordinal survey response data to compare the control and intervention groups. Results: The results indicated that the students who used the simulation tool demonstrated greater confidence in their ability to (1) explain hearing screening procedures to a child (P=.02), (2) determine whether otoscopy results are normal (P=.02), and (3) determine whether otoscopy results are abnormal (P=.03). Open-ended responses indicated that the students found that the hands-on experience provided by the simulator resulted in an easy-to-use and useful learning experience with the audiometer, which increased their confidence in their ability to perform hearing screenings. Conclusions: Software-based education simulation tools for audiology screening may provide a beneficial approach to educating students and professionals in hearing screening training. The tool tested in this study supports individualized, self-paced learning with context-sensitive feedback and performance assessment, incorporating an extensible approach to supporting simulated subjects. UR - https://formative.jmir.org/2025/1/e47150 UR - http://dx.doi.org/10.2196/47150 UR - http://www.ncbi.nlm.nih.gov/pubmed/39977027 ID - info:doi/10.2196/47150 ER - TY - JOUR AU - Wang, Jianli AU - Orpana, Heather AU - Carrington, André AU - Kephart, George AU - Vasiliadis, Helen-Maria AU - Leikin, Benjamin PY - 2025/2/19 TI - Development and Validation of Prediction Models for Perceived and Unmet Mental Health Needs in the Canadian General Population: Model-Based Synthetic Estimation Study JO - JMIR Public Health Surveill SP - e66056 VL - 11 KW - population risk prediction KW - development KW - validation KW - perceived mental health need KW - unmet mental health need N2 - Background: Research has shown that perceptions of a mental health need are closely associated with service demands and are an important dimension in needs assessment. Perceived and unmet mental health needs are important factors in the decision-making process regarding mental health services planning and resources allocation. However, few prediction tools are available to be used by policy and decision makers to forecast perceived and unmet mental health needs at the population level. Objective: We aim to develop prediction models to forecast perceived and unmet mental health needs at the provincial and health regional levels in Canada. Methods: Data from 2018, 2019, and 2020 Canadian Community Health Survey and Canadian Urban Environment were used (n=65,000 each year). Perceived and unmet mental health needs were measured by the Perceived Needs for Care Questionnaire. Using the 2018 dataset, we developed the prediction models through the application of regression synthetic estimation for the Atlantic, Central, and Western regions. The models were validated in the 2019 and 2020 datasets at the provincial level and in 10 randomly selected health regions by comparing the observed and predicted proportions of the outcomes. Results: In 2018, a total of 17.82% of the participants reported perceived mental health need and 3.81% reported unmet mental health need. The proportions were similar in 2019 (18.04% and 3.91%) and in 2020 (18.1% and 3.92%). Sex, age, self-reported mental health, physician diagnosed mood and anxiety disorders, self-reported life stress and life satisfaction were the predictors in the 3 regional models. The individual based models had good discriminative power with C statistics over 0.83 and good calibration. Applying the synthetic models in 2019 and 2020 data, the models had the best performance in Ontario, Quebec, and British Columbia; the absolute differences between observed and predicted proportions were less than 1%. The absolute differences between the predicted and observed proportion of perceived mental health needs in Newfoundland and Labrador (?4.16% in 2020) and Prince Edward Island (4.58% in 2019) were larger than those in other provinces. When applying the models in the 10 selected health regions, the models calibrated well in the health regions in Ontario and in Quebec; the absolute differences in perceived mental health needs ranged from 0.23% to 2.34%. Conclusions: Predicting perceived and unmet mental health at the population level is feasible. There are common factors that contribute to perceived and unmet mental health needs across regions, at different magnitudes, due to different population characteristics. Therefore, predicting perceived and unmet mental health needs should be region specific. The performance of the models at the provincial and health regional levels may be affected by population size. UR - https://publichealth.jmir.org/2025/1/e66056 UR - http://dx.doi.org/10.2196/66056 ID - info:doi/10.2196/66056 ER - TY - JOUR AU - Scheider, Simon AU - Mallick, Kamal Mostafa PY - 2025/2/18 TI - Exploring Metadata Catalogs in Health Care Data Ecosystems: Taxonomy Development Study JO - JMIR Form Res SP - e63396 VL - 9 KW - data catalogs KW - data ecosystems KW - findability, accessibility, interoperability, and reusability KW - FAIR KW - health care KW - metadata KW - taxonomy N2 - Background: In the European health care industry, recent years have seen increasing investments in data ecosystems to ?FAIRify? and capitalize the ever-rising amount of health data. Within such networks, health metadata catalogs (HMDCs) assume a key function as they enable data allocation, sharing, and use practices. By design, HMDCs orchestrate health information for the purpose of findability, accessibility, interoperability, and reusability (FAIR). However, despite various European initiatives pushing health care data ecosystems forward, actionable design knowledge about HMDCs is scarce. This impedes both their effective development in practice and their scientific exploration, causing huge unused innovation potential of health data. Objective: This study aims to explore the structural design elements of HMDCs, classifying them alongside empirically reasonable dimensions and characteristics. In doing so, the development of HMDCs in practice is facilitated while also closing a crucial gap in theory (ie, the literature about actionable HMDC design knowledge). Methods: We applied a rigorous methodology for taxonomy building following well-known and established guidelines from the domain of information systems. Within this methodological framework, inductive and deductive research methods were applied to iteratively design and evaluate the evolving set of HMDC dimensions and characteristics. Specifically, a systematic literature review was conducted to identify and analyze 38 articles, while a multicase study was conducted to examine 17 HMDCs from practice. These findings were evaluated and refined in 2 extensive focus group sessions by 7 interdisciplinary experts with deep knowledge about HMDCs. Results: The artifact generated by the study is an iteratively conceptualized and empirically grounded taxonomy with elaborate explanations. It proposes 20 dimensions encompassing 101 characteristics alongside which FAIR HMDCs can be structured and classified. The taxonomy describes basic design characteristics that need to be considered to implement FAIR HMDCs effectively. A major finding was that a particular focus in developing HMDCs is on the design of their published dataset offerings (ie, their metadata assets) as well as on data security and governance. The taxonomy is evaluated against the background of 4 use cases, which were cocreated with experts. These illustrative scenarios add depth and context to the taxonomy as they underline its relevance and applicability in real-world settings. Conclusions: The findings contribute fundamental, yet actionable, design knowledge for building HMDCs in European health care data ecosystems. They provide guidance for health care practitioners, while allowing both scientists and policy makers to navigate through this evolving research field and anchor their work. Therefore, this study closes the research gap outlined earlier, which has prevailed in theory and practice. UR - https://formative.jmir.org/2025/1/e63396 UR - http://dx.doi.org/10.2196/63396 UR - http://www.ncbi.nlm.nih.gov/pubmed/39964739 ID - info:doi/10.2196/63396 ER - TY - JOUR AU - Carr, James Zyad AU - Agarkov, Daniel AU - Li, Judy AU - Charchaflieh, Jean AU - Brenes-Bastos, Andres AU - Freund, Jonah AU - Zafar, Jill AU - Schonberger, B. Robert AU - Heerdt, Paul PY - 2025/2/17 TI - Implementation of Brief Submaximal Cardiopulmonary Testing in a High-Volume Presurgical Evaluation Clinic: Feasibility Cohort Study JO - JMIR Perioper Med SP - e65805 VL - 8 KW - preoperative evaluation KW - submaximal cardiopulmonary exercise test KW - risk stratification KW - perioperative medicine KW - anesthesiology N2 - Background: Precise functional capacity assessment is a critical component for preoperative risk stratification. Brief submaximal cardiopulmonary exercise testing (smCPET) has shown diagnostic utility in various cardiopulmonary conditions. Objective: This study aims to determine if smCPET could be implemented in a high-volume presurgical evaluation clinic and, when compared to structured functional capacity surveys, if smCPET could better discriminate low functional capacity (?4.6 metabolic equivalents [METs]). Methods: After institutional approval, 43 participants presenting for noncardiac surgery who met the following inclusion criteria were enrolled: aged 60 years and older, a Revised Cardiac Risk Index of ?2, and self-reported METs of ?4.6 (self-endorsed ability to climb 2 flights of stairs). Subjective METs assessments, Duke Activity Status Index (DASI) surveys, and a 6-minute smCPET trial were conducted. The primary end points were (1) operational efficiency, based on the time of the experimental session being ?20 minutes; (2) modified Borg survey of perceived exertion, with a score of ?7 indicating no more than moderate exertion; (3) high participant satisfaction with smCPET task execution, represented as a score of ?8 (out of 10); and (4) high participant satisfaction with smCPET scheduling, represented as a score of ?8 (out of 10). Student's t test was used to determine the significance of the secondary end points. Correlation between comparable structured surveys and smCPET measurements was assessed using the Pearson correlation coefficient. A Bland-Altman analysis was used to assess agreement between the methods. Results: The mean session time was 16.9 (SD 6.8) minutes. The mean posttest modified Borg survey score was 5.35 (SD 1.8). The median patient satisfaction (on a scale of 1=worst to 10=best) was 10 (IQR 10-10) for scheduling and 10 (IQR 9-10) for task execution. Subjective METs were higher when compared to smCPET equivalents (extrapolated peak METs; mean 7.6, SD 2.0 vs mean 6.7, SD 1.8; t42=2.1; P<.001). DASI-estimated peak METs were higher when compared to smCPET peak METs (mean 8.8, SD 1.2 vs mean 6.7, SD 1.8; t42=7.2; P<.001). DASI-estimated peak oxygen uptake was higher than smCPET peak oxygen uptake (mean 30.9, SD 4.3 mL kg?1 min?1 vs mean 23.6, SD 6.5 mL kg?1 min?1; t42=7.2; P<.001). Conclusions: Implementation of smCPET in a presurgical evaluation clinic is both patient centered and clinically feasible. Brief smCPET measures, supportive of published reports regarding low sensitivity of provider-driven or structured survey measures for low functional capacity, were lower than those from structured surveys. Future studies will analyze the prediction of perioperative complications and cost-effectiveness. Trial Registration: ClinicalTrials.gov NCT05743673; https://clinicaltrials.gov/study/NCT05743673 UR - https://periop.jmir.org/2025/1/e65805 UR - http://dx.doi.org/10.2196/65805 UR - http://www.ncbi.nlm.nih.gov/pubmed/39773953 ID - info:doi/10.2196/65805 ER - TY - JOUR AU - Shamebo, Dessalegn AU - Derseh Mebratie, Anagaw AU - Arsenault, Catherine PY - 2025/2/13 TI - Using an Interactive Voice Response Survey to Assess Patient Satisfaction in Ethiopia: Development and Feasibility Study JO - JMIR Form Res SP - e67452 VL - 9 KW - mobile phone surveys KW - patient satisfaction KW - interactive voice response KW - global health KW - surveys KW - Ethiopia KW - IVR KW - Africa N2 - Background: Patient satisfaction surveys can offer crucial information on the quality of care but are rarely conducted in low-income settings. In contrast with in-person exit interviews, phone-based interactive voice response (IVR) surveys may offer benefits including standardization, patient privacy, reduced social desirability bias, and cost and time efficiency. IVR surveys have rarely been tested in low-income settings, particularly for patient satisfaction surveys. Objective: In this study, we tested the feasibility of using an IVR system to assess patient satisfaction with primary care services in Addis Ababa, Ethiopia. We described the methodology, response rates, and survey costs and identified factors associated with survey participation, completion, and duration. Methods: Patients were recruited in person from 18 public and private health facilities in Addis Ababa. Patients? sex, age, education, reasons for seeking care, and mobile phone numbers were collected. The survey included 15 questions that respondents answered using their phone keypad. We used a Heckman probit regression model to identify factors influencing the likelihood of IVR survey participation (picking up and answering at least 1 question) and completion (answering all survey questions) and a Weibull regression model to identify factors influencing the survey completion time. Results: A total of 3403 individuals were approached across 18 health facilities. Nearly all eligible patients approached (2985/3167, 94.3%) had a functioning mobile phone, and 89.9% (2415/2685) of those eligible agreed to be enrolled in the study. Overall, 92.6% (2236/2415) picked up the call, 65.6% (1584/2415) answered at least 1 survey question, and 42.9% (1037/2415) completed the full survey. The average survey completion time was 8.1 (SD 1.7) minutes for 15 Likert-scale questions. We found that those aged 40-49 years and those aged 50+ years were substantially less likely to participate in (odds ratio 0.63, 95% CI 0.53-0.74) and complete the IVR survey (odds ratio 0.77, 95% CI 0.65-0.90) compared to those aged 18-30 years. Higher education levels were also strongly associated with survey participation and completion. In adjusted models, those enrolled in private facilities were less likely to participate and complete the survey compared to those in public health centers. Being male, younger, speaking Amharic, using a private hospital, and being called after 8 PM were associated with a shorter survey duration. The average survey costs were US $7.90 per completed survey. Conclusions: Our findings reveal that an IVR survey is a feasible, low-cost, and rapid solution to assess patient satisfaction in an urban context in Ethiopia. However, survey implementation must be carefully planned and tailored to local challenges. Governments and health facilities should consider IVR to routinely collect patient satisfaction data to inform quality improvement strategies. UR - https://formative.jmir.org/2025/1/e67452 UR - http://dx.doi.org/10.2196/67452 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67452 ER - TY - JOUR AU - Peasley, Dale AU - Kuplicki, Rayus AU - Sen, Sandip AU - Paulus, Martin PY - 2025/2/13 TI - Leveraging Large Language Models and Agent-Based Systems for Scientific Data Analysis: Validation Study JO - JMIR Ment Health SP - e68135 VL - 12 KW - LLM KW - agent-based systems KW - scientific data analysis KW - data contextualization KW - AI-driven research tools KW - large language model KW - scientific data KW - analysis KW - contextualization KW - AI KW - artificial intelligence KW - research tool N2 - Background: Large language models have shown promise in transforming how complex scientific data are analyzed and communicated, yet their application to scientific domains remains challenged by issues of factual accuracy and domain-specific precision. The Laureate Institute for Brain Research?Tulsa University (LIBR-TU) Research Agent (LITURAt) leverages a sophisticated agent-based architecture to mitigate these limitations, using external data retrieval and analysis tools to ensure reliable, context-aware outputs that make scientific information accessible to both experts and nonexperts. Objective: The objective of this study was to develop and evaluate LITURAt to enable efficient analysis and contextualization of complex scientific datasets for diverse user expertise levels. Methods: An agent-based system based on large language models was designed to analyze and contextualize complex scientific datasets using a ?plan-and-solve? framework. The system dynamically retrieves local data and relevant PubMed literature, performs statistical analyses, and generates comprehensive, context-aware summaries to answer user queries with high accuracy and consistency. Results: Our experiments demonstrated that LITURAt achieved an internal consistency rate of 94.8% and an external consistency rate of 91.9% across repeated and rephrased queries. Additionally, GPT-4 evaluations rated 80.3% (171/213) of the system?s answers as accurate and comprehensive, with 23.5% (50/213) receiving the highest rating of 5 for completeness and precision. Conclusions: These findings highlight the potential of LITURAt to significantly enhance the accessibility and accuracy of scientific data analysis, achieving high consistency and strong performance in complex query resolution. Despite existing limitations, such as model stability for highly variable queries, LITURAt demonstrates promise as a robust tool for democratizing data-driven insights across diverse scientific domains. UR - https://mental.jmir.org/2025/1/e68135 UR - http://dx.doi.org/10.2196/68135 ID - info:doi/10.2196/68135 ER - TY - JOUR AU - Verstegen, Amandine AU - Van Daele, Tom AU - Bonroy, Bert AU - Debard, Glen AU - Sels, Romy AU - van Loo, Marlon AU - Bernaerts, Sylvie PY - 2025/2/13 TI - Designing a Smartphone-Based Virtual Reality App for Relaxation: Qualitative Crossover Study JO - JMIR Form Res SP - e62663 VL - 9 KW - smartphone-based virtual reality KW - virtual reality KW - relaxation KW - stress KW - user experience KW - mobile phone N2 - Background: Accumulating evidence supports the use of virtual reality (VR) in mental health care, with one potential application being its use to assist individuals with relaxation exercises. Despite studies finding support for the potential of VR to effectively aid in relaxation, its implementation remains limited outside of specialized clinics. Known barriers are insufficient knowledge regarding VR operation, lack of availability of VR relaxation apps tailored to local health care systems, and cost concerns. Unfortunately, many VR relaxation apps are designed exclusively for stand-alone headsets, limiting accessibility for a broad audience. Objective: We aimed to design an accessible, smartphone-based VR relaxation app based on user preferences. This paper describes the assessment of 2 stand-alone VR relaxation apps and the resulting smartphone-based VR relaxation app design. Methods: Overall, 30 participants (n=23, 77% women; n=7, 23% men) took part in 2 separate VR sessions, assessing 1 of the 2 VR relaxation apps (Flowborne and Calm Place) in each session. After each session, participants were presented with open-ended questions to assess their experiences via a web-based survey tool. These questions explored positive and negative features, shortcomings, and suggestions for improvements while also allowing space for additional remarks concerning the 2 VR relaxation apps. Three of the authors analyzed the responses using inductive thematic analysis, a process comprising 6 phases. Results: Across both the apps, 5 recurring themes and 13 recurring subthemes were identified in the participants? answers: audio (music and sounds, guidance), visuals (content, realism, variation and dynamics in the environment), features (language, options, feedback and instructions, duration, exercise), implementation (technical aspects, cybersickness, acceptability and usability), and experience. We analyzed the participants? findings and conducted a literature review, which served as the basis for developing the app. The resulting app is a Dutch-language, smartphone-based VR relaxation app, with customization options including 3 types of relaxation exercises, 2 guiding voices, and 3 different environments. Efforts have been made to ensure maximum variation and dynamism in the environments. Calming music and nature sounds accompany the exercises. The efficacy and effectiveness of the resulting app design were not assessed. Conclusions: This study provides insights into key features of VR relaxation apps, which were subsequently used for the development of a novel smartphone-based VR relaxation app. Further research concerning the effectiveness of this app, along with a broader evaluation of the efficacy and user feedback for smartphone-based VR relaxation apps, is needed. More generally, there is a clear need for more research on the impact of interactivity, biofeedback, and type of environment in VR relaxation. UR - https://formative.jmir.org/2025/1/e62663 UR - http://dx.doi.org/10.2196/62663 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/62663 ER - TY - JOUR AU - Draucker, Burke Claire AU - Carrión, Andrés AU - Ott, A. Mary AU - Hicks, I. Ariel AU - Knopf, Amelia PY - 2025/2/13 TI - A 4-Site Public Deliberation Project on the Acceptability of Youth Self-Consent in Biomedical HIV Prevention Trials: Assessment of Facilitator Fidelity to Key Principles JO - JMIR Form Res SP - e58451 VL - 9 KW - public deliberation KW - deliberative democracy KW - bioethics KW - ethical conflict KW - biomedical KW - HIV prevention KW - HIV research KW - group facilitation KW - fidelity assessment KW - content analysis N2 - Background: Public deliberation is an approach used to engage persons with diverse perspectives in discussions and decision-making about issues affecting the public that are controversial or value laden. Because experts have identified the need to evaluate facilitator performance, our research team developed a framework to assess the fidelity of facilitator remarks to key principles of public deliberation. Objective: This report describes how the framework was used to assess facilitator fidelity in a 4-site public deliberation project on the acceptability of minor self-consent in biomedical HIV prevention research. Methods: A total of 88 individuals participated in 4 deliberation sessions held in 4 cities throughout the United States. The sessions, facilitated by 18 team members, were recorded and transcribed verbatim. Facilitator remarks were highlighted, and predetermined coding rules were used to code the remarks to 1 of 6 principles of quality deliberations. A variety of display tables were used to organize the codes and calculate the number of facilitator remarks that were consistent or inconsistent with each principle during each session across all sites. A content analysis was conducted on the remarks to describe how facilitator remarks aligned or failed to align with each principle. Results: In total, 735 remarks were coded to one of the principles; 516 (70.2%) were coded as consistent with a principle, and 219 (29.8%) were coded as inconsistent. A total of 185 remarks were coded to the principle of equal participation (n=138, 74.6% as consistent; n=185, 25.4% as inconsistent), 158 were coded to expression of diverse opinions (n=110, 69.6% as consistent; n=48, 30.4% as inconsistent), 27 were coded to respect for others (n=27, 100% as consistent), 24 were coded to adoption of a societal perspective (n=11, 46% as consistent; n=13, 54% as inconsistent), 99 were coded to reasoned justification of ideas (n=81, 82% as consistent; n=18, 18% as inconsistent), and 242 were coded to compromise or movement toward consensus (n=149, 61.6% as consistent; n=93, 38.4% as inconsistent). Therefore, the counts provided affirmation that most of the facilitator remarks were aligned with the principles of deliberation, suggesting good facilitator fidelity. By considering how the remarks aligned or failed to align with the principles, areas where facilitator fidelity can be strengthened were identified. The results indicated that facilitators should focus more on encouraging quieter members to participate, refraining from expressing personal opinions, promoting the adoption of a societal perspective and reasoned justification of opinions, and inviting deliberants to articulate their areas of common ground. Conclusions: The results provide an example of how a framework for assessing facilitator fidelity was used in a 4-site deliberation project. The framework will be refined to better address issues related to balancing personal and public perspectives, managing plurality, and mitigating social inequalities. UR - https://formative.jmir.org/2025/1/e58451 UR - http://dx.doi.org/10.2196/58451 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58451 ER - TY - JOUR AU - Kottlors, Jonathan AU - Hahnfeldt, Robert AU - Görtz, Lukas AU - Iuga, Andra-Iza AU - Fervers, Philipp AU - Bremm, Johannes AU - Zopfs, David AU - Laukamp, R. Kai AU - Onur, A. Oezguer AU - Lennartz, Simon AU - Schönfeld, Michael AU - Maintz, David AU - Kabbasch, Christoph AU - Persigehl, Thorsten AU - Schlamann, Marc PY - 2025/2/13 TI - Large Language Models?Supported Thrombectomy Decision-Making in Acute Ischemic Stroke Based on Radiology Reports: Feasibility Qualitative Study JO - J Med Internet Res SP - e48328 VL - 27 KW - artificial intelligence KW - radiology KW - report KW - large language model KW - text-based augmented supporting system KW - mechanical thrombectomy KW - GPT KW - stroke KW - decision-making KW - thrombectomy KW - imaging KW - model KW - machine learning KW - ischemia N2 - Background: The latest advancement of artificial intelligence (AI) is generative pretrained transformer large language models (LLMs). They have been trained on massive amounts of text, enabling humanlike and semantical responses to text-based inputs and requests. Foreshadowing numerous possible applications in various fields, the potential of such tools for medical data integration and clinical decision-making is not yet clear. Objective: In this study, we investigate the potential of LLMs in report-based medical decision-making on the example of acute ischemic stroke (AIS), where clinical and image-based information may indicate an immediate need for mechanical thrombectomy (MT). The purpose was to elucidate the feasibility of integrating radiology report data and other clinical information in the context of therapy decision-making using LLMs. Methods: A hundred patients with AIS were retrospectively included, for which 50% (50/100) was indicated for MT, whereas the other 50% (50/100) was not. The LLM was provided with the computed tomography report, information on neurological symptoms and onset, and patients? age. The performance of the AI decision-making model was compared with an expert consensus regarding the binary determination of MT indication, for which sensitivity, specificity, and accuracy were calculated. Results: The AI model had an overall accuracy of 88%, with a specificity of 96% and a sensitivity of 80%. The area under the curve for the report-based MT decision was 0.92. Conclusions: The LLM achieved promising accuracy in determining the eligibility of patients with AIS for MT based on radiology reports and clinical information. Our results underscore the potential of LLMs for radiological and medical data integration. This investigation should serve as a stimulus for further clinical applications of LLMs, in which this AI should be used as an augmented supporting system for human decision-making. UR - https://www.jmir.org/2025/1/e48328 UR - http://dx.doi.org/10.2196/48328 UR - http://www.ncbi.nlm.nih.gov/pubmed/39946168 ID - info:doi/10.2196/48328 ER - TY - JOUR AU - Zuidhof, Niek AU - Peters, Oscar AU - Verbeek, Peter-Paul AU - Ben Allouch, Somaya PY - 2025/2/11 TI - Social Acceptance of Smart Glasses in Health Care: Model Evaluation Study of Anticipated Adoption and Social Interaction JO - JMIR Form Res SP - e49610 VL - 9 KW - smart glasses KW - technology adoption KW - social interaction KW - instrument development KW - structural equation modeling N2 - Background: Despite the growing interest in smart glasses, it is striking that they are not widespread among health care professionals. Previous research has identified issues related to social interactions involving the use of smart glasses in public settings, which may differ from those associated with their application in health care contexts. Objective: Assuming that smart glasses mediate contact between the health care provider and patient, the objectives of this research are two-fold: (1) to develop an instrument that combines the adoption and mediation perspectives, and (2) to gain insights into how the intention to use is influenced through aspects of adoption and social interaction. Methods: A questionnaire was administered to a target audience of health care professionals (N=450), with recruitment via MTurk. The sample primarily included male participants from the United States, with the majority aged 42 years or younger. Although a large portion of respondents were medical doctors, the sample also included nurses and other health care professionals. Data were analyzed by structural equation modeling. Results: Regarding the aim of developing an instrument combining adoption and social interaction, the internal consistency was above the aspirational level (?>.70) for the instrument. Furthermore, regarding the second objective involving gaining insights into the influential constructs of the anticipated intention to use, the following results were highlighted: in testing the conceptual model, the measurement model generated a good fit and the respecified structural model also generated a good fit. The tested hypotheses confirmed that social interaction constructs could explain a higher variance of users? anticipated intention to use. Perceived social isolation and decreased attentional allocation did not have a significant effect on attitude. Furthermore, the intention to use smart glasses despite nonacceptance of smart glasses by the patient significantly influenced the anticipated intention to use. In summary, constructs that focus on social interaction could contribute to better explanation and prediction of the expected adoption of smart glasses in health care. Conclusions: The empirical findings of this study provide new insights into how the mediation perspective can increase the explained variance compared to existing knowledge about adoption. Against expectations based on previous literature and despite the social issues raised earlier, these social aspects do play important roles for health care professionals but are ultimately not decisive for the intention to use. As a result, there are fewer threats to the adoption of smart glasses from the perspective of health care professionals than might be expected based on the previous literature. Therefore, the use of smart glasses can still be considered as an innovative way of working in health care. UR - https://formative.jmir.org/2025/1/e49610 UR - http://dx.doi.org/10.2196/49610 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/49610 ER - TY - JOUR AU - Bhavnani, K. Suresh AU - Zhang, Weibin AU - Bao, Daniel AU - Raji, Mukaila AU - Ajewole, Veronica AU - Hunter, Rodney AU - Kuo, Yong-Fang AU - Schmidt, Susanne AU - Pappadis, R. Monique AU - Smith, Elise AU - Bokov, Alex AU - Reistetter, Timothy AU - Visweswaran, Shyam AU - Downer, Brian PY - 2025/2/11 TI - Subtyping Social Determinants of Health in the "All of Us" Program: Network Analysis and Visualization Study JO - J Med Internet Res SP - e48775 VL - 27 KW - social determinants of health KW - All of Us KW - bipartite networks KW - financial resources KW - health care KW - health outcomes KW - precision medicine KW - decision support KW - health industry KW - clinical implications KW - machine learning methods N2 - Background: Social determinants of health (SDoH), such as financial resources and housing stability, account for between 30% and 55% of people?s health outcomes. While many studies have identified strong associations between specific SDoH and health outcomes, little is known about how SDoH co-occur to form subtypes critical for designing targeted interventions. Such analysis has only now become possible through the All of Us program. Objective: This study aims to analyze the All of Us dataset for addressing two research questions: (1) What are the range of and responses to survey questions related to SDoH? and (2) How do SDoH co-occur to form subtypes, and what are their risks for adverse health outcomes? Methods: For question 1, an expert panel analyzed the range of and responses to SDoH questions across 6 surveys in the full All of Us dataset (N=372,397; version 6). For question 2, due to systematic missingness and uneven granularity of questions across the surveys, we selected all participants with valid and complete SDoH data and used inverse probability weighting to adjust their imbalance in demographics. Next, an expert panel grouped the SDoH questions into SDoH factors to enable more consistent granularity. To identify the subtypes, we used bipartite modularity maximization for identifying SDoH biclusters and measured their significance and replicability. Next, we measured their association with 3 outcomes (depression, delayed medical care, and emergency room visits in the last year). Finally, the expert panel inferred the subtype labels, potential mechanisms, and targeted interventions. Results: The question 1 analysis identified 110 SDoH questions across 4 surveys covering all 5 domains in Healthy People 2030. As the SDoH questions varied in granularity, they were categorized by an expert panel into 18 SDoH factors. The question 2 analysis (n=12,913; d=18) identified 4 biclusters with significant biclusteredness (Q=0.13; random-Q=0.11; z=7.5; P<.001) and significant replication (real Rand index=0.88; random Rand index=0.62; P<.001). Each subtype had significant associations with specific outcomes and had meaningful interpretations and potential targeted interventions. For example, the Socioeconomic barriers subtype included 6 SDoH factors (eg, not employed and food insecurity) and had a significantly higher odds ratio (4.2, 95% CI 3.5-5.1; P<.001) for depression when compared to other subtypes. The expert panel inferred implications of the results for designing interventions and health care policies based on SDoH subtypes. Conclusions: This study identified SDoH subtypes that had statistically significant biclusteredness and replicability, each of which had significant associations with specific adverse health outcomes and with translational implications for targeted SDoH interventions and health care policies. However, the high degree of systematic missingness requires repeating the analysis as the data become more complete by using our generalizable and scalable machine learning code available on the All of Us workbench. UR - https://www.jmir.org/2025/1/e48775 UR - http://dx.doi.org/10.2196/48775 UR - http://www.ncbi.nlm.nih.gov/pubmed/39932771 ID - info:doi/10.2196/48775 ER - TY - JOUR AU - Khairat, Saif AU - Morelli, Jennifer AU - Boynton, H. Marcella AU - Bice, Thomas AU - Gold, A. Jeffrey AU - Carson, S. Shannon PY - 2025/2/11 TI - Investigation of Information Overload in Electronic Health Records: Protocol for Usability Study JO - JMIR Res Protoc SP - e66127 VL - 14 KW - electronic health records KW - information overload KW - eye-tracking KW - EHR usability KW - EHR interface N2 - Background: Electronic health records (EHRs) have been associated with information overload, causing providers to miss critical information, make errors, and delay care. Information overload can be especially prevalent in medical intensive care units (ICUs) where patients are often critically ill and their charts contain large amounts of data points such as vitals, test and laboratory results, medications, and notes. Objective: We propose to study the relationship between information overload and EHR use among medical ICU providers in 4 major United States medical centers. In this study, we examined 2 prominent EHR systems in the United States to generate reproducible and generalizable findings. Methods: Our study collected physiological and objective data through the use of a screen-mounted eye-tracker. We aim to characterize information overload in the EHR by examining ICU providers? decision-making and EHR usability. We also surveyed providers on their institution?s EHR to better understand how they rate the system?s task load and usability using the NASA (National Aeronautics and Space Administration) Task Load Index and Computer System Usability Questionnaire. Primary outcomes include the number of eye fixations during each case, the number of correct decisions, the time to complete each case, and number of screens visited. Secondary outcomes include case complexity performance, frequency of mouse clicks, and EHR task load and usability using provided surveys. Results: This EHR usability study was funded in 2021. The study was initiated in 2022 with a completion date of 2025. Data collection for this study was completed in December 2023 and data analysis is ongoing with a total of 81 provider sessions recorded. Conclusions: Our study aims to characterize information overload in the EHR among medical ICU providers. By conducting a multisite, cross-sectional usability assessment of information overload in 2 leading EHRs, we hope to reveal mechanisms that explain information overload. The insights gained from this study may lead to potential improvements in EHR usability and interface design, which could improve health care delivery and patient safety. International Registered Report Identifier (IRRID): DERR1-10.2196/66127 UR - https://www.researchprotocols.org/2025/1/e66127 UR - http://dx.doi.org/10.2196/66127 UR - http://www.ncbi.nlm.nih.gov/pubmed/39932774 ID - info:doi/10.2196/66127 ER - TY - JOUR AU - Ray, Katherine Mary AU - Fleming, Jorie AU - Aschenbrenner, Andrew AU - Hassenstab, Jason AU - Redwine, Brooke AU - Burns, Carissa AU - Arbelaez, Maria Ana AU - Vajravelu, Ellen Mary AU - Hershey, Tamara PY - 2025/2/11 TI - Assessing Dynamic Cognitive Function in the Daily Lives of Youths With and Without Type 1 Diabetes: Usability Study JO - JMIR Form Res SP - e60275 VL - 9 KW - ecological momentary assessment KW - EMA KW - ambulatory KW - smartphone KW - continuous glucose monitoring KW - CGM KW - assessment KW - daily lives KW - youth KW - type 1 diabetes KW - diabetes KW - feasibility study KW - pilot study KW - glycemic control KW - environmental factor KW - phone KW - acceptability KW - young KW - cognitive test KW - app KW - application KW - mobile phone N2 - Background: Studies have shown a relationship between worse glycemic control and lower cognitive scores in youths with type 1 diabetes (T1D). However, most studies assess long-term glucose control (eg, years-decades) and cognition at a single time point. Understanding this relationship at a higher temporal resolution (eg, minutes-hours) and in naturalistic settings has potential clinical implications. Newer technology (eg, continuous glucose monitoring [CGM] and ecological momentary assessment) provides a unique opportunity to explore the glucose dynamics that influence dynamic cognition; that is, cognitive functions that fluctuate short-term and are influenced by environmental factors. Objective: Before we can assess this relationship, we need to determine the feasibility of measuring cognition in youths in daily life and determine the plausibility of obtaining glucose variation with CGM to be integrated with real-time cognition measures. This study?s purpose was to assess the acceptability of measuring dynamic cognition using a smartphone app and adherence to cognitive testing in daily life in youths with and without T1D. Further, we assessed CGM-derived glucose measures at temporally related timeframes to cognitive testing in naturalistic settings. Methods: Data were obtained from 3 studies including one in-laboratory study and 2 remote studies. For all studies, youths were asked to complete cognitive tests on the Ambulatory Research in Cognition (ARC) smartphone app that measured processing speed, associative memory, and working memory. For the in-laboratory study, youths completed testing 4 times during 1 session. For the remote studies, youths were asked to complete cognitive tests 5 times per day for either 10 or 14 consecutive days in daily life. Youths were asked to rate their impressions of the app. Youths with T1D wore a CGM. Results: 74 youths (n=53 control; n=21 T1D) aged 4?16 years participated. Youths generally reported liking or understanding the ARC app tasks in a laboratory and remote setting. Youths had high testing adherence in daily life (2350/3080 to 721/900, 76.3%?80.2%) and none dropped out. The percentage of measurements within each glycemic range taken immediately before the app?s cognitive testing was 3% (28/942) low glucose, 51% (484/942) euglycemia, 23% (221/942) high glucose, and 22% (210/942) very high glucose. In the 2-hour window before each cognitive task, mean glucose was 182.5 (SD 76.2) mg/dL, SD in glucose was 27.1 mg/dL (SD 18.7), and the mean maximum difference between the highest and lowest glucose was 85.5 (SD 53.7) mg/dL. Conclusions: The results suggest that using the ARC smartphone app to assess dynamic cognitive functions in youths with and without T1D is feasible. Further, we showed CGM-derived glycemic variability at temporally associated timeframes of dynamic cognitive assessments. The next steps include using ecological momentary assessment in a fully powered study to determine the relationship between short-term glycemic control and cognition in youths with T1D. UR - https://formative.jmir.org/2025/1/e60275 UR - http://dx.doi.org/10.2196/60275 ID - info:doi/10.2196/60275 ER - TY - JOUR AU - Lee, Kyungmi AU - Azuero, Andres AU - Engler, Sally AU - Kumar, Sidharth AU - Puga, Frank AU - Wright, A. Alexi AU - Kamal, Arif AU - Ritchie, S. Christine AU - Demiris, George AU - Bakitas, A. Marie AU - Odom, Nicholas J. PY - 2025/2/7 TI - Exploring the Relationship Between Smartphone GPS Patterns and Quality of Life in Patients With Advanced Cancer and Their Family Caregivers: Longitudinal Study JO - JMIR Form Res SP - e59161 VL - 9 KW - cancer KW - digital phenotyping KW - global positioning system KW - quality of life KW - smartphone KW - mobile phone KW - family caregiver N2 - Background: Patients with advanced cancer and their family caregivers often experience poor quality of life (QOL). Self-report measures are commonly used to quantify QOL of family caregivers but may have limitations such as recall bias and social desirability bias. Variables derived from passively obtained smartphone GPS data are a novel approach to measuring QOL that may overcome these limitations and enable detection of early signs of mental and physical health (PH) deterioration. Objective: This study explored the feasibility of a digital phenotyping approach by assessing participant adherence and examining correlations between smartphone GPS data and QOL levels among family caregivers and patients with advanced cancer. Methods: This was a secondary analysis involving 7 family caregivers and 4 patients with advanced cancer that assessed correlations between GPS sensor data captured by a personally owned smartphone and QOL self-report measures over 12 weeks through linear correlation coefficients. QOL as measured by the Patient-Reported Outcomes Measurement Information System (PROMIS) Global Health 10 was collected at baseline, 6, and 12 weeks. Using a Beiwe smartphone app, GPS data were collected and processed into variables including total distance, time spent at home, transition time, and number of significant locations. Results: The study identified relevant temporal correlations between QOL and smartphone GPS data across specific time periods. For instance, in terms of PH, associations were observed with the total distance traveled (12 and 13 wk, with r ranging 0.37 to 0.38), time spent at home (?4 to ?2 wk, with r ranging from ?0.41 to ?0.49), and transition time (?4 to ?2 wk, with r ranging ?0.38 to ?0.47). Conclusions: This research offers insights into using passively obtained smartphone GPS data as a novel approach for assessing and monitoring QOL among family caregivers and patients with advanced cancer, presenting potential advantages over traditional self-report measures. The observed correlations underscore the potential of this method to detect early signs of deteriorating mental health and PH, providing opportunities for timely intervention and support. UR - https://formative.jmir.org/2025/1/e59161 UR - http://dx.doi.org/10.2196/59161 ID - info:doi/10.2196/59161 ER - TY - JOUR AU - Bragazzi, Luigi Nicola AU - Buchinger, Michèle AU - Atwan, Hisham AU - Tuma, Ruba AU - Chirico, Francesco AU - Szarpak, Lukasz AU - Farah, Raymond AU - Khamisy-Farah, Rola PY - 2025/2/5 TI - Proficiency, Clarity, and Objectivity of Large Language Models Versus Specialists? Knowledge on COVID-19's Impacts in Pregnancy: Cross-Sectional Pilot Study JO - JMIR Form Res SP - e56126 VL - 9 KW - COVID-19 KW - vaccine KW - reproductive health KW - generative artificial intelligence KW - large language model KW - chatGPT KW - google bard KW - microsoft copilot KW - vaccination KW - natural language processing KW - obstetric KW - gynecology KW - women KW - text mining KW - sentiment KW - accuracy KW - zero shot KW - pregnancy KW - readability KW - infectious N2 - Background: The COVID-19 pandemic has significantly strained health care systems globally, leading to an overwhelming influx of patients and exacerbating resource limitations. Concurrently, an ?infodemic? of misinformation, particularly prevalent in women?s health, has emerged. This challenge has been pivotal for health care providers, especially gynecologists and obstetricians, in managing pregnant women?s health. The pandemic heightened risks for pregnant women from COVID-19, necessitating balanced advice from specialists on vaccine safety versus known risks. In addition, the advent of generative artificial intelligence (AI), such as large language models (LLMs), offers promising support in health care. However, they necessitate rigorous testing. Objective: This study aimed to assess LLMs? proficiency, clarity, and objectivity regarding COVID-19?s impacts on pregnancy. Methods: This study evaluates 4 major AI prototypes (ChatGPT-3.5, ChatGPT-4, Microsoft Copilot, and Google Bard) using zero-shot prompts in a questionnaire validated among 159 Israeli gynecologists and obstetricians. The questionnaire assesses proficiency in providing accurate information on COVID-19 in relation to pregnancy. Text-mining, sentiment analysis, and readability (Flesch-Kincaid grade level and Flesch Reading Ease Score) were also conducted. Results: In terms of LLMs? knowledge, ChatGPT-4 and Microsoft Copilot each scored 97% (32/33), Google Bard 94% (31/33), and ChatGPT-3.5 82% (27/33). ChatGPT-4 incorrectly stated an increased risk of miscarriage due to COVID-19. Google Bard and Microsoft Copilot had minor inaccuracies concerning COVID-19 transmission and complications. In the sentiment analysis, Microsoft Copilot achieved the least negative score (?4), followed by ChatGPT-4 (?6) and Google Bard (?7), while ChatGPT-3.5 obtained the most negative score (?12). Finally, concerning the readability analysis, Flesch-Kincaid Grade Level and Flesch Reading Ease Score showed that Microsoft Copilot was the most accessible at 9.9 and 49, followed by ChatGPT-4 at 12.4 and 37.1, while ChatGPT-3.5 (12.9 and 35.6) and Google Bard (12.9 and 35.8) generated particularly complex responses. Conclusions: The study highlights varying knowledge levels of LLMs in relation to COVID-19 and pregnancy. ChatGPT-3.5 showed the least knowledge and alignment with scientific evidence. Readability and complexity analyses suggest that each AI?s approach was tailored to specific audiences, with ChatGPT versions being more suitable for specialized readers and Microsoft Copilot for the general public. Sentiment analysis revealed notable variations in the way LLMs communicated critical information, underscoring the essential role of neutral and objective health care communication in ensuring that pregnant women, particularly vulnerable during the COVID-19 pandemic, receive accurate and reassuring guidance. Overall, ChatGPT-4, Microsoft Copilot, and Google Bard generally provided accurate, updated information on COVID-19 and vaccines in maternal and fetal health, aligning with health guidelines. The study demonstrated the potential role of AI in supplementing health care knowledge, with a need for continuous updating and verification of AI knowledge bases. The choice of AI tool should consider the target audience and required information detail level. UR - https://formative.jmir.org/2025/1/e56126 UR - http://dx.doi.org/10.2196/56126 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56126 ER - TY - JOUR AU - Chang, Shao-Hsuan AU - Chen, Daishi AU - Chen, Chi-Sheng AU - Zhou, Dong AU - Yeh, Lung-Kun PY - 2025/2/5 TI - Device Failures and Adverse Events Associated With Rhinolaryngoscopes: Analysis of the Manufacturer and User Facility Device Experience (MAUDE) Database JO - JMIR Hum Factors SP - e67036 VL - 12 KW - medical device KW - device malfunction KW - rhinolaryngoscope KW - adverse event KW - MAUDE KW - Manufacturer and User Facility Device Experience N2 - Background: Rhinolaryngoscopes are one of the most widely used tools by otolaryngologists and speech-language pathologists in current clinical practice. However, there is limited data on adverse events associated with or caused by the use of rhinolaryngoscopes. Objective: In this study, we used the Manufacturer and User Facility Device Experience (MAUDE) database with the aim of providing insights that may assist otolaryngologists in better understanding the limitations of these devices and selecting appropriate procedures for their specific clinical setting. Methods: We characterized complications associated with the postmarket use of rhinolaryngoscope devices from the US Food and Drug Administration MAUDE database from 2016 through 2023. Results: A total of 2591 reports were identified, including 2534 device malfunctions, 56 injuries, and 1 death, from 2016 through 2023. The most common device problem with rhinolaryngoscopes was breakage (n=1058 reports, 40.8%), followed by fluid leaks (n=632 reports, 24.4%). The third most common problem was poor image quality (n=467 reports, 18%). Other device issues included contamination or device reprocessing problems (n=127 reports, 4.9%), material deformation or wear (n=125 reports, 4.8%), and device detachment (n=73 reports, 2.8%). Of the 63 reported adverse events, the most common patient-related adverse event was hemorrhage or bleeding, accounting for 18 reports, with the root causes including material deformation or wear, breakage, wrinkled rubber, or improper operation. Conclusions: Our study offers valuable insights for endoscopists and manufacturers to recognize potential issues and adverse events associated with the use of rhinolaryngoscopes. It emphasizes the need for improving device reliability, training, and procedural protocols to enhance patient safety during diagnostic procedures. UR - https://humanfactors.jmir.org/2025/1/e67036 UR - http://dx.doi.org/10.2196/67036 ID - info:doi/10.2196/67036 ER - TY - JOUR AU - Ogorek, Benjamin AU - Rhoads, Thomas AU - Smith, Erica PY - 2025/2/3 TI - Collecting Real-World Data via an In-Home Smart Medication Dispenser: Longitudinal Observational Study of Survey Panel Persistency, Response Rates, and Psychometric Properties JO - JMIR Hum Factors SP - e60438 VL - 12 KW - real-world data KW - real-world evidence KW - patient-reported outcomes KW - longitudinal studies KW - survey methods N2 - Background: A smart medication dispenser called ?spencer? is a novel generator of longitudinal survey data. The patients dispensing medication act as a survey panel and respond to questions about quality of life and patient-reported outcomes. Objectives: Our goal was to evaluate panel persistency, survey response rates, reliability, and validity of surveys administered via spencer to 4138 polychronic patients residing in the United States and Canada. Methods: Patients in a Canadian health care provider?s program were included if they were dispensing via spencer in the June 2021 to February 2024 time frame and consented to have their data used for research. Panel persistency was estimated via discrete survival methods for 2 years and survey response rates were computed for 1 year. Patients were grouped by mean response rates in the 12th month (<90% vs ?90%) to observe differential response rate trends. For reliability and validity, we used a spencer question about recent falls with ternary responses value-coded ?1, 0, and 1. For reliability, we computed Pearson correlation between mean scores over 2 years of survey responses, and transitions between mean score intervals of [0, 0.5), [?0.5, 0.5), and [0.5, 1]. For validity, we measured the association between the falls question and known factors influencing fall risk: age, biological sex, quality of life, physical and emotional health, and use of selective serotonin reuptake inhibitors or serotonin-norepinephrine reuptake inhibitors, using repeated-measures regression for covariates and Kendall ? for concomitant spencer questions. Results: From 4138 patients, dispenser persistency was 68.3% (95% CI 66.8%?69.8%) at 1 year and 51% (95% CI 49%?53%) at 2 years. Within the cohort observed beyond 1 year, 82.3% (1508/1832) kept surveys enabled through the 12th month with a mean response rate of 84.1% (SD 26.4%). The large SD was apparent in the subgroup analysis, where a responder versus nonresponder dichotomy was observed. For 234 patients with ?5 fall risk responses in each of the first 2 years, the Pearson correlation estimate between yearly mean scores was 0.723 (95% CI 0.630?0.798). For mean score intervals [0, 0.5), [?0.5, 0.5), and [0.5, 1], self-transitions were the most common, with 59.8% (140/234) of patients starting and staying in [0.5, 1]. Fall risk responses were not significantly associated with sex (P=.66) or age (P=.76) but significantly related to selective serotonin reuptake inhibitor or serotonin-norepinephrine reuptake inhibitor usage, quality of life, depressive symptoms, physical health, disability, and trips to the emergency room (P<.001). Conclusions: A smart medication dispenser, spencer, generated years of longitudinal survey data from patients in their homes. Panel attrition was low, and patients continued to respond at high rates. A fall risk measure derived from the survey data showed evidence of reliability and validity. An alternative to web-based panels, spencer is a promising tool for generating patient real-world data. UR - https://humanfactors.jmir.org/2025/1/e60438 UR - http://dx.doi.org/10.2196/60438 ID - info:doi/10.2196/60438 ER - TY - JOUR AU - Hsu, Wan-Chen PY - 2025/2/3 TI - eHealth Literacy and Cyberchondria Severity Among Undergraduate Students: Mixed Methods Study JO - JMIR Form Res SP - e63449 VL - 9 KW - eHealth literacy KW - undergraduate student KW - cyberchondria KW - compucondria KW - web-based health information KW - health information seeking KW - college students N2 - Background: With the development of the internet, health care websites have become increasingly important by enabling easy access to health information, thereby influencing the attitudes and behaviors of individuals toward health issues. However, few studies have addressed public access to health information and self-diagnosis. Objective: This study investigated the background factors and status of cyberchondria severity among college students by conducting a nationwide sample survey using the Cyberchondria Severity Scale. Further, we explored the perspective of eHealth literacy of those with scores higher than 1 SD from the mean by analyzing their recent experiences using web-based health information. Methods: A nationally representative sample of college students was surveyed, and 802 valid responses were obtained (male: 435/802, 54.2%; female: 367/802, 45.8%; mean age 20.3, SD 1.4 years). The Cyberchondria Severity Scale was used, which consisted of 4 dimensions (increased anxiety, obsessive-compulsive hypochondria, perceived controllability, and web-based physician-patient interaction). Additionally, we recruited 9 volunteers who scored more than 1 SD above the mean for in-depth interviews on their web-based health information?seeking behaviors. Results: Significant differences were found across the 4 dimensions of cyberchondria severity (F3,2403=256.26; P<.001), with perceived controllability scoring the highest (mean 2.75, SD 0.87) and obsessive-compulsive hypochondria scoring the lowest (mean 2.19, SD 0.77). Positive correlations were observed between perceived controllability, web-based physician-patient interactions, increased anxiety, and obsessive-compulsive hypochondria (r=0.46-0.75, P<.001). Regression analysis indicated that health concern significantly predicted perceived controllability (? coefficient=0.12; P<.05) and web-based physician-patient interaction (? coefficient=0.16; P<.001). Interview data revealed that students often experienced heightened anxiety (8/9, 89%) and stress (7/9, 78%) after exposure to web-based health information, highlighting the need for improved health literacy and reliable information sources. Conclusions: The study identified both benefits and risks in college students? use of web-based health information, emphasizing the importance of critical consciousness and eHealth literacy. Future research should examine how college students move from self-awareness to actionable change and the development of critical health literacy, which are essential for effective digital health engagement. UR - https://formative.jmir.org/2025/1/e63449 UR - http://dx.doi.org/10.2196/63449 ID - info:doi/10.2196/63449 ER - TY - JOUR AU - Baker, Venetia AU - Mulwa, Sarah AU - Khanyile, David AU - Arnold, Georgia AU - Cousens, Simon AU - Cawood, Cherie AU - Birdthistle, Isolde PY - 2025/1/31 TI - Evaluating Reaction Videos of Young People Watching Edutainment Media (MTV Shuga): Qualitative Observational Study JO - JMIR Form Res SP - e55275 VL - 9 KW - mass media KW - edutainment KW - adolescents KW - sexual health KW - HIV prevention KW - participatory research N2 - Background: Mass media campaigns, particularly edutainment, are critical in disseminating sexual health information to young people. However, there is limited understanding of the authentic viewing experience or how viewing contexts influence engagement with media campaigns. Reaction videos, a popular format in web-based culture in which users film themselves reacting to television shows, can be adapted as a research method for immediate and unfiltered insights into young people?s engagement with edutainment media. Objective: We explored how physical and social context influences young people?s engagement with MTV Shuga, a dramatic television series based on sexual health and relationships among individuals aged 15 to 25 years. We trialed reaction videos as a novel research method to investigate how young people in South Africa experience the show, including sexual health themes and messages, in their viewing environments. Methods: In Eastern Cape, in 2020, purposively selected participants aged 18 to 24 years of an evaluation study were invited to take part in further research to video record themselves watching MTV Shuga episodes with their COVID-19 social bubble. To guide the analysis of the visual and audio data, we created a framework to examine the physical setting, group composition, social dynamics, coinciding activities, and viewers? spoken and unspoken reactions to the show. We identified patterns within and across groups to generate themes about the nature and role of viewing contexts. We also reflected on the utility of the method and analytical framework. Results: In total, 8 participants recorded themselves watching MTV Shuga episodes in family or friendship groups. Viewings occurred around a laptop in the home (living room or bedroom) and outside (garden or vehicle). In same-age groups, viewers appeared relaxed, engaging with the content through discussion, comments, empathy, and laughter. Intergenerational groups experienced discomfort, with older relatives? presence causing embarrassment and younger siblings? distractions interrupting the engagement. Scenes featuring physical intimacy prompted some viewers to hide their eyes or leave the room. While some would prefer watching MTV Shuga alone to avoid the self-consciousness experienced in group settings, others valued the social experience and the lively discussions it spurred. This illustrates varied preferences for consuming edutainment and the factors influencing these preferences. Conclusions: The use of reaction videos for research captured real-time verbal and nonverbal reactions, physical environments, and social dynamics that other methods cannot easily measure. They revealed how group composition, dynamics, settings, and storylines can maximize engagement with MTV Shuga to enhance HIV prevention education. The presence of parents and the camera may alter young people?s behavior, limiting the authenticity of their viewing experience. Still, reaction videos offer a unique opportunity to understand audience engagement with media interventions and promote participatory digital research with young people. UR - https://formative.jmir.org/2025/1/e55275 UR - http://dx.doi.org/10.2196/55275 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55275 ER - TY - JOUR AU - Bai, Anying AU - He, Shan AU - Jiang, Yu AU - Xu, Weihao AU - Lin, Zhanyi PY - 2025/1/30 TI - Comparison of 3 Aging Metrics in Dual Declines to Capture All-Cause Dementia and Mortality Risk: Cohort Study JO - JMIR Aging SP - e66104 VL - 8 KW - gerontology KW - geriatrics KW - older adults KW - older people KW - aging KW - motoric cognitive risk syndrome KW - MCR KW - physio-cognitive decline syndrome KW - PCDS KW - cognitive frailty KW - CF KW - frailty KW - discrimination KW - risk factors KW - prediction KW - dementia risk KW - mortality risk N2 - Background: The utility of aging metrics that incorporate cognitive and physical function is not fully understood. Objective: We aim to compare the predictive capacities of 3 distinct aging metrics?motoric cognitive risk syndrome (MCR), physio-cognitive decline syndrome (PCDS), and cognitive frailty (CF)?for incident dementia and all-cause mortality among community-dwelling older adults. Methods: We used longitudinal data from waves 10-15 of the Health and Retirement Study. Cox proportional hazards regression analysis was employed to evaluate the effects of MCR, PCDS, and CF on incident all-cause dementia and mortality, controlling for socioeconomic and lifestyle factors, as well as medical comorbidities. Discrimination analysis was conducted to assess and compare the predictive accuracy of the 3 aging metrics. Results: A total of 2367 older individuals aged 65 years and older, with no baseline prevalence of dementia or disability, were ultimately included. The prevalence rates of MCR, PCDS, and CF were 5.4%, 6.3%, and 1.3%, respectively. Over a decade-long follow-up period, 341 cases of dementia and 573 deaths were recorded. All 3 metrics were predictive of incident all-cause dementia and mortality when adjusting for multiple confounders, with variations in the strength of their associations (incident dementia: MCR odds ratio [OR] 1.90, 95% CI 1.30?2.78; CF 5.06, 95% CI 2.87?8.92; PCDS 3.35, 95% CI 2.44?4.58; mortality: MCR 1.60, 95% CI 1.17?2.19; CF 3.26, 95% CI 1.99?5.33; and PCDS 1.58, 95% CI 1.17?2.13). The C-index indicated that PCDS and MCR had the highest discriminatory accuracy for all-cause dementia and mortality, respectively. Conclusions: Despite the inherent differences among the aging metrics that integrate cognitive and physical functions, they consistently identified risks of dementia and mortality. This underscores the importance of implementing targeted preventive strategies and intervention programs based on these metrics to enhance the overall quality of life and reduce premature deaths in aging populations. UR - https://aging.jmir.org/2025/1/e66104 UR - http://dx.doi.org/10.2196/66104 ID - info:doi/10.2196/66104 ER - TY - JOUR AU - Puttkammer, Nancy AU - Dunbar, Elizabeth AU - Germanovych, Myroslava AU - Rosol, Mariia AU - Golden, Matthew AU - Hubashova, Anna AU - Fedorchenko, Vladyslav AU - Hetman, Larisa AU - Legkostup, Liudmyla AU - Flowers, Jan AU - Nesterova, Olena PY - 2025/1/30 TI - Human-Centered Design of an mHealth Tool for Optimizing HIV Index Testing in Wartime Ukraine: Formative Research Case Study JO - JMIR Form Res SP - e66132 VL - 9 KW - human-centered design KW - mobile health KW - mHealth KW - Ukraine KW - HIV testing KW - war and humanitarian settings N2 - Background: Assisted partner services (APSs; sometimes called index testing) are now being brought to scale as a high-yield HIV testing strategy in many nations. However, the success of APSs is often hampered by low levels of partner elicitation. The Computer-Assisted Self-Interview (CASI)?Plus study sought to develop and test a mobile health (mHealth) tool to increase the elicitation of sexual and needle-sharing partners among persons with newly diagnosed HIV. CASI-Plus provides client-facing information on APS methods and uses a standardized, self-guided questionnaire with nonjudgmental language for clients to list partners who would benefit from HIV testing. The tool also enables health care workers (HCWs) to see summarized data to facilitate partner tracking. Objective: The formative research phase of the CASI-Plus study aimed to gather client and HCW input on the design of the CASI-Plus tool to ensure its acceptability, feasibility, and usability. Methods: This study gathered input to prioritize features and tested the usability of CASI-Plus with HCWs and clients receiving HIV services in public health clinics in wartime Ukraine. The CASI-Plus study?s formative phase, carried out from May 2023 to July 2024, adapted human-centered design (HCD) methods grounded in principles of empathy, iteration, and creative ideation. The study involved 3 steps: formative HCD, including in-depth individual interviews with clients, such as men who have sex with men and people who inject drugs, and internet-based design workshops with HCWs from rural and urban HIV clinics in Chernihiv and Dnipro; software platform assessment and heuristic evaluation, including assessment of open-source mHealth platforms against CASI-Plus requirements, prototype development, and testing of the REDCap (Research Electronic Data Capture) prototype based on usability heuristics; and usability walk-throughs, including simulated cases with HCWs and clients. Results: The formative phase of the CASI-Plus study included in-depth individual interviews with 10 clients and 3 workshops with 22 HCWs. This study demonstrated how simplified HCD methods, adapted to the wartime context, gathered rich input on prioritized features and tool design. The CASI-Plus design reflected features that are both culturally sensitive and in alignment with the constraints of Ukraine?s wartime setting. Prioritized features included information about the benefits of HIV index testing; a nonjudgmental, self-guided questionnaire to report partners; client stories; and bright images to accompany the text. Two-way SMS text messaging between clients and HCWs was deemed impractical based on risks of privacy breaches, national patient privacy regulations, and HCW workload. Conclusions: It was feasible to conduct HCD research in Ukraine in a wartime setting. The CASI-Plus mHealth tool was acceptable to both HCWs and clients. The next step for this research is a randomized clinical trial of the effect of the REDCap-based CASI-Plus tool on the number of partners named and the rate of partners completing HIV testing. UR - https://formative.jmir.org/2025/1/e66132 UR - http://dx.doi.org/10.2196/66132 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/66132 ER - TY - JOUR AU - Mayer, Anja AU - Hege, Inga AU - Kononowicz, A. Andrzej AU - Müller, Anja AU - Sudacka, Ma?gorzata PY - 2025/1/30 TI - Collaborative Development of Feedback Concept Maps for Virtual Patient?Based Clinical Reasoning Education: Mixed Methods Study JO - JMIR Med Educ SP - e57331 VL - 11 KW - clinical reasoning KW - consensus building process KW - concept map KW - consensus map KW - virtual patient KW - international collaboration KW - health professionals' education KW - undergraduate KW - collaborative KW - development KW - feedback KW - content analysis KW - health professional KW - medical student KW - mixed method KW - Europe KW - questionnaire KW - descriptive analysis N2 - Background: Concept maps are a suitable method for teaching clinical reasoning (CR). For example, in a concept map, findings, tests, differential diagnoses, and treatment options can be documented and connected to each other. When combined with virtual patients, automated feedback can be provided to the students? concept maps. However, as CR is a nonlinear process, feedback concept maps that are created together by several individuals might address this issue and cover perspectives from different health professionals. Objective: In this study, we aimed to develop a collaborative process for creating feedback concept maps in virtual patient?based CR education. Methods: Health professionals of different specialties, nationalities, and levels of experience in education individually created concept maps and afterward reached a consensus on them in structured workshops. Then, medical students discussed the health professionals? concept maps in focus groups. We performed a qualitative content analysis of the transcribed audio records and field notes and a descriptive comparison of the produced concept maps. Results: A total of 14 health professionals participated in 4 workshops, each with 3?4 participants. In each workshop, they reached a consensus on 1 concept map, after discussing content and presentation, as well as rationales, and next steps. Overall, the structure of the workshops was well-received. The comparison of the produced concept maps showed that they varied widely in their scope and content. Consensus concept maps tended to contain more nodes and connections than individual ones. A total of 9 medical students participated in 2 focus groups of 4 and 5 participants. Their opinions on the concept maps? features varied widely, balancing between the wish for an in-depth explanation and the flexibility of CR. Conclusions: Although the number of participating health professionals and students was relatively low, we were able to show that consensus workshops are a constructive method to create feedback concept maps that include different perspectives of health professionals with content that is useful to and accepted by students. Further research is needed to determine which features of feedback concept maps are most likely to improve learner outcomes and how to facilitate their construction in collaborative consensus workshops. UR - https://mededu.jmir.org/2025/1/e57331 UR - http://dx.doi.org/10.2196/57331 ID - info:doi/10.2196/57331 ER - TY - JOUR AU - Choomung, Pichsinee AU - He, Yupeng AU - Matsunaga, Masaaki AU - Sakuma, Kenji AU - Kishi, Taro AU - Li, Yuanying AU - Tanihara, Shinichi AU - Iwata, Nakao AU - Ota, Atsuhiko PY - 2025/1/29 TI - Estimating the Prevalence of Schizophrenia in the General Population of Japan Using an Artificial Neural Network?Based Schizophrenia Classifier: Web-Based Cross-Sectional Survey JO - JMIR Form Res SP - e66330 VL - 9 KW - schizophrenia KW - schizophrenic KW - prevalence KW - artificial neural network KW - neural network KW - neural networks KW - ANN KW - deep learning KW - machine learning KW - SZ classifier KW - web-based survey KW - epidemiology KW - epidemiological KW - Japan KW - classifiers KW - mental illness KW - mental disorder KW - mental health N2 - Background: Estimating the prevalence of schizophrenia in the general population remains a challenge worldwide, as well as in Japan. Few studies have estimated schizophrenia prevalence in the Japanese population and have often relied on reports from hospitals and self-reported physician diagnoses or typical schizophrenia symptoms. These approaches are likely to underestimate the true prevalence owing to stigma, poor insight, or lack of access to health care among respondents. To address these issues, we previously developed an artificial neural network (ANN)?based schizophrenia classification model (SZ classifier) using data from a large-scale Japanese web-based survey to enhance the comprehensiveness of schizophrenia case identification in the general population. In addition, we also plan to introduce a population-based survey to collect general information and sample participants matching the population?s demographic structure, thereby achieving a precise estimate of the prevalence of schizophrenia in Japan. Objective: This study aimed to estimate the prevalence of schizophrenia by applying the SZ classifier to random samples from the Japanese population. Methods: We randomly selected a sample of 750 participants where the age, sex, and regional distributions were similar to Japan?s demographic structure from a large-scale Japanese web-based survey. Demographic data, health-related backgrounds, physical comorbidities, psychiatric comorbidities, and social comorbidities were collected and applied to the SZ classifier, as this information was also used for developing the SZ classifier. The crude prevalence of schizophrenia was calculated through the proportion of positive cases detected by the SZ classifier. The crude estimate was further refined by excluding false-positive cases and including false-negative cases to determine the actual prevalence of schizophrenia. Results: Out of 750 participants, 62 were classified as schizophrenia cases by the SZ classifier, resulting in a crude prevalence of schizophrenia in the general population of Japan of 8.3% (95% CI 6.6%-10.1%). Among these 62 cases, 53 were presumed to be false positives, and 3 were presumed to be false negatives. After adjustment, the actual prevalence of schizophrenia in the general population was estimated to be 1.6% (95% CI 0.7%-2.5%). Conclusions: This estimated prevalence was slightly higher than that reported in previous studies, possibly due to a more comprehensive disease classification methodology or, conversely, model limitations. This study demonstrates the capability of an ANN-based model to improve the estimation of schizophrenia prevalence in the general population, offering a novel approach to public health analysis. UR - https://formative.jmir.org/2025/1/e66330 UR - http://dx.doi.org/10.2196/66330 ID - info:doi/10.2196/66330 ER - TY - JOUR AU - Sutan, Rosnah AU - Ismail, Shahida AU - Ibrahim, Roszita PY - 2025/1/29 TI - Evaluating the Development, Reliability, and Validation of the Tele-Primary Care Oral Health Clinical Information System Questionnaire: Cross-Sectional Questionnaire Study JO - JMIR Hum Factors SP - e53630 VL - 12 KW - telehealth KW - electronic health KW - eHealth KW - public health information system KW - psychometric analysis N2 - Background: Evaluating digital health service delivery in primary health care requires a validated questionnaire to comprehensively assess users? ability to implement tasks customized to the program?s needs. Objective: This study aimed to develop, test the reliability of, and validate the Tele-Primary Care Oral Health Clinical Information System (TPC-OHCIS) questionnaire for evaluating the implementation of maternal and child digital health information systems. Methods: A cross-sectional study was conducted in 2 phases. The first phase focused on content item development and was validated by a group of 10 experts using the content validity index. The second phase was to assess its psychometric testing for reliability and validity. Results: A structured questionnaire of 65 items was constructed to assess the TPC-OHCIS delivery for primary health care use based on literature and has been validated by 10 experts, and 319 respondents answered the 65-item TPC-OHCIS questionnaire, with mean item scores ranging from 1.99 (SD 0.67) to 2.85 (SD 1.019). The content validity, reliability, and face validity showed a scale-level content validity index of 0.90, scale-level content validation ratio of 0.90, and item-level face validity index of 0.76, respectively. The internal reliability was calculated as a Cronbach ? value of 0.90, with an intraclass correlation coefficient of 0.91. Scales were determined by the scree plot with eigenvalues >1, and 13 subscales were identified based on principal component analysis. The Kaiser-Meyer-Olkin value was 0.90 (P<.049). The total variance explained was 76.07%, and factor loading scores for all variables were >0.7. The Bartlett test of sphericity, determining construct validity, was found to be significant (P<.049). Conclusions: The TPC-OHCIS questionnaire is valid to be used at the primary health care level to evaluate the TPC-OHCIS implementation. It can assess health care workers? work performance and job acceptance and improve the quality of care. UR - https://humanfactors.jmir.org/2025/1/e53630 UR - http://dx.doi.org/10.2196/53630 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53630 ER - TY - JOUR AU - Ennis, Edel AU - Bond, Raymond AU - Mulvenna, Maurice AU - Sweeney, Colm PY - 2025/1/29 TI - Understanding Individual Differences in Happiness Sources and Implications for Health Technology Design: Exploratory Analysis of an Open Dataset JO - JMIR Form Res SP - e65658 VL - 9 KW - happiness KW - sexes KW - age KW - marital status KW - parents KW - affections KW - achievements KW - datasets KW - digital health KW - well-being KW - mental health KW - digital mental health interventions KW - regression analyses KW - evidence based N2 - Background: Psychologists have developed frameworks to understand many constructs, which have subsequently informed the design of digital mental health interventions (DMHIs) aimed at improving mental health outcomes. The science of happiness is one such domain that holds significant applied importance due to its links to well-being and evidence that happiness can be cultivated through interventions. However, as with many constructs, the unique ways in which individuals experience happiness present major challenges for designing personalized DMHIs. Objective: This paper aims to (1) present an analysis of how sex may interact with age, marital status, and parental status to predict individual differences in sources of happiness, and (2) to present a preliminary discussion of how open datasets may contribute to the process of designing health-related technology innovations. Methods: The HappyDB is an open database of 100,535 statements of what people consider to have made them happy, with some people asking to consider the past 24 hours (49,831 statements) and some considering the last 3 months (50,704 statements). Demographic information is also provided. Binary logistic regression analyses are used to determine whether various groups differed in their likelihood of selecting or not selecting a category as a source of their happiness. Results: Sex and age interacted to influence what was selected as sources of happiness, with patterns being less consistent among female individuals in comparison with male individuals. For marital status, differences in sources of happiness were predominantly between married individuals and those who are divorced or separated, but these were the same for both sexes. Married, single, and widowed individuals were all largely similar in their likelihood of selecting each of the categories as a source of their happiness. However, there were some anomalies, and sex appeared to be important in these anomalies. Sex and parental status also interacted to influence what was selected as sources of happiness. Conclusions: Sex interacts with age, marital status, and parental status in the likelihood of reporting affection, bonding, leisure, achievement, or enjoying the moment as sources of happiness. The contribution of an open dataset to understanding individual differences in sources of happiness is discussed in terms of its potential role in addressing the challenges of designing DMHIs that are ethical, responsible, evidence based, acceptable, engaging, inclusive, and effective for users. The discussion considers how the content design of DMHIs in general may benefit from exploring new methods informed by diverse data sources. It is proposed that examining the extent to which insights from nondigital settings can inform requirements gathering for DMHIs is warranted. UR - https://formative.jmir.org/2025/1/e65658 UR - http://dx.doi.org/10.2196/65658 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65658 ER - TY - JOUR AU - Chan, J. Garrett AU - Fung, Mark AU - Warrington, Jill AU - Nowak, A. Sarah PY - 2025/1/29 TI - Understanding Health-Related Discussions on Reddit: Development of a Topic Assignment Method and Exploratory Analysis JO - JMIR Form Res SP - e55309 VL - 9 KW - digital health KW - internet KW - open data KW - social networking KW - social media N2 - Background: Social media has become a widely used way for people to share opinions about health care and medical topics. Social media data can be leveraged to understand patient concerns and provide insight into why patients may turn to the internet instead of the health care system for health advice. Objective: This study aimed to develop a method to investigate Reddit posts discussing health-related conditions. Our goal was to characterize these topics and identify trends in these social media?based medical discussions. Methods: Using an initial query, we collected 1 year of Reddit posts containing the phrases ?get tested? and ?get checked.? These posts were manually reviewed, and subreddits containing irrelevant posts were excluded from analysis. This selection of posts was manually read by the investigators to categorize posts into topics. A script was developed to automatically assign topics to additional posts based on keywords. Topic and keyword selections were refined based on manual review for more accurate topic assignment. Topic assignment was then performed on the entire 1-year Reddit dataset containing 347,130 posts. Related topics were grouped into broader medical disciplines. Analysis of the topic assignments was then conducted to assess condition and medical topic frequencies in medical condition?focused subreddits and general subreddits. Results: We created an automated algorithm to assign medical topics to Reddit posts. By iterating through multiple rounds of topic assignment, we improved the accuracy of the algorithm. Ultimately, this algorithm created 82 topics sorted into 17 broader medical disciplines. Of all topics, sexually transmitted infections (STIs), eye disorders, anxiety, and pregnancy had the highest post frequency overall. STIs comprised 7.44% (5876/78,980) of posts, and anxiety comprised 5.43% (4289/78,980) of posts. A total of 34% (28/82) of the topics comprised 80% (63,184/78,980) of all posts. Of the medical disciplines, those with the most posts were psychiatry and mental health; genitourinary and reproductive health; infectious diseases; and endocrinology, nutrition, and metabolism. Psychiatry and mental health comprised 26.6% (21,009/78,980) of posts, and genitourinary and reproductive health comprised 13.6% (10,741/78,980) of posts. Overall, most posts were also classified under these 4 medical disciplines. During analysis, subreddits were also classified as general if they did not focus on a specific health issue and topic-specific if they discussed a specific medical issue. Topics that appeared most frequently in the top 5 in general subreddits included addiction and drug anxiety, attention-deficit/hyperactivity disorder, abuse, and STIs. In topic-specific subreddits, most posts were found to discuss the topic of that subreddit. Conclusions: Certain health topics and medical disciplines are predominant on Reddit. These include topics such as STIs, eye disorders, anxiety, and pregnancy. Most posts were classified under the medical disciplines of psychiatry and mental health, as well as genitourinary and reproductive health. UR - https://formative.jmir.org/2025/1/e55309 UR - http://dx.doi.org/10.2196/55309 UR - http://www.ncbi.nlm.nih.gov/pubmed/39879094 ID - info:doi/10.2196/55309 ER - TY - JOUR AU - Sahandi Far, Mehran AU - Fischer, M. Jona AU - Senge, Svea AU - Rathmakers, Robin AU - Meissner, Thomas AU - Schneble, Dominik AU - Narava, Mamaka AU - Eickhoff, B. Simon AU - Dukart, Juergen PY - 2025/1/28 TI - Cross-Platform Ecological Momentary Assessment App (JTrack-EMA+): Development and Usability Study JO - J Med Internet Res SP - e51689 VL - 27 KW - digital biomarkers KW - mobile health KW - remote monitoring KW - smartphone KW - mobile phone KW - monitoring KW - biomarker KW - ecological momentary assessment KW - application KW - costly KW - user experience KW - data management KW - mobility N2 - Background: Traditional in-clinic methods of collecting self-reported information are costly, time-consuming, subjective, and often limited in the quality and quantity of observation. However, smartphone-based ecological momentary assessments (EMAs) provide complementary information to in-clinic visits by collecting real-time, frequent, and longitudinal data that are ecologically valid. While these methods are promising, they are often prone to various technical obstacles. However, despite the potential of smartphone-based EMAs, they face technical obstacles that impact adaptability, availability, and interoperability across devices and operating systems. Deficiencies in these areas can contribute to selection bias by excluding participants with unsupported devices or limited digital literacy, increase development and maintenance costs, and extend deployment timelines. Moreover, these limitations not only impede the configurability of existing solutions but also hinder their adoption for addressing diverse clinical challenges. Objective: The primary aim of this research was to develop a cross-platform EMA app that ensures a uniform user experience and core features across various operating systems. Emphasis was placed on maximizing the integration and adaptability to various study designs, all while maintaining strict adherence to security and privacy protocols. JTrack-EMA+ was designed and implemented per the FAIR (findable, accessible, interpretable, and reusable) principles in both its architecture and data management layers, thereby reducing the burden of integration for clinicians and researchers. Methods: JTrack-EMA+ was built using the Flutter framework, enabling it to run seamlessly across different platforms. This platform comprises two main components. JDash (Research Centre Jülich, Institute of Neuroscience and Medicine, Brain and Behaviour [INM-7]) is an online management tool created using Python (Python Software Foundation) with the Django (Django Software Foundation) framework. This online dashboard offers comprehensive study management tools, including assessment design, user administration, data quality control, and a reminder casting center. The JTrack-EMA+ app supports a wide range of question types, allowing flexibility in assessment design. It also has configurable assessment logic and the ability to include supplementary materials for a richer user experience. It strongly commits to security and privacy and complies with the General Data Protection Regulations to safeguard user data and ensure confidentiality. Results: We investigated our platform in a pilot study with 480 days of follow-up to assess participants? compliance. The 6-month average compliance was 49.3%, significantly declining (P=.004) from 66.7% in the first month to 42% in the sixth month. Conclusions: The JTrack-EMA+ platform prioritizes platform-independent architecture, providing an easy entry point for clinical researchers to deploy EMA in their respective clinical studies. Remote and home-based assessments of EMA using this platform can provide valuable insights into patients? daily lives, particularly in a population with limited mobility or inconsistent access to health care services. UR - https://www.jmir.org/2025/1/e51689 UR - http://dx.doi.org/10.2196/51689 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/51689 ER - TY - JOUR AU - Wang, Jiao AU - Chen, Jianrong AU - Liu, Ying AU - Xu, Jixiong PY - 2025/1/28 TI - Use of the FHTHWA Index as a Novel Approach for Predicting the Incidence of Diabetes in a Japanese Population Without Diabetes: Data Analysis Study JO - JMIR Med Inform SP - e64992 VL - 13 KW - prediction KW - diabetes KW - risk KW - index KW - population without diabetes N2 - Background: Many tools have been developed to predict the risk of diabetes in a population without diabetes; however, these tools have shortcomings that include the omission of race, inclusion of variables that are not readily available to patients, and low sensitivity or specificity. Objective: We aimed to develop and validate an easy, systematic index for predicting diabetes risk in the Asian population. Methods: We collected the data from the NAGALA (NAfld [nonalcoholic fatty liver disease] in the Gifu Area, Longitudinal Analysis) database. The least absolute shrinkage and selection operator model was used to select potentially relevant features. Multiple Cox proportional hazard analysis was used to develop a model based on the training set. Results: The final study population of 15464 participants had a mean age of 42 (range 18-79) years; 54.5% (8430) were men. The mean follow-up duration was 6.05 (SD 3.78) years. A total of 373 (2.41%) participants showed progression to diabetes during the follow-up period. Then, we established a novel parameter (the FHTHWA index), to evaluate the incidence of diabetes in a population without diabetes, comprising 6 parameters based on the training set. After multivariable adjustment, individuals in tertile 3 had a significantly higher rate of diabetes compared with those in tertile 1 (hazard ratio 32.141, 95% CI 11.545?89.476). Time receiver operating characteristic curve analyses showed that the FHTHWA index had high accuracy, with the area under the curve value being around 0.9 during the more than 12 years of follow-up. Conclusions: This research successfully developed a diabetes risk assessment index tailored for the Japanese population by utilizing an extensive dataset and a wide range of indices. By categorizing the diabetes risk levels among Japanese individuals, this study offers a novel predictive tool for identifying potential patients, while also delivering valuable insights into diabetes prevention strategies for the healthy Japanese populace. UR - https://medinform.jmir.org/2025/1/e64992 UR - http://dx.doi.org/10.2196/64992 ID - info:doi/10.2196/64992 ER - TY - JOUR AU - Subramanian, Ajan AU - Cao, Rui AU - Naeini, Kasaeyan Emad AU - Aqajari, Hossein Seyed Amir AU - Hughes, D. Thomas AU - Calderon, Michael-David AU - Zheng, Kai AU - Dutt, Nikil AU - Liljeberg, Pasi AU - Salanterä, Sanna AU - Nelson, M. Ariana AU - Rahmani, M. Amir PY - 2025/1/27 TI - Multimodal Pain Recognition in Postoperative Patients: Machine Learning Approach JO - JMIR Form Res SP - e67969 VL - 9 KW - pain intensity recognition KW - multimodal information fusion KW - signal processing KW - weak supervision KW - health care KW - pain intensity KW - pain recognition KW - machine learning approach KW - acute pain KW - pain assessment KW - behavioral pain KW - pain measurement KW - pain monitoring KW - multimodal machine learning?based framework KW - machine learning?based framework KW - electrocardiogram KW - electromyogram KW - electrodermal activity KW - self-reported pain level KW - clinical pain management N2 - Background: Acute pain management is critical in postoperative care, especially in vulnerable patient populations that may be unable to self-report pain levels effectively. Current methods of pain assessment often rely on subjective patient reports or behavioral pain observation tools, which can lead to inconsistencies in pain management. Multimodal pain assessment, integrating physiological and behavioral data, presents an opportunity to create more objective and accurate pain measurement systems. However, most previous work has focused on healthy subjects in controlled environments, with limited attention to real-world postoperative pain scenarios. This gap necessitates the development of robust, multimodal approaches capable of addressing the unique challenges associated with assessing pain in clinical settings, where factors like motion artifacts, imbalanced label distribution, and sparse data further complicate pain monitoring. Objective: This study aimed to develop and evaluate a multimodal machine learning?based framework for the objective assessment of pain in postoperative patients in real clinical settings using biosignals such as electrocardiogram, electromyogram, electrodermal activity, and respiration rate (RR) signals. Methods: The iHurt study was conducted on 25 postoperative patients at the University of California, Irvine Medical Center. The study captured multimodal biosignals during light physical activities, with concurrent self-reported pain levels using the Numerical Rating Scale. Data preprocessing involved noise filtering, feature extraction, and combining handcrafted and automatic features through convolutional and long-short-term memory autoencoders. Machine learning classifiers, including support vector machine, random forest, adaptive boosting, and k-nearest neighbors, were trained using weak supervision and minority oversampling to handle sparse and imbalanced pain labels. Pain levels were categorized into baseline and 3 levels of pain intensity (1-3). Results: The multimodal pain recognition models achieved an average balanced accuracy of over 80% across the different pain levels. RR models consistently outperformed other single modalities, particularly for lower pain intensities, while facial muscle activity (electromyogram) was most effective for distinguishing higher pain intensities. Although single-modality models, especially RR, generally provided higher performance compared to multimodal approaches, our multimodal framework still delivered results that surpassed most previous works in terms of overall accuracy. Conclusions: This study presents a novel, multimodal machine learning framework for objective pain recognition in postoperative patients. The results highlight the potential of integrating multiple biosignal modalities for more accurate pain assessment, with particular value in real-world clinical settings. UR - https://formative.jmir.org/2025/1/e67969 UR - http://dx.doi.org/10.2196/67969 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67969 ER - TY - JOUR AU - Rowley, AK Elizabeth AU - Mitchell, K. Patrick AU - Yang, Duck-Hye AU - Lewis, Ned AU - Dixon, E. Brian AU - Vazquez-Benitez, Gabriela AU - Fadel, F. William AU - Essien, J. Inih AU - Naleway, L. Allison AU - Stenehjem, Edward AU - Ong, C. Toan AU - Gaglani, Manjusha AU - Natarajan, Karthik AU - Embi, Peter AU - Wiegand, E. Ryan AU - Link-Gelles, Ruth AU - Tenforde, W. Mark AU - Fireman, Bruce PY - 2025/1/27 TI - Methods to Adjust for Confounding in Test-Negative Design COVID-19 Effectiveness Studies: Simulation Study JO - JMIR Form Res SP - e58981 VL - 9 KW - disease risk score KW - propensity score KW - vaccine effectiveness KW - COVID-19 KW - simulation study KW - usefulness KW - comorbidity KW - assessment N2 - Background: Real-world COVID-19 vaccine effectiveness (VE) studies are investigating exposures of increasing complexity accounting for time since vaccination. These studies require methods that adjust for the confounding that arises when morbidities and demographics are associated with vaccination and the risk of outcome events. Methods based on propensity scores (PS) are well-suited to this when the exposure is dichotomous, but present challenges when the exposure is multinomial. Objective: This simulation study aimed to investigate alternative methods to adjust for confounding in VE studies that have a test-negative design. Methods: Adjustment for a disease risk score (DRS) is compared with multivariable logistic regression. Both stratification on the DRS and direct covariate adjustment of the DRS are examined. Multivariable logistic regression with all the covariates and with a limited subset of key covariates is considered. The performance of VE estimators is evaluated across a multinomial vaccination exposure in simulated datasets. Results: Bias in VE estimates from multivariable models ranged from ?5.3% to 6.1% across 4 levels of vaccination. Standard errors of VE estimates were unbiased, and 95% coverage probabilities were attained in most scenarios. The lowest coverage in the multivariable scenarios was 93.7% (95% CI 92.2%-95.2%) and occurred in the multivariable model with key covariates, while the highest coverage in the multivariable scenarios was 95.3% (95% CI 94.0%-96.6%) and occurred in the multivariable model with all covariates. Bias in VE estimates from DRS-adjusted models was low, ranging from ?2.2% to 4.2%. However, the DRS-adjusted models underestimated the standard errors of VE estimates, with coverage sometimes below the 95% level. The lowest coverage in the DRS scenarios was 87.8% (95% CI 85.8%-89.8%) and occurred in the direct adjustment for the DRS model. The highest coverage in the DRS scenarios was 94.8% (95% CI 93.4%-96.2%) and occurred in the model that stratified on DRS. Although variation in the performance of VE estimates occurred across modeling strategies, variation in performance was also present across exposure groups. Conclusions: Overall, models using a DRS to adjust for confounding performed adequately but not as well as the multivariable models that adjusted for covariates individually. UR - https://formative.jmir.org/2025/1/e58981 UR - http://dx.doi.org/10.2196/58981 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58981 ER - TY - JOUR AU - Yang, Doris AU - Zhou, Doudou AU - Cai, Steven AU - Gan, Ziming AU - Pencina, Michael AU - Avillach, Paul AU - Cai, Tianxi AU - Hong, Chuan PY - 2025/1/22 TI - Robust Automated Harmonization of Heterogeneous Data Through Ensemble Machine Learning: Algorithm Development and Validation Study JO - JMIR Med Inform SP - e54133 VL - 13 KW - ensemble learning KW - semantic learning KW - distribution learning KW - variable harmonization KW - machine learning KW - cardiovascular health study KW - intracohort comparison KW - intercohort comparison KW - gold standard labels N2 - Background: Cohort studies contain rich clinical data across large and diverse patient populations and are a common source of observational data for clinical research. Because large scale cohort studies are both time and resource intensive, one alternative is to harmonize data from existing cohorts through multicohort studies. However, given differences in variable encoding, accurate variable harmonization is difficult. Objective: We propose SONAR (Semantic and Distribution-Based Harmonization) as a method for harmonizing variables across cohort studies to facilitate multicohort studies. Methods: SONAR used semantic learning from variable descriptions and distribution learning from study participant data. Our method learned an embedding vector for each variable and used pairwise cosine similarity to score the similarity between variables. This approach was built off 3 National Institutes of Health cohorts, including the Cardiovascular Health Study, the Multi-Ethnic Study of Atherosclerosis, and the Women?s Health Initiative. We also used gold standard labels to further refine the embeddings in a supervised manner. Results: The method was evaluated using manually curated gold standard labels from the 3 National Institutes of Health cohorts. We evaluated both the intracohort and intercohort variable harmonization performance. The supervised SONAR method outperformed existing benchmark methods for almost all intracohort and intercohort comparisons using area under the curve and top-k accuracy metrics. Notably, SONAR was able to significantly improve harmonization of concepts that were difficult for existing semantic methods to harmonize. Conclusions: SONAR achieves accurate variable harmonization within and between cohort studies by harnessing the complementary strengths of semantic learning and variable distribution learning. UR - https://medinform.jmir.org/2025/1/e54133 UR - http://dx.doi.org/10.2196/54133 ID - info:doi/10.2196/54133 ER - TY - JOUR AU - Zhang, Ren AU - Liu, Yi AU - Zhang, Zhiwei AU - Luo, Rui AU - Lv, Bin PY - 2025/1/20 TI - Interpretable Machine Learning Model for Predicting Postpartum Depression: Retrospective Study JO - JMIR Med Inform SP - e58649 VL - 13 KW - postpartum depression KW - machine learning KW - predictive model KW - risk factors KW - XGBoost KW - extreme gradient boosting KW - PPD N2 - Background: Postpartum depression (PPD) is a prevalent mental health issue with significant impacts on mothers and families. Exploring reliable predictors is crucial for the early and accurate prediction of PPD, which remains challenging. Objective: This study aimed to comprehensively collect variables from multiple aspects, develop and validate machine learning models to achieve precise prediction of PPD, and interpret the model to reveal clinical implications. Methods: This study recruited pregnant women who delivered at the West China Second University Hospital, Sichuan University. Various variables were collected from electronic medical record data and screened using least absolute shrinkage and selection operator penalty regression. Participants were divided into training (1358/2055, 66.1%) and validation (697/2055, 33.9%) sets by random sampling. Machine learning?based predictive models were developed in the training cohort. Models were validated in the validation cohort with receiver operating curve and decision curve analysis. Multiple model interpretation methods were implemented to explain the optimal model. Results: We recruited 2055 participants in this study. The extreme gradient boosting model was the optimal predictive model with the area under the receiver operating curve of 0.849. Shapley Additive Explanation indicated that the most influential predictors of PPD were antepartum depression, lower fetal weight, elevated thyroid-stimulating hormone, declined thyroid peroxidase antibodies, elevated serum ferritin, and older age. Conclusions: This study developed and validated a machine learning?based predictive model for PPD. Several significant risk factors and how they impact the prediction of PPD were revealed. These findings provide new insights into the early screening of individuals with high risk for PPD, emphasizing the need for comprehensive screening approaches that include both physiological and psychological factors. UR - https://medinform.jmir.org/2025/1/e58649 UR - http://dx.doi.org/10.2196/58649 ID - info:doi/10.2196/58649 ER - TY - JOUR AU - Hultman, Lisa AU - Eklund, Caroline AU - von Heideken Wågert, Petra AU - Söderlund, Anne AU - Lindén, Maria AU - Elfström, L. Magnus PY - 2025/1/20 TI - Development of an eHealth Intervention Including Self-Management for Reducing Sedentary Time in the Transition to Retirement: Participatory Design Study JO - JMIR Form Res SP - e63567 VL - 9 KW - behavior change intervention KW - adherence KW - integrated behavior change model KW - autonomous motivation KW - affective determinants N2 - Background: Having a great amount of sedentary time is common among older adults and increases with age. There is a strong need for tools to reduce sedentary time and promote adherence to reduced sedentary time, for which eHealth interventions have the potential to be useful. Interventions for reducing sedentary time in older adults have been found to be more effective when elements of self-management are included. When creating new eHealth interventions, accessibility and effectiveness can be increased by including end users as co-designers in the development process. Objective: The aim was to explore the desired features of an eHealth intervention including self-management for reducing sedentary time and promoting adherence to reduced sedentary time in older adults transitioning from working life to retirement. Further, the aim was to develop a digital prototype of such an eHealth intervention. Methods: The study used the participatory design approach to include end users, researchers, and a web designer as equal partners. Three workshops were conducted with 6 older adults transitioning to retirement, 2 researchers, and 1 web designer. Thematic analysis was used to analyze the data from the workshops. Results: Participants expressed a desire for an easy-to-use eHealth intervention, which could be accessed from mobile phones, tablets, and computers, and could be individualized to the user. The most important features for reducing sedentary time were those involving finding joyful activities, setting goals, and getting information regarding reduced sedentary time. Participants expressed that the eHealth intervention would need to first provide the user with knowledge regarding sedentary time, then offer features for measuring sedentary time and for setting goals, and lastly provide support in finding joyful activities to perform in order to avoid being sedentary. According to the participants, an eHealth intervention including self-management for reducing sedentary time in older adults in the transition to retirement should be concise, accessible, and enjoyable. A digital prototype of such an eHealth intervention was developed. Conclusions: The developed eHealth intervention including self-management for reducing sedentary time in older adults transitioning to retirement is intended to facilitate behavior change by encouraging the user to participate in autonomously motivated activities. It uses several behavior change techniques, such as goal setting and action planning through mental contrasting and implementation intention, as well as shaping knowledge. Its active components for reducing sedentary time can be explained using the integrated behavior change model. Further research is needed to evaluate the feasibility and effectiveness of the eHealth intervention. UR - https://formative.jmir.org/2025/1/e63567 UR - http://dx.doi.org/10.2196/63567 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/63567 ER - TY - JOUR AU - Bondaronek, Paulina AU - Li, Jingfeng AU - Potts, W. Henry W. PY - 2025/1/20 TI - Public Understanding and Expectations of Digital Health Evidence Generation: Focus Group Study JO - JMIR Form Res SP - e56523 VL - 9 KW - mobile apps KW - digital health KW - public expectations KW - evidence of effectiveness KW - health risk perception KW - effectiveness KW - health risk KW - health app KW - public health KW - well-being KW - public trust KW - diagnostic tools KW - safety KW - mobile phone N2 - Background: The rapid proliferation of health apps has not been matched by a comparable growth in scientific evaluations of their effectiveness, particularly for apps available to the public. This gap has prompted ongoing debate about the types of evidence necessary to validate health apps, especially as the perceived risk level varies from wellness tools to diagnostic aids. The perspectives of the general public, who are direct stakeholders, are notably underrepresented in discussions on digital health evidence generation. Objective: This study aimed to explore public understanding and expectations regarding the evidence required to demonstrate health apps? effectiveness, including at varying levels of health risk. Methods: A total of 4 focus group discussions were held with UK residents aged 18 years and older, recruited through targeted advertisements to ensure demographic diversity. Participants discussed their views on evidence requirements for 5 hypothetical health apps, ranging from low-risk wellness apps to high-risk diagnostic tools. Focus groups were moderated using a structured guide, and data were analyzed using reflexive thematic analysis to extract common themes. Results: A total of 5 key themes were established: personal needs, app functionality, social approval, expectations of testing, and authority. Participants relied on personal experiences and social endorsements when judging the effectiveness of low-risk digital health interventions, while making minimal reference to traditional scientific evidence. However, as the perceived risk of an app increased, there was a noticeable shift toward preferring evidence from authoritative sources, such as government or National Health Service endorsements. Conclusions: The public have a preference for evidence that resonates on a personal level, but also show a heightened demand for authoritative guidance as the potential risk of digital health interventions increases. These perspectives should guide developers, regulators, and policy makers as they balance how to achieve innovation, safety, and public trust in the digital health landscape. Engaging the public in evidence-generation processes and ensuring transparency in app functionality and testing can bridge the gap between public expectations and regulatory standards, fostering trust in digital health technologies. UR - https://formative.jmir.org/2025/1/e56523 UR - http://dx.doi.org/10.2196/56523 ID - info:doi/10.2196/56523 ER - TY - JOUR AU - Schindler, Lea AU - Beelich, Hilke AU - Röll, Selina AU - Katsari, Elpiniki AU - Stracke, Sylvia AU - Waltemath, Dagmar PY - 2025/1/20 TI - Applicability of Retrospective and Prospective Gender Scores for Clinical and Health Data: Protocol for a Scoping Review JO - JMIR Res Protoc SP - e57669 VL - 14 KW - gender score KW - gender medicine KW - medical informatics KW - data integration KW - gender health gap N2 - Background: Gender is known to have a strong influence on human health and disease. Despite its relevance to treatment and outcome, gender is insufficiently considered in current health research. One hindering factor is the poor representation of gender information in clinical and health (meta) data. Objective: We aim to conduct a scoping review of the literature describing gender scores. The review will provide insights into the current application of gender scores in clinical and health settings. The protocol describes how relevant literature will be identified and how gender scores will be evaluated concerning applicability and usability in scientific investigations. Methods: Our scoping review follows the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. A title and abstract screening was conducted on PubMed, followed by a full-text screening. The inclusion and exclusion criteria were discussed by a team of 5 domain experts, and a data-charting form was developed. The charted data will be categorized, summarized, and analyzed based on the research questions during the scoping review. Results: We will report our research results according to the PRISMA-ScR guidelines. The literature retrieval was carried out on June 13, 2024, and resulted in 1202 matches. As of July 2024, the scoping review is in the data extraction phase and we expect to complete and publish the results in the first quarter of 2025. Conclusions: The scoping review lays the foundation for a retrospective gender assessment by identifying scores that can be applied to existing large-scale datasets. Moreover, it will help to formulate recommendations for standardized gender scores in future investigations. International Registered Report Identifier (IRRID): DERR1-10.2196/57669 UR - https://www.researchprotocols.org/2025/1/e57669 UR - http://dx.doi.org/10.2196/57669 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57669 ER - TY - JOUR AU - Song, Shanshan AU - Ashton, Micaela AU - Yoo, Hahn Rebecca AU - Lkhagvajav, Zoljargal AU - Wright, Robert AU - Mathews, H. Debra J. AU - Taylor, Overby Casey PY - 2025/1/20 TI - Participant Contributions to Person-Generated Health Data Research Using Mobile Devices: Scoping Review JO - J Med Internet Res SP - e51955 VL - 27 KW - scoping review KW - person-generated health data KW - PGHD KW - mHealth KW - mobile device KW - smartphone KW - mobile phone KW - wearable KW - fitness tracker KW - smartwatch KW - BYOD KW - crowdsourcing KW - reporting deficiency N2 - Background: Mobile devices offer an emerging opportunity for research participants to contribute person-generated health data (PGHD). There is little guidance, however, on how to best report findings from studies leveraging those data. Thus, there is a need to characterize current reporting practices so as to better understand the potential implications for producing reproducible results. Objective: The primary objective of this scoping review was to characterize publications? reporting practices for research that collects PGHD using mobile devices. Methods: We comprehensively searched PubMed and screened the results. Qualifying publications were classified according to 6 dimensions?1 covering key bibliographic details (for all articles) and 5 covering reporting criteria considered necessary for reproducible and responsible research (ie, ?participant,? ?data,? ?device,? ?study,? and ?ethics,? for original research). For each of the 5 reporting dimensions, we also assessed reporting completeness. Results: Out of 3602 publications screened, 100 were included in this review. We observed a rapid increase in all publications from 2016 to 2021, with the largest contribution from US authors, with 1 exception, review articles. Few original research publications used crowdsourcing platforms (7%, 3/45). Among the original research publications that reported device ownership, most (75%, 21/28) reported using participant-owned devices for data collection (ie, a Bring-Your-Own-Device [BYOD] strategy). A significant deficiency in reporting completeness was observed for the ?data? and ?ethics? dimensions (5 reporting factors were missing in over half of the research publications). Reporting completeness for data ownership and participants? access to data after contribution worsened over time. Conclusions: Our work depicts the reporting practices in publications about research involving PGHD from mobile devices. We found that very few papers reported crowdsourcing platforms for data collection. BYOD strategies are increasingly popular; this creates an opportunity for improved mechanisms to transfer data from device owners to researchers on crowdsourcing platforms. Given substantial reporting deficiencies, we recommend reaching a consensus on best practices for research collecting PGHD from mobile devices. Drawing from the 5 reporting dimensions in this scoping review, we share our recommendations and justifications for 9 items. These items require improved reporting to enhance data representativeness and quality and empower participants. UR - https://www.jmir.org/2025/1/e51955 UR - http://dx.doi.org/10.2196/51955 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/51955 ER - TY - JOUR AU - Ko, Eunjung AU - Gao, Ye AU - Wang, Peng AU - Wijayasingha, Lahiru AU - Wright, D. Kathy AU - Gordon, C. Kristina AU - Wang, Hongning AU - Stankovic, A. John AU - Rose, M. Karen PY - 2025/1/17 TI - Recruitment Challenges and Strategies in a Technology-Based Intervention for Dementia Caregivers: Descriptive Study JO - JMIR Form Res SP - e59291 VL - 9 KW - recruitment challenges and strategies KW - technology-based intervention KW - dementia caregivers KW - dementia KW - mobile phone KW - Alzheimer disease KW - smart health N2 - Background: Researchers have encountered challenges in recruiting unpaid caregivers of people living with Alzheimer disease and related dementias for intervention studies. However, little is known about the reasons for nonparticipation in in-home smart health interventions in community-based settings. Objective: This study aimed to (1) assess recruitment rates in a smart health technology intervention for caregivers of people living with Alzheimer disease and related dementias and reasons for nonparticipation among them and (2) discuss lessons learned from recruitment challenges and strategies to improve recruitment. Methods: The smart health intervention was a 4-month, single-arm trial designed to evaluate an in-home, technology-based intervention that monitors stressful moments for caregiving dyads through acoustic signals and to provide the caregivers with real-time stress management strategies. The recruitment involved two main methods: on-site engagement by a recruiter from a memory clinic and social media advertising. Caregivers were screened for eligibility by phone between January 2021 and September 2023. The recruitment rates, reasons for nonparticipation, and participant demographics were analyzed using descriptive statistics. Results: Of 201 caregivers contacted, 11 were enrolled in this study. Eighty-two caregivers did not return the screening call, and others did not participate due to privacy concerns (n=30), lack of interest (n=29), and burdensome study procedures (n=26). Our recruitment strategies included addressing privacy concerns, visualizing collected data through a dashboard, boosting social media presence, increasing the recruitment budget, updating advertisements, and preparing and deploying additional study devices. Conclusions: This study highlighted barriers to participation in the smart health intervention. Despite several recruitment strategies, enrollment rates remained below expectations. These findings underscore the need for future research to explore alternative methods for increasing the recruitment of informal dementia caregivers in technology-based intervention studies. Trial Registration: ClinicalTrials.gov NCT04536701; https://clinicaltrials.gov/study/NCT04536701 International Registered Report Identifier (IRRID): RR2-10.1111/jan.14714 UR - https://formative.jmir.org/2025/1/e59291 UR - http://dx.doi.org/10.2196/59291 ID - info:doi/10.2196/59291 ER - TY - JOUR AU - Åvik Persson, Helene AU - Castor, Charlotte AU - Andersson, Nilla AU - Hylén, Mia PY - 2025/1/16 TI - Swedish Version of the System Usability Scale: Translation, Adaption, and Psychometric Evaluation JO - JMIR Hum Factors SP - e64210 VL - 12 KW - application KW - Swedish KW - System Usability Scale KW - usability KW - validation N2 - Background: The Swedish health care system is undergoing a transformation. eHealth technologies are increasingly being used. The System Usability Scale is a widely used tool, offering a standardized and reliable measure for assessing the usability of digital health solutions. However, despite the existence of several translations of the System Usability Scale into Swedish, none have undergone psychometric validation. This highlights the urgent need for a validated and standardized Swedish version of the System Usability Scale to ensure accurate and reliable usability evaluations. Objective: The aim of the study was to translate and psychometrically evaluate a Swedish version of the System Usability Scale. Methods: The study utilized a 2-phase design. The first phase translated the System Usability Scale into Swedish and the second phase tested the scale?s psychometric properties. A total of 62 participants generated a total of 82 measurements. Descriptive statistics were used to visualize participants? characteristics. The psychometric evaluation consisted of data quality, scaling assumptions, and acceptability. Construct validity was evaluated by convergent validity, and reliability was evaluated by internal consistency. Results: The Swedish version of the System Usability Scale demonstrated high conformity with the original version. The scale showed high internal consistency with a Cronbach ? of .852 and corrected item-total correlations ranging from 0.454 to 0.731. The construct validity was supported by a significant positive correlation between the System Usability Scale and domain 5 of the eHealth Literacy Questionnaire (P=.001). Conclusions: The Swedish version of the System Usability Scale demonstrated satisfactory psychometric properties. It can be recommended for use in a Swedish context. The positive correlation with domain 5 of the eHealth Literacy Questionnaire further supports the construct validity of the Swedish version of the System Usability Scale, affirming its suitability for evaluating digital health solutions. Additional tests of the Swedish version of the System Usability Scale, for example, in the evaluation of more complex eHealth technology, would further validate the scale. Trial Registration: ClinicalTrials.gov NCT04150120; https://clinicaltrials.gov/study/NCT04150120 UR - https://humanfactors.jmir.org/2025/1/e64210 UR - http://dx.doi.org/10.2196/64210 ID - info:doi/10.2196/64210 ER - TY - JOUR AU - Mumtaz, Shahzad AU - McMinn, Megan AU - Cole, Christian AU - Gao, Chuang AU - Hall, Christopher AU - Guignard-Duff, Magalie AU - Huang, Huayi AU - McAllister, A. David AU - Morales, R. Daniel AU - Jefferson, Emily AU - Guthrie, Bruce PY - 2025/1/16 TI - A Digital Tool for Clinical Evidence?Driven Guideline Development by Studying Properties of Trial Eligible and Ineligible Populations: Development and Usability Study JO - J Med Internet Res SP - e52385 VL - 27 KW - multimorbidity KW - clinical practice guideline KW - gout KW - Trusted Research Environment KW - National Institute for Health and Care Excellence KW - Scottish Intercollegiate Guidelines Network KW - clinical practice KW - development KW - efficacy KW - validity KW - epidemiological data KW - epidemiology KW - epidemiological KW - digital tool KW - tool KW - age KW - gender KW - ethnicity KW - mortality KW - feedback KW - availability N2 - Background: Clinical guideline development preferentially relies on evidence from randomized controlled trials (RCTs). RCTs are gold-standard methods to evaluate the efficacy of treatments with the highest internal validity but limited external validity, in the sense that their findings may not always be applicable to or generalizable to clinical populations or population characteristics. The external validity of RCTs for the clinical population is constrained by the lack of tailored epidemiological data analysis designed for this purpose due to data governance, consistency of disease or condition definitions, and reduplicated effort in analysis code. Objective: This study aims to develop a digital tool that characterizes the overall population and differences between clinical trial eligible and ineligible populations from the clinical populations of a disease or condition regarding demography (eg, age, gender, ethnicity), comorbidity, coprescription, hospitalization, and mortality. Currently, the process is complex, onerous, and time-consuming, whereas a real-time tool may be used to rapidly inform a guideline developer?s judgment about the applicability of evidence. Methods: The National Institute for Health and Care Excellence?particularly the gout guideline development group?and the Scottish Intercollegiate Guidelines Network guideline developers were consulted to gather their requirements and evidential data needs when developing guidelines. An R Shiny (R Foundation for Statistical Computing) tool was designed and developed using electronic primary health care data linked with hospitalization and mortality data built upon an optimized data architecture. Disclosure control mechanisms were built into the tool to ensure data confidentiality. The tool was deployed within a Trusted Research Environment, allowing only trusted preapproved researchers to conduct analysis. Results: The tool supports 128 chronic health conditions as index conditions and 161 conditions as comorbidities (33 in addition to the 128 index conditions). It enables 2 types of analyses via the graphic interface: overall population and stratified by user-defined eligibility criteria. The analyses produce an overview of statistical tables (eg, age, gender) of the index condition population and, within the overview groupings, produce details on, for example, electronic frailty index, comorbidities, and coprescriptions. The disclosure control mechanism is integral to the tool, limiting tabular counts to meet local governance needs. An exemplary result for gout as an index condition is presented to demonstrate the tool?s functionality. Guideline developers from the National Institute for Health and Care Excellence and the Scottish Intercollegiate Guidelines Network provided positive feedback on the tool. Conclusions: The tool is a proof-of-concept, and the user feedback has demonstrated that this is a step toward computer-interpretable guideline development. Using the digital tool can potentially improve evidence-driven guideline development through the availability of real-world data in real time. UR - https://www.jmir.org/2025/1/e52385 UR - http://dx.doi.org/10.2196/52385 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/52385 ER - TY - JOUR AU - Klein, Dave AU - Montgomery, Aisha AU - Begale, Mark AU - Sutherland, Scott AU - Sawyer, Sherilyn AU - McCauley, L. Jacob AU - Husbands, Letheshia AU - Joshi, Deepti AU - Ashbeck, Alan AU - Palmer, Marcy AU - Jain, Praduman PY - 2025/1/15 TI - Building a Digital Health Research Platform to Enable Recruitment, Enrollment, Data Collection, and Follow-Up for a Highly Diverse Longitudinal US Cohort of 1 Million People in the All of Us Research Program: Design and Implementation Study JO - J Med Internet Res SP - e60189 VL - 27 KW - longitudinal studies KW - cohort studies KW - health disparities KW - minority populations KW - vulnerable populations KW - precision medicine KW - biomedical research KW - decentralization KW - digital health technology KW - database management system N2 - Background: Longitudinal cohort studies have traditionally relied on clinic-based recruitment models, which limit cohort diversity and the generalizability of research outcomes. Digital research platforms can be used to increase participant access, improve study engagement, streamline data collection, and increase data quality; however, the efficacy and sustainability of digitally enabled studies rely heavily on the design, implementation, and management of the digital platform being used. Objective: We sought to design and build a secure, privacy-preserving, validated, participant-centric digital health research platform (DHRP) to recruit and enroll participants, collect multimodal data, and engage participants from diverse backgrounds in the National Institutes of Health?s (NIH) All of Us Research Program (AOU). AOU is an ongoing national, multiyear study aimed to build a research cohort of 1 million participants that reflects the diversity of the United States, including minority, health-disparate, and other populations underrepresented in biomedical research (UBR). Methods: We collaborated with community members, health care provider organizations (HPOs), and NIH leadership to design, build, and validate a secure, feature-rich digital platform to facilitate multisite, hybrid, and remote study participation and multimodal data collection in AOU. Participants were recruited by in-person, print, and online digital campaigns. Participants securely accessed the DHRP via web and mobile apps, either independently or with research staff support. The participant-facing tool facilitated electronic informed consent (eConsent), multisource data collection (eg, surveys, genomic results, wearables, and electronic health records [EHRs]), and ongoing participant engagement. We also built tools for research staff to conduct remote participant support, study workflow management, participant tracking, data analytics, data harmonization, and data management. Results: We built a secure, participant-centric DHRP with engaging functionality used to recruit, engage, and collect data from 705,719 diverse participants throughout the United States. As of April 2024, 87% (n=613,976) of the participants enrolled via the platform were from UBR groups, including racial and ethnic minorities (n=282,429, 46%), rural dwelling individuals (n=49,118, 8%), those over the age of 65 years (n=190,333, 31%), and individuals with low socioeconomic status (n=122,795, 20%). Conclusions: We built a participant-centric digital platform with tools to enable engagement with individuals from different racial, ethnic, and socioeconomic backgrounds and other UBR groups. This DHRP demonstrated successful use among diverse participants. These findings could be used as best practices for the effective use of digital platforms to build and sustain cohorts of various study designs and increase engagement with diverse populations in health research. UR - https://www.jmir.org/2025/1/e60189 UR - http://dx.doi.org/10.2196/60189 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60189 ER - TY - JOUR AU - Kim, Hyung Do AU - Jeong, Won Joo AU - Kang, Dayoung AU - Ahn, Taekyung AU - Hong, Yeonjung AU - Im, Younggon AU - Kim, Jaewon AU - Kim, Jung Min AU - Jang, Dae-Hyun PY - 2025/1/14 TI - Usefulness of Automatic Speech Recognition Assessment of Children With Speech Sound Disorders: Validation Study JO - J Med Internet Res SP - e60520 VL - 27 KW - speech sound disorder KW - speech recognition software KW - speech articulation tests KW - speech-language pathology KW - child N2 - Background: Speech sound disorders (SSDs) are common communication challenges in children, typically assessed by speech-language pathologists (SLPs) using standardized tools. However, traditional evaluation methods are time-intensive and prone to variability, raising concerns about reliability. Objective: This study aimed to compare the evaluation outcomes of SLPs and an automatic speech recognition (ASR) model using two standardized SSD assessments in South Korea, evaluating the ASR model?s performance. Methods: A fine-tuned wav2vec 2.0 XLS-R model, pretrained on 436,000 hours of adult voice data spanning 128 languages, was used. The model was further trained on 93.6 minutes of children?s voices with articulation errors to improve error detection. Participants included children referred to the Department of Rehabilitation Medicine at a general hospital in Incheon, South Korea, from August 19, 2022, to June 14, 2023. Two standardized assessments?the Assessment of Phonology and Articulation for Children (APAC) and the Urimal Test of Articulation and Phonology (U-TAP)?were used, with ASR transcriptions compared to SLP transcriptions. Results: This study included 30 children aged 3-7 years who were suspected of having SSDs. The phoneme error rates for the APAC and U-TAP were 8.42% (457/5430) and 8.91% (402/4514), respectively, indicating discrepancies between the ASR model and SLP transcriptions across all phonemes. Consonant error rates were 10.58% (327/3090) and 11.86% (331/2790) for the APAC and U-TAP, respectively. On average, there were 2.60 (SD 1.54) and 3.07 (SD 1.39) discrepancies per child for correctly produced phonemes, and 7.87 (SD 3.66) and 7.57 (SD 4.85) discrepancies per child for incorrectly produced phonemes, based on the APAC and U-TAP, respectively. The correlation between SLPs and the ASR model in terms of the percentage of consonants correct was excellent, with an intraclass correlation coefficient of 0.984 (95% CI 0.953-0.994) and 0.978 (95% CI 0.941-0.990) for the APAC and UTAP, respectively. The z scores between SLPs and ASR showed more pronounced differences with the APAC than the U-TAP, with 8 individuals showing discrepancies in the APAC compared to 2 in the U-TAP. Conclusions: The results demonstrate the potential of the ASR model in assessing children with SSDs. However, its performance varied based on phoneme or word characteristics, highlighting areas for refinement. Future research should include more diverse speech samples, clinical settings, and speech data to strengthen the model?s refinement and ensure broader clinical applicability. UR - https://www.jmir.org/2025/1/e60520 UR - http://dx.doi.org/10.2196/60520 UR - http://www.ncbi.nlm.nih.gov/pubmed/39576242 ID - info:doi/10.2196/60520 ER - TY - JOUR AU - Dushyanthen, Sathana AU - Zamri, Izzati Nadia AU - Chapman, Wendy AU - Capurro, Daniel AU - Lyons, Kayley PY - 2025/1/14 TI - Evaluation of an Interdisciplinary Educational Program to Foster Learning Health Systems: Education Evaluation JO - JMIR Med Educ SP - e54152 VL - 11 KW - continuing professional development KW - learning health system KW - flipped classroom KW - digital health informatics KW - data science KW - health professions education KW - interdisciplinary education KW - foster KW - foster learning KW - health data KW - design KW - innovative KW - innovative solution KW - health care workforce KW - Australia KW - real time KW - teaching model N2 - Background: Learning health systems (LHS) have the potential to use health data in real time through rapid and continuous cycles of data interrogation, implementing insights to practice, feedback, and practice change. However, there is a lack of an appropriately skilled interprofessional informatics workforce that can leverage knowledge to design innovative solutions. Therefore, there is a need to develop tailored professional development training in digital health, to foster skilled interprofessional learning communities in the health care workforce in Australia. Objective: This study aimed to explore participants? experiences and perspectives of participating in an interprofessional education program over 13 weeks. The evaluation also aimed to assess the benefits, barriers, and opportunities for improvements and identify future applications of the course materials. Methods: We developed a wholly online short course open to interdisciplinary professionals working in digital health in the health care sector. In a flipped classroom model, participants (n=400) undertook 2 hours of preclass learning online and then attended 2.5 hours of live synchronous learning in interactive weekly Zoom workshops for 13 weeks. Throughout the course, they collaborated in small, simulated learning communities (n=5 to 8), engaging in various activities and problem-solving exercises, contributing their unique perspectives and diverse expertise. The course covered a number of topics including background on LHS, establishing learning communities, the design thinking process, data preparation and machine learning analysis, process modeling, clinical decision support, remote patient monitoring, evaluation, implementation, and digital transformation. To evaluate the purpose of the program, we undertook a mixed methods evaluation consisting of pre- and postsurveys rating scales for usefulness, engagement, value, and applicability for various aspects of the course. Participants also completed identical measures of self-efficacy before and after (n=200), with scales mapped to specific skills and tasks that should have been achievable following each of the topics covered. Further, they undertook voluntary weekly surveys to provide feedback on which aspects to continue and recommendations for improvements, via free-text responses. Results: From the evaluation, it was evident that participants found the teaching model engaging, useful, valuable, and applicable to their work. In the self-efficacy component, we observed a significant increase (P<.001) in perceived confidence for all topics, when comparing pre- and postcourse ratings. Overall, it was evident that the program gave participants a framework to organize their knowledge and a common understanding and shared language to converse with other disciplines, changed the way they perceived their role and the possibilities of data and technologies, and provided a toolkit through the LHS framework that they could apply in their workplaces. Conclusions: We present a program to educate the health workforce on integrating the LHS model into standard practice. Interprofessional collaborative learning was a major component of the value of the program. This evaluation shed light on the multifaceted challenges and expectations of individuals embarking on a digital health program. Understanding the barriers and facilitators of the audience is crucial for creating an inclusive and supportive learning environment. Addressing these challenges will not only enhance participant engagement but also contribute to the overall success of the program and, by extension, the broader integration of digital health solutions into health care practice and, ultimately, patient outcomes. UR - https://mededu.jmir.org/2025/1/e54152 UR - http://dx.doi.org/10.2196/54152 ID - info:doi/10.2196/54152 ER - TY - JOUR AU - Antolin Muñiz, Marley AU - McMahan, M. Vanessa AU - Luna Marti, Xochitl AU - Brennan, Sarah AU - Tavasieff, Sophia AU - Rodda, N. Luke AU - Knoll, James AU - Coffin, O. Phillip PY - 2025/1/13 TI - Identification of Behavioral, Clinical, and Psychological Antecedents of Acute Stimulant Poisoning: Development and Implementation of a Mixed Methods Psychological Autopsy Study JO - JMIR Form Res SP - e64873 VL - 9 KW - psychological autopsy KW - acute stimulant poisoning KW - overdose KW - cocaine KW - methamphetamine KW - fentanyl N2 - Background: Despite increasing fatal stimulant poisoning in the United States, little is understood about the mechanism of death. The psychological autopsy (PA) has long been used to distinguish the manner of death in equivocal cases, including opioid overdose, but has not been used to explicitly explore stimulant mortality. Objective: We aimed to develop and implement a large PA study to identify antecedents of fatal stimulant poisoning, seeking to maximize data gathering and ethical interactions during the collateral interviews. Methods: We ascertained death records from the California Electronic Death Reporting System (CA-EDRS) and the San Francisco Office of the County Medical Examiner (OCME) from June 2022 through December 2023. We selected deaths determined to be due to acute poisoning from cocaine or methamphetamine, which occurred 3?12 months prior and were not attributed to suicide or homicide. We identified 31 stimulant-fentanyl and 70 stimulant-no-opioid decedents. We sought 2 informants for each decedent, who were able to describe the decedent across their life course. Informants were at least 18 years of age, communicated with the decedent within the year before death, and were aware that the decedent had been using substances during that year. Upon completion of at least one informant interview conducted by staff with bachelor?s or master?s degrees, we collected OCME, medical record, and substance use disorder treatment data for the decedent. Planned analyses include least absolute shrinkage and selection operator regressions of quantitative data and thematic analyses of qualitative data. Results: We identified and interviewed at least one informant (N=141) for each decedent (N=101). Based on feedback during recruitment, we adapted language to improve rapport, including changing the term ?accidental death? to ?premature death,? offering condolences, and providing content warnings. As expected, family members were able to provide more data about the decedent?s childhood and adolescence, and nonfamily informants provided more data regarding events proximal to death. We found that the interviews were stressful for both the interviewee and interviewer, especially when participants thought the study was intrusive or experienced significant grief during the interviews. Conclusions: In developing and implementing PA research on fatal stimulant poisoning, we noted the importance of recruitment language regarding cause of death and condolences with collateral informants. Compassion and respect were critical to facilitate the interview process and maintain an ethical framework. We discuss several barriers to success and lessons learned while conducting PA interviews, as well as recommendations for future PA studies. UR - https://formative.jmir.org/2025/1/e64873 UR - http://dx.doi.org/10.2196/64873 ID - info:doi/10.2196/64873 ER - TY - JOUR AU - Kaewboonlert, Naritsaret AU - Poontananggul, Jiraphon AU - Pongsuwan, Natthipong AU - Bhakdisongkhram, Gun PY - 2025/1/13 TI - Factors Associated With the Accuracy of Large Language Models in Basic Medical Science Examinations: Cross-Sectional Study JO - JMIR Med Educ SP - e58898 VL - 11 KW - accuracy KW - performance KW - artificial intelligence KW - AI KW - ChatGPT KW - large language model KW - LLM KW - difficulty index KW - basic medical science examination KW - cross-sectional study KW - medical education KW - datasets KW - assessment KW - medical science KW - tool KW - Google N2 - Background: Artificial intelligence (AI) has become widely applied across many fields, including medical education. Content validation and its answers are based on training datasets and the optimization of each model. The accuracy of large language model (LLMs) in basic medical examinations and factors related to their accuracy have also been explored. Objective: We evaluated factors associated with the accuracy of LLMs (GPT-3.5, GPT-4, Google Bard, and Microsoft Bing) in answering multiple-choice questions from basic medical science examinations. Methods: We used questions that were closely aligned with the content and topic distribution of Thailand?s Step 1 National Medical Licensing Examination. Variables such as the difficulty index, discrimination index, and question characteristics were collected. These questions were then simultaneously input into ChatGPT (with GPT-3.5 and GPT-4), Microsoft Bing, and Google Bard, and their responses were recorded. The accuracy of these LLMs and the associated factors were analyzed using multivariable logistic regression. This analysis aimed to assess the effect of various factors on model accuracy, with results reported as odds ratios (ORs). Results: The study revealed that GPT-4 was the top-performing model, with an overall accuracy of 89.07% (95% CI 84.76%?92.41%), significantly outperforming the others (P<.001). Microsoft Bing followed with an accuracy of 83.69% (95% CI 78.85%?87.80%), GPT-3.5 at 67.02% (95% CI 61.20%?72.48%), and Google Bard at 63.83% (95% CI 57.92%?69.44%). The multivariable logistic regression analysis showed a correlation between question difficulty and model performance, with GPT-4 demonstrating the strongest association. Interestingly, no significant correlation was found between model accuracy and question length, negative wording, clinical scenarios, or the discrimination index for most models, except for Google Bard, which showed varying correlations. Conclusions: The GPT-4 and Microsoft Bing models demonstrated equal and superior accuracy compared to GPT-3.5 and Google Bard in the domain of basic medical science. The accuracy of these models was significantly influenced by the item?s difficulty index, indicating that the LLMs are more accurate when answering easier questions. This suggests that the more accurate models, such as GPT-4 and Bing, can be valuable tools for understanding and learning basic medical science concepts. UR - https://mededu.jmir.org/2025/1/e58898 UR - http://dx.doi.org/10.2196/58898 ID - info:doi/10.2196/58898 ER - TY - JOUR AU - Kaur, Harleen AU - Tripathi, Stuti AU - Chalga, Singh Manjeet AU - Benara, K. Sudhir AU - Dhiman, Amit AU - Gupta, Shefali AU - Nair, Saritha AU - Menon, Geetha AU - Gulati, K. B. AU - Sharma, Sandeep AU - Sharma, Saurabh PY - 2025/1/10 TI - Unified Mobile App for Streamlining Verbal Autopsy and Cause of Death Assignment in India: Design and Development Study JO - JMIR Form Res SP - e59937 VL - 9 KW - verbal autopsy KW - cause of death KW - mortality KW - mHealth KW - public health KW - India KW - mobile health N2 - Background: Verbal autopsy (VA) has been a crucial tool in ascertaining population-level cause of death (COD) estimates, specifically in countries where medical certification of COD is relatively limited. The World Health Organization has released an updated instrument (Verbal Autopsy Instrument 2022) that supports electronic data collection methods along with analytical software for assigning COD. This questionnaire encompasses the primary signs and symptoms associated with prevalent diseases across all age groups. Traditional methods have primarily involved paper-based questionnaires and physician-coded approaches for COD assignment, which is time-consuming and resource-intensive. Although computer-coded algorithms have advanced the COD assignment process, data collection in densely populated countries like India remains a logistical challenge. Objective: This study aimed to develop an Android-based mobile app specifically tailored for streamlining VA data collection by leveraging the existing Indian public health workforce. The app has been designed to integrate real-time data collection by frontline health workers and seamless data transmission and digital reporting of COD by physicians. This process aimed to enhance the efficiency and accuracy of COD assignment through VA. Methods: The app was developed using Android Studio, the primary integrated development environment for developing Android apps using Java. The front-end interface was developed using XML, while SQLite and MySQL were employed to streamline complete data storage on the local and server databases, respectively. The communication between the app and the server was facilitated through a PHP application programming interface to synchronize data from the local to the server database. The complete prototype was specifically built to reduce manual intervention and automate VA data collection. Results: The app was developed to align with the current Indian public health system for district-level COD estimation. By leveraging this mobile app, the average duration required for VA data collection to ascertainment of COD, which typically ranges from 6 to 8 months, is expected to decrease by approximately 80%, reducing it to about 1?2 months. Based on annual caseload projections, the smallest administrative public health unit, health and wellness centers, is anticipated to handle 35?40 VA cases annually, while medical officers at primary health centers are projected to manage 150?200 physician-certified VAs each year. The app?s data collection and transmission efficiency were further improved based on feedback from user and subject area experts. Conclusions: The development of a unified mobile app could streamline the VA process, enabling the generation of accurate national and subnational COD estimates. This mobile app can be further piloted and scaled to different regions to integrate the automated VA model into the existing public health system for generating comprehensive mortality statistics in India. UR - https://formative.jmir.org/2025/1/e59937 UR - http://dx.doi.org/10.2196/59937 ID - info:doi/10.2196/59937 ER - TY - JOUR AU - Tanaka, Hiroki AU - Miyamoto, Kana AU - Hamet Bagnou, Jennifer AU - Prigent, Elise AU - Clavel, Céline AU - Martin, Jean-Claude AU - Nakamura, Satoshi PY - 2025/1/10 TI - Analysis of Social Performance and Action Units During Social Skills Training: Focus Group Study of Adults With Autism Spectrum Disorder and Schizophrenia JO - JMIR Form Res SP - e59261 VL - 9 KW - social performance rating scale KW - social skills training KW - autism spectrum disorder KW - schizophrenia KW - facial expressions KW - social KW - autism KW - training KW - communication KW - trainers KW - tool KW - neurological N2 - Background: Social communication is a crucial factor influencing human social life. Quantifying the degree of difficulty faced in social communication is necessary for understanding developmental and neurological disorders and for creating systems used in automatic symptom screening and assistive methods such as social skills training (SST). SST by a human trainer is a well-established method. Previous SST used a modified roleplay test to evaluate human social communication skills. However, there are no widely accepted evaluation criteria or social behavioral markers to quantify social performance during SST. Objective: This paper has 2 objectives. First, we propose applying the Social Performance Rating Scale (SPRS) to SST data to measure social communication skills. We constructed a Japanese version of the SPRS already developed in English and French. Second, we attempt to quantify action units during SST for people with autism spectrum disorder (ASD) or schizophrenia. Methods: We used videos of interactions between trainers, adults with ASD (n=16) or schizophrenia (n=15), and control participants (n=19) during SST sessions. Two raters applied the proposed scale to annotate the collected data. We investigated the differences between roleplay tasks and participant groups (ASD, schizophrenia, and control). Furthermore, the intensity of action units on the OpenFace toolkit was measured in terms of mean and SD during SST roleplaying. Results: We found significantly greater gaze scores in adults with ASD than in adults with schizophrenia. Differences were also found between the ratings of different tasks in the adults with schizophrenia and the control participants. Action units numbered AU06 and AU12 were significantly deactivated in people with schizophrenia compared with the control group. Moreover, AU02 was significantly activated in people with ASD compared with the other groups. Conclusions: The results suggest that the SPRS can be a useful tool for assessing social communication skills in different cultures and different pathologies when used with the modified roleplay test. Furthermore, facial expressions could provide effective social and behavioral markers to characterize psychometric properties. Possible future directions include using the SPRS for assessing social behavior during interaction with a digital agent. UR - https://formative.jmir.org/2025/1/e59261 UR - http://dx.doi.org/10.2196/59261 ID - info:doi/10.2196/59261 ER - TY - JOUR AU - Kaufman, Jaycee AU - Jeon, Jouhyun AU - Oreskovic, Jessica AU - Thommandram, Anirudh AU - Fossat, Yan PY - 2025/1/9 TI - Longitudinal Changes in Pitch-Related Acoustic Characteristics of the Voice Throughout the Menstrual Cycle: Observational Study JO - JMIR Form Res SP - e65448 VL - 9 KW - menstrual cycle KW - women's health KW - voice KW - acoustic analysis KW - longitudinal observational study KW - fertility tracking KW - fertility KW - reproductive health KW - feasibility KW - voice recording KW - vocal pitch KW - follicular KW - luteal phase KW - fertility status KW - mobile phone N2 - Background: Identifying subtle changes in the menstrual cycle is crucial for effective fertility tracking and understanding reproductive health. Objective: The aim of the study is to explore how fundamental frequency features vary between menstrual phases using daily voice recordings. Methods: This study analyzed smartphone-collected voice recordings from 16 naturally cycling female participants, collected every day for 1 full menstrual cycle. Fundamental frequency features (mean, SD, 5th percentile, and 95th percentile) were extracted from each voice recording. Ovulation was estimated using luteinizing hormone urine tests taken every morning. The analysis included comparisons of these features between the follicular and luteal phases and the application of changepoint detection algorithms to assess changes and pinpoint the day in which the shifts in vocal pitch occur. Results: The fundamental frequency SD was 9.0% (SD 2.9%) lower in the luteal phase compared to the follicular phase (95% CI 3.4%?14.7%; P=.002), and the 5th percentile of the fundamental frequency was 8.8% (SD 3.6%) higher (95% CI 1.7%?16.0%; P=.01). No significant differences were found between phases in mean fundamental frequency or the 95th percentile of the fundamental frequency (P=.65 and P=.07). Changepoint detection, applied separately to each feature, identified the point in time when vocal frequency behaviors shifted. For the fundamental frequency SD and 5th percentile, 81% (n=13) of participants exhibited shifts within the fertile window (P=.03). In comparison, only 63% (n=10; P=.24) and 50% (n=8; P=.50) of participants had shifts in the fertile window for the mean and 95th percentile of the fundamental frequency, respectively. Conclusions: These findings indicate that subtle variations in vocal pitch may reflect changes associated with the menstrual cycle, suggesting the potential for developing a noninvasive and convenient method for monitoring reproductive health. Changepoint detection may provide a promising avenue for future work in longitudinal fertility analysis. UR - https://formative.jmir.org/2025/1/e65448 UR - http://dx.doi.org/10.2196/65448 ID - info:doi/10.2196/65448 ER - TY - JOUR AU - Malburg, M. Carly AU - Gutreuter, Steve AU - Ruiseñor-Escudero, Horacio AU - Abdul-Quader, Abu AU - Hladik, Wolfgang PY - 2025/1/9 TI - Population Size Estimation of Men Who Have Sex With Men in Low- and Middle-Income Countries: Google Trends Analysis JO - JMIR Public Health Surveill SP - e58630 VL - 11 KW - population size estimation KW - men who have sex with men KW - MSM KW - PSE KW - google trends KW - HIV KW - AIDS KW - programming and policy KW - internet KW - porn KW - gay porn KW - male adult KW - geriatric KW - linear regression KW - homosexuality KW - sensitivity analysis KW - World Health Organization KW - WHO KW - epidemiology N2 - Background: Population size estimation (PSE) for key populations is needed to inform HIV programming and policy. Objective: This study aimed to examine the utility of applying a recently proposed method using Google Trend (GT) internet search data to generate PSE (Google Trends Population Size Estimate [GTPSE]) for men who have sex with men (MSM) in 54 countries in Africa, Asia, the Americas, and Europe. Methods: We examined GT relative search volumes (representing the relative internet search frequency of specific search terms) for ?porn? and, as a comparator term, ?gay porn? for the year 2020. We assumed ?porn? represents ?men? (denominator) while ?gay porn? represents a subset of ?MSM? (numerator) in each county, resulting in a proportional size estimate for MSM. We multiplied the proportional GTPSE values with the countries? male adult population (15?49 years) to obtain absolute size estimates. Separately, we produced subnational MSM PSE limited to countries? (commercial) capitals. Using linear regression analysis, we examined the effect of countries? levels of urbanization, internet penetration, criminalization of homosexuality, and stigma on national GTPSE results. We conducted a sensitivity analysis in a subset of countries (n=14) examining the effect of alternative English search terms, different language search terms (Spanish, French, and Swahili), and alternative search years (2019 and 2021). Results: One country was excluded from our analysis as no GT data could be obtained. Of the remaining 53 countries, all national GTPSE values exceeded the World Health Organization?s recommended minimum PSE threshold of 1% (range 1.2%?7.5%). For 44 out of 49 (89.8%) of the countries, GTPSE results were higher than Joint United Nations Programme on HIV/AIDS (UNAIDS) Key Population Atlas values but largely consistent with the regional UNAIDS Global AIDS Monitoring results. Substantial heterogeneity across same-region countries was evident in GTPSE although smaller than those based on Key Population Atlas data. Subnational GTPSE values were obtained in 51 out of 53 (96%) countries; all subnational GTPSE values exceeded 1% but often did not match or exceed the corresponding countries? national estimates. None of the covariates examined had a substantial effect on the GTPSE values (R2 values 0.01?0.28). Alternative (English) search terms in 12 out of 14 (85%) countries produced GTPSE>1%. Using non-English language terms often produced markedly lower same-country GTPSE values compared with English with 10 out of 14 (71%) countries showing national GTPSE exceeding 1%. GTPSE used search data from 2019 and 2021, yielding results similar to those of the reference year 2020. Due to a lack of absolute search volume data, credibility intervals could not be computed. The validity of key assumptions, especially who (males and females) searches for porn and gay porn, could not be assessed. Conclusions: GTPSE for MSM provides a simple, fast, essentially cost-free method. Limitations that impact the certainty of our estimates include a lack of validation of key assumptions and an inability to assign credibility intervals. GTPSE for MSM may provide an additional data source, especially for estimating national-level PSE. UR - https://publichealth.jmir.org/2025/1/e58630 UR - http://dx.doi.org/10.2196/58630 ID - info:doi/10.2196/58630 ER - TY - JOUR AU - Adamou, Marios AU - Jones, L. Sarah AU - Kyriakidou, Niki AU - Mooney, Andrew AU - Pattani, Shriti AU - Roycroft, Matthew PY - 2025/1/9 TI - Measuring Self-Reported Well-Being of Physicians Using the Well-Being Thermometer: Cohort Study JO - JMIR Form Res SP - e54158 VL - 9 KW - well-being KW - health care professionals KW - mental health KW - well-being thermometer KW - health care N2 - Background: Advancements in medical science have focused largely on patient care, often overlooking the well-being of health care professionals (HCPs). This oversight has consequences; not only are HCPs prone to mental and physical health challenges, but the quality of patient care may also endure as a result. Such concerns are also exacerbated by unprecedented crises like the COVID-19 pandemic. Compared to other sectors, HCPs report high incidence of stress, depression, and suicide, among other challenging factors that have a significant negative impact on their well-being. Objective: Given these substantial concerns, the development of a tool specifically designed to be used in clinical settings to measure the well-being of HCPs is essential. Methods: A United Kingdom?based cross-sectional pilot study was carried out to measure self-reported well-being in a cohort of 148 physicians, using the newly developed well-being thermometer. The aim of the tool is to allow respondents to develop an individual sense of ?well-being intelligence? thus supporting HCPs to have better insight and control over their well-being and allow insights into how to manage it. The tool consists of 5 well-being domains?health, thoughts, emotions, spiritual, and social. Each domain can be measured individually or combined to produce an overall well-being score. Results: The tool demonstrated good internal consistency; the Cronbach ? in this study was 0.84 for the total scale. Conclusions: Results from this cohort demonstrated that the well-being thermometer can be used to gather intelligence of staff well-being. This is a promising new tool that will assist HCPs to recognize their own well-being needs and allow health care organizations to facilitate change in policies and practices to reflect a better understanding of staff well-being. UR - https://formative.jmir.org/2025/1/e54158 UR - http://dx.doi.org/10.2196/54158 ID - info:doi/10.2196/54158 ER - TY - JOUR AU - Atchison, J. Christina AU - Gilby, Nicholas AU - Pantelidou, Galini AU - Clemens, Sam AU - Pickering, Kevin AU - Chadeau-Hyam, Marc AU - Ashby, Deborah AU - Barclay, S. Wendy AU - Cooke, S. Graham AU - Darzi, Ara AU - Riley, Steven AU - Donnelly, A. Christl AU - Ward, Helen AU - Elliott, Paul PY - 2025/1/9 TI - Strategies to Increase Response Rate and Reduce Nonresponse Bias in Population Health Research: Analysis of a Series of Randomized Controlled Experiments during a Large COVID-19 Study JO - JMIR Public Health Surveill SP - e60022 VL - 11 KW - study recruitment KW - response rate KW - population-based research KW - COVID-19 KW - SARS-CoV-2 KW - web-based questionnaires N2 - Background: High response rates are needed in population-based studies, as nonresponse reduces effective sample size and bias affects accuracy and decreases the generalizability of the study findings. Objective: We tested different strategies to improve response rate and reduce nonresponse bias in a national population?based COVID-19 surveillance program in England, United Kingdom. Methods: Over 19 rounds, a random sample of individuals aged 5 years and older from the general population in England were invited by mail to complete a web-based questionnaire and return a swab for SARS-CoV-2 testing. We carried out several nested randomized controlled experiments to measure the impact on response rates of different interventions, including (1) variations in invitation and reminder letters and SMS text messages and (2) the offer of a conditional monetary incentive to return a swab, reporting absolute changes in response and relative response rate (95% CIs). Results: Monetary incentives increased the response rate (completed swabs returned as a proportion of the number of individuals invited) across all age groups, sex at birth, and area deprivation with the biggest increase among the lowest responders, namely teenagers and young adults and those living in more deprived areas. With no monetary incentive, the response rate was 3.4% in participants aged 18?22 years, increasing to 8.1% with a £10 (US $12.5) incentive, 11.9% with £20 (US $25.0), and 18.2% with £30 (US $37.5) (relative response rate 2.4 [95% CI 2.0-2.9], 3.5 [95% CI 3.0-4.2], and 5.4 [95% CI 4.4-6.7], respectively). Nonmonetary strategies had a modest, if any, impact on response rate. The largest effect was observed for sending an additional swab reminder (SMS text message or email). For example, those receiving an additional SMS text message were more likely to return a completed swab compared to those receiving the standard email-SMS approach, 73.3% versus 70.2%: percentage difference 3.1% (95% CI 2.2%-4.0%). Conclusions: Conditional monetary incentives improved response rates to a web-based survey, which required the return of a swab test, particularly for younger age groups. Used in a selective way, incentives may be an effective strategy for improving sample response and representativeness in population-based studies. UR - https://publichealth.jmir.org/2025/1/e60022 UR - http://dx.doi.org/10.2196/60022 ID - info:doi/10.2196/60022 ER - TY - JOUR AU - Danoff-Burg, Sharon AU - Gottlieb, Elie AU - Weaver, A. Morgan AU - Carmon, C. Kiara AU - Lara Ledesma, Duvia AU - Rus, M. Holly PY - 2025/1/3 TI - Effects of Smart Goggles Used at Bedtime on Objectively Measured Sleep and Self-Reported Anxiety, Stress, and Relaxation: Pre-Post Pilot Study JO - JMIR Form Res SP - e58461 VL - 9 KW - relaxation KW - stress KW - anxiety KW - sleep KW - health technology KW - intervention N2 - Background: Insufficient sleep is a problem affecting millions. Poor sleep can trigger or worsen anxiety; conversely, anxiety can lead to or exacerbate poor sleep. Advances in innovative consumer products designed to promote relaxation and support healthy sleep are emerging, and their effectiveness can be evaluated accurately using sleep measurement technologies in the home environment. Objective: This pilot study examined the effects of smart goggles used at bedtime to deliver gentle, slow vibration to the eyes and temples. The study hypothesized that objective sleep, perceived sleep, self-reported stress, anxiety, relaxation, and sleepiness would improve after using the smart goggles. Methods: A within-participants, pre-post study design was implemented. Healthy adults with subclinical threshold sleep problems (N=20) tracked their sleep nightly using a polysomnography-validated noncontact biomotion device and completed daily questionnaires over two phases: a 3-week baseline period and a 3-week intervention period. During the baseline period, participants followed their usual sleep routines at home. During the intervention period, participants used Therabody SmartGoggles in ?Sleep? mode at bedtime. This mode, designed for relaxation, delivers a gentle eye and temple massage through the inflation of internal compartments to create a kneading sensation combined with vibrating motors. Each night, the participants completed questionnaires assessing relaxation, stress, anxiety, and sleepiness immediately before and after using the goggles. Daily morning questionnaires assessed perceived sleep, complementing the objective sleep data measured every night. Results: Multilevel regression analysis of 676 nights of objective sleep parameters showed improvements during nights when the goggles were used compared to the baseline period. Key findings include sleep duration (increased by 12 minutes, P=.01); duration of deep sleep (increased by 6 minutes, P=.002); proportion of deep sleep (7% relative increase, P=.02); BodyScore, an age- and gender-normalized measure of deep sleep (4% increase, P=.002); number of nighttime awakenings (7% decrease, P=.02); total time awake after sleep onset (reduced by 6 minutes, P=.047); and SleepScore, a measure of overall sleep quality (3% increase, P=.02). Questionnaire responses showed that compared to baseline, participants felt they had better sleep quality (P<.001) and woke feeling more well-rested (P<.001). Additionally, participants reported feeling sleepier, less stressed, less anxious, and more relaxed (all P values <.05) immediately after using the goggles each night, compared to immediately before use. A standardized inventory administered before and after the 3-week intervention period indicated reduced anxiety (P=.03), confirming the nightly analysis. Conclusions: The use of smart goggles at bedtime significantly improved objectively measured sleep metrics and perceived sleep quality. Further, participants reported increased feelings of relaxation along with reduced stress and anxiety. Future research expanding on this pilot study is warranted to confirm and expand on the preliminary evidence presented in this brief report. UR - https://formative.jmir.org/2025/1/e58461 UR - http://dx.doi.org/10.2196/58461 ID - info:doi/10.2196/58461 ER - TY - JOUR AU - Law, Vivienne AU - Afolalu, F. Esther AU - Abetz-Webb, Linda AU - Wemyss, Andrew Lee AU - Turner, Andrew AU - Chrea, Christelle PY - 2025/1/2 TI - International Expert Consensus on Relevant Health and Functioning Concepts to Assess in Users of Tobacco and Nicotine Products: Delphi Study JO - JMIR Form Res SP - e58614 VL - 9 KW - Delphi study KW - expert consensus KW - outcome measures KW - health and functioning KW - tobacco and/or nicotine products N2 - Background: A Delphi study was conducted to reach a consensus among international clinical and health care experts on the most important health and functioning self-reported concepts when evaluating a switch from smoking cigarettes to using smoke-free tobacco and/or nicotine products (sf-TNPs). Objective: The aim of this research was to identify concepts considered important to measure when assessing the health and functioning status of users of tobacco and/or nicotine products. Methods: Experts (n=105), including health care professionals, researchers, and policy makers, from 26 countries with professional experience and knowledge of sf-TNPs completed a 3-round, adapted Delphi panel. Online surveys combining quantitative (MaxDiff best-worst scaling and latent class analysis) and qualitative assessments were used to rank and achieve alignment on the importance of 69 health and functioning concepts. All experts participating in round I completed round II, and 101 (95%) completed round III. Results: The round I analysis identified 36 (52%) out of 69 concepts that were refined for the round II assessment. The highest-ranked concepts reflected health-related impacts, while the lowest-ranked ranked concepts were related to aesthetics and social impacts. Round II ranking reinforced the importance of concepts relating to health impacts, and the analysis resulted in 20 concepts retained for round III assessment. In round III, the 4 highest-ranked concepts were cardiovascular symptoms, shortness of breath, chest pain, and worry about smoking-related diseases and impact on general health, and they made up 50% of the total score in the MaxDiff analysis. Experts reported likelihood of seeing measurable levels of change in the final 20 concepts with a switch to an sf-TNP. The majority of experts felt it was ?likely? or ?extremely likely? to observe changes in concepts such as gum problems (74/101, 73%), phlegm or mucus while coughing or not coughing (72/101, 71%), general perception of well-being (72/101, 71%), and throat irritation or sore throat (72/101, 71%). Latent class analysis revealed subgroups of experts with different perceptions of the relative importance of the concepts, which varied depending on professional specialty and geographic region. For example, 74% (14/19) of oncologists aligned with the subgroup prioritizing physical health symptoms, while 71% (12/17) of experts from Asia aligned with the subgroup considering both physical health and psychosocial aspects. Conclusions: This study identified key concepts to be considered in the development of a new measurement instrument to assess the self-reported health and functioning status of individuals using sf-TNPs. The findings contribute to the scientific evidence base for understanding and evaluating both the individual and public health impacts of sf-TNPs. UR - https://formative.jmir.org/2025/1/e58614 UR - http://dx.doi.org/10.2196/58614 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58614 ER - TY - JOUR AU - Strojny, Pawe? AU - Kapela, Ksawery AU - Lipp, Natalia AU - Sikström, Sverker PY - 2024/12/31 TI - Use of 4 Open-Ended Text Responses to Help Identify People at Risk of Gaming Disorder: Preregistered Development and Usability Study Using Natural Language Processing JO - JMIR Serious Games SP - e56663 VL - 12 KW - gaming disorder KW - natural language processing KW - machine learning KW - mental health KW - NLP KW - text KW - open-ended KW - response KW - risk KW - psychological KW - Question-based Computational Language Assessment KW - QCLA KW - transformers-based KW - language model analysis KW - Polish KW - Pearson KW - correlation KW - Python N2 - Background: Words are a natural way to describe mental states in humans, while numerical values are a convenient and effective way to carry out quantitative psychological research. With the growing interest of researchers in gaming disorder, the number of screening tools is growing. However, they all require self-quantification of mental states. The rapid development of natural language processing creates an opportunity to supplement traditional rating scales with a question-based computational language assessment approach that gives a deeper understanding of the studied phenomenon without losing the rigor of quantitative data analysis. Objective: The aim of the study was to investigate whether transformer-based language model analysis of text responses from active gamers is a potential supplement to traditional rating scales. We compared a tool consisting of 4 open-ended questions formulated based on the clinician's intuition (not directly related to any existing rating scales for measuring gaming disorders) with the results of one of the commonly used rating scales. Methods: Participants recruited using an online panel were asked to answer the Word-Based Gaming Disorder Test, consisting of 4 open-ended questions about gaming. Subsequently, they completed a closed-ended Gaming Disorders Test based on a numerical scale. Of the initial 522 responses collected, we removed a total of 105 due to 1 of 3 criteria (suspiciously low survey completion time, providing nonrelevant or incomplete responses). Final analyses were conducted on the responses of 417 participants. The responses to the open-ended questions were vectorized using HerBERT, a large language model based on Google's Bidirectional Encoder Representations from Transformers (BERT). Last, a machine learning model, specifically ridge regression, was used to predict the scores of the Gaming Disorder Test based on the features of the vectorized open-ended responses. Results: The Pearson correlation between the observable scores from the Gaming Disorder test and the predictions made by the model was 0.476 when using the answers of the 4 respondents as features. When using only 1 of the 4 text responses, the correlation ranged from 0.274 to 0.406. Conclusions: Short open responses analyzed using natural language processing can contribute to a deeper understanding of gaming disorder at no additional cost in time. The obtained results confirmed 2 of 3 preregistered hypotheses. The written statements analyzed using the results of the model correlated with the rating scale. Furthermore, the inclusion in the model of data from more responses that take into account different perspectives on gaming improved the performance of the model. However, there is room for improvement, especially in terms of supplementing the questions with content that corresponds more directly to the definition of gaming disorder. Trial Registration: OSF Registries osf.io/957nz; https://osf.io/957nz UR - https://games.jmir.org/2024/1/e56663 UR - http://dx.doi.org/10.2196/56663 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56663 ER - TY - JOUR AU - Rinehart, Marie Linda AU - Anker, Justin AU - Unruh, Amanda AU - Degeneffe, Nikki AU - Thuras, Paul AU - Norden, Amie AU - Hartnett, Lilly AU - Kushner, Matt PY - 2024/12/31 TI - Supplemental Intervention for Alcohol Use Disorder Treatment Patients With a Co-Occurring Anxiety Disorder: Technical Development and Functional Testing of an Autonomous Digital Program JO - JMIR Form Res SP - e62995 VL - 8 KW - alcohol use disorder KW - anxiety disorder KW - comorbidity KW - digital intervention KW - psychological treatments KW - addiction KW - community-based practice KW - therapy KW - stress KW - depression KW - therapist-delivered therapies N2 - Background: Anxiety disorders are common in alcohol use disorder (AUD) treatment patients. Such co-occurring conditions (?comorbidity?) have negative prognostic implications for AUD treatment outcomes, yet they commonly go unaddressed in standard AUD care. Over a decade ago, we developed and validated a cognitive behavioral therapy intervention to supplement standard AUD care that, when delivered by trained therapists, improves outcomes in comorbid patients. However, this validated intervention, like many others in addiction care, has not been taken up in community-based AUD treatment programs. This phenomenon?empirically validated treatments that fail to be widely adopted in community care?has been termed the ?research-to-practice gap.? Researchers have suggested that the availability of fully autonomous digital equivalents of validated therapist-delivered therapies could reduce some barriers underlying the research-to-practice gap, especially by eliminating the need for costly and intensive therapist training and supervision. Objective: With this in mind, we obtained a Program Development Grant (R34) to conduct formative work in the development of a fully autonomous digital version of our previously validated therapist-delivered intervention for AUD treatment patients with a comorbid anxiety disorder. Methods: In the first phase of the project, we developed the digital intervention. This process included: (1) identifying appropriate collaborators and vendors; (2) consultation with an e-learning expert to develop a storyboard and accompanying graphics and narrative; (3) video production and editing; and (4) interactive programming. The second phase of the project was functional testing of the newly developed digital intervention conducted in 52 residential AUD treatment patients with a comorbid anxiety disorder. Patients underwent the 3 one-hour segments of the newly developed intervention and completed user surveys, knowledge quizzes, and behavioral competence tests. Results: While the development of the digital intervention was successful, the timeline was approximately double that projected (1 vs 2 years) due to false starts and inefficiencies that we describe, including lessons learned. Functional testing of the newly developed digital intervention showed that, on average, patients rated the user experience in the upper (favorable) 20% of the response scales. Knowledge quizzes and behavioral demonstrations showed that over 80% of participants gained functional mastery of the key skills and information taught in the program. Conclusions: Functional testing results in this study justify a randomized controlled trial of the digital intervention?s efficacy, which is currently ongoing. In sharing the details of our challenges and solutions in developing the digital intervention, we hope to inform others developing digital tools. The extent to which the availability of empirically validated, fully autonomous digital interventions achieves their potential to reduce the research-to-practice gap remains an open but important empirical question. The present work stands as a necessary first step toward that end. UR - https://formative.jmir.org/2024/1/e62995 UR - http://dx.doi.org/10.2196/62995 ID - info:doi/10.2196/62995 ER - TY - JOUR AU - Dimitsaki, Stella AU - Natsiavas, Pantelis AU - Jaulent, Marie-Christine PY - 2024/12/30 TI - Applying AI to Structured Real-World Data for Pharmacovigilance Purposes: Scoping Review JO - J Med Internet Res SP - e57824 VL - 26 KW - pharmacovigilance KW - drug safety KW - artificial intelligence KW - machine learning KW - real-world data KW - scoping review N2 - Background: Artificial intelligence (AI) applied to real-world data (RWD; eg, electronic health care records) has been identified as a potentially promising technical paradigm for the pharmacovigilance field. There are several instances of AI approaches applied to RWD; however, most studies focus on unstructured RWD (conducting natural language processing on various data sources, eg, clinical notes, social media, and blogs). Hence, it is essential to investigate how AI is currently applied to structured RWD in pharmacovigilance and how new approaches could enrich the existing methodology. Objective: This scoping review depicts the emerging use of AI on structured RWD for pharmacovigilance purposes to identify relevant trends and potential research gaps. Methods: The scoping review methodology is based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We queried the MEDLINE database through the PubMed search engine. Relevant scientific manuscripts published from January 2010 to January 2024 were retrieved. The included studies were ?mapped? against a set of evaluation criteria, including applied AI approaches, code availability, description of the data preprocessing pipeline, clinical validation of AI models, and implementation of trustworthy AI criteria following the guidelines of the FUTURE (Fairness, Universality, Traceability, Usability, Robustness, and Explainability)-AI initiative. Results: The scoping review ultimately yielded 36 studies. There has been a significant increase in relevant studies after 2019. Most of the articles focused on adverse drug reaction detection procedures (23/36, 64%) for specific adverse effects. Furthermore, a substantial number of studies (34/36, 94%) used nonsymbolic AI approaches, emphasizing classification tasks. Random forest was the most popular machine learning approach identified in this review (17/36, 47%). The most common RWD sources used were electronic health care records (28/36, 78%). Typically, these data were not available in a widely acknowledged data model to facilitate interoperability, and they came from proprietary databases, limiting their availability for reproducing results. On the basis of the evaluation criteria classification, 10% (4/36) of the studies published their code in public registries, 16% (6/36) tested their AI models in clinical environments, and 36% (13/36) provided information about the data preprocessing pipeline. In addition, in terms of trustworthy AI, 89% (32/36) of the studies followed at least half of the trustworthy AI initiative guidelines. Finally, selection and confounding biases were the most common biases in the included studies. Conclusions: AI, along with structured RWD, constitutes a promising line of work for drug safety and pharmacovigilance. However, in terms of AI, some approaches have not been examined extensively in this field (such as explainable AI and causal AI). Moreover, it would be helpful to have a data preprocessing protocol for RWD to support pharmacovigilance processes. Finally, because of personal data sensitivity, evaluation procedures have to be investigated further. UR - https://www.jmir.org/2024/1/e57824 UR - http://dx.doi.org/10.2196/57824 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57824 ER - TY - JOUR AU - Hoffman, Jane AU - Hattingh, Laetitia AU - Shinners, Lucy AU - Angus, L. Rebecca AU - Richards, Brent AU - Hughes, Ian AU - Wenke, Rachel PY - 2024/12/30 TI - Allied Health Professionals? Perceptions of Artificial Intelligence in the Clinical Setting: Cross-Sectional Survey JO - JMIR Form Res SP - e57204 VL - 8 KW - allied health KW - artificial intelligence KW - hospital KW - digital health KW - impact KW - AI KW - mHealth KW - cross sectional KW - survey KW - health professional KW - medical professional KW - perception KW - clinical setting KW - opportunity KW - challenge KW - healthcare KW - delivery KW - Australia KW - clinician KW - confirmatory factor analysis KW - linear regression N2 - Background: Artificial intelligence (AI) has the potential to address growing logistical and economic pressures on the health care system by reducing risk, increasing productivity, and improving patient safety; however, implementing digital health technologies can be disruptive. Workforce perception is a powerful indicator of technology use and acceptance, however, there is little research available on the perceptions of allied health professionals (AHPs) toward AI in health care. Objective: This study aimed to explore AHP perceptions of AI and the opportunities and challenges for its use in health care delivery. Methods: A cross-sectional survey was conducted at a health service in, Queensland, Australia, using the Shinners Artificial Intelligence Perception tool. Results: A total of 231 (22.1%) participants from 11 AHPs responded to the survey. Participants were mostly younger than 40 years (157/231, 67.9%), female (189/231, 81.8%), working in a clinical role (196/231, 84.8%) with a median of 10 years? experience in their profession. Most participants had not used AI (185/231, 80.1%), had little to no knowledge about AI (201/231, 87%), and reported workforce knowledge and skill as the greatest challenges to incorporating AI in health care (178/231, 77.1%). Age (P=.01), profession (P=.009), and AI knowledge (P=.02) were strong predictors of the perceived professional impact of AI. AHPs generally felt unprepared for the implementation of AI in health care, with concerns about a lack of workforce knowledge on AI and losing valued tasks to AI. Prior use of AI (P=.02) and years of experience as a health care professional (P=.02) were significant predictors of perceived preparedness for AI. Most participants had not received education on AI (190/231, 82.3%) and desired training (170/231, 73.6%) and believed AI would improve health care. Ideas and opportunities suggested for the use of AI within the allied health setting were predominantly nonclinical, administrative, and to support patient assessment tasks, with a view to improving efficiencies and increasing clinical time for direct patient care. Conclusions: Education and experience with AI are needed in health care to support its implementation across allied health, the second largest workforce in health. Industry and academic partnerships with clinicians should not be limited to AHPs with high AI literacy as clinicians across all knowledge levels can identify many opportunities for AI in health care. UR - https://formative.jmir.org/2024/1/e57204 UR - http://dx.doi.org/10.2196/57204 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57204 ER - TY - JOUR AU - Luo, Waylon AU - Jin, Ruoming AU - Kenne, Deric AU - Phan, NhatHai AU - Tang, Tang PY - 2024/12/30 TI - An Analysis of the Prevalence and Trends in Drug-Related Lyrics on Twitter (X): Quantitative Approach JO - JMIR Form Res SP - e49567 VL - 8 KW - Twitter (X) KW - popular music KW - big data analysis KW - music KW - lyrics KW - big data KW - substance abuse KW - tweet KW - social media KW - drug KW - alcohol N2 - Background: The pervasiveness of drug culture has become evident in popular music and social media. Previous research has examined drug abuse content in both social media and popular music; however, to our knowledge, the intersection of drug abuse content in these 2 domains has not been explored. To address the ongoing drug epidemic, we analyzed drug-related content on Twitter (subsequently rebranded X), with a specific focus on lyrics. Our study provides a novel finding on the prevalence of drug abuse by defining a new subcategory of X content: ?tweets that reference established drug lyrics.? Objective: We aim to investigate drug trends in popular music on X, identify and classify popular drugs, and analyze related artists? gender, genre, and popularity. Based on the collected data, our goal is to create a prediction model for future drug trends and gain a deeper understanding of the characteristics of users who cite drug lyrics on X. Methods: X data were collected from 2015 to 2017 through the X streaming application programming interface (API). Drug lyrics were obtained from the Genius lyrics database using the Genius API based on drug keywords. The Smith-Waterman text-matching algorithm is used to detect the drug lyrics in posts. We identified famous drugs in lyrics that were posted. Consequently, the analysis was extended to related artists, songs, genres, and popularity on X. The frequency of drug-related lyrics on X was aggregated into a time-series, which was then used to create prediction models using linear regression, Facebook Prophet, and NIXTLA TimeGPT-1. In addition, we analyzed the number of followers of users posting drug-related lyrics to explore user characteristics. Results: We analyzed over 1.97 billion publicly available posts from 2015 to 2017, identifying more than 157 million that matched drug-related keywords. Of these, 150,746 posts referenced drug-related lyrics. Cannabinoids, opioids, stimulants, and hallucinogens were the most cited drugs in lyrics on X. Rap and hip-hop dominated, with 91.98% of drug-related lyrics from these genres and 84.21% performed by male artists. Predictions from all 3 models, linear regression, Facebook Prophet, and NIXTLA TimeGPT-1, indicate a slight decline in the prevalence of drug-related lyrics on X over time. Conclusions: Our study revealed 2 significant findings. First, we identified a previously unexamined subset of drug-related content on X: drug lyrics, which could play a critical role in models predicting the surge in drug-related incidents. Second, we demonstrated the use of cutting-edge time-series forecasting tools, including Facebook Prophet and NIXTLA TimeGPT-1, in accurately predicting these trends. These insights contribute to our understanding of how social media shapes public behavior and sentiment toward drug use. UR - https://formative.jmir.org/2024/1/e49567 UR - http://dx.doi.org/10.2196/49567 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/49567 ER - TY - JOUR AU - Beiler, Donielle AU - Chopra, Aanya AU - Gregor, M. Christina AU - Tusing, D. Lorraine AU - Pradhan, M. Apoorva AU - Romagnoli, M. Katrina AU - Kraus, K. Chadd AU - Piper, J. Brian AU - Wright, A. Eric AU - Troiani, Vanessa PY - 2024/12/23 TI - Medical Marijuana Documentation Practices in Patient Electronic Health Records: Retrospective Observational Study Using Smart Data Elements and a Review of Medical Records JO - JMIR Form Res SP - e65957 VL - 8 KW - cannabis KW - learning health system KW - Epic KW - prescription drug monitoring program KW - medical marijuana KW - electronic health records KW - physician KW - cannabis use KW - drug use KW - data sharing KW - patient care KW - legalization KW - dosage KW - chart review protocol KW - human data extraction KW - data collection N2 - Background: Medical marijuana (MMJ) is available in Pennsylvania, and participation in the state-regulated program requires patient registration and receiving certification by an approved physician. Currently, no integration of MMJ certification data with health records exists in Pennsylvania that would allow clinicians to rapidly identify patients using MMJ, as exists with other scheduled drugs. This absence of a formal data sharing structure necessitates tools aiding in consistent documentation practices to enable comprehensive patient care. Customized smart data elements (SDEs) were made available to clinicians at an integrated health system, Geisinger, following MMJ legalization in Pennsylvania. Objective: The purpose of this project was to examine and contextualize the use of MMJ SDEs in the Geisinger population. We accomplished this goal by developing a systematic protocol for review of medical records and creating a tool that resulted in consistent human data extraction. Methods: We developed a protocol for reviewing medical records for extracting MMJ-related information. The protocol was developed between August and December of 2022 and focused on a patient group that received one of several MMJ SDEs between January 25, 2019, and May 26, 2022. Characteristics were first identified on a pilot sample (n=5), which were then iteratively reviewed to optimize for consistency. Following the pilot, 2 reviewers were assigned 200 randomly selected patients? medical records, with a third reviewer examining a subsample (n=30) to determine reliability. We then summarized the clinician- and patient-level features from 156 medical records with a table-format SDE that best captured MMJ information. Results: We found the review protocol for medical records was feasible for those with minimal medical background to complete, with high interrater reliability (?=0.966; P<.001; odds ratio 0.97, 95% CI 0.954-0.978). MMJ certification was largely documented by nurses and medical assistants (n=138, 88.5%) and typically within primary care settings (n=107, 68.6%). The SDE has 6 preset field prompts with heterogeneous documentation completion rates, including certifying conditions (n=146, 93.6%), product (n=145, 92.9%), authorized dispensary (n=137, 87.8%), active ingredient (n=130, 83.3%), certifying provider (n=96, 61.5%), and dosage (n=48, 30.8%). We found preset fields were overall well-recorded (mean 76.6%, SD 23.7% across all fields). Primary diagnostic codes recorded at documentation encounters varied, with the most frequent being routine examinations and testing (n=34, 21.8%), musculoskeletal or nervous conditions, and signs and symptoms not classified elsewhere (n=21, 13.5%). Conclusions: This method of reviewing medical records yields high-quality data extraction that can serve as a model for other health record inquiries. Our evaluation showed relatively high completeness of SDE fields, primarily by clinical staff responsible for rooming patients, with an overview of conditions under which MMJ is documented. Improving the adoption and fidelity of SDE data collection may present a valuable data source for future research on patient MMJ use, treatment efficacy, and outcomes. UR - https://formative.jmir.org/2024/1/e65957 UR - http://dx.doi.org/10.2196/65957 ID - info:doi/10.2196/65957 ER - TY - JOUR AU - Wu, Sixuan AU - Song, Kefan AU - Cobb, Jason AU - Adams, T. Alexander PY - 2024/12/23 TI - Pump-Free Microfluidics for Cell Concentration Analysis on Smartphones in Clinical Settings (SmartFlow): Design, Development, and Evaluation JO - JMIR Biomed Eng SP - e62770 VL - 9 KW - mobile health KW - mHealth KW - ubiquitous health KW - smartphone KW - chip KW - microscope KW - microfluidics KW - cells counting, body fluid analysis, blood test, urinalysis, computer vision, machine learning KW - fluid KW - cell KW - cellular KW - concentration N2 - Background: Cell concentration in body fluid is an important factor for clinical diagnosis. The traditional method involves clinicians manually counting cells under microscopes, which is labor-intensive. Automated cell concentration estimation can be achieved using flow cytometers; however, their high cost limits accessibility. Microfluidic systems, although cheaper than flow cytometers, still require high-speed cameras and syringe pumps to drive the flow and ensure video quality. In this paper, we present SmartFlow, a low-cost solution for cell concentration estimation using smartphone-based computer vision on 3D-printed, pump-free microfluidic platforms. Objective: The objective was to design and fabricate microfluidic chips, coupled with clinical utilities, for cell counting and concentration analysis. We answered the following research questions (RQs): RQ1, Can gravity drive the flow within the microfluidic chips, eliminating the need for external pumps? RQ2, How does the microfluidic chip design impact video quality for cell analysis? RQ3, Can smartphone-captured videos be used to estimate cell count and concentration in microfluidic chips? Methods: To answer the 3 RQs, 2 experiments were conducted. In the cell flow velocity experiment, diluted sheep blood flowed through the microfluidic chips with and without a bottleneck design to answer RQ1 and RQ2, respectively. In the cell concentration analysis experiment, sheep blood diluted into 13 concentrations flowed through the microfluidic chips while videos were recorded by smartphones for the concentration measurement. Results: In the cell flow velocity experiment, we designed and fabricated 2 versions of microfluidic chips. The ANOVA test (Straight: F6, 99=6144.45, P<.001; Bottleneck: F6, 99=3475.78, P<.001) showed the height difference had a significant impact on the cell velocity, which implied gravity could drive the flow. The video sharpness analysis demonstrated that video quality followed an exponential decay with increasing height differences (video quality=100e?k×Height) and a bottleneck design could effectively preserve video quality (Straight: R2=0.95, k=4.33; Bottleneck: R2=0.91, k=0.59). Samples from the 13 cell concentrations were used for cell counting and cell concentration estimation analysis. The accuracy of cell counting (n=35, 60-second samples, R2=0.96, mean absolute error=1.10, mean squared error=2.24, root mean squared error=1.50) and cell concentration regression (n=39, 150-second samples, R2=0.99, mean absolute error=0.24, mean squared error=0.11, root mean squared error=0.33 on a logarithmic scale, mean average percentage error=0.25) were evaluated using 5-fold cross-validation by comparing the algorithmic estimation to ground truth. Conclusions: In conclusion, we demonstrated the importance of the flow velocity in a microfluidic system, and we proposed SmartFlow, a low-cost system for computer vision?based cellular analysis. The proposed system could count the cells and estimate cell concentrations in the samples. UR - https://biomedeng.jmir.org/2024/1/e62770 UR - http://dx.doi.org/10.2196/62770 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/62770 ER - TY - JOUR AU - Iino, Haru AU - Kizaki, Hayato AU - Imai, Shungo AU - Hori, Satoko PY - 2024/12/23 TI - Identifying the Relative Importance of Factors Influencing Medication Compliance in General Patients Using Regularized Logistic Regression and LightGBM: Web-Based Survey Analysis JO - JMIR Form Res SP - e65882 VL - 8 KW - medication adherence KW - pharmacological management KW - medication compliance KW - Japan KW - drugs KW - dose KW - psychological KW - questionnaire survey KW - LightGBM KW - logistic regression model KW - regularization KW - machine learning KW - AI KW - artificial intelligence N2 - Background: Medication compliance, which refers to the extent to which patients correctly adhere to prescribed regimens, is influenced by various psychological, behavioral, and demographic factors. When analyzing these factors, challenges such as multicollinearity and variable selection often arise, complicating the interpretation of results. To address the issue of multicollinearity and better analyze the importance of each factor, machine learning methods are considered to be useful. Objective: This study aimed to identify key factors influencing medication compliance by applying regularized logistic regression and LightGBM. Methods: A questionnaire survey was conducted among 638 adult patients in Japan who had been continuously taking medications for at least 3 months. The survey collected data on demographics, medication habits, psychological adherence factors, and compliance. Logistic regression with regularization was used to handle multicollinearity, while LightGBM was used to calculate feature importance. Results: The regularized logistic regression model identified significant predictors, including ?using the drug at approximately the same time each day? (coefficient 0.479; P=.02), ?taking meals at approximately the same time each day? (coefficient 0.407; P=.02), and ?I would like to have my medication reduced? (coefficient ?0.410; P=.01). The top 5 variables with the highest feature importance scores in the LightGBM results were ?Age? (feature importance 179.1), ?Using the drug at approximately the same time each day? (feature importance 148.4), ?Taking meals at approximately the same time each day? (feature importance 109.0), ?I would like to have my medication reduced? (feature importance 77.48), and ?I think I want to take my medicine? (feature importance 70.85). Additionally, the feature importance scores for the groups of medication adherence?related factors were 77.92 for lifestyle-related items, 52.04 for awareness of medication, 20.30 for relationships with health care professionals, and 5.05 for others. Conclusions: The most significant factors for medication compliance were the consistency of medication and meal timing (mean of feature importance), followed by the number of medications and patient attitudes toward their treatment. This study is the first to use a machine learning model to calculate and compare the relative importance of factors affecting medication adherence. Our findings demonstrate that, in terms of relative importance, lifestyle habits are the most significant contributors to medication compliance among the general patient population. The findings suggest that regularization and machine learning methods, such as LightGBM, are useful for better understanding the numerous adherence factors affected by multicollinearity. UR - https://formative.jmir.org/2024/1/e65882 UR - http://dx.doi.org/10.2196/65882 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65882 ER - TY - JOUR AU - Kim, Sanghwan AU - Jang, Sowon AU - Kim, Borham AU - Sunwoo, Leonard AU - Kim, Seok AU - Chung, Jin-Haeng AU - Nam, Sejin AU - Cho, Hyeongmin AU - Lee, Donghyoung AU - Lee, Keehyuck AU - Yoo, Sooyoung PY - 2024/12/20 TI - Automated Pathologic TN Classification Prediction and Rationale Generation From Lung Cancer Surgical Pathology Reports Using a Large Language Model Fine-Tuned With Chain-of-Thought: Algorithm Development and Validation Study JO - JMIR Med Inform SP - e67056 VL - 12 KW - AJCC Cancer Staging Manual 8th edition KW - American Joint Committee on Cancer KW - large language model KW - chain-of-thought KW - rationale KW - lung cancer KW - report analysis KW - AI KW - surgery KW - pathology reports KW - tertiary hospital KW - generative language models KW - efficiency KW - accuracy KW - automated N2 - Background: Traditional rule-based natural language processing approaches in electronic health record systems are effective but are often time-consuming and prone to errors when handling unstructured data. This is primarily due to the substantial manual effort required to parse and extract information from diverse types of documentation. Recent advancements in large language model (LLM) technology have made it possible to automatically interpret medical context and support pathologic staging. However, existing LLMs encounter challenges in rapidly adapting to specialized guideline updates. In this study, we fine-tuned an LLM specifically for lung cancer pathologic staging, enabling it to incorporate the latest guidelines for pathologic TN classification. Objective: This study aims to evaluate the performance of fine-tuned generative language models in automatically inferring pathologic TN classifications and extracting their rationale from lung cancer surgical pathology reports. By addressing the inefficiencies and extensive parsing efforts associated with rule-based methods, this approach seeks to enable rapid and accurate reclassification aligned with the latest cancer staging guidelines. Methods: We conducted a comparative performance evaluation of 6 open-source LLMs for automated TN classification and rationale generation, using 3216 deidentified lung cancer surgical pathology reports based on the American Joint Committee on Cancer (AJCC) Cancer Staging Manual8th edition, collected from a tertiary hospital. The dataset was preprocessed by segmenting each report according to lesion location and morphological diagnosis. Performance was assessed using exact match ratio (EMR) and semantic match ratio (SMR) as evaluation metrics, which measure classification accuracy and the contextual alignment of the generated rationales, respectively. Results: Among the 6 models, the Orca2_13b model achieved the highest performance with an EMR of 0.934 and an SMR of 0.864. The Orca2_7b model also demonstrated strong performance, recording an EMR of 0.914 and an SMR of 0.854. In contrast, the Llama2_7b model achieved an EMR of 0.864 and an SMR of 0.771, while the Llama2_13b model showed an EMR of 0.762 and an SMR of 0.690. The Mistral_7b and Llama3_8b models, on the other hand, showed lower performance, with EMRs of 0.572 and 0.489, and SMRs of 0.377 and 0.456, respectively. Overall, the Orca2 models consistently outperformed the others in both TN stage classification and rationale generation. Conclusions: The generative language model approach presented in this study has the potential to enhance and automate TN classification in complex cancer staging, supporting both clinical practice and oncology data curation. With additional fine-tuning based on cancer-specific guidelines, this approach can be effectively adapted to other cancer types. UR - https://medinform.jmir.org/2024/1/e67056 UR - http://dx.doi.org/10.2196/67056 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/67056 ER - TY - JOUR AU - Grover, Ashoo AU - Nair, Saritha AU - Sharma, Saurabh AU - Gupta, Shefali AU - Shrivastava, Suyesh AU - Singh, Pushpendra AU - Kanungo, Srikanta AU - Ovung, Senthanro AU - Singh, Charan AU - Khan, Mabood Abdul AU - Sharma, Sandeep AU - Palo, Kumar Subrata AU - Chakma, Tapas AU - Bajaj, Anjali PY - 2024/12/20 TI - Strengthening Cause of Death Statistics in Selected Districts of 3 States in India: Protocol for an Uncontrolled, Before-After, Mixed Method Study JO - JMIR Res Protoc SP - e51493 VL - 13 KW - cause of death KW - Medical Certification of Cause of Death KW - capacity building KW - Civil Registration and Vital Statistics KW - training N2 - Background: Mortality statistics are vital for health policy development, epidemiological research, and health care service planning. A robust surveillance system is essential for obtaining vital information such as cause of death (CoD) information. Objective: This study aims to develop a comprehensive model to strengthen the CoD information in the selected study sites. The specific objectives are (1) to identify the best practices and challenges in the functioning of the Civil Registration and Vital Statistics (CRVS) system with respect to mortality statistics and CoD information; (2) to develop and implement interventions to strengthen the CoD information; (3) to evaluate the quality improvement of the Medical Certification of Cause of Death (MCCD); and (4) to improve the CoD information at the population level through verbal autopsy for noninstitutional deaths in the selected study sites. Methods: An uncontrolled, before-after, mixed method study will be conducted in 3 blocks located in the districts of 3 states (Madhya Pradesh, Uttar Pradesh, and Odisha) in India. A baseline assessment to identify the best practices and challenges in the functioning of the CRVS system, along with a quality assessment of the MCCD, will be conducted. An intervention informed by existing literature and the baseline assessment will be developed and implemented in the study sites. The major components of intervention will include a Training of Trainers workshop, orientation of stakeholders in the functioning of the CRVS system, training of physicians and medical officers in the MCCD, and training of community health workers in World Health Organization Verbal Autopsy 2022 instrument. Postintervention evaluation will be carried out to assess the impact made by the intervention on the availability and quality improvement of CoD information in the selected study sites. The outcome will be measured in terms of the quality improvement of the MCCD and the availability of CoD information at population level through verbal autopsy in the selected study sites. Results: The project has been funded, and regulatory approval has been obtained from the Institutional Ethics Committee. The data collection process began in May 2023. The duration of the study will be for 24 months. Conclusions: Our study is expected to provide a valuable contribution toward strengthening CoD information, which could be helpful for policy making and further research. The intervention model will be developed in collaboration with the existing functionaries of the health and CRVS systems in the selected study sites that are engaged in reporting and recording CoD information; this will ensure sustainability and provide lessons for upscaling, with the aim to improve the reporting of CoD information in the country. International Registered Report Identifier (IRRID): DERR1-10.2196/51493 UR - https://www.researchprotocols.org/2024/1/e51493 UR - http://dx.doi.org/10.2196/51493 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/51493 ER - TY - JOUR AU - Austin, A. Jodie AU - Lobo, H. Elton AU - Samadbeik, Mahnaz AU - Engstrom, Teyl AU - Philip, Reji AU - Pole, D. Jason AU - Sullivan, M. Clair PY - 2024/12/20 TI - Decades in the Making: The Evolution of Digital Health Research Infrastructure Through Synthetic Data, Common Data Models, and Federated Learning JO - J Med Internet Res SP - e58637 VL - 26 KW - real-world data KW - digital health research KW - synthetic data KW - common data models KW - federated learning KW - university-industry collaboration UR - https://www.jmir.org/2024/1/e58637 UR - http://dx.doi.org/10.2196/58637 UR - http://www.ncbi.nlm.nih.gov/pubmed/39705072 ID - info:doi/10.2196/58637 ER - TY - JOUR AU - Jones, Debra AU - Sowerbutts, Marie Anne AU - Burden, Sorrel PY - 2024/12/18 TI - Exploring Individuals? Views and Feedback on a Nutritional Screening Mobile App: Qualitative Focus Group Study JO - JMIR Form Res SP - e63680 VL - 8 KW - malnutrition KW - malnutrition risk KW - malnutrition screening KW - MUST KW - mobile application KW - mHealth app KW - malnutrition universal screening tool N2 - Background: Malnutrition is a major global health challenge. Worldwide, approximately 390 million adults are underweight, while 2.5 billion are overweight. The Malnutrition Universal Screening Tool (MUST) has been implemented successfully in the United Kingdom to assess the nutritional status of patients in health care settings. Currently, MUST is available as a web-based tool or as a paper-based version, However, the paper tool can lead to calculation errors, and web-based tools require internet access, limiting use in some communities. The MUST app uses clear and simple navigation and processes information precisely, so could potentially improve the accuracy and accessibility of malnutrition screening for health care professionals (HCP) in all settings. Objective: This study aimed to explore the views of HCPs on the content, functionality, and usability of a newly developed mobile app for MUST. Methods: We performed a qualitative study using deductive and inductive framework analysis. A series of online focus groups (~1 hour each) were conducted, exploring potential users? views on the app?s content design, functionality, and usefulness, which was set in demonstration mode and not available for direct use with patients. Each focus group used a semistructured approach and predefined topic guide. Participants were recruited consecutively and United Kingdom?wide using advertisements through emails, newsletters, and on social media across appropriate local and national networks. Participants had the opportunity to look at the app on their phones before giving feedback and an on-screen demonstration of the app was provided during the focus group. Data were analyzed using deductive and inductive framework analysis. Results: In total, 8 online focus groups were conducted between August 2022 and January 2023. Participants (n=32) were dietetic and nutrition HCPs or educators with experience in using MUST in clinical or community settings. Data analysis revealed three broad themes: (1) improving the app for better use in practice, (2) user experience of design, and (3) barriers and facilitators in different settings. Overall feedback for the app was positive with potential users considering it to be very useful for improving routine and accurate screening, particularly in the community, and mainly because of the automatic calculation feature, which may help with improving discrepancies. Participants generally considered the app to be for professional use only, stating that patients may find it too clinical or technical. Participants also made suggestions for app sustainability and improvements, such as incentives to complete the demographics section or the option to skip questions, and the addition of more subjective measures and instructions on measuring ulna length. Conclusions: The MUST app was positively evaluated by potential users, who reported it was user-friendly and an accessible way to screen for malnutrition risk, whilst improving the accuracy of screening and availability in community settings. UR - https://formative.jmir.org/2024/1/e63680 UR - http://dx.doi.org/10.2196/63680 UR - http://www.ncbi.nlm.nih.gov/pubmed/39693128 ID - info:doi/10.2196/63680 ER - TY - JOUR AU - Wagh, Vaidehi AU - Scott, W. Matthew AU - Kraeutner, N. Sarah PY - 2024/12/17 TI - Quantifying Similarities Between MediaPipe and a Known Standard to Address Issues in Tracking 2D Upper Limb Trajectories: Proof of Concept Study JO - JMIR Form Res SP - e56682 VL - 8 KW - markerless pose estimation KW - procrustes analysis KW - artificial intelligence KW - motion KW - movement tracking KW - touchscreen KW - markerless tracking KW - upper limb KW - motor N2 - Background: Markerless motion tracking methods have promise for use in a range of domains, including clinical settings where traditional marker-based systems for human pose estimation are not feasible. Artificial intelligence (AI)?based systems can offer a markerless, lightweight approach to motion capture. However, the accuracy of such systems, such as MediaPipe, for tracking fine upper limb movements involving the hand has not been explored. Objective: The aim of this study is to evaluate the 2D accuracy of MediaPipe against a known standard. Methods: Participants (N=10) performed a touchscreen-based shape-tracing task requiring them to trace the trajectory of a moving cursor using their index finger. Cursor trajectories created a reoccurring or random shape at 5 different speeds (500-2500 ms, in increments of 500 ms). Movement trajectories on each trial were simultaneously captured by the touchscreen and a separate video camera. Movement coordinates for each trial were extracted from the touchscreen and compared to those predicted by MediaPipe. Specifically, following resampling, normalization, and Procrustes transformations, root-mean-squared error (RMSE; primary outcome measure) was calculated between predicted coordinates and those generated by the touchscreen computer. Results: Although there was some size distortion in the frame-by-frame estimates predicted by MediaPipe, shapes were similar between the 2 methods and transformations improved the general overlap and similarity of the shapes. The resultant mean RMSE between predicted coordinates and those generated by the touchscreen was 0.28 (SD 0.06) normalized px. Equivalence testing revealed that accuracy differed between MediaPipe and the touchscreen, but that the true difference was between 0 and 0.30 normalized px (t114=?3.02; P=.002). Additional analyses revealed no differences in resultant RMSE between methods when comparing across lower frame rates (30 and 60 frames per second [FPS]), although there was greater RMSE for 120 FPS than for 60 FPS (t35.43=?2.51; P=.03). Conclusions: Overall, we quantified similarities between one AI-based approach to motion capture and a known standard for tracking fine upper limb movements, informing applications of such systems in domains such as clinical and research settings. Future work should address accuracy in 3 dimensions to further validate the use of AI-based systems, including MediaPipe, in such domains. UR - https://formative.jmir.org/2024/1/e56682 UR - http://dx.doi.org/10.2196/56682 ID - info:doi/10.2196/56682 ER - TY - JOUR AU - Silvey, Scott AU - Liu, Jinze PY - 2024/12/17 TI - Sample Size Requirements for Popular Classification Algorithms in Tabular Clinical Data: Empirical Study JO - J Med Internet Res SP - e60231 VL - 26 KW - medical informatics KW - machine learning KW - sample size KW - research design KW - decision trees KW - classification algorithm KW - clinical research KW - learning-curve analysis KW - analysis KW - analyses KW - guidelines KW - ML KW - decision making KW - algorithm KW - curve analysis KW - dataset N2 - Background: The performance of a classification algorithm eventually reaches a point of diminishing returns, where the additional sample added does not improve the results. Thus, there is a need to determine an optimal sample size that maximizes performance while accounting for computational burden or budgetary concerns. Objective: This study aimed to determine optimal sample sizes and the relationships between sample size and dataset-level characteristics over a variety of binary classification algorithms. Methods: A total of 16 large open-source datasets were collected, each containing a binary clinical outcome. Furthermore, 4 machine learning algorithms were assessed: XGBoost (XGB), random forest (RF), logistic regression (LR), and neural networks (NNs). For each dataset, the cross-validated area under the curve (AUC) was calculated at increasing sample sizes, and learning curves were fit. Sample sizes needed to reach the observed full?dataset AUC minus 2 points (0.02) were calculated from the fitted learning curves and compared across the datasets and algorithms. Dataset?level characteristics, minority class proportion, full?dataset AUC, number of features, type of features, and degree of nonlinearity were examined. Negative binomial regression models were used to quantify relationships between these characteristics and expected sample sizes within each algorithm. A total of 4 multivariable models were constructed, which selected the best-fitting combination of dataset?level characteristics. Results: Among the 16 datasets (full-dataset sample sizes ranging from 70,000-1,000,000), median sample sizes were 9960 (XGB), 3404 (RF), 696 (LR), and 12,298 (NN) to reach AUC stability. For all 4 algorithms, more balanced classes (multiplier: 0.93-0.96 for a 1% increase in minority class proportion) were associated with decreased sample size. Other characteristics varied in importance across algorithms?in general, more features, weaker features, and more complex relationships between the predictors and the response increased expected sample sizes. In multivariable analysis, the top selected predictors were minority class proportion among all 4 algorithms assessed, full?dataset AUC (XGB, RF, and NN), and dataset nonlinearity (XGB, RF, and NN). For LR, the top predictors were minority class proportion, percentage of strong linear features, and number of features. Final multivariable sample size models had high goodness-of-fit, with dataset?level predictors explaining a majority (66.5%-84.5%) of the total deviance in the data among all 4 models. Conclusions: The sample sizes needed to reach AUC stability among 4 popular classification algorithms vary by dataset and method and are associated with dataset?level characteristics that can be influenced or estimated before the start of a research study. UR - https://www.jmir.org/2024/1/e60231 UR - http://dx.doi.org/10.2196/60231 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60231 ER - TY - JOUR AU - Roos, Jonas AU - Martin, Ron AU - Kaczmarczyk, Robert PY - 2024/12/17 TI - Evaluating Bard Gemini Pro and GPT-4 Vision Against Student Performance in Medical Visual Question Answering: Comparative Case Study JO - JMIR Form Res SP - e57592 VL - 8 KW - medical education KW - visual question answering KW - image analysis KW - large language model KW - LLM KW - student KW - performance KW - comparative KW - case study KW - artificial intelligence KW - AI KW - ChatGPT KW - effectiveness KW - diagnostic KW - training KW - accuracy KW - utility KW - image-based KW - question KW - image KW - AMBOSS KW - English KW - German KW - question and answer KW - Python KW - AI in health care KW - health care N2 - Background: The rapid development of large language models (LLMs) such as OpenAI?s ChatGPT has significantly impacted medical research and education. These models have shown potential in fields ranging from radiological imaging interpretation to medical licensing examination assistance. Recently, LLMs have been enhanced with image recognition capabilities. Objective: This study aims to critically examine the effectiveness of these LLMs in medical diagnostics and training by assessing their accuracy and utility in answering image-based questions from medical licensing examinations. Methods: This study analyzed 1070 image-based multiple-choice questions from the AMBOSS learning platform, divided into 605 in English and 465 in German. Customized prompts in both languages directed the models to interpret medical images and provide the most likely diagnosis. Student performance data were obtained from AMBOSS, including metrics such as the ?student passed mean? and ?majority vote.? Statistical analysis was conducted using Python (Python Software Foundation), with key libraries for data manipulation and visualization. Results: GPT-4 1106 Vision Preview (OpenAI) outperformed Bard Gemini Pro (Google), correctly answering 56.9% (609/1070) of questions compared to Bard?s 44.6% (477/1070), a statistically significant difference (?2?=32.1, P<.001). However, GPT-4 1106 left 16.1% (172/1070) of questions unanswered, significantly higher than Bard?s 4.1% (44/1070; ?2?=83.1, P<.001). When considering only answered questions, GPT-4 1106?s accuracy increased to 67.8% (609/898), surpassing both Bard (477/1026, 46.5%; ?2?=87.7, P<.001) and the student passed mean of 63% (674/1070, SE 1.48%; ?2?=4.8, P=.03). Language-specific analysis revealed both models performed better in German than English, with GPT-4 1106 showing greater accuracy in German (282/465, 60.65% vs 327/605, 54.1%; ?2?=4.4, P=.04) and Bard Gemini Pro exhibiting a similar trend (255/465, 54.8% vs 222/605, 36.7%; ?2?=34.3, P<.001). The student majority vote achieved an overall accuracy of 94.5% (1011/1070), significantly outperforming both artificial intelligence models (GPT-4 1106: ?2?=408.5, P<.001; Bard Gemini Pro: ?2?=626.6, P<.001). Conclusions: Our study shows that GPT-4 1106 Vision Preview and Bard Gemini Pro have potential in medical visual question-answering tasks and to serve as a support for students. However, their performance varies depending on the language used, with a preference for German. They also have limitations in responding to non-English content. The accuracy rates, particularly when compared to student responses, highlight the potential of these models in medical education, yet the need for further optimization and understanding of their limitations in diverse linguistic contexts remains critical. UR - https://formative.jmir.org/2024/1/e57592 UR - http://dx.doi.org/10.2196/57592 ID - info:doi/10.2196/57592 ER - TY - JOUR AU - Dahu, M. Butros AU - Khan, Solaiman AU - Toubal, Eddine Imad AU - Alshehri, Mariam AU - Martinez-Villar, I. Carlos AU - Ogundele, B. Olabode AU - Sheets, R. Lincoln AU - Scott, J. Grant PY - 2024/12/17 TI - Geospatial Modeling of Deep Neural Visual Features for Predicting Obesity Prevalence in Missouri: Quantitative Study JO - JMIR AI SP - e64362 VL - 3 KW - geospatial modeling KW - deep convolutional neural network KW - DCNN KW - Residual Network-50 KW - ResNet-50 KW - satellite imagery KW - Moran I KW - local indicators of spatial association KW - LISA KW - spatial lag model KW - obesity rate KW - artificial intelligence KW - AI N2 - Background: The global obesity epidemic demands innovative approaches to understand its complex environmental and social determinants. Spatial technologies, such as geographic information systems, remote sensing, and spatial machine learning, offer new insights into this health issue. This study uses deep learning and spatial modeling to predict obesity rates for census tracts in Missouri. Objective: This study aims to develop a scalable method for predicting obesity prevalence using deep convolutional neural networks applied to satellite imagery and geospatial analysis, focusing on 1052 census tracts in Missouri. Methods: Our analysis followed 3 steps. First, Sentinel-2 satellite images were processed using the Residual Network-50 model to extract environmental features from 63,592 image chips (224×224 pixels). Second, these features were merged with obesity rate data from the Centers for Disease Control and Prevention for Missouri census tracts. Third, a spatial lag model was used to predict obesity rates and analyze the association between deep neural visual features and obesity prevalence. Spatial autocorrelation was used to identify clusters of obesity rates. Results: Substantial spatial clustering of obesity rates was found across Missouri, with a Moran I value of 0.68, indicating similar obesity rates among neighboring census tracts. The spatial lag model demonstrated strong predictive performance, with an R2 of 0.93 and a spatial pseudo R2 of 0.92, explaining 93% of the variation in obesity rates. Local indicators from a spatial association analysis revealed regions with distinct high and low clusters of obesity, which were visualized through choropleth maps. Conclusions: This study highlights the effectiveness of integrating deep convolutional neural networks and spatial modeling to predict obesity prevalence based on environmental features from satellite imagery. The model?s high accuracy and ability to capture spatial patterns offer valuable insights for public health interventions. Future work should expand the geographical scope and include socioeconomic data to further refine the model for broader applications in obesity research. UR - https://ai.jmir.org/2024/1/e64362 UR - http://dx.doi.org/10.2196/64362 UR - http://www.ncbi.nlm.nih.gov/pubmed/39688897 ID - info:doi/10.2196/64362 ER - TY - JOUR AU - Fernandez, Diana Isabel AU - Yang, Yu-Ching AU - Chang, Wonkyung AU - Kautz, Amber AU - Farchaus Stein, Karen PY - 2024/12/13 TI - Developing Components of an Integrated mHealth Dietary Intervention for Mexican Immigrant Farmworkers: Feasibility Usability Study of a Food Photography Protocol for Dietary Assessment JO - JMIR Form Res SP - e54664 VL - 8 KW - Mexican immigrant farmworker KW - diet-related noncommunicable diseases KW - mHealth KW - dietary assessment KW - image-based KW - healthcare disparities KW - minority KW - feasibility study KW - food photography KW - rural health KW - health literacy KW - culutural adaptation KW - women KW - technology acceptance KW - mobile health N2 - Background: Rural-urban disparities in access to health services and the burden of diet-related noncommunicable diseases are exacerbated among Mexican immigrant farmworkers due to work demands, social and geographical isolation, literacy issues, and limited access to culturally and language-competent health services. Although mobile health (mHealth) tools have the potential to overcome structural barriers to health services access, efficacious mHealth interventions to promote healthy eating have not considered issues of low literacy and health literacy, and food preferences and norms in the Mexican immigrant farmworker population. To address this critical gap, we conducted a series of preliminary studies among Mexican immigrant farmworkers with the long-term goal of developing a culture- and literacy-specific smartphone app integrating dietary assessment through food photography, diet analyses, and a non?text-based dietary intervention. Objective: This study aimed to report adherence and reactivity to a 14-day food photography dietary assessment protocol, in which Mexican immigrant farmworker women were instructed to take photos of all foods and beverages consumed. Methods: We developed a secure mobile app with an intuitive graphical user interface to collect food images. Adult Mexican immigrant farmworker women were recruited and oriented to the photography protocol. Adherence and reactivity were examined by calculating the mean number of food photos per day over time, differences between the first and second week, and differences between weekdays and weekends. The type of foods and meals photographed were compared with reported intake in three 24-hour dietary recalls. Results: In total, 16 Mexican farmworker women took a total of 1475 photos in 14 days, with a mean of 6.6 (SD 2.3) photos per day per participant. On average, participants took 1 fewer photo per day in week 2 compared with week 1 (mean 7.1, SD 2.5 in week 1 vs mean 6.1, SD 2.6 in week 2; P=.03), and there was a decrease of 0.6 photos on weekdays versus weekends (mean 6.4, SD 2.5 on weekdays vs mean 7, SD 2.7 on weekends; P=.50). Of individual food items, 71% (352/495) of foods in the photos matched foods in the recalls. Of all missing food items (n=138) and meals (n=36) in the photos, beverages (74/138, 54%), tortillas (15/138, 11%), snacks 16/36, 44%), and dinners (10/36, 28%) were the most frequently missed. Most of the meals not photographed (27/36, 75%) were in the second week of the protocol. Conclusions: Dietary assessment through food photography is feasible among Mexican immigrant farmworker women. For future protocols, substantive adjustments will be introduced to reduce the frequency of missing foods and meals. Our preliminary studies are a step in the right direction to extend the benefits of mHealth technologies to a hard-to-reach group and contribute to the prevention and control of diet-related noncommunicable diseases. UR - https://formative.jmir.org/2024/1/e54664 UR - http://dx.doi.org/10.2196/54664 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54664 ER - TY - JOUR AU - Georgescu, Livia Alexandra AU - Cummins, Nicholas AU - Molimpakis, Emilia AU - Giacomazzi, Eduardo AU - Rodrigues Marczyk, Joana AU - Goria, Stefano PY - 2024/12/12 TI - Screening for Depression and Anxiety Using a Nonverbal Working Memory Task in a Sample of Older Brazilians: Observational Study of Preliminary Artificial Intelligence Model Transferability JO - JMIR Form Res SP - e55856 VL - 8 KW - depression KW - anxiety KW - Brazil KW - machine learning KW - n-back KW - working memory KW - artificial intelligence KW - gerontology KW - older adults KW - mental health KW - AI KW - transferability KW - detection KW - screening KW - questionnaire KW - longitudinal study N2 - Background: Anxiety and depression represent prevalent yet frequently undetected mental health concerns within the older population. The challenge of identifying these conditions presents an opportunity for artificial intelligence (AI)?driven, remotely available, tools capable of screening and monitoring mental health. A critical criterion for such tools is their cultural adaptability to ensure effectiveness across diverse populations. Objective: This study aims to illustrate the preliminary transferability of two established AI models designed to detect high depression and anxiety symptom scores. The models were initially trained on data from a nonverbal working memory game (1- and 2-back tasks) in a dataset by thymia, a company that develops AI solutions for mental health and well-being assessments, encompassing over 6000 participants from the United Kingdom, United States, Mexico, Spain, and Indonesia. We seek to validate the models? performance by applying it to a new dataset comprising older Brazilian adults, thereby exploring its transferability and generalizability across different demographics and cultures. Methods: A total of 69 Brazilian participants aged 51-92 years old were recruited with the help of Laços Saúde, a company specializing in nurse-led, holistic home care. Participants received a link to the thymia dashboard every Monday and Thursday for 6 months. The dashboard had a set of activities assigned to them that would take 10-15 minutes to complete, which included a 5-minute game with two levels of the n-back tasks. Two Random Forest models trained on thymia data to classify depression and anxiety based on thresholds defined by scores of the Patient Health Questionnaire (8 items) (PHQ-8) ?10 and those of the Generalized Anxiety Disorder Assessment (7 items) (GAD-7) ?10, respectively, were subsequently tested on the Laços Saúde patient cohort. Results: The depression classification model exhibited robust performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.78, a specificity of 0.69, and a sensitivity of 0.72. The anxiety classification model showed an initial AUC of 0.63, with a specificity of 0.58 and a sensitivity of 0.64. This performance surpassed a benchmark model using only age and gender, which had AUCs of 0.47 for PHQ-8 and 0.53 for GAD-7. After recomputing the AUC scores on a cross-sectional subset of the data (the first n-back game session), we found AUCs of 0.79 for PHQ-8 and 0.76 for GAD-7. Conclusions: This study successfully demonstrates the preliminary transferability of two AI models trained on a nonverbal working memory task, one for depression and the other for anxiety classification, to a novel sample of older Brazilian adults. Future research could seek to replicate these findings in larger samples and other cultural contexts. Trial Registration: ISRCTN Registry ISRCTN90727704; https://www.isrctn.com/ISRCTN90727704 UR - https://formative.jmir.org/2024/1/e55856 UR - http://dx.doi.org/10.2196/55856 ID - info:doi/10.2196/55856 ER - TY - JOUR AU - Chen, Xiaolan AU - Zhao, Ziwei AU - Zhang, Weiyi AU - Xu, Pusheng AU - Wu, Yue AU - Xu, Mingpu AU - Gao, Le AU - Li, Yinwen AU - Shang, Xianwen AU - Shi, Danli AU - He, Mingguang PY - 2024/12/11 TI - EyeGPT for Patient Inquiries and Medical Education: Development and Validation of an Ophthalmology Large Language Model JO - J Med Internet Res SP - e60063 VL - 26 KW - large language model KW - generative pretrained transformer KW - generative artificial intelligence KW - ophthalmology KW - retrieval-augmented generation KW - medical assistant KW - EyeGPT KW - generative AI N2 - Background: Large language models (LLMs) have the potential to enhance clinical flow and improve medical education, but they encounter challenges related to specialized knowledge in ophthalmology. Objective: This study aims to enhance ophthalmic knowledge by refining a general LLM into an ophthalmology-specialized assistant for patient inquiries and medical education. Methods: We transformed Llama2 into an ophthalmology-specialized LLM, termed EyeGPT, through the following 3 strategies: prompt engineering for role-playing, fine-tuning with publicly available data sets filtered for eye-specific terminology (83,919 samples), and retrieval-augmented generation leveraging a medical database and 14 ophthalmology textbooks. The efficacy of various EyeGPT variants was evaluated by 4 board-certified ophthalmologists through comprehensive use of 120 diverse category questions in both simple and complex question-answering scenarios. The performance of the best EyeGPT model was then compared with that of the unassisted human physician group and the EyeGPT+human group. We proposed 4 metrics for assessment: accuracy, understandability, trustworthiness, and empathy. The proportion of hallucinations was also reported. Results: The best fine-tuned model significantly outperformed the original Llama2 model at providing informed advice (mean 9.30, SD 4.42 vs mean 13.79, SD 5.70; P<.001) and mitigating hallucinations (97/120, 80.8% vs 53/120, 44.2%, P<.001). Incorporating information retrieval from reliable sources, particularly ophthalmology textbooks, further improved the model's response compared with solely the best fine-tuned model (mean 13.08, SD 5.43 vs mean 15.14, SD 4.64; P=.001) and reduced hallucinations (71/120, 59.2% vs 57/120, 47.4%, P=.02). Subgroup analysis revealed that EyeGPT showed robustness across common diseases, with consistent performance across different users and domains. Among the variants, the model integrating fine-tuning and book retrieval ranked highest, closely followed by the combination of fine-tuning and the manual database, standalone fine-tuning, and pure role-playing methods. EyeGPT demonstrated competitive capabilities in understandability and empathy when compared with human ophthalmologists. With the assistance of EyeGPT, the performance of the ophthalmologist was notably enhanced. Conclusions: We pioneered and introduced EyeGPT by refining a general domain LLM and conducted a comprehensive comparison and evaluation of different strategies to develop an ophthalmology-specific assistant. Our results highlight EyeGPT?s potential to assist ophthalmologists and patients in medical settings. UR - https://www.jmir.org/2024/1/e60063 UR - http://dx.doi.org/10.2196/60063 UR - http://www.ncbi.nlm.nih.gov/pubmed/39661433 ID - info:doi/10.2196/60063 ER - TY - JOUR AU - Portillo-Van Diest, Ana AU - Mortier, Philippe AU - Ballester, Laura AU - Amigo, Franco AU - Carrasco, Paula AU - Falcó, Raquel AU - Gili, Margalida AU - Kiekens, Glenn AU - H Machancoses, Francisco AU - Piqueras, A. Jose AU - Rebagliato, Marisa AU - Roca, Miquel AU - Rodríguez-Jiménez, Tíscar AU - Alonso, Jordi AU - Vilagut, Gemma PY - 2024/12/10 TI - Ecological Momentary Assessment of Mental Health Problems Among University Students: Data Quality Evaluation Study JO - J Med Internet Res SP - e55712 VL - 26 KW - experience sampling method KW - ecological momentary assessment KW - mental health KW - university students KW - participation KW - compliance KW - reliability KW - sensitivity analysis KW - mobile phone N2 - Background: The use of ecological momentary assessment (EMA) designs has been on the rise in mental health epidemiology. However, there is a lack of knowledge of the determinants of participation in and compliance with EMA studies, reliability of measures, and underreporting of methodological details and data quality indicators. Objective: This study aims to evaluate the quality of EMA data in a large sample of university students by estimating participation rate and mean compliance, identifying predictors of individual-level participation and compliance, evaluating between- and within-person reliability of measures of negative and positive affect, and identifying potential careless responding. Methods: A total of 1259 university students were invited to participate in a 15-day EMA study on mental health problems. Logistic and Poisson regressions were used to investigate the associations between sociodemographic factors, lifetime adverse experiences, stressful events in the previous 12 months, and mental disorder screens and EMA participation and compliance. Multilevel reliability and intraclass correlation coefficients were obtained for positive and negative affect measures. Careless responders were identified based on low compliance or individual reliability coefficients. Results: Of those invited, 62.1% (782/1259) participated in the EMA study, with a mean compliance of 76.9% (SD 27.7%). Participation was higher among female individuals (odds ratio [OR] 1.41, 95% CI 1.06-1.87) and lower among those aged ?30 years (OR 0.20, 95% CI 0.08-0.43 vs those aged 18-21 years) and those who had experienced the death of a friend or family member in the previous 12 months (OR 0.73, 95% CI 0.57-0.94) or had a suicide attempt in the previous 12 months (OR 0.26, 95% CI 0.10-0.64). Compliance was particularly low among those exposed to sexual abuse before the age of 18 years (exponential of ?=0.87) or to sexual assault or rape in the previous year (exponential of ?=0.80) and among those with 12-month positive alcohol use disorder screens (exponential of ?=0.89). Between-person reliability of negative and positive affect was strong (RkRn>0.97), whereas within-person reliability was fair to moderate (Rcn>0.43). Of all answered assessments, 0.86% (291/33,626) were flagged as careless responses because the response time per item was <1 second or the participants gave the same response to all items. Of the participants, 17.5% (137/782) could be considered careless responders due to low compliance (<25/56, 45%) or very low to null individual reliability (raw Cronbach ?<0.11) for either negative or positive affect. Conclusions: Data quality assessments should be carried out in EMA studies in a standardized manner to provide robust conclusions to advance the field. Future EMA research should implement strategies to mitigate nonresponse bias as well as conduct sensitivity analyses to assess possible exclusion of careless responders. UR - https://www.jmir.org/2024/1/e55712 UR - http://dx.doi.org/10.2196/55712 UR - http://www.ncbi.nlm.nih.gov/pubmed/39657180 ID - info:doi/10.2196/55712 ER - TY - JOUR AU - Du, Jianchao AU - Ding, Junyao AU - Wu, Yuan AU - Chen, Tianyan AU - Lian, Jianqi AU - Shi, Lei AU - Zhou, Yun PY - 2024/12/9 TI - A Pathological Diagnosis Method for Fever of Unknown Origin Based on Multipath Hierarchical Classification: Model Design and Validation JO - JMIR Form Res SP - e58423 VL - 8 KW - fever of unknown origin KW - FUO KW - intelligent diagnosis KW - machine learning KW - hierarchical classification KW - feature selection KW - model design KW - validation KW - diagnostic KW - prediction model N2 - Background: Fever of unknown origin (FUO) is a significant challenge for the medical community due to its association with a wide range of diseases, the complexity of diagnosis, and the likelihood of misdiagnosis. Machine learning can extract valuable information from the extensive data of patient indicators, aiding doctors in diagnosing the underlying cause of FUO. Objective: The study aims to design a multipath hierarchical classification algorithm to diagnose FUO due to the hierarchical structure of the etiology of FUO. In addition, to improve the diagnostic performance of the model, a mechanism for feature selection is added to the model. Methods: The case data of patients with FUO admitted to the First Affiliated Hospital of Xi?an Jiaotong University between 2011 and 2020 in China were used as the dataset for model training and validation. The hierarchical structure tree was then characterized according to etiology. The structure included 3 layers, with the top layer representing the FUO, the middle layer dividing the FUO into 5 categories of etiology (bacterial infection, viral infection, other infection, autoimmune diseases, and other noninfection), and the last layer further refining them to 16 etiologies. Finally, ablation experiments were set to determine the optimal structure of the proposed method, and comparison experiments were to verify the diagnostic performance. Results: According to ablation experiments, the model achieved the best performance with an accuracy of 76.08% when the number of middle paths was 3%, and 25% of the features were selected. According to comparison experiments, the proposed model outperformed the comparison methods, both from the perspective of feature selection methods and hierarchical classification methods. Specifically, brucellosis had an accuracy of 100%, and liver abscess, viral infection, and lymphoma all had an accuracy of more than 80%. Conclusions: In this study, a novel multipath feature selection and hierarchical classification model was designed for the diagnosis of FUO and was adequately evaluated quantitatively. Despite some limitations, this model enriches the exploration of FUO in machine learning and assists physicians in their work. UR - https://formative.jmir.org/2024/1/e58423 UR - http://dx.doi.org/10.2196/58423 ID - info:doi/10.2196/58423 ER - TY - JOUR AU - Parker, N. Jayelin AU - Rager, L. Theresa AU - Burns, Jade AU - Mmeje, Okeoma PY - 2024/12/9 TI - Data Verification and Respondent Validity for a Web-Based Sexual Health Survey: Tutorial JO - JMIR Form Res SP - e56788 VL - 8 KW - sexually transmitted infections KW - adolescent and young adults KW - sexual health KW - recruitment KW - survey design KW - social media KW - data verification KW - web-based surveys KW - data integrity KW - social media advertisements KW - online advertisements KW - STI KW - STD KW - sexual health survey KW - sexually transmitted disease N2 - Background: As technology continues to shape the landscape of health research, the utilization of web-based surveys for collecting sexual health information among adolescents and young adults has become increasingly prevalent. However, this shift toward digital platforms brings forth a new set of challenges, particularly the infiltration of automated bots that can compromise data integrity and the reliability of survey results. Objective: We aimed to outline the data verification process used in our study design, which employed survey programming and data cleaning protocols. Methods: A 26-item survey was developed and programmed with several data integrity functions, including reCAPTCHA scores, RelevantID fraud and duplicate scores, verification of IP addresses, and honeypot questions. Participants aged 15?24 years were recruited via social media advertisements over 7 weeks and received a US $15 incentive after survey completion. Data verification occurred through a 2-part cleaning process, which removed responses that were incomplete, flagged as spam by Qualtrics, or from duplicate IP addresses, or those that did not meet the inclusion criteria. Final comparisons of reported age with date of birth and reported state with state inclusion criteria were performed. Participants who completed the study survey were linked to a second survey to receive their incentive. Responses without first and last names and full addresses were removed, as were those with duplicate IP addresses or the exact same longitude and latitude coordinates. Finally, IP addresses used to complete both surveys were compared, and consistent responses were eligible for an incentive. Results: Over 7 weeks, online advertisements for a web-based survey reached 1.4 million social media users. Of the 20,585 survey responses received, 4589 (22.3%) were verified. Incentives were sent to 462 participants; of these, 14 responses were duplicates and 3 contained discrepancies, resulting in a final sample of 445 responses. Conclusions: Confidential web-based surveys are an appealing method for reaching populations?particularly adolescents and young adults, who may be reluctant to disclose sensitive information to family, friends, or clinical providers. Web-based surveys are a useful tool for researchers targeting hard-to-reach populations due to the difficulty in obtaining a representative sample. However, researchers face the ongoing threat of bots and fraudulent participants in a technology-driven world, necessitating the adoption of evolving bot detection software and tailored protocols for data collection in unique contexts. UR - https://formative.jmir.org/2024/1/e56788 UR - http://dx.doi.org/10.2196/56788 ID - info:doi/10.2196/56788 ER - TY - JOUR AU - Sugiura, Ayaka AU - Saegusa, Satoshi AU - Jin, Yingzi AU - Yoshimoto, Riki AU - Smith, D. Nicholas AU - Dohi, Koji AU - Higuchi, Tadashi AU - Kozu, Tomotake PY - 2024/12/9 TI - Evaluation of RMES, an Automated Software Tool Utilizing AI, for Literature Screening with Reference to Published Systematic Reviews as Case-Studies: Development and Usability Study JO - JMIR Form Res SP - e55827 VL - 8 KW - artificial intelligence KW - automated literature screening KW - natural language processing KW - randomized controlled trials KW - Rapid Medical Evidence Synthesis KW - RMES KW - systematic reviews KW - text mining N2 - Background: Systematic reviews and meta-analyses are important to evidence-based medicine, but the information retrieval and literature screening procedures are burdensome tasks. Rapid Medical Evidence Synthesis (RMES; Deloitte Tohmatsu Risk Advisory LLC) is a software designed to support information retrieval, literature screening, and data extraction for evidence-based medicine. Objective: This study aimed to evaluate the accuracy of RMES for literature screening with reference to published systematic reviews. Methods: We used RMES to automatically screen the titles and abstracts of PubMed-indexed articles included in 12 systematic reviews across 6 medical fields, by applying 4 filters: (1) study type; (2) study type + disease; (3) study type + intervention; and (4) study type + disease + intervention. We determined the numbers of articles correctly included by each filter relative to those included by the authors of each systematic review. Only PubMed-indexed articles were assessed. Results: Across the 12 reviews, the number of articles analyzed by RMES ranged from 46 to 5612. The number of PubMed-cited articles included in the reviews ranged from 4 to 47. The median (range) percentage of articles correctly labeled by RMES using filters 1-4 were: 80.9% (57.1%-100%), 65.2% (34.1%-81.8%), 70.5% (0%-100%), and 58.6% (0%-81.8%), respectively. Conclusions: This study demonstrated good performance and accuracy of RMES for the initial screening of the titles and abstracts of articles for use in systematic reviews. RMES has the potential to reduce the workload involved in the initial screening of published studies. UR - https://formative.jmir.org/2024/1/e55827 UR - http://dx.doi.org/10.2196/55827 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55827 ER - TY - JOUR AU - Linden-Carmichael, Ashley AU - Stull, W. Samuel AU - Wang, Danny AU - Bhandari, Sandesh AU - Lanza, T. Stephanie PY - 2024/12/5 TI - Impact of Providing a Personalized Data Dashboard on Ecological Momentary Assessment Compliance Among College Students Who Use Substances: Pilot Microrandomized Trial JO - JMIR Form Res SP - e60193 VL - 8 KW - ecological momentary assessment KW - data dashboard KW - study compliance KW - substance use KW - substance use behavior KW - college student KW - alcohol KW - cannabis KW - cannabis use KW - personalized data dashboard KW - EMA protocol KW - EMA KW - health behaviors KW - survey KW - compliance KW - self-reported N2 - Background: The landscape of substance use behavior among young adults has observed rapid changes over time. Intensive longitudinal designs are ideal for examining and intervening in substance use behavior in real time but rely on high participant compliance in the study protocol, representing a significant challenge for researchers. Objective: This study aimed to evaluate the effect of including a personalized data dashboard (DD) in a text-based survey prompt on study compliance outcomes among college students participating in a 21-day ecological momentary assessment (EMA) study. Methods: Participants (N=91; 61/91, 67% female and 84/91, 92% White) were college students who engaged in recent alcohol and cannabis use. Participants were randomized to either complete a 21-day EMA protocol with 4 prompts/d (EMA Group) or complete the same EMA protocol with 1 personalized message and a DD indicating multiple metrics of progress in the study, delivered at 1 randomly selected prompt/d (EMA+DD Group) via a microrandomized design. Study compliance, completion time, self-reported protocol experiences, and qualitative responses were assessed for both groups. Results: Levels of compliance were similar across groups. Participants in the EMA+DD Group had overall faster completion times, with significant week-level differences in weeks 2 and 3 of the study (P=.047 and P=.03, respectively). Although nonsignificant, small-to-medium effect sizes were observed when comparing the groups in terms of compensation level (P=.08; Cohen w=0.19) and perceived burden (P=.09; Cohen d=-0.36). Qualitative findings revealed that EMA+DD participants perceived that seeing their progress facilitated engagement. Within the EMA+DD Group, providing a DD at the moment level did not significantly impact participants? likelihood of completing the EMA or completion time at that particular prompt (all P>.05), with the exception of the first prompt of the day (P=.01 and P<.001). Conclusions: Providing a DD may be useful to increase engagement, particularly for researchers aiming to assess health behaviors shortly after a survey prompt is deployed to participants? mobile devices. International Registered Report Identifier (IRRID): RR2-10.2196/57664 UR - https://formative.jmir.org/2024/1/e60193 UR - http://dx.doi.org/10.2196/60193 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60193 ER - TY - JOUR AU - Bouchacourt, Lindsay AU - Smith, Sarah AU - Mackert, Michael AU - Almalki, Shoaa AU - Awad, Germine AU - Barczyk, Amanda AU - Bearman, Kate Sarah AU - Castelli, Darla AU - Champagne, Frances AU - de Barbaro, Kaya AU - Garcia, Shirene AU - Johnson, Karen AU - Kinney, Kerry AU - Lawson, Karla AU - Nagy, Zoltan AU - Quiñones Camacho, Laura AU - Rodríguez, Lourdes AU - Schnyer, David AU - Thomaz, Edison AU - Upshaw, Sean AU - Zhang, Yan PY - 2024/12/5 TI - Strategies to Implement a Community-Based, Longitudinal Cohort Study: The Whole Communities-Whole Health Case Study JO - JMIR Form Res SP - e60368 VL - 8 KW - community-based KW - longitudinal KW - health disparities KW - cohort study KW - case study KW - family health KW - child KW - children KW - families KW - child development KW - mobile phone UR - https://formative.jmir.org/2024/1/e60368 UR - http://dx.doi.org/10.2196/60368 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60368 ER - TY - JOUR AU - Harakeh, Zeena AU - de Hoogh, Iris AU - Krijger-Dijkema, Anne-Margreeth AU - Berbée, Susanne AU - Kalkman, Gino AU - van Empelen, Pepijn AU - Otten, Wilma PY - 2024/12/4 TI - A 360° Approach to Personalize Lifestyle Treatment in Primary Care for People With Type 2 Diabetes: Feasibility Study JO - JMIR Form Res SP - e57312 VL - 8 KW - type 2 diabetes KW - diagnostic tool KW - holistic approach KW - personalized treatment KW - shared decision-making KW - health professionals KW - intervention KW - feasibility study KW - primary care N2 - Background: Given the multifactorial nature of type 2 diabetes (T2D), health care for this condition would benefit from a holistic approach and multidisciplinary consultation. To address this, we developed the web-based 360-degree (360°) diagnostic tool, which assesses 4 key domains: ?body? (physical health parameters), ?thinking and feeling? (eg, mental health and stress), ?behavior? (lifestyle factors), and ?environment? (eg, work and housing conditions). Objective: This work examines the acceptability, implementation, and potential effects of the 360° diagnostic tool and subsequent tailored treatment (360° approach) in a 6-month intervention and feasibility study conducted in standard primary health care settings in the Netherlands. Methods: A single-group design with baseline, 3-month, and 6-month follow-ups was used. A total of 15 people with T2D and their health care providers from 2 practices participated in a 6-month intervention, which included the 360° diagnosis, tailored treatment, and both individual and group consultations. The 360° diagnosis involved clinical measurements for the ?body? domain and self-reports for the ?thinking and feeling,? ?behavior,? and ?environment? domains. After multidisciplinary consultations involving the general practitioner, pharmacist, nurse practitioner (NP), and dietitian, the NP and dietitian provided tailored advice, lifestyle treatment, and ongoing support. At the end of the intervention, face-to-face semistructured interviews were conducted with health care professionals (n=6) and participants (n=13) to assess the acceptability and implementation of the 360° approach in primary health care. Additionally, data from 14 participants on the ?thinking and feeling? and ?behavior? domains at baseline, 3 months, and 6 months were analyzed to assess changes over time. Results: The semistructured interviews revealed that both participants with T2D and health care professionals were generally positive about various aspects of the 360° approach, including onboarding, data collection with the 360° diagnosis, consultations and advice from the NP and dietitian, the visual representation of parameters in the profile wheel, counseling during the intervention (including professional collaboration), and the group meetings. The interviews also identified factors that promoted or hindered the implementation of the 360° approach. Promoting factors included (1) the care, attention, support, and experience of professionals; (2) the multidisciplinary team; (3) social support; and (4) the experience of positive health effects. Hindering factors included (1) too much information, (2) survey-related issues, and (3) time-consuming counseling. In terms of effects over time, improvements were observed at 3 months in mental health, diabetes-related problems, and fast-food consumption. At 6 months, there was a reduction in perceived stress and fast-food consumption. Additionally, fruit intake decreased at both 3 and 6 months. Conclusions: Our findings suggest that the 360° approach is acceptable to both people with T2D and health care professionals, implementable, and potentially effective in fostering positive health changes. Overall, it appears feasible to implement the 360° approach in standard primary health care. Trial Registration: Netherlands Trial Register NL-7509/NL-OMON45788; https://onderzoekmetmensen.nl/nl/trial/45788 UR - https://formative.jmir.org/2024/1/e57312 UR - http://dx.doi.org/10.2196/57312 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57312 ER - TY - JOUR AU - Hammarberg, Karin AU - Bandyopadhyay, Mridula AU - Nguyen, Hau AU - Cicuttini, Flavia AU - Stanzel, Andrea Karin AU - Brown, Helen AU - Hickey, Martha AU - Fisher, Jane PY - 2024/12/2 TI - Development and Evaluation of 4 Short, Animated Videos for Women in Midlife Promoting Positive Health Behaviors: Survey Study JO - Interact J Med Res SP - e60949 VL - 13 KW - health promotion KW - healthy aging KW - self-management KW - midlife KW - menopause KW - internet KW - video KW - animation KW - survey KW - questionnaire KW - education KW - women KW - gynecology N2 - Background: Health and health behaviors in midlife are important determinants of healthy aging. There is evidence of unmet needs for health-promoting information for women from culturally and linguistically diverse backgrounds and women with low literacy. Objective: This study aimed to (1) develop accessible short, animated videos viewable and downloadable from YouTube aimed at promoting positive health behaviors in women in midlife and (2) evaluate their accessibility, acceptability, understanding, and usability and whether this was influenced by the level of education or socioeconomic disadvantage. Methods: In collaboration with a video production company, a multidisciplinary team of academics and health professionals developed 2 short, animated videos on self-management of menopause health and 2 promoting joint health. Their accessibility, acceptability, understanding, and usability to women were evaluated in an anonymous web-based survey. Results: A total of 490 women viewed the videos and responded to the survey. Of these, 353 (72%) completed all questions. Almost all (from 321/353, 91% to 334/363, 92%) agreed that the information in the videos was ?very easy to understand.? The proportions reporting that all or some of the information in the video was new to them varied between videos from 36% (137/386) to 66% (233/353), the reported likelihood of using the practical tips offered in the videos varied from 70% (271/386) to 89% (331/373), and between 61% (235/386) and 70% (263/373) of respondents stated that they would recommend the videos to others. Education-level group comparisons revealed few differences in opinions about the videos, except that women with lower education were more likely than those with higher education to state that they would recommend the 2 joint health videos to others (36/45, 80% vs 208/318, 65%; P=.051 for video 3; and 36/44, 80% vs 197/309, 64%; P=.04 for video 4). There were no differences between women living in the least advantaged areas (Socioeconomic Indexes for Areas quintile areas 1 and 2) and those living in the most advantaged areas (Socioeconomic Indexes for Areas quintile areas 3, 4, and 5) in their responses to any of the questions about the 4 videos. Conclusions: Most women found the videos easy to understand, learned something new from watching them, planned to use the practical tips they offered, and were likely to recommend them to other women. This suggests that short, animated videos about health self-management strategies in midlife to improve the chance of healthy aging are perceived as accessible, acceptable, easy to understand, and useful by women. UR - https://www.i-jmr.org/2024/1/e60949 UR - http://dx.doi.org/10.2196/60949 UR - http://www.ncbi.nlm.nih.gov/pubmed/39621404 ID - info:doi/10.2196/60949 ER - TY - JOUR AU - Herman, M. Patricia AU - Slaughter, E. Mary AU - Qureshi, Nabeel AU - Azzam, Tarek AU - Cella, David AU - Coulter, D. Ian AU - DiGuiseppi, Graham AU - Edelen, Orlando Maria AU - Kapteyn, Arie AU - Rodriguez, Anthony AU - Rubinstein, Max AU - Hays, D. Ron PY - 2024/11/29 TI - Comparing Health Survey Data Cost and Quality Between Amazon?s Mechanical Turk and Ipsos? KnowledgePanel: Observational Study JO - J Med Internet Res SP - e63032 VL - 26 KW - data collection KW - probability panel KW - convenience sample KW - data quality KW - weighting KW - back pain KW - misrepresentation KW - Amazon KW - Mechanical Turk KW - MTurk KW - convenience panel KW - KnowledgePanel N2 - Background: Researchers have many options for web-based survey data collection, ranging from access to curated probability-based panels, where individuals are selectively invited to join based on their membership in a representative population, to convenience panels, which are open for anyone to join. The mix of respondents available also varies greatly regarding representation of a population of interest and in motivation to provide thoughtful and accurate responses. Despite the additional dataset-building labor required of the researcher, convenience panels are much less expensive than probability-based panels. However, it is important to understand what may be given up regarding data quality for those cost savings. Objective: This study examined the relative costs and data quality of fielding equivalent surveys on Amazon?s Mechanical Turk (MTurk), a convenience panel, and KnowledgePanel, a nationally representative probability-based panel. Methods: We administered the same survey measures to MTurk (in 2021) and KnowledgePanel (in 2022) members. We applied several recommended quality assurance steps to enhance the data quality achieved using MTurk. Ipsos, the owner of KnowledgePanel, followed their usual (industry standard) protocols. The survey was designed to support psychometric analyses and included >60 items from the Patient-Reported Outcomes Measurement Information System (PROMIS), demographics, and a list of health conditions. We used 2 fake conditions (?syndomitis? and ?chekalism?) to identify those more likely to be honest respondents. We examined the quality of each platform?s data using several recommended metrics (eg, consistency, reliability, representativeness, missing data, and correlations) including and excluding those respondents who had endorsed a fake condition and examined the impact of weighting on representativeness. Results: We found that prescreening in the MTurk sample (removing those who endorsed a fake health condition) improved data quality but KnowledgePanel data quality generally remained superior. While MTurk?s unweighted point estimates for demographics exhibited the usual mismatch with national averages (younger, better educated, and lower income), weighted MTurk data matched national estimates. KnowledgePanel?s point estimates better matched national benchmarks even before poststratification weighting. Correlations between PROMIS measures and age and income were similar in MTurk and KnowledgePanel; the mean absolute value of the difference between each platform?s 137 correlations was 0.06, and 92% were <0.15. However, correlations between PROMIS measures and educational level were dramatically different; the mean absolute value of the difference across these 17 correlation pairs was 0.15, the largest difference was 0.29, and the direction of more than half of these relationships in the MTurk sample was the opposite from that expected from theory. Therefore, caution is needed if using MTurk for studies where educational level is a key variable. Conclusions: The data quality of our MTurk sample was often inferior to that of the KnowledgePanel sample but possibly not so much as to negate the benefits of its cost savings for some uses. International Registered Report Identifier (IRRID): RR2-10.1186/s12891-020-03696-2 UR - https://www.jmir.org/2024/1/e63032 UR - http://dx.doi.org/10.2196/63032 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/63032 ER - TY - JOUR AU - Cerrito, Brianna AU - Xiao, Jamie AU - Fialk, Amanda AU - Buono, D. Frank PY - 2024/11/29 TI - Therapy Mode Preference Scale: Preliminary Validation Methodological Design JO - JMIR Form Res SP - e65477 VL - 8 KW - virtual teletherapy KW - young adult mental health treatment KW - in-person therapy KW - virtual mental health care KW - telehealth KW - mental health treatment KW - virtual care KW - therapeutic KW - virtual therapy KW - in-person treatment KW - exploratory factor analysis KW - hierarchical linear regression KW - standardized tool KW - herapeutic impact N2 - Background: The use of tele?mental health care increased rapidly in 2020 as a critical response to the COVID-19 pandemic, serving as an effective contact-free alternative to treatment. Today, tele?mental health care remains a viable option for individuals with geographic and physical barriers to treatment. However, there are several potential therapeutic disadvantages to tele?mental health care (ie, missing nonverbal signals, handling crises, confidentiality, weakened social connection in group therapy) that should be evaluated. While published literature has explored client satisfaction within teletherapy and the effect of using technology for tele?mental health care demands, there is a need for published surveys that evaluate the therapeutic experience in teletherapy and in-person mediums of care. Objective: The authors of this study sought to develop and validate a survey that could evaluate the comparative impact of teletherapy and in-person care from a therapeutic perspective across key factors (ie, therapeutic alliance, engagement, rapport, and confidentiality). Methods: Participants were clients who experienced both tele?mental health care and in-person therapy at an intensive outpatient mental health treatment program for young adults from April 2020 through June 2022. Generated items on the survey were formulated based on input from experts in the field and existing validated scales. All individuals completed the survey on the internet, following informed consent (n=89). An exploratory factor analysis was conducted to understand factor structure, and Cronbach ? was used to determine internal consistency. Incremental validity was demonstrated through a hierarchical linear regression. Results: The exploratory factor analysis revealed a 14-item, 3-factor structure. All 14 items correlated at a minimum of 0.30 with at least one other item. Kaiser-Meyer-Olkin measure of sampling adequacy was 0.75 and Bartlett?s test of sphericity was significant (?291=528.41, P<.001). In total, 3 factors accounted for 61% of the variance, and the preliminary Cronbach ? (?=0.71) indicates a satisfactory level of internal consistency. The Zoom Exhaustion and Fatigue Scale (ZEF) and Client Satisfaction Questionnaire (CSQ; ?0.29) were significantly correlated, as well as the ZEF and Therapy Mode Preference Scale (TMPS; ?0.31), and CSQ and TMPS (0.50; P<.001). Hierarchical linear regression revealed that the CSQ significantly accounted for additional variance in the TMPS (P<.001). With the ZEF entered into the model, no further variance was accounted for (P=.06). Conclusions: Continual research is warranted to expand the current findings by validating this standardized tool for assessing the therapeutic impact of teletherapy versus in-person care in a generalizable population. UR - https://formative.jmir.org/2024/1/e65477 UR - http://dx.doi.org/10.2196/65477 ID - info:doi/10.2196/65477 ER - TY - JOUR AU - Wang, Ruijing AU - Rezaeian, Olya AU - Asan, Onur AU - Zhang, Linghan AU - Liao, Ting PY - 2024/11/27 TI - Relationship Between Heart Rate and Perceived Stress in Intensive Care Unit Residents: Exploratory Analysis Using Fitbit Data JO - JMIR Form Res SP - e60759 VL - 8 KW - stress KW - perceived stress KW - heart rate KW - Fitbit KW - wearable KW - provider KW - occupational health KW - resident KW - trainee KW - physician KW - health care worker KW - intensive care unit KW - secondary data analysis KW - mental health KW - self-reported N2 - Background: Intensive care unit (ICU) residents are exposed to high stress levels due to the intense nature of their work, which can impact their mental health and job performance. Heart rate measured through wearable devices has the potential to provide insights into residents? self-reported stress and aid in developing targeted interventions. Objective: This exploratory study aims to analyze continuous heart rate data and self-reported stress levels and stressors in ICU residents to examine correlations between physiological responses, stress levels, and daily stressors reported. Methods: A secondary data analysis was conducted on heart rate measurements and stress assessments collected from 57 ICU residents over a 3-week period using Fitbit Charge 3 devices. These devices captured continuous physiological data alongside daily surveys that assessed stress levels and identified stressors. The study used Spearman rank correlation, point-biserial correlation analysis, 2-tailed paired t tests, and mixed-effect models to analyze the relationship between heart rate features and stress indicators. Results: The findings reveal complex interactions between stress levels and heart rate patterns. The correlation analysis between stress levels and median heart rate values across different percentile ranges showed that lower percentile heart rates (bottom 5%, 10%, 25%, and 50%) had modest correlations with stress, whereas higher percentiles (top 50%, 25%, 10%, and 5%) did not correlate significantly (all P>.05). The 2-tailed paired t test indicated significant differences in stress levels reported in midday versus end-of-day surveys (P<.001), although these changes in stress levels were not consistently reflected in heart rate patterns. Additionally, we explored and found that stressors related to ?other health? issues had the highest positive correlation with stress level changes from midday to end-of-day surveys. However, the weak effect of these stressors on peak heart rate suggests that their impact on physiological measures like heart rate is not yet clear. According to our mixed-effects model, stress levels significantly influenced heart rate variations when hierarchical data were taken into account (P=.03), meaning that as the stress level increased, there was a significant increase in mean heart rate. Conclusions: This study highlights the complexity of using heart rate as an indicator of stress, particularly in high-stress environments like the ICU. Our findings suggest that while heart rate is found to correlate with self-reported stress in the mixed-effect model, its impact is modest, and it should be combined with other physiological and psychological measures to obtain a more accurate and comprehensive assessment of residents? stress levels. UR - https://formative.jmir.org/2024/1/e60759 UR - http://dx.doi.org/10.2196/60759 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60759 ER - TY - JOUR AU - Nagino, Ken AU - Sung, Jaemyoung AU - Midorikawa-Inomata, Akie AU - Akasaki, Yasutsugu AU - Adachi, Takeya AU - Ebihara, Nobuyuki AU - Fukuda, Ken AU - Fukushima, Atsuki AU - Fujio, Kenta AU - Okumura, Yuichi AU - Eguchi, Atsuko AU - Fujimoto, Keiichi AU - Shokirova, Hurramhon AU - Yee, Alan AU - Morooka, Yuki AU - Huang, Tianxiang AU - Hirosawa, Kunihiko AU - Nakao, Shintaro AU - Kobayashi, Hiroyuki AU - Inomata, Takenori PY - 2024/11/26 TI - Minimal Clinically Important Differences With the Outcomes of the App-Based Japanese Allergic Conjunctival Diseases Quality of Life Questionnaire: Cross-Sectional Observational Study JO - JMIR Form Res SP - e60731 VL - 8 KW - allergic conjunctivitis KW - hay fever KW - Japanese Allergic Conjunctival Disease Quality of Life Questionnaire KW - minimal clinically important difference KW - pollinosis KW - telemedicine KW - mobile phone N2 - Background: Assessing changes in quality of life in patients with hay fever?related allergic conjunctivitis requires validated and clinically meaningful metrics. A minimal clinically important difference (MCID) that can be applied to assess Domain II of the Japanese Allergic Conjunctival Disease Quality of Life Questionnaire (JACQLQ) in a smartphone app setting has yet to be determined. Objective: This cross-sectional observational study aimed to determine MCIDs for the app-based JACQLQ in assessing hay fever?related allergic conjunctivitis. Methods: This study used data from a crowdsourced, cross-sectional, observational study conducted via the smartphone app ?AllerSearch? between February 1, 2018, and May 1, 2020. Participants were recruited through digital media and social networking platforms and voluntarily provided electronic informed consent. Participants completed the JACQLQ, which includes items on daily activity and psychological well-being, as well as a visual analog scale to measure stress levels related to hay fever. Data were collected through the app, ensuring comprehensive user input. MCIDs were determined using both anchor- and distribution-based methods. The face scale of the JACQLQ Domain III and stress level scale for hay fever were used as anchors to estimate the MCID; ranges were derived from these MCID estimates. In the distribution-based method, MCIDs were calculated using half the SD and SE of the JACQLQ Domain II scores. SEs were derived from the intraclass correlation coefficient of an app-based JACQLQ test-retest reliability metric. Results: A total of 17,597 individuals were identified, of which 15,749 individuals provided electronic consent. After excluding those with incomplete data, 7590 participants with hay fever were included in the study (mean age 35.3, SD 13.9 years; n=4331, 57.1% of women). MCID ranges calculated using the anchor-based method were 1.0-6.9, 1.2-5.6, and 2.1-12.6 for daily activity, psychological well-being, and total JACQLQ Domain II scores, respectively. Using the distribution-based method, the intraclass correlation coefficients were odds ratio (OR) 0.813 (95% CI 0.769-0.849) for daily activity, OR 0.791 (95% CI 0.743-0.832) for psychological well-being, and OR 0.841 (95% CI 0.791-0.864) for total JACQLQ Domain II scores. In addition, the distribution-based method resulted in 2 MCIDs based on half the SD and SE of measurement for daily activity (4.8 and 4.2), psychological well-being (3.4 and 3.1), and total JACQLQ Domain II (7.8 and 6.4) scores. The final suggested MCID ranges for daily activity, psychological well-being, and total JACQLQ Domain II scores were 4.2-6.0, 3.1-4.7, and 6.4-10.5, respectively. Conclusions: MCID ranges for the JACQLQ estimation could help to standardize the app-based quality of life assessment for patients with hay fever?related allergic conjunctivitis. These MCIDs enhanced the precision of remote symptom monitoring and facilitated timely, data-driven interventions, ultimately improving the overall management and outcomes of allergic conjunctivitis through mobile health platforms. UR - https://formative.jmir.org/2024/1/e60731 UR - http://dx.doi.org/10.2196/60731 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60731 ER - TY - JOUR AU - Zhou, Weiqiang AU - Liu, Dongliang AU - Yi, Zhaoxu AU - Lei, Yang AU - Zhang, Zhenming AU - Deng, Yu AU - Tan, Ying PY - 2024/11/22 TI - Web-Based Platform for Systematic Reviews and Meta-Analyses of Traditional Chinese Medicine: Platform Development Study JO - JMIR Form Res SP - e49328 VL - 8 KW - evidence-based medicine KW - information science KW - medical librarian KW - web development KW - web design KW - meta-analysis KW - traditional Chinese medicine KW - systematic review KW - review methodology KW - Chinese medicine KW - traditional medicine N2 - Background: There are many problems associated with systematic reviews of traditional Chinese medicine (TCM), such as considering ?integrated traditional Chinese and Western medicine? or treatment methods as intervention measures without considering the differences in drug use, disregarding dosage and courses of treatment, disregarding interindividual differences in control groups, etc. Classifying a large number of heterogeneous intervention measures into the same measure is easy but results in inaccurate results. In April 2023, Cochrane launched RevMan Web to digitalize systematic reviews and meta-analyses. We believe that this web-based working model helps solve the abovementioned problems. Objective: This study aims to (1) develop a web-based platform that is more suitable for systematic review and meta-analysis of TCM and (2) explore the characteristics and future development directions of this work through the testing of digital workflow. Methods: We developed TCMeta (Traditional Chinese Medicine Meta-analysis)?a platform focused on systematic reviews of TCM types. All systematic review?related work can be completed on the web, including creating topics, writing protocols, arranging personnel, obtaining literature, screening literature, inputting and analyzing data, and designing illustrations. The platform was developed using the latest internet technology and can be continuously modified and updated based on user feedback. When screening the literature on the platform, in addition to the traditional manual screening mode, the platform also creatively provides a query mode where users input keywords and click on Search to find literature with the same characteristics; this better reflects the objectivity of the screening with higher efficiency. Productivity can be improved by analyzing data and generating graphs digitally. Results: We used some test data in TCMeta to simulate data processing in a systematic review. In the literature screening stage, researchers could rapidly screen 19 sources of literature from among multiple sources with the manual screening mode. This traditional method could result in bias due to differences in the researchers? cognitive levels. The query mode is much more complex and involves inputting of data regarding drug compatibility, dosage, syndrome type, etc; different query methods can yield very different results, thus increasing the stringency of screening. We integrated data analysis tools on the platform and used third-party software to generate graphs. Conclusions: TCMeta has shown great potential in improving the quality of systematic reviews of TCM types in simulation tests. Several indicators show that this web-based mode of working is superior to the traditional way. Future research is required to focus on validating and refining the performance of TCMeta, emphasizing the ability to handle complex data. The system has good scalability and adaptability, and it has the potential to have a positive impact on the field of evidence-based medicine. UR - https://formative.jmir.org/2024/1/e49328 UR - http://dx.doi.org/10.2196/49328 ID - info:doi/10.2196/49328 ER - TY - JOUR AU - Chua, Chien Mei AU - Hadimaja, Matthew AU - Wong, Jill AU - Mukherjee, Subhra Sankha AU - Foussat, Agathe AU - Chan, Daniel AU - Nandal, Umesh AU - Yap, Fabian PY - 2024/11/22 TI - Exploring the Use of a Length AI Algorithm to Estimate Children?s Length from Smartphone Images in a Real-World Setting: Algorithm Development and Usability Study JO - JMIR Pediatr Parent SP - e59564 VL - 7 KW - computer vision KW - length estimation KW - artificial intelligence KW - smartphone images KW - children KW - AI KW - algorithm KW - imaging KW - height KW - length KW - measure KW - pediatric KW - infant KW - neonatal KW - newborn KW - smartphone KW - mHealth KW - mobile health KW - mobile phone N2 - Background: Length measurement in young children younger than 18 months is important for monitoring growth and development. Accurate length measurement requires proper equipment, standardized methods, and trained personnel. In addition, length measurement requires young children?s cooperation, making it particularly challenging during infancy and toddlerhood. Objective: This study aimed to develop a length artificial intelligence (LAI) algorithm to aid users in determining recumbent length conveniently from smartphone images and explore its performance and suitability for personal and clinical use. Methods: This proof-of-concept study in healthy children (aged 0-18 months) was performed at KK Women?s and Children?s Hospital, Singapore, from November 2021 to March 2022. Smartphone images were taken by parents and investigators. Standardized length-board measurements were taken by trained investigators. Performance was evaluated by comparing the tool?s image-based length estimations with length-board measurements (bias [mean error, mean difference between measured and predicted length]; absolute error [magnitude of error]). Prediction performance was evaluated on an individual-image basis and participant-averaged basis. User experience was collected through questionnaires. Results: A total of 215 participants (median age 4.4, IQR 1.9-9.7 months) were included. The tool produced a length prediction for 99.4% (2211/2224) of photos analyzed. The mean absolute error was 2.47 cm for individual image predictions and 1.77 cm for participant-averaged predictions. Investigators and parents reported no difficulties in capturing the required photos for most participants (182/215, 84.7% participants and 144/200, 72% participants, respectively). Conclusions: The LAI algorithm is an accessible and novel way of estimating children?s length from smartphone images without the need for specialized equipment or trained personnel. The LAI algorithm?s current performance and ease of use suggest its potential for use by parents or caregivers with an accuracy approaching what is typically achieved in general clinics or community health settings. The results show that the algorithm is acceptable for use in a personal setting, serving as a proof of concept for use in clinical settings. Trial Registration: ClinicalTrials.gov NCT05079776; https://clinicaltrials.gov/ct2/show/NCT05079776 UR - https://pediatrics.jmir.org/2024/1/e59564 UR - http://dx.doi.org/10.2196/59564 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59564 ER - TY - JOUR AU - Meng, Jian AU - Niu, Xiaoyu AU - Luo, Can AU - Chen, Yueyue AU - Li, Qiao AU - Wei, Dongmei PY - 2024/11/22 TI - Development and Validation of a Machine Learning?Based Early Warning Model for Lichenoid Vulvar Disease: Prediction Model Development Study JO - J Med Internet Res SP - e55734 VL - 26 KW - female KW - lichenoid vulvar disease KW - risk factors KW - evidence-based medicine KW - early warning model N2 - Background: Given the complexity and diversity of lichenoid vulvar disease (LVD) risk factors, it is crucial to actively explore these factors and construct personalized warning models using relevant clinical variables to assess disease risk in patients. Yet, to date, there has been insufficient research, both nationwide and internationally, on risk factors and warning models for LVD. In light of these gaps, this study represents the first systematic exploration of the risk factors associated with LVD. Objective: The risk factors of LVD in women were explored and a medically evidence-based warning model was constructed to provide an early alert tool for the high-risk target population. The model can be applied in the clinic to identify high-risk patients and evaluate its accuracy and practicality in predicting LVD in women. Simultaneously, it can also enhance the diagnostic and treatment proficiency of medical personnel in primary community health service centers, which is of great significance in reducing overall health care spending and disease burden. Methods: A total of 2990 patients who attended West China Second Hospital of Sichuan University from January 2013 to December 2017 were selected as the study candidates and were divided into 1218 cases in the normal vulvovagina group (group 0) and 1772 cases in the lichenoid vulvar disease group (group 1) according to the results of the case examination. We investigated and collected routine examination data from patients for intergroup comparisons, included factors with significant differences in multifactorial analysis, and constructed logistic regression, random forests, gradient boosting machine (GBM), adaboost, eXtreme Gradient Boosting, and Categorical Boosting analysis models. The predictive efficacy of these six models was evaluated using receiver operating characteristic curve and area under the curve. Results: Univariate analysis revealed that vaginitis, urinary incontinence, humidity of the long-term residential environment, spicy dietary habits, regular intake of coffee or caffeinated beverages, daily sleep duration, diabetes mellitus, smoking history, presence of autoimmune diseases, menopausal status, and hypertension were all significant risk factors affecting female LVD. Furthermore, the area under the receiver operating characteristic curve, accuracy, sensitivity, and F1-score of the GBM warning model were notably higher than the other 5 predictive analysis models. The GBM analysis model indicated that menopausal status had the strongest impact on female LVD, showing a positive correlation, followed by the presence of autoimmune diseases, which also displayed a positive dependency. Conclusions: In accordance with evidence-based medicine, the construction of a predictive warning model for female LVD can be used to identify high-risk populations at an early stage, aiding in the formulation of effective preventive measures, which is of paramount importance for reducing the incidence of LVD in women. UR - https://www.jmir.org/2024/1/e55734 UR - http://dx.doi.org/10.2196/55734 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55734 ER - TY - JOUR AU - Lukmanjaya, Wilson AU - Butler, Tony AU - Taflan, Patricia AU - Simpson, Paul AU - Ginnivan, Natasha AU - Buchan, Iain AU - Nenadic, Goran AU - Karystianis, George PY - 2024/11/22 TI - Population Characteristics in Justice Health Research Based on PubMed Abstracts From 1963 to 2023: Text Mining Study JO - JMIR Form Res SP - e60878 VL - 8 KW - epidemiology KW - PubMed KW - criminology KW - text mining KW - justice health KW - offending and incarcerated populations KW - population characteristics KW - open research KW - health research KW - text mining study KW - epidemiological criminology KW - public health KW - justice systems KW - bias KW - population KW - men KW - women KW - prison KW - prisoner KW - researcher N2 - Background: The field of epidemiological criminology (or justice health research) has emerged in the past decade, studying the intersection between the public health and justice systems. To ensure research efforts are focused and equitable, it is important to reflect on the outputs in this area and address knowledge gaps. Objective: This study aimed to examine the characteristics of populations researched in a large sample of published outputs and identify research gaps and biases. Methods: A rule-based, text mining method was applied to 34,481 PubMed abstracts published from 1963 to 2023 to identify 4 population characteristics (sex, age, offender type, and nationality). Results: We evaluated our method in a random sample of 100 PubMed abstracts. Microprecision was 94.3%, with microrecall at 85.9% and micro?F1-score at 89.9% across the 4 characteristics. Half (n=17,039, 49.4%) of the 34,481 abstracts did not have any characteristic mentions and only 1.3% (n=443) reported sex, age, offender type, and nationality. From the 5170 (14.9%) abstracts that reported age, 3581 (69.3%) mentioned young people (younger than 18 years) and 3037 (58.7%) mentioned adults. Since 1990, studies reporting female-only populations increased, and in 2023, these accounted for almost half (105/216, 48.6%) of the research outputs, as opposed to 33.3% (72/216) for male-only populations. Nordic countries (Sweden, Norway, Finland, and Denmark) had the highest number of abstracts proportional to their incarcerated populations. Offenders with mental illness were the most common group of interest (840/4814, 17.4%), with an increase from 1990 onward. Conclusions: Research reporting on female populations increased, surpassing that involving male individuals, despite female individuals representing 5% of the incarcerated population; this suggests that male prisoners are underresearched. Although calls have been made for the justice health area to focus more on young people, our results showed that among the abstracts reporting age, most mentioned a population aged <18 years, reflecting a rise of youth involvement in the youth justice system. Those convicted of sex offenses and crimes relating to children were not as researched as the existing literature suggests, with a focus instead on populations with mental illness, whose rates rose steadily in the last 30 years. After adjusting for the size of the incarcerated population, Nordic countries have conducted proportionately the most research. Our findings highlight that despite the presence of several research reporting guidelines, justice health abstracts still do not adequately describe the investigated populations. Our study offers new insights in the field of justice health with implications for promoting diversity in the selection of research participants. UR - https://formative.jmir.org/2024/1/e60878 UR - http://dx.doi.org/10.2196/60878 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60878 ER - TY - JOUR AU - Afolabi, Aliyyat AU - Cheung, Elaine AU - Lyu, Chen Joanne AU - Ling, M. Pamela PY - 2024/11/22 TI - Short-Form Video Informed Consent Compared With Written Consent for Adolescents and Young Adults: Randomized Experiment JO - JMIR Form Res SP - e57747 VL - 8 KW - health communication KW - video informed consent KW - randomized experiment KW - informed consent KW - adolescent KW - video KW - consent KW - e-cigarette KW - vaping KW - health research KW - social media KW - vaping cessation KW - smoking cessation N2 - Background: Adolescents and young adults have the highest prevalence of e-cigarette use (?vaping?), but they are difficult to enroll in health research studies. Previous studies have found that video consent can improve comprehension and make informed consent procedures more accessible, but the videos in previous studies are much longer than videos on contemporary social media platforms that are popular among young people. Objective: This study aimed to examine the effectiveness of a short-form (90-second) video consent compared with a standard written consent for a vaping cessation study for adolescents and young adults. Methods: We conducted a web-based experiment with 435 adolescents and young adults (aged 13-24 years) recruited by a web-based survey research provider. Each participant was randomly assigned to view either a short-form video consent or a written consent form describing a behavioral study of a social media?based vaping cessation program. Participants completed a postexposure survey measuring three outcomes: (1) comprehension of the consent information, (2) satisfaction with the consent process, and (3) willingness to participate in the described study. Independent sample 2-tailed t tests and chi-square tests were conducted to compare the outcomes between the 2 groups. Results: In total, 435 cases comprised the final analytic sample (video: n=215, 49.4%; written: n=220, 50.6%). There was no significant difference in characteristics between the 2 groups (all P>.05). Participants who watched the short-form video completed the consent review and postconsent survey process in less time (average 4.5 minutes) than those in the written consent group (5.1 minutes). A total of 83.2% (179/215) of the participants in the video consent condition reported satisfaction with the overall consent process compared with 76.3% (168/220) in the written consent condition (P=.047). There was no difference in the ability to complete consent unassisted and satisfaction with the amount of time between study conditions. There was no difference in the composite measure of overall comprehension, although in individual measures, participants who watched the short-form video consent performed better in 4 measures of comprehension about risk, privacy, and procedures, while participants who read the written document consent had better comprehension of 2 measures of study procedures. There was no difference between the groups in willingness to participate in the described study. Conclusions: Short-form informed consent videos had similar comprehension and satisfaction with the consent procedure among adolescents and young adults. Short-form informed consent videos may be a feasible and acceptable alternative to the standard written consent process, although video and written consent forms have different strengths with respect to comprehension. Because they match how young people consume media, short-form videos may be particularly well suited for adolescents and young adults participating in research. UR - https://formative.jmir.org/2024/1/e57747 UR - http://dx.doi.org/10.2196/57747 UR - http://www.ncbi.nlm.nih.gov/pubmed/39576682 ID - info:doi/10.2196/57747 ER - TY - JOUR AU - Lee, Haeun AU - Kim, Seok AU - Moon, Hui-Woun AU - Lee, Ho-Young AU - Kim, Kwangsoo AU - Jung, Young Se AU - Yoo, Sooyoung PY - 2024/11/22 TI - Hospital Length of Stay Prediction for Planned Admissions Using Observational Medical Outcomes Partnership Common Data Model: Retrospective Study JO - J Med Internet Res SP - e59260 VL - 26 KW - length of stay KW - machine learning KW - Observational Medical Outcomes Partnership Common Data Model KW - allocation of resources KW - reproducibility of results KW - hospital KW - admission KW - retrospective study KW - prediction model KW - electronic health record KW - EHR KW - South Korea KW - logistic regression KW - algorithm KW - Shapley Additive Explanation KW - health care KW - clinical informatics N2 - Background: Accurate hospital length of stay (LoS) prediction enables efficient resource management. Conventional LoS prediction models with limited covariates and nonstandardized data have limited reproducibility when applied to the general population. Objective: In this study, we developed and validated a machine learning (ML)?based LoS prediction model for planned admissions using the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM). Methods: Retrospective patient-level prediction models used electronic health record (EHR) data converted to the OMOP CDM (version 5.3) from Seoul National University Bundang Hospital (SNUBH) in South Korea. The study included 137,437 hospital admission episodes between January 2016 and December 2020. Covariates from the patient, condition occurrence, medication, observation, measurement, procedure, and visit occurrence tables were included in the analysis. To perform feature selection, we applied Lasso regularization in the logistic regression. The primary outcome was an LoS of 7 days or longer, while the secondary outcome was an LoS of 3 days or longer. The prediction models were developed using 6 ML algorithms, with the training and test set split in a 7:3 ratio. The performance of each model was evaluated based on the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC). Shapley Additive Explanations (SHAP) analysis measured feature importance, while calibration plots assessed the reliability of the prediction models. External validation of the developed models occurred at an independent institution, the Seoul National University Hospital. Results: The final sample included 129,938 patient entry events in the planned admissions. The Extreme Gradient Boosting (XGB) model achieved the best performance in binary classification for predicting an LoS of 7 days or longer, with an AUROC of 0.891 (95% CI 0.887-0.894) and an AUPRC of 0.819 (95% CI 0.813-0.826) on the internal test set. The Light Gradient Boosting (LGB) model performed the best in the multiclassification for predicting an LoS of 3 days or more, with an AUROC of 0.901 (95% CI 0.898-0.904) and an AUPRC of 0.770 (95% CI 0.762-0.779). The most important features contributing to the models were the operation performed, frequency of previous outpatient visits, patient admission department, age, and day of admission. The RF model showed robust performance in the external validation set, achieving an AUROC of 0.804 (95% CI 0.802-0.807). Conclusions: The use of the OMOP CDM in predicting hospital LoS for planned admissions demonstrates promising predictive capabilities for stays of varying durations. It underscores the advantage of standardized data in achieving reproducible results. This approach should serve as a model for enhancing operational efficiency and patient care coordination across health care settings. UR - https://www.jmir.org/2024/1/e59260 UR - http://dx.doi.org/10.2196/59260 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59260 ER - TY - JOUR AU - Zimmermann, Jannik AU - Morf, Harriet AU - Scharf, Florian AU - Knitza, Johannes AU - Moeller, Heidi AU - Muehlensiepen, Felix AU - Nathrath, Michaela AU - Orlemann, Till AU - Voelker, Thomas AU - Deckers, Merlin PY - 2024/11/21 TI - German Version of the Telehealth Usability Questionnaire and Derived Short Questionnaires for Usability and Perceived Usefulness in Health Care Assessment in Telehealth and Digital Therapeutics: Instrument Validation Study JO - JMIR Hum Factors SP - e57771 VL - 11 KW - mHealth KW - mobile health KW - telehealth KW - usability KW - questionnaire validation KW - technology acceptance model KW - validity KW - questionnaire translation KW - Net Promoter Scale KW - NPS KW - usefulness KW - autoimmune chronic diseases KW - questionnaire KW - German KW - digital therapeutics KW - therapeutics KW - feasibility N2 - Background: The exponential growth of telehealth is revolutionizing health care delivery, but its evaluation has not matched the pace of its uptake. Various forms of assessment, from single-item to more extensive questionnaires, have been used to assess telehealth and digital therapeutics and their usability. The most frequently used questionnaire is the ?Telehealth Usability Questionnaire? (TUQ). The use of the TUQ is limited by its restricted availability in languages other than English and its feasibility. Objective: The aims of this study were to create a translated German TUQ version and to derive a short questionnaire for patients??Telehealth Usability and Perceived Usefulness Short Questionnaire for patients? (TUUSQ). Methods: As a first step, the original 21-item TUQ was forward and back-translated twice. In the second step, 13 TUQ items were selected for their suitability for the general evaluation of telehealth on the basis of expert opinion. These 13 items were surveyed between July 2022 and September 2023 in 4 studies with patients and family members of palliative care, as well as patients with chronic autoimmune diseases, evaluating 13 health care apps, including digital therapeutics and a telehealth system (n1=128, n2=220, n3=30, and n4=12). Psychometric exploratory factor analysis was conducted. Results: The analysis revealed that a parsimonious factor structure with 2 factors (?perceived usefulness in health care? and ?usability?) is sufficient to describe the patient?s perception. Consequently, the questionnaire could be shortened to 6 items without compromising its informativeness. Conclusions: We provide a linguistically precise German version of the TUQ for assessing the usability and perceived usefulness of telehealth. Beyond that, we supply a highly feasible shortened version that is versatile for general use in telehealth, mobile health, and digital therapeutics, which distinguishes between the 2 factors ?perceived usefulness in health care? and ?usability? in patients. Trial Registration: German Clinical Trials Register DRKS00030546; https://drks.de/search/de/trial/DRKS00030546 UR - https://humanfactors.jmir.org/2024/1/e57771 UR - http://dx.doi.org/10.2196/57771 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57771 ER - TY - JOUR AU - Seo, Junhyuk AU - Choi, Dasol AU - Kim, Taerim AU - Cha, Chul Won AU - Kim, Minha AU - Yoo, Haanju AU - Oh, Namkee AU - Yi, YongJin AU - Lee, Hwa Kye AU - Choi, Edward PY - 2024/11/20 TI - Evaluation Framework of Large Language Models in Medical Documentation: Development and Usability Study JO - J Med Internet Res SP - e58329 VL - 26 KW - large language models KW - health care documentation KW - clinical evaluation KW - emergency department KW - artificial intelligence KW - medical record accuracy N2 - Background: The advancement of large language models (LLMs) offers significant opportunities for health care, particularly in the generation of medical documentation. However, challenges related to ensuring the accuracy and reliability of LLM outputs, coupled with the absence of established quality standards, have raised concerns about their clinical application. Objective: This study aimed to develop and validate an evaluation framework for assessing the accuracy and clinical applicability of LLM-generated emergency department (ED) records, aiming to enhance artificial intelligence integration in health care documentation. Methods: We organized the Healthcare Prompt-a-thon, a competitive event designed to explore the capabilities of LLMs in generating accurate medical records. The event involved 52 participants who generated 33 initial ED records using HyperCLOVA X, a Korean-specialized LLM. We applied a dual evaluation approach. First, clinical evaluation: 4 medical professionals evaluated the records using a 5-point Likert scale across 5 criteria?appropriateness, accuracy, structure/format, conciseness, and clinical validity. Second, quantitative evaluation: We developed a framework to categorize and count errors in the LLM outputs, identifying 7 key error types. Statistical methods, including Pearson correlation and intraclass correlation coefficients (ICC), were used to assess consistency and agreement among evaluators. Results: The clinical evaluation demonstrated strong interrater reliability, with ICC values ranging from 0.653 to 0.887 (P<.001), and a test-retest reliability Pearson correlation coefficient of 0.776 (P<.001). Quantitative analysis revealed that invalid generation errors were the most common, constituting 35.38% of total errors, while structural malformation errors had the most significant negative impact on the clinical evaluation score (Pearson r=?0.654; P<.001). A strong negative correlation was found between the number of quantitative errors and clinical evaluation scores (Pearson r=?0.633; P<.001), indicating that higher error rates corresponded to lower clinical acceptability. Conclusions: Our research provides robust support for the reliability and clinical acceptability of the proposed evaluation framework. It underscores the framework?s potential to mitigate clinical burdens and foster the responsible integration of artificial intelligence technologies in health care, suggesting a promising direction for future research and practical applications in the field. UR - https://www.jmir.org/2024/1/e58329 UR - http://dx.doi.org/10.2196/58329 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58329 ER - TY - JOUR AU - Hirosawa, Takanobu AU - Harada, Yukinori AU - Tokumasu, Kazuki AU - Shiraishi, Tatsuya AU - Suzuki, Tomoharu AU - Shimizu, Taro PY - 2024/11/19 TI - Comparative Analysis of Diagnostic Performance: Differential Diagnosis Lists by LLaMA3 Versus LLaMA2 for Case Reports JO - JMIR Form Res SP - e64844 VL - 8 KW - artificial intelligence KW - clinical decision support system KW - generative artificial intelligence KW - large language models KW - natural language processing KW - NLP KW - AI KW - clinical decision making KW - decision support KW - decision making KW - LLM: diagnostic KW - case report KW - diagnosis KW - generative AI KW - LLaMA N2 - Background: Generative artificial intelligence (AI), particularly in the form of large language models, has rapidly developed. The LLaMA series are popular and recently updated from LLaMA2 to LLaMA3. However, the impacts of the update on diagnostic performance have not been well documented. Objective: We conducted a comparative evaluation of the diagnostic performance in differential diagnosis lists generated by LLaMA3 and LLaMA2 for case reports. Methods: We analyzed case reports published in the American Journal of Case Reports from 2022 to 2023. After excluding nondiagnostic and pediatric cases, we input the remaining cases into LLaMA3 and LLaMA2 using the same prompt and the same adjustable parameters. Diagnostic performance was defined by whether the differential diagnosis lists included the final diagnosis. Multiple physicians independently evaluated whether the final diagnosis was included in the top 10 differentials generated by LLaMA3 and LLaMA2. Results: In our comparative evaluation of the diagnostic performance between LLaMA3 and LLaMA2, we analyzed differential diagnosis lists for 392 case reports. The final diagnosis was included in the top 10 differentials generated by LLaMA3 in 79.6% (312/392) of the cases, compared to 49.7% (195/392) for LLaMA2, indicating a statistically significant improvement (P<.001). Additionally, LLaMA3 showed higher performance in including the final diagnosis in the top 5 differentials, observed in 63% (247/392) of cases, compared to LLaMA2?s 38% (149/392, P<.001). Furthermore, the top diagnosis was accurately identified by LLaMA3 in 33.9% (133/392) of cases, significantly higher than the 22.7% (89/392) achieved by LLaMA2 (P<.001). The analysis across various medical specialties revealed variations in diagnostic performance with LLaMA3 consistently outperforming LLaMA2. Conclusions: The results reveal that the LLaMA3 model significantly outperforms LLaMA2 per diagnostic performance, with a higher percentage of case reports having the final diagnosis listed within the top 10, top 5, and as the top diagnosis. Overall diagnostic performance improved almost 1.5 times from LLaMA2 to LLaMA3. These findings support the rapid development and continuous refinement of generative AI systems to enhance diagnostic processes in medicine. However, these findings should be carefully interpreted for clinical application, as generative AI, including the LLaMA series, has not been approved for medical applications such as AI-enhanced diagnostics. UR - https://formative.jmir.org/2024/1/e64844 UR - http://dx.doi.org/10.2196/64844 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/64844 ER - TY - JOUR AU - Pak, Lam Sharon Hoi AU - Wu, Chanchan AU - Choi, Ying Kitty Wai AU - Choi, Hang Edmond Pui PY - 2024/11/19 TI - Measuring Technology-Facilitated Sexual Violence and Abuse in the Chinese Context: Development Study and Content Validity Analysis JO - JMIR Form Res SP - e65199 VL - 8 KW - technology-facilitated sexual violence and abuse KW - TFSVA KW - image-based sexual abuse KW - sexual abuse KW - content validity KW - measurement KW - questionnaire KW - China N2 - Background: Technology-facilitated sexual violence and abuse (TFSVA) encompasses a range of behaviors where digital technologies are used to enable both virtual and in-person sexual violence. Given that TFSVA is an emerging and continually evolving form of sexual abuse, it has been challenging to establish a universally accepted definition or to develop standardized measures for its assessment. Objective: This study aimed to address the significant gap in research on TFSVA within the Chinese context. Specifically, it sought to develop a TFSVA measurement tool with robust content validity, tailored for use in subsequent epidemiological studies within the Chinese context. Methods: The first step in developing the measurement approach for TFSVA victimization and perpetration was to conduct a thorough literature review of existing empirical research on TFSVA and relevant measurement tools. After the initial generation of items, all the items were reviewed by an expert panel to assess the face validity. The measurement items were further reviewed by potential research participants, who were recruited through snowball sampling via online platforms. The assessment results were quantified by computing the content validity index (CVI). The participants were asked to rate each scale item in terms of its relevance, appropriateness, and clarity regarding the topic. Results: The questionnaire was reviewed by 24 lay experts, with a mean age of 27.96 years. They represented different genders and sexual orientations. The final questionnaire contained a total of 89 items. Three key domains were identified to construct the questionnaire, which included image-based sexual abuse, nonimage-based TFSVA, and online-initiated physical sexual violence. The overall scale CVI values of relevance, appropriateness, and clarity for the scale were 0.90, 0.96, and 0.97, respectively, which indicated high content validity for all the instrument items. To ensure the measurement accurately reflects the experiences of diverse demographic groups, the content validity was further analyzed by gender and sexual orientation. This analysis revealed variations in item validity among participants from different genders and sexual orientations. For instance, heterosexual male respondents showed a particularly low CVI for relevance of 0.20 in the items related to nudity, including ?male?s chest/nipples are visible? and ?the person is sexually suggestive.? This underscored the importance of an inclusive approach when developing a measurement for TFSVA. Conclusions: This study greatly advances the assessment of TFSVA by examining the content validity of our newly developed measurement. The findings revealed that our measurement tool demonstrated adequate content validity, thereby providing a strong foundation for assessing TFSVA within the Chinese context. Implementing this tool is anticipated to enhance our understanding of TFSVA and aid in the development of effective interventions to combat this form of abuse. UR - https://formative.jmir.org/2024/1/e65199 UR - http://dx.doi.org/10.2196/65199 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/65199 ER - TY - JOUR AU - Ding, Huitong AU - Gifford, Katherine AU - Shih, C. Ludy AU - Ho, Kristi AU - Rahman, Salman AU - Igwe, Akwaugo AU - Low, Spencer AU - Popp, Zachary AU - Searls, Edward AU - Li, Zexu AU - Madan, Sanskruti AU - Burk, Alexa AU - Hwang, H. Phillip AU - Anda-Duran, De Ileana AU - Kolachalama, B. Vijaya AU - Au, Rhoda AU - Lin, Honghuang PY - 2024/11/18 TI - Exploring the Perspectives of Older Adults on a Digital Brain Health Platform Using Natural Language Processing: Cohort Study JO - JMIR Form Res SP - e60453 VL - 8 KW - digital brain health KW - older adults KW - perspectives KW - semistructured interviews KW - natural language processing KW - mobile phone N2 - Background: Although digital technology represents a growing field aiming to revolutionize early Alzheimer disease risk prediction and monitoring, the perspectives of older adults on an integrated digital brain health platform have not been investigated. Objective: This study aims to understand the perspectives of older adults on a digital brain health platform by conducting semistructured interviews and analyzing their transcriptions by natural language processing. Methods: The study included 28 participants from the Boston University Alzheimer?s Disease Research Center, all of whom engaged with a digital brain health platform over an initial assessment period of 14 days. Semistructured interviews were conducted to collect data on participants? experiences with the digital brain health platform. The transcripts generated from these interviews were analyzed using natural language processing techniques. The frequency of positive and negative terms was evaluated through word count analysis. A sentiment analysis was used to measure the emotional tone and subjective perceptions of the participants toward the digital platform. Results: Word count analysis revealed a generally positive sentiment toward the digital platform, with ?like,? ?well,? and ?good? being the most frequently mentioned positive terms. However, terms such as ?problem? and ?hard? indicated certain challenges faced by participants. Sentiment analysis showed a slightly positive attitude with a median polarity score of 0.13 (IQR 0.08-0.15), ranging from ?1 (completely negative) to 1 (completely positive), and a median subjectivity score of 0.51 (IQR 0.47-0.53), ranging from 0 (completely objective) to 1 (completely subjective). These results suggested an overall positive attitude among the study cohort. Conclusions: The study highlights the importance of understanding older adults? attitudes toward digital health platforms amid the comprehensive evolution of the digitalization era. Future research should focus on refining digital solutions to meet the specific needs of older adults, fostering a more personalized approach to brain health. UR - https://formative.jmir.org/2024/1/e60453 UR - http://dx.doi.org/10.2196/60453 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60453 ER - TY - JOUR AU - Slade, Christopher AU - Benzo, M. Roberto AU - Washington, Peter PY - 2024/11/18 TI - Design Guidelines for Improving Mobile Sensing Data Collection: Prospective Mixed Methods Study JO - J Med Internet Res SP - e55694 VL - 26 KW - mobile health sensing KW - mHealth KW - active data collection KW - passive data collection KW - ecological momentary assessment KW - mobile data KW - mobile phone KW - machine learning KW - real-world setting KW - mixed method KW - college KW - student KW - user data KW - data consistency N2 - Background: Machine learning models often use passively recorded sensor data streams as inputs to train machine learning models that predict outcomes captured through ecological momentary assessments (EMA). Despite the growth of mobile data collection, challenges in obtaining proper authorization to send notifications, receive background events, and perform background tasks persist. Objective: We investigated challenges faced by mobile sensing apps in real-world settings in order to develop design guidelines. For active data, we compared 2 prompting strategies: setup prompting, where the app requests authorization during its initial run, and contextual prompting, where authorization is requested when an event or notification occurs. Additionally, we evaluated 2 passive data collection paradigms: collection during scheduled background tasks and persistent reminders that trigger passive data collection. We investigated the following research questions (RQs): (RQ1) how do setup prompting and contextual prompting affect scheduled notification delivery and the response rate of notification-initiated EMA? (RQ2) Which authorization paradigm, setup or contextual prompting, is more successful in leading users to grant authorization to receive background events? and (RQ3) Which polling-based method, persistent reminders or scheduled background tasks, completes more background sessions? Methods: We developed mobile sensing apps for iOS and Android devices and tested them through a 30-day user study asking college students (n=145) about their stress levels. Participants responded to a daily EMA question to test active data collection. The sensing apps collected background location events, polled for passive data with persistent reminders, and scheduled background tasks to test passive data collection. Results: For RQ1, setup and contextual prompting yielded no significant difference (ANOVA F1,144=0.0227; P=.88) in EMA compliance, with an average of 23.4 (SD 7.36) out of 30 assessments completed. However, qualitative analysis revealed that contextual prompting on iOS devices resulted in inconsistent notification deliveries. For RQ2, contextual prompting for background events was 55.5% (?21=4.4; P=.04) more effective in gaining authorization. For RQ3, users demonstrated resistance to installing the persistent reminder, but when installed, the persistent reminder performed 226.5% more background sessions than traditional background tasks. Conclusions: We developed design guidelines for improving mobile sensing on consumer mobile devices based on our qualitative and quantitative results. Our qualitative results demonstrated that contextual prompts on iOS devices resulted in inconsistent notification deliveries, unlike setup prompting on Android devices. We therefore recommend using setup prompting for EMA when possible. We found that contextual prompting is more efficient for authorizing background events. We therefore recommend using contextual prompting for passive sensing. Finally, we conclude that developing a persistent reminder and requiring participants to install it provides an additional way to poll for sensor and user data and could improve data collection to support adaptive interventions powered by machine learning. UR - https://www.jmir.org/2024/1/e55694 UR - http://dx.doi.org/10.2196/55694 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55694 ER - TY - JOUR AU - Cho, Na Ha AU - Jun, Joon Tae AU - Kim, Young-Hak AU - Kang, Heejun AU - Ahn, Imjin AU - Gwon, Hansle AU - Kim, Yunha AU - Seo, Jiahn AU - Choi, Heejung AU - Kim, Minkyoung AU - Han, Jiye AU - Kee, Gaeun AU - Park, Seohyun AU - Ko, Soyoung PY - 2024/11/18 TI - Task-Specific Transformer-Based Language Models in Health Care: Scoping Review JO - JMIR Med Inform SP - e49724 VL - 12 KW - transformer-based language models KW - medicine KW - health care KW - medical language model N2 - Background: Transformer-based language models have shown great potential to revolutionize health care by advancing clinical decision support, patient interaction, and disease prediction. However, despite their rapid development, the implementation of transformer-based language models in health care settings remains limited. This is partly due to the lack of a comprehensive review, which hinders a systematic understanding of their applications and limitations. Without clear guidelines and consolidated information, both researchers and physicians face difficulties in using these models effectively, resulting in inefficient research efforts and slow integration into clinical workflows. Objective: This scoping review addresses this gap by examining studies on medical transformer-based language models and categorizing them into 6 tasks: dialogue generation, question answering, summarization, text classification, sentiment analysis, and named entity recognition. Methods: We conducted a scoping review following the Cochrane scoping review protocol. A comprehensive literature search was performed across databases, including Google Scholar and PubMed, covering publications from January 2017 to September 2024. Studies involving transformer-derived models in medical tasks were included. Data were categorized into 6 key tasks. Results: Our key findings revealed both advancements and critical challenges in applying transformer-based models to health care tasks. For example, models like MedPIR involving dialogue generation show promise but face privacy and ethical concerns, while question-answering models like BioBERT improve accuracy but struggle with the complexity of medical terminology. The BioBERTSum summarization model aids clinicians by condensing medical texts but needs better handling of long sequences. Conclusions: This review attempted to provide a consolidated understanding of the role of transformer-based language models in health care and to guide future research directions. By addressing current challenges and exploring the potential for real-world applications, we envision significant improvements in health care informatics. Addressing the identified challenges and implementing proposed solutions can enable transformer-based language models to significantly improve health care delivery and patient outcomes. Our review provides valuable insights for future research and practical applications, setting the stage for transformative advancements in medical informatics. UR - https://medinform.jmir.org/2024/1/e49724 UR - http://dx.doi.org/10.2196/49724 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/49724 ER - TY - JOUR AU - Bazie, Hiwot AU - Lemma, Bekele AU - Workneh, Anteneh AU - Estifanos, Ashebir PY - 2024/11/15 TI - The Effect of Virtual Laboratories on the Academic Achievement of Undergraduate Chemistry Students: Quasi-Experimental Study JO - JMIR Form Res SP - e64476 VL - 8 KW - virtual laboratory KW - practical chemistry KW - student achievement KW - undergraduate student KW - Dilla University KW - simulation KW - chemistry education N2 - Background: Experimentation is crucial in chemistry education as it links practical experience with theoretical concepts. However, practical chemistry courses typically rely on real laboratory experiments and often face challenges such as limited resources, equipment shortages, and logistical constraints in university settings. To address these challenges, computer-based laboratories have been introduced as a potential solution, offering electronic simulations that replicate real laboratory experiences. Objective: This study examines the effect of virtual laboratories on the academic achievement of undergraduate chemistry students and evaluates their potential as a viable alternative or complement to traditional laboratory-based instruction. Methods: A quasi-experimental design was implemented to examine the cause-and-effect relationship between instructional methods and student outcomes. The study involved 60 fourth-year BSc chemistry students from Dilla University, divided into 3 groups: a real laboratory group (n=20), which performed real laboratory experiments; a virtual group (n=20), which used virtual laboratory simulations; and a lecture group (n=20), which received lecture-based instruction. Quantitative data were collected through tests administered before and after the intervention to assess academic performance. The data analysis used descriptive and inferential statistics, such as means and SDs, 1-way ANOVA, the Tukey honestly significant difference test, and independent-sample t tests (2-tailed), with a P value of .05 set for determining statistical significance. Results: Before the intervention, the results indicated no significant differences in academic achievement among the 3 groups (P=.99). However, after the intervention, notable differences were observed in student performance across the methods. The real laboratory group had the highest mean posttest score (mean 62.6, SD 10.7), followed by the virtual laboratory group (mean 55.5, SD 6.8) and the lecture-only group, which had the lowest mean score (mean 43.7, SD 11.5). ANOVA results confirmed significant differences between the groups (F2,57=18.429; P<.001). The Tukey post hoc test further revealed that the real laboratory group significantly outperformed the lecture-only group (mean difference 18.88; P<.001), while the virtual laboratory group also performed significantly better than the lecture-only group (mean difference 11.7; P=.001). However, no statistically significant difference was found between the real laboratory and virtual laboratory groups (mean difference 7.12; P=.07). In addition, gender did not significantly influence performance in the virtual laboratory group (P=.21), with no substantial difference in posttest scores between male and female students. Conclusions: These findings suggest that computer-based laboratories are a viable and effective alternative when real laboratories are unavailable, enhancing learning outcomes when compared with traditional lecture-based methods. Therefore, universities should consider integrating computer-based laboratories into their practical chemistry curricula to provide students with interactive and engaging learning experiences, especially when physical laboratories are inaccessible. UR - https://formative.jmir.org/2024/1/e64476 UR - http://dx.doi.org/10.2196/64476 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/64476 ER - TY - JOUR AU - Hudon, Alexandre AU - Beaudoin, Mélissa AU - Phraxayavong, Kingsada AU - Potvin, Stéphane AU - Dumais, Alexandre PY - 2024/11/15 TI - Exploring the Intersection of Schizophrenia, Machine Learning, and Genomics: Scoping Review JO - JMIR Bioinform Biotech SP - e62752 VL - 5 KW - schizophrenia KW - genomic data KW - machine learning KW - artificial intelligence KW - classification techniques KW - psychiatry KW - mental health KW - genomics KW - predictions KW - ML KW - psychiatric KW - synthesis KW - review methods KW - searches KW - scoping review KW - prediction models N2 - Background: An increasing body of literature highlights the integration of machine learning with genomic data in psychiatry, particularly for complex mental health disorders such as schizophrenia. These advanced techniques offer promising potential for uncovering various facets of these disorders. A comprehensive review of the current applications of machine learning in conjunction with genomic data within this context can significantly enhance our understanding of the current state of research and its future directions. Objective: This study aims to conduct a systematic scoping review of the use of machine learning algorithms with genomic data in the field of schizophrenia. Methods: To conduct a systematic scoping review, a search was performed in the electronic databases MEDLINE, Web of Science, PsycNet (PsycINFO), and Google Scholar from 2013 to 2024. Studies at the intersection of schizophrenia, genomic data, and machine learning were evaluated. Results: The literature search identified 2437 eligible articles after removing duplicates. Following abstract screening, 143 full-text articles were assessed, and 121 were subsequently excluded. Therefore, 21 studies were thoroughly assessed. Various machine learning algorithms were used in the identified studies, with support vector machines being the most common. The studies notably used genomic data to predict schizophrenia, identify schizophrenia features, discover drugs, classify schizophrenia amongst other mental health disorders, and predict the quality of life of patients. Conclusions: Several high-quality studies were identified. Yet, the application of machine learning with genomic data in the context of schizophrenia remains limited. Future research is essential to further evaluate the portability of these models and to explore their potential clinical applications. UR - https://bioinform.jmir.org/2024/1/e62752 UR - http://dx.doi.org/10.2196/62752 UR - http://www.ncbi.nlm.nih.gov/pubmed/39546776 ID - info:doi/10.2196/62752 ER - TY - JOUR AU - Johnson, K. Amy AU - Devlin, A. Samantha AU - Pyra, Maria AU - Etshokin, Eriika AU - Ducheny, Kelly AU - Friedman, E. Eleanor AU - Hirschhorn, R. Lisa AU - Haider, Sadia AU - Ridgway, P. Jessica PY - 2024/11/15 TI - Mapping Implementation Strategies to Address Barriers to Pre-Exposure Prophylaxis Use Among Women Through POWER Up (Pre-Exposure Prophylaxis Optimization Among Women to Enhance Retention and Uptake): Content Analysis JO - JMIR Form Res SP - e59800 VL - 8 KW - pre-exposure prophylaxis KW - PrEP KW - Consolidated Framework for Implementation Research KW - CFIR KW - Expert Recommendations for Implementing Change KW - ERIC KW - implementation science KW - HIV prevention KW - AIDS KW - United States KW - Black women KW - women?s health N2 - Background: Black cisgender women (hereafter referred to as ?women?) experience one of the highest incidences of HIV among all populations in the United States. Pre-exposure prophylaxis (PrEP) is an effective biomedical HIV prevention option, but uptake among women is low. Despite tailored strategies for certain populations, including men who have sex with men and transgender women, Black women are frequently overlooked in HIV prevention efforts. Strategies to increase PrEP awareness and use among Black women are needed at multiple levels (ie, community, system or clinic, provider, and individual or patient). Objective: This study aimed to identify barriers and facilitators to PrEP uptake and persistence among Black cisgender women and to map implementation strategies to identified barriers using the CFIR (Consolidated Framework for Implementation Research)-ERIC (Expert Recommendations for Implementing Change) Implementation Strategy Matching Tool. Methods: We conducted a secondary analysis of previous qualitative studies completed by a multidisciplinary team of HIV physicians, implementation scientists, and epidemiologists. Studies involved focus groups and interviews with medical providers and women at a federally qualified health center in Chicago, Illinois. Implementation science frameworks such as the CFIR were used to investigate determinants of PrEP use among Black women. In this secondary analysis, data from 45 total transcripts were analyzed. We identified barriers and facilitators to PrEP uptake and persistence among cisgender women across each CFIR domain. The CFIR-ERIC Implementation Strategy Matching Tool was used to map appropriate implementation strategies to address barriers and increase PrEP uptake among Black women. Results: Barriers to PrEP uptake were identified across the CFIR domains. Barriers included being unaware that PrEP was available (characteristics of individuals), worrying about side effects and impacts on fertility and pregnancy (intervention characteristics), and being unsure about how to pay for PrEP (outer setting). Providers identified lack of training (characteristics of individuals), need for additional clinical support for PrEP protocols (inner setting), and need for practicing discussions about PrEP with women (intervention characteristics). ERIC mapping resulted in 5 distinct implementation strategies to address barriers and improve PrEP uptake: patient education, provider training, PrEP navigation, clinical champions, and electronic medical record optimization. Conclusions: Evidence-based implementation strategies that address individual, provider, and clinic factors are needed to engage women in the PrEP care continuum. Tailoring implementation strategies to address identified barriers increases the probability of successfully improving PrEP uptake. Our results provide an overview of a comprehensive, multilevel implementation strategy (ie, ?POWER Up?) to improve PrEP uptake among women. International Registered Report Identifier (IRRID): RR2-10.1371/journal.pone.0285858 UR - https://formative.jmir.org/2024/1/e59800 UR - http://dx.doi.org/10.2196/59800 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59800 ER - TY - JOUR AU - Kang, Jin Soo AU - Oh, Hye-Kyung AU - Han, Hae-Ra PY - 2024/11/13 TI - Developing and Validating the Health Literacy Scale for Migrant Workers: Instrument Development and Validation Study JO - JMIR Public Health Surveill SP - e59293 VL - 10 KW - transients and migrants KW - psychometrics KW - scale development KW - health literacy KW - validation study KW - Rasch model N2 - Background: Research concerning health literacy among migrant workers in South Korea has been limited, especially given the lack of validated instruments and the lack of focus on the cultural diversity of migrant workers. Objective: This study aimed to develop and validate a health literacy scale for unskilled migrant workers (HLS-MW) in South Korea. Methods: We first generated a pool of potential items based on a literature review and in-depth interviews with 23 migrant workers. Subsequently, we reviewed empirical referents from the first step to select relevant medical terminologies and passages, ultimately choosing 709 words. The study team initially generated 35 items with 709 health-related terms through empirical referent reviews. After content validity testing by an expert panel, 28 items comprising 89 terms on the 2 subscales of prose and documents were selected for psychometric testing. Overall, 402 unskilled migrant workers in South Korea completed a web-based survey between August and September 2021, with 334 responses included in the final analysis. We used multiple analytic approaches, including exploratory factor analysis, Rasch analysis (item response theory), and descriptive analysis, to examine the new scale?s validity and reliability. Results: The final sample primarily included young male workers from South Asian countries. The HLS-MW yielded 2 factors: prose and documents. The item difficulty scores ranged from ?1.36 to 2.56. The scale was reduced to 13 items (10 prose and 3 document items), with the final version exhibiting good internal reliability (Kuder-Richardson index=0.88; intraclass correlation coefficient=0.94, 95% CI 0.93?0.95) and test-retest reliability (r=0.74, 95% CI 0.57?0.92). HLS-MW scores differed significantly by Korean language proficiency (F2,331=3.54, P=.004). Conclusions: The HLS-MW is a reliable and valid measure to assess health literacy among migrant workers in South Korea. Further studies are needed to test the psychometric properties of the HLS-MW in diverse migrant groups in South Korea while also establishing cutoffs to help identify those in need of health literacy support. UR - https://publichealth.jmir.org/2024/1/e59293 UR - http://dx.doi.org/10.2196/59293 ID - info:doi/10.2196/59293 ER - TY - JOUR AU - Gao, Hongxin AU - Schneider, Stefan AU - Hernandez, Raymond AU - Harris, Jenny AU - Maupin, Danny AU - Junghaenel, U. Doerte AU - Kapteyn, Arie AU - Stone, Arthur AU - Zelinski, Elizabeth AU - Meijer, Erik AU - Lee, Pey-Jiuan AU - Orriens, Bart AU - Jin, Haomiao PY - 2024/11/13 TI - Early Identification of Cognitive Impairment in Community Environments Through Modeling Subtle Inconsistencies in Questionnaire Responses: Machine Learning Model Development and Validation JO - JMIR Form Res SP - e54335 VL - 8 KW - machine learning KW - artificial intelligence KW - cognitive impairments KW - surveys and questionnaires KW - community health services KW - public health KW - early identification KW - elder care KW - dementia N2 - Background: The underdiagnosis of cognitive impairment hinders timely intervention of dementia. Health professionals working in the community play a critical role in the early detection of cognitive impairment, yet still face several challenges such as a lack of suitable tools, necessary training, and potential stigmatization. Objective: This study explored a novel application integrating psychometric methods with data science techniques to model subtle inconsistencies in questionnaire response data for early identification of cognitive impairment in community environments. Methods: This study analyzed questionnaire response data from participants aged 50 years and older in the Health and Retirement Study (waves 8-9, n=12,942). Predictors included low-quality response indices generated using the graded response model from four brief questionnaires (optimism, hopelessness, purpose in life, and life satisfaction) assessing aspects of overall well-being, a focus of health professionals in communities. The primary and supplemental predicted outcomes were current cognitive impairment derived from a validated criterion and dementia or mortality in the next ten years. Seven predictive models were trained, and the performance of these models was evaluated and compared. Results: The multilayer perceptron exhibited the best performance in predicting current cognitive impairment. In the selected four questionnaires, the area under curve values for identifying current cognitive impairment ranged from 0.63 to 0.66 and was improved to 0.71 to 0.74 when combining the low-quality response indices with age and gender for prediction. We set the threshold for assessing cognitive impairment risk in the tool based on the ratio of underdiagnosis costs to overdiagnosis costs, and a ratio of 4 was used as the default choice. Furthermore, the tool outperformed the efficiency of age or health-based screening strategies for identifying individuals at high risk for cognitive impairment, particularly in the 50- to 59-year and 60- to 69-year age groups. The tool is available on a portal website for the public to access freely. Conclusions: We developed a novel prediction tool that integrates psychometric methods with data science to facilitate ?passive or backend? cognitive impairment assessments in community settings, aiming to promote early cognitive impairment detection. This tool simplifies the cognitive impairment assessment process, making it more adaptable and reducing burdens. Our approach also presents a new perspective for using questionnaire data: leveraging, rather than dismissing, low-quality data. UR - https://formative.jmir.org/2024/1/e54335 UR - http://dx.doi.org/10.2196/54335 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54335 ER - TY - JOUR AU - Yang, Si Myung AU - Taira, Kazuya PY - 2024/11/11 TI - Predicting Prefecture-Level Well-Being Indicators in Japan Using Search Volumes in Internet Search Engines: Infodemiology Study JO - J Med Internet Res SP - e64555 VL - 26 KW - well-being KW - spatial indicator KW - infodemiology KW - search engine KW - public health KW - health policy KW - policy-making KW - Google KW - Japan N2 - Background: In recent years, the adoption of well-being indicators by national governments and international organizations has emerged as an important tool for evaluating state governance and societal progress. Traditionally, well-being has been gauged primarily through economic metrics such as gross domestic product, which fall short of capturing multifaceted well-being, including socioeconomic inequalities, life satisfaction, and health status. Current well-being indicators, including both subjective and objective measures, offer a broader evaluation but face challenges such as high survey costs and difficulties in evaluating at regional levels within countries. The emergence of web log data as an alternative source of well-being indicators offers the potential for more cost-effective, timely, and less biased assessments. Objective: This study aimed to develop a model using internet search data to predict well-being indicators at the regional level in Japan, providing policy makers with a more accessible and cost-effective tool for assessing public well-being and making informed decisions. Methods: This study used the Regional Well-Being Index (RWI) for Japan, which evaluates prefectural well-being across 47 prefectures for the years 2010, 2013, 2016, and 2019, as the outcome variable. The RWI includes a comprehensive approach integrating both subjective and objective indicators across 11 domains, including income, job, and life satisfaction. Predictor variables included z score?normalized relative search volume (RSV) data from Google Trends for words relevant to each domain. Unrelated words were excluded from the analysis to ensure relevance. The Elastic Net methodology was applied to predict RWI using RSVs, with ? balancing ridge and lasso effects and ? regulating their strengths. The model was optimized by cross-validation, determining the best mix and strength of regularization parameters to minimize prediction error. Root mean square errors (RMSE) and coefficients of determination (R2) were used to assess the model?s predictive accuracy and fit. Results: An analysis of Google Trends data yielded 275 words related to the RWI domains, and RSVs were collected for 211 words after filtering out irrelevant terms. The mean search frequencies for these words during 2010, 2013, 2016, and 2019 ranged from ?1.587 to 3.902, with SDs between 3.025 and 0.053. The best Elastic Net model (?=0.1, ?=0.906, RMSE=1.290, and R2=0.904) was built using 2010-2016 training data and 2-13 variables per domain. Applied to 2019 test data, it yielded an RMSE of 2.328 and R2 of 0.665. Conclusions: This study demonstrates the effectiveness of using internet search log data through the Elastic Net machine learning method to predict the RWI in Japanese prefectures with high accuracy, offering a rapid and cost-efficient alternative to traditional survey approaches. This study highlights the potential of this methodology to provide foundational data for evidence-based policy making aimed at enhancing well-being. UR - https://www.jmir.org/2024/1/e64555 UR - http://dx.doi.org/10.2196/64555 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/64555 ER - TY - JOUR AU - Agnello, Marie Danielle AU - Balaskas, George AU - Steiner, Artur AU - Chastin, Sebastien PY - 2024/11/11 TI - Methods Used in Co-Creation Within the Health CASCADE Co-Creation Database and Gray Literature: Systematic Methods Overview JO - Interact J Med Res SP - e59772 VL - 13 KW - co-creation KW - coproduction KW - co-design KW - methods KW - participatory KW - inventory KW - text mining KW - methodology KW - research methods KW - CASCADE N2 - Background: Co-creation is increasingly recognized for its potential to generate innovative solutions, particularly in addressing complex and wicked problems in public health. Despite this growing recognition, there are no standards or recommendations for method use in co-creation, leading to confusion and inconsistency. While some studies have examined specific methods, a comprehensive overview is lacking, limiting the collective understanding and ability to make informed decisions about the most appropriate methods for different contexts and research objectives. Objective: This study aimed to systematically compile and analyze methods used in co-creation to enhance transparency and deepen understanding of how co-creation is practiced. Methods: To enhance transparency and deepen understanding of how co-creation is practiced, this study systematically inventoried and analyzed methods used in co-creation. We conducted a systematic methods overview, applying 2 parallel processes: one within the peer-reviewed Health CASCADE Co-Creation Database and another within gray literature. An artificial intelligence?assisted recursive search strategy, coupled with a 2-step screening process, ensured that we captured relevant methods. We then extracted method names and conducted textual, comparative, and bibliometric analyses to assess the content, relationship between methods, fields of research, and the methodological underpinnings of the included sources. Results: We examined a total of 2627 academic papers and gray literature sources, with the literature primarily drawn from health sciences, medical research, and health services research. The dominant methodologies identified were co-creation, co-design, coproduction, participatory research methodologies, and public and patient involvement. From these sources, we extracted and analyzed 956 co-creation methods, noting that only 10% (n=97) of the methods overlap between academic and gray literature. Notably, 91.3% (230/252) of the methods in academic literature co-occurred, often involving combinations of multiple qualitative methods. The most frequently used methods in academic literature included surveys, focus groups, photo voice, and group discussion, whereas gray literature highlighted methods such as world café, focus groups, role-playing, and persona. Conclusions: This study presents the first systematic overview of co-creation methods, providing a clear understanding of the diverse methods currently in use. Our findings reveal a significant methodological gap between researchers and practitioners, offering insights into the relative prevalence and combinations of methods. By shedding light on these methods, this study helps bridge the gap and supports researchers in making informed decisions about which methods to apply in their work. Additionally, it offers a foundation for further investigation into method use in co-creation. This systematic investigation is a valuable resource for anyone engaging in co-creation or similar participatory methodologies, helping to navigate the diverse landscape of methods. UR - https://www.i-jmr.org/2024/1/e59772 UR - http://dx.doi.org/10.2196/59772 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59772 ER - TY - JOUR AU - Qureshi, Nabeel AU - Hays, D. Ron AU - Herman, M. Patricia PY - 2024/11/11 TI - Dropout in a Longitudinal Survey of Amazon Mechanical Turk Workers With Low Back Pain: Observational Study JO - Interact J Med Res SP - e58771 VL - 13 KW - chronic low back pain KW - Mechanical Turk KW - MTurk KW - survey attrition KW - survey weights KW - Amazon KW - occupational health KW - manual labor N2 - Background: Surveys of internet panels such as Amazon?s Mechanical Turk (MTurk) are common in health research. Nonresponse in longitudinal studies can limit inferences about change over time. Objective: This study aimed to (1) describe the patterns of survey responses and nonresponse among MTurk members with back pain, (2) identify factors associated with survey response over time, (3) assess the impact of nonresponse on sample characteristics, and (4) assess how well inverse probability weighting can account for differences in sample composition. Methods: We surveyed adult MTurk workers who identified as having back pain. We report participation trends over 3 survey waves and use stepwise logistic regression to identify factors related to survey participation in successive waves. Results: A total of 1678 adults participated in wave 1. Of those, 983 (59%) participated in wave 2 and 703 (42%) in wave 3. Participants who did not drop out took less time to complete previous surveys (30 min vs 35 min in wave 1, P<.001; 24 min vs 26 min in wave 2, P=.02) and reported having fewer health conditions (5.88 vs 6.6, P<.001). In multivariate models predicting responding at wave 2, lower odds of participation were associated with more time to complete the baseline survey (odds ratio [OR] 0.98, 95% CI 0.97-0.99), being Hispanic (compared with non-Hispanic, OR 0.69, 95% CI 0.49-0.96), having a bachelor?s degree as their terminal degree (compared with all other levels of education, OR 0.58, 95% CI 0.46-0.73), having more pain interference and intensity (OR 0.75, 95% CI 0.64-0.89), and having more health conditions. In contrast, older respondents (older than 45 years age compared with 18-24 years age) were more likely to respond to the wave 2 survey (OR 2.63 and 3.79, respectively) and those whose marital status was divorced (OR 1.81) and separated (OR 1.77) were also more likely to respond to the wave 2 survey. Weighted analysis showed slight differences in sample demographics and conditions and larger differences in pain assessments, particularly for those who responded to wave 2. Conclusions: Longitudinal studies on MTurk have large, differential dropouts between waves. This study provided information about the individuals more likely to drop out over time, which can help researchers prepare for future surveys. UR - https://www.i-jmr.org/2024/1/e58771 UR - http://dx.doi.org/10.2196/58771 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58771 ER - TY - JOUR AU - Scaramozza, Matthew AU - Ruet, Aurélie AU - Chiesa, A. Patrizia AU - Ahamada, Laïtissia AU - Bartholomé, Emmanuel AU - Carment, Loïc AU - Charre-Morin, Julie AU - Cosne, Gautier AU - Diouf, Léa AU - Guo, C. Christine AU - Juraver, Adrien AU - Kanzler, M. Christoph AU - Karatsidis, Angelos AU - Mazzà, Claudia AU - Penalver-Andres, Joaquin AU - Ruiz, Marta AU - Saubusse, Aurore AU - Simoneau, Gabrielle AU - Scotland, Alf AU - Sun, Zhaonan AU - Tang, Minao AU - van Beek, Johan AU - Zajac, Lauren AU - Belachew, Shibeshih AU - Brochet, Bruno AU - Campbell, Nolan PY - 2024/11/8 TI - Sensor-Derived Measures of Motor and Cognitive Functions in People With Multiple Sclerosis Using Unsupervised Smartphone-Based Assessments: Proof-of-Concept Study JO - JMIR Form Res SP - e60673 VL - 8 KW - multiple sclerosis KW - sensor-derived measure KW - smartphone KW - cognitive function KW - motor function KW - digital biomarkers KW - mobile phone N2 - Background: Smartphones and wearables are revolutionizing the assessment of cognitive and motor function in neurological disorders, allowing for objective, frequent, and remote data collection. However, these assessments typically provide a plethora of sensor-derived measures (SDMs), and selecting the most suitable measure for a given context of use is a challenging, often overlooked problem. Objective: This analysis aims to develop and apply an SDM selection framework, including automated data quality checks and the evaluation of statistical properties, to identify robust SDMs that describe the cognitive and motor function of people with multiple sclerosis (MS). Methods: The proposed framework was applied to data from a cross-sectional study involving 85 people with MS and 68 healthy participants who underwent in-clinic supervised and remote unsupervised smartphone-based assessments. The assessment provided high-quality recordings from cognitive, manual dexterity, and mobility tests, from which 47 SDMs, based on established literature, were extracted using previously developed and publicly available algorithms. These SDMs were first separately and then jointly screened for bias and normality by 2 expert assessors. Selected SDMs were then analyzed to establish their reliability, using an intraclass correlation coefficient and minimal detectable change at 95% CI. The convergence of selected SDMs with in-clinic MS functional measures and patient-reported outcomes was also evaluated. Results: A total of 16 (34%) of the 47 SDMs passed the selection framework. All selected SDMs demonstrated moderate-to-good reliability in remote settings (intraclass correlation coefficient 0.5-0.85; minimal detectable change at 95% CI 19%-35%). Selected SDMs extracted from the smartphone-based cognitive test demonstrated good-to-excellent correlation (Spearman correlation coefficient, |?|>0.75) with the in-clinic Symbol Digit Modalities Test and fair correlation with Expanded Disability Status Scale (EDSS) scores (0.25?|?|<0.5). SDMs extracted from the manual dexterity tests showed either fair correlation (0.25?|?|<0.5) or were not correlated (|?|<0.25) with the in-clinic 9-hole peg test and EDSS scores. Most selected SDMs from mobility tests showed fair correlation with the in-clinic timed 25-foot walk test and fair to moderate-to-good correlation (0.5<|?|?0.75) with EDSS scores. SDM correlations with relevant patient-reported outcomes varied by functional domain, ranging from not correlated (cognitive test SDMs) to good-to-excellent correlation (|?|>0.75) for mobility test SDMs. Overall, correlations were similar when smartphone-based tests were performed in a clinic or remotely. Conclusions: Reported results highlight that smartphone-based assessments are suitable tools to remotely obtain high-quality SDMs of cognitive and motor function in people with MS. The presented SDM selection framework promises to increase the interpretability and standardization of smartphone-based SDMs in people with MS, paving the way for their future use in interventional trials. UR - https://formative.jmir.org/2024/1/e60673 UR - http://dx.doi.org/10.2196/60673 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/60673 ER - TY - JOUR AU - Kissler, Katherine AU - Phillippi, C. Julia AU - Erickson, Elise AU - Holmes, Leah AU - Tilden, Ellen PY - 2024/11/8 TI - Collecting Real-Time Patient-Reported Outcome Data During Latent Labor: Feasibility Study of the MyCap Mobile App in Prospective Person-Centered Research JO - JMIR Form Res SP - e59155 VL - 8 KW - patient-reported outcomes KW - survey methods KW - smartphone KW - labor onset KW - prodromal symptoms KW - prospective studies N2 - Background: The growing emphasis on patient experience in medical research has increased the focus on patient-reported outcomes and symptom measures. However, patient-reported outcomes data are subject to recall bias, limiting reliability. Patient-reported data are most valid when reported by patients in real time; however, this type of data is difficult to collect from patients experiencing acute health events such as labor. Mobile technologies such as the MyCap app, integrated with the REDCap (Research Electronic Data Capture) platform, have emerged as tools for collecting patient-generated health data in real time offering potential improvements in data quality and relevance. Objective: This study aimed to evaluate the feasibility of using MyCap for real-time, patient-reported data collection during latent labor. The objective was to assess the usability of MyCap in characterizing patient experiences during this acute health event and to identify any challenges in data collection that could inform future research. Methods: In this descriptive cohort study, we quantified and characterized data collected prospectively through MyCap and the extent to which participants engaged with the app as a research tool for collecting patient-reported data in real time. Longitudinal quantitative and qualitative surveys were sent to (N=18) enrolled patients with term pregnancies planning vaginal birth at Oregon Health Sciences University. Participants were trained in app use prenatally. Then participants were invited to initiate the research survey on their personal smartphone via MyCap when they experienced labor symptoms and were asked to return to MyCap every 3 hours to provide additional longitudinal symptom data. Results: Out of 18 enrolled participants, 17 completed the study. During latent labor, 13 (76.5%) participants (all those who labored at home and two-thirds of those who were induced) recorded at least 1 symptom report during latent labor. A total of 191 quantitative symptom reports (mean of 10 per participant) were recorded. The most commonly reported symptoms were fatigue, contractions, and pain, with nausea and diarrhea being less frequent but more intense. Four participants recorded qualitative data during labor and 14 responded to qualitative prompts in the postpartum period. The study demonstrated that MyCap could effectively capture real-time patient-reported data during latent labor, although qualitative data collection during active symptoms was less robust. Conclusions: MyCap is a feasible tool for collecting prospective data on patient-reported symptoms during latent labor. Participants engaged actively with quantitative symptom reporting, though qualitative data collection was more challenging. The use of MyCap appears to reduce recall bias and facilitate more accurate data collection for patient-reported symptoms during acute health events outside of health care settings. Future research should explore strategies to enhance qualitative data collection and assess the tool?s usability across more diverse populations and disease states. UR - https://formative.jmir.org/2024/1/e59155 UR - http://dx.doi.org/10.2196/59155 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59155 ER - TY - JOUR AU - Perme, Natalie AU - Reid, Endia AU - Eluagu, Chinwenwa Macdonald AU - Thompson, John AU - Hebert, Courtney AU - Gabbe, Steven AU - Swoboda, Marie Christine PY - 2024/11/8 TI - Development and Usability of the OHiFamily Mobile App to Enhance Accessibility to Maternal and Infant Information for Expectant Families in Ohio: Qualitative Study JO - JMIR Form Res SP - e53299 VL - 8 KW - health resources KW - pregnancy KW - patient engagement KW - mHealth KW - maternal KW - mobile health KW - app KW - focus group KW - landscape analysis KW - birth KW - preterm KW - premature KW - mortality KW - death KW - pediatric KW - infant KW - neonatal KW - design KW - development KW - obstetric KW - mobile phone N2 - Background: The Infant Mortality Research Partnership in Ohio is working to help pregnant individuals and families on Medicaid who are at risk for infant mortality and preterm birth. As part of this initiative, researchers at The Ohio State University worked to develop a patient-facing mobile app, OHiFamily, targeted toward, and created for, this population. To address the social determinants of health that can affect maternal and infant health, the app provides curated information on community resources, health care services, and educational materials in a format that is easily accessible and intended to facilitate contact between families and resources. The OHiFamily app includes 3 distinct features, that is, infant care logging (eg, feeding and diaper changes), curated educational resources, and a link to the curated Ohio resource database (CORD). This paper describes the development and assessment of the OHiFamily app as well as CORD. Objective: This study aimed to describe the development of the OHiFamily mobile app and CORD and the qualitative feedback received by the app?s intended audience. Methods: The researchers performed a landscape analysis and held focus groups to determine the resources and app features of interest to Ohio families on Medicaid. Results: Participants from several focus groups were interested in an app that could offer community resources with contact information, information about medical providers and information and ways to contact them, health tips, and information about pregnancy and infant development. Feedback was provided by 9 participants through 3 focus group sessions. Using this feedback, the team created a curated resource database and mobile app to help users locate and access resources, as well as access education materials and infant tracking features. Conclusions: OHiFamily offers a unique combination of features and access to local resources for families on Medicaid in Ohio not seen in other smartphone apps. UR - https://formative.jmir.org/2024/1/e53299 UR - http://dx.doi.org/10.2196/53299 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53299 ER - TY - JOUR AU - Knowlden, P. Adam AU - Winchester, J. Lee AU - MacDonald, V. Hayley AU - Geyer, D. James AU - Higginbotham, C. John PY - 2024/11/8 TI - Associations Among Cardiometabolic Risk Factors, Sleep Duration, and Obstructive Sleep Apnea in a Southeastern US Rural Community: Cross-Sectional Analysis From the SLUMBRx-PONS Study JO - JMIR Form Res SP - e54792 VL - 8 KW - obstructive sleep apnea KW - obesity KW - adiposity KW - cardiometabolic KW - cardiometabolic disease KW - risk factors KW - sleep KW - sleep duration KW - sleep apnea KW - Short Sleep Undermines Cardiometabolic Health-Public Health Observational study KW - SLUMBRx study N2 - Background: Short sleep and obstructive sleep apnea are underrecognized strains on the public health infrastructure. In the United States, over 35% of adults report short sleep and more than 80% of individuals with obstructive sleep apnea remain undiagnosed. The associations between inadequate sleep and cardiometabolic disease risk factors have garnered increased attention. However, challenges persist in modeling sleep-associated cardiometabolic disease risk factors. Objective: This study aimed to report early findings from the Short Sleep Undermines Cardiometabolic Health-Public Health Observational study (SLUMBRx-PONS). Methods: Data for the SLUMBRx-PONS study were collected cross-sectionally and longitudinally from a nonclinical, rural community sample (n=47) in the southeast United States. Measures included 7 consecutive nights of wrist-based actigraphy (eg, mean of 7 consecutive nights of total sleep time [TST7N]), 1 night of sleep apnea home testing (eg, apnea-hypopnea index [AHI]), and a cross-sectional clinical sample of anthropometric (eg, BMI), cardiovascular (eg, blood pressure), and blood-based biomarkers (eg, triglycerides and glucose). Correlational analyses and regression models assessed the relationships between the cardiometabolic disease risk factors and the sleep indices (eg, TST7N and AHI). Linear regression models were constructed to examine associations between significant cardiometabolic indices of TST7N (model 1) and AHI (model 2). Results: Correlational assessment in model 1 identified significant associations between TST7N and AHI (r=?0.45, P=.004), BMI (r=?0.38, P=.02), systolic blood pressure (r=0.40, P=.01), and diastolic blood pressure (r=0.32, P=.049). Pertaining to model 1, composite measures of AHI, BMI, systolic blood pressure, and diastolic blood pressure accounted for 25.1% of the variance in TST7N (R2adjusted=0.25; F2,38=7.37; P=.002). Correlational analyses in model 2 revealed significant relationships between AHI and TST7N (r=?0.45, P<.001), BMI (r=0.71, P<.001), triglycerides (r=0.36, P=.03), and glucose (r=0.34, P=.04). Results from model 2 found that TST7N, triglycerides, and glucose accounted for 37.6% of the variance in the composite measure of AHI and BMI (R2adjusted=0.38; F3,38=8.63; P<.001). Conclusions: Results from the SLUMBRx-PONS study highlight the complex interplay between sleep-associated risk factors for cardiometabolic disease. Early findings underscore the need for further investigations incorporating the collection of clinical, epidemiological, and ambulatory measures to inform public health, health promotion, and health education interventions addressing the cardiometabolic consequences of inadequate sleep. International Registered Report Identifier (IRRID): RR2-10.2196/27139 UR - https://formative.jmir.org/2024/1/e54792 UR - http://dx.doi.org/10.2196/54792 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54792 ER - TY - JOUR AU - Saraya, Norah AU - McBride, Jonathon AU - Singh, Karandeep AU - Sohail, Omar AU - Das, Jeet Porag PY - 2024/11/8 TI - Comparison of Auscultation Quality Using Contemporary Digital Stethoscopes JO - JMIR Cardio SP - e54746 VL - 8 KW - auscultation KW - digital stethoscopes KW - valvular heart disease UR - https://cardio.jmir.org/2024/1/e54746 UR - http://dx.doi.org/10.2196/54746 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54746 ER - TY - JOUR AU - Naicker, Sundresan AU - Tariq, Amina AU - Donovan, Raelene AU - Magon, Honor AU - White, Nicole AU - Simmons, Joshua AU - McPhail, M. Steven PY - 2024/11/8 TI - Patterns and Perceptions of Standard Order Set Use Among Physicians Working Within a Multihospital System: Mixed Methods Study JO - JMIR Form Res SP - e54022 VL - 8 KW - medical informatics KW - adoption and implementation KW - behavior KW - health systems KW - testing KW - electronic medical records KW - behavioral model KW - quantitative data KW - semistructured interview KW - clinical practice KW - user preference KW - user KW - user experience N2 - Background: Electronic standard order sets automate the ordering of specific treatment, testing, and investigative protocols by physicians. These tools may help reduce unwarranted clinical variation and improve health care efficiency. Despite their routine implementation within electronic medical records (EMRs), little is understood about how they are used and what factors influence their adoption in practice. Objective: This study aims to (1) describe the patterns of use of standard order sets implemented in a widely used EMR (PowerPlans and Cerner Millennium) within a multihospital digital health care system; (2) explore the experiences and perceptions of implementers and users regarding the factors contributing to the use of these standard order sets; and (3) map these findings to the Capability, Opportunity, and Motivation Behavior (COM-B) model of behavior change to assist those planning to develop, improve, implement, and iterate the use of standard order sets in hospital settings. Methods: Quantitative data on standard order set usage were captured from 5 hospitals over 5-month intervals for 3 years (2019, 2020, and 2021). Qualitative data, comprising unstructured and semistructured interviews (n=15), were collected and analyzed using a reflexive thematic approach. Interview themes were then mapped to a theory-informed model of behavior change (COM-B) to identify determinants of standard order set usage in routine clinical practice. The COM-B model is an evidence-based, multicomponent framework that posits that human actions result from multiple contextual influences, which can be categorized across 3 dimensions: capability, opportunity, and motivation, all of which intersect. Results: The total count of standard order set usage across the health system during the 2019 observation period was 267,253, increasing to 293,950 in 2020 and 335,066 in 2021. There was a notable shift toward using specialty order sets that received upgrades during the study period. Four emergent themes related to order set use were derived from clinician interviews: (1) Knowledge and Skills; (2) Perceptions; (3) Technical Dependencies; and (4) Unintended Consequences, all of which were mapped to the COM-B model. Findings indicate a user preference for customized order sets that respond to local context and user experience. Conclusions: The study findings suggest that ongoing investment in the development and functionality of specialty order sets has the potential to enhance usage as these sets continue to be customized in response to local context and user experience. Sustained and continuous uptake of appropriate Computerized Provider Order Entry use may require implementation strategies that address the capability, opportunity, and motivational influencers of behavior. UR - https://formative.jmir.org/2024/1/e54022 UR - http://dx.doi.org/10.2196/54022 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54022 ER - TY - JOUR AU - Lange-Drenth, Lukas AU - Schulz, Holger AU - Suck, Isabell AU - Bleich, Christiane PY - 2024/11/8 TI - Barriers, Facilitators, and Requirements for a Telerehabilitation Aftercare Program for Patients After Occupational Injuries: Semistructured Interviews With Key Stakeholders JO - JMIR Form Res SP - e51865 VL - 8 KW - telerehabilitation KW - rehabilitation KW - eHealth development KW - value specification KW - stakeholder participation KW - occupational injuries KW - vocational rehabilitation KW - aftercare KW - mobile phone N2 - Background: Patients with occupational injuries often receive multidisciplinary rehabilitation for a rapid return to work. Rehabilitation aftercare programs give patients the opportunity to help patients apply the progress they have made during the rehabilitation to their everyday activities. Telerehabilitation aftercare programs can help reduce barriers, such as lack of time due to other commitments, because they can be used regardless of time or location. Careful identification of barriers, facilitators, and design requirements with key stakeholders is a critical step in developing a telerehabilitation aftercare program. Objective: This study aims to identify barriers, facilitators, and design requirements for a future telerehabilitation aftercare program for patients with occupational injuries from the perspective of the key stakeholders. Methods: We used a literature review and expert recommendations to identify key stakeholders. We conducted semistructured interviews in person and via real-time video calls with 27 key stakeholders to collect data. Interviews were transcribed verbatim, and thematic analysis was applied. We selected key stakeholder statements about facilitators and barriers and categorized them as individual, technical, environmental, and organizational facilitators and barriers. We identified expressions that captured aspects that the telerehabilitation aftercare program should fulfill and clustered them into attributes and overarching values. We translated the attributes into one or more requirements and grouped them into content, functional, service, user experience, and work context requirements. Results: The key stakeholders identified can be grouped into the following categories: patients, health care professionals, administrative personnel, and members of the telerehabilitation program design and development team. The most frequently reported facilitators of a future telerehabilitation aftercare program were time savings for patients, high motivation of the patients to participate in telerehabilitation aftercare program, high usability of the program, and regular in-person therapy meetings during the telerehabilitation aftercare program. The most frequently reported barriers were low digital affinity and skills of the patients and personnel, patients? lack of trust and acceptance of the telerehabilitation aftercare program, slow internet speed, program functionality problems (eg, application crashes or freezes), and inability of telerehabilitation to deliver certain elements of in-person rehabilitation aftercare such as monitoring exercise performance. In our study, the most common design requirements were reducing barriers and implementing facilitators. The 2 most frequently discussed overarching values were tailoring of telerehabilitation, such as a tailored exercise plan and tailored injury-related information, and social interaction, such as real-time psychotherapy and digital and in-person rehabilitation aftercare in a blended care approach. Conclusions: Key stakeholders reported on facilitators, barriers, and design requirements that should be considered throughout the development process. Tailoring telerehabilitation content was the key value for stakeholders to ensure the program could meet the needs of patients with different types of occupational injuries. UR - https://formative.jmir.org/2024/1/e51865 UR - http://dx.doi.org/10.2196/51865 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/51865 ER - TY - JOUR AU - Ferguson, Caleb AU - William, Scott AU - Allida, M. Sabine AU - Fulcher, Jordan AU - Jenkins, J. Alicia AU - Lattimore, Jo-Dee AU - Loch, L-J AU - Keech, Anthony PY - 2024/11/7 TI - The Development of Heart Failure Electronic-Message Driven Tips to Support Self-Management: Co-Design Case Study JO - JMIR Cardio SP - e57328 VL - 8 KW - heart failure KW - co-design KW - smartphone KW - app design KW - patient education KW - e-TIPS KW - electronic-message driven tips N2 - Background: Heart failure (HF) is a complex syndrome associated with high morbidity and mortality and increased health care use. Patient education is key to improving health outcomes, achieved by promoting self-management to optimize medical management. Newer digital tools like SMS text messaging and smartphone apps provide novel patient education approaches. Objective: This study aimed to partner with clinicians and people with lived experience of HF to identify the priority educational topic areas to inform the development and delivery of a bank of electronic-message driven tips (e-TIPS) to support HF self-management. Methods: We conducted 3 focus groups with cardiovascular clinicians, people with lived experience of HF, and their caregivers, which consisted of 2 stages: stage 1 (an exploratory qualitative study to identify the unmet educational needs of people living with HF; previously reported) and stage 2 (a co-design feedback session to identify educational topic areas and inform the delivery of e-TIPS). This paper reports the findings of the co-design feedback session. Results: We identified 5 key considerations in delivering e-TIPS and 5 relevant HF educational topics for their content. Key considerations in e-TIP delivery included (1) timing of the e-TIPS; (2) clear and concise e-TIPS; (3) embedding a feedback mechanism; (4) distinguishing actionable and nonactionable e-TIPS; and (5) frequency of e-TIP delivery. Relevant educational topic areas included the following: (1) cardiovascular risk reduction, (2) self-management, (3) food and nutrition, (4) sleep hygiene, and (5) mental health. Conclusions: The findings from this co-design case study have provided a foundation for developing a bank of e-TIPS. These will now be evaluated for usability in the BANDAIDS e-TIPS, a single-group, quasi-experimental study of a 24-week e-TIP program (personalized educational messages) delivered via SMS text messaging (ACTRN12623000644662). UR - https://cardio.jmir.org/2024/1/e57328 UR - http://dx.doi.org/10.2196/57328 ID - info:doi/10.2196/57328 ER - TY - JOUR AU - Xiong, Eddy AU - Bonner, Carissa AU - King, Amanda AU - Bourne, Maxwell Zoltan AU - Morgan, Mark AU - Tolosa, Ximena AU - Stanton, Tony AU - Greaves, Kim PY - 2024/11/6 TI - Insights From the Development of a Dynamic Consent Platform for the Australians Together Health Initiative (ATHENA) Program: Interview and Survey Study JO - JMIR Form Res SP - e57165 VL - 8 KW - dynamic consent KW - research KW - clinical trials KW - consumer engagement KW - digital consent KW - development KW - decision making KW - decision KW - feedback KW - user platform KW - users KW - communication N2 - Background: Dynamic consent has the potential to address many of the issues facing traditional paper-based or electronic consent, including enrolling informed and engaged participants in the decision-making process. The Australians Together Health Initiative (ATHENA) program aims to connect participants across Queensland, Australia, with new research opportunities. At its core is dynamic consent, an interactive and participant-centric digital platform that enables users to view ongoing research activities, update consent preferences, and have ongoing engagement with researchers. Objective: This study aimed to describe the development of the ATHENA dynamic consent platform within the framework of the ATHENA program, including how the platform was designed, its utilization by participants, and the insights gained. Methods: One-on-one interviews were undertaken with consumers, followed by a workshop with health care staff to gain insights into the dynamic consent concept. Five problem statements were developed, and solutions were posed, from which a dynamic consent platform was constructed, tested, and used for implementation in a clinical trial. Potential users were randomly recruited from a pre-existing pool of 615 participants in the ATHENA program. Feedback on user platform experience was gained from a survey hosted on the platform. Results: In the 13 consumer interviews undertaken, participants were positive about dynamic consent, valuing privacy, ease of use, and adequate communication. Motivators for registration were feedback on data usage and its broader community benefits. Problem statements were security, trust and governance, ease of use, communication, control, and need for a scalable platform. Using the newly constructed dynamic consent platform, 99 potential participants were selected, of whom 67 (68%) were successfully recontacted. Of these, 59 (88%) agreed to be sent the platform, 44 (74%) logged on (indicating use), and 22 (57%) registered for the clinical trial. Survey feedback was favorable, with an average positive rating of 78% across all questions, reflecting satisfaction with the clarity, brevity, and flexibility of the platform. Barriers to implementation included technological and health literacy. Conclusions: This study describes the successful development and testing of a dynamic consent platform that was well-accepted, with users recognizing its advantages over traditional methods of consent regarding flexibility, ease of communication, and participant satisfaction. This information may be useful to other researchers who plan to use dynamic consent in health care research. UR - https://formative.jmir.org/2024/1/e57165 UR - http://dx.doi.org/10.2196/57165 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57165 ER - TY - JOUR AU - Tagi, Masato AU - Hamada, Yasuhiro AU - Shan, Xiao AU - Ozaki, Kazumi AU - Kubota, Masanori AU - Amano, Sosuke AU - Sakaue, Hiroshi AU - Suzuki, Yoshiko AU - Konishi, Takeshi AU - Hirose, Jun PY - 2024/11/5 TI - A Food Intake Estimation System Using an Artificial Intelligence?Based Model for Estimating Leftover Hospital Liquid Food in Clinical Environments: Development and Validation Study JO - JMIR Form Res SP - e55218 VL - 8 KW - artificial intelligence KW - machine learning KW - system development KW - food intake KW - dietary intake KW - dietary assessment KW - food consumption KW - image visual estimation KW - AI estimation KW - direct visual estimation N2 - Background: Medical staff often conduct assessments, such as food intake and nutrient sufficiency ratios, to accurately evaluate patients? food consumption. However, visual estimations to measure food intake are difficult to perform with numerous patients. Hence, the clinical environment requires a simple and accurate method to measure dietary intake. Objective: This study aims to develop a food intake estimation system through an artificial intelligence (AI) model to estimate leftover food. The accuracy of the AI?s estimation was compared with that of visual estimation for liquid foods served to hospitalized patients. Methods: The estimations were evaluated by a dietitian who looked at the food photo (image visual estimation) and visual measurement evaluation was carried out by a nurse who looked directly at the food (direct visual estimation) based on actual measurements. In total, 300 dishes of liquid food (100 dishes of thin rice gruel, 100 of vegetable soup, 31 of fermented milk, and 18, 12, 13, and 26 of peach, grape, orange, and mixed juices, respectively) were used. The root-mean-square error (RMSE) and coefficient of determination (R2) were used as metrics to determine the accuracy of the evaluation process. Corresponding t tests and Spearman rank correlation coefficients were used to verify the accuracy of the measurements by each estimation method with the weighing method. Results: The RMSE obtained by the AI estimation approach was 8.12 for energy. This tended to be smaller and larger than that obtained by the image visual estimation approach (8.49) and direct visual estimation approach (4.34), respectively. In addition, the R2 value for the AI estimation tended to be larger and smaller than the image and direct visual estimations, respectively. There was no difference between the AI estimation (mean 71.7, SD 23.9 kcal, P=.82) and actual values with the weighing method. However, the mean nutrient intake from the image visual estimation (mean 75.5, SD 23.2 kcal, P<.001) and direct visual estimation (mean 73.1, SD 26.4 kcal, P=.007) were significantly different from the actual values. Spearman rank correlation coefficients were high for energy (?=0.89-0.97), protein (?=0.94-0.97), fat (?=0.91-0.94), and carbohydrate (?=0.89-0.97). Conclusions: The measurement from the food intake estimation system by an AI-based model to estimate leftover liquid food intake in patients showed a high correlation with the actual values with the weighing method. Furthermore, it also showed a higher accuracy than the image visual estimation. The errors of the AI estimation method were within the acceptable range of the weighing method, which indicated that the AI-based food intake estimation system could be applied in clinical environments. However, its lower accuracy than that of direct visual estimation was still an issue. UR - https://formative.jmir.org/2024/1/e55218 UR - http://dx.doi.org/10.2196/55218 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55218 ER - TY - JOUR AU - Al-Attar, Mariam AU - Assawamartbunlue, Kesmanee AU - Gandrup, Julie AU - van der Veer, N. Sabine AU - Dixon, G. William PY - 2024/10/28 TI - Exploring the Potential of Electronic Patient-Generated Health Data for Evaluating Treatment Response to Intramuscular Steroids in Rheumatoid Arthritis: Case Series JO - JMIR Form Res SP - e55715 VL - 8 KW - patient-reported outcome measures KW - remote monitoring KW - patient-generated health data KW - mobile health KW - intramuscular steroid injections KW - rheumatoid arthritis KW - app KW - case series KW - symptom tracking KW - pain score N2 - Background: Mobile health devices are increasingly available, presenting exciting opportunities to remotely collect high-frequency, electronic patient-generated health data (ePGHD). This novel data type may provide detailed insights into disease activity outside usual clinical settings. Assessing treatment responses, which can be hampered by the infrequency of appointments and recall bias, is a promising, novel application of ePGHD. Drugs with short treatment effects, such as intramuscular steroid injections, illustrate the challenge, as patients are unlikely to accurately recall treatment responses at follow-ups, which often occur several months later. Retrospective assessment means that responses may be over- or underestimated. High-frequency ePGHD, such as daily, app-collected, patient-reported symptoms between clinic appointments, may bridge this gap. However, the potential of ePGHD remains untapped due to the absence of established definitions for treatment response using ePGHD or established methodological approaches for analyzing this type of data. Objective: This study aims to explore the feasibility of evaluating treatment responses to intramuscular steroid therapy in a case series of patients with rheumatoid arthritis tracking daily symptoms using a smartphone app. Methods: We report a case series of patients who collected ePGHD through the REmote Monitoring Of Rheumatoid Arthritis (REMORA) smartphone app for daily remote symptom tracking. Symptoms were tracked on a 0-10 scale. We described the patients? longitudinal pain scores before and after intramuscular steroid injections. The baseline pain score was calculated as the mean pain score in the 10 days prior to the injection. This was compared to the pain scores in the days following the injection. ?Response? was defined as any improvement from the baseline score on the first day following the injection. The response end time was defined as the first date when the pain score exceeded the pre-steroid baseline. Results: We included 6 patients who, between them, received 9 steroid injections. Average pre-injection pain scores ranged from 3.3 to 9.3. Using our definitions, 7 injections demonstrated a response. Among the responders, the duration of response ranged from 1 to 54 days (median 9, IQR 7-41), average pain score improvement ranged from 0.1 to 5.3 (median 3.3, IQR 2.2-4.0), and maximum pain score improvement ranged from 0.1 to 7.0 (median 4.3, IQR 1.7 to 6.0). Conclusions: This case series demonstrates the feasibility of using ePGHD to evaluate treatment response and is an important exploratory step toward developing more robust methodological approaches for analysis of this novel data type. Issues highlighted by our analysis include the importance of accounting for one-off data points, varying response start times, and confounders such as other medications. Future analysis of ePGHD across a larger population is required to address issues highlighted by our analysis and to develop meaningful consensus definitions for treatment response in time-series data. UR - https://formative.jmir.org/2024/1/e55715 UR - http://dx.doi.org/10.2196/55715 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55715 ER - TY - JOUR AU - So, Jae-hee AU - Chang, Joonhwan AU - Kim, Eunji AU - Na, Junho AU - Choi, JiYeon AU - Sohn, Jy-yong AU - Kim, Byung-Hoon AU - Chu, Hui Sang PY - 2024/10/24 TI - Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study JO - JMIR Form Res SP - e58418 VL - 8 KW - large language model KW - psychiatric interview KW - interview summarization KW - symptom delineation N2 - Background: Recent advancements in large language models (LLMs) have accelerated their use across various domains. Psychiatric interviews, which are goal-oriented and structured, represent a significantly underexplored area where LLMs can provide substantial value. In this study, we explore the application of LLMs to enhance psychiatric interviews by analyzing counseling data from North Korean defectors who have experienced traumatic events and mental health issues. Objective: This study aims to investigate whether LLMs can (1) delineate parts of the conversation that suggest psychiatric symptoms and identify those symptoms, and (2) summarize stressors and symptoms based on the interview dialogue transcript. Methods: Given the interview transcripts, we align the LLMs to perform 3 tasks: (1) extracting stressors from the transcripts, (2) delineating symptoms and their indicative sections, and (3) summarizing the patients based on the extracted stressors and symptoms. These 3 tasks address the 2 objectives, where delineating symptoms is based on the output from the second task, and generating the summary of the interview incorporates the outputs from all 3 tasks. In this context, the transcript data were labeled by mental health experts for the training and evaluation of the LLMs. Results: First, we present the performance of LLMs in estimating (1) the transcript sections related to psychiatric symptoms and (2) the names of the corresponding symptoms. In the zero-shot inference setting using the GPT-4 Turbo model, 73 out of 102 transcript segments demonstrated a recall mid-token distance d<20 for estimating the sections associated with the symptoms. For evaluating the names of the corresponding symptoms, the fine-tuning method demonstrates a performance advantage over the zero-shot inference setting of the GPT-4 Turbo model. On average, the fine-tuning method achieves an accuracy of 0.82, a precision of 0.83, a recall of 0.82, and an F1-score of 0.82. Second, the transcripts are used to generate summaries for each interviewee using LLMs. This generative task was evaluated using metrics such as Generative Evaluation (G-Eval) and Bidirectional Encoder Representations from Transformers Score (BERTScore). The summaries generated by the GPT-4 Turbo model, utilizing both symptom and stressor information, achieve high average G-Eval scores: coherence of 4.66, consistency of 4.73, fluency of 2.16, and relevance of 4.67. Furthermore, it is noted that the use of retrieval-augmented generation did not lead to a significant improvement in performance. Conclusions: LLMs, using either (1) appropriate prompting techniques or (2) fine-tuning methods with data labeled by mental health experts, achieved an accuracy of over 0.8 for the symptom delineation task when measured across all segments in the transcript. Additionally, they attained a G-Eval score of over 4.6 for coherence in the summarization task. This research contributes to the emerging field of applying LLMs in psychiatric interviews and demonstrates their potential effectiveness in assisting mental health practitioners. UR - https://formative.jmir.org/2024/1/e58418 UR - http://dx.doi.org/10.2196/58418 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58418 ER - TY - JOUR AU - Cho, Minseo AU - Park, Doeun AU - Choo, Myounglee AU - Kim, Jinwoo AU - Han, Hyun Doug PY - 2024/10/24 TI - Development and Initial Evaluation of a Digital Phenotype Collection System for Adolescents: Proof-of-Concept Study JO - JMIR Form Res SP - e59623 VL - 8 KW - adolescents KW - adolescent mental health KW - smartphone apps KW - self-monitoring KW - qualitative research KW - phenotypes KW - proof of concept KW - digital phenotyping KW - phenotype data KW - ecological momentary assessment N2 - Background: The growing concern on adolescent mental health calls for proactive early detection and intervention strategies. There is a recognition of the link between digital phenotypes and mental health, drawing attention to their potential use. However, the process of collecting digital phenotype data presents challenges despite its promising prospects. Objective: This study aims to develop and validate system concepts for collecting adolescent digital phenotypes that effectively manage inherent challenges in the process. Methods: In a formative investigation (N=34), we observed adolescent self-recording behaviors and conducted interviews to develop design goals. These goals were then translated into system concepts, which included planners resembling interfaces, simplified data input with tags, visual reports on behaviors and moods, and supportive ecological momentary assessment (EMA) prompts. A proof-of-concept study was conducted over 2 weeks (n=16), using tools that simulated the concepts to record daily activities and complete EMA surveys. The effectiveness of the system was evaluated through semistructured interviews, supplemented by an analysis of the frequency of records and responses. Results: The interview findings revealed overall satisfaction with the system concepts, emphasizing strong support for self-recording. Participants consistently maintained daily records throughout the study period, with no missing data. They particularly valued the recording procedures that aligned well with their self-recording goal of time management, facilitated by the interface design and simplified recording procedures. Visualizations during recording and subsequent report viewing further enhanced engagement by identifying missing data and encouraging deeper self-reflection. The average EMA compliance reached 72%, attributed to a design that faithfully reflected adolescents? lives, with surveys scheduled at convenient times and supportive messages tailored to their daily routines. The high compliance rates observed and positive feedback from participants underscore the potential of our approach in addressing the challenges of collecting digital phenotypes among adolescents. Conclusions: Integrating observations of adolescents? recording behavior into the design process proved to be beneficial for developing an effective and highly compliant digital phenotype collection system. UR - https://formative.jmir.org/2024/1/e59623 UR - http://dx.doi.org/10.2196/59623 UR - http://www.ncbi.nlm.nih.gov/pubmed/39446465 ID - info:doi/10.2196/59623 ER - TY - JOUR AU - Wen, Fred Cheng K. AU - Schneider, Stefan AU - Junghaenel, U. Doerte AU - Toledo, L. Meynard John AU - Lee, Pey-Jiuan AU - Smyth, M. Joshua AU - Stone, A. Arthur PY - 2024/10/24 TI - Evaluating the Psychometric Properties of a Physical Activity and Sedentary Behavior Identity Scale: Survey Study With Two Independent Samples of Adults in the United States JO - JMIR Form Res SP - e59950 VL - 8 KW - physical activity KW - sedentary behavior KW - geriatrics KW - exercise KW - lifestyle KW - physical health KW - mental health KW - social-cognitive approach N2 - Background: Emerging evidence suggests a positive association between relevant aspects of one?s psychological identity and physical activity engagement, but the current understanding of this relationship is primarily based on scales designed to assess identity as a person who exercises, leaving out essential aspects of physical activities (eg, incidental and occupational physical activity) and sedentary behavior. Objective: The goal of this study is to evaluate the validity of a new physical activity and sedentary behavior (PA/SB) identity scale using 2 independent samples of US adults. Methods: In study 1, participants answered 21 candidate items for the PA/SB identity scale and completed the International Physical Activity Questionnaire-Short Form (IPAQ-SF). Study 2 participants completed the same PA/SB identity items twice over a 1-week interval and completed the IPAQ-SF at the end. We performed factor analyses to evaluate the structure of the PA/SB identity scale, evaluated convergent validity and test-retest reliability (in study 2) of the final scale scores, and examined their discriminant validity using tests for differences in dependent correlations. Results: The final PA/SB identity measure was comprised of 3 scales: physical activity role identity (F1), physical activity belief (F2), and sedentary behavior role identity (F3). The scales had high test-retest reliability (Pearson correlation coefficient: F1, r=0.87; F2, r=0.75; F3, r=0.84; intraclass correlation coefficient [ICC]: F1: ICC=0.85; F2: ICC=0.75; F3: ICC=0.84). F1 and F2 were positively correlated with each other (study 1, r=0.76; study 2, r=0.69), while both were negatively correlated with F3 (Pearson correlation coefficient between F1 and F3: r=?0.58 for study 1 and r=?0.73 for study 2; F2 and F3: r=?0.46 for studies 1 and 2). Data from both studies also demonstrated adequate discriminant validity of the scale developed. Significantly larger correlations with time in vigorous and moderate activities and time walking and sitting assessed by IPAQ-SF with F1, compared with F2, were observed. Significantly larger correlations with time in vigorous and moderate activities with F1, compared with F3, were also observed. Similarly, a larger correlation with time in vigorous activities and a smaller correlation with time walking were observed with F2, compared with F3. Conclusions: This study provided initial empirical evidence from 2 independent studies on the reliability and validity of the PA/SB identity scales for adults. UR - https://formative.jmir.org/2024/1/e59950 UR - http://dx.doi.org/10.2196/59950 UR - http://www.ncbi.nlm.nih.gov/pubmed/39446463 ID - info:doi/10.2196/59950 ER - TY - JOUR AU - Juhl, Haase Marie AU - Soerensen, Lykkegaard Ann AU - Vardinghus-Nielsen, Henrik AU - Mortensen, Sinding Lea AU - Kolding Kristensen, Jette AU - Olesen, Estrup Anne PY - 2024/10/9 TI - Designing an Intervention to Improve Medication Safety for Nursing Home Residents Based on Experiential Knowledge Related to Patient Safety Culture at the Nursing Home Front Line: Cocreative Process Study JO - JMIR Form Res SP - e54977 VL - 8 KW - intervention development KW - nursing home KW - frontline professionals KW - medication safety KW - quality improvement KW - patient safety culture KW - experiential knowledge KW - cocreation KW - resilient health care systems KW - safety II perspective KW - human resources KW - integrated knowledge translation N2 - Background: Despite years of attention, avoiding medication-related harm remains a global challenge. Nursing homes provide essential health care for frail older individuals, who often experience multiple chronic diseases and polypharmacy, increasing their risk of medication errors. Evidence of effective interventions to improve medication safety in these settings is inconclusive. Focusing on patient safety culture is a potential key to intervention development as it forms the foundation for overall patient safety and is associated with medication errors. Objective: This study aims to develop an intervention to improve medication safety for nursing home residents through a cocreative process guided by integrated knowledge translation and experience-based codesign. Methods: This study used a cocreative process guided by integrated knowledge translation and experience-based co-design principles. Evidence on patient safety culture was used as an inspirational source for exploration of medication safety. Data collection involved semistructured focus groups to generate experiential knowledge (stage 1) to inform intervention design in a multidisciplinary workshop (stage 2). Research validation engaging different types of research expertise and municipal managerial representatives in finalizing the intervention design was essential. Acceptance of the final intervention for evaluation was aimed for through contextualization focused on partnership with a municipal advisory board. An abductive, rapid qualitative analytical approach to data analysis was chosen using elements from analyzing in the present, addressing the time-dependent, context-bound aspects of the cocreative process. Results: Experiential knowledge was represented by three main themes: (1) closed systems and gaps between functions, (2) resource interpretation and untapped potential, and (3) community of medication safety and surveillance. The main themes informed the design of preliminary intervention components in a multidisciplinary workshop. An intervention design process focused on research validation in addition to contextualization resulted in the Safe Medication in Nursing Home Residents (SAME) intervention covering (1) campaign material visualizing key roles and responsibilities regarding medication for nursing home residents and (2) ?Medication safety reflexive spaces? focused on social and health care assistants. Conclusions: The cocreative process successfully resulted in the multifaceted SAME intervention, grounded in lived experiences shared by some of the most important (but often underrepresented in research) stakeholders: frontline health care professionals and representatives of nursing home residents. This study brought attention toward closed systems related to functions in medication management and surveillance, not only informing the SAME intervention design but as opportunities for further exploration in future research. Evaluation of the intervention is an important next step. Overall, this study represents an important contribution to the complex field of medication safety. International Registered Report Identifier (IRRID): RR2-10.2196/43538 UR - https://formative.jmir.org/2024/1/e54977 UR - http://dx.doi.org/10.2196/54977 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54977 ER - TY - JOUR AU - Zhou, Weipeng AU - Youngbloom, Amy AU - Ren, Xinyang AU - Saelens, E. Brian AU - Mooney, D. Sean AU - Mooney, J. Stephen PY - 2024/10/4 TI - The Automatic Context Measurement Tool (ACMT) to Compile Participant-Specific Built and Social Environment Measures for Health Research: Development and Usability Study JO - JMIR Form Res SP - e56510 VL - 8 KW - built environment KW - social environment KW - geocoding KW - GIS KW - geographic information systems KW - ACMT KW - automatic context measurement tool KW - linkage KW - privacy N2 - Background: The environment shapes health behaviors and outcomes. Studies exploring this influence have been limited to research groups with the geographic information systems expertise required to develop built and social environment measures (eg, groups that include a researcher with geographic information system expertise). Objective: The goal of this study was to develop an open-source, user-friendly, and privacy-preserving tool for conveniently linking built, social, and natural environmental variables to study participant addresses. Methods: We built the automatic context measurement tool (ACMT). The ACMT comprises two components: (1) a geocoder, which identifies a latitude and longitude given an address (currently limited to the United States), and (2) a context measure assembler, which computes measures from publicly available data sources linked to a latitude and longitude. ACMT users access both of these components using an RStudio/RShiny-based web interface that is hosted within a Docker container, which runs on a local computer and keeps user data stored in local to protect sensitive data. We illustrate ACMT with 2 use cases: one comparing population density patterns within several major US cities, and one identifying correlates of cannabis licensure status in Washington State. Results: In the population density analysis, we created a line plot showing the population density (x-axis) in relation to distance from the center of the city (y-axis, using city hall location as a proxy) for Seattle, Los Angeles, Chicago, New York City, Nashville, Houston, and Boston with the distances being 1000, 2000, 3000, 4000, and 5000 m. We found the population density tended to decrease as distance from city hall increased except for Nashville and Houston, 2 cities that are notably more sprawling than the others. New York City had a significantly higher population density than the others. We also observed that Los Angeles and Seattle had similarly low population densities within up to 2500 m of City Hall. In the cannabis licensure status analysis, we gathered neighborhood measures such as age, sex, commute time, and education. We found the strongest predictive characteristic of cannabis license approval to be the count of female children aged 5 to 9 years and the proportion of females aged 62 to 64 years who were not in the labor force. However, after accounting for Bonferroni error correction, none of the measures were significantly associated with cannabis retail license approval status. Conclusions: The ACMT can be used to compile environmental measures to study the influence of environmental context on population health. The portable and flexible nature of ACMT makes it optimal for neighborhood study research seeking to attribute environmental data to specific locations within the United States. UR - https://formative.jmir.org/2024/1/e56510 UR - http://dx.doi.org/10.2196/56510 UR - http://www.ncbi.nlm.nih.gov/pubmed/39365663 ID - info:doi/10.2196/56510 ER - TY - JOUR AU - Bennett, Verity AU - Spasi?, Irena AU - Filimonov, Maxim AU - Muralidaran, Vigneshwaran AU - Kemp, Mary Alison AU - Allen, Stuart AU - Watkins, John William PY - 2024/9/26 TI - Assessing the Feasibility of Using Parents? Social Media Conversations to Inform Burn First Aid Interventions: Mixed Methods Study JO - JMIR Form Res SP - e48695 VL - 8 KW - social media KW - burn first aid KW - health interventions KW - parents KW - burns N2 - Background: Burns are common childhood injuries, which can lead to serious physical and psychological outcomes. Appropriate first aid is essential in managing the pain and severity of these injuries; hence, parents who need timely access to such information often seek it from the web. In particular, social media allow them to reach other parents, hence these conversations may provide insight to aid the design and evaluation of burn first aid interventions for parents. Objective: This study aims to determine the feasibility of finding, accessing, and analyzing parent burn first aid conversations on social media to inform intervention research. Methods: The initial choice of the relevant social media was made based on the results of a parent focus group and survey. We considered Facebook (Meta Platforms, Inc), Mumsnet (Mumsnet Limited), Netmums (Aufeminin Group), Twitter (subsequently rebranded as ?X?; X Corp), Reddit (Reddit, Inc), and YouTube (Google LLC). To locate the relevant data on these platforms, we collated a taxonomy of search terms and designed a search strategy. A combination of natural language processing and manual inspection was used to filter out irrelevant data. The remaining data were analyzed manually to determine the length of conversations, the number of participants, the purpose of the initial post (eg, asking for or offering advice), burn types, and distribution of relevant keywords. Results: Facebook parenting groups were not accessed due to privacy, and public influencer pages yielded scant data. No relevant data were found on Reddit. Data were collected from Mumsnet, Netmums, YouTube, and Twitter. The amount of available data varied across these platforms and through time. Sunburn was identified as a topic across all 4 platforms. Conversations on the parenting forums Mumsnet and Netmums were started predominantly to seek advice (112/116, 96.6% and 25/25, 100%, respectively). Conversely, YouTube and Twitter were used mainly to provide advice (362/328, 94.8% and 126/197, 64%, respectively). Contact burns and sunburn were the most frequent burn types discussed on Mumsnet (30/94, 32% and 23/94, 25%, respectively) and Netmums (2/25, 8% and 14/26, 56%, respectively). Conclusions: This study provides a suite of bespoke search strategies, tailored to a range of social media platforms, for the extraction and analysis of burn first aid conversation data. Our methodology provides a template for other topics not readily accessible via a specific search term or hashtag. YouTube and Twitter show potential utility in measuring advice offered before and after interventions and extending the reach of messaging. Mumsnet and Netmums present the best opportunity for informing burn first aid intervention design via an in-depth qualitative investigation into parents? knowledge, attitudes, and behaviors. UR - https://formative.jmir.org/2024/1/e48695 UR - http://dx.doi.org/10.2196/48695 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/48695 ER - TY - JOUR AU - Lim, Sachiko AU - Johannesson, Paul PY - 2024/9/26 TI - An Ontology to Bridge the Clinical Management of Patients and Public Health Responses for Strengthening Infectious Disease Surveillance: Design Science Study JO - JMIR Form Res SP - e53711 VL - 8 KW - infectious disease KW - ontology KW - IoT KW - infectious disease surveillance KW - patient monitoring KW - infectious disease management KW - risk analysis KW - early warning KW - data integration KW - semantic interoperability KW - public health N2 - Background: Novel surveillance approaches using digital technologies, including the Internet of Things (IoT), have evolved, enhancing traditional infectious disease surveillance systems by enabling real-time detection of outbreaks and reaching a wider population. However, disparate, heterogenous infectious disease surveillance systems often operate in silos due to a lack of interoperability. As a life-changing clinical use case, the COVID-19 pandemic has manifested that a lack of interoperability can severely inhibit public health responses to emerging infectious diseases. Interoperability is thus critical for building a robust ecosystem of infectious disease surveillance and enhancing preparedness for future outbreaks. The primary enabler for semantic interoperability is ontology. Objective: This study aims to design the IoT-based management of infectious disease ontology (IoT-MIDO) to enhance data sharing and integration of data collected from IoT-driven patient health monitoring, clinical management of individual patients, and disparate heterogeneous infectious disease surveillance. Methods: The ontology modeling approach was chosen for its semantic richness in knowledge representation, flexibility, ease of extensibility, and capability for knowledge inference and reasoning. The IoT-MIDO was developed using the basic formal ontology (BFO) as the top-level ontology. We reused the classes from existing BFO-based ontologies as much as possible to maximize the interoperability with other BFO-based ontologies and databases that rely on them. We formulated the competency questions as requirements for the ontology to achieve the intended goals. Results: We designed an ontology to integrate data from heterogeneous sources, including IoT-driven patient monitoring, clinical management of individual patients, and infectious disease surveillance systems. This integration aims to facilitate the collaboration between clinical care and public health domains. We also demonstrate five use cases using the simplified ontological models to show the potential applications of IoT-MIDO: (1) IoT-driven patient monitoring, risk assessment, early warning, and risk management; (2) clinical management of patients with infectious diseases; (3) epidemic risk analysis for timely response at the public health level; (4) infectious disease surveillance; and (5) transforming patient information into surveillance information. Conclusions: The development of the IoT-MIDO was driven by competency questions. Being able to answer all the formulated competency questions, we successfully demonstrated that our ontology has the potential to facilitate data sharing and integration for orchestrating IoT-driven patient health monitoring in the context of an infectious disease epidemic, clinical patient management, infectious disease surveillance, and epidemic risk analysis. The novelty and uniqueness of the ontology lie in building a bridge to link IoT-based individual patient monitoring and early warning based on patient risk assessment to infectious disease epidemic surveillance at the public health level. The ontology can also serve as a starting point to enable potential decision support systems, providing actionable insights to support public health organizations and practitioners in making informed decisions in a timely manner. UR - https://formative.jmir.org/2024/1/e53711 UR - http://dx.doi.org/10.2196/53711 UR - http://www.ncbi.nlm.nih.gov/pubmed/39325530 ID - info:doi/10.2196/53711 ER - TY - JOUR AU - Goehringer, Jessica AU - Kosmin, Abigail AU - Laible, Natalie AU - Romagnoli, Katrina PY - 2024/9/26 TI - Assessing the Utility of a Patient-Facing Diagnostic Tool Among Individuals With Hypermobile Ehlers-Danlos Syndrome: Focus Group Study JO - JMIR Form Res SP - e49720 VL - 8 KW - diagnostic tool KW - hypermobile Ehlers-Danlos syndrome KW - patient experiences KW - diagnostic odyssey KW - affinity mapping KW - mobile health app KW - mobile phone N2 - Background: Hypermobile Ehlers-Danlos syndrome (hEDS), characterized by joint hypermobility, skin laxity, and tissue fragility, is thought to be the most common inherited connective tissue disorder, with millions affected worldwide. Diagnosing this condition remains a challenge that can impact quality of life for individuals with hEDS. Many with hEDS describe extended diagnostic odysseys involving exorbitant time and monetary investment. This delay is due to the complexity of diagnosis, symptom overlap with other conditions, and limited access to providers. Many primary care providers are unfamiliar with hEDS, compounded by genetics clinics that do not accept referrals for hEDS evaluation and long waits for genetics clinics that do evaluate for hEDS, leaving patients without sufficient options. Objective: This study explored the user experience, quality, and utility of a prototype of a patient-facing diagnostic tool intended to support clinician diagnosis for individuals with symptoms of hEDS. The questions included within the prototype are aligned with the 2017 international classification of Ehlers-Danlos syndromes. This study explored how this tool may help patients communicate information about hEDS to their physicians, influencing the diagnosis of hEDS and affecting patient experience. Methods: Participants clinically diagnosed with hEDS were recruited from either a medical center or private groups on a social media platform. Interested participants provided verbal consent, completed questionnaires about their diagnosis, and were invited to join an internet-based focus group to share their thoughts and opinions on a diagnostic tool prototype. Participants were invited to complete the Mobile App Rating Scale (MARS) to evaluate their experience viewing the diagnostic tool. The MARS is a framework for evaluating mobile health apps across 4 dimensions: engagement, functionality, esthetics, and information quality. Qualitative data were analyzed using affinity mapping to organize information and inductively create themes that were categorized within the MARS framework dimensions to help identify strengths and weaknesses of the diagnostic tool prototype. Results: In total, 15 individuals participated in the internet-based focus groups; 3 (20%) completed the MARS. Through affinity diagramming, 2 main categories of responses were identified, including responses related to the user interface and responses related to the application of the tool. Each category included several themes and subthemes that mapped well to the 4 MARS dimensions. The analysis showed that the tool held value and utility among the participants diagnosed with hEDS. The shareable ending summary sheet provided by the tool stood out as a strength for facilitating communication between patient and provider during the diagnostic evaluation. Conclusions: The results provide insights on the perceived utility and value of the tool, including preferred phrasing, layout and design preferences, and tool accessibility. The participants expressed that the tool may improve the hEDS diagnostic odyssey and help educate providers about the diagnostic process. UR - https://formative.jmir.org/2024/1/e49720 UR - http://dx.doi.org/10.2196/49720 UR - http://www.ncbi.nlm.nih.gov/pubmed/39325533 ID - info:doi/10.2196/49720 ER - TY - JOUR AU - Rossi, S. Fernanda AU - Wu, Justina AU - Timko, Christine AU - Nevedal, L. Andrea AU - Wiltsey Stirman, Shannon PY - 2024/9/25 TI - A Clinical Decision Support Tool for Intimate Partner Violence Screening Among Women Veterans: Development and Qualitative Evaluation of Provider Perspectives JO - JMIR Form Res SP - e57633 VL - 8 KW - intimate partner violence KW - clinical decision support KW - intimate partner violence screening KW - women veterans N2 - Background: Women veterans, compared to civilian women, are especially at risk of experiencing intimate partner violence (IPV), pointing to the critical need for IPV screening and intervention in the Veterans Health Administration (VHA). However, implementing paper-based IPV screening and intervention in the VHA has revealed substantial barriers, including health care providers? inadequate IPV training, competing demands, time constraints, and discomfort addressing IPV and making decisions about the appropriate type or level of intervention. Objective: This study aimed to address IPV screening implementation barriers and hence developed and tested a novel IPV clinical decision support (CDS) tool for physicians in the Women?s Health Clinic (WHC), a primary care clinic within the Veterans Affairs Palo Alto Health Care System. This tool provides intelligent, evidence-based, step-by-step guidance on how to conduct IPV screening and intervention. Methods: Informed by existing CDS development frameworks, developing the IPV CDS tool prototype involved six steps: (1) identifying the scope of the tool, (2) identifying IPV screening and intervention content, (3) incorporating IPV-related VHA and clinic resources, (4) identifying the tool?s components, (5) designing the tool, and (6) conducting initial tool revisions. We obtained preliminary physician feedback on user experience and clinical utility of the CDS tool via the System Usability Scale (SUS) and semistructured interviews with 6 WHC physicians. SUS scores were examined using descriptive statistics. Interviews were analyzed using rapid qualitative analysis to extract actionable feedback to inform design updates and improvements. Results: This study includes a detailed description of the IPV CDS tool. Findings indicated that the tool was generally well received by physicians, who indicated good tool usability (SUS score: mean 77.5, SD 12.75). They found the tool clinically useful, needed in their practice, and feasible to implement in primary care. They emphasized that it increased their confidence in managing patients reporting IPV but expressed concerns regarding its length, workflow integration, flexibility, and specificity of information. Several physicians, for example, found the tool too time consuming when encountering patients at high risk; they suggested multiple uses of the tool (eg, an educational tool for less-experienced health care providers and a checklist for more-experienced health care providers) and including more detailed information (eg, a list of local shelters). Conclusions: Physician feedback on the IPV CDS tool is encouraging and will be used to improve the tool. This study offers an example of an IPV CDS tool that clinics can adapt to potentially enhance the quality and efficiency of their IPV screening and intervention process. Additional research is needed to determine the tool?s clinical utility in improving IPV screening and intervention rates and patient outcomes (eg, increased patient safety, reduced IPV risk, and increased referrals to mental health treatment). UR - https://formative.jmir.org/2024/1/e57633 UR - http://dx.doi.org/10.2196/57633 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57633 ER - TY - JOUR AU - French, Blandine AU - Babbage, Camilla AU - Bird, Katherine AU - Marsh, Lauren AU - Pelton, Mirabel AU - Patel, Shireen AU - Cassidy, Sarah AU - Rennick-Egglestone, Stefan PY - 2024/9/16 TI - Data Integrity Issues With Web-Based Studies: An Institutional Example of a Widespread Challenge JO - JMIR Ment Health SP - e58432 VL - 11 KW - web-based research KW - web-based studies KW - qualitative studies KW - surveys KW - mental health KW - data integrity, misrepresentation UR - https://mental.jmir.org/2024/1/e58432 UR - http://dx.doi.org/10.2196/58432 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58432 ER - TY - JOUR AU - Swain, Alexander T. AU - McNarry, A. Melitta AU - Mackintosh, A. Kelly PY - 2024/9/13 TI - Assessing and Enhancing Movement Quality Using Wearables and Consumer Technologies: Thematic Analysis of Expert Perspectives JO - JMIR Form Res SP - e56784 VL - 8 KW - physical activity KW - exercise KW - wellness KW - qualitative KW - sensors KW - motor skill KW - motor learning KW - movement skills KW - skill development KW - movement assessment N2 - Background: Improvements in movement quality (ie, how well an individual moves) facilitate increases in movement quantity, subsequently improving general health and quality of life. Wearable technology offers a convenient, affordable means of measuring and assessing movement quality for the general population, while technology more broadly can provide constructive feedback through various modalities. Considering the perspectives of professionals involved in the development and implementation of technology helps translate user needs into effective strategies for the optimal application of consumer technologies to enhance movement quality. Objective: This study aimed to obtain the opinions of wearable technology experts regarding the use of wearable devices to measure movement quality and provide feedback. A secondary objective was to determine potential strategies for integrating preferred assessment and feedback characteristics into a technology-based movement quality intervention for the general, recreationally active population. Methods: Semistructured interviews were conducted with 12 participants (age: mean 42, SD 9 years; 5 males) between August and September 2022 using a predetermined interview schedule. Participants were categorized based on their professional roles: commercial (n=4) and research and development (R&D; n=8). All participants had experience in the development or application of wearable technology for sports, exercise, and wellness. The verbatim interview transcripts were analyzed using reflexive thematic analysis in QSR NVivo (release 1.7), resulting in the identification of overarching themes and subthemes. Results: Three main themes were generated as follows: (1) ?Grab and Go,? (2) ?Adjust and Adapt,? and (3) ?Visualize and Feedback.? Participants emphasized the importance of convenience to enhance user engagement when using wearables to collect movement data. However, it was suggested that users would tolerate minor inconveniences if the benefits were perceived as valuable. Simple, easily interpretable feedback was recommended to accommodate diverse audiences and aid understanding of their movement quality, while avoiding excessive detail was advised to prevent overload, which could deter users. Adaptability was endorsed to accommodate progressions in user movement quality, and customizable systems were advocated to offer variety, thereby increasing user interest and engagement. The findings indicate that visual feedback representative of the user (ie, an avatar) should be used, supplemented with concise text or audible instructions to form a comprehensive, multimodal feedback system. Conclusions: The study provides insights from wearable technology experts on the use of consumer technologies for enhancing movement quality. The findings recommend the prioritization of user convenience and simplistic, multimodal feedback centered around visualizations, and an adaptable system suitable for a diverse audience. Emphasizing individualized feedback and user-centric design, this study provides valuable findings around the use of wearables and other consumer technologies to enhance movement quality among the general population. These findings, in conjunction with those of future research into user perspectives, should be applied in practical settings to evaluate their effectiveness in enhancing movement quality. UR - https://formative.jmir.org/2024/1/e56784 UR - http://dx.doi.org/10.2196/56784 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56784 ER - TY - JOUR AU - Tesfaye, Wubshet AU - Jordan, Margaret AU - Chen, F. Timothy AU - Castelino, Lynel Ronald AU - Sud, Kamal AU - Dabliz, Racha AU - Aslani, Parisa PY - 2024/9/12 TI - Usability Evaluation Methods Used in Electronic Discharge Summaries: Literature Review JO - J Med Internet Res SP - e55247 VL - 26 KW - electronic discharge summaries KW - usability testing KW - heuristic evaluation KW - heuristics, think-aloud KW - adoption KW - digital health KW - usability KW - electronic KW - discharge summary KW - end users KW - evaluation KW - user-centered N2 - Background: With the widespread adoption of digital health records, including electronic discharge summaries (eDS), it is important to assess their usability in order to understand whether they meet the needs of the end users. While there are established approaches for evaluating the usability of electronic health records, there is a lack of knowledge regarding suitable evaluation methods specifically for eDS. Objective: This literature review aims to identify the usability evaluation approaches used in eDS. Methods: We conducted a comprehensive search of PubMed, CINAHL, Web of Science, ACM Digital Library, MEDLINE, and ProQuest databases from their inception until July 2023. The study information was extracted and reported in accordance with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses). We included studies that assessed the usability of eDS, and the systems used to display eDS. Results: A total of 12 records, including 11 studies and 1 thesis, met the inclusion criteria. The included studies used qualitative, quantitative, or mixed methods approaches and reported the use of various usability evaluation methods. Heuristic evaluation was the most used method to assess the usability of eDS systems (n=7), followed by the think-aloud approach (n=5) and laboratory testing (n=3). These methods were used either individually or in combination with usability questionnaires (n=3) and qualitative semistructured interviews (n=4) for evaluating eDS usability issues. The evaluation processes incorporated usability metrics such as user performance, satisfaction, efficiency, and impact rating. Conclusions: There are a limited number of studies focusing on usability evaluations of eDS. The identified studies used expert-based and user-centered approaches, which can be used either individually or in combination to identify usability issues. However, further research is needed to determine the most appropriate evaluation method which can assess the fitness for purpose of discharge summaries. UR - https://www.jmir.org/2024/1/e55247 UR - http://dx.doi.org/10.2196/55247 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55247 ER - TY - JOUR AU - Tak, Won Yae AU - Lee, Won Jong AU - Kim, Junetae AU - Lee, Yura PY - 2024/9/9 TI - Predicting Long-Term Engagement in mHealth Apps: Comparative Study of Engagement Indices JO - J Med Internet Res SP - e59444 VL - 26 KW - treatment adherence and compliance KW - patient compliance KW - medication adherence KW - digital therapeutics KW - engagement index KW - mobile phone N2 - Background: Digital health care apps, including digital therapeutics, have the potential to increase accessibility and improve patient engagement by overcoming the limitations of traditional facility-based medical treatments. However, there are no established tools capable of quantitatively measuring long-term engagement at present. Objective: This study aimed to evaluate an existing engagement index (EI) in a commercial health management app for long-term use and compare it with a newly developed EI. Methods: Participants were recruited from cancer survivors enrolled in a randomized controlled trial that evaluated the impact of mobile health apps on recovery. Of these patients, 240 were included in the study and randomly assigned to the Noom app (Noom Inc). The newly developed EI was compared with the existing EI, and a long-term use analysis was conducted. Furthermore, the new EI was evaluated based on adapted measurements from the Web Matrix Visitor Index, focusing on click depth, recency, and loyalty indices. Results: The newly developed EI model outperformed the existing EI model in terms of predicting EI of a 6- to 9-month period based on the EI of a 3- to 6-month period. The existing model had a mean squared error of 0.096, a root mean squared error of 0.310, and an R2 of 0.053. Meanwhile, the newly developed EI models showed improved performance, with the best one achieving a mean squared error of 0.025, root mean squared error of 0.157, and R2 of 0.610. The existing EI exhibited significant associations: the click depth index (hazard ratio [HR] 0.49, 95% CI 0.29-0.84; P<.001) and loyalty index (HR 0.17, 95% CI 0.09-0.31; P<.001) were significantly associated with improved survival, whereas the recency index exhibited no significant association (HR 1.30, 95% CI 1.70-2.42; P=.41). Among the new EI models, the EI with a menu combination of menus available in the app?s free version yielded the most promising result. Furthermore, it exhibited significant associations with the loyalty index (HR 0.32, 95% CI 0.16-0.62; P<.001) and the recency index (HR 0.47, 95% CI 0.30-0.75; P<.001). Conclusions: The newly developed EI model outperformed the existing model in terms of the prediction of long-term user engagement and compliance in a mobile health app context. We emphasized the importance of log data and suggested avenues for future research to address the subjectivity of the EI and incorporate a broader range of indices for comprehensive evaluation. UR - https://www.jmir.org/2024/1/e59444 UR - http://dx.doi.org/10.2196/59444 UR - http://www.ncbi.nlm.nih.gov/pubmed/39250192 ID - info:doi/10.2196/59444 ER - TY - JOUR AU - Akyon, Handan Seyma AU - Akyon, Cagatay Fatih AU - Camyar, Sefa Ahmet AU - H?zl?, Fatih AU - Sari, Talha AU - H?zl?, ?amil PY - 2024/9/4 TI - Evaluating the Capabilities of Generative AI Tools in Understanding Medical Papers: Qualitative Study JO - JMIR Med Inform SP - e59258 VL - 12 KW - large language models KW - LLM KW - LLMs KW - ChatGPT KW - artificial intelligence KW - AI KW - natural language processing KW - medicine KW - health care KW - GPT KW - machine learning KW - language model KW - language models KW - generative KW - research paper KW - research papers KW - scientific research KW - answer KW - answers KW - response KW - responses KW - comprehension KW - STROBE KW - Strengthening the Reporting of Observational Studies in Epidemiology N2 - Background: Reading medical papers is a challenging and time-consuming task for doctors, especially when the papers are long and complex. A tool that can help doctors efficiently process and understand medical papers is needed. Objective: This study aims to critically assess and compare the comprehension capabilities of large language models (LLMs) in accurately and efficiently understanding medical research papers using the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist, which provides a standardized framework for evaluating key elements of observational study. Methods: The study is a methodological type of research. The study aims to evaluate the understanding capabilities of new generative artificial intelligence tools in medical papers. A novel benchmark pipeline processed 50 medical research papers from PubMed, comparing the answers of 6 LLMs (GPT-3.5-Turbo, GPT-4-0613, GPT-4-1106, PaLM 2, Claude v1, and Gemini Pro) to the benchmark established by expert medical professors. Fifteen questions, derived from the STROBE checklist, assessed LLMs? understanding of different sections of a research paper. Results: LLMs exhibited varying performance, with GPT-3.5-Turbo achieving the highest percentage of correct answers (n=3916, 66.9%), followed by GPT-4-1106 (n=3837, 65.6%), PaLM 2 (n=3632, 62.1%), Claude v1 (n=2887, 58.3%), Gemini Pro (n=2878, 49.2%), and GPT-4-0613 (n=2580, 44.1%). Statistical analysis revealed statistically significant differences between LLMs (P<.001), with older models showing inconsistent performance compared to newer versions. LLMs showcased distinct performances for each question across different parts of a scholarly paper?with certain models like PaLM 2 and GPT-3.5 showing remarkable versatility and depth in understanding. Conclusions: This study is the first to evaluate the performance of different LLMs in understanding medical papers using the retrieval augmented generation method. The findings highlight the potential of LLMs to enhance medical research by improving efficiency and facilitating evidence-based decision-making. Further research is needed to address limitations such as the influence of question formats, potential biases, and the rapid evolution of LLM models. UR - https://medinform.jmir.org/2024/1/e59258 UR - http://dx.doi.org/10.2196/59258 UR - http://www.ncbi.nlm.nih.gov/pubmed/39230947 ID - info:doi/10.2196/59258 ER - TY - JOUR AU - Liu, Jingkun AU - Tai, Jiaojiao AU - Han, Junying AU - Zhang, Meng AU - Li, Yang AU - Yang, Hongjuan AU - Yan, Ziqiang PY - 2024/9/4 TI - Constructing a Hospital Department Development?Level Assessment Model: Machine Learning and Expert Consultation Approach in Complex Hospital Data Environments JO - JMIR Form Res SP - e54638 VL - 8 KW - machine algorithms KW - hospital management KW - model construction KW - support vector machine KW - clustering N2 - Background: Every hospital manager aims to build harmonious, mutually beneficial, and steady-state departments. Therefore, it is important to explore a hospital department development assessment model based on objective hospital data. Objective: This study aims to use a novel machine learning algorithm to identify key evaluation indexes for hospital departments, offering insights for strategic planning and resource allocation in hospital management. Methods: Data related to the development of a hospital department over the past 3 years were extracted from various hospital information systems. The resulting data set was mined using neural machine algorithms to assess the possible role of hospital departments in the development of a hospital. A questionnaire was used to consult senior experts familiar with the hospital to assess the actual work in each hospital department and the impact of each department?s development on overall hospital discipline. We used the results from this questionnaire to verify the accuracy of the departmental risk scores calculated by the machine learning algorithm. Results: Deep machine learning was performed and modeled on the hospital system training data set. The model successfully leveraged the hospital?s training data set to learn, predict, and evaluate the working and development of hospital departments. A comparison of the questionnaire results with the risk ranking set from the departments machine learning algorithm using the cosine similarity algorithm and Pearson correlation analysis showed a good match. This indicates that the department development assessment model and risk score based on the objective data of hospital systems are relatively accurate and objective. Conclusions: This study demonstrated that our machine learning algorithm provides an accurate and objective assessment model for hospital department development. The strong alignment of the model's risk assessments with expert opinions, validated through statistical analysis, highlights its reliability and potential to guide strategic hospital management decisions. UR - https://formative.jmir.org/2024/1/e54638 UR - http://dx.doi.org/10.2196/54638 UR - http://www.ncbi.nlm.nih.gov/pubmed/39230941 ID - info:doi/10.2196/54638 ER - TY - JOUR AU - Hawkins, T. Alexander AU - Fa, Andrea AU - Younan, A. Samuel AU - Ivatury, Joga Srinivas AU - Bonnet, Kemberlee AU - Schlundt, David AU - Gordon, J. Elisa AU - Cavanaugh, L. Kerri PY - 2024/9/3 TI - Decision Aid for Colectomy in Recurrent Diverticulitis: Development and Usability Study JO - JMIR Form Res SP - e59952 VL - 8 KW - design sprint KW - diverticulitis KW - decision aid KW - shared decision-making KW - colectomy KW - decision-making KW - diverticular diseases KW - gastrointestinal diagnosis KW - American KW - America KW - tools KW - tool KW - effectiveness KW - surgeon KW - patients KW - patient KW - communication KW - synopsis N2 - Background: Diverticular disease is a common gastrointestinal diagnosis with over 2.7 million clinic visits yearly. National guidelines from the American Society of Colon and Rectal Surgeons state that ?the decision to recommend elective sigmoid colectomy after recovery from uncomplicated acute diverticulitis should be individualized.? However, tools to individualize this decision are lacking. Objective: This study aimed to develop an online educational decision aid (DA) to facilitate effective surgeon and patient communication about treatment options for recurrent left-sided diverticulitis. Methods: We used a modified design sprint methodology to create a prototype DA. We engaged a multidisciplinary team and adapted elements from the Ottawa Personal Decision Guide. We then iteratively refined the prototype by conducting a mixed methods assessment of content and usability testing, involving cognitive interviews with patients and surgeons. The findings informed the refinement of the DA. Further testing included an in-clinic feasibility review. Results: Over a 4-day in-person rapid design sprint, including patients, surgeons, and health communication experts, we developed a prototype of a diverticulitis DA, comprising an interactive website and handout with 3 discrete sections. The first section contains education about diverticulitis and treatment options. The second section clarifies the potential risks and benefits of both clinical treatment options (medical management vs colectomy). The third section invites patients to participate in a value clarification exercise. After navigating the DA, the patient prints a synopsis that they bring to their clinic appointment, which serves as a guide for shared decision-making. Conclusions: Design sprint methodology, emphasizing stakeholder co-design and complemented by extensive user testing, is an effective and efficient strategy to create a DA for patients living with recurrent diverticulitis facing critical treatment decisions. UR - https://formative.jmir.org/2024/1/e59952 UR - http://dx.doi.org/10.2196/59952 UR - http://www.ncbi.nlm.nih.gov/pubmed/39226090 ID - info:doi/10.2196/59952 ER - TY - JOUR AU - Wirtz, L. Andrea AU - Poteat, Tonia AU - Borquez, Annick AU - Linton, Sabriya AU - Stevenson, Megan AU - Case, James AU - Brown, Carter AU - Lint, Arianna AU - Miller, Marissa AU - Radix, Asa AU - Althoff, N. Keri AU - Schneider, S. Jason AU - Haw, Sonya J. AU - Wawrzyniak, J. Andrew AU - Rodriguez, Allan AU - Cooney, Erin AU - Humes, Elizabeth AU - Pontes, Ceza AU - Seopaul, Shannon AU - White, Camille AU - Beyrer, Chris AU - Reisner, L. Sari AU - PY - 2024/8/27 TI - Enhanced Cohort Methods for HIV Research and Epidemiology (ENCORE): Protocol for a Nationwide Hybrid Cohort for Transgender Women in the United States JO - JMIR Res Protoc SP - e59846 VL - 13 KW - transgender women KW - cohort KW - United States KW - HIV KW - digital methods N2 - Background: In the United States, transgender women are disproportionately impacted by HIV and prioritized in the national strategy to end the epidemic. Individual, interpersonal, and structural vulnerabilities underlie HIV acquisition among transgender women and fuel syndemic conditions, yet no nationwide cohort monitors their HIV and other health outcomes. Objective: Our objective is to develop a nationwide cohort to estimate HIV incidence, identify risk factors, and investigate syndemic conditions co-occurring with HIV vulnerability or acquisition among US transgender women. The study is informed by the Syndemics Framework and the Social Ecological Model, positing that stigma-related conditions are synergistically driven by shared multilevel vulnerabilities. Methods: To address logistical and cost challenges while minimizing technology barriers and research distrust, we aim to establish a novel, hybrid community hub?supported digital cohort (N=3000). The digital cohort is the backbone of the study and is enhanced by hubs strategically located across the United States for increased engagement and in-person support. Study participants are English or Spanish speakers, are aged ?18 years, identify as transgender women or along the transfeminine spectrum, reside in 1 of the 50 states or Puerto Rico, and do not have HIV (laboratory confirmed). Participants are followed for 24 months, with semiannual assessments. These include a questionnaire and laboratory-based HIV testing using self-collected specimens. Using residential zip codes, person-level data will be merged with contextual geolocated data, including population health measures and economic, housing, and other social and structural factors. Analyses will (1) evaluate the contribution of hub support to the digital cohort using descriptive statistics; (2) estimate and characterize syndemic patterns among transgender women using latent class analysis; (3) examine the role of contextual factors in driving syndemics and HIV prevention over time using multilevel regression models; (4) estimate HIV incidence in transgender women and examine the effect of syndemics and contextual factors on HIV incidence using Poisson regression models; and (5) develop dynamic, compartmental models of multilevel combination HIV prevention interventions among transgender women to simulate their impact on HIV incidence through 2030. Results: Enrollment launched on March 15, 2023, with data collection phases occurring in spring and fall. As of February 24, 2024, a total of 3084 individuals were screened, and 996 (32.3%) met the inclusion criteria and enrolled into the cohort: 2.3% (23/996) enrolled at a hub, and 53.6% (534/996) enrolled through a community hub?supported strategy. Recruitment through purely digital methods contributed 61.5% (1895/3084) of those screened and 42.7% (425/996) of those enrolled in the cohort. Conclusions: Study findings will inform the development of evidence-based interventions to reduce HIV acquisition and syndemic conditions among US transgender women and advance efforts to end the US HIV epidemic. Methodological findings will also have critical implications for the design of future innovative approaches to HIV research. International Registered Report Identifier (IRRID): DERR1-10.2196/59846 UR - https://www.researchprotocols.org/2024/1/e59846 UR - http://dx.doi.org/10.2196/59846 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59846 ER - TY - JOUR AU - Milliken, Tabitha AU - Beiler, Donielle AU - Hoffman, Samantha AU - Olenginski, Ashlee AU - Troiani, Vanessa PY - 2024/8/22 TI - Recruitment in Appalachian, Rural and Older Adult Populations in an Artificial Intelligence World: Study Using Human-Mediated Follow-Up JO - JMIR Form Res SP - e38189 VL - 8 KW - telecommunication KW - enrollment rate KW - Northern Appalachia KW - web-based KW - aging KW - recruitment KW - rural N2 - Background: Participant recruitment in rural and hard-to-reach (HTR) populations can present unique challenges. These challenges are further exacerbated by the need for low-cost recruiting, which often leads to use of web-based recruitment methods (eg, email, social media). Despite these challenges, recruitment strategy statistics that support effective enrollment strategies for underserved and HTR populations are underreported. This study highlights how a recruitment strategy that uses email in combination with follow-up, mostly phone calls and email reminders, produced a higher-than-expected enrollment rate that includes a diversity of participants from rural, Appalachian populations in older age brackets and reports recruitment and demographic statistics within a subset of HTR populations. Objective: This study aims to provide evidence that a recruitment strategy that uses a combination of email, telephonic, and follow-up recruitment strategies increases recruitment rates in various HTR populations, specifically in rural, older, and Appalachian populations. Methods: We evaluated the overall enrollment rate of 1 recruitment arm of a larger study that aims to understand the relationship between genetics and substance use disorders. We evaluated the enrolled population?s characteristics to determine recruitment success of a combined email and follow-up recruitment strategy, and the enrollment rate of HTR populations. These characteristics included (1) enrollment rate before versus after follow-up; (2) zip code and county of enrollee to determine rural or urban and Appalachian status; (3) age to verify recruitment in all eligible age brackets; and (4) sex distribution among age brackets and rural or urban status. Results: The email and follow-up arm of the study had a 17.4% enrollment rate. Of the enrolled participants, 76.3% (4602/6030) lived in rural counties and 23.7% (1428/6030) lived in urban counties in Pennsylvania. In addition, of patients enrolled, 98.7% (5956/6030) were from Appalachian counties and 1.3% (76/6030) were from non-Appalachian counties. Patients from rural Appalachia made up 76.2% (4603/6030) of the total rural population. Enrolled patients represented all eligible age brackets from ages 20 to 75 years, with the 60-70 years age bracket having the most enrollees. Females made up 72.5% (4371/6030) of the enrolled population and males made up 27.5% (1659/6030) of the population. Conclusions: Results indicate that a web-based recruitment method with participant follow-up, such as a phone call and email follow-up, increases enrollment numbers more than web-based methods alone for rural, Appalachian, and older populations. Adding a humanizing component, such as a live person phone call, may be a key element needed to establish trust and encourage patients from underserved and rural areas to enroll in studies via web-based recruitment methods. Supporting statistics on this recruitment strategy should help researchers identify whether this strategy may be useful in future studies and HTR populations. UR - https://formative.jmir.org/2024/1/e38189 UR - http://dx.doi.org/10.2196/38189 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/38189 ER - TY - JOUR AU - Li, Danis Kevin AU - Fernandez, M. Adrian AU - Schwartz, Rachel AU - Rios, Natalie AU - Carlisle, Nathaniel Marvin AU - Amend, M. Gregory AU - Patel, V. Hiren AU - Breyer, N. Benjamin PY - 2024/8/21 TI - Comparing GPT-4 and Human Researchers in Health Care Data Analysis: Qualitative Description Study JO - J Med Internet Res SP - e56500 VL - 26 KW - artificial intelligence KW - ChatGPT KW - large language models KW - qualitative analysis KW - content analysis KW - buried penis KW - qualitative interviews KW - qualitative description KW - urology N2 - Background: Large language models including GPT-4 (OpenAI) have opened new avenues in health care and qualitative research. Traditional qualitative methods are time-consuming and require expertise to capture nuance. Although large language models have demonstrated enhanced contextual understanding and inferencing compared with traditional natural language processing, their performance in qualitative analysis versus that of humans remains unexplored. Objective: We evaluated the effectiveness of GPT-4 versus human researchers in qualitative analysis of interviews with patients with adult-acquired buried penis (AABP). Methods: Qualitative data were obtained from semistructured interviews with 20 patients with AABP. Human analysis involved a structured 3-stage process?initial observations, line-by-line coding, and consensus discussions to refine themes. In contrast, artificial intelligence (AI) analysis with GPT-4 underwent two phases: (1) a naïve phase, where GPT-4 outputs were independently evaluated by a blinded reviewer to identify themes and subthemes and (2) a comparison phase, where AI-generated themes were compared with human-identified themes to assess agreement. We used a general qualitative description approach. Results: The study population (N=20) comprised predominantly White (17/20, 85%), married (12/20, 60%), heterosexual (19/20, 95%) men, with a mean age of 58.8 years and BMI of 41.1 kg/m2. Human qualitative analysis identified ?urinary issues? in 95% (19/20) and GPT-4 in 75% (15/20) of interviews, with the subtheme ?spray or stream? noted in 60% (12/20) and 35% (7/20), respectively. ?Sexual issues? were prominent (19/20, 95% humans vs 16/20, 80% GPT-4), although humans identified a wider range of subthemes, including ?pain with sex or masturbation? (7/20, 35%) and ?difficulty with sex or masturbation? (4/20, 20%). Both analyses similarly highlighted ?mental health issues? (11/20, 55%, both), although humans coded ?depression? more frequently (10/20, 50% humans vs 4/20, 20% GPT-4). Humans frequently cited ?issues using public restrooms? (12/20, 60%) as impacting social life, whereas GPT-4 emphasized ?struggles with romantic relationships? (9/20, 45%). ?Hygiene issues? were consistently recognized (14/20, 70% humans vs 13/20, 65% GPT-4). Humans uniquely identified ?contributing factors? as a theme in all interviews. There was moderate agreement between human and GPT-4 coding (?=0.401). Reliability assessments of GPT-4?s analyses showed consistent coding for themes including ?body image struggles,? ?chronic pain? (10/10, 100%), and ?depression? (9/10, 90%). Other themes like ?motivation for surgery? and ?weight challenges? were reliably coded (8/10, 80%), while less frequent themes were variably identified across multiple iterations. Conclusions: Large language models including GPT-4 can effectively identify key themes in analyzing qualitative health care data, showing moderate agreement with human analysis. While human analysis provided a richer diversity of subthemes, the consistency of AI suggests its use as a complementary tool in qualitative research. With AI rapidly advancing, future studies should iterate analyses and circumvent token limitations by segmenting data, furthering the breadth and depth of large language model?driven qualitative analyses. UR - https://www.jmir.org/2024/1/e56500 UR - http://dx.doi.org/10.2196/56500 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56500 ER - TY - JOUR AU - Klassen, A. Stephen AU - Jabbar, Jesica AU - Osborne, Jenna AU - Iannarelli, J. Nathaniel AU - Kirby, S. Emerson AU - O'Leary, D. Deborah AU - Locke, Sean PY - 2024/8/20 TI - Examining the Light Heart Mobile Device App for Assessing Human Pulse Interval and Heart Rate Variability: Validation Study JO - JMIR Form Res SP - e56921 VL - 8 KW - pulse interval KW - mobile app KW - validation KW - mHealth KW - mHealth app KW - app mobile device KW - mobile device app KW - pulse KW - heart KW - heart rate KW - validation study KW - biomarker KW - psychological KW - physiological KW - pulse rate KW - young adults KW - youth KW - linear correlation KW - heart rate variability KW - examining KW - examine KW - validity KW - psychological health KW - physiological health KW - interval data KW - mobile phone N2 - Background: Pulse interval is a biomarker of psychological and physiological health. Pulse interval can now be assessed using mobile phone apps, which expands researchers? ability to assess pulse interval in the real world. Prior to implementation, measurement accuracy should be established. Objective: This investigation evaluated the validity of the Light Heart mobile app to measure pulse interval and pulse rate variability in healthy young adults. Methods: Validity was assessed by comparing the pulse interval and SD of normal pulse intervals obtained by Light Heart to the gold standard, electrocardiogram (ECG), in 14 young healthy individuals (mean age 24, SD 5 years; n=9, 64% female) in a seated posture. Results: Mean pulse interval (Light Heart: 859, SD 113 ms; ECG: 857, SD 112 ms) demonstrated a strong positive linear correlation (r=0.99; P<.001) and strong agreement (intraclass correlation coefficient=1.00, 95% CI 0.99-1.00) between techniques. The Bland-Altman plot demonstrated good agreement for the mean pulse interval measured with Light Heart and ECG with evidence of fixed bias (?1.56, SD 1.86; 95% CI ?5.2 to 2.1 ms), suggesting that Light Heart overestimates pulse interval by a small margin. When Bland-Altman plots were constructed for each participant?s beat-by-beat pulse interval data, all participants demonstrated strong agreement between Light Heart and ECG with no evidence of fixed bias between measures. Heart rate variability, assessed by SD of normal pulse intervals, demonstrated strong agreement between techniques (Light Heart: mean 73, SD 23 ms; ECG: mean 73, SD 22 ms; r=0.99; P<.001; intraclass correlation coefficient=0.99, 95% CI 0.97-1.00). Conclusions: This study provides evidence to suggest that the Light Heart mobile app provides valid measures of pulse interval and heart rate variability in healthy young adults. UR - https://formative.jmir.org/2024/1/e56921 UR - http://dx.doi.org/10.2196/56921 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56921 ER - TY - JOUR AU - Wunderlich, Markus Maximilian AU - Krampe, Henning AU - Fuest, Kristina AU - Leicht, Dominik AU - Probst, Benedikt Moriz AU - Runge, Julian AU - Schmid, Sebastian AU - Spies, Claudia AU - Weiß, Björn AU - Balzer, Felix AU - Poncette, Akira-Sebastian AU - PY - 2024/8/8 TI - Evaluating the Construct Validity of the Charité Alarm Fatigue Questionnaire using Confirmatory Factor Analysis JO - JMIR Hum Factors SP - e57658 VL - 11 KW - patient monitoring KW - intensive care unit KW - alarm KW - alarms KW - validity KW - validation KW - safety KW - intensive KW - care KW - alarm fatigue KW - alarm management KW - patient safety KW - ICU KW - alarm system KW - alarm system quality KW - medical devices KW - clinical alarms KW - questionnaire KW - questionnaires KW - warning KW - factor analysis N2 - Background: The Charité Alarm Fatigue Questionnaire (CAFQa) is a 9-item questionnaire that aims to standardize how alarm fatigue in nurses and physicians is measured. We previously hypothesized that it has 2 correlated scales, one on the psychosomatic effects of alarm fatigue and the other on staff?s coping strategies in working with alarms. Objective: We aimed to validate the hypothesized structure of the CAFQa and thus underpin the instrument?s construct validity. Methods: We conducted 2 independent studies with nurses and physicians from intensive care units in Germany (study 1: n=265; study 2: n=1212). Responses to the questionnaire were analyzed using confirmatory factor analysis with the unweighted least-squares algorithm based on polychoric covariances. Convergent validity was assessed by participants? estimation of their own alarm fatigue and exposure to false alarms as a percentage. Results: In both studies, the ?2 test reached statistical significance (study 1: ?226=44.9; P=.01; study 2: ?226=92.4; P<.001). Other fit indices suggested a good model fit (in both studies: root mean square error of approximation <0.05, standardized root mean squared residual <0.08, relative noncentrality index >0.95, Tucker-Lewis index >0.95, and comparative fit index >0.995). Participants? mean scores correlated moderately with self-reported alarm fatigue (study 1: r=0.45; study 2: r=0.53) and weakly with self-perceived exposure to false alarms (study 1: r=0.3; study 2: r=0.33). Conclusions: The questionnaire measures the construct of alarm fatigue as proposed in our previous study. Researchers and clinicians can rely on the CAFQa to measure the alarm fatigue of nurses and physicians. Trial Registration: ClinicalTrials.gov NCT04994600; https://www.clinicaltrials.gov/study/NCT04994600 UR - https://humanfactors.jmir.org/2024/1/e57658 UR - http://dx.doi.org/10.2196/57658 ID - info:doi/10.2196/57658 ER - TY - JOUR AU - Bather, R. Jemar AU - Kaphingst, A. Kimberly AU - Goodman, S. Melody PY - 2024/8/8 TI - Racial Composition of Social Environments Over the Life Course Using the Pictorial Racial Composition Measure: Development and Validation Study JO - JMIR Public Health Surveill SP - e55461 VL - 10 KW - racial residential segregation KW - racial composition KW - health equity KW - social environment KW - place KW - neighborhood composition KW - health inequities KW - social determinants of health N2 - Background: Studies investigating the impact of racial segregation on health have reported mixed findings and tended to focus on the racial composition of neighborhoods. These studies use varying racial composition measures, such as census data or investigator-adapted questions, which are currently limited to assessing one dimension of neighborhood racial composition. Objective: This study aims to develop and validate a novel racial segregation measure, the Pictorial Racial Composition Measure (PRCM). Methods: The PRCM is a 10-item questionnaire of pictures representing social environments across adolescence and adulthood: neighborhoods and blocks (adolescent and current), schools and classrooms (junior high and high school), workplace, and place of worship. Cognitive interviews (n=13) and surveys (N=549) were administered to medically underserved patients at a primary care clinic at the Barnes-Jewish Hospital. Development of the PRCM occurred across pilot and main phases. For each social environment and survey phase (pilot and main), we computed positive versus negative pairwise comparisons: mostly Black versus all other categories, half Black versus all other categories, and mostly White versus all other categories. We calculated the following validity metrics for each pairwise comparison: sensitivity, specificity, correct classification rate, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, false positive rate, and false negative rate. Results: For each social environment, the mostly Black and mostly White dichotomizations generated better validity metrics relative to the half Black dichotomization. Across all 10 social environments in the pilot and main phases, mostly Black and mostly White dichotomizations exhibited a moderate-to-high sensitivity, specificity, correct classification rate, positive predictive value, and negative predictive value. The positive likelihood ratio values were >1, and the negative likelihood ratio values were close to 0. The false positive and negative rates were low to moderate. Conclusions: These findings support that using either the mostly Black versus other categories or the mostly White versus other categories dichotomizations may provide accurate and reliable measures of racial composition across the 10 social environments. The PRCM can serve as a uniform measure across disciplines, capture multiple social environments over the life course, and be administered during one study visit. The PRCM also provides an added window into understanding how structural racism has impacted minoritized communities and may inform equitable intervention and prevention efforts to improve lives. UR - https://publichealth.jmir.org/2024/1/e55461 UR - http://dx.doi.org/10.2196/55461 UR - http://www.ncbi.nlm.nih.gov/pubmed/39115929 ID - info:doi/10.2196/55461 ER - TY - JOUR AU - Gaur, Pooja AU - Temple, S. Dorota AU - Hegarty-Craver, Meghan AU - Boyce, D. Matthew AU - Holt, R. Jonathan AU - Wenger, F. Michael AU - Preble, A. Edward AU - Eckhoff, P. Randall AU - McCombs, S. Michelle AU - Davis-Wilson, C. Hope AU - Walls, J. Howard AU - Dausch, E. David PY - 2024/8/7 TI - Continuous Monitoring of Heart Rate Variability in Free-Living Conditions Using Wearable Sensors: Exploratory Observational Study JO - JMIR Form Res SP - e53977 VL - 8 KW - heart rate variability KW - physiological monitoring KW - wearable sensors KW - smartwatch KW - PPG KW - photoplethysmography KW - monitoring KW - physiological KW - heart rate KW - wearable KW - wearables KW - sensor KW - sensors KW - observation study KW - wearable devices KW - devices KW - remote monitoring KW - community KW - data platform KW - data collection KW - health risk N2 - Background: Wearable physiological monitoring devices are promising tools for remote monitoring and early detection of potential health changes of interest. The widespread adoption of such an approach across communities and over long periods of time will require an automated data platform for collecting, processing, and analyzing relevant health information. Objective: In this study, we explore prospective monitoring of individual health through an automated data collection, metrics extraction, and health anomaly analysis pipeline in free-living conditions over a continuous monitoring period of several months with a focus on viral respiratory infections, such as influenza or COVID-19. Methods: A total of 59 participants provided smartwatch data and health symptom and illness reports daily over an 8-month window. Physiological and activity data from photoplethysmography sensors, including high-resolution interbeat interval (IBI) and step counts, were uploaded directly from Garmin Fenix 6 smartwatches and processed automatically in the cloud using a stand-alone, open-source analytical engine. Health risk scores were computed based on a deviation in heart rate and heart rate variability metrics from each individual?s activity-matched baseline values, and scores exceeding a predefined threshold were checked for corresponding symptoms or illness reports. Conversely, reports of viral respiratory illnesses in health survey responses were also checked for corresponding changes in health risk scores to qualitatively assess the risk score as an indicator of acute respiratory health anomalies. Results: The median average percentage of sensor data provided per day indicating smartwatch wear compliance was 70%, and survey responses indicating health reporting compliance was 46%. A total of 29 elevated health risk scores were detected, of which 12 (41%) had concurrent survey data and indicated a health symptom or illness. A total of 21 influenza or COVID-19 illnesses were reported by study participants; 9 (43%) of these reports had concurrent smartwatch data, of which 6 (67%) had an increase in health risk score. Conclusions: We demonstrate a protocol for data collection, extraction of heart rate and heart rate variability metrics, and prospective analysis that is compatible with near real-time health assessment using wearable sensors for continuous monitoring. The modular platform for data collection and analysis allows for a choice of different wearable sensors and algorithms. Here, we demonstrate its implementation in the collection of high-fidelity IBI data from Garmin Fenix 6 smartwatches worn by individuals in free-living conditions, and the prospective, near real-time analysis of the data, culminating in the calculation of health risk scores. To our knowledge, this study demonstrates for the first time the feasibility of measuring high-resolution heart IBI and step count using smartwatches in near real time for respiratory illness detection over a long-term monitoring period in free-living conditions. UR - https://formative.jmir.org/2024/1/e53977 UR - http://dx.doi.org/10.2196/53977 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53977 ER - TY - JOUR AU - Larrabee Sonderlund, Anders AU - Quirina Bang Van Sas, Tessa AU - Wehberg, Sonja AU - Huibers, Linda AU - Nielsen, Bo Jesper AU - Søndergaard, Jens AU - Assing Hvidt, Elisabeth PY - 2024/8/2 TI - Development of a Video Consultation Patient-Satisfaction Questionnaire (vCare-PSQ): A Cross-Sectional Explorative Study JO - JMIR Form Res SP - e58928 VL - 8 KW - video consultation KW - patient satisfaction KW - patient-physician relationship KW - telehealth KW - general practice KW - pilot-testing KW - COVID-19 KW - SARS-CoV-2 KW - pandemic KW - primary care KW - healthcare KW - health professional KW - health professionals KW - Danish KW - adult KW - adults KW - IT-literacy KW - methodological KW - vCare-PSQ KW - COSMIN N2 - Background: Since the COVID-19 pandemic, the use of video consultation (VC) in primary care has expanded considerably in many countries. VC and other telehealth formats are often touted as a solution to improved health care access, with numerous studies showing high satisfaction with this care format among health professionals and patients. However, operationalization and measurement of patient satisfaction with VC varies across studies and often lacks consideration of dynamic contextual factors (eg, convenience, ease-of-use, or privacy) and doctor-patient relational variables that may influence patient satisfaction. Objective: We aim to develop a comprehensive and evidence-based questionnaire for assessing patient satisfaction with VC in general practice. Methods: The vCare Patient-Satisfaction Questionnaire (the vCare-PSQ) was developed according to the COSMIN (Consensus-Based Standards for the Selection of Health Measurement Instruments) guidelines. To achieve our overall objective, we pursued three aims: (1) a validation analysis of an existing patient-satisfaction scale (the PS-14), (2) an assessment of extrinsic contextual factors that may impact patient satisfaction, and (3) an assessment of pertinent intrinsic and relational satisfaction correlates (eg, health anxiety, information technology literacy, trust in the general practitioner, or convenience). For validation purposes, the questionnaire was filled out by a convenience sample of 188 Danish adults who had attended at least 1 VC. Results: Our validation analysis of the PS-14 in a Danish population produced reliable results, indicating that the PS-14 is an appropriate measure of patient satisfaction with VC in Danish patient populations. Regressing situational and doctor-patient relational factors onto patient satisfaction further suggested that patient satisfaction is contingent on several factors not measured by the PS-14. These include information technology literacy and patient trust in the general practitioner, as well as several contextual pros and cons. Conclusions: Supplementing the PS-14 with dynamic measures of situational and doctor-patient relational factors may provide a more comprehensive understanding of patient satisfaction with VC. The vCare-PSQ may thus contribute to an enhanced methodological approach to assessing patient satisfaction with VC. We hope that the vCare-PSQ format may be useful for future research and implementation efforts regarding VC in a general practice setting. UR - https://formative.jmir.org/2024/1/e58928 UR - http://dx.doi.org/10.2196/58928 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58928 ER - TY - JOUR AU - Wang, D. Shirlene AU - Hatzinger, Lori AU - Morales, Jeremy AU - Hewus, Micaela AU - Intille, Stephen AU - Dunton, F. Genevieve PY - 2024/8/2 TI - Burden and Inattentive Responding in a 12-Month Intensive Longitudinal Study: Interview Study Among Young Adults JO - JMIR Form Res SP - e52165 VL - 8 KW - data quality KW - burden KW - exit interview KW - careless responding KW - ecological momentary assessment KW - intensive longitudinal data collection KW - mobile phone N2 - Background: Intensive longitudinal data (ILD) collection methods have gained popularity in social and behavioral research as a tool to better understand behavior and experiences over time with reduced recall bias. Engaging participants in these studies over multiple months and ensuring high data quality are crucial but challenging due to the potential burden of repeated measurements. It is suspected that participants may engage in inattentive responding (IR) behavior to combat burden, but the processes underlying this behavior are unclear as previous studies have focused on the barriers to compliance rather than the barriers to providing high-quality data. Objective: This study aims to broaden researchers? knowledge about IR during ILD studies using qualitative analysis and uncover the underlying IR processes to aid future hypothesis generation. Methods: We explored the process of IR by conducting semistructured qualitative exit interviews with 31 young adult participants (aged 18-29 years) who completed a 12-month ILD health behavior study with daily evening smartphone-based ecological momentary assessment (EMA) surveys and 4-day waves of hourly EMA surveys. The interviews assessed participants? motivations, the impact of time-varying contexts, changes in motivation and response patterns over time, and perceptions of attention check questions (ACQs) to understand participants? response patterns and potential factors leading to IR. Results: Thematic analysis revealed 5 overarching themes on factors that influence participant engagement: (1) friends and family also had to tolerate the frequent surveys, (2) participants tried to respond to surveys quickly, (3) the repetitive nature of surveys led to neutral responses, (4) ACQs within the surveys helped to combat overly consistent response patterns, and (5) different motivations for answering the surveys may have led to different levels of data quality. Conclusions: This study aimed to examine participants? perceptions of the quality of data provided in an ILD study to contribute to the field?s understanding of engagement. These findings provide insights into the complex process of IR and participant engagement in ILD studies with EMA. The study identified 5 factors influencing IR that could guide future research to improve EMA survey design. The identified themes offer practical implications for researchers and study designers, including the importance of considering social context, the consideration of dynamic motivations, and the potential benefit of including ACQs as a technique to reduce IR and leveraging the intrinsic motivators of participants. By incorporating these insights, researchers might maximize the scientific value of their multimonth ILD studies through better data collection protocols. International Registered Report Identifier (IRRID): RR2-10.2196/36666 UR - https://formative.jmir.org/2024/1/e52165 UR - http://dx.doi.org/10.2196/52165 UR - http://www.ncbi.nlm.nih.gov/pubmed/39093606 ID - info:doi/10.2196/52165 ER - TY - JOUR AU - Madanay, Farrah AU - Tu, Karissa AU - Campagna, Ada AU - Davis, Kelly J. AU - Doerstling, S. Steven AU - Chen, Felicia AU - Ubel, A. Peter PY - 2024/8/1 TI - Classification of Patients? Judgments of Their Physicians in Web-Based Written Reviews Using Natural Language Processing: Algorithm Development and Validation JO - J Med Internet Res SP - e50236 VL - 26 KW - web-based physician reviews KW - patient judgments KW - RoBERTa KW - natural language processing KW - text classification KW - machine learning KW - patient experience KW - patient-authored reviews KW - healthcare quality KW - patient care KW - psychology N2 - Background: Patients increasingly rely on web-based physician reviews to choose a physician and share their experiences. However, the unstructured text of these written reviews presents a challenge for researchers seeking to make inferences about patients? judgments. Methods previously used to identify patient judgments within reviews, such as hand-coding and dictionary-based approaches, have posed limitations to sample size and classification accuracy. Advanced natural language processing methods can help overcome these limitations and promote further analysis of physician reviews on these popular platforms. Objective: This study aims to train, test, and validate an advanced natural language processing algorithm for classifying the presence and valence of 2 dimensions of patient judgments in web-based physician reviews: interpersonal manner and technical competence. Methods: We sampled 345,053 reviews for 167,150 physicians across the United States from Healthgrades.com, a commercial web-based physician rating and review website. We hand-coded 2000 written reviews and used those reviews to train and test a transformer classification algorithm called the Robustly Optimized BERT (Bidirectional Encoder Representations from Transformers) Pretraining Approach (RoBERTa). The 2 fine-tuned models coded the reviews for the presence and positive or negative valence of patients? interpersonal manner or technical competence judgments of their physicians. We evaluated the performance of the 2 models against 200 hand-coded reviews and validated the models using the full sample of 345,053 RoBERTa-coded reviews. Results: The interpersonal manner model was 90% accurate with precision of 0.89, recall of 0.90, and weighted F1-score of 0.89. The technical competence model was 90% accurate with precision of 0.91, recall of 0.90, and weighted F1-score of 0.90. Positive-valence judgments were associated with higher review star ratings whereas negative-valence judgments were associated with lower star ratings. Analysis of the data by review rating and physician gender corresponded with findings in prior literature. Conclusions: Our 2 classification models coded interpersonal manner and technical competence judgments with high precision, recall, and accuracy. These models were validated using review star ratings and results from previous research. RoBERTa can accurately classify unstructured, web-based review text at scale. Future work could explore the use of this algorithm with other textual data, such as social media posts and electronic health records. UR - https://www.jmir.org/2024/1/e50236 UR - http://dx.doi.org/10.2196/50236 UR - http://www.ncbi.nlm.nih.gov/pubmed/39088259 ID - info:doi/10.2196/50236 ER - TY - JOUR AU - Shi, Yulin AU - Wang, Baohua AU - Zhao, Jian AU - Wang, Chunping AU - Li, Ning AU - Chen, Min AU - Wan, Xia PY - 2024/7/31 TI - Summary Measure of Health-Related Quality of Life and Its Related Factors Based on the Chinese Version of the Core Healthy Days Measures: Cross-Sectional Study JO - JMIR Public Health Surveill SP - e52019 VL - 10 KW - health-related quality of life KW - Healthy Days KW - summary measure KW - health status indicators KW - exploratory factor analyses KW - confirmatory factor analyses N2 - Background: The core Healthy Days measures were used to track the population-level health status in the China Chronic Disease and Risk Factor Surveillance; however, they were not easily combined to create a summary of the overall health-related quality of life (HRQOL), limiting this indicator?s use. Objective: This study aims to develop a summary score based on the Chinese version of the core Healthy Days measures (HRQOL-5) and apply it to estimate HRQOL and its determinants in a Chinese population. Methods: From November 2018 to May 2019, a multistage stratified cluster survey was conducted to examine population health status and behavioral risk factors among the resident population older than 15 years in Weifang City, Shandong Province, China. Both exploratory factor analyses and confirmatory factor analyses were performed to reveal the underlying latent construct of HRQOL-5 and then to quantify the overall HRQOL by calculating its summary score. Tobit regression models were finally carried out to identify the influencing factors of the summary score. Results: A total of 26,269 participants (male: n=13,571, 51.7%; mean age 55.9, SD 14.9 years) were included in this study. A total of 71% (n=18,663) of respondents reported that they had excellent or very good general health. One summary factor was extracted to capture overall HRQOL using exploratory factor analysis. The confirmatory factor analysis further confirmed this one-factor model (Tucker-Lewis index, comparative fit index, and goodness-of-fit index >0.90; root mean square error of approximation 0.02). Multivariate Tobit regression analysis showed that age (?=?0.06), educational attainments (primary school: ?=0.72; junior middle school: ?=1.46; senior middle school or more: ?=2.58), average income (?¥30,000 [US $4200]: ?=0.69), physical activity (?=0.75), alcohol use (?=0.46), self-reported disease (?=?6.36), and self-reported injury (?=?5.00) were the major influencing factors on the summary score of the HRQOL-5. Conclusions: This study constructs a summary score from the HRQOL-5, providing a comprehensive representation of population-level HRQOL. Differences in summary scores of different subpopulations may help set priorities for health planning in China to improve population HRQOL. UR - https://publichealth.jmir.org/2024/1/e52019 UR - http://dx.doi.org/10.2196/52019 ID - info:doi/10.2196/52019 ER - TY - JOUR AU - Ladune, Raphaelle AU - Hayotte, Meggy AU - Vuillemin, Anne AU - d'Arripe-Longueville, Fabienne PY - 2024/7/30 TI - Development of a Web App to Enhance Physical Activity in People With Cystic Fibrosis: Co-Design and Acceptability Evaluation by Patients and Health Professionals JO - JMIR Form Res SP - e54322 VL - 8 KW - cystic fibrosis KW - decisional balance KW - digital app KW - acceptability KW - physical activity KW - mobile phone N2 - Background: Cystic fibrosis (CF) is a genetic disease affecting the respiratory and digestive systems, with recent treatment advances improving life expectancy. However, many people with CF lack adequate physical activity (PA). PA can enhance lung function and quality of life, but barriers exist. The Cystic Fibrosis Decisional Balance of Physical Activity questionnaire assesses the decisional balance for PA in adults with CF, but it is not optimal for clinical use. A digital app might overcome this limitation by improving the efficiency of administration, interpretation of results, and communication between patients and health care professionals. Objective: This paper presents the development process and reports on the acceptability of a web app designed to measure and monitor the decisional balance for PA in people with CF. Methods: This study comprised two stages: (1) the co-design of a digital app and (2) the evaluation of its acceptability among health care professionals and people with CF. A participatory approach engaged stakeholders in the app?s creation. The app?s acceptability, based on factors outlined in the Unified Theory of Acceptance and Use of Technology 2, is vital for its successful adoption. Participants volunteered, gave informed consent, and were aged >18 years and fluent in French. Data collection was performed through qualitative interviews, video presentations, surveys, and individual semistructured interviews, followed by quantitative and qualitative data analyses. Results: In total, 11 health care professionals, 6 people with CF, and 5 researchers were involved in the co-design phase. Results of this phase led to the coconstruction of an app named MUCO_BALAD, designed for people with CF aged ?18 years, health care professionals, and researchers to monitor the decisional balance for PA in people with CF. In the acceptability evaluation phase, the sample included 47 health care professionals, 44 people with CF, and 12 researchers. The analysis revealed that the acceptability measures were positive and that app acceptability did not differ according to user types. Semistructured interviews helped identify positive and negative perceptions of the app and the interface, as well as missing functionalities. Conclusions: This study assessed the acceptability of an app and demonstrated promising qualitative and quantitative results. The digital tool for measuring the decisional balance in PA for people with CF is encouraging for health care professionals, people with CF, and researchers, according to the valuable insights gained from this study. UR - https://formative.jmir.org/2024/1/e54322 UR - http://dx.doi.org/10.2196/54322 UR - http://www.ncbi.nlm.nih.gov/pubmed/39078689 ID - info:doi/10.2196/54322 ER - TY - JOUR AU - Bisby, Madelyne AU - Staples, Lauren AU - Dear, Blake AU - Titov, Nickolai PY - 2024/7/25 TI - Changes in the Frequency of Actions Associated With Mental Health During Online Treatment: Analysis of Demographic and Clinical Factors JO - JMIR Form Res SP - e57938 VL - 8 KW - anxiety KW - depression KW - daily actions KW - treatments KW - personalization KW - mental health KW - digital treatment KW - analysis KW - clinical factors KW - questionnaire KW - depression symptoms KW - anxiety symptoms KW - patients KW - Australian KW - Australia KW - digital psychology service KW - psychology KW - symptom severity KW - severity N2 - Background: Specific daily actions (eg, goal setting, meaningful activities) are associated with mental health. Performing specific daily actions at a higher frequency is associated with significantly lower baseline symptoms of depression and anxiety, as well as better psychological treatment outcomes for depression and anxiety. Objective: This study explored how the frequency of specific daily actions associated with mental health may differ prior to, during, and following treatment according to demographic and clinical characteristics. Methods: Using a sample of 448 patients from an Australian national digital psychology service, we examined baseline differences in daily action frequency and changes in daily action frequency during a digital psychological treatment according to demographic and clinical subgroups. A total of 5 specific types of daily actions were measured using the Things You Do Questionnaire: healthy thinking, meaningful activities, goals and plans, healthy habits, and social connections. Results: The frequency of daily actions differed according to employment status (largest P=.005) and educational level (largest P=.004). Daily action frequency was lower in those participants with more severe or chronic depression or anxiety symptoms (largest P=.004). Participants reported larger increases in how often they did these daily actions from baseline to midtreatment compared to mid- to posttreatment. Depression duration (P=.01) and severity (P<.001) were associated with differences in how daily action frequency changed during treatment. Conclusions: The findings of this study support continued research exploring the relationship between daily actions and mental health, how this relationship might differ between individuals, and the clinical potential of supporting individuals to increase the frequency of daily actions to improve mental health. UR - https://formative.jmir.org/2024/1/e57938 UR - http://dx.doi.org/10.2196/57938 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57938 ER - TY - JOUR AU - Bijker, Rimke AU - Merkouris, S. Stephanie AU - Dowling, A. Nicki AU - Rodda, N. Simone PY - 2024/7/25 TI - ChatGPT for Automated Qualitative Research: Content Analysis JO - J Med Internet Res SP - e59050 VL - 26 KW - ChatGPT KW - natural language processing KW - qualitative content analysis KW - Theoretical Domains Framework N2 - Background: Data analysis approaches such as qualitative content analysis are notoriously time and labor intensive because of the time to detect, assess, and code a large amount of data. Tools such as ChatGPT may have tremendous potential in automating at least some of the analysis. Objective: The aim of this study was to explore the utility of ChatGPT in conducting qualitative content analysis through the analysis of forum posts from people sharing their experiences on reducing their sugar consumption. Methods: Inductive and deductive content analysis were performed on 537 forum posts to detect mechanisms of behavior change. Thorough prompt engineering provided appropriate instructions for ChatGPT to execute data analysis tasks. Data identification involved extracting change mechanisms from a subset of forum posts. The precision of the extracted data was assessed through comparison with human coding. On the basis of the identified change mechanisms, coding schemes were developed with ChatGPT using data-driven (inductive) and theory-driven (deductive) content analysis approaches. The deductive approach was informed by the Theoretical Domains Framework using both an unconstrained coding scheme and a structured coding matrix. In total, 10 coding schemes were created from a subset of data and then applied to the full data set in 10 new conversations, resulting in 100 conversations each for inductive and unconstrained deductive analysis. A total of 10 further conversations coded the full data set into the structured coding matrix. Intercoder agreement was evaluated across and within coding schemes. ChatGPT output was also evaluated by the researchers to assess whether it reflected prompt instructions. Results: The precision of detecting change mechanisms in the data subset ranged from 66% to 88%. Overall ? scores for intercoder agreement ranged from 0.72 to 0.82 across inductive coding schemes and from 0.58 to 0.73 across unconstrained coding schemes and structured coding matrix. Coding into the best-performing coding scheme resulted in category-specific ? scores ranging from 0.67 to 0.95 for the inductive approach and from 0.13 to 0.87 for the deductive approaches. ChatGPT largely followed prompt instructions in producing a description of each coding scheme, although the wording for the inductively developed coding schemes was lengthier than specified. Conclusions: ChatGPT appears fairly reliable in assisting with qualitative analysis. ChatGPT performed better in developing an inductive coding scheme that emerged from the data than adapting an existing framework into an unconstrained coding scheme or coding directly into a structured matrix. The potential for ChatGPT to act as a second coder also appears promising, with almost perfect agreement in at least 1 coding scheme. The findings suggest that ChatGPT could prove useful as a tool to assist in each phase of qualitative content analysis, but multiple iterations are required to determine the reliability of each stage of analysis. UR - https://www.jmir.org/2024/1/e59050 UR - http://dx.doi.org/10.2196/59050 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59050 ER - TY - JOUR AU - Galvin, Karyn AU - Tomlin, Dani AU - Timmer, B. Barbra H. AU - McNeice, Zoe AU - Mount, Nicole AU - Gray, Kathleen AU - Short, E. Camille PY - 2024/7/23 TI - Consumer Perspectives for a Future Mobile App to Document Real-World Listening Difficulties: Qualitative Study JO - JMIR Form Res SP - e47578 VL - 8 KW - adults KW - hearing loss KW - listening difficulties KW - digital health KW - app KW - self-management KW - mobile health KW - smartphone KW - mobile phone N2 - Background: By enabling individuals with hearing loss to collect their own hearing data in their personal real-world settings, there is scope to improve clinical care, empower consumers, and support shared clinical decision-making and problem-solving. Clinician support for this approach has been established in a separate study. Objective: This study aims to explore, for consumers with hearing loss, their (1) experiences of listening difficulties, to identify the data an app could usefully collect; (2) preferences regarding the features of mobile apps in general; and (3) opinions on the potential value and desirable features of a yet-to-be designed app for documenting listening difficulties in real-world settings. Methods: A total of 3 focus groups involved 27 adults who self-reported hearing loss. Most were fitted with hearing devices. A facilitator used a topic guide to generate discussion, which was video- and audio-recorded. Verbatim transcriptions were analyzed using inductive content analysis. Results: Consumers supported the concept of a mobile app that would facilitate the documenting of listening difficulties in real-world settings important to the individual. Consumers shared valuable insights about their listening difficulties, which will help determine the data that should be collected through an app designed to document these challenges. This information included early indicators of hearing loss (eg, mishearing, difficulty communicating in groups and on the phone, and speaking overly loudly) and prompts to seek hearing devices (eg, spousal pressure and the advice or example provided by others, and needing to rely on lipreading or to constantly request others to repeat themselves). It also included the well-known factors that influence listening difficulties (eg, reverberation, background noise, group conversations) and the impacts and consequences of their difficulties (eg, negative impacts on relationships and employment, social isolation and withdrawal, and negative emotions). Consumers desired a visual-based app that provided options for how data could be collected and how the user could enter data into an app, and which enabled data sharing with a clinician. Conclusions: These findings provide directions for the future co-design and piloting of a prototype mobile app to provide data that are useful for increasing self-awareness of listening difficulties and can be shared with a clinician. UR - https://formative.jmir.org/2024/1/e47578 UR - http://dx.doi.org/10.2196/47578 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/47578 ER - TY - JOUR AU - Bisby, A. Madelyne AU - Jones, P. Michael AU - Staples, Lauren AU - Dear, Blake AU - Titov, Nickolai PY - 2024/7/22 TI - Measurement of Daily Actions Associated With Mental Health Using the Things You Do Questionnaire?15-Item: Questionnaire Development and Validation Study JO - JMIR Form Res SP - e57804 VL - 8 KW - daily actions KW - depression KW - anxiety KW - psychometric KW - mental health KW - questionnaire KW - activities KW - goals KW - plans KW - healthy habits KW - habits KW - treatment-seeking KW - treatment KW - confirmatory factor analysis KW - survey KW - adult KW - assessment KW - digital psychology service KW - digital KW - psychology KW - depression symptoms KW - anxiety symptoms N2 - Background: A large number of modifiable and measurable daily actions are thought to impact mental health. The ?Things You Do? refers to 5 types of daily actions that have been associated with mental health: healthy thinking, meaningful activities, goals and plans, healthy habits, and social connections. Previous studies have reported the psychometric properties of the Things You Do Questionnaire (TYDQ)?21-item (TYDQ21). The 21-item version, however, has an uneven distribution of items across the 5 aforementioned factors and may be lengthy to administer on a regular basis. Objective: This study aimed to develop and evaluate a brief version of the TYDQ. To accomplish this, we identified the top 10 and 15 items on the TYDQ21 and then evaluated the performance of the 10-item and 15-item versions of the TYDQ in community and treatment-seeking samples. Methods: Using confirmatory factor analysis, the top 2 or 3 items were used to develop the 10-item and 15-item versions, respectively. Model fit, reliability, and validity were examined for both versions in 2 samples: a survey of community adults (n=6070) and adults who completed an assessment at a digital psychology service (n=14,878). Treatment responsivity was examined in a subgroup of participants (n=448). Results: Parallel analysis supported the 5-factor structure of the TYDQ. The brief (10-item and 15-item) versions were associated with better model fit than the 21-item version, as revealed by its comparative fit index, root-mean-square error of approximation, and Tucker-Lewis index. Configural, metric, and scalar invariance were supported. The 15-item version explained more variance in the 21-item scores than the 10-item version. Internal consistency was appropriate (eg, the 15-item version had a Cronbach ? of >0.90 in both samples) and there were no marked differences between how the brief versions correlated with validated measures of depression or anxiety symptoms. The measure was responsive to treatment. Conclusions: The 15-item version is appropriate for use as a brief measure of daily actions associated with mental health while balancing brevity and clinical utility. Further research is encouraged to replicate our psychometric evaluation in other settings (eg, face-to-face services). Trial Registration: Australian New Zealand Clinical Trials Registry ACTRN12613000407796; https://tinyurl.com/2s67a6ps UR - https://formative.jmir.org/2024/1/e57804 UR - http://dx.doi.org/10.2196/57804 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/57804 ER - TY - JOUR AU - Wang, Yan AU - DeVito Dabbs, Annette AU - Thomas, Hagan Teresa AU - Campbell, Grace AU - Donovan, Heidi PY - 2024/7/22 TI - Measuring Engagement in Provider-Guided Digital Health Interventions With a Conceptual and Analytical Framework Using Nurse WRITE as an Exemplar: Exploratory Study With an Iterative Approach JO - JMIR Form Res SP - e57529 VL - 8 KW - engagement KW - digital health intervention KW - framework KW - symptom management KW - eHealth KW - gynecological cancer N2 - Background: Limited guidance exists for analyzing participant engagement in provider-guided digital health interventions (DHIs). System usage is commonly assessed, with acknowledged limitations in measuring socio-affective and cognitive aspects of engagement. Nurse WRITE, an 8-week web-based nurse-guided DHI for managing symptoms among women with recurrent ovarian cancer, offers an opportunity to develop a framework for assessing multidimensional engagement. Objective: This study aims to develop a conceptual and analytic framework to measure socio-affective, cognitive, and behavioral engagement with provider-guided DHIs. We then illustrate the framework?s ability to describe and categorize engagement using Nurse WRITE as an example. Methods: A sample of 68 participants from Nurse WRITE who posted on the message boards were included. We adapted a prior framework for conceptualizing and operationalizing engagement across 3 dimensions and finalized a set of 6 distinct measures. Using patients' posts, we created 2 socio-affective engagement measures?total count of socio-affective engagement classes (eg, sharing personal experience) and total word count?and 2 cognitive engagement measures?total count of cognitive engagement classes (eg, asking information-seeking questions) and average question completion percentage. Additionally, we devised behavioral engagement measures using website data?the total count of symptom care plans and plan reviews. k-Means clustering categorized the participants into distinct groups based on levels of engagement across 3 dimensions. Descriptive statistics and narratives were used to describe engagement in 3 dimensions. Results: On average, participants displayed socio-affective engagement 34.7 times, writing 14,851 words. They showed cognitive engagement 19.4 times, with an average of 78.3% completion of nurses' inquiries. Participants also submitted an average of 1.6 symptom care plans and 0.7 plan reviews. Participants were clustered into high (n=13), moderate (n=17), and low engagers (n=38) based on the 6 measures. High engagers wrote a median of 36,956 (IQR 26,199-46,265) words. They demonstrated socio-affective engagement approximately 81 times and cognitive engagement around 46 times, approximately 6 times that of the low engagers and twice that of the moderate engagers. High engagers had a median of 91.7% (IQR 82.2%-93.7%) completion of the nurses? queries, whereas moderate engagers had 86.4% (IQR 80%-96.4%), and low engagers had 68.3% (IQR 60.1%-79.6%). High engagers completed a median of 3 symptom care plans and 2 reviews, while moderate engagers completed 2 plans and 1 review. Low engagers completed a median of 1 plan with no reviews. Conclusions: This study developed and reported an engagement framework to guide behavioral intervention scientists in understanding and analyzing participants? engagement with provider-guided DHIs. Significant variations in engagement levels across 3 dimensions highlight the importance of measuring engagement with provider-guided DHIs in socio-affective, cognitive, and behavioral dimensions. Future studies should validate the framework with other DHIs, explore the influence of patient and provider factors on engagement, and investigate how engagement influences intervention efficacy. UR - https://formative.jmir.org/2024/1/e57529 UR - http://dx.doi.org/10.2196/57529 UR - http://www.ncbi.nlm.nih.gov/pubmed/39037757 ID - info:doi/10.2196/57529 ER - TY - JOUR AU - Pirmani, Ashkan AU - Oldenhof, Martijn AU - Peeters, M. Liesbet AU - De Brouwer, Edward AU - Moreau, Yves PY - 2024/7/17 TI - Accessible Ecosystem for Clinical Research (Federated Learning for Everyone): Development and Usability Study JO - JMIR Form Res SP - e55496 VL - 8 KW - federated learning KW - multistakeholder collaboration KW - real-world data KW - integrity KW - reliability KW - clinical research KW - implementation KW - inclusivity KW - inclusive KW - accessible KW - ecosystem KW - design effectiveness N2 - Background: The integrity and reliability of clinical research outcomes rely heavily on access to vast amounts of data. However, the fragmented distribution of these data across multiple institutions, along with ethical and regulatory barriers, presents significant challenges to accessing relevant data. While federated learning offers a promising solution to leverage insights from fragmented data sets, its adoption faces hurdles due to implementation complexities, scalability issues, and inclusivity challenges. Objective: This paper introduces Federated Learning for Everyone (FL4E), an accessible framework facilitating multistakeholder collaboration in clinical research. It focuses on simplifying federated learning through an innovative ecosystem-based approach. Methods: The ?degree of federation? is a fundamental concept of FL4E, allowing for flexible integration of federated and centralized learning models. This feature provides a customizable solution by enabling users to choose the level of data decentralization based on specific health care settings or project needs, making federated learning more adaptable and efficient. By using an ecosystem-based collaborative learning strategy, FL4E encourages a comprehensive platform for managing real-world data, enhancing collaboration and knowledge sharing among its stakeholders. Results: Evaluating FL4E?s effectiveness using real-world health care data sets has highlighted its ecosystem-oriented and inclusive design. By applying hybrid models to 2 distinct analytical tasks?classification and survival analysis?within real-world settings, we have effectively measured the ?degree of federation? across various contexts. These evaluations show that FL4E?s hybrid models not only match the performance of fully federated models but also avoid the substantial overhead usually linked with these models. Achieving this balance greatly enhances collaborative initiatives and broadens the scope of analytical possibilities within the ecosystem. Conclusions: FL4E represents a significant step forward in collaborative clinical research by merging the benefits of centralized and federated learning. Its modular ecosystem-based design and the ?degree of federation? feature make it an inclusive, customizable framework suitable for a wide array of clinical research scenarios, promising to revolutionize the field through improved collaboration and data use. Detailed implementation and analyses are available on the associated GitHub repository. UR - https://formative.jmir.org/2024/1/e55496 UR - http://dx.doi.org/10.2196/55496 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/55496 ER - TY - JOUR AU - Avnat, Eden AU - Samin, Michael AU - Ben Joya, Daniel AU - Schneider, Eyal AU - Yanko, Elia AU - Eshel, Dafna AU - Ovadia, Shahar AU - Lev, Yossi AU - Souroujon, Daniel PY - 2024/7/16 TI - The Potential of Evidence-Based Clinical Intake Tools to Discover or Ground Prevalence of Symptoms Using Real-Life Digital Health Encounters: Retrospective Cohort Study JO - J Med Internet Res SP - e49570 VL - 26 KW - clinical intake tool KW - evidence-based medicine KW - big data KW - digital health KW - symptoms KW - prevalence N2 - Background: Evidence-based clinical intake tools (EBCITs) are structured assessment tools used to gather information about patients and help health care providers make informed decisions. The growing demand for personalized medicine, along with the big data revolution, has rendered EBCITs a promising solution. EBCITs have the potential to provide comprehensive and individualized assessments of symptoms, enabling accurate diagnosis, while contributing to the grounding of medical care. Objective: This work aims to examine whether EBCITs cover data concerning disorders and symptoms to a similar extent as physicians, and thus can reliably address medical conditions in clinical settings. We also explore the potential of EBCITs to discover and ground the real prevalence of symptoms in different disorders thereby expanding medical knowledge and further supporting medical diagnoses made by physicians. Methods: Between August 1, 2022, and January 15, 2023, patients who used the services of a digital health care (DH) provider in the United States were first assessed by the Kahun EBCIT. Kahun platform gathered and analyzed the information from the sessions. This study estimated the prevalence of patients? symptoms in medical disorders using 2 data sets. The first data set analyzed symptom prevalence, as determined by Kahun?s knowledge engine. The second data set analyzed symptom prevalence, relying solely on data from the DH patients gathered by Kahun. The variance difference between these 2 prevalence data sets helped us assess Kahun?s ability to incorporate new data while integrating existing knowledge. To analyze the comprehensiveness of Kahun?s knowledge engine, we compared how well it covers weighted data for the symptoms and disorders found in the 2019 National Ambulatory Medical Care Survey (NMCAS). To assess Kahun?s diagnosis accuracy, physicians independently diagnosed 250 of Kahun-DH?s sessions. Their diagnoses were compared with Kahun?s diagnoses. Results: In this study, 2550 patients used Kahun to complete a full assessment. Kahun proposed 108,523 suggestions related to symptoms during the intake process. At the end of the intake process, 6496 conditions were presented to the caregiver. Kahun covered 94% (526,157,569/562,150,572) of the weighted symptoms and 91% (1,582,637,476/173,4783,244) of the weighted disorders in the 2019 NMCAS. In 90% (224/250) of the sessions, both physicians and Kahun suggested at least one identical disorder, with a 72% (367/507) total accuracy rate. Kahun?s engine yielded 519 prevalences while the Kahun-DH cohort yielded 599; 156 prevalences were unique to the latter and 443 prevalences were shared by both data sets. Conclusions: ECBITs, such as Kahun, encompass extensive amounts of knowledge and could serve as a reliable database for inferring medical insights and diagnoses. Using this credible database, the potential prevalence of symptoms in different disorders was discovered or grounded. This highlights the ability of ECBITs to refine the understanding of relationships between disorders and symptoms, which further supports physicians in medical diagnosis. UR - https://www.jmir.org/2024/1/e49570 UR - http://dx.doi.org/10.2196/49570 UR - http://www.ncbi.nlm.nih.gov/pubmed/39012659 ID - info:doi/10.2196/49570 ER - TY - JOUR AU - Sebo, Paul AU - Tudrej, Benoit AU - Bernard, Augustin AU - Delaunay, Bruno AU - Dupuy, Alexandra AU - Malavergne, Claire AU - Maisonneuve, Hubert PY - 2024/7/16 TI - Validation and Refinement of the Sense of Coherence Scale for a French Population: Observational Study JO - Interact J Med Res SP - e50284 VL - 13 KW - French KW - sense of coherence KW - salutogenesis KW - SOC KW - Sense of Coherence scale KW - validation KW - validscale KW - well-being KW - promoting KW - resilience KW - validity KW - reliability KW - primary care patients KW - manageability N2 - Background: Salutogenesis focuses on understanding the factors that contribute to positive health outcomes. At the core of the model lies the sense of coherence (SOC), which plays a crucial role in promoting well-being and resilience. Objective: Using the validscale Stata command, we aimed to assess the psychometric properties of the French version of the 3-dimension 13-item SOC questionnaire (SOC-13), encompassing the comprehensibility, manageability, and meaningfulness dimensions. We also aimed to determine if a refined scale, assessed through this method, exhibits superior psychometric properties compared to the SOC-13. Methods: A sample of 880 consecutive primary care patients recruited from 35 French practices were asked to complete the SOC-13. We tested for internal consistency and scalability using the Cronbach ? and Loevinger H coefficients, respectively, and we tested for construct validity using confirmatory factor analysis and goodness-of-fit indices (root mean square error of approximation [RMSEA] and comparative fit index [CFI]). Results: Of the 880 eligible patients, 804 (91.4%) agreed to participate (n=527, 65.6% women; median age 51 years). Cronbach ? and Loevinger H coefficients for the SOC-13 were all <0.70 and <0.30, respectively, indicating poor internal consistency and poor scalability (0.64 and 0.29 for comprehensibility, 0.56 and 0.26 for manageability, and 0.46 and 0.17 for meaningfulness, respectively). The RMSEA and CFI were >0.06 (0.09) and <0.90 (0.83), respectively, indicating a poor fit. By contrast, the psychometric properties of a unidimensional 8-item version of the SOC questionnaire (SOC-8) were excellent (Cronbach ?=0.82, Loevinger H=0.38, RMSEA=0.05, and CFI=0.97). Conclusions: The psychometric properties of the 3-dimension SOC-13 were poor, unlike the unidimensional SOC-8. A questionnaire built only with these 8 items could be a good candidate to measure the SOC. However, further validation studies are needed before recommending its use in research. UR - https://www.i-jmr.org/2024/1/e50284 UR - http://dx.doi.org/10.2196/50284 UR - http://www.ncbi.nlm.nih.gov/pubmed/39012689 ID - info:doi/10.2196/50284 ER - TY - JOUR AU - Zha, Bowen AU - Cai, Angshu AU - Wang, Guiqi PY - 2024/7/15 TI - Diagnostic Accuracy of Artificial Intelligence in Endoscopy: Umbrella Review JO - JMIR Med Inform SP - e56361 VL - 12 KW - endoscopy KW - artificial intelligence KW - umbrella review KW - meta-analyses KW - AI KW - diagnostic KW - researchers KW - researcher KW - tools KW - tool KW - assessment N2 - Background: Some research has already reported the diagnostic value of artificial intelligence (AI) in different endoscopy outcomes. However, the evidence is confusing and of varying quality. Objective: This review aimed to comprehensively evaluate the credibility of the evidence of AI?s diagnostic accuracy in endoscopy. Methods: Before the study began, the protocol was registered on PROSPERO (CRD42023483073). First, 2 researchers searched PubMed, Web of Science, Embase, and Cochrane Library using comprehensive search terms. Then, researchers screened the articles and extracted information. We used A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR2) to evaluate the quality of the articles. When there were multiple studies aiming at the same result, we chose the study with higher-quality evaluations for further analysis. To ensure the reliability of the conclusions, we recalculated each outcome. Finally, the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) was used to evaluate the credibility of the outcomes. Results: A total of 21 studies were included for analysis. Through AMSTAR2, it was found that 8 research methodologies were of moderate quality, while other studies were regarded as having low or critically low quality. The sensitivity and specificity of 17 different outcomes were analyzed. There were 4 studies on esophagus, 4 studies on stomach, and 4 studies on colorectal regions. Two studies were associated with capsule endoscopy, two were related to laryngoscopy, and one was related to ultrasonic endoscopy. In terms of sensitivity, gastroesophageal reflux disease had the highest accuracy rate, reaching 97%, while the invasion depth of colon neoplasia, with 71%, had the lowest accuracy rate. On the other hand, the specificity of colorectal cancer was the highest, reaching 98%, while the gastrointestinal stromal tumor, with only 80%, had the lowest specificity. The GRADE evaluation suggested that the reliability of most outcomes was low or very low. Conclusions: AI proved valuabe in endoscopic diagnoses, especially in esophageal and colorectal diseases. These findings provide a theoretical basis for developing and evaluating AI-assisted systems, which are aimed at assisting endoscopists in carrying out examinations, leading to improved patient health outcomes. However, further high-quality research is needed in the future to fully validate AI?s effectiveness. UR - https://medinform.jmir.org/2024/1/e56361 UR - http://dx.doi.org/10.2196/56361 ID - info:doi/10.2196/56361 ER - TY - JOUR AU - Squire, M. Claudia AU - Giombi, C. Kristen AU - Rupert, J. Douglas AU - Amoozegar, Jacqueline AU - Williams, Peyton PY - 2024/7/9 TI - Determining an Appropriate Sample Size for Qualitative Interviews to Achieve True and Near Code Saturation: Secondary Analysis of Data JO - J Med Internet Res SP - e52998 VL - 26 KW - saturation KW - sample size KW - web-based data collection KW - semistructured interviews KW - qualitative KW - research methods KW - research methodology KW - data collection KW - coding KW - interviews KW - interviewing KW - in-depth N2 - Background: In-depth interviews are a common method of qualitative data collection, providing rich data on individuals? perceptions and behaviors that would be challenging to collect with quantitative methods. Researchers typically need to decide on sample size a priori. Although studies have assessed when saturation has been achieved, there is no agreement on the minimum number of interviews needed to achieve saturation. To date, most research on saturation has been based on in-person data collection. During the COVID-19 pandemic, web-based data collection became increasingly common, as traditional in-person data collection was possible. Researchers continue to use web-based data collection methods post the COVID-19 emergency, making it important to assess whether findings around saturation differ for in-person versus web-based interviews. Objective: We aimed to identify the number of web-based interviews needed to achieve true code saturation or near code saturation. Methods: The analyses for this study were based on data from 5 Food and Drug Administration?funded studies conducted through web-based platforms with patients with underlying medical conditions or with health care providers who provide primary or specialty care to patients. We extracted code- and interview-specific data and examined the data summaries to determine when true saturation or near saturation was reached. Results: The sample size used in the 5 studies ranged from 30 to 70 interviews. True saturation was reached after 91% to 100% (n=30-67) of planned interviews, whereas near saturation was reached after 33% to 60% (n=15-23) of planned interviews. Studies that relied heavily on deductive coding and studies that had a more structured interview guide reached both true saturation and near saturation sooner. We also examined the types of codes applied after near saturation had been reached. In 4 of the 5 studies, most of these codes represented previously established core concepts or themes. Codes representing newly identified concepts, other or miscellaneous responses (eg, ?in general?), uncertainty or confusion (eg, ?don?t know?), or categorization for analysis (eg, correct as compared with incorrect) were less commonly applied after near saturation had been reached. Conclusions: This study provides support that near saturation may be a sufficient measure to target and that conducting additional interviews after that point may result in diminishing returns. Factors to consider in determining how many interviews to conduct include the structure and type of questions included in the interview guide, the coding structure, and the population under study. Studies with less structured interview guides, studies that rely heavily on inductive coding and analytic techniques, and studies that include populations that may be less knowledgeable about the topics discussed may require a larger sample size to reach an acceptable level of saturation. Our findings also build on previous studies looking at saturation for in-person data collection conducted at a small number of sites. UR - https://www.jmir.org/2024/1/e52998 UR - http://dx.doi.org/10.2196/52998 UR - http://www.ncbi.nlm.nih.gov/pubmed/38980711 ID - info:doi/10.2196/52998 ER - TY - JOUR AU - Pretorius, Kelly PY - 2024/7/9 TI - A Simple and Systematic Approach to Qualitative Data Extraction From Social Media for Novice Health Care Researchers: Tutorial JO - JMIR Form Res SP - e54407 VL - 8 KW - social media analysis KW - data extraction KW - health care research KW - extraction tutorial KW - Facebook extraction KW - Facebook analysis KW - safe sleep KW - sudden unexpected infant death KW - social media KW - analysis KW - systematic approach KW - qualitative data KW - Facebook KW - health-related KW - maternal perspective KW - maternal perspectives KW - sudden infant death syndrome KW - mother KW - mothers KW - women KW - United States KW - SIDS KW - SUID KW - post KW - posts UR - https://formative.jmir.org/2024/1/e54407 UR - http://dx.doi.org/10.2196/54407 UR - http://www.ncbi.nlm.nih.gov/pubmed/38980712 ID - info:doi/10.2196/54407 ER - TY - JOUR AU - Jo, Eunbeen AU - Song, Sanghoun AU - Kim, Jong-Ho AU - Lim, Subin AU - Kim, Hyeon Ju AU - Cha, Jung-Joon AU - Kim, Young-Min AU - Joo, Joon Hyung PY - 2024/7/8 TI - Assessing GPT-4?s Performance in Delivering Medical Advice: Comparative Analysis With Human Experts JO - JMIR Med Educ SP - e51282 VL - 10 KW - GPT-4 KW - medical advice KW - ChatGPT KW - cardiology KW - cardiologist KW - heart KW - advice KW - recommendation KW - recommendations KW - linguistic KW - linguistics KW - artificial intelligence KW - NLP KW - natural language processing KW - chatbot KW - chatbots KW - conversational agent KW - conversational agents KW - response KW - responses N2 - Background: Accurate medical advice is paramount in ensuring optimal patient care, and misinformation can lead to misguided decisions with potentially detrimental health outcomes. The emergence of large language models (LLMs) such as OpenAI?s GPT-4 has spurred interest in their potential health care applications, particularly in automated medical consultation. Yet, rigorous investigations comparing their performance to human experts remain sparse. Objective: This study aims to compare the medical accuracy of GPT-4 with human experts in providing medical advice using real-world user-generated queries, with a specific focus on cardiology. It also sought to analyze the performance of GPT-4 and human experts in specific question categories, including drug or medication information and preliminary diagnoses. Methods: We collected 251 pairs of cardiology-specific questions from general users and answers from human experts via an internet portal. GPT-4 was tasked with generating responses to the same questions. Three independent cardiologists (SL, JHK, and JJC) evaluated the answers provided by both human experts and GPT-4. Using a computer interface, each evaluator compared the pairs and determined which answer was superior, and they quantitatively measured the clarity and complexity of the questions as well as the accuracy and appropriateness of the responses, applying a 3-tiered grading scale (low, medium, and high). Furthermore, a linguistic analysis was conducted to compare the length and vocabulary diversity of the responses using word count and type-token ratio. Results: GPT-4 and human experts displayed comparable efficacy in medical accuracy (?GPT-4 is better? at 132/251, 52.6% vs ?Human expert is better? at 119/251, 47.4%). In accuracy level categorization, humans had more high-accuracy responses than GPT-4 (50/237, 21.1% vs 30/238, 12.6%) but also a greater proportion of low-accuracy responses (11/237, 4.6% vs 1/238, 0.4%; P=.001). GPT-4 responses were generally longer and used a less diverse vocabulary than those of human experts, potentially enhancing their comprehensibility for general users (sentence count: mean 10.9, SD 4.2 vs mean 5.9, SD 3.7; P<.001; type-token ratio: mean 0.69, SD 0.07 vs mean 0.79, SD 0.09; P<.001). Nevertheless, human experts outperformed GPT-4 in specific question categories, notably those related to drug or medication information and preliminary diagnoses. These findings highlight the limitations of GPT-4 in providing advice based on clinical experience. Conclusions: GPT-4 has shown promising potential in automated medical consultation, with comparable medical accuracy to human experts. However, challenges remain particularly in the realm of nuanced clinical judgment. Future improvements in LLMs may require the integration of specific clinical reasoning pathways and regulatory oversight for safe use. Further research is needed to understand the full potential of LLMs across various medical specialties and conditions. UR - https://mededu.jmir.org/2024/1/e51282 UR - http://dx.doi.org/10.2196/51282 ID - info:doi/10.2196/51282 ER - TY - JOUR AU - Dionne, Maude AU - Sauvageau, Chantal AU - Etienne, Doriane AU - Kiely, Marilou AU - Witteman, Holly AU - Dubé, Eve PY - 2024/7/8 TI - Development of Promising Interventions to Improve Human Papillomavirus Vaccination in a School-Based Program in Quebec, Canada: Results From a Formative Evaluation Using a Mixed Methods Design JO - JMIR Form Res SP - e57118 VL - 8 KW - immunization KW - human papillomavirus KW - HPV KW - HPV vaccine KW - school-based immunization program KW - intervention KW - strategies KW - vaccination KW - vaccine KW - Quebec KW - school-based KW - vaccine coverage KW - decision aid KW - student KW - students KW - nurse KW - nurses KW - parent KW - parents KW - focus group KW - descriptive analyses KW - user-centered KW - effectiveness KW - data collection KW - vaccine safety N2 - Background: Despite the availability of school-based human papillomavirus (HPV) vaccination programs, disparities in vaccine coverage persist. Barriers to HPV vaccine acceptance and uptake include parental attitudes, knowledge, beliefs, and system-level barriers. A total of 3 interventions were developed to address these barriers: an in-person presentation by school nurses, an email reminder with a web-based information and decision aid tool, and a telephone reminder using motivational interviewing (MI) techniques. Objective: Here we report on the development and formative evaluation of interventions to improve HPV vaccine acceptance and uptake among grade 4 students? parents in Quebec, Canada. Methods: In the summer of 2019, we conducted a formative evaluation of the interventions to assess the interventions? relevance, content, and format and to identify any unmet needs. We conducted 3 focus group discussions with parents of grade 3 students and nurses. Interviews were recorded, transcribed, and analyzed for thematic content using NVivo software (Lumivero). Nurses received training on MI techniques and we evaluated the effect on nurses? knowledge and skills using a pre-post questionnaire. Descriptive quantitative analyses were carried out on data from questionnaires relating to the training. Comparisons were made using the proportions of the results. Finally, we developed a patient decision aid using an iterative, user-centered design process. The iterative refinement process involved feedback from parents, nurses, and experts to ensure the tool?s relevance and effectiveness. The evaluation protocol and data collection tools were approved by the CHU (Centre Hospitalier Universitaire) de Québec Research Ethics Committee (MP-20-2019-4655, May 16, 2019). Results: The data collection was conducted from April 2019 to March 2021. Following feedback (n=28) from the 3 focus group discussions in June 2019, several changes were made to the in-person presentation intervention. Experts (n=27) and school nurses (n=29) recruited for the project appreciated the visual and simplified information on vaccination in it. The results of the MI training for school nurses conducted in August 2019 demonstrated an increase in the skills and knowledge of nurses (n=29). School nurses who took the web-based course (n=24) filled out a pretest and posttest questionnaire to evaluate their learning. The rating increased by 19% between the pretest and posttest questionnaires. Several changes were made between the first draft of the web-based decision-aid tool and the final version during the summer of 2019 after an expert consultation of experts (n=3), focus group participants (n=28), and parents in the iterative process (n=5). More information about HPV and vaccines was added, and users could click if more detail is desired. Conclusions: We developed and pilot-tested 3 interventions using an iterative process. The interventions were perceived as potentially effective to increase parents? knowledge and positive attitudes toward HPV vaccination, and ultimately, vaccine acceptance. Future research will assess the effectiveness of these interventions on a larger scale. UR - https://formative.jmir.org/2024/1/e57118 UR - http://dx.doi.org/10.2196/57118 UR - http://www.ncbi.nlm.nih.gov/pubmed/38976317 ID - info:doi/10.2196/57118 ER - TY - JOUR AU - Sobolewski, Jessica AU - Rothschild, Allie AU - Freeman, Andrew PY - 2024/7/4 TI - The Impact of Incentives on Data Collection for Online Surveys: Social Media Recruitment Study JO - JMIR Form Res SP - e50240 VL - 8 KW - social media KW - online survey recruitment KW - incentive KW - experiment KW - online surveys KW - Facebook KW - Instagram KW - data collection KW - users KW - cost KW - social media recruitment KW - survey N2 - Background: The use of targeted advertisements on social media platforms (eg, Facebook and Instagram) has become increasingly popular for recruiting participants for online survey research. Many of these surveys offer monetary incentives for survey completion in the form of gift cards; however, little is known about whether the incentive amount impacts the cost, speed, and quality of data collection. Objective: This experiment addresses this gap in the literature by examining how different incentives in paid advertising campaigns on Instagram for completing a 10-minute online survey influence the response rate, recruitment advertising cost, data quality, and length of data collection. Methods: This experiment tested three incentive conditions using three Instagram campaigns that were each allocated a US $1400 budget to spend over a maximum of 4 days; ads targeted users aged 15-24 years in three nonadjacent designated market areas of similar size to avoid overlapping audiences. Four ad creatives were designed for each campaign; all ads featured the same images and text, but the incentive amount varied: no incentive, US $5 gift card, and US $15 gift card. All ads had a clickable link that directed users to an eligibility screener and a 10-minute online survey, if eligible. Each campaign ran for either the full allotted time (4 days) or until there were 150 total survey completes, prior to data quality checks for fraud. Results: The US $15 incentive condition resulted in the quickest and cheapest data collection, requiring 17 hours and ad spending of US $338.64 to achieve 142 survey completes. The US $5 condition took more than twice as long (39 hours) and cost US $864.33 in ad spending to achieve 148 survey completes. The no-incentive condition ran for 60 hours, spending nearly the full budget (US $1398.23), and achieved only 24 survey completes. The US $15 and US $5 incentive conditions had similar levels of fraudulent respondents, whereas the no-incentive condition had no fraudulent respondents. The completion rate for the US $15 and US $5 incentive conditions were 93.4% (155/166) and 89.8% (149/166), respectively, while the completion rate for the no-incentive condition was 43.6% (24/55). Conclusions: Overall, we found that a higher incentive resulted in quicker data collection, less money spent on ads, and higher response rates, despite some fraudulent cases that had to be dropped from the sample. However, when considering the total incentive amounts in addition to the ad spending, a US $5 incentive appeared to be the most cost-effective data collection option. Other costs associated with running a campaign for a longer period should also be considered. A longer experiment is warranted to determine whether fraud varies over time across conditions. UR - https://formative.jmir.org/2024/1/e50240 UR - http://dx.doi.org/10.2196/50240 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/50240 ER - TY - JOUR AU - Soehnchen, Clarissa AU - Burmann, Anja AU - Henningsen, Maike AU - Meister, Sven PY - 2024/7/3 TI - A Digital Sexual Health Education Web Application for Resource-Poor Regions in Kenya: Implementation-Oriented Case Study Using the Intercultural Research Model JO - JMIR Form Res SP - e58549 VL - 8 KW - sexual health education KW - Intercultural Research Model KW - semistructured interview KW - SUS analysis KW - user-centered design N2 - Background: Developing a digital educational application focused on sexual health education necessitates a framework that integrates cultural considerations effectively. Drawing from previous research, we identified the problem and essential requirements to incorporate cultural insights into the development of a solution. Objective: This study aims to explore the Solution Room of the self-established Intercultural Research Model, with a focus on creating a reusable framework for developing and implementing a widely accessible digital educational tool for sexual health. The study centers on advancing from a low-fidelity prototype (She!Masomo) to a high-fidelity prototype (We!Masomo), while evaluating its system usability through differentiation. This research contributes to the pursuit of Sustainable Development Goals 3, 4, and 5. Methods: The research methodology is anchored in the Solution Room of the self-expanded Intercultural Research Model, which integrates cultural considerations. It uses a multimethod, user-centered design thinking approach, focusing on extensive human involvement for the open web-based application. This includes gathering self-assessed textual user feedback, conducting a System Usability Scale (SUS) analysis, and conducting 4 face-to-face semistructured expert interviews, following COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines. Results: Based on the identified limitations of the low-fidelity prototype, She!Masomo (SUS score 67), which were highlighted through textual user feedback (63/77) and prototype feature comparisons, iterative development and improvement were implemented. This process led to the creation of an enhanced high-fidelity prototype (We!Masomo). The improved effectiveness of the enhanced prototype was evaluated using the qualitative SUS analysis (82/90), resulting in a favorable score of 77.3, compared with the previous SUS score of 67 for the low-fidelity prototype. Highlighting the importance of accessible digital educational tools, this study conducted 4 expert interviews (4/4) and reported e-survey results following the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) guideline. The digital educational platform, We!Masomo, is specifically designed to promote universal and inclusive free access to information. Therefore, the developed high-fidelity prototype was implemented in Kenya. Conclusions: The primary outcome of this research provides a comprehensive exploration of utilizing a case study methodology to advance the development of digital educational web tools, particularly focusing on cultural sensitivity and sensitive educational subjects. It offers critical insights for effectively introducing such tools in regions with limited resources. Nonetheless, it is crucial to emphasize that the findings underscore the importance of integrating culture-specific components during the design phase. This highlights the necessity of conducting a thorough requirement engineering analysis and developing a low-fidelity prototype, followed by an SUS analysis. These measures are particularly critical when disseminating sensitive information, such as sexual health, through digital platforms. International Registered Report Identifier (IRRID): RR2-10.1186/s12905-023-02839-6 UR - https://formative.jmir.org/2024/1/e58549 UR - http://dx.doi.org/10.2196/58549 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/58549 ER - TY - JOUR AU - Ferrario, Andrea AU - Sedlakova, Jana AU - Trachsel, Manuel PY - 2024/7/2 TI - The Role of Humanization and Robustness of Large Language Models in Conversational Artificial Intelligence for Individuals With Depression: A Critical Analysis JO - JMIR Ment Health SP - e56569 VL - 11 KW - generative AI KW - large language models KW - large language model KW - LLM KW - LLMs KW - machine learning KW - ML KW - natural language processing KW - NLP KW - deep learning KW - depression KW - mental health KW - mental illness KW - mental disease KW - mental diseases KW - mental illnesses KW - artificial intelligence KW - AI KW - digital health KW - digital technology KW - digital intervention KW - digital interventions KW - ethics UR - https://mental.jmir.org/2024/1/e56569 UR - http://dx.doi.org/10.2196/56569 ID - info:doi/10.2196/56569 ER - TY - JOUR AU - Herman Bernardim Andrade, Gabriel AU - Yada, Shuntaro AU - Aramaki, Eiji PY - 2024/7/2 TI - Is Boundary Annotation Necessary? Evaluating Boundary-Free Approaches to Improve Clinical Named Entity Annotation Efficiency: Case Study JO - JMIR Med Inform SP - e59680 VL - 12 KW - natural language processing KW - named entity recognition KW - information extraction KW - text annotation KW - entity boundaries KW - lenient annotation KW - case reports KW - annotation KW - case study KW - medical case report KW - efficiency KW - model KW - model performance KW - dataset KW - Japan KW - Japanese KW - entity KW - clinical domain KW - clinical N2 - Background: Named entity recognition (NER) is a fundamental task in natural language processing. However, it is typically preceded by named entity annotation, which poses several challenges, especially in the clinical domain. For instance, determining entity boundaries is one of the most common sources of disagreements between annotators due to questions such as whether modifiers or peripheral words should be annotated. If unresolved, these can induce inconsistency in the produced corpora, yet, on the other hand, strict guidelines or adjudication sessions can further prolong an already slow and convoluted process. Objective: The aim of this study is to address these challenges by evaluating 2 novel annotation methodologies, lenient span and point annotation, aiming to mitigate the difficulty of precisely determining entity boundaries. Methods: We evaluate their effects through an annotation case study on a Japanese medical case report data set. We compare annotation time, annotator agreement, and the quality of the produced labeling and assess the impact on the performance of an NER system trained on the annotated corpus. Results: We saw significant improvements in the labeling process efficiency, with up to a 25% reduction in overall annotation time and even a 10% improvement in annotator agreement compared to the traditional boundary-strict approach. However, even the best-achieved NER model presented some drop in performance compared to the traditional annotation methodology. Conclusions: Our findings demonstrate a balance between annotation speed and model performance. Although disregarding boundary information affects model performance to some extent, this is counterbalanced by significant reductions in the annotator?s workload and notable improvements in the speed of the annotation process. These benefits may prove valuable in various applications, offering an attractive compromise for developers and researchers. UR - https://medinform.jmir.org/2024/1/e59680 UR - http://dx.doi.org/10.2196/59680 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/59680 ER - TY - JOUR AU - Hodson, Nathan AU - Woods, Peter AU - Sobolev, Michael AU - Giacco, Domenico PY - 2024/6/28 TI - A Digital Microintervention Supporting Evidence-Based Parenting Skills: Development Study Using the Agile Scrum Methodology JO - JMIR Form Res SP - e54892 VL - 8 KW - parenting KW - child behavior KW - mental health KW - app development KW - digital N2 - Background: Conduct disorder increases risks of educational dropout, future mental illness, and incarceration if untreated. First-line treatment of conduct disorder involves evidence-based parenting skills programs. Time-outs, a frequent tool in these programs, can be effective at improving behavior, and recent apps have been developed to aid this process. However, these apps promote the use of time-outs in inconsistent or developmentally inappropriate ways, potentially worsening behavior problems. Digital microinterventions like these apps could guide parents through high-quality time-outs in the moment, but current time-out apps lack features promoting adherence to the evidence-based best practice. Agile scrum is a respected approach in the software development industry. Objective: We aimed to explore the feasibility of using the agile scrum approach to build a digital microintervention to help parents deliver an evidence-based time-out. Methods: The agile scrum methodology was used. Four sprints were conducted. Figma software was used for app design and wireframing. Insights from 42 expert stakeholders were used during 3 sprint reviews. We consulted experts who were identified from councils around the Midlands region of the United Kingdom and charities through personal contacts and a snowballing approach. Results: Over 4 development sprints from August 2022 to March 2023, the app was iteratively designed and refined based on consultation with a diverse group of 42 experts who shared their knowledge about the content of common parenting programs and the challenges parents commonly face. Modifications made throughout the process resulted in significant app enhancements, including tailored timer algorithms and enhanced readability, as well as an onboarding zone, mindfulness module, and pictorial information to increase inclusivity. By the end of the fourth sprint, the app was deemed ready for home use by stakeholders, demonstrating the effectiveness of our agile scrum development approach. Conclusions: We developed an app to support parents to use the evidence-based time-out technique. We recommend the agile scrum approach to create mobile health apps. Our experience highlights the valuable role that frontline health and social care professionals, particularly those working with vulnerable families, can play as experts in scrum reviews. There is a need for research to both evaluate the impact of digital microinterventions on child behavioral change and also create digital microinterventions that cater to non?English speakers and individuals who participate in parenting programs in settings outside the United Kingdom. UR - https://formative.jmir.org/2024/1/e54892 UR - http://dx.doi.org/10.2196/54892 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/54892 ER - TY - JOUR AU - Courtney, Elizabeth Kelly AU - Liu, Weichen AU - Andrade, Gianna AU - Schulze, Jurgen AU - Doran, Neal PY - 2024/6/27 TI - Attentional Bias, Pupillometry, and Spontaneous Blink Rate: Eye Characteristic Assessment Within a Translatable Nicotine Cue Virtual Reality Paradigm JO - JMIR Serious Games SP - e54220 VL - 12 KW - nicotine KW - craving KW - cue exposure KW - virtual reality KW - attentional bias KW - pupillometry KW - spontaneous blink rate KW - eye-tracking KW - tobacco KW - VR KW - development KW - addiction KW - eye KW - pupil KW - biomarker KW - biomarkers KW - tobacco product N2 - Background: Incentive salience processes are important for the development and maintenance of addiction. Eye characteristics such as gaze fixation time, pupil diameter, and spontaneous eyeblink rate (EBR) are theorized to reflect incentive salience and may serve as useful biomarkers. However, conventional cue exposure paradigms have limitations that may impede accurate assessment of these markers. Objective: This study sought to evaluate the validity of these eye-tracking metrics as indicators of incentive salience within a virtual reality (VR) environment replicating real-world situations of nicotine and tobacco product (NTP) use. Methods: NTP users from the community were recruited and grouped by NTP use patterns: nondaily (n=33) and daily (n=75) use. Participants underwent the NTP cue VR paradigm and completed measures of nicotine craving, NTP use history, and VR-related assessments. Eye-gaze fixation time (attentional bias) and pupillometry in response to NTP versus control cues and EBR during the active and neutral VR scenes were recorded and analyzed using ANOVA and analysis of covariance models. Results: Greater subjective craving, as measured by the Tobacco Craving Questionnaire?Short Form, following active versus neutral scenes was observed (F1,106=47.95; P<.001). Greater mean eye-gaze fixation time (F1,106=48.34; P<.001) and pupil diameter (F1,102=5.99; P=.02) in response to NTP versus control cues were also detected. Evidence of NTP use group effects was observed in fixation time and pupillometry analyses, as well as correlations between these metrics, NTP use history, and nicotine craving. No significant associations were observed with EBR. Conclusions: This study provides additional evidence for attentional bias, as measured via eye-gaze fixation time, and pupillometry as useful biomarkers of incentive salience, and partially supports theories suggesting that incentive salience diminishes as nicotine dependence severity increases. UR - https://games.jmir.org/2024/1/e54220 UR - http://dx.doi.org/10.2196/54220 ID - info:doi/10.2196/54220 ER - TY - JOUR AU - Soni, Hiral AU - Ivanova, Julia AU - Wilczewski, Hattie AU - Ong, Triton AU - Ross, Nalubega J. AU - Bailey, Alexandra AU - Cummins, Mollie AU - Barrera, Janelle AU - Bunnell, Brian AU - Welch, Brandon PY - 2024/6/25 TI - User Preferences and Needs for Health Data Collection Using Research Electronic Data Capture: Survey Study JO - JMIR Med Inform SP - e49785 VL - 12 KW - Research Electronic Data Capture KW - REDCap KW - user experience KW - electronic data collection KW - health data KW - personal health information KW - clinical research KW - mobile phone N2 - Background: Self-administered web-based questionnaires are widely used to collect health data from patients and clinical research participants. REDCap (Research Electronic Data Capture; Vanderbilt University) is a global, secure web application for building and managing electronic data capture. Unfortunately, stakeholder needs and preferences of electronic data collection via REDCap have rarely been studied. Objective: This study aims to survey REDCap researchers and administrators to assess their experience with REDCap, especially their perspectives on the advantages, challenges, and suggestions for the enhancement of REDCap as a data collection tool. Methods: We conducted a web-based survey with representatives of REDCap member organizations in the United States. The survey captured information on respondent demographics, quality of patient-reported data collected via REDCap, patient experience of data collection with REDCap, and open-ended questions focusing on the advantages, challenges, and suggestions to enhance REDCap?s data collection experience. Descriptive and inferential analysis measures were used to analyze quantitative data. Thematic analysis was used to analyze open-ended responses focusing on the advantages, disadvantages, and enhancements in data collection experience. Results: A total of 207 respondents completed the survey. Respondents strongly agreed or agreed that the data collected via REDCap are accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). More than half of respondents strongly agreed or agreed that patients find REDCap easy to use (165/207, 79.7%), could successfully complete tasks without help (151/207, 72.9%), and could do so in a timely manner (163/207, 78.7%). Thematic analysis of open-ended responses yielded 8 major themes: survey development, user experience, survey distribution, survey results, training and support, technology, security, and platform features. The user experience category included more than half of the advantage codes (307/594, 51.7% of codes); meanwhile, respondents reported higher challenges in survey development (169/516, 32.8% of codes), also suggesting the highest enhancement suggestions for the category (162/439, 36.9% of codes). Conclusions: Respondents indicated that REDCap is a valued, low-cost, secure resource for clinical research data collection. REDCap?s data collection experience was generally positive among clinical research and care staff members and patients. However, with the advancements in data collection technologies and the availability of modern, intuitive, and mobile-friendly data collection interfaces, there is a critical opportunity to enhance the REDCap experience to meet the needs of researchers and patients. UR - https://medinform.jmir.org/2024/1/e49785 UR - http://dx.doi.org/10.2196/49785 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/49785 ER - TY - JOUR AU - O'Driscoll, Fiona AU - O'Brien, Niki AU - Guo, Chaohui AU - Prime, Matthew AU - Darzi, Ara AU - Ghafur, Saira PY - 2024/6/25 TI - Clinical Simulation in the Regulation of Software as a Medical Device: An eDelphi Study JO - JMIR Form Res SP - e56241 VL - 8 KW - digital health technology KW - software as a medical device KW - clinical simulation KW - Delphi study KW - eDelphi study KW - artificial intelligence KW - digital health N2 - Background: Accelerated digitalization in the health sector requires the development of appropriate evaluation methods to ensure that digital health technologies (DHTs) are safe and effective. Software as a medical device (SaMD) is a commonly used DHT by clinicians to provide care to patients. Traditional research methods for evaluating health care products, such as randomized clinical trials, may not be suitable for DHTs, such as SaMD. However, evidence to show their safety and efficacy is needed by regulators before they can be used in practice. Clinical simulation can be used by researchers to test SaMD in an agile and low-cost way; yet, there is limited research on criteria to assess the robustness of simulations and, subsequently, their relevance for a regulatory decision. Objective: The objective of this study was to gain consensus on the criteria that should be used to assess clinical simulation from a regulatory perspective when it is used to generate evidence for SaMD. Methods: An eDelphi study approach was chosen to develop a set of criteria to assess clinical simulation when used to evaluate SaMD. Participants were recruited through purposive and snowball sampling based on their experience and knowledge in relevant sectors. They were guided through an initial scoping questionnaire with key themes identified from the literature to obtain a comprehensive list of criteria. Participants voted upon these criteria in 2 Delphi rounds, with criteria being excluded if consensus was not met. Participants were invited to add qualitative comments during rounds and qualitative analysis was performed on the comments gathered during the first round. Consensus was predefined by 2 criteria: if <10% of the panelists deemed the criteria as ?not important? or ?not important at all? and >60% ?important? or ?very important.? Results: In total, 33 international experts in the digital health field, including academics, regulators, policy makers, and industry representatives, completed both Delphi rounds, and 43 criteria gained consensus from the participants. The research team grouped these criteria into 7 domains?background and context, overall study design, study population, delivery of the simulation, fidelity, software and artificial intelligence, and study analysis. These 7 domains were formulated into the simulation for regulation of SaMD framework. There were key areas of concern identified by participants regarding the framework criteria, such as the importance of how simulation fidelity is achieved and reported and the avoidance of bias throughout all stages. Conclusions: This study proposes the simulation for regulation of SaMD framework, developed through an eDelphi consensus process, to evaluate clinical simulation when used to assess SaMD. Future research should prioritize the development of safe and effective SaMD, while implementing and refining the framework criteria to adapt to new challenges. UR - https://formative.jmir.org/2024/1/e56241 UR - http://dx.doi.org/10.2196/56241 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/56241 ER - TY - JOUR AU - Luo, Xufei AU - Chen, Fengxian AU - Zhu, Di AU - Wang, Ling AU - Wang, Zijun AU - Liu, Hui AU - Lyu, Meng AU - Wang, Ye AU - Wang, Qi AU - Chen, Yaolong PY - 2024/6/25 TI - Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses JO - J Med Internet Res SP - e56780 VL - 26 KW - large language model KW - ChatGPT KW - systematic review KW - chatbot KW - meta-analysis UR - https://www.jmir.org/2024/1/e56780 UR - http://dx.doi.org/10.2196/56780 UR - http://www.ncbi.nlm.nih.gov/pubmed/38819655 ID - info:doi/10.2196/56780 ER - TY - JOUR AU - Kramer, Joanna AU - Wilens, E. Timothy AU - Rao, Vinod AU - Villa, Richard AU - Yule, M. Amy PY - 2024/6/25 TI - Feasibility of a 2-Part Substance Use Screener Self-Administered by Patients on Paper: Observational Study JO - JMIR Form Res SP - e52801 VL - 8 KW - patient reported outcome measures KW - patient reported outcomes KW - substance use screening KW - paper and pencil screening KW - screening KW - tobacco KW - prescription medication KW - medication KW - substance use KW - care KW - mental health KW - symptoms N2 - Background: Measurement-based care in behavioral health uses patient-reported outcome measures (PROMs) to screen for mental health symptoms and substance use and to assess symptom change over time. While PROMs are increasingly being integrated into electronic health record systems and administered electronically, paper-based PROMs continue to be used. It is unclear if it is feasible to administer a PROM on paper when the PROM was initially developed for electronic administration. Objective: This study aimed to examine the feasibility of patient self-administration of a 2-part substance use screener?the Tobacco, Alcohol, Prescription medications, and other Substances (TAPS)?on paper. This screener was originally developed for electronic administration. It begins with a limited number of questions and branches to either skip or reflex to additional questions based on an individual?s responses. In this study, the TAPS was adapted for paper use due to barriers to electronic administration within an urgent care behavioral health clinic at an urban health safety net hospital. Methods: From August 2021 to March 2022, research staff collected deidentified paper TAPS responses and tracked TAPS completion rates and adherence to questionnaire instructions. A retrospective chart review was subsequently conducted to obtain demographic information for the patients who presented to the clinic between August 2021 and March 2022. Since the initial information collected from TAPS responses was deidentified, demographic information was not linked to the individual TAPS screeners that were tracked by research staff. Results: A total of 507 new patients were seen in the clinic with a mean age of 38.7 (SD 16.6) years. In all, 258 (50.9%) patients were male. They were predominantly Black (n=212, 41.8%), White (n=152, 30%), and non-Hispanic or non-Latino (n=403, 79.5%). Most of the patients were publicly insured (n=411, 81.1%). Among these 507 patients, 313 (61.7%) completed the TAPS screener. Of these 313 patients, 76 (24.3%) adhered to the instructions and 237 (75.7%) did not follow the instructions correctly. Of the 237 respondents who did not follow the instructions correctly, 166 (70%) answered more questions and 71 (30%) answered fewer questions than required in TAPS part 2. Among the 237 patients who did not adhere to questionnaire instructions, 44 (18.6%) responded in a way that contradicted their response in part 1 of the screener and ultimately affected their overall TAPS score. Conclusions: It was challenging for patients to adhere to questionnaire instructions when completing a substance use screener on paper that was originally developed for electronic use. When selecting PROMs for measurement-based care, it is important to consider the structure of the questionnaire and how the PROM will be administered to determine if additional support for PROM self-administration needs to be implemented. UR - https://formative.jmir.org/2024/1/e52801 UR - http://dx.doi.org/10.2196/52801 UR - http://www.ncbi.nlm.nih.gov/pubmed/38916950 ID - info:doi/10.2196/52801 ER - TY - JOUR AU - Depauw, Tanguy AU - Boasen, Jared AU - Léger, Pierre-Majorique AU - Sénécal, Sylvain PY - 2024/6/14 TI - Assessing the Relationship Between Digital Trail Making Test Performance and IT Task Performance: Empirical Study JO - JMIR Hum Factors SP - e49992 VL - 11 KW - Trail Making Test KW - user experience KW - cognitive profile KW - information technology KW - task performance KW - cognitive assessment KW - human factors KW - cognitive function KW - CAPTCHA N2 - Background: Cognitive functional ability affects the accessibility of IT and is thus something that should be controlled for in user experience (UX) research. However, many cognitive function assessment batteries are long and complex, making them impractical for use in conventional experimental time frames. Therefore, there is a need for a short and reliable cognitive assessment that has discriminant validity for cognitive functions needed for general IT tasks. One potential candidate is the Trail Making Test (TMT). Objective: This study investigated the usefulness of a digital TMT as a cognitive profiling tool in IT-related UX research by assessing its predictive validity on general IT task performance and exploring its discriminant validity according to discrete cognitive functions required to perform the IT task. Methods: A digital TMT (parts A and B) named Axon was administered to 27 healthy participants, followed by administration of 5 IT tasks in the form of CAPTCHAs (Completely Automated Public Turing tests to Tell Computers and Humans Apart). The discrete cognitive functions required to perform each CAPTCHA were rated by trained evaluators. To further explain and cross-validate our results, the original TMT and 2 psychological assessments of visuomotor and short-term memory function were administered. Results: Axon A and B were administrable in less than 5 minutes, and overall performance was significantly predictive of general IT task performance (F5,19=6.352; P=.001; ?=0.374). This result was driven by performance on Axon B (F5,19=3.382; P=.02; ?=0.529), particularly for IT tasks involving the combination of executive processing with visual object and pattern recognition. Furthermore, Axon was cross-validated with the original TMT (Pcorr=.001 and Pcorr=.017 for A and B, respectively) and visuomotor and short-term memory tasks. Conclusions: The results demonstrate that variance in IT task performance among an age-homogenous neurotypical population can be related to intersubject variance in cognitive function as assessed by Axon. Although Axon?s predictive validity seemed stronger for tasks involving the combination of executive function with visual object and pattern recognition, these cognitive functions are arguably relevant to the majority of IT interfaces. Considering its short administration time and remote implementability, the Axon digital TMT demonstrates the potential to be a useful cognitive profiling tool for IT-based UX research. UR - https://humanfactors.jmir.org/2024/1/e49992 UR - http://dx.doi.org/10.2196/49992 UR - http://www.ncbi.nlm.nih.gov/pubmed/38875007 ID - info:doi/10.2196/49992 ER - TY - JOUR AU - Stang, S. Garrett AU - Tanner, T. Nichole AU - Hatch, Ashley AU - Godbolt, Jakarri AU - Toll, A. Benjamin AU - Rojewski, M. Alana PY - 2024/6/12 TI - Development of an Electronic Health Record Self-Referral Tool for Lung Cancer Screening: One-Group Posttest Study JO - JMIR Form Res SP - e53159 VL - 8 KW - lung cancer screening KW - LCS KW - electronic health records KW - EHR KW - Health Belief Model KW - HBM KW - self-refer KW - tobacco treatment KW - cancer screening KW - development KW - self-referral tool KW - electronic health record KW - decision-making N2 - Background: Approximately 14 million individuals in the United States are eligible for lung cancer screening (LCS), but only 5.8% completed screening in 2021. Given the low uptake despite the potential great health benefit of LCS, interventions aimed at increasing uptake are warranted. The use of a patient-facing electronic health record (EHR) patient portal direct messaging tool offers a new opportunity to both engage eligible patients in preventative screening and provide a unique referral pathway for tobacco treatment. Objective: This study sought to develop and pilot an EHR patient-facing self-referral tool for an established LCS program in an academic medical center. Methods: Guided by constructs of the Health Belief Model associated with LCS uptake (eg, knowledge and self-efficacy), formative development of an EHR-delivered engagement message, infographic, and self-referring survey was conducted. The survey submits eligible self-reported patient information to a scheduler for the LCS program. The materials were pretested using an interviewer-administered mixed methods survey captured through venue-day-time sampling in 5 network-affiliated pulmonology clinics. Materials were then integrated into the secure patient messaging feature in the EHR system. Next, a one-group posttest quality improvement pilot test was conducted. Results: A total of 17 individuals presenting for lung screening shared-decision visits completed the pretest survey. More than half were newly referred for LCS (n=10, 60%), and the remaining were returning patients. When asked if they would use a self-referring tool through their EHR messaging portal, 94% (n=16) reported yes. In it, 15 participants provided oral feedback that led to refinement in the tool and infographic prior to pilot-testing. When the initial application of the tool was sent to a convenience sample of 150 random patients, 13% (n=20) opened the self-referring survey. Of the 20 who completed the pilot survey, 45% (n=9) were eligible for LCS based on self-reported smoking data. A total of 3 self-referring individuals scheduled an LCS. Conclusions: Pretest and initial application data suggest this tool is a positive stimulus to trigger the decision-making process to engage in a self-referral process to LCS among eligible patients. This self-referral tool may increase the number of patients engaging in LCS and could also be used to aid in self-referral to other preventative health screenings. This tool has implications for clinical practice. Tobacco treatment clinical services or health care systems should consider using EHR messaging for LCS self-referral. This approach may be cost-effective to improve LCS engagement and uptake. Additional referral pathways could be built into this EHR tool to not only refer patients who currently smoke to LCS but also simultaneously trigger a referral to clinical tobacco treatment. UR - https://formative.jmir.org/2024/1/e53159 UR - http://dx.doi.org/10.2196/53159 UR - http://www.ncbi.nlm.nih.gov/pubmed/38865702 ID - info:doi/10.2196/53159 ER - TY - JOUR AU - Salvi, Amey AU - Gillenwater, A. Logan AU - Cockrum, P. Brandon AU - Wiehe, E. Sarah AU - Christian, Kaitlyn AU - Cayton, John AU - Bailey, Timothy AU - Schwartz, Katherine AU - Dir, L. Allyson AU - Ray, Bradley AU - Aalsma, C. Matthew AU - Reda, Khairi PY - 2024/6/11 TI - Development of a Real-Time Dashboard for Overdose Touchpoints: User-Centered Design Approach JO - JMIR Hum Factors SP - e57239 VL - 11 KW - overdose prevention KW - dashboards KW - fatality review KW - data integration KW - visualizations KW - visualization KW - dashboard KW - fatality KW - death KW - overdose KW - overdoses KW - overdosing KW - prevention KW - develop KW - development KW - design KW - interview KW - interviews KW - focus group KW - focus groups KW - touchpoints KW - touchpoint KW - substance abuse KW - drug abuse N2 - Background: Overdose Fatality Review (OFR) is an important public health tool for shaping overdose prevention strategies in communities. However, OFR teams review only a few cases at a time, which typically represent a small fraction of the total fatalities in their jurisdiction. Such limited review could result in a partial understanding of local overdose patterns, leading to policy recommendations that do not fully address the broader community needs. Objective: This study explored the potential to enhance conventional OFRs with a data dashboard, incorporating visualizations of touchpoints?events that precede overdoses?to highlight prevention opportunities. Methods: We conducted 2 focus groups and a survey of OFR experts to characterize their information needs and design a real-time dashboard that tracks and measures decedents? past interactions with services in Indiana. Experts (N=27) were engaged, yielding insights on essential data features to incorporate and providing feedback to guide the development of visualizations. Results: The findings highlighted the importance of showing decedents? interactions with health services (emergency medical services) and the justice system (incarcerations). Emphasis was also placed on maintaining decedent anonymity, particularly in small communities, and the need for training OFR members in data interpretation. The developed dashboard summarizes key touchpoint metrics, including prevalence, interaction frequency, and time intervals between touchpoints and overdoses, with data viewable at the county and state levels. In an initial evaluation, the dashboard was well received for its comprehensive data coverage and its potential for enhancing OFR recommendations and case selection. Conclusions: The Indiana touchpoints dashboard is the first to display real-time visualizations that link administrative and overdose mortality data across the state. This resource equips local health officials and OFRs with timely, quantitative, and spatiotemporal insights into overdose risk factors in their communities, facilitating data-driven interventions and policy changes. However, fully integrating the dashboard into OFR practices will likely require training teams in data interpretation and decision-making. UR - https://humanfactors.jmir.org/2024/1/e57239 UR - http://dx.doi.org/10.2196/57239 UR - http://www.ncbi.nlm.nih.gov/pubmed/38861717 ID - info:doi/10.2196/57239 ER - TY - JOUR AU - Ulm, Clayton AU - Chen, Sixia AU - Fleshman, Brianna AU - Benson, Lizbeth AU - Kendzor, E. Darla AU - Frank-Pearce, Summer AU - Neil, M. Jordan AU - Vidrine, Damon AU - De La Torre, Irene AU - Businelle, S. Michael PY - 2024/6/7 TI - Smartphone-Based Survey and Message Compliance in Adults Initially Unready to Quit Smoking: Secondary Analysis of a Randomized Controlled Trial JO - JMIR Form Res SP - e56003 VL - 8 KW - just-in-time adaptive intervention KW - tailored messaging KW - smoking cessation KW - mobile health KW - survey compliance KW - phase-based model KW - smoking KW - smoker KW - survey KW - smokers KW - messaging KW - smartphone KW - efficacy KW - pilot randomized controlled trial KW - adult smokers KW - linear regression KW - age KW - intervention engagement KW - engagement N2 - Background: Efficacy of smartphone-based interventions depends on intervention content quality and level of exposure to that content. Smartphone-based survey completion rates tend to decline over time; however, few studies have identified variables that predict this decline over longer-term interventions (eg, 26 weeks). Objective: This study aims to identify predictors of survey completion and message viewing over time within a 26-week smoking cessation trial. Methods: This study examined data from a 3-group pilot randomized controlled trial of adults who smoke (N=152) and were not ready to quit smoking within the next 30 days. For 182 days, two intervention groups received smartphone-based morning and evening messages based on current readiness to quit smoking. The control group received 2 daily messages unrelated to smoking. All participants were prompted to complete 26 weekly smartphone-based surveys that assessed smoking behavior, quit attempts, and readiness to quit. Compliance was operationalized as percentages of weekly surveys completed and daily messages viewed. Linear regression and mixed-effects models were used to identify predictors (eg, intervention group, age, and sex) of weekly survey completion and daily message viewing and decline in compliance over time. Results: The sample (mean age 50, SD 12.5, range 19-75 years; mean years of education 13.3, SD 1.6, range 10-20 years) was 67.8% (n=103) female, 74.3% (n=113) White, 77% (n=117) urban, and 52.6% (n=80) unemployed, and 61.2% (n=93) had mental health diagnoses. On average, participants completed 18.3 (71.8%) out of 25.5 prompted weekly surveys and viewed 207.3 (60.6%) out of 345.1 presented messages (31,503/52,460 total). Age was positively associated with overall weekly survey completion (P=.003) and daily message viewing (P=.02). Mixed-effects models indicated a decline in survey completion from 77% (114/148) in the first week of the intervention to 56% (84/150) in the last week of the intervention (P<.001), which was significantly moderated by age, sex, ethnicity, municipality (ie, rural/urban), and employment status. Similarly, message viewing declined from 72.3% (1533/2120) in the first week of the intervention to 44.6% (868/1946) in the last week of the intervention (P<.001). This decline in message viewing was significantly moderated by age, sex, municipality, employment status, and education. Conclusions: This study demonstrated the feasibility of a 26-week smartphone-based smoking cessation intervention. Study results identified subgroups that displayed accelerated rates in the decline of survey completion and message viewing. Future research should identify ways to maintain high levels of interaction with mobile health interventions that span long intervention periods, especially among subgroups that have demonstrated declining rates of intervention engagement over time. Trial Registration: ClinicalTrials.gov NCT03405129; https://clinicaltrials.gov/ct2/show/NCT03405129 UR - https://formative.jmir.org/2024/1/e56003 UR - http://dx.doi.org/10.2196/56003 UR - http://www.ncbi.nlm.nih.gov/pubmed/38848557 ID - info:doi/10.2196/56003 ER - TY - JOUR AU - Dagli, Marcel Mert AU - Oettl, Conrad Felix AU - Gujral, Jaskeerat AU - Malhotra, Kashish AU - Ghenbot, Yohannes AU - Yoon, W. Jang AU - Ozturk, K. Ali AU - Welch, C. William PY - 2024/6/7 TI - Clinical Accuracy, Relevance, Clarity, and Emotional Sensitivity of Large Language Models to Surgical Patient Questions: Cross-Sectional Study JO - JMIR Form Res SP - e56165 VL - 8 KW - artificial intelligence KW - AI KW - natural language processing KW - NLP KW - large language model KW - LLM KW - generative AI KW - cross-sectional study KW - health information KW - patient education KW - clinical accuracy KW - emotional sensitivity KW - surgical patient KW - surgery KW - surgical UR - https://formative.jmir.org/2024/1/e56165 UR - http://dx.doi.org/10.2196/56165 UR - http://www.ncbi.nlm.nih.gov/pubmed/38848553 ID - info:doi/10.2196/56165 ER - TY - JOUR AU - Chuang, Hai-Hua AU - Lin, Chen AU - Lee, Li-Ang AU - Chang, Hsiang-Chih AU - She, Guan-Jie AU - Lin, Yu-Hsuan PY - 2024/6/5 TI - Comparing Human-Smartphone Interactions and Actigraphy Measurements for Circadian Rhythm Stability and Adiposity: Algorithm Development and Validation Study JO - J Med Internet Res SP - e50149 VL - 26 KW - actigraphy KW - body composition KW - circadian rhythm KW - human-smartphone interaction KW - interdaily stability KW - obesity N2 - Background: This study aimed to investigate the relationships between adiposity and circadian rhythm and compare the measurement of circadian rhythm using both actigraphy and a smartphone app that tracks human-smartphone interactions. Objective: We hypothesized that the app-based measurement may provide more comprehensive information, including light-sensitive melatonin secretion and social rhythm, and have stronger correlations with adiposity indicators. Methods: We enrolled a total of 78 participants (mean age 41.5, SD 9.9 years; 46/78, 59% women) from both an obesity outpatient clinic and a workplace health promotion program. All participants (n=29 with obesity, n=16 overweight, and n=33 controls) were required to wear a wrist actigraphy device and install the Rhythm app for a minimum of 4 weeks, contributing to a total of 2182 person-days of data collection. The Rhythm app estimates sleep and circadian rhythm indicators by tracking human-smartphone interactions, which correspond to actigraphy. We examined the correlations between adiposity indices and sleep and circadian rhythm indicators, including sleep time, chronotype, and regularity of circadian rhythm, while controlling for physical activity level, age, and gender. Results: Sleep onset and wake time measurements did not differ significantly between the app and actigraphy; however, wake after sleep onset was longer (13.5, SD 19.5 minutes) with the app, resulting in a longer actigraphy-measured total sleep time (TST) of 20.2 (SD 66.7) minutes. The obesity group had a significantly longer TST with both methods. App-measured circadian rhythm indicators were significantly lower than their actigraphy-measured counterparts. The obesity group had significantly lower interdaily stability (IS) than the control group with both methods. The multivariable-adjusted model revealed a negative correlation between BMI and app-measured IS (P=.007). Body fat percentage (BF%) and visceral adipose tissue area (VAT) showed significant correlations with both app-measured IS and actigraphy-measured IS. The app-measured midpoint of sleep showed a positive correlation with both BF% and VAT. Actigraphy-measured TST exhibited a positive correlation with BMI, VAT, and BF%, while no significant correlation was found between app-measured TST and either BMI, VAT, or BF%. Conclusions: Our findings suggest that IS is strongly correlated with various adiposity indicators. Further exploration of the role of circadian rhythm, particularly measured through human-smartphone interactions, in obesity prevention could be warranted. UR - https://www.jmir.org/2024/1/e50149 UR - http://dx.doi.org/10.2196/50149 UR - http://www.ncbi.nlm.nih.gov/pubmed/38838328 ID - info:doi/10.2196/50149 ER - TY - JOUR AU - Ohno, Yukiko AU - Kato, Riri AU - Ishikawa, Haruki AU - Nishiyama, Tomohiro AU - Isawa, Minae AU - Mochizuki, Mayumi AU - Aramaki, Eiji AU - Aomori, Tohru PY - 2024/6/4 TI - Using the Natural Language Processing System Medical Named Entity Recognition-Japanese to Analyze Pharmaceutical Care Records: Natural Language Processing Analysis JO - JMIR Form Res SP - e55798 VL - 8 KW - natural language processing KW - NLP KW - named entity recognition KW - pharmaceutical care records KW - machine learning KW - cefazolin sodium KW - electronic medical record KW - EMR KW - extraction KW - Japanese N2 - Background: Large language models have propelled recent advances in artificial intelligence technology, facilitating the extraction of medical information from unstructured data such as medical records. Although named entity recognition (NER) is used to extract data from physicians? records, it has yet to be widely applied to pharmaceutical care records. Objective: In this study, we aimed to investigate the feasibility of automatic extraction of the information regarding patients? diseases and symptoms from pharmaceutical care records. The verification was performed using Medical Named Entity Recognition-Japanese (MedNER-J), a Japanese disease-extraction system designed for physicians? records. Methods: MedNER-J was applied to subjective, objective, assessment, and plan data from the care records of 49 patients who received cefazolin sodium injection at Keio University Hospital between April 2018 and March 2019. The performance of MedNER-J was evaluated in terms of precision, recall, and F1-score. Results: The F1-scores of NER for subjective, objective, assessment, and plan data were 0.46, 0.70, 0.76, and 0.35, respectively. In NER and positive-negative classification, the F1-scores were 0.28, 0.39, 0.64, and 0.077, respectively. The F1-scores of NER for objective (0.70) and assessment data (0.76) were higher than those for subjective and plan data, which supported the superiority of NER performance for objective and assessment data. This might be because objective and assessment data contained many technical terms, similar to the training data for MedNER-J. Meanwhile, the F1-score of NER and positive-negative classification was high for assessment data alone (F1-score=0.64), which was attributed to the similarity of its description format and contents to those of the training data. Conclusions: MedNER-J successfully read pharmaceutical care records and showed the best performance for assessment data. However, challenges remain in analyzing records other than assessment data. Therefore, it will be necessary to reinforce the training data for subjective data in order to apply the system to pharmaceutical care records. UR - https://formative.jmir.org/2024/1/e55798 UR - http://dx.doi.org/10.2196/55798 UR - http://www.ncbi.nlm.nih.gov/pubmed/38833694 ID - info:doi/10.2196/55798 ER - TY - JOUR AU - Azizi, Mehrnoosh AU - Jamali, Akbar Ali AU - Spiteri, J. Raymond PY - 2024/6/4 TI - Identifying X (Formerly Twitter) Posts Relevant to Dementia and COVID-19: Machine Learning Approach JO - JMIR Form Res SP - e49562 VL - 8 KW - machine learning KW - dementia KW - Alzheimer disease KW - COVID-19 KW - X (Twitter) KW - natural language processing N2 - Background: During the pandemic, patients with dementia were identified as a vulnerable population. X (formerly Twitter) became an important source of information for people seeking updates on COVID-19, and, therefore, identifying posts (formerly tweets) relevant to dementia can be an important support for patients with dementia and their caregivers. However, mining and coding relevant posts can be daunting due to the sheer volume and high percentage of irrelevant posts. Objective: The objective of this study was to automate the identification of posts relevant to dementia and COVID-19 using natural language processing and machine learning (ML) algorithms. Methods: We used a combination of natural language processing and ML algorithms with manually annotated posts to identify posts relevant to dementia and COVID-19. We used 3 data sets containing more than 100,000 posts and assessed the capability of various algorithms in correctly identifying relevant posts. Results: Our results showed that (pretrained) transfer learning algorithms outperformed traditional ML algorithms in identifying posts relevant to dementia and COVID-19. Among the algorithms tested, the transfer learning algorithm A Lite Bidirectional Encoder Representations from Transformers (ALBERT) achieved an accuracy of 82.92% and an area under the curve of 83.53%. ALBERT substantially outperformed the other algorithms tested, further emphasizing the superior performance of transfer learning algorithms in the classification of posts. Conclusions: Transfer learning algorithms such as ALBERT are highly effective in identifying topic-specific posts, even when trained with limited or adjacent data, highlighting their superiority over other ML algorithms and applicability to other studies involving analysis of social media posts. Such an automated approach reduces the workload of manual coding of posts and facilitates their analysis for researchers and policy makers to support patients with dementia and their caregivers and other vulnerable populations. UR - https://formative.jmir.org/2024/1/e49562 UR - http://dx.doi.org/10.2196/49562 UR - http://www.ncbi.nlm.nih.gov/pubmed/38833288 ID - info:doi/10.2196/49562 ER - TY - JOUR AU - Kumarasamy, Vithusa AU - Goodfellow, Nicole AU - Ferron, Mae Era AU - Wright, L. Amy PY - 2024/6/4 TI - Evaluating the Problem of Fraudulent Participants in Health Care Research: Multimethod Pilot Study JO - JMIR Form Res SP - e51530 VL - 8 KW - fraudulent participants KW - threats to data integrity KW - online recruitment KW - multimethod study KW - health care research KW - bots KW - social media N2 - Background: The shift toward online recruitment methods, accelerated by the COVID-19 pandemic, has brought to the forefront the growing concern of encountering fraudulent participants in health care research. The increasing prevalence of this issue poses a serious threat to the reliability and integrity of research data and subsequent findings. Objective: This study aims to explore the experiences of health care researchers (HCRs) who have encountered fraudulent participants while using online recruitment methods and platforms. The primary objective was to gain insights into how researchers detect and mitigate fraudulent behavior in their work and provide prevention recommendations. Methods: A multimethod sequential design was used for this pilot study, comprising a quantitative arm involving a web-based survey followed by a qualitative arm featuring semistructured interviews. The qualitative description approach framed the qualitative arm of the study. Sample sizes for the quantitative and qualitative arms were based on pragmatic considerations that in part stemmed from encountering fraudulent participants in a concurrent study. Content analysis was used to analyze open-ended survey questions and interview data. Results: A total of 37 HCRs participated, with 35% (13/37) of them engaging in qualitative interviews. Online platforms such as Facebook, email, Twitter (subsequently rebranded X), and newsletters were the most used methods for recruitment. A total of 84% (31/37) of participants indicated that fraudulent participation occurred in studies that mentioned incentives in their recruitment communications, with 71% (26/37) of HCRs offering physical or electronic gift cards as incentives. Researchers identified several indicators of suspicious behavior, including email surges, discrepancies in contact or personal information, geographical inconsistencies, and suspicious responses to survey questions. HCRs emphasized the need for a comprehensive screening protocol that extends beyond eligibility checks and is seamlessly integrated into the study protocol, grant applications, and research ethics board submissions. Conclusions: This study sheds light on the intricate and pervasive problem of fraudulent participation in health care research using online recruitment methods. The findings underscore the importance of vigilance and proactivity among HCRs in identifying, preventing, and addressing fraudulent behavior. To effectively tackle this challenge, researchers are encouraged to develop a comprehensive prevention strategy and establish a community of practice, facilitating real-time access to solutions and support and the promotion of ethical research practices. This collaborative approach will enable researchers to effectively address the issue of fraudulent participation, ensuring the conduct of high-quality and ethically sound research in the digital age. UR - https://formative.jmir.org/2024/1/e51530 UR - http://dx.doi.org/10.2196/51530 UR - http://www.ncbi.nlm.nih.gov/pubmed/38833292 ID - info:doi/10.2196/51530 ER - TY - JOUR AU - Umibe, Akiko AU - Fushiki, Hiroaki AU - Tsunoda, Reiko AU - Kuroda, Tatsuaki AU - Kuroda, Kazuhiro AU - Tanaka, Yasuhiro PY - 2024/6/4 TI - Development of a Subjective Visual Vertical Test System Using a Smartphone With Virtual Reality Goggles for Screening of Otolithic Dysfunction: Observational Study JO - JMIR Form Res SP - e53642 VL - 8 KW - vestibular function tests KW - telemedicine KW - smartphone KW - virtual reality KW - otolith dysfunction screening tool KW - vestibular evoked myogenic potential KW - iPhone KW - mobile phone N2 - Background: The subjective visual vertical (SVV) test can evaluate otolith function and spatial awareness and is performed in dedicated vertigo centers using specialized equipment; however, it is not otherwise widely used because of the specific equipment and space requirements. An SVV test smartphone app was developed to easily perform assessments in outpatient facilities. Objective: This study aimed to verify whether the SVV test smartphone app with commercially available virtual reality goggles can be used in a clinical setting. Methods: The reference range was calculated for 15 healthy participants. We included 14 adult patients with unilateral vestibular neuritis, sudden sensorineural hearing loss with vertigo, and Meniere disease and investigated the correlation between the SVV test results and vestibular evoked myogenic potential (VEMP) results. Results: The SVV reference range of healthy participants for the sitting front-facing position was small, ranging from ?2.6º to 2.3º. Among the 14 patients, 6 (43%) exceeded the reference range for healthy participants. The SVV of patients with vestibular neuritis and sudden sensorineural hearing loss tended to deviate to the affected side. A total of 9 (64%) had abnormal cervical VEMP (cVEMP) values and 6 (43%) had abnormal ocular VEMP (oVEMP) values. No significant difference was found between the presence or absence of abnormal SVV values and the presence or absence of abnormal cVEMP and oVEMP values; however, the odds ratios (ORs) suggested a higher likelihood of abnormal SVV values among those with abnormal cVEMP and oVEMP responses (OR 2.40, 95% CI 0.18-32.88; P>.99; and OR 2, 95% CI 0.90-4.45; P=.46, respectively). Conclusions: The SVV app can be used anywhere and in a short period while reducing directional bias by using virtual reality goggles, thus making it highly versatile and useful as a practical otolith dysfunction screening tool. UR - https://formative.jmir.org/2024/1/e53642 UR - http://dx.doi.org/10.2196/53642 UR - http://www.ncbi.nlm.nih.gov/pubmed/38833295 ID - info:doi/10.2196/53642 ER - TY - JOUR AU - Kim, Min Hyung AU - Kang, Hyoeun AU - Lee, Chaeyoon AU - Park, Hyuk Jong AU - Chung, Kyung Mi AU - Kim, Miran AU - Kim, Young Na AU - Lee, Jun Hye PY - 2024/6/3 TI - Evaluation of the Clinical Efficacy and Trust in AI-Assisted Embryo Ranking: Survey-Based Prospective Study JO - J Med Internet Res SP - e52637 VL - 26 KW - assisted reproductive technology KW - in vitro fertilization KW - artificial intelligence KW - intraobserver and interobserver agreements KW - embryos KW - embryologists N2 - Background: Current embryo assessment methods for in vitro fertilization depend on subjective morphological assessments. Recently, artificial intelligence (AI) has emerged as a promising tool for embryo assessment; however, its clinical efficacy and trustworthiness remain unproven. Simulation studies may provide additional evidence, provided that they are meticulously designed to mitigate bias and variance. Objective: The primary objective of this study was to evaluate the benefits of an AI model for predicting clinical pregnancy through well-designed simulations. The secondary objective was to identify the characteristics of and potential bias in the subgroups of embryologists with varying degrees of experience. Methods: This simulation study involved a questionnaire-based survey conducted on 61 embryologists with varying levels of experience from 12 in vitro fertilization clinics. The survey was conducted via Google Forms (Google Inc) in three phases: (1) phase 1, an initial assessment (December 23, 2022, to January 22, 2023); (2) phase 2, a validation assessment (March 6, 2023, to April 5, 2023); and (3) phase 3 an AI-guided assessment (March 6, 2023, to April 5, 2023). Inter- and intraobserver assessments and the accuracy of embryo selection from 360 day-5 embryos before and after AI guidance were analyzed for all embryologists and subgroups of senior and junior embryologists. Results: With AI guidance, the interobserver agreement increased from 0.355 to 0.527 and from 0.440 to 0.524 for junior and senior embryologists, respectively, thus reaching similar levels of agreement. In a test of accurate embryo selection with 90 questions, the numbers of correct responses by the embryologists only, embryologists with AI guidance, and AI only were 34 (38%), 45 (50%), and 59 (66%), respectively. Without AI, the average score (accuracy) of the junior group was 33.516 (37%), while that of the senior group was 35.967 (40%), with P<.001 in the t test. With AI guidance, the average score (accuracy) of the junior group increased to 46.581 (52%), reaching a level similar to that of the senior embryologists of 44.833 (50%), with P=.34. Junior embryologists had a higher level of trust in the AI score. Conclusions: This study demonstrates the potential benefits of AI in selecting embryos with high chances of pregnancy, particularly for embryologists with 5 years or less of experience, possibly due to their trust in AI. Thus, using AI as an auxiliary tool in clinical practice has the potential to improve embryo assessment and increase the probability of a successful pregnancy. UR - https://www.jmir.org/2024/1/e52637 UR - http://dx.doi.org/10.2196/52637 UR - http://www.ncbi.nlm.nih.gov/pubmed/38830209 ID - info:doi/10.2196/52637 ER - TY - JOUR AU - Meczner, András AU - Cohen, Nathan AU - Qureshi, Aleem AU - Reza, Maria AU - Sutaria, Shailen AU - Blount, Emily AU - Bagyura, Zsolt AU - Malak, Tamer PY - 2024/5/31 TI - Controlling Inputter Variability in Vignette Studies Assessing Web-Based Symptom Checkers: Evaluation of Current Practice and Recommendations for Isolated Accuracy Metrics JO - JMIR Form Res SP - e49907 VL - 8 KW - symptom checker KW - accuracy KW - vignette studies KW - variability KW - methods KW - triage KW - evaluation KW - vignette KW - performance KW - metrics KW - mobile phone N2 - Background: The rapid growth of web-based symptom checkers (SCs) is not matched by advances in quality assurance. Currently, there are no widely accepted criteria assessing SCs? performance. Vignette studies are widely used to evaluate SCs, measuring the accuracy of outcome. Accuracy behaves as a composite metric as it is affected by a number of individual SC- and tester-dependent factors. In contrast to clinical studies, vignette studies have a small number of testers. Hence, measuring accuracy alone in vignette studies may not provide a reliable assessment of performance due to tester variability. Objective: This study aims to investigate the impact of tester variability on the accuracy of outcome of SCs, using clinical vignettes. It further aims to investigate the feasibility of measuring isolated aspects of performance. Methods: Healthily?s SC was assessed using 114 vignettes by 3 groups of 3 testers who processed vignettes with different instructions: free interpretation of vignettes (free testers), specified chief complaints (partially free testers), and specified chief complaints with strict instruction for answering additional symptoms (restricted testers). ? statistics were calculated to assess agreement of top outcome condition and recommended triage. Crude and adjusted accuracy was measured against a gold standard. Adjusted accuracy was calculated using only results of consultations identical to the vignette, following a review and selection process. A feasibility study for assessing symptom comprehension of SCs was performed using different variations of 51 chief complaints across 3 SCs. Results: Intertester agreement of most likely condition and triage was, respectively, 0.49 and 0.51 for the free tester group, 0.66 and 0.66 for the partially free group, and 0.72 and 0.71 for the restricted group. For the restricted group, accuracy ranged from 43.9% to 57% for individual testers, averaging 50.6% (SD 5.35%). Adjusted accuracy was 56.1%. Assessing symptom comprehension was feasible for all 3 SCs. Comprehension scores ranged from 52.9% and 68%. Conclusions: We demonstrated that by improving standardization of the vignette testing process, there is a significant improvement in the agreement of outcome between testers. However, significant variability remained due to uncontrollable tester-dependent factors, reflected by varying outcome accuracy. Tester-dependent factors, combined with a small number of testers, limit the reliability and generalizability of outcome accuracy when used as a composite measure in vignette studies. Measuring and reporting different aspects of SC performance in isolation provides a more reliable assessment of SC performance. We developed an adjusted accuracy measure using a review and selection process to assess data algorithm quality. In addition, we demonstrated that symptom comprehension with different input methods can be feasibly compared. Future studies reporting accuracy need to apply vignette testing standardization and isolated metrics. UR - https://formative.jmir.org/2024/1/e49907 UR - http://dx.doi.org/10.2196/49907 UR - http://www.ncbi.nlm.nih.gov/pubmed/38820578 ID - info:doi/10.2196/49907 ER - TY - JOUR AU - Adedoja, Dorcas AU - Kuhns, M. Lisa AU - Radix, Asa AU - Garofalo, Robert AU - Brin, Maeve AU - Schnall, Rebecca PY - 2024/5/30 TI - MyPEEPS Mobile App for HIV Prevention Among Transmasculine Youth: Adaptation Through Community-Based Feedback and Usability Evaluation JO - JMIR Form Res SP - e56561 VL - 8 KW - HIV KW - mobile app KW - transgender men KW - transmasculine N2 - Background: Transgender men and transmasculine youth are at high risk for acquiring HIV. Growing research on transgender men demonstrates increased HIV risk and burden compared with the general US population. Despite biomedical advancements in HIV prevention, there remains a dearth of evidence-based, sexual health HIV prevention interventions for young transgender men. MyPEEPS (Male Youth Pursuing Empowerment, Education, and Prevention around Sexuality) Mobile is a web-based app that builds on extensive formative community?informed work to develop an evidence-based HIV prevention intervention. Our study team developed and tested the MyPEEPS Mobile intervention for 13- to 18-year-old cisgender young men in a national randomized controlled trial, which demonstrated efficacy to reduce sexual risk in the short term?at 3-month follow-up. Trans men and transmasculine youth resonated with basic HIV educational information and sexual scenarios of the original MyPEEPS app for cisgender men, but recognized the app's lack of transmasculine specificity. Objective: The purpose of this study is to detail the user-centered design methods to adapt, improve the user interface, and enhance the usability of the MyPEEPS Mobile app for young transgender men and transmasculine youth. Methods: The MyPEEPS Mobile app for young transgender men was adapted through a user-centered design approach, which included an iterative review of the adapted prototype by expert advisors and a youth advisory board. The app was then evaluated through a rigorous usability evaluation. Results: MyPEEPS Mobile is among the first mobile health interventions developed to meet the specific needs of young transgender men and transmasculine youth to reduce HIV risk behaviors. While many of the activities in the original MyPEEPS Mobile were rigorously developed and tested, there was a need to adapt our intervention to meet the specific needs and risk factors among young transgender men and transmasculine youth. The findings from this study describe the adaptation of these activities through feedback from a youth advisory board and expert advisors. Following adaptation of the content, the app underwent a rigorous usability assessment through an evaluation with experts in human-computer interaction (n=5) and targeted end users (n=20). Conclusions: Usability and adaptation findings demonstrate that the MyPEEPS Mobile app is highly usable and perceived as potentially useful for targeting HIV risk behaviors in young transgender men and transmasculine youth. UR - https://formative.jmir.org/2024/1/e56561 UR - http://dx.doi.org/10.2196/56561 UR - http://www.ncbi.nlm.nih.gov/pubmed/38814701 ID - info:doi/10.2196/56561 ER - TY - JOUR AU - Invernici, Francesco AU - Bernasconi, Anna AU - Ceri, Stefano PY - 2024/5/30 TI - Searching COVID-19 Clinical Research Using Graph Queries: Algorithm Development and Validation JO - J Med Internet Res SP - e52655 VL - 26 KW - big data corpus KW - clinical research KW - co-occurrence network KW - COVID-19 Open Research Dataset KW - CORD-19 KW - graph search KW - Named Entity Recognition KW - Neo4j KW - text mining N2 - Background: Since the beginning of the COVID-19 pandemic, >1 million studies have been collected within the COVID-19 Open Research Dataset, a corpus of manuscripts created to accelerate research against the disease. Their related abstracts hold a wealth of information that remains largely unexplored and difficult to search due to its unstructured nature. Keyword-based search is the standard approach, which allows users to retrieve the documents of a corpus that contain (all or some of) the words in a target list. This type of search, however, does not provide visual support to the task and is not suited to expressing complex queries or compensating for missing specifications. Objective: This study aims to consider small graphs of concepts and exploit them for expressing graph searches over existing COVID-19?related literature, leveraging the increasing use of graphs to represent and query scientific knowledge and providing a user-friendly search and exploration experience. Methods: We considered the COVID-19 Open Research Dataset corpus and summarized its content by annotating the publications? abstracts using terms selected from the Unified Medical Language System and the Ontology of Coronavirus Infectious Disease. Then, we built a co-occurrence network that includes all relevant concepts mentioned in the corpus, establishing connections when their mutual information is relevant. A sophisticated graph query engine was built to allow the identification of the best matches of graph queries on the network. It also supports partial matches and suggests potential query completions using shortest paths. Results: We built a large co-occurrence network, consisting of 128,249 entities and 47,198,965 relationships; the GRAPH-SEARCH interface allows users to explore the network by formulating or adapting graph queries; it produces a bibliography of publications, which are globally ranked; and each publication is further associated with the specific parts of the query that it explains, thereby allowing the user to understand each aspect of the matching. Conclusions: Our approach supports the process of query formulation and evidence search upon a large text corpus; it can be reapplied to any scientific domain where documents corpora and curated ontologies are made available. UR - https://www.jmir.org/2024/1/e52655 UR - http://dx.doi.org/10.2196/52655 UR - http://www.ncbi.nlm.nih.gov/pubmed/38814687 ID - info:doi/10.2196/52655 ER - TY - JOUR AU - Li, Wenhao AU - O'Hara, Rebecca AU - Hull, Louise M. AU - Slater, Helen AU - Sirohi, Diksha AU - Parker, A. Melissa AU - Bidargaddi, Niranjan PY - 2024/5/29 TI - Enabling Health Information Recommendation Using Crowdsourced Refinement in Web-Based Health Information Applications: User-Centered Design Approach and EndoZone Informatics Case Study JO - JMIR Hum Factors SP - e52027 VL - 11 KW - information recommendation KW - crowdsourcing KW - health informatics KW - digital health KW - endometriosis N2 - Background: In the digital age, search engines and social media platforms are primary sources for health information, yet their commercial interests?focused algorithms often prioritize irrelevant content. Web-based health applications by reputable sources offer a solution to circumvent these biased algorithms. Despite this advantage, there remains a significant gap in research on the effective integration of content-ranking algorithms within these specialized health applications to ensure the delivery of personalized and relevant health information. Objective: This study introduces a generic methodology designed to facilitate the development and implementation of health information recommendation features within web-based health applications. Methods: We detail our proposed methodology, covering conceptual foundation and practical considerations through the stages of design, development, operation, review, and optimization in the software development life cycle. Using a case study, we demonstrate the practical application of the proposed methodology through the implementation of recommendation functionalities in the EndoZone platform, a platform dedicated to providing targeted health information on endometriosis. Results: Application of the proposed methodology in the EndoZone platform led to the creation of a tailored health information recommendation system known as EndoZone Informatics. Feedback from EndoZone stakeholders as well as insights from the implementation process validate the methodology?s utility in enabling advanced recommendation features in health information applications. Preliminary assessments indicate that the system successfully delivers personalized content, adeptly incorporates user feedback, and exhibits considerable flexibility in adjusting its recommendation logic. While certain project-specific design flaws were not caught in the initial stages, these issues were subsequently identified and rectified in the review and optimization stages. Conclusions: We propose a generic methodology to guide the design and implementation of health information recommendation functionality within web-based health information applications. By harnessing user characteristics and feedback for content ranking, this methodology enables the creation of personalized recommendations that align with individual user needs within trusted health applications. The successful application of our methodology in the development of EndoZone Informatics marks a significant progress toward personalized health information delivery at scale, tailored to the specific needs of users. UR - https://humanfactors.jmir.org/2024/1/e52027 UR - http://dx.doi.org/10.2196/52027 UR - http://www.ncbi.nlm.nih.gov/pubmed/38809588 ID - info:doi/10.2196/52027 ER - TY - JOUR AU - Tran, D. Amanda AU - White, E. Alice AU - Torok, R. Michelle AU - Jervis, H. Rachel AU - Albanese, A. Bernadette AU - Scallan Walter, J. Elaine PY - 2024/5/27 TI - Lessons Learned From a Sequential Mixed-Mode Survey Design to Recruit and Collect Data From Case-Control Study Participants: Formative Evaluation JO - JMIR Form Res SP - e56218 VL - 8 KW - case-control studies KW - mixed-mode design KW - epidemiologic study methods KW - web-based survey KW - telephone interview KW - public health KW - outbreak preparedness KW - COVID-19 KW - survey KW - recruitment KW - epidemiology KW - methods N2 - Background: Sequential mixed-mode surveys using both web-based surveys and telephone interviews are increasingly being used in observational studies and have been shown to have many benefits; however, the application of this survey design has not been evaluated in the context of epidemiological case-control studies. Objective: In this paper, we discuss the challenges, benefits, and limitations of using a sequential mixed-mode survey design for a case-control study assessing risk factors during the COVID-19 pandemic. Methods: Colorado adults testing positive for SARS-CoV-2 were randomly selected and matched to those with a negative SARS-CoV-2 test result from March to April 2021. Participants were first contacted by SMS text message to complete a self-administered web-based survey asking about community exposures and behaviors. Those who did not respond were contacted for a telephone interview. We evaluated the representativeness of survey participants to sample populations and compared sociodemographic characteristics, participant responses, and time and resource requirements by survey mode using descriptive statistics and logistic regression models. Results: Of enrolled case and control participants, most were interviewed by telephone (308/537, 57.4% and 342/648, 52.8%, respectively), with overall enrollment more than doubling after interviewers called nonresponders. Participants identifying as female or White non-Hispanic, residing in urban areas, and not working outside the home were more likely to complete the web-based survey. Telephone participants were more likely than web-based participants to be aged 18-39 years or 60 years and older and reside in areas with lower levels of education, more linguistic isolation, lower income, and more people of color. While there were statistically significant sociodemographic differences noted between web-based and telephone case and control participants and their respective sample pools, participants were more similar to sample pools when web-based and telephone responses were combined. Web-based participants were less likely to report close contact with an individual with COVID-19 (odds ratio [OR] 0.70, 95% CI 0.53-0.94) but more likely to report community exposures, including visiting a grocery store or retail shop (OR 1.55, 95% CI 1.13-2.12), restaurant or cafe or coffee shop (OR 1.52, 95% CI 1.20-1.92), attending a gathering (OR 1.69, 95% CI 1.34-2.15), or sport or sporting event (OR 1.05, 95% CI 1.05-1.88). The web-based survey required an average of 0.03 (SD 0) person-hours per enrolled participant and US $920 in resources, whereas the telephone interview required an average of 5.11 person-hours per enrolled participant and US $70,000 in interviewer wages. Conclusions: While we still encountered control recruitment challenges noted in other observational studies, the sequential mixed-mode design was an efficient method for recruiting a more representative group of participants for a case-control study with limited impact on data quality and should be considered during public health emergencies when timely and accurate exposure information is needed to inform control measures. UR - https://formative.jmir.org/2024/1/e56218 UR - http://dx.doi.org/10.2196/56218 UR - http://www.ncbi.nlm.nih.gov/pubmed/38801768 ID - info:doi/10.2196/56218 ER - TY - JOUR AU - Arango-Ibanez, Pablo Juan AU - Posso-Nuñez, Alejandro Jose AU - Díaz-Solórzano, Pablo Juan AU - Cruz-Suárez, Gustavo PY - 2024/5/24 TI - Evidence-Based Learning Strategies in Medicine Using AI JO - JMIR Med Educ SP - e54507 VL - 10 KW - artificial intelligence KW - large language models KW - ChatGPT KW - active recall KW - memory cues KW - LLMs KW - evidence-based KW - learning strategy KW - medicine KW - AI KW - medical education KW - knowledge KW - relevance UR - https://mededu.jmir.org/2024/1/e54507 UR - http://dx.doi.org/10.2196/54507 ID - info:doi/10.2196/54507 ER - TY - JOUR AU - Shao, Jian AU - Pan, Ying AU - Kou, Wei-Bin AU - Feng, Huyi AU - Zhao, Yu AU - Zhou, Kaixin AU - Zhong, Shao PY - 2024/5/24 TI - Generalization of a Deep Learning Model for Continuous Glucose Monitoring?Based Hypoglycemia Prediction: Algorithm Development and Validation Study JO - JMIR Med Inform SP - e56909 VL - 12 KW - hypoglycemia prediction KW - hypoglycemia KW - hypoglycemic KW - blood sugar KW - prediction KW - predictive KW - deep learning KW - generalization KW - machine learning KW - glucose KW - diabetes KW - continuous glucose monitoring KW - type 1 diabetes KW - type 2 diabetes KW - LSTM KW - long short-term memory N2 - Background: Predicting hypoglycemia while maintaining a low false alarm rate is a challenge for the wide adoption of continuous glucose monitoring (CGM) devices in diabetes management. One small study suggested that a deep learning model based on the long short-term memory (LSTM) network had better performance in hypoglycemia prediction than traditional machine learning algorithms in European patients with type 1 diabetes. However, given that many well-recognized deep learning models perform poorly outside the training setting, it remains unclear whether the LSTM model could be generalized to different populations or patients with other diabetes subtypes. Objective: The aim of this study was to validate LSTM hypoglycemia prediction models in more diverse populations and across a wide spectrum of patients with different subtypes of diabetes. Methods: We assembled two large data sets of patients with type 1 and type 2 diabetes. The primary data set including CGM data from 192 Chinese patients with diabetes was used to develop the LSTM, support vector machine (SVM), and random forest (RF) models for hypoglycemia prediction with a prediction horizon of 30 minutes. Hypoglycemia was categorized into mild (glucose=54-70 mg/dL) and severe (glucose<54 mg/dL) levels. The validation data set of 427 patients of European-American ancestry in the United States was used to validate the models and examine their generalizations. The predictive performance of the models was evaluated according to the sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Results: For the difficult-to-predict mild hypoglycemia events, the LSTM model consistently achieved AUC values greater than 97% in the primary data set, with a less than 3% AUC reduction in the validation data set, indicating that the model was robust and generalizable across populations. AUC values above 93% were also achieved when the LSTM model was applied to both type 1 and type 2 diabetes in the validation data set, further strengthening the generalizability of the model. Under different satisfactory levels of sensitivity for mild and severe hypoglycemia prediction, the LSTM model achieved higher specificity than the SVM and RF models, thereby reducing false alarms. Conclusions: Our results demonstrate that the LSTM model is robust for hypoglycemia prediction and is generalizable across populations or diabetes subtypes. Given its additional advantage of false-alarm reduction, the LSTM model is a strong candidate to be widely implemented in future CGM devices for hypoglycemia prediction. UR - https://medinform.jmir.org/2024/1/e56909 UR - http://dx.doi.org/10.2196/56909 ID - info:doi/10.2196/56909 ER - TY - JOUR AU - Li, Xiancheng AU - Gill, Aneet AU - Panzarasa, Pietro AU - Bestwick, Jonathan AU - Schrag, Anette AU - Noyce, Alastair AU - De Simoni, Anna PY - 2024/5/24 TI - Web Application to Enable Online Social Interactions in a Parkinson Disease Risk Cohort: Feasibility Study and Social Network Analysis JO - JMIR Form Res SP - e51977 VL - 8 KW - pilot studies KW - network analysis KW - Parkinson disease KW - risk factors KW - risk KW - risk cohort KW - social interaction KW - development KW - neurodegenerative disease KW - neurodegenerative KW - United Kingdom KW - feasibility KW - design KW - pilot KW - engagement KW - users KW - online forum KW - online network KW - online KW - regression analysis N2 - Background: There is evidence that social interaction has an inverse association with the development of neurodegenerative diseases. PREDICT-Parkinson Disease (PREDICT-PD) is an online UK cohort study that stratifies participants for risk of future Parkinson disease (PD). Objective: This study aims to explore the methodological approach and feasibility of assessing the digital social characteristics of people at risk of developing PD and their social capital within the PREDICT-PD platform, making hypotheses about the relationship between web-based social engagement and potential predictive risk indicators of PD. Methods: A web-based application was built to enable social interaction through the PREDICT-PD portal. Feedback from existing members of the cohort was sought and informed the design of the pilot. Dedicated staff used weekly engagement activities, consisting of PD-related research, facts, and queries, to stimulate discussion. Data were collected by the hosting platform. We examined the pattern of connections generated over time through the cumulative number of posts and replies and ego networks using social network analysis. We used network metrics to describe the bonding, bridging, and linking of social capital among participants on the platform. Relevant demographic data and Parkinson risk scores (expressed as an odd 1:x) were analyzed using descriptive statistics. Regression analysis was conducted to estimate the relationship between risk scores (after log transformation) and network measures. Results: Overall, 219 participants took part in a 4-month pilot forum embedded in the study website. In it, 200 people (n=80, 40% male and n=113, 57% female) connected in a large group, where most pairs of users could reach one another either directly or indirectly through other users. A total of 59% (20/34) of discussions were spontaneously started by participants. Participation was asynchronous, with some individuals acting as ?brokers? between groups of discussions. As more participants joined the forum and connected to one another through online posts, distinct groups of connected users started to emerge. This pilot showed that a forum application within the cohort web platform was feasible and acceptable and fostered digital social interaction. Matching participants? web-based social engagement with previously collected data at individual level in the PREDICT-PD study was feasible, showing potential for future analyses correlating online network characteristics with the risk of PD over time, as well as testing digital social engagement as an intervention to modify the risk of developing neurodegenerative diseases. Conclusions: The results from the pilot suggest that an online forum can serve as an intervention to enhance social connectedness and investigate whether patterns of online engagement can impact the risk of developing PD through long-term follow-up. This highlights the potential of leveraging online platforms to study the role of social capital in moderating PD risk and underscores the feasibility of such approaches in future research or interventions. UR - https://formative.jmir.org/2024/1/e51977 UR - http://dx.doi.org/10.2196/51977 UR - http://www.ncbi.nlm.nih.gov/pubmed/38788211 ID - info:doi/10.2196/51977 ER - TY - JOUR AU - Yanez Touzet, Alvaro AU - Houhou, Tatiana AU - Rahic, Zerina AU - Kolias, Angelos AU - Yordanov, Stefan AU - Anderson, B. David AU - Laufer, Ilya AU - Li, Maggie AU - Grahovac, Gordan AU - Kotter, RN Mark AU - Davies, M. Benjamin AU - PY - 2024/5/24 TI - Reliability of a Smartphone App to Objectively Monitor Performance Outcomes in Degenerative Cervical Myelopathy: Observational Study JO - JMIR Form Res SP - e56889 VL - 8 KW - reproducibility of results KW - patient outcome assessment KW - smartphone KW - neurology KW - psychometrics KW - spinal cord compression KW - mobile phone N2 - Background: Developing new clinical measures for degenerative cervical myelopathy (DCM) is an AO Spine RECODE-DCM Research, an international and multi-stakeholder partnership, priority. Difficulties in detecting DCM and its changes cause diagnostic and treatment delays in clinical settings and heightened costs in clinical trials due to elevated recruitment targets. Digital outcome measures can tackle these challenges due to their ability to measure disease remotely, repeatedly, and more economically. Objective: The aim of this study is to assess the reliability of the MoveMed battery of performance outcome measures. Methods: A prospective observational study in decentralized secondary care was performed in England, United Kingdom. The primary outcome was to determine the test-retest reliability of the MoveMed performance outcomes using the intraclass correlation (ICC) of agreement . The secondary outcome was to determine the measurement error of the MoveMed performance outcomes using both the SE of the mean (SEM) of agreement and the smallest detectable change (SDC) of agreement . Criteria from the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) manual were used to determine adequate reliability (ie, ICC of agreement ?0.7) and risk of bias. Disease stability was controlled using 2 minimum clinically important difference (MCID) thresholds obtained from the literature on the patient-derived modified Japanese Orthopaedic Association (p-mJOA) score, namely, MCID ?1 point and MCID ?2 points. Results: In total, 7 adults aged 59.5 (SD 12.4) years who live with DCM and possess an approved smartphone participated in the study. All tests demonstrated moderate to excellent test-retest coefficients and low measurement errors. In the MCID ?1 group, ICC of agreement values were 0.84-0.94 in the fast tap test, 0.89-0.95 in the hold test, 0.95 in the typing test, and 0.98 in the stand and walk test. SEM of agreement values were ±1 tap, ±1%-3% stability score points, ±0.06 keys per second, and ±10 steps per minute, respectively. SDC of agreement values were ±3 taps, ±4%-7% stability score points, ±0.2 keys per second, and ±27 steps per minute, respectively. In the MCID ?2 group, ICC of agreement values were 0.61-0.91, 0.75-0.77, 0.98, and 0.62, respectively; SEM of agreement values were ±1 tap, ±2%-4% stability score points, ±0.06 keys per second, and ±10 steps per minute, respectively; and SDC of agreement values were ±3-7 taps, ±7%-10% stability score points, ±0.2 keys per second, and ±27 steps per minute, respectively. Furthermore, the fast tap, hold, and typing tests obtained sufficient ratings (ICC of agreement ?0.7) in both MCID ?1 and MCID ?2 groups. No risk of bias factors from the COSMIN Risk of Bias checklist were recorded. Conclusions: The criteria from COSMIN provide ?very good? quality evidence of the reliability of the MoveMed tests in an adult population living with DCM. UR - https://formative.jmir.org/2024/1/e56889 UR - http://dx.doi.org/10.2196/56889 UR - http://www.ncbi.nlm.nih.gov/pubmed/38787602 ID - info:doi/10.2196/56889 ER - TY - JOUR AU - Bandiera, Carole AU - Pasquier, Jérôme AU - Locatelli, Isabella AU - Schneider, P. Marie PY - 2024/5/22 TI - Using a Semiautomated Procedure (CleanADHdata.R Script) to Clean Electronic Adherence Monitoring Data: Tutorial JO - JMIR Form Res SP - e51013 VL - 8 KW - medication adherence KW - digital technology KW - digital pharmacy KW - electronic adherence monitoring KW - data management KW - data cleaning KW - research methodology KW - algorithms KW - R KW - semiautomated KW - code KW - coding KW - computer science KW - computer programming KW - medications KW - computer script N2 - Background: Patient adherence to medications can be assessed using interactive digital health technologies such as electronic monitors (EMs). Changes in treatment regimens and deviations from EM use over time must be characterized to establish the actual level of medication adherence. Objective: We developed the computer script CleanADHdata.R to clean raw EM adherence data, and this tutorial is a guide for users. Methods: In addition to raw EM data, we collected adherence start and stop monitoring dates and identified the prescribed regimens, the expected number of EM openings per day based on the prescribed regimen, EM use deviations, and patients? demographic data. The script formats the data longitudinally and calculates each day?s medication implementation. Results: We provided a simulated data set for 10 patients, for which 15 EMs were used over a median period of 187 (IQR 135-342) days. The median patient implementation before and after EM raw data cleaning was 83.3% (IQR 71.5%-93.9%) and 97.3% (IQR 95.8%-97.6%), respectively (?+14%). This difference is substantial enough to consider EM data cleaning to be capable of avoiding data misinterpretation and providing a cleaned data set for the adherence analysis in terms of implementation and persistence. Conclusions: The CleanADHdata.R script is a semiautomated procedure that increases standardization and reproducibility. This script has broader applicability within the realm of digital health, as it can be used to clean adherence data collected with diverse digital technologies. UR - https://formative.jmir.org/2024/1/e51013 UR - http://dx.doi.org/10.2196/51013 UR - http://www.ncbi.nlm.nih.gov/pubmed/38776539 ID - info:doi/10.2196/51013 ER - TY - JOUR AU - Anthonimuthu, Jeganathan Danny AU - Hejlesen, Ole AU - Zwisler, Olsen Ann-Dorthe AU - Udsen, Witt Flemming PY - 2024/5/20 TI - Application of Machine Learning in Multimorbidity Research: Protocol for a Scoping Review JO - JMIR Res Protoc SP - e53761 VL - 13 KW - multimorbidity KW - multiple long-term conditions KW - machine learning KW - artificial intelligence KW - scoping review KW - protocol KW - chronic conditions KW - health care system KW - health care N2 - Background: Multimorbidity, defined as the coexistence of multiple chronic conditions, poses significant challenges to health care systems on a global scale. It is associated with increased mortality, reduced quality of life, and increased health care costs. The burden of multimorbidity is expected to worsen if no effective intervention is taken. Machine learning has the potential to assist in addressing these challenges since it offers advanced analysis and decision-making capabilities, such as disease prediction, treatment development, and clinical strategies. Objective: This paper represents the protocol of a scoping review that aims to identify and explore the current literature concerning the use of machine learning for patients with multimorbidity. More precisely, the objective is to recognize various machine learning models, the patient groups involved, features considered, types of input data, the maturity of the machine learning algorithms, and the outcomes from these machine learning models. Methods: The scoping review will be based on the guidelines of the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Five databases (PubMed, Embase, IEEE, Web of Science, and Scopus) are chosen to conduct a literature search. Two reviewers will independently screen the titles, abstracts, and full texts of identified studies based on predefined eligibility criteria. Covidence (Veritas Health Innovation Ltd) will be used as a tool for managing and screening papers. Only studies that examine more than 1 chronic disease or individuals with a single chronic condition at risk of developing another will be included in the scoping review. Data from the included studies will be collected using Microsoft Excel (Microsoft Corp). The focus of the data extraction will be on bibliographical information, objectives, study populations, types of input data, types of algorithm, performance, maturity of the algorithms, and outcome. Results: The screening process will be presented in a PRISMA-ScR flow diagram. The findings of the scoping review will be conveyed through a narrative synthesis. Additionally, data extracted from the studies will be presented in more comprehensive formats, such as charts or tables. The results will be presented in a forthcoming scoping review, which will be published in a peer-reviewed journal. Conclusions: To our knowledge, this may be the first scoping review to investigate the use of machine learning in multimorbidity research. The goal of the scoping review is to summarize the field of literature on machine learning in patients with multiple chronic conditions, highlight different approaches, and potentially discover research gaps. The results will offer insights for future research within this field, contributing to developments that can enhance patient outcomes. International Registered Report Identifier (IRRID): PRR1-10.2196/53761 UR - https://www.researchprotocols.org/2024/1/e53761 UR - http://dx.doi.org/10.2196/53761 UR - http://www.ncbi.nlm.nih.gov/pubmed/38767948 ID - info:doi/10.2196/53761 ER - TY - JOUR AU - Harada, Yukinori AU - Sakamoto, Tetsu AU - Sugimoto, Shu AU - Shimizu, Taro PY - 2024/5/17 TI - Longitudinal Changes in Diagnostic Accuracy of a Differential Diagnosis List Developed by an AI-Based Symptom Checker: Retrospective Observational Study JO - JMIR Form Res SP - e53985 VL - 8 KW - atypical presentations KW - diagnostic accuracy KW - diagnosis KW - diagnostics KW - symptom checker KW - uncommon diseases KW - symptom checkers KW - uncommon KW - rare KW - artificial intelligence N2 - Background: Artificial intelligence (AI) symptom checker models should be trained using real-world patient data to improve their diagnostic accuracy. Given that AI-based symptom checkers are currently used in clinical practice, their performance should improve over time. However, longitudinal evaluations of the diagnostic accuracy of these symptom checkers are limited. Objective: This study aimed to assess the longitudinal changes in the accuracy of differential diagnosis lists created by an AI-based symptom checker used in the real world. Methods: This was a single-center, retrospective, observational study. Patients who visited an outpatient clinic without an appointment between May 1, 2019, and April 30, 2022, and who were admitted to a community hospital in Japan within 30 days of their index visit were considered eligible. We only included patients who underwent an AI-based symptom checkup at the index visit, and the diagnosis was finally confirmed during follow-up. Final diagnoses were categorized as common or uncommon, and all cases were categorized as typical or atypical. The primary outcome measure was the accuracy of the differential diagnosis list created by the AI-based symptom checker, defined as the final diagnosis in a list of 10 differential diagnoses created by the symptom checker. To assess the change in the symptom checker?s diagnostic accuracy over 3 years, we used a chi-square test to compare the primary outcome over 3 periods: from May 1, 2019, to April 30, 2020 (first year); from May 1, 2020, to April 30, 2021 (second year); and from May 1, 2021, to April 30, 2022 (third year). Results: A total of 381 patients were included. Common diseases comprised 257 (67.5%) cases, and typical presentations were observed in 298 (78.2%) cases. Overall, the accuracy of the differential diagnosis list created by the AI-based symptom checker was 172 (45.1%), which did not differ across the 3 years (first year: 97/219, 44.3%; second year: 32/72, 44.4%; and third year: 43/90, 47.7%; P=.85). The accuracy of the differential diagnosis list created by the symptom checker was low in those with uncommon diseases (30/124, 24.2%) and atypical presentations (12/83, 14.5%). In the multivariate logistic regression model, common disease (P<.001; odds ratio 4.13, 95% CI 2.50-6.98) and typical presentation (P<.001; odds ratio 6.92, 95% CI 3.62-14.2) were significantly associated with the accuracy of the differential diagnosis list created by the symptom checker. Conclusions: A 3-year longitudinal survey of the diagnostic accuracy of differential diagnosis lists developed by an AI-based symptom checker, which has been implemented in real-world clinical practice settings, showed no improvement over time. Uncommon diseases and atypical presentations were independently associated with a lower diagnostic accuracy. In the future, symptom checkers should be trained to recognize uncommon conditions. UR - https://formative.jmir.org/2024/1/e53985 UR - http://dx.doi.org/10.2196/53985 UR - http://www.ncbi.nlm.nih.gov/pubmed/38758588 ID - info:doi/10.2196/53985 ER - TY - JOUR AU - Gandrup, Julie AU - Selby, A. David AU - Dixon, G. William PY - 2024/5/14 TI - Classifying Self-Reported Rheumatoid Arthritis Flares Using Daily Patient-Generated Data From a Smartphone App: Exploratory Analysis Applying Machine Learning Approaches JO - JMIR Form Res SP - e50679 VL - 8 KW - rheumatoid arthritis KW - flare KW - patient-generated health data KW - smartphone KW - mobile health KW - machine learning KW - arthritis KW - rheumatic KW - rheumatism KW - joint KW - joints KW - arthritic KW - musculoskeletal KW - flares KW - classify KW - classification KW - symptom KW - symptoms KW - mobile phone N2 - Background: The ability to predict rheumatoid arthritis (RA) flares between clinic visits based on real-time, longitudinal patient-generated data could potentially allow for timely interventions to avoid disease worsening. Objective: This exploratory study aims to investigate the feasibility of using machine learning methods to classify self-reported RA flares based on a small data set of daily symptom data collected on a smartphone app. Methods: Daily symptoms and weekly flares reported on the Remote Monitoring of Rheumatoid Arthritis (REMORA) smartphone app from 20 patients with RA over 3 months were used. Predictors were several summary features of the daily symptom scores (eg, pain and fatigue) collected in the week leading up to the flare question. We fitted 3 binary classifiers: logistic regression with and without elastic net regularization, a random forest, and naive Bayes. Performance was evaluated according to the area under the curve (AUC) of the receiver operating characteristic curve. For the best-performing model, we considered sensitivity and specificity for different thresholds in order to illustrate different ways in which the predictive model could behave in a clinical setting. Results: The data comprised an average of 60.6 daily reports and 10.5 weekly reports per participant. Participants reported a median of 2 (IQR 0.75-4.25) flares each over a median follow-up time of 81 (IQR 79-82) days. AUCs were broadly similar between models, but logistic regression with elastic net regularization had the highest AUC of 0.82. At a cutoff requiring specificity to be 0.80, the corresponding sensitivity to detect flares was 0.60 for this model. The positive predictive value (PPV) in this population was 53%, and the negative predictive value (NPV) was 85%. Given the prevalence of flares, the best PPV achieved meant only around 2 of every 3 positive predictions were correct (PPV 0.65). By prioritizing a higher NPV, the model correctly predicted over 9 in every 10 non-flare weeks, but the accuracy of predicted flares fell to only 1 in 2 being correct (NPV and PPV of 0.92 and 0.51, respectively). Conclusions: Predicting self-reported flares based on daily symptom scorings in the preceding week using machine learning methods was feasible. The observed predictive accuracy might improve as we obtain more data, and these exploratory results need to be validated in an external cohort. In the future, analysis of frequently collected patient-generated data may allow us to predict flares before they unfold, opening opportunities for just-in-time adaptative interventions. Depending on the nature and implication of an intervention, different cutoff values for an intervention decision need to be considered, as well as the level of predictive certainty required. UR - https://formative.jmir.org/2024/1/e50679 UR - http://dx.doi.org/10.2196/50679 UR - http://www.ncbi.nlm.nih.gov/pubmed/38743480 ID - info:doi/10.2196/50679 ER - TY - JOUR AU - Farah, Line AU - Borget, Isabelle AU - Martelli, Nicolas AU - Vallee, Alexandre PY - 2024/5/13 TI - Suitability of the Current Health Technology Assessment of Innovative Artificial Intelligence-Based Medical Devices: Scoping Literature Review JO - J Med Internet Res SP - e51514 VL - 26 KW - artificial intelligence KW - machine learning KW - health technology assessment KW - medical devices KW - evaluation N2 - Background: Artificial intelligence (AI)?based medical devices have garnered attention due to their ability to revolutionize medicine. Their health technology assessment framework is lacking. Objective: This study aims to analyze the suitability of each health technology assessment (HTA) domain for the assessment of AI-based medical devices. Methods: We conducted a scoping literature review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. We searched databases (PubMed, Embase, and Cochrane Library), gray literature, and HTA agency websites. Results: A total of 10.1% (78/775) of the references were included. Data quality and integration are vital aspects to consider when describing and assessing the technical characteristics of AI-based medical devices during an HTA process. When it comes to implementing specialized HTA for AI-based medical devices, several practical challenges and potential barriers could be highlighted and should be taken into account (AI technological evolution timeline, data requirements, complexity and transparency, clinical validation and safety requirements, regulatory and ethical considerations, and economic evaluation). Conclusions: The adaptation of the HTA process through a methodological framework for AI-based medical devices enhances the comparability of results across different evaluations and jurisdictions. By defining the necessary expertise, the framework supports the development of a skilled workforce capable of conducting robust and reliable HTAs of AI-based medical devices. A comprehensive adapted HTA framework for AI-based medical devices can provide valuable insights into the effectiveness, cost-effectiveness, and societal impact of AI-based medical devices, guiding their responsible implementation and maximizing their benefits for patients and health care systems. UR - https://www.jmir.org/2024/1/e51514 UR - http://dx.doi.org/10.2196/51514 UR - http://www.ncbi.nlm.nih.gov/pubmed/38739911 ID - info:doi/10.2196/51514 ER - TY - JOUR AU - Attarha, Mouna AU - Mahncke, Henry AU - Merzenich, Michael PY - 2024/5/13 TI - The Real-World Usability, Feasibility, and Performance Distributions of Deploying a Digital Toolbox of Computerized Assessments to Remotely Evaluate Brain Health: Development and Usability Study JO - JMIR Form Res SP - e53623 VL - 8 KW - web-based cognitive assessment KW - remote data collection KW - neurocognition KW - cognitive profiles KW - normative assessment data KW - brain health KW - cognitive status KW - assessment accessibility N2 - Background: An ongoing global challenge is managing brain health and understanding how performance changes across the lifespan. Objective: We developed and deployed a set of self-administrable, computerized assessments designed to measure key indexes of brain health across the visual and auditory sensory modalities. In this pilot study, we evaluated the usability, feasibility, and performance distributions of the assessments in a home-based, real-world setting without supervision. Methods: Potential participants were untrained users who self-registered on an existing brain training app called BrainHQ. Participants were contacted via a recruitment email and registered remotely to complete a demographics questionnaire and 29 unique assessments on their personal devices. We examined participant engagement, descriptive and psychometric properties of the assessments, associations between performance and self-reported demographic variables, cognitive profiles, and factor loadings. Results: Of the 365,782 potential participants contacted via a recruitment email, 414 (0.11%) registered, of whom 367 (88.6%) completed at least one assessment and 104 (25.1%) completed all 29 assessments. Registered participants were, on average, aged 63.6 (SD 14.8; range 13-107) years, mostly female (265/414, 64%), educated (329/414, 79.5% with a degree), and White (349/414, 84.3% White and 48/414, 11.6% people of color). A total of 72% (21/29) of the assessments showed no ceiling or floor effects or had easily modifiable score bounds to eliminate these effects. When correlating performance with self-reported demographic variables, 72% (21/29) of the assessments were sensitive to age, 72% (21/29) of the assessments were insensitive to gender, 93% (27/29) of the assessments were insensitive to race and ethnicity, and 93% (27/29) of the assessments were insensitive to education-based differences. Assessments were brief, with a mean duration of 3 (SD 1.0) minutes per task. The pattern of performance across the assessments revealed distinctive cognitive profiles and loaded onto 4 independent factors. Conclusions: The assessments were both usable and feasible and warrant a full normative study. A digital toolbox of scalable and self-administrable assessments that can evaluate brain health at a glance (and longitudinally) may lead to novel future applications across clinical trials, diagnostics, and performance optimization. UR - https://formative.jmir.org/2024/1/e53623 UR - http://dx.doi.org/10.2196/53623 UR - http://www.ncbi.nlm.nih.gov/pubmed/38739916 ID - info:doi/10.2196/53623 ER - TY - JOUR AU - Denecke, Kerstin AU - May, Richard AU - AU - Rivera Romero, Octavio PY - 2024/5/13 TI - Potential of Large Language Models in Health Care: Delphi Study JO - J Med Internet Res SP - e52399 VL - 26 KW - large language models KW - LLMs KW - health care KW - Delphi study KW - natural language processing KW - NLP KW - artificial intelligence KW - language model KW - Delphi KW - future KW - innovation KW - interview KW - interviews KW - informatics KW - experience KW - experiences KW - attitude KW - attitudes KW - opinion KW - perception KW - perceptions KW - perspective KW - perspectives KW - implementation N2 - Background: A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications. Objective: The aim of this adapted Delphi study was to collect researchers? opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care. Methods: We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items. Results: The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data. Conclusions: Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice. UR - https://www.jmir.org/2024/1/e52399 UR - http://dx.doi.org/10.2196/52399 UR - http://www.ncbi.nlm.nih.gov/pubmed/38739445 ID - info:doi/10.2196/52399 ER - TY - JOUR AU - Hernández Encuentra, Eulàlia AU - Robles, Noemí AU - Angulo-Brunet, Ariadna AU - Cullen, David AU - del Arco, Ignacio PY - 2024/5/10 TI - Spanish and Catalan Versions of the eHealth Literacy Questionnaire: Translation, Cross-Cultural Adaptation, and Validation Study JO - J Med Internet Res SP - e49227 VL - 26 KW - eHealth literacy KW - eHealth KW - digital health KW - health literacy KW - questionnaire KW - eHealth Literacy Questionnaire KW - eHLQ KW - validation N2 - Background: The rise of digital health services, especially following the outbreak of COVID-19, has led to a need for health literacy policies that respond to people?s needs. Spain is a country with a highly developed digital health infrastructure, but there are currently no tools available to measure digital health literacy fully. A well-thought-through questionnaire with strong psychometric properties such as the eHealth Literacy Questionnaire (eHLQ) is important to assess people?s eHealth literacy levels, especially in the context of a fast-growing field such as digital health. Objective: This study aims to adapt the eHLQ and gather evidence of its psychometric quality in 2 of Spain?s official languages: Spanish and Catalan. Methods: A systematic cultural adaptation process was followed. Data from Spanish-speaking (n=400) and Catalan-speaking (n=400) people were collected. Confirmatory factor analysis was used to confirm the previously established factor structure. For reliability, the Cronbach ? and categorical ? were obtained for every subscale. Evidence of convergent and discriminant validity was provided through the correlation with the total score of the eHealth Literacy Scale. Evidence based on relations to other variables was evaluated by examining extreme values for educational level, socioeconomic level, and use of technology variables. Results: Regarding the confirmatory factor analysis, the 7-factor correlated model and the 7 one-factor models had adequate goodness-of-fit indexes for both Spanish and Catalan. Moreover, measurement invariance was established between the Spanish and Catalan versions. Reliability estimates were considered adequate as all the scales in both versions had values of >0.80. For convergent and discriminant validity evidence, the eHealth Literacy Scale showed moderate correlation with eHLQ scales in both versions (Spanish: range 0.57-0.76 and P<.001; Catalan: range 0.41-0.64 and P<.001). According to the relationship with external variables, all the eHLQ scales in both languages could discriminate between the maximum and minimum categories in level of education, socioeconomic level, and level of technology use. Conclusions: The Spanish and Catalan versions of the eHLQ appear to be psychometrically sound questionnaires for assessing digital health literacy. They could both be useful tools in Spain and Catalonia for researchers, policy makers, and health service managers to explore people?s needs, skills, and competencies and provide interesting insights into their interactions and engagement regarding their own experiences with digital health services, especially in the context of digital health growth in Spain. UR - https://www.jmir.org/2024/1/e49227 UR - http://dx.doi.org/10.2196/49227 UR - http://www.ncbi.nlm.nih.gov/pubmed/38728072 ID - info:doi/10.2196/49227 ER - TY - JOUR AU - Vujkovic, Branko AU - Brkovic, Voin AU - Paji?i?, Ana AU - Pavlovic, Vedrana AU - Stanisavljevic, Dejana AU - Krajnovi?, Du?anka AU - Jovic Vranes, Aleksandra PY - 2024/5/9 TI - Serbian Version of the eHealth Literacy Questionnaire (eHLQ): Translation, Cultural Adaptation, and Validation Study Among Primary Health Care Users JO - J Med Internet Res SP - e57963 VL - 26 KW - eHealth KW - digital health KW - eHLQ KW - eHealth Literacy Questionnaire KW - digital health literacy KW - primary healthcare KW - Serbia KW - questionnaire KW - technology KW - communication N2 - Background: As digital health services are increasingly developing and becoming more interactive in Serbia, a comprehensive instrument for measuring eHealth literacy (EHL) is needed. Objective: This study aimed to translate, culturally adapt, and investigate the psychometric properties of the Serbian version of the eHealth Literacy Questionnaire (eHLQ); to evaluate EHL in the population of primary health care (PHC) users in Serbia; and to explore factors associated with their EHL. Methods: The validation study was conducted in 8 PHC centers in the territory of the Ma?va district in Western Serbia. A stratified sampling method was used to obtain a representative sample. The Translation Integrity Procedure was followed to adapt the questionnaire to the Serbian language. The psychometric properties of the Serbian version of the eHLQ were analyzed through the examination of factorial structure, internal consistency, and test-retest reliability. Descriptive statistics were calculated to determine participant characteristics. Differences between groups were tested by the 2-tailed Students t test and ANOVA. Univariable and multivariable linear regression analyses were used to determine factors related to EHL. Results: A total of 475 PHC users were enrolled. The mean age was 51.0 (SD 17.3; range 19-94) years, and most participants were female (328/475, 69.1%). Confirmatory factor analysis validated the 7-factor structure of the questionnaire. Values for incremental fit index (0.96) and comparative fit index (0.95) were above the cutoff of ?0.95. The root mean square error of approximation value of 0.05 was below the suggested value of ?0.06. Cronbach ? of the entire scale was 0.95, indicating excellent scale reliability, with Cronbach ? ranging from 0.81 to 0.90 for domains. The intraclass correlation coefficient ranged from 0.63 to 0.82, indicating moderate to good test-retest reliability. The highest EHL mean scores were obtained for the understanding of health concepts and language (mean 2.86, SD 0.32) and feel safe and in control (mean 2.89, SD 0.33) domains. Statistically significant differences (all P<.05) for all 7 eHLQ scores were observed for age, education, perceived material status, perceived health status, searching for health information on the internet, and occupation (except domain 4). In multivariable regression models, searching for health information on the internet and being aged younger than 65 years were associated with higher values of all domain scores except the domain feel safe and in control for variable age. Conclusions: This study demonstrates that the Serbian version of the eHLQ can be a useful tool in the measurement of EHL and in the planning of digital health interventions at the population and individual level due to its strong psychometric properties in the Serbian context. UR - https://www.jmir.org/2024/1/e57963 UR - http://dx.doi.org/10.2196/57963 UR - http://www.ncbi.nlm.nih.gov/pubmed/38722675 ID - info:doi/10.2196/57963 ER - TY - JOUR AU - Tostain, Jean-Baptiste AU - Mathieu, Marina AU - Oude Engberink, Agnès AU - Clary, Bernard AU - Amouyal, Michel AU - Lognos, Béatrice AU - Demoly, Pascal AU - Annesi-Maesano, Isabella AU - Ninot, Grégory AU - Molinari, Nicolas AU - Richard, Arnaud AU - Badreddine, Maha AU - Duflos, Claire AU - Carbonnel, Francois PY - 2024/5/9 TI - The Primary Care and Environmental Health e-Learning Course to Integrate Environmental Health in General Practice: Before-and-After Feasibility Study JO - JMIR Form Res SP - e56130 VL - 8 KW - environmental health KW - medical education KW - One Health KW - environment KW - environmental KW - eLearning KW - e-learning KW - remote KW - learning KW - online learning KW - primary care KW - satisfaction KW - awareness KW - behavioral KW - behavior change KW - questionnaire KW - survey KW - course KW - educational KW - teaching KW - GP KW - general practice KW - general practitioner N2 - Background: Environmental and behavioral factors are responsible for 12.6 million deaths annually and contribute to 25% of deaths and chronic diseases worldwide. Through the One Health initiative, the World Health Organization and other international health organizations plan to improve these indicators to create healthier environments by 2030. To meet this challenge, training primary care professionals should be the priority of national policies. General practitioners (GPs) are ready to become involved but need in-depth training to gain and apply environmental health (EH) knowledge to their practice. In response, we designed the Primary Care Environment and Health (PCEH) online course in partnership with the Occitanie Regional Health Agency in France. This course was used to train GP residents from the Montpelier-Nimes Faculty of Medicine in EH knowledge. The course was organized in 2 successive parts: (1) an asynchronous e-learning modular course focusing on EH knowledge and tools and (2) 1 day of face-to-face sessions. Objective: This study assessed the impact of the e-learning component of the PCEH course on participants? satisfaction, knowledge, and behavior changes toward EH. Methods: This was a pilot before-and-after study. Four modules were available in the 6-hour e-learning course: introduction to EH, population-based approach (mapping tools and resources), clinical cases, and communication tools. From August to September 2021, we recruited first-year GP residents from the University of Montpellier (N=130). Participants? satisfaction, knowledge improvements for 19 EH risks, procedure to report EH risks to health authorities online, and behavior change (to consider the possible effects of the environment on their own and their patients? health) were assessed using self-reported questionnaires on a Likert scale (1-5). Paired Student t tests and the McNemar ?2 test were used to compare quantitative and qualitative variables, respectively, before and after the course. Results: A total of 74 GP residents completed the e-learning and answered the pre- and posttest questionnaires. The mean satisfaction score was 4.0 (SD 0.9) out of 5. Knowledge scores of EH risks increased significantly after the e-learning course, with a mean difference of 30% (P<.001) for all items. Behavioral scores improved significantly by 18% for the participant?s health and by 26% for patients? health (P<.001). These improvements did not vary significantly according to participant characteristics (eg, sex, children, place of work). Conclusions: The e-learning course improved knowledge and behavior related to EH. Further studies are needed to assess the impact of the PCEH course on clinical practice and potential benefits for patients. This course was designed to serve as a knowledge base that could be reused each year with a view toward sustainability. This course will integrate new modules and will be adapted to the evolution of EH status indicators and target population needs. UR - https://formative.jmir.org/2024/1/e56130 UR - http://dx.doi.org/10.2196/56130 UR - http://www.ncbi.nlm.nih.gov/pubmed/38722679 ID - info:doi/10.2196/56130 ER - TY - JOUR AU - Skryd, Anthony AU - Lawrence, Katharine PY - 2024/5/8 TI - ChatGPT as a Tool for Medical Education and Clinical Decision-Making on the Wards: Case Study JO - JMIR Form Res SP - e51346 VL - 8 KW - ChatGPT KW - medical education KW - large language models KW - LLMs KW - clinical decision-making N2 - Background: Large language models (LLMs) are computational artificial intelligence systems with advanced natural language processing capabilities that have recently been popularized among health care students and educators due to their ability to provide real-time access to a vast amount of medical knowledge. The adoption of LLM technology into medical education and training has varied, and little empirical evidence exists to support its use in clinical teaching environments. Objective: The aim of the study is to identify and qualitatively evaluate potential use cases and limitations of LLM technology for real-time ward-based educational contexts. Methods: A brief, single-site exploratory evaluation of the publicly available ChatGPT-3.5 (OpenAI) was conducted by implementing the tool into the daily attending rounds of a general internal medicine inpatient service at a large urban academic medical center. ChatGPT was integrated into rounds via both structured and organic use, using the web-based ?chatbot? style interface to interact with the LLM through conversational free-text and discrete queries. A qualitative approach using phenomenological inquiry was used to identify key insights related to the use of ChatGPT through analysis of ChatGPT conversation logs and associated shorthand notes from the clinical sessions. Results: Identified use cases for ChatGPT integration included addressing medical knowledge gaps through discrete medical knowledge inquiries, building differential diagnoses and engaging dual-process thinking, challenging medical axioms, using cognitive aids to support acute care decision-making, and improving complex care management by facilitating conversations with subspecialties. Potential additional uses included engaging in difficult conversations with patients, exploring ethical challenges and general medical ethics teaching, personal continuing medical education resources, developing ward-based teaching tools, supporting and automating clinical documentation, and supporting productivity and task management. LLM biases, misinformation, ethics, and health equity were identified as areas of concern and potential limitations to clinical and training use. A code of conduct on ethical and appropriate use was also developed to guide team usage on the wards. Conclusions: Overall, ChatGPT offers a novel tool to enhance ward-based learning through rapid information querying, second-order content exploration, and engaged team discussion regarding generated responses. More research is needed to fully understand contexts for educational use, particularly regarding the risks and limitations of the tool in clinical settings and its impacts on trainee development. UR - https://formative.jmir.org/2024/1/e51346 UR - http://dx.doi.org/10.2196/51346 UR - http://www.ncbi.nlm.nih.gov/pubmed/38717811 ID - info:doi/10.2196/51346 ER - TY - JOUR AU - El Emam, Khaled AU - Leung, I. Tiffany AU - Malin, Bradley AU - Klement, William AU - Eysenbach, Gunther PY - 2024/5/2 TI - Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Models (CREMLS) JO - J Med Internet Res SP - e52508 VL - 26 KW - reporting guidelines KW - machine learning KW - predictive models KW - diagnostic models KW - prognostic models KW - artificial intelligence KW - editorial policy UR - https://www.jmir.org/2024/1/e52508 UR - http://dx.doi.org/10.2196/52508 UR - http://www.ncbi.nlm.nih.gov/pubmed/38696776 ID - info:doi/10.2196/52508 ER - TY - JOUR AU - Green, Shaw Sara AU - Lee, Sung-Jae AU - Chahin, Samantha AU - Pooler-Burgess, Meardith AU - Green-Jones, Monique AU - Gurung, Sitaji AU - Outlaw, Y. Angulique AU - Naar, Sylvie PY - 2024/5/2 TI - Regulatory Issues in Electronic Health Records for Adolescent HIV Research: Strategies and Lessons Learned JO - JMIR Form Res SP - e46420 VL - 8 KW - electronic health record KW - HIV KW - pragmatic trial KW - regulatory KW - EHR KW - pre-exposure prophylaxis KW - retention KW - attrition KW - dropout KW - legal KW - regulation KW - adherence KW - ethic KW - review board KW - implementation KW - data use KW - privacy N2 - Background: Electronic health records (EHRs) are a cost-effective approach to provide the necessary foundations for clinical trial research. The ability to use EHRs in real-world clinical settings allows for pragmatic approaches to intervention studies with the emerging adult HIV population within these settings; however, the regulatory components related to the use of EHR data in multisite clinical trials poses unique challenges that researchers may find themselves unprepared to address, which may result in delays in study implementation and adversely impact study timelines, and risk noncompliance with established guidance. Objective: As part of the larger Adolescent Trials Network (ATN) for HIV/AIDS Interventions Protocol 162b (ATN 162b) study that evaluated clinical-level outcomes of an intervention including HIV treatment and pre-exposure prophylaxis services to improve retention within the emerging adult HIV population, the objective of this study is to highlight the regulatory process and challenges in the implementation of a multisite pragmatic trial using EHRs to assist future researchers conducting similar studies in navigating the often time-consuming regulatory process and ensure compliance with adherence to study timelines and compliance with institutional and sponsor guidelines. Methods: Eight sites were engaged in research activities, with 4 sites selected from participant recruitment venues as part of the ATN, who participated in the intervention and data extraction activities, and an additional 4 sites were engaged in data management and analysis. The ATN 162b protocol team worked with site personnel to establish the necessary regulatory infrastructure to collect EHR data to evaluate retention in care and viral suppression, as well as para-data on the intervention component to assess the feasibility and acceptability of the mobile health intervention. Methods to develop this infrastructure included site-specific training activities and the development of both institutional reliance and data use agreements. Results: Due to variations in site-specific activities, and the associated regulatory implications, the study team used a phased approach with the data extraction sites as phase 1 and intervention sites as phase 2. This phased approach was intended to address the unique regulatory needs of all participating sites to ensure that all sites were properly onboarded and all regulatory components were in place. Across all sites, the regulatory process spanned 6 months for the 4 data extraction and intervention sites, and up to 10 months for the data management and analysis sites. Conclusions: The process for engaging in multisite clinical trial studies using EHR data is a multistep, collaborative effort that requires proper advanced planning from the proposal stage to adequately implement the necessary training and infrastructure. Planning, training, and understanding the various regulatory aspects, including the necessity of data use agreements, reliance agreements, external institutional review board review, and engagement with clinical sites, are foremost considerations to ensure successful implementation and adherence to pragmatic trial timelines and outcomes. UR - https://formative.jmir.org/2024/1/e46420 UR - http://dx.doi.org/10.2196/46420 UR - http://www.ncbi.nlm.nih.gov/pubmed/38696775 ID - info:doi/10.2196/46420 ER - TY - JOUR AU - Kluge, Felix AU - Brand, E. Yonatan AU - Micó-Amigo, Encarna M. AU - Bertuletti, Stefano AU - D'Ascanio, Ilaria AU - Gazit, Eran AU - Bonci, Tecla AU - Kirk, Cameron AU - Küderle, Arne AU - Palmerini, Luca AU - Paraschiv-Ionescu, Anisoara AU - Salis, Francesca AU - Soltani, Abolfazl AU - Ullrich, Martin AU - Alcock, Lisa AU - Aminian, Kamiar AU - Becker, Clemens AU - Brown, Philip AU - Buekers, Joren AU - Carsin, Anne-Elie AU - Caruso, Marco AU - Caulfield, Brian AU - Cereatti, Andrea AU - Chiari, Lorenzo AU - Echevarria, Carlos AU - Eskofier, Bjoern AU - Evers, Jordi AU - Garcia-Aymerich, Judith AU - Hache, Tilo AU - Hansen, Clint AU - Hausdorff, M. Jeffrey AU - Hiden, Hugo AU - Hume, Emily AU - Keogh, Alison AU - Koch, Sarah AU - Maetzler, Walter AU - Megaritis, Dimitrios AU - Niessen, Martijn AU - Perlman, Or AU - Schwickert, Lars AU - Scott, Kirsty AU - Sharrack, Basil AU - Singleton, David AU - Vereijken, Beatrix AU - Vogiatzis, Ioannis AU - Yarnall, Alison AU - Rochester, Lynn AU - Mazzà, Claudia AU - Del Din, Silvia AU - Mueller, Arne PY - 2024/5/1 TI - Real-World Gait Detection Using a Wrist-Worn Inertial Sensor: Validation Study JO - JMIR Form Res SP - e50035 VL - 8 KW - digital mobility outcomes KW - validation KW - wearable sensor KW - walking KW - digital health KW - inertial measurement unit KW - accelerometer KW - Mobilise-D N2 - Background: Wrist-worn inertial sensors are used in digital health for evaluating mobility in real-world environments. Preceding the estimation of spatiotemporal gait parameters within long-term recordings, gait detection is an important step to identify regions of interest where gait occurs, which requires robust algorithms due to the complexity of arm movements. While algorithms exist for other sensor positions, a comparative validation of algorithms applied to the wrist position on real-world data sets across different disease populations is missing. Furthermore, gait detection performance differences between the wrist and lower back position have not yet been explored but could yield valuable information regarding sensor position choice in clinical studies. Objective: The aim of this study was to validate gait sequence (GS) detection algorithms developed for the wrist position against reference data acquired in a real-world context. In addition, this study aimed to compare the performance of algorithms applied to the wrist position to those applied to lower back?worn inertial sensors. Methods: Participants with Parkinson disease, multiple sclerosis, proximal femoral fracture (hip fracture recovery), chronic obstructive pulmonary disease, and congestive heart failure and healthy older adults (N=83) were monitored for 2.5 hours in the real-world using inertial sensors on the wrist, lower back, and feet including pressure insoles and infrared distance sensors as reference. In total, 10 algorithms for wrist-based gait detection were validated against a multisensor reference system and compared to gait detection performance using lower back?worn inertial sensors. Results: The best-performing GS detection algorithm for the wrist showed a mean (per disease group) sensitivity ranging between 0.55 (SD 0.29) and 0.81 (SD 0.09) and a mean (per disease group) specificity ranging between 0.95 (SD 0.06) and 0.98 (SD 0.02). The mean relative absolute error of estimated walking time ranged between 8.9% (SD 7.1%) and 32.7% (SD 19.2%) per disease group for this algorithm as compared to the reference system. Gait detection performance from the best algorithm applied to the wrist inertial sensors was lower than for the best algorithms applied to the lower back, which yielded mean sensitivity between 0.71 (SD 0.12) and 0.91 (SD 0.04), mean specificity between 0.96 (SD 0.03) and 0.99 (SD 0.01), and a mean relative absolute error of estimated walking time between 6.3% (SD 5.4%) and 23.5% (SD 13%). Performance was lower in disease groups with major gait impairments (eg, patients recovering from hip fracture) and for patients using bilateral walking aids. Conclusions: Algorithms applied to the wrist position can detect GSs with high performance in real-world environments. Those periods of interest in real-world recordings can facilitate gait parameter extraction and allow the quantification of gait duration distribution in everyday life. Our findings allow taking informed decisions on alternative positions for gait recording in clinical studies and public health. Trial Registration: ISRCTN Registry 12246987; https://www.isrctn.com/ISRCTN12246987 International Registered Report Identifier (IRRID): RR2-10.1136/bmjopen-2021-050785 UR - https://formative.jmir.org/2024/1/e50035 UR - http://dx.doi.org/10.2196/50035 UR - http://www.ncbi.nlm.nih.gov/pubmed/38691395 ID - info:doi/10.2196/50035 ER - TY - JOUR AU - Olsen, Christine AU - Lungu, Adrian Daniel PY - 2024/4/30 TI - Effectiveness of a Smartphone App (Heia Meg) in Improving Decisions About Nutrition and Physical Activity: Prospective Longitudinal Study JO - JMIR Form Res SP - e48185 VL - 8 KW - app KW - BMI KW - diet KW - exercise KW - health KW - Heia Meg KW - lifestyle change KW - longitudinal KW - mHealth KW - mobile health KW - motivation KW - nutrition KW - obese KW - obesity KW - overweight KW - physical activity KW - smartphone apps KW - weight N2 - Background: Obesity is a prevalent and serious chronic condition associated with abnormal or excessive fat buildup that poses significant health risks. The rates of overweight and obesity in adults and children continue to rise, with global rates of children with overweight or obesity aged 5-19 years growing from 4% to 18% between 1975 and 2016. Furthermore, in 2017, nearly 4 million people died due to complications arising from being overweight or obese. Objective: This study aims to investigate the potential impact of the mobile app Heia Meg on promoting healthier lifestyle choices regarding nutrition and physical activity. Methods: A prospective longitudinal study was conducted in collaboration with the Norwegian Directorate of Health. Participants were recruited through the Heia Meg app and were asked to complete a questionnaire before and after using the app. A total of 199 responses were included in the first (preintervention) questionnaire, while 99 valid responses were obtained in the second (postintervention) questionnaire. Results: The majority (159/199, 79.9%) of participants were female, and their age ranged from 18 years to 70 years and older. The results show a reduction in BMI after the digital intervention. However, some variables influence the BMI reduction effect: sex, age, education, and smoking. The group that obtained the most benefit from the intervention consisted of those who were male, aged 30-39 years, highly educated, and nonsmokers. Although positive, some of the findings are slightly above the statistical significance threshold and therefore should be interpreted carefully. Conclusions: Our study found weak evidence to support the effectiveness of the Heia Meg app in promoting healthier lifestyle choices. However, limitations and confounding factors suggest that further research in different populations with larger sample sizes is needed to confirm or disprove our findings. UR - https://formative.jmir.org/2024/1/e48185 UR - http://dx.doi.org/10.2196/48185 UR - http://www.ncbi.nlm.nih.gov/pubmed/38687565 ID - info:doi/10.2196/48185 ER - TY - JOUR AU - Anikamadu, Onyekachukwu AU - Ezechi, Oliver AU - Engelhart, Alexis AU - Nwaozuru, Ucheoma AU - Obiezu-Umeh, Chisom AU - Ogunjemite, Ponmile AU - Bale, Ismail Babatunde AU - Nwachukwu, Daniel AU - Gbaja-biamila, Titilola AU - Oladele, David AU - Musa, Z. Adesola AU - Mason, Stacey AU - Ojo, Temitope AU - Tucker, Joseph AU - Iwelunmor, Juliet PY - 2024/4/30 TI - Expanding Youth-Friendly HIV Self-Testing Services During the COVID-19 Pandemic: Qualitative Analysis of a Crowdsourcing Open Call in Nigeria JO - JMIR Form Res SP - e46945 VL - 8 KW - crowdsourcing KW - World AIDS Day KW - HIV KW - self-testing KW - young people KW - COVID-19 pandemic restrictions KW - Nigeria KW - HIV self-testing KW - health promotion KW - crowdsourcing open call KW - young adult N2 - Background: HIV self-testing (HIVST) among young people is an effective approach to enhance the uptake of HIV testing recommended by the World Health Organization. However, the COVID-19 pandemic disrupted conventional facility-based HIV testing services, necessitating the exploration of innovative strategies for the effective delivery of HIVST. Objective: This study analyzed the outcomes of a digital World AIDS Day crowdsourcing open call, designed to elicit youth responses on innovative approaches to promote HIVST among young people (14-24 years) in Nigeria during COVID-19 restrictions. Methods: From November 2 to 22, 2020, a World AIDS Day 2020 crowdsourcing open call was held digitally due to COVID-19 restrictions. The crowdsourcing open call followed World Health Organization standardized steps, providing a structured framework for participant engagement. Young people in Nigeria, aged 10-24 years, participated by submitting ideas digitally through Google Forms or email in response to this crowdsourcing open call prompt: ?How will you promote HIV self-testing among young people during COVID-19 pandemic?? Data and responses from each submission were analyzed, and proposed ideas were closely examined to identify common themes. Four independent reviewers (AE, SM, AZM, and TG) judged each submission based on the desirability, feasibility, and impact on a 9-point scale (3-9, with 3 being the lowest and 9 being the highest). Results: The crowdsourcing open call received 125 eligible entries, 44 from women and 65 from men. The median age of participants was 20 (IQR 24-20) years, with the majority having completed their highest level of education at the senior secondary school level. The majority of participants lived in the South-West region (n=61) and Lagos state (n=36). Of the 125 eligible entries, the top 20 submissions received an average total score of 7.5 (SD 2.73) or above. The panel of judges ultimately selected 3 finalists to receive a monetary award. Three prominent themes were identified from the 125 crowdsourcing open call submissions as specific ways that HIVST can adapt during the COVID-19 pandemic: (1) digital approaches (such as gamification, photoverification system, and digital media) to generate demand for HIVST and avoid risks associated with attending clinics, (2) awareness and sensitization through existing infrastructures (such as churches, schools, and health facilities), and (3) partnerships with influencers, role models, and leaders (such as religious and youth leaders and social influencers in businesses, churches, organizations, and schools) to build trust in HIVST services. Conclusions: The crowdsourcing open call effectively engaged a diverse number of young people who proposed a variety of ways to improve the uptake of HIVST during the COVID-19 pandemic. Findings contribute to the need for innovative HIVST strategies that close critical knowledge and practice gaps on ways to reach young people with HIVST during and beyond the pandemic. Trial Registration: ClinicalTrials.gov NCT04710784; https://clinicaltrials.gov/study/NCT04710784 UR - https://formative.jmir.org/2024/1/e46945 UR - http://dx.doi.org/10.2196/46945 UR - http://www.ncbi.nlm.nih.gov/pubmed/38687582 ID - info:doi/10.2196/46945 ER - TY - JOUR AU - Vidal Bustamante, M. Constanza AU - Coombs III, Garth AU - Rahimi-Eichi, Habiballah AU - Mair, Patrick AU - Onnela, Jukka-Pekka AU - Baker, T. Justin AU - Buckner, L. Randy PY - 2024/4/30 TI - Precision Assessment of Real-World Associations Between Stress and Sleep Duration Using Actigraphy Data Collected Continuously for an Academic Year: Individual-Level Modeling Study JO - JMIR Form Res SP - e53441 VL - 8 KW - deep phenotyping KW - individualized models KW - intensive longitudinal data KW - sleep KW - stress KW - actigraphy KW - accelerometer KW - wearable KW - mobile phone KW - digital health N2 - Background: Heightened stress and insufficient sleep are common in the transition to college, often co-occur, and have both been linked to negative health outcomes. A challenge concerns disentangling whether perceived stress precedes or succeeds changes in sleep. These day-to-day associations may vary across individuals, but short study periods and group-level analyses in prior research may have obscured person-specific phenotypes. Objective: This study aims to obtain stable estimates of lead-lag associations between perceived stress and objective sleep duration in the individual, unbiased by the group, by developing an individual-level linear model that can leverage intensive longitudinal data while remaining parsimonious. Methods: In total, 55 college students (n=6, 11% second-year students and n=49, 89% first-year students) volunteered to provide daily self-reports of perceived stress via a smartphone app and wore an actigraphy wristband for the estimation of daily sleep duration continuously throughout the academic year (median usable daily observations per participant: 178, IQR 65.5). The individual-level linear model, developed in a Bayesian framework, included the predictor and outcome of interest and a covariate for the day of the week to account for weekly patterns. We validated the model on the cohort of second-year students (n=6, used as a pilot sample) by applying it to variables expected to correlate positively within individuals: objective sleep duration and self-reported sleep quality. The model was then applied to the fully independent target sample of first-year students (n=49) for the examination of bidirectional associations between daily stress levels and sleep duration. Results: Proof-of-concept analyses captured expected associations between objective sleep duration and subjective sleep quality in every pilot participant. Target analyses revealed negative associations between sleep duration and perceived stress in most of the participants (45/49, 92%), but their temporal association varied. Of the 49 participants, 19 (39%) showed a significant association (probability of direction>0.975): 8 (16%) showed elevated stress in the day associated with shorter sleep later that night, 5 (10%) showed shorter sleep associated with elevated stress the next day, and 6 (12%) showed both directions of association. Of note, when analyzed using a group-based multilevel model, individual estimates were systematically attenuated, and some even reversed sign. Conclusions: The dynamic interplay of stress and sleep in daily life is likely person specific. Paired with intensive longitudinal data, our individual-level linear model provides a precision framework for the estimation of stable real-world behavioral and psychological dynamics and may support the personalized prioritization of intervention targets for health and well-being. UR - https://formative.jmir.org/2024/1/e53441 UR - http://dx.doi.org/10.2196/53441 UR - http://www.ncbi.nlm.nih.gov/pubmed/38687600 ID - info:doi/10.2196/53441 ER - TY - JOUR AU - González-Castro, Ana AU - Leirós-Rodríguez, Raquel AU - Prada-García, Camino AU - Benítez-Andrades, Alberto José PY - 2024/4/29 TI - The Applications of Artificial Intelligence for Assessing Fall Risk: Systematic Review JO - J Med Internet Res SP - e54934 VL - 26 KW - machine learning KW - accidental falls KW - public health KW - patient care KW - artificial intelligence KW - AI KW - fall risk N2 - Background: Falls and their consequences are a serious public health problem worldwide. Each year, 37.3 million falls requiring medical attention occur. Therefore, the analysis of fall risk is of great importance for prevention. Artificial intelligence (AI) represents an innovative tool for creating predictive statistical models of fall risk through data analysis. Objective: The aim of this review was to analyze the available evidence on the applications of AI in the analysis of data related to postural control and fall risk. Methods: A literature search was conducted in 6 databases with the following inclusion criteria: the articles had to be published within the last 5 years (from 2018 to 2024), they had to apply some method of AI, AI analyses had to be applied to data from samples consisting of humans, and the analyzed sample had to consist of individuals with independent walking with or without the assistance of external orthopedic devices. Results: We obtained a total of 3858 articles, of which 22 were finally selected. Data extraction for subsequent analysis varied in the different studies: 82% (18/22) of them extracted data through tests or functional assessments, and the remaining 18% (4/22) of them extracted through existing medical records. Different AI techniques were used throughout the articles. All the research included in the review obtained accuracy values of >70% in the predictive models obtained through AI. Conclusions: The use of AI proves to be a valuable tool for creating predictive models of fall risk. The use of this tool could have a significant socioeconomic impact as it enables the development of low-cost predictive models with a high level of accuracy. Trial Registration: PROSPERO CRD42023443277; https://tinyurl.com/4sb72ssv UR - https://www.jmir.org/2024/1/e54934 UR - http://dx.doi.org/10.2196/54934 UR - http://www.ncbi.nlm.nih.gov/pubmed/38684088 ID - info:doi/10.2196/54934 ER - TY - JOUR AU - Khalilnejad, Arash AU - Sun, Ruo-Ting AU - Kompala, Tejaswi AU - Painter, Stefanie AU - James, Roberta AU - Wang, Yajuan PY - 2024/4/26 TI - Proactive Identification of Patients with Diabetes at Risk of Uncontrolled Outcomes during a Diabetes Management Program: Conceptualization and Development Study Using Machine Learning JO - JMIR Form Res SP - e54373 VL - 8 KW - diabetes KW - diabetic KW - DM KW - diabetes mellitus KW - type 2 diabetes KW - type 1 diabetes KW - self-monitoring KW - predictive model KW - predictive models KW - predictive analytics KW - predictive system KW - practical model KW - practical models KW - ML KW - machine learning KW - AI KW - artificial intelligence KW - algorithm KW - algorithms KW - behavior KW - behaviour KW - telehealth KW - tele-health KW - chronic condition KW - chronic conditions KW - chronic disease KW - chronic diseases KW - chronic illness KW - chronic illnesses N2 - Background: The growth in the capabilities of telehealth have made it possible to identify individuals with a higher risk of uncontrolled diabetes and provide them with targeted support and resources to help them manage their condition. Thus, predictive modeling has emerged as a valuable tool for the advancement of diabetes management. Objective: This study aimed to conceptualize and develop a novel machine learning (ML) approach to proactively identify participants enrolled in a remote diabetes monitoring program (RDMP) who were at risk of uncontrolled diabetes at 12 months in the program. Methods: Registry data from the Livongo for Diabetes RDMP were used to design separate dynamic predictive ML models to predict participant outcomes at each monthly checkpoint of the participants? program journey (month-n models) from the first day of onboarding (month-0 model) up to the 11th month (month-11 model). A participant?s program journey began upon onboarding into the RDMP and monitoring their own blood glucose (BG) levels through the RDMP-provided BG meter. Each participant passed through 12 predicative models through their first year enrolled in the RDMP. Four categories of participant attributes (ie, survey data, BG data, medication fills, and health signals) were used for feature construction. The models were trained using the light gradient boosting machine and underwent hyperparameter tuning. The performance of the models was evaluated using standard metrics, including precision, recall, specificity, the area under the curve, the F1-score, and accuracy. Results: The ML models exhibited strong performance, accurately identifying observable at-risk participants, with recall ranging from 70% to 94% and precision from 40% to 88% across the 12-month program journey. Unobservable at-risk participants also showed promising performance, with recall ranging from 61% to 82% and precision from 42% to 61%. Overall, model performance improved as participants progressed through their program journey, demonstrating the importance of engagement data in predicting long-term clinical outcomes. Conclusions: This study explored the Livongo for Diabetes RDMP participants? temporal and static attributes, identification of diabetes management patterns and characteristics, and their relationship to predict diabetes management outcomes. Proactive targeting ML models accurately identified participants at risk of uncontrolled diabetes with a high level of precision that was generalizable through future years within the RDMP. The ability to identify participants who are at risk at various time points throughout the program journey allows for personalized interventions to improve outcomes. This approach offers significant advancements in the feasibility of large-scale implementation in remote monitoring programs and can help prevent uncontrolled glycemic levels and diabetes-related complications. Future research should include the impact of significant changes that can affect a participant?s diabetes management. UR - https://formative.jmir.org/2024/1/e54373 UR - http://dx.doi.org/10.2196/54373 UR - http://www.ncbi.nlm.nih.gov/pubmed/38669074 ID - info:doi/10.2196/54373 ER - TY - JOUR AU - Marsall, Matthias AU - Dinse, Hannah AU - Schröder, Julia AU - Skoda, Eva-Maria AU - Teufel, Martin AU - Bäuerle, Alexander PY - 2024/4/25 TI - Assessing Electronic Health Literacy in Individuals With the Post?COVID-19 Condition Using the German Revised eHealth Literacy Scale: Validation Study JO - JMIR Form Res SP - e52189 VL - 8 KW - eHealth literacy KW - eHEALS KW - factor analysis KW - measurement invariance KW - psychometric properties KW - infodemic N2 - Background: The eHealth Literacy Scale (eHEALS) is a widely used instrument for measuring eHealth literacy (eHL). However, little is known so far about whether the instrument is valid for the assessment of eHL in persons who are affected by the post?COVID-19 condition. This is particularly important as people with the post?COVID-19 condition are frequently affected by false information from the internet. Objective: The objective of our study was to evaluate the validity and reliability of the German Revised eHealth Literacy Scale (GR-eHEALS) in individuals with the post?COVID-19 condition. Methods: A cross-sectional study was conducted from January to May 2022. The self-assessment survey consisted of the GR-eHEALS, health status? and internet use?related variables, sociodemographic data, and (post)?COVID-19?related medical data. Confirmatory factor analysis (CFA), correlational analyses, and tests of measurement invariance were deployed. Results: In total, 330 participants were included in the statistical analyses. CFA revealed that the 2-factor model reached an excellent model fit (comparative fit index=1.00, Tucker?Lewis index=0.99, root mean square error of approximation=0.036, standardized root mean square residual=0.038). Convergent validity was confirmed by significant positive correlations between eHL and knowledge of internet-based health promotion programs, experience in using these programs, and the duration of private internet use. In addition, a significantly negative relationship of eHL with internet anxiety supported convergent validity. Further, significant relationships of eHL with mental health status and internal health locus of control confirmed the criterion validity of the instrument. However, relationships of eHL with physical health status and quality of life could not be confirmed. The 2-factor model was fully measurement invariant regarding gender. Regarding age and educational level, partial measurement invariance was confirmed. The subscales as well as the overall GR-eHEALS reached good-to-excellent reliability (Cronbach ??.86). Conclusions: The GR-eHEALS is a reliable and largely valid instrument for assessing eHL in individuals with the post?COVID-19 condition. Measurement invariance regarding gender was fully confirmed and allows the interpretation of group differences. Regarding age and educational level, group differences should be interpreted with caution. Given the high likelihood that individuals with the post?COVID-19 condition will be confronted with misinformation on the Internet, eHL is a core competency that is highly relevant in this context, in both research and clinical practice. Therefore, future research should also explore alternative instruments to capture eHL to overcome shortcomings in the validity of the GR-eHEALS. UR - https://formative.jmir.org/2024/1/e52189 UR - http://dx.doi.org/10.2196/52189 UR - http://www.ncbi.nlm.nih.gov/pubmed/38662429 ID - info:doi/10.2196/52189 ER - TY - JOUR AU - Ahmed, Sabbir Md AU - Hasan, Tanvir AU - Islam, Salekul AU - Ahmed, Nova PY - 2024/4/24 TI - Investigating Rhythmicity in App Usage to Predict Depressive Symptoms: Protocol for Personalized Framework Development and Validation Through a Countrywide Study JO - JMIR Res Protoc SP - e51540 VL - 13 KW - depressive symptoms KW - app usage rhythm KW - behavioral markers KW - personalization KW - multitask learning framework N2 - Background: Understanding a student?s depressive symptoms could facilitate significantly more precise diagnosis and treatment. However, few studies have focused on depressive symptom prediction through unobtrusive systems, and these studies are limited by small sample sizes, low performance, and the requirement for higher resources. In addition, research has not explored whether statistically significant rhythms based on different app usage behavioral markers (eg, app usage sessions) exist that could be useful in finding subtle differences to predict with higher accuracy like the models based on rhythms of physiological data. Objective: The main objective of this study is to explore whether there exist statistically significant rhythms in resource-insensitive app usage behavioral markers and predict depressive symptoms through these marker-based rhythmic features. Another objective of this study is to understand whether there is a potential link between rhythmic features and depressive symptoms. Methods: Through a countrywide study, we collected 2952 students? raw app usage behavioral data and responses to the 9 depressive symptoms in the 9-item Patient Health Questionnaire (PHQ-9). The behavioral data were retrieved through our developed app, which was previously used in our pilot studies in Bangladesh on different research problems. To explore whether there is a rhythm based on app usage data, we will conduct a zero-amplitude test. In addition, we will develop a cosinor model for each participant to extract rhythmic parameters (eg, acrophase). In addition, to obtain a comprehensive picture of the rhythms, we will explore nonparametric rhythmic features (eg, interdaily stability). Furthermore, we will conduct regression analysis to understand the association of rhythmic features with depressive symptoms. Finally, we will develop a personalized multitask learning (MTL) framework to predict symptoms through rhythmic features. Results: After applying inclusion criteria (eg, having app usage data of at least 2 days to explore rhythmicity), we kept the data of 2902 (98.31%) students for analysis, with 24.48 million app usage events, and 7 days? app usage of 2849 (98.17%) students. The students are from all 8 divisions of Bangladesh, both public and private universities (19 different universities and 52 different departments). We are analyzing the data and will publish the findings in a peer-reviewed publication. Conclusions: Having an in-depth understanding of app usage rhythms and their connection with depressive symptoms through a countrywide study can significantly help health care professionals and researchers better understand depressed students and may create possibilities for using app usage?based rhythms for intervention. In addition, the MTL framework based on app usage rhythmic features may more accurately predict depressive symptoms due to the rhythms? capability to find subtle differences. International Registered Report Identifier (IRRID): DERR1-10.2196/51540 UR - https://www.researchprotocols.org/2024/1/e51540 UR - http://dx.doi.org/10.2196/51540 UR - http://www.ncbi.nlm.nih.gov/pubmed/38657238 ID - info:doi/10.2196/51540 ER - TY - JOUR AU - Tate, D. Allan AU - Fertig, R. Angela AU - de Brito, N. Junia AU - Ellis, M. Émilie AU - Carr, Patrick Christopher AU - Trofholz, Amanda AU - Berge, M. Jerica PY - 2024/4/24 TI - Momentary Factors and Study Characteristics Associated With Participant Burden and Protocol Adherence: Ecological Momentary Assessment JO - JMIR Form Res SP - e49512 VL - 8 KW - adherence KW - burden KW - data quality KW - ecological momentary assessment KW - mental health KW - mHealth KW - mobile health KW - participant adherence KW - public health KW - stress KW - study design KW - survey burden KW - survey N2 - Background: Ecological momentary assessment (EMA) has become a popular mobile health study design to understand the lived experiences of dynamic environments. The numerous study design choices available to EMA researchers, however, may quickly increase participant burden and could affect overall adherence, which could limit the usability of the collected data. Objective: This study quantifies what study design, participant attributes, and momentary factors may affect self-reported burden and adherence. Methods: The EMA from the Phase 1 Family Matters Study (n=150 adult Black, Hmong, Latino or Latina, Native American, Somali, and White caregivers; n=1392 observation days) was examined to understand how participant self-reported survey burden was related to both design and momentary antecedents of adherence. The daily burden was measured by the question ?Overall, how difficult was it for you to fill out the surveys today?? on a 5-item Likert scale (0=not at all and 4=extremely). Daily protocol adherence was defined as completing at least 2 signal-contingent surveys, 1 event-contingent survey, and 1 end-of-day survey each. Stress and mood were measured earlier in the day, sociodemographic and psychosocial characteristics were reported using a comprehensive cross-sectional survey, and EMA timestamps for weekends and weekdays were used to parameterize time-series models to evaluate prospective correlates of end-of-day study burden. Results: The burden was low at 1.2 (SD 1.14) indicating ?a little? burden on average. Participants with elevated previous 30-day chronic stress levels (mean burden difference: 0.8; P=.04), 1 in 5 more immigrant households (P=.02), and the language primarily spoken in the home (P=.04; 3 in 20 more non-English?speaking households) were found to be population attributes of elevated moderate-high burden. Current and 1-day lagged nonadherence were correlated with elevated 0.39 and 0.36 burdens, respectively (P=.001), and the association decayed by the second day (?=0.08; P=.47). Unit increases in momentary antecedents, including daily depressed mood (P=.002) and across-day change in stress (P=.008), were positively associated with 0.15 and 0.07 higher end-of-day burdens after controlling for current-day adherence. Conclusions: The 8-day EMA implementation appeared to capture momentary sources of stress and depressed mood without substantial burden to a racially or ethnically diverse and immigrant or refugee sample of parents. Attention to sociodemographic attributes (eg, EMA in the primary language of the caregiver) was important for minimizing participant burden and improving data quality. Momentary stress and depressed mood were strong determinants of participant-experienced EMA burden and may affect adherence to mobile health study protocols. There were no strong indicators of EMA design attributes that created a persistent burden for caregivers. EMA stands to be an important observational design to address dynamic public health challenges related to human-environment interactions when the design is carefully tailored to the study population and to study research objectives. UR - https://formative.jmir.org/2024/1/e49512 UR - http://dx.doi.org/10.2196/49512 UR - http://www.ncbi.nlm.nih.gov/pubmed/38656787 ID - info:doi/10.2196/49512 ER - TY - JOUR AU - Aggarwal, Monica AU - Hutchison, G. Brian AU - Kokorelias, M. Kristina AU - Ramsden, R. Vivian AU - Ivers, M. Noah AU - Pinto, Andrew AU - Uphsur, G. Ross E. AU - Wong, T. Sabrina AU - Pimlott, Nick AU - Slade, Steve PY - 2024/4/23 TI - The Conceptualization and Measurement of Research Impact in Primary Health Care: Protocol for a Rapid Scoping Review JO - JMIR Res Protoc SP - e55860 VL - 13 KW - research impact KW - primary health care KW - measurement KW - definition KW - concept KW - development KW - implementation KW - health policy KW - policy KW - health service KW - rapid review KW - review KW - research KW - policies KW - societal KW - productivity KW - literature database N2 - Background: The generation of research evidence and knowledge in primary health care (PHC) is crucial for informing the development and implementation of interventions and innovations and driving health policy, health service improvements, and potential societal changes. PHC research has broad effects on patients, practices, services, population health, community, and policy formulation. The in-depth exploration of the definition and measures of research impact within PHC is essential for broadening our understanding of research impact in the discipline and how it compares to other health services research. Objective: The objectives of the study are (1) to understand the conceptualizations and measures of research impact within the realm of PHC and (2) to identify methodological frameworks for evaluation and research impact and the benefits and challenges of using these approaches. The forthcoming review seeks to guide future research endeavors and enhance methodologies used in assessing research impact within PHC. Methods: The protocol outlines the rapid review and environmental scan approach that will be used to explore research impact in PHC and will be guided by established frameworks such as the Canadian Academy of Health Sciences Impact Framework and the Canadian Health Services and Policy Research Alliance. The rapid review follows scoping review guidelines (PRISMA-ScR; Preferred Reporting Items for Systematic Review and Meta-Analysis Extension for Scoping Reviews). The environmental scan will be done by consulting with professional organizations, academic institutions, information science, and PHC experts. The search strategy will involve multiple databases, citation and forward citation searching, and manual searches of gray literature databases, think tank websites, and relevant catalogs. We will include gray and scientific literature focusing explicitly on research impact in PHC from high-income countries using the World Bank classification. Publications published in English from 1978 will be considered. The collected papers will undergo a 2-stage independent review process based on predetermined inclusion criteria. The research team will extract data from selected studies based on the research questions and the CRISP (Consensus Reporting Items for Studies in Primary Care) protocol statement. The team will discuss the extracted data, enabling the identification and categorization of key themes regarding research impact conceptualization and measurement in PHC. The narrative synthesis will evolve iteratively based on the identified literature. Results: The results of this study are expected at the end of 2024. Conclusions: The forthcoming review will explore the conceptualization and measurement of research impact in PHC. The synthesis will offer crucial insights that will guide subsequent research, emphasizing the need for a standardized approach that incorporates diverse perspectives to comprehensively gauge the true impact of PHC research. Furthermore, trends and gaps in current methodologies will set the stage for future studies aimed at enhancing our understanding and measurement of research impact in PHC. International Registered Report Identifier (IRRID): PRR1-10.2196/55860 UR - https://www.researchprotocols.org/2024/1/e55860 UR - http://dx.doi.org/10.2196/55860 UR - http://www.ncbi.nlm.nih.gov/pubmed/38652900 ID - info:doi/10.2196/55860 ER - TY - JOUR AU - Zhu, Yue AU - Zhang, Ran AU - Yin, Shuluo AU - Sun, Yihui AU - Womer, Fay AU - Liu, Rongxun AU - Zeng, Sheng AU - Zhang, Xizhe AU - Wang, Fei PY - 2024/4/22 TI - Digital Dietary Behaviors in Individuals With Depression: Real-World Behavioral Observation JO - JMIR Public Health Surveill SP - e47428 VL - 10 KW - dietary behaviors KW - digital marker KW - depression KW - mental health KW - appetite disturbance KW - behavioral monitoring KW - eating pattern KW - electronic record KW - digital health KW - behavioral KW - surveillance N2 - Background: Depression is often accompanied by changes in behavior, including dietary behaviors. The relationship between dietary behaviors and depression has been widely studied, yet previous research has relied on self-reported data which is subject to recall bias. Electronic device?based behavioral monitoring offers the potential for objective, real-time data collection of a large amount of continuous, long-term behavior data in naturalistic settings. Objective: The study aims to characterize digital dietary behaviors in depression, and to determine whether these behaviors could be used to detect depression. Methods: A total of 3310 students (2222 healthy controls [HCs], 916 with mild depression, and 172 with moderate-severe depression) were recruited for the study of their dietary behaviors via electronic records over a 1-month period, and depression severity was assessed in the middle of the month. The differences in dietary behaviors across the HCs, mild depression, and moderate-severe depression were determined by ANCOVA (analyses of covariance) with age, gender, BMI, and educational level as covariates. Multivariate logistic regression analyses were used to examine the association between dietary behaviors and depression severity. Support vector machine analysis was used to determine whether changes in dietary behaviors could detect mild and moderate-severe depression. Results: The study found that individuals with moderate-severe depression had more irregular eating patterns, more fluctuated feeding times, spent more money on dinner, less diverse food choices, as well as eating breakfast less frequently, and preferred to eat only lunch and dinner, compared with HCs. Moderate-severe depression was found to be negatively associated with the daily 3 regular meals pattern (breakfast-lunch-dinner pattern; OR 0.467, 95% CI 0.239-0.912), and mild depression was positively associated with daily lunch and dinner pattern (OR 1.460, 95% CI 1.016-2.100). These changes in digital dietary behaviors were able to detect mild and moderate-severe depression (accuracy=0.53, precision=0.60), with better accuracy for detecting moderate-severe depression (accuracy=0.67, precision=0.64). Conclusions: This is the first study to develop a profile of changes in digital dietary behaviors in individuals with depression using real-world behavioral monitoring. The results suggest that digital markers may be a promising approach for detecting depression. UR - https://publichealth.jmir.org/2024/1/e47428 UR - http://dx.doi.org/10.2196/47428 UR - http://www.ncbi.nlm.nih.gov/pubmed/38648087 ID - info:doi/10.2196/47428 ER - TY - JOUR AU - Karimian Sichani, Elnaz AU - Smith, Aaron AU - El Emam, Khaled AU - Mosquera, Lucy PY - 2024/4/22 TI - Creating High-Quality Synthetic Health Data: Framework for Model Development and Validation JO - JMIR Form Res SP - e53241 VL - 8 KW - synthetic data KW - tensor decomposition KW - data sharing KW - data utility KW - data privacy KW - electronic health record KW - longitudinal KW - model development KW - model validation KW - generative models N2 - Background: Electronic health records are a valuable source of patient information that must be properly deidentified before being shared with researchers. This process requires expertise and time. In addition, synthetic data have considerably reduced the restrictions on the use and sharing of real data, allowing researchers to access it more rapidly with far fewer privacy constraints. Therefore, there has been a growing interest in establishing a method to generate synthetic data that protects patients? privacy while properly reflecting the data. Objective: This study aims to develop and validate a model that generates valuable synthetic longitudinal health data while protecting the privacy of the patients whose data are collected. Methods: We investigated the best model for generating synthetic health data, with a focus on longitudinal observations. We developed a generative model that relies on the generalized canonical polyadic (GCP) tensor decomposition. This model also involves sampling from a latent factor matrix of GCP decomposition, which contains patient factors, using sequential decision trees, copula, and Hamiltonian Monte Carlo methods. We applied the proposed model to samples from the MIMIC-III (version 1.4) data set. Numerous analyses and experiments were conducted with different data structures and scenarios. We assessed the similarity between our synthetic data and the real data by conducting utility assessments. These assessments evaluate the structure and general patterns present in the data, such as dependency structure, descriptive statistics, and marginal distributions. Regarding privacy disclosure, our model preserves privacy by preventing the direct sharing of patient information and eliminating the one-to-one link between the observed and model tensor records. This was achieved by simulating and modeling a latent factor matrix of GCP decomposition associated with patients. Results: The findings show that our model is a promising method for generating synthetic longitudinal health data that is similar enough to real data. It can preserve the utility and privacy of the original data while also handling various data structures and scenarios. In certain experiments, all simulation methods used in the model produced the same high level of performance. Our model is also capable of addressing the challenge of sampling patients from electronic health records. This means that we can simulate a variety of patients in the synthetic data set, which may differ in number from the patients in the original data. Conclusions: We have presented a generative model for producing synthetic longitudinal health data. The model is formulated by applying the GCP tensor decomposition. We have provided 3 approaches for the synthesis and simulation of a latent factor matrix following the process of factorization. In brief, we have reduced the challenge of synthesizing massive longitudinal health data to synthesizing a nonlongitudinal and significantly smaller data set. UR - https://formative.jmir.org/2024/1/e53241 UR - http://dx.doi.org/10.2196/53241 UR - http://www.ncbi.nlm.nih.gov/pubmed/38648097 ID - info:doi/10.2196/53241 ER - TY - JOUR AU - Blanchard, Marc AU - Koller, Nadana Cinja AU - Azevedo, Ming Pedro AU - Prétat, Tiffany AU - Hügle, Thomas PY - 2024/4/19 TI - Development of a Management App for Postviral Fibromyalgia-Like Symptoms: Patient Preference-Guided Approach JO - JMIR Form Res SP - e50832 VL - 8 KW - digital health KW - patient preference KW - user experience KW - patient-centricity KW - platform KW - development KW - fibromyalgia KW - self-management KW - quality of life KW - patient outcome KW - musculoskeletal KW - usability testing KW - digital health solution N2 - Background: Persistent fibromyalgia-like symptoms have been increasingly reported following viral infections, including SARS-CoV-2. About 30% of patients with post?COVID-19 syndrome fulfill the fibromyalgia criteria. This complex condition presents significant challenges in terms of self-management. Digital health interventions offer a viable means to assist patients in managing their health conditions. However, the challenge of ensuring their widespread adoption and adherence persists. This study responds to this need by developing a patient-centered digital health management app, incorporating patient preferences to enhance usability and effectiveness, ultimately aiming to improve patient outcomes and quality of life. Objective: This research aims to develop a digital health self-management app specifically for patients experiencing postviral fibromyalgia-like symptoms. By prioritizing patient preferences and engagement through the app?s design and functionality, the study intends to facilitate better self-management practices and improve adherence. Methods: Using an exploratory study design, the research used patient preference surveys and usability testing as primary tools to inform the development process of the digital health solution. We gathered and analyzed patients? expectations regarding design features, content, and usability to steer the iterative app development. Results: The study uncovered crucial insights from patient surveys and usability testing, which influenced the app?s design and functionality. Key findings included a preference for a symptom list over an automated chatbot, a desire to report on a moderate range of symptoms and activities, and the importance of an intuitive onboarding process. While usability testing identified some challenges in the onboarding process, it also confirmed the importance of aligning the app with patient needs to enhance engagement and satisfaction. Conclusions: Incorporating patient feedback has been a significant factor in the development of the digital health app. Challenges encountered with user onboarding during usability testing have highlighted the importance of this process for user adoption. The study acknowledges the role of patient input in developing digital health technologies and suggests further research to improve onboarding procedures, aiming to enhance patient engagement and their ability to manage digital health resources effectively. International Registered Report Identifier (IRRID): RR2-10.2196/32193 UR - https://formative.jmir.org/2024/1/e50832 UR - http://dx.doi.org/10.2196/50832 UR - http://www.ncbi.nlm.nih.gov/pubmed/38639986 ID - info:doi/10.2196/50832 ER - TY - JOUR AU - Wang, Echo H. AU - Weiner, P. Jonathan AU - Saria, Suchi AU - Kharrazi, Hadi PY - 2024/4/18 TI - Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis JO - J Med Internet Res SP - e47125 VL - 26 KW - algorithmic bias KW - model bias KW - predictive models KW - model fairness KW - health disparity KW - hospital readmission KW - retrospective analysis N2 - Background: The adoption of predictive algorithms in health care comes with the potential for algorithmic bias, which could exacerbate existing disparities. Fairness metrics have been proposed to measure algorithmic bias, but their application to real-world tasks is limited. Objective: This study aims to evaluate the algorithmic bias associated with the application of common 30-day hospital readmission models and assess the usefulness and interpretability of selected fairness metrics. Methods: We used 10.6 million adult inpatient discharges from Maryland and Florida from 2016 to 2019 in this retrospective study. Models predicting 30-day hospital readmissions were evaluated: LACE Index, modified HOSPITAL score, and modified Centers for Medicare & Medicaid Services (CMS) readmission measure, which were applied as-is (using existing coefficients) and retrained (recalibrated with 50% of the data). Predictive performances and bias measures were evaluated for all, between Black and White populations, and between low- and other-income groups. Bias measures included the parity of false negative rate (FNR), false positive rate (FPR), 0-1 loss, and generalized entropy index. Racial bias represented by FNR and FPR differences was stratified to explore shifts in algorithmic bias in different populations. Results: The retrained CMS model demonstrated the best predictive performance (area under the curve: 0.74 in Maryland and 0.68-0.70 in Florida), and the modified HOSPITAL score demonstrated the best calibration (Brier score: 0.16-0.19 in Maryland and 0.19-0.21 in Florida). Calibration was better in White (compared to Black) populations and other-income (compared to low-income) groups, and the area under the curve was higher or similar in the Black (compared to White) populations. The retrained CMS and modified HOSPITAL score had the lowest racial and income bias in Maryland. In Florida, both of these models overall had the lowest income bias and the modified HOSPITAL score showed the lowest racial bias. In both states, the White and higher-income populations showed a higher FNR, while the Black and low-income populations resulted in a higher FPR and a higher 0-1 loss. When stratified by hospital and population composition, these models demonstrated heterogeneous algorithmic bias in different contexts and populations. Conclusions: Caution must be taken when interpreting fairness measures? face value. A higher FNR or FPR could potentially reflect missed opportunities or wasted resources, but these measures could also reflect health care use patterns and gaps in care. Simply relying on the statistical notions of bias could obscure or underplay the causes of health disparity. The imperfect health data, analytic frameworks, and the underlying health systems must be carefully considered. Fairness measures can serve as a useful routine assessment to detect disparate model performances but are insufficient to inform mechanisms or policy changes. However, such an assessment is an important first step toward data-driven improvement to address existing health disparities. UR - https://www.jmir.org/2024/1/e47125 UR - http://dx.doi.org/10.2196/47125 UR - http://www.ncbi.nlm.nih.gov/pubmed/38422347 ID - info:doi/10.2196/47125 ER - TY - JOUR AU - Herrmann-Werner, Anne AU - Festl-Wietek, Teresa AU - Holderried, Friederike AU - Herschbach, Lea AU - Griewatz, Jan AU - Masters, Ken AU - Zipfel, Stephan AU - Mahling, Moritz PY - 2024/4/16 TI - Authors? Reply: ?Evaluating GPT-4?s Cognitive Functions Through the Bloom Taxonomy: Insights and Clarifications? JO - J Med Internet Res SP - e57778 VL - 26 KW - answer KW - artificial intelligence KW - assessment KW - Bloom?s taxonomy KW - ChatGPT KW - classification KW - error KW - exam KW - examination KW - generative KW - GPT-4 KW - Generative Pre-trained Transformer 4 KW - language model KW - learning outcome KW - LLM KW - MCQ KW - medical education KW - medical exam KW - multiple-choice question KW - natural language processing KW - NLP KW - psychosomatic KW - question KW - response KW - taxonomy UR - https://www.jmir.org/2024/1/e57778 UR - http://dx.doi.org/10.2196/57778 UR - http://www.ncbi.nlm.nih.gov/pubmed/38625723 ID - info:doi/10.2196/57778 ER - TY - JOUR AU - Huang, Kuan-Ju PY - 2024/4/16 TI - Evaluating GPT-4?s Cognitive Functions Through the Bloom Taxonomy: Insights and Clarifications JO - J Med Internet Res SP - e56997 VL - 26 KW - artificial intelligence KW - ChatGPT KW - Bloom taxonomy KW - AI KW - cognition UR - https://www.jmir.org/2024/1/e56997 UR - http://dx.doi.org/10.2196/56997 UR - http://www.ncbi.nlm.nih.gov/pubmed/38625725 ID - info:doi/10.2196/56997 ER - TY - JOUR AU - Nwosu, Chinonyelum AU - Khan, Hamda AU - Masese, Rita AU - Nocek, M. Judith AU - Gollan, Siera AU - Varughese, Taniya AU - Bourne, Sarah AU - Clesca, Cindy AU - Jacobs, R. Sara AU - Baumann, Ana AU - Klesges, M. Lisa AU - Shah, Nirmish AU - Hankins, S. Jane AU - Smeltzer, P. Matthew PY - 2024/4/16 TI - Recruitment Strategies in the Integration of Mobile Health Into Sickle Cell Disease Care to Increase Hydroxyurea Utilization Study (meSH): Multicenter Survey Study JO - JMIR Form Res SP - e48767 VL - 8 KW - sickle cell KW - recruitment KW - eHealth KW - multicenter KW - utilization KW - strategy KW - hydroxyurea KW - mobile health KW - mhealth KW - intervention N2 - Background: Hydroxyurea is an evidence-based disease-modifying therapy for sickle cell disease (SCD) but is underutilized. The Integration of Mobile Health into Sickle Cell Disease Care to Increase Hydroxyurea Utilization (meSH) multicenter study leveraged mHealth to deliver targeted interventions to patients and providers. SCD studies often underenroll; and recruitment strategies in the SCD population are not widely studied. Unanticipated events can negatively impact enrollment, making it important to study strategies that ensure adequate study accrual. Objective: The goal of this study was to evaluate enrollment barriers and the impact of modified recruitment strategies among patients and providers in the meSH study in response to a global emergency. Methods: Recruitment was anticipated to last 2 months for providers and 6 months for patients. The recruitment strategies used with patients and providers, new recruitment strategies, and recruitment rates were captured and compared. To document recruitment adaptations and their reasons, study staff responsible for recruitment completed an open-ended 9-item questionnaire eliciting challenges to recruitment and strategies used. Themes were extrapolated using thematic content analysis. Results: Total enrollment across the 7 sites included 89 providers and 293 patients. The study acceptance rate was 85.5% (382/447) for both patients and providers. The reasons patients declined participation were most frequently a lack of time and interest in research, while providers mostly declined because of self-perceived high levels of SCD expertise, believing they did not need the intervention. Initially, recruitment involved an in-person invitation to participate during clinic visits (patients), staff meetings (providers), or within the office (providers). We identified several important recruitment challenges, including (1) lack of interest in research, (2) lack of human resources, (3) unavailable physical space for recruitment activities, and (4) lack of documentation to verify eligibility. Adaptive strategies were crucial to alleviate enrollment disruptions due to the COVID-19 pandemic. These included remote approaching and consenting (eg, telehealth, email, and telephone) for patients and providers. Additionally, for patients, recruitment was enriched by simplification of enrollment procedures (eg, directly approaching patients without a referral from the provider) and a multitouch method (ie, warm introductions with flyers, texts, and patient portal messages). We found that patient recruitment rates were similar between in-person and adapted (virtual with multitouch) approaches (167/200, 83.5% and 126/143, 88.1%, respectively; P=.23). However, for providers, recruitment was significantly higher for in-person vs remote recruitment (48/50, 96% and 41/54, 76%, respectively, P<.001). Conclusions: We found that timely adaptation in recruitment strategies secured high recruitment rates using an assortment of enriched remote recruitment strategies. Flexibility in approach and reducing the burden of enrollment procedures for participants aided enrollment. It is important to continue identifying effective recruitment strategies in studies involving patients with SCD and their providers and the impact and navigation of recruitment challenges. Trial Registration: ClinicalTrials.Gov NCT03380351; https://clinicaltrials.gov/study/NCT03380351 International Registered Report Identifier (IRRID): RR2-10.2196/16319 UR - https://formative.jmir.org/2024/1/e48767 UR - http://dx.doi.org/10.2196/48767 UR - http://www.ncbi.nlm.nih.gov/pubmed/38625729 ID - info:doi/10.2196/48767 ER - TY - JOUR AU - Shulha, Michael AU - Hovdebo, Jordan AU - D?Souza, Vinita AU - Thibault, Francis AU - Harmouche, Rola PY - 2024/4/16 TI - Integrating Explainable Machine Learning in Clinical Decision Support Systems: Study Involving a Modified Design Thinking Approach JO - JMIR Form Res SP - e50475 VL - 8 KW - explainable machine learning KW - XML KW - design thinking approach KW - NASSS framework KW - clinical decision support KW - clinician engagement KW - clinician-facing interface KW - clinician trust in machine learning KW - COVID-19 KW - chest x-ray KW - severity prediction N2 - Background: Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools. Objective: This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface. Methods: We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians? assessments of the domain representation, action ability, and consistency of the tool. Results: Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity. Conclusions: The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools. UR - https://formative.jmir.org/2024/1/e50475 UR - http://dx.doi.org/10.2196/50475 UR - http://www.ncbi.nlm.nih.gov/pubmed/38625728 ID - info:doi/10.2196/50475 ER - TY - JOUR AU - Beverly, A. Elizabeth AU - Koopman-Gonzalez, Sarah AU - Wright, Jackson AU - Dungan, Kathleen AU - Pallerla, Harini AU - Gubitosi-Klug, Rose AU - Baughman, Kristin AU - Konstan, W. Michael AU - Bolen, D. Shari PY - 2024/4/12 TI - Assessing Priorities in a Statewide Cardiovascular and Diabetes Health Collaborative Based on the Results of a Needs Assessment: Cross-Sectional Survey Study JO - JMIR Form Res SP - e55285 VL - 8 KW - health collaborative KW - cardiovascular disease KW - type 2 diabetes KW - needs assessment N2 - Background: The Ohio Cardiovascular and Diabetes Health Collaborative (Cardi-OH) unites general and subspecialty medical staff at the 7 medical schools in Ohio with community and public health partnerships to improve cardiovascular and diabetes health outcomes and eliminate disparities in Ohio?s Medicaid population. Although statewide collaboratives exist to address health improvements, few deploy needs assessments to inform their work. Objective: Cardi-OH conducts an annual needs assessment to identify high-priority clinical topics, screening practices, policy changes for home monitoring devices and referrals, and preferences for the dissemination and implementation of evidence-based best practices. The results of the statewide needs assessment could also be used by others interested in disseminating best practices to primary care teams. Methods: A cross-sectional survey was distributed electronically via REDCap (Research Electronic Data Capture; Vanderbilt University) to both Cardi-OH grant-funded and non?grant-funded members (ie, people who have engaged with Cardi-OH but are not funded by the grant). Results: In total, 88% (103/117) of Cardi-OH grant-funded members and 8.14% (98/1204) of non?grant-funded members completed the needs assessment survey. Of these, 51.5% (53/103) of Cardi-OH grant-funded members and 47% (46/98) of non?grant-funded members provided direct clinical care. The top cardiovascular medicine and diabetes clinical topics for Cardi-OH grant-funded members (clinical and nonclinical) were lifestyle prescriptions (50/103, 48.5%), atypical diabetes (38/103, 36.9%), COVID-19 and cardiovascular disease (CVD; 38/103, 36.9%), and mental health and CVD (38/103, 36.9%). For non?grant-funded members, the top topics were lifestyle prescriptions (53/98, 54%), mental health and CVD (39/98, 40%), alcohol and CVD (27/98, 28%), and cardiovascular complications (27/98, 28%). Regarding social determinants of health, Cardi-OH grant-funded members prioritized 3 topics: weight bias and stigma (44/103, 42.7%), family-focused interventions (40/103, 38.8%), and adverse childhood events (37/103, 35.9%). Non?grant-funded members? choices were family-focused interventions (51/98, 52%), implicit bias (43/98, 44%), and adverse childhood events (39/98, 40%). Assessment of other risk factors for CVD and diabetes across grant- and non?grant-funded members revealed screening for social determinants of health in approximately 50% of patients in each practice, whereas some frequency of depression and substance abuse screening occurred in 80% to 90% of the patients. Access to best practice home monitoring devices was challenging, with 30% (16/53) and 41% (19/46) of clinical grant-funded and non?grant-funded members reporting challenges in obtaining home blood pressure monitoring devices and 68% (36/53) and 43% (20/46) reporting challenges with continuous glucose monitors. Conclusions: Cardi-OH grant- and non?grant-funded members shared the following high-priority topics: lifestyle prescriptions, CVD and mental health, family-focused interventions, alcohol and CVD, and adverse childhood experiences. Identifying high-priority educational topics and preferred delivery modalities for evidence-based materials is essential for ensuring that the dissemination of resources is practical and useful for providers. UR - https://formative.jmir.org/2024/1/e55285 UR - http://dx.doi.org/10.2196/55285 UR - http://www.ncbi.nlm.nih.gov/pubmed/38607661 ID - info:doi/10.2196/55285 ER - TY - JOUR AU - Ominami, Kaya AU - Kushida, Osamu PY - 2024/4/11 TI - Examining and Comparing the Validity and Reproducibility of Scales to Determine the Variety of Vegetables Consumed: Validation Study JO - JMIR Form Res SP - e55795 VL - 8 KW - vegetable KW - variety KW - scale KW - validity KW - reproducibility KW - dietary records KW - nutrition N2 - Background: Previous studies have reported that vegetable variety reduces the risk for noncommunicable diseases independent of the amount consumed. Objective: This study aimed to examine and compare the validity and reproducibility of several scales to determine vegetable variety. Methods: In total, 23 nutrition students in Japan reported their vegetable intake over the past month using a self-administered questionnaire between July and August 2021. Specifically, four scales were used: (1) a single question regarding the number of vegetables consumed (scale A); (2) a scale containing 9 vegetable subgroups included in the brief-type self-administered diet history questionnaire (scale B); (3) a scale containing 19 vegetable items included in a self-administered diet history questionnaire (scale C); and (4) a scale containing 20 vegetable items from the Ranking of Vegetable Consumers in Japan, which was analyzed based on a report on the National Health and Nutrition Survey in Japan (scale D). Scale validity was assessed by correlation with the number of vegetables consumed, which was collected from dietary records for 7 consecutive days. Reproducibility was assessed by test-retest reliability. Results: Regarding the validity of the 4 scales, significant correlations were found between scales C (?=0.51) and D (?=0.44) with vegetable variety based on dietary records, but scales A (?=0.28) and B (?=0.22) were not significantly correlated. Reproducibility showed a significant correlation in scale B (?=0.45) and strong correlations in scales C (?=0.73) and D (?=0.75). Conclusions: The scales for vegetable items have acceptable validity and reproducibility compared to the scales that used a single question or vegetable subgroup and, therefore, may determine the variety of vegetables consumed. UR - https://formative.jmir.org/2024/1/e55795 UR - http://dx.doi.org/10.2196/55795 UR - http://www.ncbi.nlm.nih.gov/pubmed/38603775 ID - info:doi/10.2196/55795 ER - TY - JOUR AU - Segur-Ferrer, Joan AU - Moltó-Puigmartí, Carolina AU - Pastells-Peiró, Roland AU - Vivanco-Hidalgo, Maria Rosa PY - 2024/4/10 TI - Methodological Frameworks and Dimensions to Be Considered in Digital Health Technology Assessment: Scoping Review and Thematic Analysis JO - J Med Internet Res SP - e48694 VL - 26 KW - digital health KW - eHealth KW - mHealth KW - mobile health KW - AI KW - artificial intelligence KW - framework KW - health technology assessment KW - scoping review KW - technology KW - health care system KW - methodological framework KW - thematic analysis N2 - Background: Digital health technologies (dHTs) offer a unique opportunity to address some of the major challenges facing health care systems worldwide. However, the implementation of dHTs raises some concerns, such as the limited understanding of their real impact on health systems and people?s well-being or the potential risks derived from their use. In this context, health technology assessment (HTA) is 1 of the main tools that health systems can use to appraise evidence and determine the value of a given dHT. Nevertheless, due to the nature of dHTs, experts highlight the need to reconsider the frameworks used in traditional HTA. Objective: This scoping review (ScR) aimed to identify the methodological frameworks used worldwide for digital health technology assessment (dHTA); determine what domains are being considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature. Methods: The ScR was performed in accordance with the guidelines established in the PRISMA-ScR guidelines. We searched 7 databases for peer reviews and gray literature published between January 2011 and December 2021. The retrieved studies were screened using Rayyan in a single-blind manner by 2 independent authors, and data were extracted using ATLAS.ti software. The same software was used for thematic analysis. Results: The systematic search retrieved 3061 studies (n=2238, 73.1%, unique), of which 26 (0.8%) studies were included. From these, we identified 102 methodological frameworks designed for dHTA. These frameworks revealed great heterogeneity between them due to their different structures, approaches, and items to be considered in dHTA. In addition, we identified different wording used to refer to similar concepts. Through thematic analysis, we reduced this heterogeneity. In the first phase of the analysis, 176 provisional codes related to different assessment items emerged. In the second phase, these codes were clustered into 86 descriptive themes, which, in turn, were grouped in the third phase into 61 analytical themes and organized through a vertical hierarchy of 3 levels: level 1 formed by 13 domains, level 2 formed by 38 dimensions, and level 3 formed by 11 subdimensions. From these 61 analytical themes, we developed a proposal for a methodological framework for dHTA. Conclusions: There is a need to adapt the existing frameworks used for dHTA or create new ones to more comprehensively assess different kinds of dHTs. Through this ScR, we identified 26 studies including 102 methodological frameworks and tools for dHTA. The thematic analysis of those 26 studies led to the definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered in dHTA. UR - https://www.jmir.org/2024/1/e48694 UR - http://dx.doi.org/10.2196/48694 UR - http://www.ncbi.nlm.nih.gov/pubmed/38598288 ID - info:doi/10.2196/48694 ER - TY - JOUR AU - Choi, Soyoung PY - 2024/4/10 TI - Comparison of Self-Tracking Health Practices, eHealth Literacy, and Subjective Well-Being Between College Students With and Without Disabilities: Cross-Sectional Survey JO - JMIR Form Res SP - e48783 VL - 8 KW - college students KW - personal health data KW - self-tracking KW - eHealth literacy KW - well-being KW - tracking KW - students KW - disability KW - cross-sectional survey KW - pediatric care KW - adult care KW - smartphone health app KW - application KW - literacy N2 - Background: College students with disabilities need to transition from pediatric-centered care to adult care. However, they may become overwhelmed by multiple responsibilities, such as academic activities, peer relationships, career preparation, job seeking, independent living, as well as managing their health and promoting healthy behaviors. Objective: As the use of smartphones and wearable devices for collecting personal health data becomes popular, this study aimed to compare the characteristics of self-tracking health practices between college students with disabilities and their counterparts. In addition, this study examined the relationships between disability status, self-tracking health practices, eHealth literacy, and subjective well-being among college students. Methods: The web-based questionnaire was designed using Qualtrics for the cross-sectional online survey. The survey data were collected from February 2023 to April 2023 and included responses from 702 participants. Results: More than 80% (563/702, 80.2%) of the respondents participated voluntarily in self-tracking health practices. College students with disabilities (n=83) showed significantly lower levels of eHealth literacy and subjective well-being compared with college students without disabilities (n=619). The group with disabilities reported significantly lower satisfaction (t411=?5.97, P<.001) and perceived efficacy (t411=?4.85, P<.001) when using smartphone health apps and wearable devices. Finally, the study identified a significant correlation between subjective well-being in college students and disability status (?=3.81, P<.001), self-tracking health practices (?=2.22, P=.03), and eHealth literacy (?=24.29, P<.001). Conclusions: Given the significant relationships among disability status, self-tracking health practices, eHealth literacy, and subjective well-being in college students, it is recommended to examine their ability to leverage digital technology for self-care. Offering learning opportunities to enhance eHealth literacy and self-tracking health strategies within campus environments could be a strategic approach to improve the quality of life and well-being of college students. UR - https://formative.jmir.org/2024/1/e48783 UR - http://dx.doi.org/10.2196/48783 UR - http://www.ncbi.nlm.nih.gov/pubmed/38598285 ID - info:doi/10.2196/48783 ER - TY - JOUR AU - Hadar-Shoval, Dorit AU - Asraf, Kfir AU - Mizrachi, Yonathan AU - Haber, Yuval AU - Elyoseph, Zohar PY - 2024/4/9 TI - Assessing the Alignment of Large Language Models With Human Values for Mental Health Integration: Cross-Sectional Study Using Schwartz?s Theory of Basic Values JO - JMIR Ment Health SP - e55988 VL - 11 KW - large language models KW - LLMs KW - large language model KW - LLM KW - machine learning KW - ML KW - natural language processing KW - NLP KW - deep learning KW - ChatGPT KW - Chat-GPT KW - chatbot KW - chatbots KW - chat-bot KW - chat-bots KW - Claude KW - values KW - Bard KW - artificial intelligence KW - AI KW - algorithm KW - algorithms KW - predictive model KW - predictive models KW - predictive analytics KW - predictive system KW - practical model KW - practical models KW - mental health KW - mental illness KW - mental illnesses KW - mental disease KW - mental diseases KW - mental disorder KW - mental disorders KW - mobile health KW - mHealth KW - eHealth KW - mood disorder KW - mood disorders N2 - Background: Large language models (LLMs) hold potential for mental health applications. However, their opaque alignment processes may embed biases that shape problematic perspectives. Evaluating the values embedded within LLMs that guide their decision-making have ethical importance. Schwartz?s theory of basic values (STBV) provides a framework for quantifying cultural value orientations and has shown utility for examining values in mental health contexts, including cultural, diagnostic, and therapist-client dynamics. Objective: This study aimed to (1) evaluate whether the STBV can measure value-like constructs within leading LLMs and (2) determine whether LLMs exhibit distinct value-like patterns from humans and each other. Methods: In total, 4 LLMs (Bard, Claude 2, Generative Pretrained Transformer [GPT]-3.5, GPT-4) were anthropomorphized and instructed to complete the Portrait Values Questionnaire?Revised (PVQ-RR) to assess value-like constructs. Their responses over 10 trials were analyzed for reliability and validity. To benchmark the LLMs? value profiles, their results were compared to published data from a diverse sample of 53,472 individuals across 49 nations who had completed the PVQ-RR. This allowed us to assess whether the LLMs diverged from established human value patterns across cultural groups. Value profiles were also compared between models via statistical tests. Results: The PVQ-RR showed good reliability and validity for quantifying value-like infrastructure within the LLMs. However, substantial divergence emerged between the LLMs? value profiles and population data. The models lacked consensus and exhibited distinct motivational biases, reflecting opaque alignment processes. For example, all models prioritized universalism and self-direction, while de-emphasizing achievement, power, and security relative to humans. Successful discriminant analysis differentiated the 4 LLMs? distinct value profiles. Further examination found the biased value profiles strongly predicted the LLMs? responses when presented with mental health dilemmas requiring choosing between opposing values. This provided further validation for the models embedding distinct motivational value-like constructs that shape their decision-making. Conclusions: This study leveraged the STBV to map the motivational value-like infrastructure underpinning leading LLMs. Although the study demonstrated the STBV can effectively characterize value-like infrastructure within LLMs, substantial divergence from human values raises ethical concerns about aligning these models with mental health applications. The biases toward certain cultural value sets pose risks if integrated without proper safeguards. For example, prioritizing universalism could promote unconditional acceptance even when clinically unwise. Furthermore, the differences between the LLMs underscore the need to standardize alignment processes to capture true cultural diversity. Thus, any responsible integration of LLMs into mental health care must account for their embedded biases and motivation mismatches to ensure equitable delivery across diverse populations. Achieving this will require transparency and refinement of alignment techniques to instill comprehensive human values. UR - https://mental.jmir.org/2024/1/e55988 UR - http://dx.doi.org/10.2196/55988 UR - http://www.ncbi.nlm.nih.gov/pubmed/38593424 ID - info:doi/10.2196/55988 ER - TY - JOUR AU - Mugaanyi, Joseph AU - Cai, Liuying AU - Cheng, Sumei AU - Lu, Caide AU - Huang, Jing PY - 2024/4/5 TI - Evaluation of Large Language Model Performance and Reliability for Citations and References in Scholarly Writing: Cross-Disciplinary Study JO - J Med Internet Res SP - e52935 VL - 26 KW - large language models KW - accuracy KW - academic writing KW - AI KW - cross-disciplinary evaluation KW - scholarly writing KW - ChatGPT KW - GPT-3.5 KW - writing tool KW - scholarly KW - academic discourse KW - LLMs KW - machine learning algorithms KW - NLP KW - natural language processing KW - citations KW - references KW - natural science KW - humanities KW - chatbot KW - artificial intelligence N2 - Background: Large language models (LLMs) have gained prominence since the release of ChatGPT in late 2022. Objective: The aim of this study was to assess the accuracy of citations and references generated by ChatGPT (GPT-3.5) in two distinct academic domains: the natural sciences and humanities. Methods: Two researchers independently prompted ChatGPT to write an introduction section for a manuscript and include citations; they then evaluated the accuracy of the citations and Digital Object Identifiers (DOIs). Results were compared between the two disciplines. Results: Ten topics were included, including 5 in the natural sciences and 5 in the humanities. A total of 102 citations were generated, with 55 in the natural sciences and 47 in the humanities. Among these, 40 citations (72.7%) in the natural sciences and 36 citations (76.6%) in the humanities were confirmed to exist (P=.42). There were significant disparities found in DOI presence in the natural sciences (39/55, 70.9%) and the humanities (18/47, 38.3%), along with significant differences in accuracy between the two disciplines (18/55, 32.7% vs 4/47, 8.5%). DOI hallucination was more prevalent in the humanities (42/55, 89.4%). The Levenshtein distance was significantly higher in the humanities than in the natural sciences, reflecting the lower DOI accuracy. Conclusions: ChatGPT?s performance in generating citations and references varies across disciplines. Differences in DOI standards and disciplinary nuances contribute to performance variations. Researchers should consider the strengths and limitations of artificial intelligence writing tools with respect to citation accuracy. The use of domain-specific models may enhance accuracy. UR - https://www.jmir.org/2024/1/e52935 UR - http://dx.doi.org/10.2196/52935 UR - http://www.ncbi.nlm.nih.gov/pubmed/38578685 ID - info:doi/10.2196/52935 ER - TY - JOUR AU - Loeb, Talia AU - Willis, Kalai AU - Velishavo, Frans AU - Lee, Daniel AU - Rao, Amrita AU - Baral, Stefan AU - Rucinski, Katherine PY - 2024/4/4 TI - Leveraging Routinely Collected Program Data to Inform Extrapolated Size Estimates for Key Populations in Namibia: Small Area Estimation Study JO - JMIR Public Health Surveill SP - e48963 VL - 10 KW - female sex workers KW - HIV KW - key populations KW - men who have sex with men KW - Namibia KW - population size estimation KW - small area estimation N2 - Background: Estimating the size of key populations, including female sex workers (FSW) and men who have sex with men (MSM), can inform planning and resource allocation for HIV programs at local and national levels. In geographic areas where direct population size estimates (PSEs) for key populations have not been collected, small area estimation (SAE) can help fill in gaps using supplemental data sources known as auxiliary data. However, routinely collected program data have not historically been used as auxiliary data to generate subnational estimates for key populations, including in Namibia. Objective: To systematically generate regional size estimates for FSW and MSM in Namibia, we used a consensus-informed estimation approach with local stakeholders that included the integration of routinely collected HIV program data provided by key populations? HIV service providers. Methods: We used quarterly program data reported by key population implementing partners, including counts of the number of individuals accessing HIV services over time, to weight existing PSEs collected through bio-behavioral surveys using a Bayesian triangulation approach. SAEs were generated through simple imputation, stratified imputation, and multivariable Poisson regression models. We selected final estimates using an iterative qualitative ranking process with local key population implementing partners. Results: Extrapolated national estimates for FSW ranged from 4777 to 13,148 across Namibia, comprising 1.5% to 3.6% of female individuals aged between 15 and 49 years. For MSM, estimates ranged from 4611 to 10,171, comprising 0.7% to 1.5% of male individuals aged between 15 and 49 years. After the inclusion of program data as priors, the estimated proportion of FSW derived from simple imputation increased from 1.9% to 2.8%, and the proportion of MSM decreased from 1.5% to 0.75%. When stratified imputation was implemented using HIV prevalence to inform strata, the inclusion of program data increased the proportion of FSW from 2.6% to 4.0% in regions with high prevalence and decreased the proportion from 1.4% to 1.2% in regions with low prevalence. When population density was used to inform strata, the inclusion of program data also increased the proportion of FSW in high-density regions (from 1.1% to 3.4%) and decreased the proportion of MSM in all regions. Conclusions: Using SAE approaches, we combined epidemiologic and program data to generate subnational size estimates for key populations in Namibia. Overall, estimates were highly sensitive to the inclusion of program data. Program data represent a supplemental source of information that can be used to align PSEs with real-world HIV programs, particularly in regions where population-based data collection methods are challenging to implement. Future work is needed to determine how best to include and validate program data in target settings and in key population size estimation studies, ultimately bridging research with practice to support a more comprehensive HIV response. UR - https://publichealth.jmir.org/2024/1/e48963 UR - http://dx.doi.org/10.2196/48963 UR - http://www.ncbi.nlm.nih.gov/pubmed/38573760 ID - info:doi/10.2196/48963 ER - TY - JOUR AU - McMurry, J. Andrew AU - Zipursky, R. Amy AU - Geva, Alon AU - Olson, L. Karen AU - Jones, R. James AU - Ignatov, Vladimir AU - Miller, A. Timothy AU - Mandl, D. Kenneth PY - 2024/4/4 TI - Moving Biosurveillance Beyond Coded Data Using AI for Symptom Detection From Physician Notes: Retrospective Cohort Study JO - J Med Internet Res SP - e53367 VL - 26 KW - natural language processing KW - COVID-19 KW - artificial intelligence KW - AI KW - public health, biosurveillance KW - surveillance KW - respiratory KW - infectious KW - pulmonary KW - SARS-CoV-2 KW - symptom KW - symptoms KW - detect KW - detection KW - pipeline KW - pipelines KW - clinical note KW - clinical notes KW - documentation KW - emergency KW - urgent KW - pediatric KW - pediatrics KW - paediatric KW - paediatrics KW - child KW - children KW - youth KW - adolescent KW - adolescents KW - teen KW - teens KW - teenager KW - teenagers KW - diagnose KW - diagnosis KW - diagnostic KW - diagnostics N2 - Background: Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records. Objective: This study sought to validate and test an artificial intelligence (AI)?based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak. Methods: Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children?s hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras. Results: There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras. Conclusions: This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance. UR - https://www.jmir.org/2024/1/e53367 UR - http://dx.doi.org/10.2196/53367 UR - http://www.ncbi.nlm.nih.gov/pubmed/38573752 ID - info:doi/10.2196/53367 ER - TY - JOUR AU - Choo, Mei Sim AU - Sartori, Daniele AU - Lee, Chet Sing AU - Yang, Hsuan-Chia AU - Syed-Abdul, Shabbir PY - 2024/4/3 TI - Data-Driven Identification of Factors That Influence the Quality of Adverse Event Reports: 15-Year Interpretable Machine Learning and Time-Series Analyses of VigiBase and QUEST JO - JMIR Med Inform SP - e49643 VL - 12 KW - pharmacovigilance KW - medication safety KW - big data analysis KW - feature selection KW - interpretable machine learning N2 - Background: The completeness of adverse event (AE) reports, crucial for assessing putative causal relationships, is measured using the vigiGrade completeness score in VigiBase, the World Health Organization global database of reported potential AEs. Malaysian reports have surpassed the global average score (approximately 0.44), achieving a 5-year average of 0.79 (SD 0.23) as of 2019 and approaching the benchmark for well-documented reports (0.80). However, the contributing factors to this relatively high report completeness score remain unexplored. Objective: This study aims to explore the main drivers influencing the completeness of Malaysian AE reports in VigiBase over a 15-year period using vigiGrade. A secondary objective was to understand the strategic measures taken by the Malaysian authorities leading to enhanced report completeness across different time frames. Methods: We analyzed 132,738 Malaysian reports (2005-2019) recorded in VigiBase up to February 2021 split into historical International Drug Information System (INTDIS; n=63,943, 48.17% in 2005-2016) and newer E2B (n=68,795, 51.83% in 2015-2019) format subsets. For machine learning analyses, we performed a 2-stage feature selection followed by a random forest classifier to identify the top features predicting well-documented reports. We subsequently applied tree Shapley additive explanations to examine the magnitude, prevalence, and direction of feature effects. In addition, we conducted time-series analyses to evaluate chronological trends and potential influences of key interventions on reporting quality. Results: Among the analyzed reports, 42.84% (56,877/132,738) were well documented, with an increase of 65.37% (53,929/82,497) since 2015. Over two-thirds (46,186/68,795, 67.14%) of the Malaysian E2B reports were well documented compared to INTDIS reports at 16.72% (10,691/63,943). For INTDIS reports, higher pharmacovigilance center staffing was the primary feature positively associated with being well documented. In recent E2B reports, the top positive features included reaction abated upon drug dechallenge, reaction onset or drug use duration of <1 week, dosing interval of <1 day, reports from public specialist hospitals, reports by pharmacists, and reaction duration between 1 and 6 days. In contrast, reports from product registration holders and other health care professionals and reactions involving product substitution issues negatively affected the quality of E2B reports. Multifaceted strategies and interventions comprising policy changes, continuity of education, and human resource development laid the groundwork for AE reporting in Malaysia, whereas advancements in technological infrastructure, pharmacovigilance databases, and reporting tools concurred with increases in both the quantity and quality of AE reports. Conclusions: Through interpretable machine learning and time-series analyses, this study identified key features that positively or negatively influence the completeness of Malaysian AE reports and unveiled how Malaysia has developed its pharmacovigilance capacity via multifaceted strategies and interventions. These findings will guide future work in enhancing pharmacovigilance and public health. UR - https://medinform.jmir.org/2024/1/e49643 UR - http://dx.doi.org/10.2196/49643 UR - http://www.ncbi.nlm.nih.gov/pubmed/38568722 ID - info:doi/10.2196/49643 ER - TY - JOUR AU - He, Yunfan AU - Zhu, Wei AU - Wang, Tong AU - Chen, Han AU - Xin, Junyi AU - Liu, Yongcheng AU - Lei, Jianbo AU - Liang, Jun PY - 2024/3/28 TI - Mining User Reviews From Hypertension Management Mobile Health Apps to Explore Factors Influencing User Satisfaction and Their Asymmetry: Comparative Study JO - JMIR Mhealth Uhealth SP - e55199 VL - 12 KW - hypertension management KW - mobile health KW - topic modeling KW - satisfaction KW - 2-factor model KW - comparative study N2 - Background: Hypertension significantly impacts the well-being and health of individuals globally. Hypertension management apps (HMAs) have been shown to assist patients in controlling blood pressure (BP), with their efficacy validated in clinical trials. However, the utilization of HMAs continues to be suboptimal. Presently, there is a dearth of real-world research based on big data and exploratory mining that compares Chinese and American HMAs. Objective: This study aims to systematically gather HMAs and their user reviews from both China and the United States. Subsequently, using data mining techniques, the study aims to compare the user experience, satisfaction levels, influencing factors, and asymmetry between Chinese and American users of HMAs. In addition, the study seeks to assess the disparities in satisfaction and its determinants while delving into the asymmetry of these factors. Methods: The study sourced HMAs and user reviews from 10 prominent Chinese and American app stores globally. Using the latent Dirichlet allocation (LDA) topic model, the research identified various topics within user reviews. Subsequently, the Tobit model was used to investigate the impact and distinctions of each topic on user satisfaction. The Wald test was applied to analyze differences in effects across various factors. Results: We examined a total of 261 HMAs along with their associated user reviews, amounting to 116,686 reviews in total. In terms of quantity and overall satisfaction levels, Chinese HMAs (n=91) and corresponding reviews (n=16,561) were notably fewer compared with their American counterparts (n=220 HMAs and n=100,125 reviews). The overall satisfaction rate among HMA users was 75.22% (87,773/116,686), with Chinese HMAs demonstrating a higher satisfaction rate (13,866/16,561, 83.73%) compared with that for American HMAs (73,907/100,125, 73.81%). Chinese users primarily focus on reliability (2165/16,561, 13.07%) and measurement accuracy (2091/16,561, 12.63%) when considering HMAs, whereas American users prioritize BP tracking (17,285/100,125, 17.26%) and data synchronization (12,837/100,125, 12.82%). Seven factors (easy to use: P<.001; measurement accuracy: P<.001; compatibility: P<.001; cost: P<.001; heart rate detection function: P=.02; blood pressure tracking function: P<.001; and interface design: P=.01) significantly influenced the positive deviation (PD) of Chinese HMA user satisfaction, while 8 factors (easy to use: P<.001; reliability: P<.001; measurement accuracy: P<.001; compatibility: P<.001; cost: P<.001; interface design: P<.001; real-time: P<.001; and data privacy: P=.001) affected the negative deviation (ND). Notably, BP tracking had the greatest effect on PD (?=.354, P<.001), while cost had the most significant impact on ND (?=3.703, P<.001). All 12 factors (easy to use: P<.001; blood pressure tracking function: P<.001; data synchronization: P<.001; blood pressure management effect: P<.001; heart rate detection function: P<.001; data sharing: P<.001; reliability: P<.001; compatibility: P<.001; interface design: P<.001; advertisement distribution: P<.001; measurement accuracy: P<.001; and cost: P<.001) significantly influenced the PD and ND of American HMA user satisfaction. Notably, BP tracking had the greatest effect on PD (?=0.312, P<.001), while data synchronization had the most significant impact on ND (?=2.662, P<.001). In addition, the influencing factors of PD and ND in user satisfaction of HMA in China and the United States are different. Conclusions: User satisfaction factors varied significantly between different countries, showing considerable asymmetry. For Chinese HMA users, ease of use and interface design emerged as motivational factors, while factors such as cost, measurement accuracy, and compatibility primarily contributed to user dissatisfaction. For American HMA users, motivational factors were ease of use, BP tracking, BP management effect, interface design, measurement accuracy, and cost. Moreover, users expect features such as data sharing, synchronization, software reliability, compatibility, heart rate detection, and nonintrusive advertisement distribution. Tailored experience plans should be devised for different user groups in various countries to address these diverse preferences and requirements. UR - https://mhealth.jmir.org/2024/1/e55199 UR - http://dx.doi.org/10.2196/55199 UR - http://www.ncbi.nlm.nih.gov/pubmed/38547475 ID - info:doi/10.2196/55199 ER - TY - JOUR AU - Macdonald, Trystan AU - Dinnes, Jacqueline AU - Maniatopoulos, Gregory AU - Taylor-Phillips, Sian AU - Shinkins, Bethany AU - Hogg, Jeffry AU - Dunbar, Kevin John AU - Solebo, Lola Ameenat AU - Sutton, Hannah AU - Attwood, John AU - Pogose, Michael AU - Given-Wilson, Rosalind AU - Greaves, Felix AU - Macrae, Carl AU - Pearson, Russell AU - Bamford, Daniel AU - Tufail, Adnan AU - Liu, Xiaoxuan AU - Denniston, K. Alastair PY - 2024/3/27 TI - Target Product Profile for a Machine Learning?Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study JO - JMIR Res Protoc SP - e50568 VL - 13 KW - artificial intelligence KW - design KW - developers KW - diabetes mellitus KW - diabetic eye screening KW - diabetic retinopathy KW - diabetic KW - DM KW - England KW - eye screening KW - imaging analysis software KW - implementation KW - machine learning KW - retinal imaging KW - study protocol KW - target product profile N2 - Background: Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. Objective: This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. Methods: This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence?s Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from ?definitely exclude? to ?definitely include,? and suggest edits. The document will be iterated between rounds based on participants? feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. Results: Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. Conclusions: The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. International Registered Report Identifier (IRRID): DERR1-10.2196/50568 UR - https://www.researchprotocols.org/2024/1/e50568 UR - http://dx.doi.org/10.2196/50568 UR - http://www.ncbi.nlm.nih.gov/pubmed/38536234 ID - info:doi/10.2196/50568 ER - TY - JOUR AU - Kilshaw, E. Robyn AU - Boggins, Abigail AU - Everett, Olivia AU - Butner, Emma AU - Leifker, R. Feea AU - Baucom, W. Brian R. PY - 2024/3/27 TI - Benchmarking Mental Health Status Using Passive Sensor Data: Protocol for a Prospective Observational Study JO - JMIR Res Protoc SP - e53857 VL - 13 KW - audio data KW - computational psychiatry KW - data repository KW - digital phenotyping KW - machine learning KW - passive sensor data N2 - Background: Computational psychiatry has the potential to advance the diagnosis, mechanistic understanding, and treatment of mental health conditions. Promising results from clinical samples have led to calls to extend these methods to mental health risk assessment in the general public; however, data typically used with clinical samples are neither available nor scalable for research in the general population. Digital phenotyping addresses this by capitalizing on the multimodal and widely available data created by sensors embedded in personal digital devices (eg, smartphones) and is a promising approach to extending computational psychiatry methods to improve mental health risk assessment in the general population. Objective: Building on recommendations from existing computational psychiatry and digital phenotyping work, we aim to create the first computational psychiatry data set that is tailored to studying mental health risk in the general population; includes multimodal, sensor-based behavioral features; and is designed to be widely shared across academia, industry, and government using gold standard methods for privacy, confidentiality, and data integrity. Methods: We are using a stratified, random sampling design with 2 crossed factors (difficulties with emotion regulation and perceived life stress) to recruit a sample of 400 community-dwelling adults balanced across high- and low-risk for episodic mental health conditions. Participants first complete self-report questionnaires assessing current and lifetime psychiatric and medical diagnoses and treatment, and current psychosocial functioning. Participants then complete a 7-day in situ data collection phase that includes providing daily audio recordings, passive sensor data collected from smartphones, self-reports of daily mood and significant events, and a verbal description of the significant daily events during a nightly phone call. Participants complete the same baseline questionnaires 6 and 12 months after this phase. Self-report questionnaires will be scored using standard methods. Raw audio and passive sensor data will be processed to create a suite of daily summary features (eg, time spent at home). Results: Data collection began in June 2022 and is expected to conclude by July 2024. To date, 310 participants have consented to the study; 149 have completed the baseline questionnaire and 7-day intensive data collection phase; and 61 and 31 have completed the 6- and 12-month follow-up questionnaires, respectively. Once completed, the proposed data set will be made available to academic researchers, industry, and the government using a stepped approach to maximize data privacy. Conclusions: This data set is designed as a complementary approach to current computational psychiatry and digital phenotyping research, with the goal of advancing mental health risk assessment within the general population. This data set aims to support the field?s move away from siloed research laboratories collecting proprietary data and toward interdisciplinary collaborations that incorporate clinical, technical, and quantitative expertise at all stages of the research process. International Registered Report Identifier (IRRID): DERR1-10.2196/53857 UR - https://www.researchprotocols.org/2024/1/e53857 UR - http://dx.doi.org/10.2196/53857 UR - http://www.ncbi.nlm.nih.gov/pubmed/38536220 ID - info:doi/10.2196/53857 ER - TY - JOUR AU - Raja, Hina AU - Munawar, Asim AU - Mylonas, Nikolaos AU - Delsoz, Mohammad AU - Madadi, Yeganeh AU - Elahi, Muhammad AU - Hassan, Amr AU - Abu Serhan, Hashem AU - Inam, Onur AU - Hernandez, Luis AU - Chen, Hao AU - Tran, Sang AU - Munir, Wuqaas AU - Abd-Alrazaq, Alaa AU - Yousefi, Siamak PY - 2024/3/22 TI - Automated Category and Trend Analysis of Scientific Articles on Ophthalmology Using Large Language Models: Development and Usability Study JO - JMIR Form Res SP - e52462 VL - 8 KW - Bidirectional and Auto-Regressive Transformers KW - BART KW - bidirectional encoder representations from transformers KW - BERT KW - ophthalmology KW - text classification KW - large language model KW - LLM KW - trend analysis N2 - Background: In this paper, we present an automated method for article classification, leveraging the power of large language models (LLMs). Objective: The aim of this study is to evaluate the applicability of various LLMs based on textual content of scientific ophthalmology papers. Methods: We developed a model based on natural language processing techniques, including advanced LLMs, to process and analyze the textual content of scientific papers. Specifically, we used zero-shot learning LLMs and compared Bidirectional and Auto-Regressive Transformers (BART) and its variants with Bidirectional Encoder Representations from Transformers (BERT) and its variants, such as distilBERT, SciBERT, PubmedBERT, and BioBERT. To evaluate the LLMs, we compiled a data set (retinal diseases [RenD] ) of 1000 ocular disease?related articles, which were expertly annotated by a panel of 6 specialists into 19 distinct categories. In addition to the classification of articles, we also performed analysis on different classified groups to find the patterns and trends in the field. Results: The classification results demonstrate the effectiveness of LLMs in categorizing a large number of ophthalmology papers without human intervention. The model achieved a mean accuracy of 0.86 and a mean F1-score of 0.85 based on the RenD data set. Conclusions: The proposed framework achieves notable improvements in both accuracy and efficiency. Its application in the domain of ophthalmology showcases its potential for knowledge organization and retrieval. We performed a trend analysis that enables researchers and clinicians to easily categorize and retrieve relevant papers, saving time and effort in literature review and information gathering as well as identification of emerging scientific trends within different disciplines. Moreover, the extendibility of the model to other scientific fields broadens its impact in facilitating research and trend analysis across diverse disciplines. UR - https://formative.jmir.org/2024/1/e52462 UR - http://dx.doi.org/10.2196/52462 UR - http://www.ncbi.nlm.nih.gov/pubmed/38517457 ID - info:doi/10.2196/52462 ER - TY - JOUR AU - van Alen, Marie Catharina AU - Brenner, Alexander AU - Warnecke, Tobias AU - Varghese, Julian PY - 2024/3/20 TI - Smartwatch Versus Routine Tremor Documentation: Descriptive Comparison JO - JMIR Form Res SP - e51249 VL - 8 KW - Parkinson disease KW - tremor KW - smart wearables KW - smartwatch KW - mobile apps KW - movement disorders KW - tremor documentation KW - tremor occurrence KW - tremor score UR - https://formative.jmir.org/2024/1/e51249 UR - http://dx.doi.org/10.2196/51249 UR - http://www.ncbi.nlm.nih.gov/pubmed/38506919 ID - info:doi/10.2196/51249 ER - TY - JOUR AU - Baek, Jinyoung AU - Lawson, Jonathan AU - Rahimzadeh, Vasiliki PY - 2024/3/20 TI - Investigating the Roles and Responsibilities of Institutional Signing Officials After Data Sharing Policy Reform for Federally Funded Research in the United States: National Survey JO - JMIR Form Res SP - e49822 VL - 8 KW - biomedical research KW - survey KW - surveys KW - data sharing KW - data management KW - secondary use KW - National Institutes of Health KW - signing official KW - information sharing KW - exchange KW - access KW - data science KW - accessibility KW - policy KW - policies N2 - Background: New federal policies along with rapid growth in data generation, storage, and analysis tools are together driving scientific data sharing in the United States. At the same, triangulating human research data from diverse sources can also create situations where data are used for future research in ways that individuals and communities may consider objectionable. Institutional gatekeepers, namely, signing officials (SOs), are therefore at the helm of compliant management and sharing of human data for research. Of those with data governance responsibilities, SOs most often serve as signatories for investigators who deposit, access, and share research data between institutions. Although SOs play important leadership roles in compliant data sharing, we know surprisingly little about their scope of work, roles, and oversight responsibilities. Objective: The purpose of this study was to describe existing institutional policies and practices of US SOs who manage human genomic data access, as well as how these may change in the wake of new Data Management and Sharing requirements for National Institutes of Health?funded research in the United States. Methods: We administered an anonymous survey to institutional SOs recruited from biomedical research institutions across the United States. Survey items probed where data generated from extramurally funded research are deposited, how researchers outside the institution access these data, and what happens to these data after extramural funding ends. Results: In total, 56 institutional SOs participated in the survey. We found that SOs frequently approve duplicate data deposits and impose stricter access controls when data use limitations are unclear or unspecified. In addition, 21% (n=12) of SOs knew where data from federally funded projects are deposited after project funding sunsets. As a consequence, most investigators deposit their scientific data into ?a National Institutes of Health?funded repository? to meet the Data Management and Sharing requirements but also within the ?institution?s own repository? or a third-party repository. Conclusions: Our findings inform 5 policy recommendations and best practices for US SOs to improve coordination and develop comprehensive and consistent data governance policies that balance the need for scientific progress with effective human data protections. UR - https://formative.jmir.org/2024/1/e49822 UR - http://dx.doi.org/10.2196/49822 UR - http://www.ncbi.nlm.nih.gov/pubmed/38506894 ID - info:doi/10.2196/49822 ER - TY - JOUR AU - Fellows, E. Ian AU - Corcoran, Carl AU - McIntyre, F. Anne PY - 2024/3/19 TI - Triangulating Truth and Reaching Consensus on Population Size, Prevalence, and More: Modeling Study JO - JMIR Public Health Surveill SP - e48738 VL - 10 KW - HIV KW - epidemiology KW - population size estimation KW - key populations KW - Bayesian models KW - consensus estimation KW - statistical tool KW - prevalence KW - Bayesian model KW - population KW - estimate KW - consensus KW - population size N2 - Background: Population size, prevalence, and incidence are essential metrics that influence public health programming and policy. However, stakeholders are frequently tasked with setting performance targets, reporting global indicators, and designing policies based on multiple (often incongruous) estimates of these variables, and they often do so in the absence of a formal, transparent framework for reaching a consensus estimate. Objective: This study aims to describe a model to synthesize multiple study estimates while incorporating stakeholder knowledge, introduce an R Shiny app to implement the model, and demonstrate the model and app using real data. Methods: In this study, we developed a Bayesian hierarchical model to synthesize multiple study estimates that allow the user to incorporate the quality of each estimate as a confidence score. The model was implemented as a user-friendly R Shiny app aimed at practitioners of population size estimation. The underlying Bayesian model was programmed in Stan for efficient sampling and computation. Results: The app was demonstrated using biobehavioral survey-based population size estimates (and accompanying confidence scores) of female sex workers and men who have sex with men from 3 survey locations in a country in sub-Saharan Africa. The consensus results incorporating confidence scores are compared with the case where they are absent, and the results with confidence scores are shown to perform better according to an app-supplied metric for unaccounted-for variation. Conclusions: The utility of the triangulator model, including the incorporation of confidence scores, as a user-friendly app is demonstrated using a use case example. Our results offer empirical evidence of the model?s effectiveness in producing an accurate consensus estimate and emphasize the significant impact that the accessible model and app offer for public health. It offers a solution to the long-standing problem of synthesizing multiple estimates, potentially leading to more informed and evidence-based decision-making processes. The Triangulator has broad utility and flexibility to be adapted and used in various other contexts and regions to address similar challenges. UR - https://publichealth.jmir.org/2024/1/e48738 UR - http://dx.doi.org/10.2196/48738 UR - http://www.ncbi.nlm.nih.gov/pubmed/38502183 ID - info:doi/10.2196/48738 ER - TY - JOUR AU - Abdulai, Abdul-Fatawu AU - Naghdali, Hasti AU - Noga, Heather AU - Yong, J. Paul PY - 2024/3/15 TI - Patient-Centered Approaches for Designing Destigmatizing Sexual Pain-Related Web-Based Platforms: Qualitative Study JO - JMIR Form Res SP - e53742 VL - 8 KW - stigma KW - digital health KW - sexual pain KW - destigmatizing KW - end user patients N2 - Background: Sexual pain is a common but neglected disorder that affects approximately 3% to 18% of women and an unmeasured number of gender-diverse people worldwide. Despite its wide prevalence, many people feel reluctant to visit conventional health care services or disclose their symptoms due to the fear of stigmatization. To alleviate this stigma, various web-based interventions have been developed to complement and, in some cases, replace conventional sexual health interventions. However, the way these web-based interventions are developed could inadvertently reproduce, perpetuate, or exacerbate stigma among end user patients. Objective: The purpose of this study was to understand patients? perspectives on how sexual pain?related web platforms can be designed to alleviate stigma or prevent the unintended effects of stigma among patients who use web-based interventions. Methods: Individual semistructured interviews were conducted among 16 participants with lived experiences of painful sex in a large urban city in Western Canada. Participants were recruited via social media platforms, newsletters, and a provincial health volunteer website. Using a sample sexual pain website to provide context, participants were interviewed about their experiences of stigma and how they think web platforms could be designed to address stigma. The interviews were conducted via Zoom (Zoom Technologies Inc) and analyzed using thematic analysis. Results: The findings revealed 4 overarching themes that represented participants? perspectives on designing web platforms that may alleviate or prevent the unintended effects of stigma. These findings suggested the design of inclusive web platforms, having a nonprovocative and calming user interface, having features that facilitate connections among users and between users and providers, and displaying personal testimonials and experiences of sexual pain. Conclusions: This study highlighted patient-centered design approaches that could serve as a reference guide in developing web platforms that alleviate or prevent the unintended effects of stigma, particularly among nonheterosexual and gender-diverse people. While this study was conducted in the context of sexual pain, the results might also apply to web platforms on other potentially stigmatizing health-related disorders or conditions. UR - https://formative.jmir.org/2024/1/e53742 UR - http://dx.doi.org/10.2196/53742 UR - http://www.ncbi.nlm.nih.gov/pubmed/38488844 ID - info:doi/10.2196/53742 ER - TY - JOUR AU - Hatef, Elham AU - Chang, Hsien-Yen AU - Richards, M. Thomas AU - Kitchen, Christopher AU - Budaraju, Janya AU - Foroughmand, Iman AU - Lasser, C. Elyse AU - Weiner, P. Jonathan PY - 2024/3/12 TI - Development of a Social Risk Score in the Electronic Health Record to Identify Social Needs Among Underserved Populations: Retrospective Study JO - JMIR Form Res SP - e54732 VL - 8 KW - AI KW - algorithms KW - artificial intelligence KW - community health KW - deep learning KW - EHR KW - electronic health record KW - machine learning KW - ML KW - population demographics KW - population health KW - practical models KW - predictive analytics KW - predictive modeling KW - predictive modelling KW - predictive models KW - predictive system KW - public health KW - public surveillance KW - SDOH KW - social determinants of health KW - social needs KW - social risks N2 - Background: Patients with unmet social needs and social determinants of health (SDOH) challenges continue to face a disproportionate risk of increased prevalence of disease, health care use, higher health care costs, and worse outcomes. Some existing predictive models have used the available data on social needs and SDOH challenges to predict health-related social needs or the need for various social service referrals. Despite these one-off efforts, the work to date suggests that many technical and organizational challenges must be surmounted before SDOH-integrated solutions can be implemented on an ongoing, wide-scale basis within most US-based health care organizations. Objective: We aimed to retrieve available information in the electronic health record (EHR) relevant to the identification of persons with social needs and to develop a social risk score for use within clinical practice to better identify patients at risk of having future social needs. Methods: We conducted a retrospective study using EHR data (2016-2021) and data from the US Census American Community Survey. We developed a prospective model using current year-1 risk factors to predict future year-2 outcomes within four 2-year cohorts. Predictors of interest included demographics, previous health care use, comorbidity, previously identified social needs, and neighborhood characteristics as reflected by the area deprivation index. The outcome variable was a binary indicator reflecting the likelihood of the presence of a patient with social needs. We applied a generalized estimating equation approach, adjusting for patient-level risk factors, the possible effect of geographically clustered data, and the effect of multiple visits for each patient. Results: The study population of 1,852,228 patients included middle-aged (mean age range 53.76-55.95 years), White (range 324,279/510,770, 63.49% to 290,688/488,666, 64.79%), and female (range 314,741/510,770, 61.62% to 278,488/448,666, 62.07%) patients from neighborhoods with high socioeconomic status (mean area deprivation index percentile range 28.76-30.31). Between 8.28% (37,137/448,666) and 11.55% (52,037/450,426) of patients across the study cohorts had at least 1 social need documented in their EHR, with safety issues and economic challenges (ie, financial resource strain, employment, and food insecurity) being the most common documented social needs (87,152/1,852,228, 4.71% and 58,242/1,852,228, 3.14% of overall patients, respectively). The model had an area under the curve of 0.702 (95% CI 0.699-0.705) in predicting prospective social needs in the overall study population. Previous social needs (odds ratio 3.285, 95% CI 3.237-3.335) and emergency department visits (odds ratio 1.659, 95% CI 1.634-1.684) were the strongest predictors of future social needs. Conclusions: Our model provides an opportunity to make use of available EHR data to help identify patients with high social needs. Our proposed social risk score could help identify the subset of patients who would most benefit from further social needs screening and data collection to avoid potentially more burdensome primary data collection on all patients in a target population of interest. UR - https://formative.jmir.org/2024/1/e54732 UR - http://dx.doi.org/10.2196/54732 UR - http://www.ncbi.nlm.nih.gov/pubmed/38470477 ID - info:doi/10.2196/54732 ER - TY - JOUR AU - Kazemi, Alireza AU - Boyd, Marisha AU - Choi, Fiona AU - Tai, Yeung Andy Man AU - Tsang, WL Vivian AU - To, Tam AU - Kim, Jane AU - Jang, Kerry AU - Shams, Farhud AU - Schreiter, Stefanie AU - Cabanis, Maurice AU - Krausz, Michael Reinhard PY - 2024/3/11 TI - Architecture and Development Framework for a Web-Based Risk Assessment and Management Platform Developed on WordPress to Address Opioid Overdose JO - JMIR Form Res SP - e49759 VL - 8 KW - software designs KW - risks management KW - risk assessments KW - opioid overdose KW - crisis intervention KW - substance related disorders UR - https://formative.jmir.org/2024/1/e49759 UR - http://dx.doi.org/10.2196/49759 UR - http://www.ncbi.nlm.nih.gov/pubmed/38466977 ID - info:doi/10.2196/49759 ER - TY - JOUR AU - Burch, Vanessa AU - McGuire, Jessica AU - Buch, Eric AU - Sathekge, Mike AU - M'bouaffou, Francis AU - Senkubuge, Flavia AU - Fagan, Johannes PY - 2024/3/6 TI - Feasibility and Acceptability of Web-Based Structured Oral Examinations for Postgraduate Certification: Mixed Methods Preliminary Evaluation JO - JMIR Form Res SP - e40868 VL - 8 KW - web-based certification examinations KW - web-based structured oral examinations KW - medical education KW - specialist and subspecialist examinations KW - structured oral examinations KW - Colleges of Medicine of South Africa N2 - Background: The COVID-19 pandemic disrupted postgraduate certification examinations globally. The Colleges of Medicine of South Africa continued hosting certification examinations through the pandemic. This was achieved by effecting a rapid transition from in-person to web-based certification examinations. Objective: This formative evaluation explored candidates? acceptability of web-based structured oral examinations (SOEs) hosted via Zoom (Zoom Communications Inc). We also reported the audiovisual quality and technical challenges encountered while using Zoom and candidates? overall experience with these examinations conducted during the early part of the COVID-19 pandemic. Additionally, performance in web-based certification examinations was compared with previous in-person certification examinations. Methods: This mixed methods, single-arm evaluation anonymously gathered candidates? perceptions of web-based SOE acceptability, audiovisual quality, and overall experience with Zoom using a web-based survey. Pass rates of web-based and previous in-person certification examinations were compared using chi-square tests, with a Yates correction. A thematic analysis approach was adopted for qualitative data. Results: Between June 2020 and June 2021, 3105 candidates registered for certification examinations, 293 (9.4%) withdrew, 2812 (90.6%) wrote, and 2799 (99.9%) passed, and 1525 (54.2%) were invited to a further web-based SOE. Examination participation was 96.2% (n=1467). During the first web-based examination cycle (2020), 542 (87.1%) of 622 web-based SOE candidates completed the web-based survey. They reported web-based SOEs as fair (374/542, 69%) and adequately testing their clinical reasoning and insight (396/542, 73.1%). Few would have preferred real patient encounters (173/542, 31.9%) or in-person oral examinations (152/542, 28%). Most found Zoom acceptable (434/542, 80%) and fair (396/542, 73.1%) for hosting web-based SOEs. SOEs resulted in financial (434/542, 80%) and time (428/542, 79%) savings for candidates. Many (336/542, 62%) supported the ongoing use of web-based certification examinations. Only 169 technical challenges in using Zoom were reported, which included connectivity-related issues, poor audio quality, and poor image quality. The thematic analysis identified 4 themes of positive and negative experiences related to web-based SOE station design and content, examination station environment, examiner-candidate interactions, and personal benefits for candidates. Our qualitative analysis identified 10 improvements for future web-based SOEs. Candidates achieved high pass rates in web-based certification examinations in 2020 (1583/1732, 91.39%) and 2021 (850/1067, 79.66%). These were significantly higher (2020: N=8635; ?21=667; P<.001; 2021: N=7988; ?21=178; P<.001) than the previous in-person certification examination pass rate of 58.23% (4030/6921; 2017-2019). Conclusions: Web-based SOEs conducted by the Colleges of Medicine of South Africa during the COVID-19 pandemic were well received by candidates, and few technical difficulties were encountered while using Zoom. Better performance was observed in web-based examinations than in previous in-person certification examinations. These early findings support the ongoing use of this assessment method. UR - https://formative.jmir.org/2024/1/e40868 UR - http://dx.doi.org/10.2196/40868 UR - http://www.ncbi.nlm.nih.gov/pubmed/38064633 ID - info:doi/10.2196/40868 ER - TY - JOUR AU - Abd-alrazaq, Alaa AU - Nashwan, J. Abdulqadir AU - Shah, Zubair AU - Abujaber, Ahmad AU - Alhuwail, Dari AU - Schneider, Jens AU - AlSaad, Rawan AU - Ali, Hazrat AU - Alomoush, Waleed AU - Ahmed, Arfan AU - Aziz, Sarah PY - 2024/3/5 TI - Machine Learning?Based Approach for Identifying Research Gaps: COVID-19 as a Case Study JO - JMIR Form Res SP - e49411 VL - 8 KW - research gaps KW - research gap KW - research topic KW - research topics KW - scientific literature KW - literature review KW - machine learning KW - COVID-19 KW - BERTopic KW - topic clustering KW - text analysis KW - BERT KW - NLP KW - natural language processing KW - review methods KW - review methodology KW - SARS-CoV-2 KW - coronavirus KW - COVID N2 - Background: Research gaps refer to unanswered questions in the existing body of knowledge, either due to a lack of studies or inconclusive results. Research gaps are essential starting points and motivation in scientific research. Traditional methods for identifying research gaps, such as literature reviews and expert opinions, can be time consuming, labor intensive, and prone to bias. They may also fall short when dealing with rapidly evolving or time-sensitive subjects. Thus, innovative scalable approaches are needed to identify research gaps, systematically assess the literature, and prioritize areas for further study in the topic of interest. Objective: In this paper, we propose a machine learning?based approach for identifying research gaps through the analysis of scientific literature. We used the COVID-19 pandemic as a case study. Methods: We conducted an analysis to identify research gaps in COVID-19 literature using the COVID-19 Open Research (CORD-19) data set, which comprises 1,121,433 papers related to the COVID-19 pandemic. Our approach is based on the BERTopic topic modeling technique, which leverages transformers and class-based term frequency-inverse document frequency to create dense clusters allowing for easily interpretable topics. Our BERTopic-based approach involves 3 stages: embedding documents, clustering documents (dimension reduction and clustering), and representing topics (generating candidates and maximizing candidate relevance). Results: After applying the study selection criteria, we included 33,206 abstracts in the analysis of this study. The final list of research gaps identified 21 different areas, which were grouped into 6 principal topics. These topics were: ?virus of COVID-19,? ?risk factors of COVID-19,? ?prevention of COVID-19,? ?treatment of COVID-19,? ?health care delivery during COVID-19,? ?and impact of COVID-19.? The most prominent topic, observed in over half of the analyzed studies, was ?the impact of COVID-19.? Conclusions: The proposed machine learning?based approach has the potential to identify research gaps in scientific literature. This study is not intended to replace individual literature research within a selected topic. Instead, it can serve as a guide to formulate precise literature search queries in specific areas associated with research questions that previous publications have earmarked for future exploration. Future research should leverage an up-to-date list of studies that are retrieved from the most common databases in the target area. When feasible, full texts or, at minimum, discussion sections should be analyzed rather than limiting their analysis to abstracts. Furthermore, future studies could evaluate more efficient modeling algorithms, especially those combining topic modeling with statistical uncertainty quantification, such as conformal prediction. UR - https://formative.jmir.org/2024/1/e49411 UR - http://dx.doi.org/10.2196/49411 UR - http://www.ncbi.nlm.nih.gov/pubmed/38441952 ID - info:doi/10.2196/49411 ER - TY - JOUR AU - Beatty, R. Jessica AU - Zelenak, Logan AU - Gillon, Spencer AU - McGoron, Lucy AU - Goyert, Gregory AU - Ondersma, J. Steven PY - 2024/3/4 TI - Risk Identification in Perinatal Health Care Settings via Technology-Based Recruitment Methods: Comparative Study JO - JMIR Form Res SP - e48823 VL - 8 KW - participant recruitment KW - engagement KW - health care screening KW - mobile phone N2 - Background: Digital screening and intervention tools have shown promise in the identification and reduction of substance use in health care settings. However, research in this area is impeded by challenges in integrating recruitment efforts into ongoing clinical workflows or staffing multiple study clinics with full-time research assistants, as well as by the underreporting of substance use. Objective: The aim of the study is to evaluate pragmatic methods for facilitating study recruitment in health care settings by examining recruitment rates and participant characteristics using in-person?based versus flyer approaches. Methods: This study compared recruitment rates at a Women?s Health clinic in the Midwest under 2 different recruitment strategies: in person versus via a flyer with a QR code. We also examined the disclosure of substance use and risk screener positivity for the 2 strategies. We also obtained information about the current use of technology and willingness to use it for study participation. Results: A greater percentage of patients recruited in person participated than those recruited via flyers (57/63, 91% vs 64/377, 17%). However, the final number recruited in each group was roughly equal (n=57 vs n=64). Additionally, participants recruited via flyers were more likely to screen positive for alcohol use risk on the Tolerance, Annoyed, Cut Down, Eye-Opener alcohol screen than those recruited at the clinic (24/64, 38% vs 11/57, 19%; ?21=4.9; P=.03). Participants recruited via flyers were also more likely to screen positive for drug use risk on the Wayne Indirect Drug Use Screener than those recruited at the clinic (20/64, 31% vs 9/57, 16%; ?21=4.0; P=.05). Furthermore, of the 121 pregnant women, 117 (96.7%) reported owning a smartphone, 111 (91.7%) had an SMS text message plan on their phone, and 94 (77.7%) reported being willing to receive SMS text messages or participate in a study if sent a link to their phone. Conclusions: The distribution of flyers with a QR code by medical staff appears to be an efficient and cost-effective method of recruitment that also facilitates disclosure while reducing the impact on clinic workflows. This method of recruitment can be useful for data collection at multiple locations and lead to larger samples across and between health systems. Participant recruitment via technology in perinatal health care appears to facilitate disclosure, particularly when participants can learn about the research and complete screening using their own device at a place and time convenient for them. Pregnant women in an urban Midwestern hospital had access to and were comfortable using technology. UR - https://formative.jmir.org/2024/1/e48823 UR - http://dx.doi.org/10.2196/48823 UR - http://www.ncbi.nlm.nih.gov/pubmed/38437004 ID - info:doi/10.2196/48823 ER - TY - JOUR AU - Worthington, A. Michelle AU - Christie, H. Richard AU - Masino, J. Aaron AU - Kark, M. Sarah PY - 2024/3/1 TI - Identifying Unmet Needs in Major Depressive Disorder Using a Computer-Assisted Alternative to Conventional Thematic Analysis: Qualitative Interview Study With Psychiatrists JO - JMIR Form Res SP - e48894 VL - 8 KW - consumer health informatics KW - interview KW - major depressive disorder KW - medical informatics applications KW - needs assessment KW - psychiatry and psychology N2 - Background: The development of digital health tools that are clinically relevant requires a deep understanding of the unmet needs of stakeholders, such as clinicians and patients. One way to reveal unforeseen stakeholder needs is through qualitative research, including stakeholder interviews. However, conventional qualitative data analytical approaches are time-consuming and resource-intensive, rendering them untenable in many industry settings where digital tools are conceived of and developed. Thus, a more time-efficient process for identifying clinically relevant target needs for digital tool development is needed. Objective: The objective of this study was to address the need for an accessible, simple, and time-efficient alternative to conventional thematic analysis of qualitative research data through text analysis of semistructured interview transcripts. In addition, we sought to identify important themes across expert psychiatrist advisor interview transcripts to efficiently reveal areas for the development of digital tools that target unmet clinical needs. Methods: We conducted 10 (1-hour-long) semistructured interviews with US-based psychiatrists treating major depressive disorder. The interviews were conducted using an interview guide that comprised open-ended questions predesigned to (1) understand the clinicians? experience of the care management process and (2) understand the clinicians? perceptions of the patients? experience of the care management process. We then implemented a hybrid analytical approach that combines computer-assisted text analyses with deductive analyses as an alternative to conventional qualitative thematic analysis to identify word combination frequencies, content categories, and broad themes characterizing unmet needs in the care management process. Results: Using this hybrid computer-assisted analytical approach, we were able to identify several key areas that are of interest to clinicians in the context of major depressive disorder and would be appropriate targets for digital tool development. Conclusions: A hybrid approach to qualitative research combining computer-assisted techniques with deductive techniques provides a time-efficient approach to identifying unmet needs, targets, and relevant themes to inform digital tool development. This can increase the likelihood that useful and practical tools are built and implemented to ultimately improve health outcomes for patients. UR - https://formative.jmir.org/2024/1/e48894 UR - http://dx.doi.org/10.2196/48894 UR - http://www.ncbi.nlm.nih.gov/pubmed/38427407 ID - info:doi/10.2196/48894 ER - TY - JOUR AU - Hashiguchi, Akiko AU - Asashima, Makoto AU - Takahashi, Satoru PY - 2024/2/28 TI - The Influence of Human Connections and Collaboration on Research Grant Success at Various Career Stages: Regression Analysis JO - JMIR Form Res SP - e49905 VL - 8 KW - biomedical researchers KW - grant success KW - human connection KW - peer researchers KW - synergistic collaborations KW - research development N2 - Background: Documenting the grant acquisition characteristics of a highly selective group of researchers could provide insights into the research and faculty development of talented individuals, and the insights gained to foster such researchers will help university management strengthen their research capacity. Objective: This study examines the role of human connections in the success of biomedical researchers in Japanese universities. Methods: This study used grant data from the Grants-in-Aid for Scientific Research (GIA) program, the largest competitive research funding program in Japan, to collect information on projects and their implementation systems obtained throughout the participants? careers. Grant success was measured by the number and amounts of the awards obtained while participants occupied the role of principal investigator. Human connections were quantified by the number of projects in which the participants took part as members and were classified by their relationship with the project leader. Data were matched with information on career history, publication performance, and experience of the participants with government-funded programs apart from GIA and were analyzed using univariate and multivariate regression analyses. Results: Early-career interpersonal relationships, as measured using the h-index value of the researchers who provided the participants with their initial experience as project members, had a positive effect on grant success. The experience of contributing to prestigious research programs led by top researchers dramatically increased the cumulative amount of GIA awards received by the participants over time. Univariate logistic regression analyses revealed that more interactions with upper-level researchers resulted in fewer acquisitions of large programs (odds ratio [OR] 0.67, 95% CI 0.50-0.89). Collaboration with peers increased the success rate of ?2 research grants in large programs in situations in which both the participant and project leader were professors (OR 1.16, 95% CI 1.06-1.26). Tracking the process of research development, we found that collaboration during the periods of 10 to 14 years and 15 to 19 years after completing a doctorate degree determined the size of the project that the participant would obtain?interactions with peer researchers and subordinates during the 10- to 14-year postdegree period had positive effects on ?2 large-program acquisitions (OR 1.51, 95% CI 1.09-2.09 and OR 1.31, 95% CI 1.10-1.57, respectively), whereas interactions with subordinates during the 15- to 19-year postdegree period also had positive effects (OR 1.25, 95% CI 1.06-1.47). Furthermore, relationships that remained narrowly focused resulted in limited grant success for small programs. Conclusions: Human networking is important for improving an individual?s ability to obtain external funding. The results emphasize the importance of having a high-h-indexed collaborator to obtain quality information early in one?s career; working with diverse, nonsupervisory personnel at the midcareer stage; and engaging in synergistic collaborations upon establishing a research area in which one can take more initiatives. UR - https://formative.jmir.org/2024/1/e49905 UR - http://dx.doi.org/10.2196/49905 UR - http://www.ncbi.nlm.nih.gov/pubmed/38416548 ID - info:doi/10.2196/49905 ER - TY - JOUR AU - Luken, Amanda AU - Rabinowitz, A. Jill AU - Wells, L. Jonathan AU - Sosnowski, W. David AU - Strickland, C. Justin AU - Thrul, Johannes AU - Kirk, D. Gregory AU - Maher, S. Brion PY - 2024/2/27 TI - Designing and Validating a Novel Method for Assessing Delay Discounting Associated With Health Behaviors: Ecological Momentary Assessment Study JO - JMIR Form Res SP - e48954 VL - 8 KW - delay discounting KW - measurement KW - Monetary Choice Questionnaire KW - ecological momentary assessment KW - substance use KW - substance abuse KW - questionnaire KW - validity KW - validation KW - monetary KW - reward KW - rewards KW - survey KW - mobile phone N2 - Background: Delay discounting quantifies an individual?s preference for smaller, short-term rewards over larger, long-term rewards and represents a transdiagnostic factor associated with numerous adverse health outcomes. Rather than a fixed trait, delay discounting may vary over time and place, influenced by individual and contextual factors. Continuous, real-time measurement could inform adaptive interventions for various health conditions. Objective: The goals of this paper are 2-fold. First, we present and validate a novel, short, ecological momentary assessment (EMA)?based delay discounting scale we developed. Second, we assess this tool?s ability to reproduce known associations between delay discounting and health behaviors (ie, substance use and craving) using a convenience-based sample. Methods: Participants (N=97) were adults (age range 18-71 years), recruited on social media. In phase 1, data were collected on participant sociodemographic characteristics, and delay discounting was evaluated via the traditional Monetary Choice Questionnaire (MCQ) and our novel method (ie, 7-item time-selection and 7-item monetary-selection scales). During phase 2 (approximately 6 months later), participants completed the MCQ, our novel delay discounting measures, and health outcomes questions. The correlations between our method and the traditional MCQ within and across phases were examined. For scale reduction, a random number of items were iteratively selected, and the correlation between the full and random scales was assessed. We then examined the association between our time- and monetary-selection scales assessed during phase 2 and the percentage of assessments that participants endorsed using or craving alcohol, tobacco, or cannabis. Results: In total, 6 of the 7 individual time-selection items were highly correlated with the full scale (r>0.89). Both time-selection (r=0.71; P<.001) and monetary-selection (r=0.66; P<.001) delay discounting rates had high test-retest reliability across phases 1 and 2. Phase 1 MCQ delay discounting function highly correlated with phase 1 (r=0.76; P<.001) and phase 2 (r=0.45; P<.001) time-selection delay discounting scales. One or more randomly chosen time-selection items were highly correlated with the full scale (r>0.94). Greater delay discounting measured via the time-selection measure (adjusted mean difference=5.89, 95% CI 1.99-9.79), but not the monetary-selection scale (adjusted mean difference=?0.62, 95% CI ?3.57 to 2.32), was associated with more past-hour tobacco use endorsement in follow-up surveys. Conclusions: This study evaluated a novel EMA-based scale?s ability to validly and reliably assess delay discounting. By measuring delay discounting with fewer items and in situ via EMA in natural environments, researchers may be better able to identify individuals at risk for poor health outcomes. UR - https://formative.jmir.org/2024/1/e48954 UR - http://dx.doi.org/10.2196/48954 UR - http://www.ncbi.nlm.nih.gov/pubmed/38412027 ID - info:doi/10.2196/48954 ER - TY - JOUR AU - Virto, Naiara AU - Río, Xabier AU - Angulo-Garay, Garazi AU - García Molina, Rafael AU - Avendaño Céspedes, Almudena AU - Cortés Zamora, Belen Elisa AU - Gómez Jiménez, Elena AU - Alcantud Córcoles, Ruben AU - Rodriguez Mañas, Leocadio AU - Costa-Grille, Alba AU - Matheu, Ander AU - Marcos-Pérez, Diego AU - Lazcano, Uxue AU - Vergara, Itziar AU - Arjona, Laura AU - Saeteros, Morelva AU - Lopez-de-Ipiña, Diego AU - Coca, Aitor AU - Abizanda Soler, Pedro AU - Sanabria, J. Sergio PY - 2024/2/23 TI - Development of Continuous Assessment of Muscle Quality and Frailty in Older Patients Using Multiparametric Combinations of Ultrasound and Blood Biomarkers: Protocol for the ECOFRAIL Study JO - JMIR Res Protoc SP - e50325 VL - 13 KW - muscle KW - ultrasound KW - blood-based biomarkers KW - sarcopenia KW - frailty KW - older adults N2 - Background: Frailty resulting from the loss of muscle quality can potentially be delayed through early detection and physical exercise interventions. There is a demand for cost-effective tools for the objective evaluation of muscle quality, in both cross-sectional and longitudinal assessments. Literature suggests that quantitative analysis of ultrasound data captures morphometric, compositional, and microstructural muscle properties, while biological assays derived from blood samples are associated with functional information. Objective: This study aims to assess multiparametric combinations of ultrasound and blood-based biomarkers to offer a cross-sectional evaluation of the patient frailty phenotype and to track changes in muscle quality associated with supervised exercise programs. Methods: This prospective observational multicenter study will include patients aged 70 years and older who are capable of providing informed consent. We aim to recruit 100 patients from hospital environments and 100 from primary care facilities. Each patient will undergo at least two examinations (baseline and follow-up), totaling a minimum of 400 examinations. In hospital environments, 50 patients will be measured before/after a 16-week individualized and supervised exercise program, while another 50 patients will be followed up after the same period without intervention. Primary care patients will undergo a 1-year follow-up evaluation. The primary objective is to compare cross-sectional evaluations of physical performance, functional capacity, body composition, and derived scales of sarcopenia and frailty with biomarker combinations obtained from muscle ultrasound and blood-based assays. We will analyze ultrasound raw data obtained with a point-of-care device, along with a set of biomarkers previously associated with frailty, using quantitative real-time polymerase chain reaction and enzyme-linked immunosorbent assay. Additionally, we will examine the sensitivity of these biomarkers to detect short-term muscle quality changes and functional improvement after a supervised exercise intervention compared with usual care. Results: At the time of manuscript submission, the enrollment of volunteers is ongoing. Recruitment started on March 1, 2022, and ends on June 30, 2024. Conclusions: The outlined study protocol will integrate portable technologies, using quantitative muscle ultrasound and blood biomarkers, to facilitate an objective cross-sectional assessment of muscle quality in both hospital and primary care settings. The primary objective is to generate data that can be used to explore associations between biomarker combinations and the cross-sectional clinical assessment of frailty and sarcopenia. Additionally, the study aims to investigate musculoskeletal changes following multicomponent physical exercise programs. Trial Registration: ClinicalTrials.gov NCT05294757; https://clinicaltrials.gov/ct2/show/NCT05294757 International Registered Report Identifier (IRRID): DERR1-10.2196/50325 UR - https://www.researchprotocols.org/2024/1/e50325 UR - http://dx.doi.org/10.2196/50325 UR - http://www.ncbi.nlm.nih.gov/pubmed/38393761 ID - info:doi/10.2196/50325 ER - TY - JOUR AU - Tegegne, Kassaw Teketo AU - Tran, Ly-Duyen AU - Nourse, Rebecca AU - Gurrin, Cathal AU - Maddison, Ralph PY - 2024/2/21 TI - Daily Activity Lifelogs of People With Heart Failure: Observational Study JO - JMIR Form Res SP - e51248 VL - 8 KW - heart failure KW - self-management KW - lifelogs KW - daily activity KW - wearable camera KW - E-Myscéal KW - activities of daily living KW - ADL KW - intervention KW - self-report method KW - wearable KW - chronic condition N2 - Background: Globally, heart failure (HF) affects more than 64 million people, and attempts to reduce its social and economic burden are a public health priority. Interventions to support people with HF to self-manage have been shown to reduce hospitalizations, improve quality of life, and reduce mortality rates. Understanding how people self-manage is imperative to improve future interventions; however, most approaches to date, have used self-report methods to achieve this. Wearable cameras provide a unique tool to understand the lived experiences of people with HF and the daily activities they undertake, which could lead to more effective interventions. However, their potential for understanding chronic conditions such as HF is unclear. Objective: This study aimed to determine the potential utility of wearable cameras to better understand the activities of daily living in people living with HF. Methods: The ?Seeing is Believing (SIB)? study involved 30 patients with HF who wore wearable cameras for a maximum of 30 days. We used the E-Myscéal web-based lifelog retrieval system to process and analyze the wearable camera image data set. Search terms for 7 daily activities (physical activity, gardening, shopping, screen time, drinking, eating, and medication intake) were developed and used for image retrieval. Sensitivity analysis was conducted to compare the number of images retrieved using different search terms. Temporal patterns in daily activities were examined, and differences before and after hospitalization were assessed. Results: E-Myscéal exhibited sensitivity to specific search terms, leading to significant variations in the number of images retrieved for each activity. The highest number of images returned were related to eating and drinking, with fewer images for physical activity, screen time, and taking medication. The majority of captured activities occurred before midday. Notably, temporal differences in daily activity patterns were observed for participants hospitalized during this study. The number of medication images increased after hospital discharge, while screen time images decreased. Conclusions: Wearable cameras offer valuable insights into daily activities and self-management in people living with HF. E-Myscéal efficiently retrieves relevant images, but search term sensitivity underscores the need for careful selection. UR - https://formative.jmir.org/2024/1/e51248 UR - http://dx.doi.org/10.2196/51248 UR - http://www.ncbi.nlm.nih.gov/pubmed/38381484 ID - info:doi/10.2196/51248 ER - TY - JOUR AU - Williams, Hants AU - Steinberg, Sarah AU - Leon, Kendall AU - Vingum, Ryan AU - Hu, Mengyao AU - Berzin, Robin AU - Hagg, Heather AU - Hanaway, Patrick PY - 2024/2/16 TI - Predictive Criterion Validity of the Parsley Symptom Index Against the Patient-Reported Outcomes Measurement Information System-10 in a Chronic Disease Cohort: Retrospective Cohort Study JO - JMIR Form Res SP - e53316 VL - 8 KW - chronic disease KW - eHealth KW - ePROM KW - mHealth KW - Parsley Symptom Index KW - patient-reported outcome measure KW - PROM KW - PSI KW - telehealth KW - telemedicine KW - validation KW - web-based N2 - Background: Approximately 60% of US adults live with chronic disease, imposing a significant burden on patients and the health care system. With the rise of telehealth, patient-reported outcomes measures (PROMs) have emerged as pivotal tools for managing chronic disease. While numerous PROMs exist, few have been designed explicitly for telehealth settings. The Parsley Symptom Index (PSI) is an electronic patient-reported outcome measure (ePROM) developed specifically for telehealth environments. Objective: Our aim is to determine whether the PSI predicts changes in the established Patient-Reported Outcomes Measurement Information System-10 (PROMIS-10) Global Health, a 10-question short form. Methods: We conducted a retrospective cohort study using data from 367 unique patients, amassing 1170 observations between August 30, 2017, and January 30, 2023. Patients completed the PSI and the PROMIS-10 multiple times throughout the study period. Using univariate regression models, we assess the predictive criterion validity of the PSI against PROMIS-10 scores. Results: This study revealed significant relationships between the PSI and PROMIS-10 physical and mental health scores through comprehensive univariate analyses, thus establishing support for the criterion validity of the PSI. These analyses highlighted the PSI?s potential as an insightful tool for understanding and predicting both mental and physical health dimensions. Conclusions: Our findings emphasize the importance of the PSI in capturing the nuanced interactions between symptomatology and health outcomes. These insights reinforce the value of the PSI in clinical contexts and support its potential as a versatile tool in both research and practice. UR - https://formative.jmir.org/2024/1/e53316 UR - http://dx.doi.org/10.2196/53316 UR - http://www.ncbi.nlm.nih.gov/pubmed/38363587 ID - info:doi/10.2196/53316 ER - TY - JOUR AU - Yordanov, Stefan AU - Akhter, Kalsoom AU - Quan Teh, Jye AU - Naushahi, Jawad AU - Jalloh, Ibrahim PY - 2024/2/14 TI - Measurement of Head Circumference Using a Smartphone: Feasibility Cohort Study JO - JMIR Form Res SP - e54194 VL - 8 KW - head circumference KW - HC KW - hydrocephalus KW - neurosurgery KW - pediatric neurosurgery KW - paediatric neurosurgery KW - neurology KW - neuro KW - neurosurgeon KW - neurologist KW - mobile health KW - mHealth KW - app KW - apps KW - application KW - applications KW - digital health KW - smartphone KW - smartphones KW - pediatric KW - pediatrics KW - paediatric KW - paediatrics KW - infant KW - infants KW - infancy KW - baby KW - babies KW - neonate KW - neonates KW - neonatal KW - toddler KW - toddlers KW - child KW - children N2 - Background: Accurate head circumference (HC) measurement is essential when assessing neonates and infants. Tape measure HC measurements are prone to errors, particularly when performed by parents/guardians, due to individual differences in head shape, hair style and texture, subject cooperation, and examiner techniques, including tape measure placement and tautness. There is, therefore, the need for a more reliable method. Objective: The primary objective of this study was to evaluate the validity, reliability, and consistency of HC app measurement compared to the current standard of practice, serving as a proof-of-concept for use by health care professionals. Methods: We recruited infants attending the neurosurgery clinic, and parents/guardians were approached and consented to participate in the study. Along with the standard head circumference measurement, measurements were taken with the head circumference app (HC app) developed in-house, and we also collected baseline medical history and characteristics. For the statistical analysis, we used RStudio (version 4.1.1). In summary, we analyzed covariance and intraclass correlation coefficient (ICC) to compare the measurement's within-rater and interrater reliability. The F test was used to analyze the variance between measurements and the Bland-Altman agreement, t test, and correlation coefficients were used to compare the tape measurement to the measures taken by the HC app. We also used nonvalidated questionnaires to explore parental or guardians? experiences, assess their views on app utility, and collect feedback. Results: The total number of recruited patients was 37. Comparison between the app measurements and the measurements with a tape measure showed poor reliability (ICC=0.177) and wide within-app variations (ICC=0.341). The agreement between the measurements done by parents/guardians and the tape measurements done by the researcher was good (ICC=0.901). Parental/guardian feedback was overall very positive, with most of the parents/guardians reporting that the app was easy to use (n=31, 84%) and that they are happy to use the app in an unsupervised setting, provided that they are assured of the measurement quality. Conclusions: We developed this project as a proof-of-concept study, and as such, the app has shown great potential to be used both in a clinical setting and by parents/guardians in their own homes. UR - https://formative.jmir.org/2024/1/e54194 UR - http://dx.doi.org/10.2196/54194 UR - http://www.ncbi.nlm.nih.gov/pubmed/38354022 ID - info:doi/10.2196/54194 ER - TY - JOUR AU - Gagnon, Julie AU - Probst, Sebastian AU - Chartrand, Julie AU - Lalonde, Michelle PY - 2024/2/13 TI - mHealth App Usability Questionnaire for Stand-Alone mHealth Apps Used by Health Care Providers: Canadian French Translation, Cross-Cultural Adaptation, and Validation (Part 1) JO - JMIR Form Res SP - e50839 VL - 8 KW - cross-cultural adaptation KW - French language KW - mHealth App Usability Questionnaire KW - MAUQ KW - mobile health KW - mHealth KW - mobile app KW - questionnaire translation KW - usability KW - validation KW - health care providers KW - French translation N2 - Background: An increasing number of health care professionals are using mobile apps. The mHealth App Usability Questionnaire (MAUQ) was designed to evaluate the usability of mobile health apps by patients and providers. However, this questionnaire is not available in French. Objective: This study aims to translate (from English to Canadian French), cross-culturally adapt, and initiate the validation of the original version of MAUQ for stand-alone mobile health apps used by French-speaking health care providers. Methods: A cross-cultural research study using a well-established method was conducted to translate MAUQ to Canadian French by certified translators and subsequently review it with a translation committee. It was then back translated to English. The back translations were compared with the original by the members of the committee to reach consensus regarding the prefinal version. A pilot test of the prefinal version was conducted with a sample of 49 potential users and 10 experts for content validation. Results: The statements are considered clear, with interrater agreement of 99.14% among potential users and 90% among experts. Of 21 statements, 5 (24%) did not exceed the 80% interrater agreement of the experts regarding clarity. Following the revisions, interrater agreement exceeded 80%. The content validity index of the items varied from 0.90 to 1, and the overall content validity index was 0.981. Individual Fleiss multirater ? of each item was between 0.89 and 1, showing excellent agreement and increasing confidence in the questionnaire?s content validity. Conclusions: This process of translation and cultural adaptation produced a new version of MAUQ that was validated for later use among the Canadian French?speaking population. An upcoming separate study will investigate the psychometric properties of the adapted questionnaire. UR - https://formative.jmir.org/2024/1/e50839 UR - http://dx.doi.org/10.2196/50839 UR - http://www.ncbi.nlm.nih.gov/pubmed/38349710 ID - info:doi/10.2196/50839 ER - TY - JOUR AU - Kearney, E. Lauren AU - Jansen, Emily AU - Kathuria, Hasmeena AU - Steiling, Katrina AU - Jones, C. Kayla AU - Walkey, Allan AU - Cordella, Nicholas PY - 2024/2/9 TI - Efficacy of Digital Outreach Strategies for Collecting Smoking Data: Pragmatic Randomized Trial JO - JMIR Form Res SP - e50465 VL - 8 KW - electronic health records KW - EHR KW - informatics KW - learning health system KW - lung cancer screening KW - smoking history N2 - Background: Tobacco smoking is an important risk factor for disease, but inaccurate smoking history data in the electronic medical record (EMR) limits the reach of lung cancer screening (LCS) and tobacco cessation interventions. Patient-generated health data is a novel approach to documenting smoking history; however, the comparative effectiveness of different approaches is unclear. Objective: We designed a quality improvement intervention to evaluate the effectiveness of portal questionnaires compared to SMS text message?based surveys, to compare message frames, and to evaluate the completeness of patient-generated smoking histories. Methods: We randomly assigned patients aged between 50 and 80 years with a history of tobacco use who identified English as a preferred language and have never undergone LCS to receive an EMR portal questionnaire or a text survey. The portal questionnaire used a ?helpfulness? message, while the text survey tested frame types informed by behavior economics (?gain,? ?loss,? and ?helpfulness?) and nudge messaging. The primary outcome was the response rate for each modality and framing type. Completeness and consistency with documented structured smoking data were also evaluated. Results: Participants were more likely to respond to the text survey (191/1000, 19.1%) compared to the portal questionnaire (35/504, 6.9%). Across all text survey rounds, patients were less responsive to the ?helpfulness? frame compared with the ?gain? frame (odds ratio [OR] 0.29, 95% CI 0.09-0.91; P<.05) and ?loss? frame (OR 0.32, 95% CI 11.8-99.4; P<.05). Compared to the structured data in the EMR, the patient-generated data were significantly more likely to be complete enough to determine LCS eligibility both compared to the portal questionnaire (OR 34.2, 95% CI 3.8-11.1; P<.05) and to the text survey (OR 6.8, 95% CI 3.8-11.1; P<.05). Conclusions: We found that an approach using patient-generated data is a feasible way to engage patients and collect complete smoking histories. Patients are likely to respond to a text survey using ?gain? or ?loss? framing to report detailed smoking histories. Optimizing an SMS text message approach to collect medical information has implications for preventative and follow-up clinical care beyond smoking histories, LCS, and smoking cessation therapy. UR - https://formative.jmir.org/2024/1/e50465 UR - http://dx.doi.org/10.2196/50465 UR - http://www.ncbi.nlm.nih.gov/pubmed/38335012 ID - info:doi/10.2196/50465 ER - TY - JOUR AU - Wang, Lei AU - Bi, Wenshuai AU - Zhao, Suling AU - Ma, Yinyao AU - Lv, Longting AU - Meng, Chenwei AU - Fu, Jingru AU - Lv, Hanlin PY - 2024/2/8 TI - Investigating the Impact of Prompt Engineering on the Performance of Large Language Models for Standardizing Obstetric Diagnosis Text: Comparative Study JO - JMIR Form Res SP - e53216 VL - 8 KW - obstetric data KW - similarity embedding KW - term standardization KW - large language models KW - LLMs N2 - Background: The accumulation of vast electronic medical records (EMRs) through medical informatization creates significant research value, particularly in obstetrics. Diagnostic standardization across different health care institutions and regions is vital for medical data analysis. Large language models (LLMs) have been extensively used for various medical tasks. Prompt engineering is key to use LLMs effectively. Objective: This study aims to evaluate and compare the performance of LLMs with various prompt engineering techniques on the task of standardizing obstetric diagnostic terminology using real-world obstetric data. Methods: The paper describes a 4-step approach used for mapping diagnoses in electronic medical records to the International Classification of Diseases, 10th revision, observation domain. First, similarity measures were used for mapping the diagnoses. Second, candidate mapping terms were collected based on similarity scores above a threshold, to be used as the training data set. For generating optimal mapping terms, we used two LLMs (ChatGLM2 and Qwen-14B-Chat [QWEN]) for zero-shot learning in step 3. Finally, a performance comparison was conducted by using 3 pretrained bidirectional encoder representations from transformers (BERTs), including BERT, whole word masking BERT, and momentum contrastive learning with BERT (MC-BERT), for unsupervised optimal mapping term generation in the fourth step. Results: LLMs and BERT demonstrated comparable performance at their respective optimal levels. LLMs showed clear advantages in terms of performance and efficiency in unsupervised settings. Interestingly, the performance of the LLMs varied significantly across different prompt engineering setups. For instance, when applying the self-consistency approach in QWEN, the F1-score improved by 5%, with precision increasing by 7.9%, outperforming the zero-shot method. Likewise, ChatGLM2 delivered similar rates of accurately generated responses. During the analysis, the BERT series served as a comparative model with comparable results. Among the 3 models, MC-BERT demonstrated the highest level of performance. However, the differences among the versions of BERT in this study were relatively insignificant. Conclusions: After applying LLMs to standardize diagnoses and designing 4 different prompts, we compared the results to those generated by the BERT model. Our findings indicate that QWEN prompts largely outperformed the other prompts, with precision comparable to that of the BERT model. These results demonstrate the potential of unsupervised approaches in improving the efficiency of aligning diagnostic terms in daily research and uncovering hidden information values in patient data. UR - https://formative.jmir.org/2024/1/e53216 UR - http://dx.doi.org/10.2196/53216 UR - http://www.ncbi.nlm.nih.gov/pubmed/38329787 ID - info:doi/10.2196/53216 ER - TY - JOUR AU - Bjelkarøy, Torheim Maria AU - Simonsen, Breines Tone AU - Siddiqui, Ghazal Tahreem AU - Halset, Sigrid AU - Cheng, Socheat AU - Grambaite, Ramune AU - Benth, ?altyt? J?rat? AU - Gerwing, Jennifer AU - Kristoffersen, Saxhaug Espen AU - Lundqvist, Christofer PY - 2024/2/8 TI - Brief Intervention as a Method to Reduce Z-Hypnotic Use by Older Adults: Feasibility Case Series JO - JMIR Form Res SP - e51862 VL - 8 KW - prescription medication misuse KW - older adults KW - brief intervention KW - z-drugs KW - benzodiazepine-related drugs KW - BZD-related drugs KW - z-hypnotic KW - intervention KW - feasibility KW - case series KW - insomnia KW - sleep KW - substance overuse KW - older adult KW - treatment KW - reduction KW - benzodiazepine KW - hypnotics N2 - Background: Z-hypnotics or z-drugs are commonly prescribed for insomnia and sleep difficulties in older adults. These drugs are associated with adverse events and dependence and are not recommended for long-term use. Despite evidence of older adults being more sensitive to a wide array of adverse events and clinical guidelines advocating limiting use, inappropriate use in this population is still prevalent. Previous intervention studies have focused mainly on prescriber information. Simple, individually focused intervention designs are less studied. Brief intervention (BI) is a simple, easily transferable method mainly used to treat patients at risk of alcohol overuse. Objective: Our objective was to design and test the feasibility and acceptability of a BI intervention adapted to address individual, inappropriate use of z-hypnotics among older adults. This preparatory study aimed to optimize the intervention in advance of a quantitative randomized controlled trial investigating the treatment effect in a larger population. Methods: This feasibility case series was conducted at Akershus University Hospital, Norway, in autumn 2021. We included 5 adults aged ?65 years with long-term (?4 weeks) use of z-hypnotics and 2 intervening physicians. Additionally, 2 study investigators contributed with process evaluation notes. The BI consists of information on the risk of inappropriate use and individualized advice on how to reduce use. The focus of the intervention is behavioral and aims, in cooperation with the patient and based on shared decision-making, to change patient behavior regarding sleep medication rather than physician-based detoxification and termination of z-hypnotic prescriptions. Qualitative and descriptive quantitative data were collected from intervening physicians, study investigators, and participants at baseline, immediately after the intervention, and at the 6-week follow-up. Results: Data were obtained from 2 physicians, 2 study investigators, and 5 participants (4 women) with a median age of 84 years. The average time spent on the BI consultation was 15 minutes. All 5 participants completed the intervention without problems. The participants and 2 intervening physicians reported the intervention as acceptable and were satisfied with the delivery of the intervention. After the intervention, 2 participants stopped their use of z-hypnotics completely and participated in the follow-up interview. Study investigators identified logistical challenges regarding location and time requirements. Identified aspects that may improve the intervention and reduce dropouts included revising the intervention content, focusing on rebound insomnia, adding an information leaflet, and supporting the patient in the period between the intervention and follow-up. The notion that the intervention should best be located and conducted by the patient?s own general practitioner was supported by the participants. Conclusions: We identified important aspects to improve the designed intervention and found that the BI is feasible and acceptable for incorporation into a larger randomized trial investigating the treatment effect of BI for reducing z-hypnotic use by older adults. Trial Registration: ClinicalTrials.gov NCT03162081; http://tinyurl.com/rmzx6brn UR - https://formative.jmir.org/2024/1/e51862 UR - http://dx.doi.org/10.2196/51862 UR - http://www.ncbi.nlm.nih.gov/pubmed/38329779 ID - info:doi/10.2196/51862 ER - TY - JOUR AU - Jones, Catherine AU - Chandarana, Shikha AU - Vyas, Amita AU - Napolitano, Melissa PY - 2024/2/6 TI - Attitudes, Barriers, and Motivators Toward Daily Walking and a Mobile App to Increase Walking Among Women: Web-Based Anonymous Survey JO - JMIR Form Res SP - e48668 VL - 8 KW - mHealth KW - mobile health KW - mobile app KW - walking KW - physical activity KW - step counts KW - women?s health KW - age KW - wearable activity tracker KW - chronic disease KW - mental health KW - mobile phone KW - COVID-19 N2 - Background: There are disparities in the prevalence of physical activity (PA) with women engaging in less PA than men, a gap which widens during midlife. Walking is a generally accepted form of PA among women and should be encouraged. Motivations, barriers, and attitudes to engaging in walking change with age, but the influencing factors are not well understood nor are the features of mobile apps that facilitate daily walking. Objective: This study explores the relationship between age and women?s self-reported motivations, barriers, attitudes, and beliefs toward daily walking. It further assesses attitudes toward features of a mobile app designed to sync with a wearable step tracker to increase and maintain levels of daily walking among women. Methods: A web-based anonymous survey was completed by 400 women, aged 21-75 years. The 31-item survey captured women?s perceived barriers and motivators toward daily walking and attitudes toward mobile apps to support and maintain daily walking. For analysis, responses to the survey were grouped into 2 categories of women: ages 21-49 years and ages 50-75 years. Bivariate analyses were conducted through SPSS (IBM Corp) for each of the survey questions using chi-square for dichotomous variables and 1-tailed t tests for scales and continuous variables to identify significant differences between the groups. One-tailed t tests were run for scaled variables to identify significant differences between the 10-year age increments. Results: Significant barriers to daily walking were observed in the 21-49?year group for personal and work responsibilities, motivational and psychosocial factors, and physical and environmental factors. Motivators to walk daily in the 21- 49?year group were significantly higher to reduce stress and anxiety, and motivators to walk daily in the 50-75?year group were significantly higher to help manage or lose weight and to reduce the risk of chronic illness. Women?s walking preferences, beliefs around their walking behaviors, and their perceived importance of the features of a future mobile app for walking designed specifically for women showed significant variation according to age. When asked about the importance of features for a mobile app, women aged 21-49 years indicated a significantly higher number of positive responses for the following features: digital community support, rewards or point system, and seeing a daily or weekly or monthly progress chart. Conclusions: Our findings indicate that barriers, motivators, and beliefs around daily walking and the importance of preferred features of a mobile app vary according to women?s ages. Messaging and app features should be tailored to different age groups of women. These study results can be viewed as a foundation for future research and development of mobile health interventions to effectively increase daily walking among women of all ages. UR - https://formative.jmir.org/2024/1/e48668 UR - http://dx.doi.org/10.2196/48668 UR - http://www.ncbi.nlm.nih.gov/pubmed/38319695 ID - info:doi/10.2196/48668 ER - TY - JOUR AU - Heinze, Clara AU - Hartmeyer, Dalgaard Rikke AU - Sidenius, Anne AU - Ringgaard, Winther Lene AU - Bjerregaard, Anne-Louise AU - Krølner, Fredenslund Rikke AU - Allender, Steven AU - Bauman, Adrian AU - Klinker, Demant Charlotte PY - 2024/2/6 TI - Developing and Evaluating a Data-Driven and Systems Approach to Health Promotion Among Vocational Students: Protocol for the Data Health Study JO - JMIR Res Protoc SP - e52571 VL - 13 KW - health promotion KW - health behavior KW - well-being KW - organizational readiness KW - cocreation KW - causal loop diagram KW - systems thinking KW - systems-based evaluation KW - vocational schools KW - youth N2 - Background: Vocational school students exhibit significant risk behaviors in terms of poor diet, frequent use of nicotine products, inadequate fruit and vegetable intake, low levels of physical activity, and poor mental health. This makes vocational students vulnerable to the development of noncommunicable diseases. Therefore, effective health promotion programs targeting vocational students are required. Objective: The Danish study ?Data-driven and Systems Approach to Health Promotion Among Vocational Students? (Data Health) aims to develop, implement, and evaluate a systems approach to support vocational schools, municipalities, and local communities in implementing locally relevant health promotion actions among and for vocational students. This paper describes the Data Health program and how implementation and preliminary effectiveness will be evaluated. Methods: The Data Health program offers an iterative 5-step process to develop changes in the systems that shape health behavior and well-being among vocational students. The program will be implemented and evaluated in 8 Danish vocational schools in 4 municipalities. The implementation of the process and actions will be explored using a systems-based evaluation design that assesses contextual differences and the mechanisms through which the program leads to changes in the systems. Preliminary effectiveness at the individual level (students? self-reported health behavior and well-being) and organizational level (school organizational readiness reported by school staff) will be assessed using a quasi-experimental design, and cross-sectional data will be collected at all 8 schools simultaneously 4 times during the 2-year study period. Results: This study was launched in 2021, and data collection is expected to be completed in June 2024. The first results are expected to be submitted for publication in January 2024. Conclusions: We expect that the Data Health study will make significant contributions to complex intervention research by contributing to the paucity of research studies that have used systems approaches in school settings. The study will also provide evidence of successful elements for systems change and effectiveness to determine whether a national scale-up can be recommended. Trial Registration: ClinicalTrials.gov NCT05308459; https://clinicaltrials.gov/study/NCT05308459 International Registered Report Identifier (IRRID): DERR1-10.2196/52571 UR - https://www.researchprotocols.org/2024/1/e52571 UR - http://dx.doi.org/10.2196/52571 UR - http://www.ncbi.nlm.nih.gov/pubmed/38319698 ID - info:doi/10.2196/52571 ER - TY - JOUR AU - Ahmadzia, Khorrami Homa AU - Dzienny, C. Alexa AU - Bopf, Mike AU - Phillips, M. Jaclyn AU - Federspiel, Jeffrey Jerome AU - Amdur, Richard AU - Rice, Murguia Madeline AU - Rodriguez, Laritza PY - 2024/2/5 TI - Machine Learning Models for Prediction of Maternal Hemorrhage and Transfusion: Model Development Study JO - JMIR Bioinform Biotech SP - e52059 VL - 5 KW - postpartum hemorrhage KW - machine learning KW - prediction KW - maternal KW - predict KW - predictive KW - bleeding KW - hemorrhage KW - hemorrhaging KW - birth KW - postnatal KW - blood KW - transfusion KW - antepartum KW - obstetric KW - obstetrics KW - women's health KW - gynecology KW - gynecological N2 - Background: Current postpartum hemorrhage (PPH) risk stratification is based on traditional statistical models or expert opinion. Machine learning could optimize PPH prediction by allowing for more complex modeling. Objective: We sought to improve PPH prediction and compare machine learning and traditional statistical methods. Methods: We developed models using the Consortium for Safe Labor data set (2002-2008) from 12 US hospitals. The primary outcome was a transfusion of blood products or PPH (estimated blood loss of ?1000 mL). The secondary outcome was a transfusion of any blood product. Fifty antepartum and intrapartum characteristics and hospital characteristics were included. Logistic regression, support vector machines, multilayer perceptron, random forest, and gradient boosting (GB) were used to generate prediction models. The area under the receiver operating characteristic curve (ROC-AUC) and area under the precision/recall curve (PR-AUC) were used to compare performance. Results: Among 228,438 births, 5760 (3.1%) women had a postpartum hemorrhage, 5170 (2.8%) had a transfusion, and 10,344 (5.6%) met the criteria for the transfusion-PPH composite. Models predicting the transfusion-PPH composite using antepartum and intrapartum features had the best positive predictive values, with the GB machine learning model performing best overall (ROC-AUC=0.833, 95% CI 0.828-0.838; PR-AUC=0.210, 95% CI 0.201-0.220). The most predictive features in the GB model predicting the transfusion-PPH composite were the mode of delivery, oxytocin incremental dose for labor (mU/minute), intrapartum tocolytic use, presence of anesthesia nurse, and hospital type. Conclusions: Machine learning offers higher discriminability than logistic regression in predicting PPH. The Consortium for Safe Labor data set may not be optimal for analyzing risk due to strong subgroup effects, which decreases accuracy and limits generalizability. UR - https://bioinform.jmir.org/2024/1/e52059 UR - http://dx.doi.org/10.2196/52059 UR - http://www.ncbi.nlm.nih.gov/pubmed/38935950 ID - info:doi/10.2196/52059 ER - TY - JOUR AU - Recsky, Chantelle AU - Rush, L. Kathy AU - MacPhee, Maura AU - Stowe, Megan AU - Blackburn, Lorraine AU - Muniak, Allison AU - Currie, M. Leanne PY - 2024/2/5 TI - Clinical Informatics Team Members? Perspectives on Health Information Technology Safety After Experiential Learning and Safety Process Development: Qualitative Descriptive Study JO - JMIR Form Res SP - e53302 VL - 8 KW - informatics KW - community health services KW - knowledge translation KW - qualitative research KW - patient safety N2 - Background: Although intended to support improvement, the rapid adoption and evolution of technologies in health care can also bring about unintended consequences related to safety. In this project, an embedded researcher with expertise in patient safety and clinical education worked with a clinical informatics team to examine safety and harm related to health information technologies (HITs) in primary and community care settings. The clinical informatics team participated in learning activities around relevant topics (eg, human factors, high reliability organizations, and sociotechnical systems) and cocreated a process to address safety events related to technology (ie, safety huddles and sociotechnical analysis of safety events). Objective: This study aimed to explore clinical informaticians? experiences of incorporating safety practices into their work. Methods: We used a qualitative descriptive design and conducted web-based focus groups with clinical informaticians. Thematic analysis was used to analyze the data. Results: A total of 10 informants participated. Barriers to addressing safety and harm in their context included limited prior knowledge of HIT safety, previous assumptions and perspectives, competing priorities and organizational barriers, difficulty with the reporting system and processes, and a limited number of reports for learning. Enablers to promoting safety and mitigating harm included participating in learning sessions, gaining experience analyzing reported events, participating in safety huddles, and role modeling and leadership from the embedded researcher. Individual outcomes included increased ownership and interest in HIT safety, the development of a sociotechnical systems perspective, thinking differently about safety, and increased consideration for user perspectives. Team outcomes included enhanced communication within the team, using safety events to inform future work and strategic planning, and an overall promotion of a culture of safety. Conclusions: As HITs are integrated into care delivery, it is important for clinical informaticians to recognize the risks related to safety. Experiential learning activities, including reviewing safety event reports and participating in safety huddles, were identified as particularly impactful. An HIT safety learning initiative is a feasible approach for clinical informaticians to become more knowledgeable and engaged in HIT safety issues in their work. UR - https://formative.jmir.org/2024/1/e53302 UR - http://dx.doi.org/10.2196/53302 UR - http://www.ncbi.nlm.nih.gov/pubmed/38315544 ID - info:doi/10.2196/53302 ER - TY - JOUR AU - Stoffel, T. Sandro AU - Law, Hui Jing AU - Kerrison, Robert AU - Brewer, R. Hannah AU - Flanagan, M. James AU - Hirst, Yasemin PY - 2024/2/5 TI - Testing Behavioral Messages to Increase Recruitment to Health Research When Embedded Within Social Media Campaigns on Twitter: Web-Based Experimental Study JO - JMIR Form Res SP - e48538 VL - 8 KW - advertise KW - advertisement KW - advertisements KW - advertising KW - behavior change KW - behavioral KW - behaviour change KW - behavioural KW - campaign KW - campaigns KW - experimental design KW - message KW - messages KW - messaging KW - recruit KW - recruiting KW - recruitment KW - social media KW - social norms KW - Twitter N2 - Background: Social media is rapidly becoming the primary source to disseminate invitations to the public to consider taking part in research studies. There is, however, little information on how the contents of the advertisement can be communicated to facilitate engagement and subsequently promote intentions to participate in research. Objective: This paper describes an experimental study that tested different behavioral messages for recruiting study participants for a real-life observational case-control study. Methods: We included 1060 women in a web-based experiment and randomized them to 1 of 3 experimental conditions: standard advertisement (n=360), patient endorsement advertisement (n=345), and social norms advertisement (n=355). After seeing 1 of the 3 advertisements, participants were asked to state (1) their intention to take part in the advertised case-control study, (2) the ease of understanding the message and study aims, and (3) their willingness to be redirected to the website of the case-control study after completing the survey. Individuals were further asked to suggest ways to improve the messages. Intentions were compared between groups using ordinal logistic regression, reported in percentages, adjusted odds ratio (aOR), and 95% CIs. Results: Those who were in the patient endorsement and social norms?based advertisement groups had significantly lower intentions to take part in the advertised study compared with those in the standard advertisement group (aOR 0.73, 95% CI 0.55-0.97; P=.03 and aOR 0.69, 95% CI 0.52-0.92; P=.009, respectively). The patient endorsement advertisement was perceived to be more difficult to understand (aOR 0.65, 95% CI 0.48-0.87; P=.004) and to communicate the study aims less clearly (aOR 0.72, 95% CI 0.55-0.95; P=.01). While the patient endorsement advertisement had no impact on intention to visit the main study website, the social norms advertisement decreased willingness compared with the standard advertisement group (157/355, 44.2% vs 191/360, 53.1%; aOR 0.74, 95% CI 0.54-0.99; P=.02). The majority of participants (395/609, 64.8%) stated that the messages did not require changes, but some preferred clearer (75/609, 12.3%) and shorter (59/609, 9.7%) messages. Conclusions: The results of this study indicate that adding normative behavioral messages to simulated tweets decreased participant intention to take part in our web-based case-control study, as this made the tweet harder to understand. This suggests that simple messages should be used for participant recruitment through Twitter (subsequently rebranded X). UR - https://formative.jmir.org/2024/1/e48538 UR - http://dx.doi.org/10.2196/48538 UR - http://www.ncbi.nlm.nih.gov/pubmed/38315543 ID - info:doi/10.2196/48538 ER - TY - JOUR AU - Dryden, M. Eileen AU - Anwar, Chitra AU - Conti, Jennifer AU - Boudreau, H. Jacqueline AU - Kennedy, A. Meaghan AU - Hung, W. William AU - Nearing, A. Kathryn AU - Pimentel, B. Camilla AU - Moo, Lauren PY - 2024/2/1 TI - The Development and Use of a New Visual Tool (REVISIT) to Support Participant Recall: Web-Based Interview Study Among Older Adults JO - JMIR Form Res SP - e52096 VL - 8 KW - qualitative interviews KW - visual recall aid KW - older adults KW - health services research KW - web-based methods KW - visual tool KW - recall KW - qualitative interview KW - experience KW - perspective KW - motivation KW - patient KW - recall capacity KW - medical information KW - visual appointment KW - geriatric KW - older people KW - telemedicine KW - videoconference KW - e-consultation KW - e-medicine KW - internet medicine KW - REVISIT KW - Remembering Healthcare Encounters Visually and Interactively KW - mobile phone N2 - Background: Qualitative health services research often relies on semistructured or in-depth interviews to develop a deeper understanding of patient experiences, motivations, and perspectives. The quality of data gathered is contingent upon a patient?s recall capacity; yet, studies have shown that recall of medical information is low. Threats to generating rich and detailed interview data may be more prevalent when interviewing older adults. Objective: We developed and studied the feasibility of using a tool, Remembering Healthcare Encounters Visually and Interactively (REVISIT), which has been created to aid the recall of a specific telemedicine encounter to provide health services research teams with a visual tool, to improve qualitative interviews with older adults. Methods: The REVISIT visual appointment summary was developed to facilitate web-based interviews with our participants as part of an evaluation of a geriatric telemedicine program. Our primary aims were to aid participant recall, maintain focus on the index visit, and establish a shared understanding of the visit between participants and interviewers. The authors? experiences and observations developing REVISIT and using it during videoconference interviews (N=16) were systematically documented and synthesized. We discuss these experiences with REVISIT and suggest considerations for broader implementation and future research to expand upon this preliminary work. Results: REVISIT enhanced the interview process by providing a focus and catalyst for discussion and supporting rapport-building with participants. REVISIT appeared to support older patients? and caregivers? recollection of a clinical visit, helping them to share additional details about their experience. REVISIT was difficult to read for some participants, however, and could not be used for phone interviews. Conclusions: REVISIT is a promising tool to enhance the quality of data collected during interviews with older, rural adults and caregivers about a health care encounter. This novel tool may aid recall of health care experiences for those groups for whom it may be more challenging to collect accurate, rich qualitative data (eg, those with cognitive impairment or complex medical care), allowing health services research to include more diverse patient experiences. UR - https://formative.jmir.org/2024/1/e52096 UR - http://dx.doi.org/10.2196/52096 UR - http://www.ncbi.nlm.nih.gov/pubmed/38300691 ID - info:doi/10.2196/52096 ER - TY - JOUR AU - van der Heijden, Zoë AU - de Gooijer, Femke AU - Camps, Guido AU - Lucassen, Desiree AU - Feskens, Edith AU - Lasschuijt, Marlou AU - Brouwer-Brolsma, Elske PY - 2024/2/1 TI - User Requirements in Developing a Novel Dietary Assessment Tool for Children: Mixed Methods Study JO - JMIR Form Res SP - e47850 VL - 8 KW - diet KW - children KW - dietary assessment KW - recall KW - technological innovation KW - mobile health KW - mHealth KW - mobile phone N2 - Background: The prevalence of childhood obesity and comorbidities is rising alarmingly, and diet is an important modifiable determinant. Numerous dietary interventions in children have been developed to reduce childhood obesity and overweight rates, but their long-term effects are unsatisfactory. Stakeholders call for more personalized approaches, which require detailed dietary intake data. In the case of primary school children, caregivers are key to providing such dietary information. However, as school-aged children are not under the full supervision of one specific caregiver anymore, data are likely to be biased. Recent technological advancements provide opportunities for the role of children themselves, which would serve the overall quality of the obtained dietary data. Objective: This study aims to conduct a child-centered exploratory sequential mixed methods study to identify user requirements for a dietary assessment tool for children aged 5 to 6 years. Methods: Formative, nonsystematic narrative literature research was undertaken to delineate initial user requirements and inform prototype ideation in an expert panel workshop (n=11). This yielded 3 prototype dietary assessment tools: FoodBear (tangible piggy bank), myBear (smartphone or tablet app), and FoodCam (physical camera). All 3 prototypes were tested for usability by means of a usability task (video analyses) and user experience (This or That method) among 14 Dutch children aged 5 to 6 years (n=8, 57% boys and n=6, 43% girls). Results: Most children were able to complete FoodBear?s (11/14, 79%), myBear?s (10/14, 71%), and FoodCam?s (9/14, 64%) usability tasks, but all children required assistance (14/14, 100%) and most of the children encountered usability problems (13/14, 93%). Usability issues were related to food group categorization and recognition, frustrations owing to unsatisfactory functioning of (parts) of the prototypes, recall of food products, and the distinction between eating moments. No short-term differences in product preference between the 3 prototypes were observed, but autonomy, challenge, gaming elements, being tablet based, appearance, social elements, and time frame were identified as determinants of liking the product. Conclusions: Our results suggest that children can play a complementary role in dietary data collection to enhance the data collected by their parents. Incorporation of a training program, auditory or visual prompts, reminders and feedback, a user-friendly and intuitive interaction design, child-friendly food groups or icons, and room for children?s autonomy were identified as requirements for the future development of a novel and usable dietary assessment tool for children aged 5 to 6 years. Our findings can serve as valuable guidance for ongoing innovations in the field of children?s dietary assessment and the provision of personalized dietary support. UR - https://formative.jmir.org/2024/1/e47850 UR - http://dx.doi.org/10.2196/47850 UR - http://www.ncbi.nlm.nih.gov/pubmed/38300689 ID - info:doi/10.2196/47850 ER - TY - JOUR AU - Blasini, Romina AU - Strantz, Cosima AU - Gulden, Christian AU - Helfer, Sven AU - Lidke, Jakub AU - Prokosch, Hans-Ulrich AU - Sohrabi, Keywan AU - Schneider, Henning PY - 2024/1/31 TI - Evaluation of Eligibility Criteria Relevance for the Purpose of IT-Supported Trial Recruitment: Descriptive Quantitative Analysis JO - JMIR Form Res SP - e49347 VL - 8 KW - CTRSS KW - clinical trial recruitment support system KW - PRS KW - patient recruitment system KW - clinical trials KW - classifications KW - data groups KW - data elements KW - data classification KW - criteria KW - relevance KW - automated clinical trials KW - participants KW - clinical trial N2 - Background: Clinical trials (CTs) are crucial for medical research; however, they frequently fall short of the requisite number of participants who meet all eligibility criteria (EC). A clinical trial recruitment support system (CTRSS) is developed to help identify potential participants by performing a search on a specific data pool. The accuracy of the search results is directly related to the quality of the data used for comparison. Data accessibility can present challenges, making it crucial to identify the necessary data for a CTRSS to query. Prior research has examined the data elements frequently used in CT EC but has not evaluated which criteria are actually used to search for participants. Although all EC must be met to enroll a person in a CT, not all criteria have the same importance when searching for potential participants in an existing data pool, such as an electronic health record, because some of the criteria are only relevant at the time of enrollment. Objective: In this study, we investigated which groups of data elements are relevant in practice for finding suitable participants and whether there are typical elements that are not relevant and can therefore be omitted. Methods: We asked trial experts and CTRSS developers to first categorize the EC of their CTs according to data element groups and then to classify them into 1 of 3 categories: necessary, complementary, and irrelevant. In addition, the experts assessed whether a criterion was documented (on paper or digitally) or whether it was information known only to the treating physicians or patients. Results: We reviewed 82 CTs with 1132 unique EC. Of these 1132 EC, 350 (30.9%) were considered necessary, 224 (19.8%) complementary, and 341 (30.1%) total irrelevant. To identify the most relevant data elements, we introduced the data element relevance index (DERI). This describes the percentage of studies in which the corresponding data element occurs and is also classified as necessary or supplementary. We found that the query of ?diagnosis? was relevant for finding participants in 79 (96.3%) of the CTs. This group was followed by ?date of birth/age? with a DERI of 85.4% (n=70) and ?procedure? with a DERI of 35.4% (n=29). Conclusions: The distribution of data element groups in CTs has been heterogeneously described in previous works. Therefore, we recommend identifying the percentage of CTs in which data element groups can be found as a more reliable way to determine the relevance of EC. Only necessary and complementary criteria should be included in this DERI. UR - https://formative.jmir.org/2024/1/e49347 UR - http://dx.doi.org/10.2196/49347 UR - http://www.ncbi.nlm.nih.gov/pubmed/38294862 ID - info:doi/10.2196/49347 ER - TY - JOUR AU - Cranston, D. Kaela AU - Grieve, J. Natalie AU - Dineen, E. Tineke AU - Jung, E. Mary PY - 2024/1/26 TI - Designing and Developing Online Training for Diabetes Prevention Program Coaches Using an Integrated Knowledge Translation Approach: Development and Usability Study JO - JMIR Form Res SP - e50942 VL - 8 KW - program evaluation KW - prediabetic state KW - e-learning education KW - e-learning KW - platform KW - usability KW - diabetes KW - prevention KW - knowledge translation KW - end user KW - type 2 diabetes KW - framework N2 - Background: e-Learning has rapidly become a popular alternative to in-person learning due to its flexibility, convenience, and wide reach. Using a systematic and partnered process to transfer in-person training to an e-learning platform helps to ensure the training will be effective and acceptable to learners. Objective: This study aimed to develop an e-learning platform for Small Steps for Big Changes (SSBC) type 2 diabetes prevention program coaches to improve the viability of coach training. Methods: An integrated knowledge translation approach was used in the first 3 stages of the technology-enhanced learning (TEL) evaluation framework to address the study objective. This included three steps: (1) conducting a needs analysis based on focus groups with previously trained SSBC coaches, meetings with the SSBC research team, and a review of research results on the effectiveness of the previous in-person version of the training; (2) documenting processes and decisions in the design and development of the e-learning training platform; and (3) performing usability testing. Previously trained SSBC coaches and the SSBC research team were included in all stages of this study. Results: Step 1 identified components from the in-person training that should be maintained in the e-learning training (ie, a focus on motivational interviewing), additional components to be added to the e-learning training (ie, how to deliver culturally safe and inclusive care), and mode of delivery (videos and opportunities to synchronously practice skills). Step 2 documented the processes and decisions made in the design and development of the e-learning training, including the resources (ie, time and finances) used, the content of the training modules, and how coaches would flow through the training process. The design and development process consisted of creating a blueprint of the training. The training included 7 e-learning modules, the learning modalities of which included narrated demonstration videos and user-engaging activities, a mock session with feedback from the research team, and a final knowledge test. Step 3, usability testing, demonstrated high levels of learnability, efficiency, memorability, and satisfaction, with minor bugs documented and resolved. Conclusions: Using an integrated knowledge translation approach to the technology-enhanced learning evaluation framework was successful in developing an e-learning training platform for SSBC coaches. Incorporating end users in this process can increase the chances that the e-learning training platform is usable, engaging, and acceptable. Future research will include examining the satisfaction of coaches using the SSBC coach e-learning training platform, assessing coach learning outcomes (ie, knowledge and behavior), and estimating the cost and viability of implementing this training. UR - https://formative.jmir.org/2024/1/e50942 UR - http://dx.doi.org/10.2196/50942 UR - http://www.ncbi.nlm.nih.gov/pubmed/38277214 ID - info:doi/10.2196/50942 ER - TY - JOUR AU - Ma, Shaoying AU - Jiang, Shuning AU - Yang, Olivia AU - Zhang, Xuanzhi AU - Fu, Yu AU - Zhang, Yusen AU - Kaareen, Aadeeba AU - Ling, Meng AU - Chen, Jian AU - Shang, Ce PY - 2024/1/24 TI - Use of Machine Learning Tools in Evidence Synthesis of Tobacco Use Among Sexual and Gender Diverse Populations: Algorithm Development and Validation JO - JMIR Form Res SP - e49031 VL - 8 KW - machine learning KW - natural language processing KW - tobacco control KW - sexual and gender diverse populations KW - lesbian KW - gay KW - bisexual KW - transgender KW - queer KW - LGBTQ+ KW - evidence synthesis N2 - Background: From 2016 to 2021, the volume of peer-reviewed publications related to tobacco has experienced a significant increase. This presents a considerable challenge in efficiently summarizing, synthesizing, and disseminating research findings, especially when it comes to addressing specific target populations, such as the LGBTQ+ (lesbian, gay, bisexual, transgender, queer, intersex, asexual, Two Spirit, and other persons who identify as part of this community) populations. Objective: In order to expedite evidence synthesis and research gap discoveries, this pilot study has the following three aims: (1) to compile a specialized semantic database for tobacco policy research to extract information from journal article abstracts, (2) to develop natural language processing (NLP) algorithms that comprehend the literature on nicotine and tobacco product use among sexual and gender diverse populations, and (3) to compare the discoveries of the NLP algorithms with an ongoing systematic review of tobacco policy research among LGBTQ+ populations. Methods: We built a tobacco research domain?specific semantic database using data from 2993 paper abstracts from 4 leading tobacco-specific journals, with enrichment from other publicly available sources. We then trained an NLP model to extract named entities after learning patterns and relationships between words and their context in text, which further enriched the semantic database. Using this iterative process, we extracted and assessed studies relevant to LGBTQ+ tobacco control issues, further comparing our findings with an ongoing systematic review that also focuses on evidence synthesis for this demographic group. Results: In total, 33 studies were identified as relevant to sexual and gender diverse individuals? nicotine and tobacco product use. Consistent with the ongoing systematic review, the NLP results showed that there is a scarcity of studies assessing policy impact on this demographic using causal inference methods. In addition, the literature is dominated by US data. We found that the product drawing the most attention in the body of existing research is cigarettes or cigarette smoking and that the number of studies of various age groups is almost evenly distributed between youth or young adults and adults, consistent with the research needs identified by the US health agencies. Conclusions: Our pilot study serves as a compelling demonstration of the capabilities of NLP tools in expediting the processes of evidence synthesis and the identification of research gaps. While future research is needed to statistically test the NLP tool?s performance, there is potential for NLP tools to fundamentally transform the approach to evidence synthesis. UR - https://formative.jmir.org/2024/1/e49031 UR - http://dx.doi.org/10.2196/49031 UR - http://www.ncbi.nlm.nih.gov/pubmed/38265858 ID - info:doi/10.2196/49031 ER - TY - JOUR AU - Herrmann-Werner, Anne AU - Festl-Wietek, Teresa AU - Holderried, Friederike AU - Herschbach, Lea AU - Griewatz, Jan AU - Masters, Ken AU - Zipfel, Stephan AU - Mahling, Moritz PY - 2024/1/23 TI - Assessing ChatGPT?s Mastery of Bloom?s Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study JO - J Med Internet Res SP - e52113 VL - 26 KW - answer KW - artificial intelligence KW - assessment KW - Bloom?s taxonomy KW - ChatGPT KW - classification KW - error KW - exam KW - examination KW - generative KW - GPT-4 KW - Generative Pre-trained Transformer 4 KW - language model KW - learning outcome KW - LLM KW - MCQ KW - medical education KW - medical exam KW - multiple-choice question KW - natural language processing KW - NLP KW - psychosomatic KW - question KW - response KW - taxonomy N2 - Background: Large language models such as GPT-4 (Generative Pre-trained Transformer 4) are being increasingly used in medicine and medical education. However, these models are prone to ?hallucinations? (ie, outputs that seem convincing while being factually incorrect). It is currently unknown how these errors by large language models relate to the different cognitive levels defined in Bloom?s taxonomy. Objective: This study aims to explore how GPT-4 performs in terms of Bloom?s taxonomy using psychosomatic medicine exam questions. Methods: We used a large data set of psychosomatic medicine multiple-choice questions (N=307) with real-world results derived from medical school exams. GPT-4 answered the multiple-choice questions using 2 distinct prompt versions: detailed and short. The answers were analyzed using a quantitative approach and a qualitative approach. Focusing on incorrectly answered questions, we categorized reasoning errors according to the hierarchical framework of Bloom?s taxonomy. Results: GPT-4?s performance in answering exam questions yielded a high success rate: 93% (284/307) for the detailed prompt and 91% (278/307) for the short prompt. Questions answered correctly by GPT-4 had a statistically significant higher difficulty than questions answered incorrectly (P=.002 for the detailed prompt and P<.001 for the short prompt). Independent of the prompt, GPT-4?s lowest exam performance was 78.9% (15/19), thereby always surpassing the ?pass? threshold. Our qualitative analysis of incorrect answers, based on Bloom?s taxonomy, showed that errors were primarily in the ?remember? (29/68) and ?understand? (23/68) cognitive levels; specific issues arose in recalling details, understanding conceptual relationships, and adhering to standardized guidelines. Conclusions: GPT-4 demonstrated a remarkable success rate when confronted with psychosomatic medicine multiple-choice exam questions, aligning with previous findings. When evaluated through Bloom?s taxonomy, our data revealed that GPT-4 occasionally ignored specific facts (remember), provided illogical reasoning (understand), or failed to apply concepts to a new situation (apply). These errors, which were confidently presented, could be attributed to inherent model biases and the tendency to generate outputs that maximize likelihood. UR - https://www.jmir.org/2024/1/e52113 UR - http://dx.doi.org/10.2196/52113 UR - http://www.ncbi.nlm.nih.gov/pubmed/38261378 ID - info:doi/10.2196/52113 ER - TY - JOUR AU - Zrubka, Zsombor AU - Kertész, Gábor AU - Gulácsi, László AU - Czere, János AU - Hölgyesi, Áron AU - Nezhad, Motahari Hossein AU - Mosavi, Amir AU - Kovács, Levente AU - Butte, J. Atul AU - Péntek, Márta PY - 2024/1/19 TI - The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review JO - J Med Internet Res SP - e47430 VL - 26 KW - diabetes mellitus KW - children KW - adolescent KW - pediatric KW - machine learning KW - Minimum Information About Clinical Artificial Intelligence Modelling KW - MI-CLAIM KW - reporting quality N2 - Background: Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine. Objective: We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies. Methods: We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility. Results: After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant. Conclusions: The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable. UR - https://www.jmir.org/2024/1/e47430 UR - http://dx.doi.org/10.2196/47430 UR - http://www.ncbi.nlm.nih.gov/pubmed/38241075 ID - info:doi/10.2196/47430 ER - TY - JOUR AU - Pang, MengWei AU - Dong, YanLing AU - Zhao, XiaoHan AU - Wan, JiaWu AU - Jiang, Li AU - Song, JinLin AU - Ji, Ping AU - Jiang, Lin PY - 2024/1/17 TI - Virtual and Interprofessional Objective Structured Clinical Examination in Dentistry and Dental Technology: Development and User Evaluations JO - JMIR Form Res SP - e44653 VL - 8 KW - dentist KW - dental technician KW - objective structured clinical examination KW - OSCE KW - interprofessional education KW - interprofessional collaborative practice N2 - Background: Interprofessional education (IPE) facilitates interprofessional collaborative practice (IPCP) to encourage teamwork among dental care professionals and is increasingly becoming a part of training programs for dental and dental technology students. However, the focus of previous IPE and IPCP studies has largely been on subjective student and instructor perceptions without including objective assessments of collaborative practice as an outcome measure. Objective: The purposes of this study were to develop the framework for a novel virtual and interprofessional objective structured clinical examination (viOSCE) applicable to dental and dental technology students, to assess the effectiveness of the framework as a tool for measuring the outcomes of IPE, and to promote IPCP among dental and dental technology students. Methods: The framework of the proposed novel viOSCE was developed using the modified Delphi method and then piloted. The lead researcher and a group of experts determined the content and scoring system. Subjective data were collected using the Readiness for Interprofessional Learning Scale and a self-made scale, and objective data were collected using examiner ratings. Data were analyzed using nonparametric tests. Results: We successfully developed a viOSCE framework applicable to dental and dental technology students. Of 50 students, 32 (64%) participated in the pilot study and completed the questionnaires. On the basis of the Readiness for Interprofessional Learning Scale, the subjective evaluation indicated that teamwork skills were improved, and the only statistically significant difference in participant motivation between the 2 professional groups was in the mutual evaluation scale (P=.004). For the viOSCE evaluation scale, the difference between the professional groups in removable prosthodontics was statistically significant, and a trend for negative correlation between subjective and objective scores was noted, but it was not statistically significant. Conclusions: The results confirm that viOSCE can be used as an objective evaluation tool to assess the outcomes of IPE and IPCP. This study also revealed an interesting relationship between mutual evaluation and IPCP results, further demonstrating that the IPE and IPCP results urgently need to be supplemented with objective evaluation tools. Therefore, the implementation of viOSCE as part of a large and more complete objective structured clinical examination to test the ability of students to meet undergraduate graduation requirements will be the focus of our future studies. UR - https://formative.jmir.org/2024/1/e44653 UR - http://dx.doi.org/10.2196/44653 UR - http://www.ncbi.nlm.nih.gov/pubmed/38231556 ID - info:doi/10.2196/44653 ER - TY - JOUR AU - Bonett, Stephen AU - Lin, Willey AU - Sexton Topper, Patrina AU - Wolfe, James AU - Golinkoff, Jesse AU - Deshpande, Aayushi AU - Villarruel, Antonia AU - Bauermeister, José PY - 2024/1/12 TI - Assessing and Improving Data Integrity in Web-Based Surveys: Comparison of Fraud Detection Systems in a COVID-19 Study JO - JMIR Form Res SP - e47091 VL - 8 KW - web-based survey KW - data quality KW - fraud KW - survey methodology KW - COVID-19 KW - survey KW - fraud detection KW - Philadelphia KW - data privacy KW - data protection KW - privacy KW - security KW - data KW - information security KW - data validation KW - cross-sectional KW - web-based N2 - Background: Web-based surveys increase access to study participation and improve opportunities to reach diverse populations. However, web-based surveys are vulnerable to data quality threats, including fraudulent entries from automated bots and duplicative submissions. Widely used proprietary tools to identify fraud offer little transparency about the methods used, effectiveness, or representativeness of resulting data sets. Robust, reproducible, and context-specific methods of accurately detecting fraudulent responses are needed to ensure integrity and maximize the value of web-based survey research. Objective: This study aims to describe a multilayered fraud detection system implemented in a large web-based survey about COVID-19 attitudes, beliefs, and behaviors; examine the agreement between this fraud detection system and a proprietary fraud detection system; and compare the resulting study samples from each of the 2 fraud detection methods. Methods: The PhillyCEAL Common Survey is a cross-sectional web-based survey that remotely enrolled residents ages 13 years and older to assess how the COVID-19 pandemic impacted individuals, neighborhoods, and communities in Philadelphia, Pennsylvania. Two fraud detection methods are described and compared: (1) a multilayer fraud detection strategy developed by the research team that combined automated validation of response data and real-time verification of study entries by study personnel and (2) the proprietary fraud detection system used by the Qualtrics (Qualtrics) survey platform. Descriptive statistics were computed for the full sample and for responses classified as valid by 2 different fraud detection methods, and classification tables were created to assess agreement between the methods. The impact of fraud detection methods on the distribution of vaccine confidence by racial or ethnic group was assessed. Results: Of 7950 completed surveys, our multilayer fraud detection system identified 3228 (40.60%) cases as valid, while the Qualtrics fraud detection system identified 4389 (55.21%) cases as valid. The 2 methods showed only ?fair? or ?minimal? agreement in their classifications (?=0.25; 95% CI 0.23-0.27). The choice of fraud detection method impacted the distribution of vaccine confidence by racial or ethnic group. Conclusions: The selection of a fraud detection method can affect the study?s sample composition. The findings of this study, while not conclusive, suggest that a multilayered approach to fraud detection that includes conservative use of automated fraud detection and integration of human review of entries tailored to the study?s specific context and its participants may be warranted for future survey research. UR - https://formative.jmir.org/2024/1/e47091 UR - http://dx.doi.org/10.2196/47091 UR - http://www.ncbi.nlm.nih.gov/pubmed/38214962 ID - info:doi/10.2196/47091 ER - TY - JOUR AU - Maenhout, Laura AU - Latomme, Julie AU - Cardon, Greet AU - Crombez, Geert AU - Van Hove, Geert AU - Compernolle, Sofie PY - 2024/1/11 TI - Synergizing the Behavior Change Wheel and a Cocreative Approach to Design a Physical Activity Intervention for Adolescents and Young Adults With Intellectual Disabilities: Development Study JO - JMIR Form Res SP - e51693 VL - 8 KW - Behavior Change Wheel KW - cocreation KW - intervention KW - physical activity KW - adolescents KW - young adults KW - intellectual disabilities N2 - Background: There is a need for physical activity promotion interventions in adolescents and young adults with intellectual disabilities. Current interventions have shown limited effectiveness, which may be attributed to the absence of theory and a population-specific development. Combining a planning model (including theory) and cocreation with the target audience during intervention development could potentially address this gap. Objective: This study aimed to report the systematic development of the Move it, Move ID! intervention by describing how the 8 different steps of the Behavior Change Wheel (BCW) were applied and present the results that emerged from those steps. In doing so, the (theoretical) content of the intervention is described in detail. Methods: A total of 23 adolescents and young adults (aged 14-22 years) with mild to moderate intellectual disabilities were designated as cocreators of the intervention. Across 2 groups, 6 similar cocreation sessions were organized in each. The content and sequence of the sessions were structured to align with the 8 steps of the BCW. All sessions were recorded and transcribed verbatim. Both a deductive (ie, steps of the BCW) and inductive (ie, resonating the voice of the participants) analysis approach were applied specifically focusing on identifying and describing the findings within each of the BCW steps. Results: After behavioral analysis (steps 1-4), 10 intervention goals were chosen and linked to Capability, Opportunity, and Motivation?Behavior components (theory within the BCW) that needed to be addressed. Psychological capability, social opportunity, and reflective motivation were emphasized as the first targets to focus on. A key finding was the urge for real-life social connectedness and social integration, which makes the social component as part of physical activity a central theme to focus on within intervention development. Judgments on the most suitable intervention functions (step 5) and behavior change techniques (step 7) were explained. When discussing the mode of delivery of the intervention (step 8), it was underscored that solely relying on a mobile health app would not fulfill participants? social needs. Hence, the chosen intervention adopts a dyadic approach in which young individuals with intellectual disabilities are matched with peers without intellectual disabilities to engage in physical activities together, with a mobile app playing a supportive role in this partnership. Conclusions: The transparent description of the development process highlights why certain intervention components and behavior change techniques were chosen and how they are intertwined by means of the selected intervention design. This paper provides a detailed blueprint for practitioners wanting to integrate the BCW and its associated behavior change techniques, in combination with actively involving the target group, into their intervention development for people with intellectual disabilities. UR - https://formative.jmir.org/2024/1/e51693 UR - http://dx.doi.org/10.2196/51693 UR - http://www.ncbi.nlm.nih.gov/pubmed/38206648 ID - info:doi/10.2196/51693 ER - TY - JOUR AU - Liang, Bingyu AU - Sun, Rujing AU - Liao, Yanyan AU - Nong, Aidan AU - He, Jinfeng AU - Qin, Fengxiang AU - Ou, Yanyun AU - Che, Jianhua AU - Wu, Zhenxian AU - Yang, Yuan AU - Qin, Jiao AU - Cai, Jie AU - Bao, Lijuan AU - Ye, Li AU - Liang, Hao PY - 2024/1/9 TI - CD4/CD8 Ratio Recovered as a Predictor of Decreased Liver Damage in Adults Infected With HIV: 16-Year Observational Cohort Study JO - JMIR Public Health Surveill SP - e45818 VL - 10 KW - AIDS KW - antiretroviral therapy KW - CD4/CD8 KW - efavirenz KW - HIV KW - liver damage KW - lopinavir KW - nevirapine N2 - Background: As the life expectancy of individuals infected with HIV continues to increase, vigilant monitoring of non?AIDS-related events becomes imperative, particularly those pertaining to liver diseases. In comparison to the general population, patients infected with HIV experience a higher frequency of liver-related deaths. The CD4/CD8 ratio is emerging as a potential biomarker for non?AIDS-related events. However, few existing studies have been specially designed to explore the relationship between the CD4/CD8 ratio and specific types of non?AIDS-related events, notably liver damage. Objective: This study aimed to investigate the potential association between the CD4/CD8 ratio and the development of liver damage in a sizable cohort of patients infected with HIV receiving antiretroviral treatment (ART). Additionally, the study sought to assess the effectiveness of 3 antiretroviral drugs in recovering the CD4/CD8 ratio and reducing the occurrence of liver damage in this population. Methods: We conducted an observational cohort study among adults infected with HIV receiving ART from 2004 to 2020 in Guangxi, China. Propensity score matching, multivariable Cox proportional hazard, and Fine-Gray competing risk regression models were used to determine the relationship between the CD4/CD8 ratio recovered and liver damage. Results: The incidence of liver damage was 20.12% among 2440 eligible individuals during a median follow-up period of 4 person-years. Patients whose CD4/CD8 ratio did not recover to 1.0 exhibited a higher incidence of liver damage compared to patients with a CD4/CD8 ratio recovered (adjusted hazard ratio 7.90, 95% CI 4.39-14.21; P<.001; subdistribution hazard ratio 6.80, 95% CI 3.83-12.11; P<.001), findings consistent with the propensity score matching analysis (adjusted hazard ratio 6.94, 95% CI 3.41-14.12; P<.001; subdistribution hazard ratio 5.67, 95% CI 2.74-11.73; P<.001). The Efavirenz-based regimen exhibited the shortest time for CD4/CD8 ratio recovery (median 71, IQR 49-88 months) and demonstrated a lower prevalence of liver damage (4.18/100 person-years). Conclusions: Recovery of the CD4/CD8 ratio was associated with a decreased risk of liver damage in patients infected with HIV receiving ART, adding evidence for considering the CD4/CD8 ratio as a potential marker for identifying individuals at risk of non?AIDS-related diseases. An efavirenz-based regimen emerged as a recommended choice for recovering the CD4/CD8 ratio and mitigating the risk of liver damage. UR - https://publichealth.jmir.org/2024/1/e45818 UR - http://dx.doi.org/10.2196/45818 UR - http://www.ncbi.nlm.nih.gov/pubmed/37846087 ID - info:doi/10.2196/45818 ER - TY - JOUR AU - Bentley, H. Kate AU - Madsen, M. Emily AU - Song, Eugene AU - Zhou, Yu AU - Castro, Victor AU - Lee, Hyunjoon AU - Lee, H. Younga AU - Smoller, W. Jordan PY - 2024/1/8 TI - Determining Distinct Suicide Attempts From Recurrent Electronic Health Record Codes: Classification Study JO - JMIR Form Res SP - e46364 VL - 8 KW - suicide KW - suicide attempt KW - self-injury KW - electronic health record KW - EHR KW - prediction KW - predictive model KW - predict KW - model KW - suicidal KW - informatics KW - automated rule KW - psychiatry KW - machine learning N2 - Background: Prior suicide attempts are a relatively strong risk factor for future suicide attempts. There is growing interest in using longitudinal electronic health record (EHR) data to derive statistical risk prediction models for future suicide attempts and other suicidal behavior outcomes. However, model performance may be inflated by a largely unrecognized form of ?data leakage? during model training: diagnostic codes for suicide attempt outcomes may refer to prior attempts that are also included in the model as predictors. Objective: We aimed to develop an automated rule for determining when documented suicide attempt diagnostic codes identify distinct suicide attempt events. Methods: From a large health care system?s EHR, we randomly sampled suicide attempt codes for 300 patients with at least one pair of suicide attempt codes documented at least one but no more than 90 days apart. Supervised chart reviewers assigned the clinical settings (ie, emergency department [ED] versus non-ED), methods of suicide attempt, and intercode interval (number of days). The probability (or positive predictive value) that the second suicide attempt code in a given pair of codes referred to a distinct suicide attempt event from its preceding suicide attempt code was calculated by clinical setting, method, and intercode interval. Results: Of 1015 code pairs reviewed, 835 (82.3%) were nonindependent (ie, the 2 codes referred to the same suicide attempt event). When the second code in a pair was documented in a clinical setting other than the ED, it represented a distinct suicide attempt 3.3% of the time. The more time elapsed between codes, the more likely the second code in a pair referred to a distinct suicide attempt event from its preceding code. Code pairs in which the second suicide attempt code was assigned in an ED at least 5 days after its preceding suicide attempt code had a positive predictive value of 0.90. Conclusions: EHR-based suicide risk prediction models that include International Classification of Diseases codes for prior suicide attempts as a predictor may be highly susceptible to bias due to data leakage in model training. We derived a simple rule to distinguish codes that reflect new, independent suicide attempts: suicide attempt codes documented in an ED setting at least 5 days after a preceding suicide attempt code can be confidently treated as new events in EHR-based suicide risk prediction models. This rule has the potential to minimize upward bias in model performance when prior suicide attempts are included as predictors in EHR-based suicide risk prediction models. UR - https://formative.jmir.org/2024/1/e46364 UR - http://dx.doi.org/10.2196/46364 UR - http://www.ncbi.nlm.nih.gov/pubmed/38190236 ID - info:doi/10.2196/46364 ER - TY - JOUR AU - Saito, Junichi AU - Kumano, Hiroaki AU - Ghazizadeh, Mohammad AU - Shimokawa, Chigusa AU - Tanemura, Hideki PY - 2024/1/3 TI - Differences in Psychological Inflexibility Among Men With Erectile Dysfunction Younger and Older Than 40 Years: Web-Based Cross-Sectional Study JO - JMIR Form Res SP - e45998 VL - 8 KW - erectile dysfunction KW - acceptance and commitment therapy KW - psychological inflexibility KW - depression KW - anxiety KW - men KW - cross-sectional study KW - psychological KW - utility KW - psychosocial KW - therapy KW - impotence KW - erection N2 - Background: Psychological inflexibility is a core concept of acceptance and commitment therapy (ACT), which is a comprehensive, transdiagnostic interpretation of mental health symptoms. Erectile dysfunction (ED) is a condition that affects male sexual performance, involving the inability to achieve and maintain a penile erection sufficient for satisfactory sexual activity. Psychosocial factors primarily influence ED in men younger than 40 years, whereas biological factors are more likely to be the underlying cause in older men. Objective: This web-based cross-sectional study examined differences in depression, anxiety, and psychological inflexibility among men with ED younger and older than 40 years in a Japanese population. Methods: We used a web-based survey to gather data from various community samples. ED was assessed by the International Index of Erectile Function?5 (IIEF-5) questionnaire, while depression, anxiety, and psychological inflexibility were evaluated by the Patient Health Questionnaire-9 (PHQ-9), General Anxiety Disorder-7 (GAD-7), Acceptance and Action Questionnaire-II (AAQ-II), Cognitive Fusion Questionnaire (CFQ), and Valuing Questionnaire?Obstacle Subscale (VQ-OB) questionnaires. The chi?square test estimated the scores of PHQ-9 and GAD-7 among men with ED, comparing those younger than 40 years and those older than 40 years. Additionally, a two-way ANOVA was conducted with ED severity and age group as independent variables, assessing psychological inflexibility. Results: Valid responses from 643 individuals (mean age 36.19, SD 7.54 years) were obtained. Of these, 422 were younger than 40 years (mean age 31.76, SD 5.00 years), and 221 were older than 40 years (mean age 44.67, SD 2.88 years). There was a statistical difference in the prevalence of depression as judged by PHQ?10 between men with ED younger and older than 40 years (P<.001). On the other hand, there was no difference in the prevalence of anxiety as judged by GAD?10 (P=.12). The two-way ANOVA revealed that the interactions for CFQ (P=.04) and VQ-OB (P=.01) were significant. The simple main effect was that men with ED younger than 40 years had significantly higher CFQ (P=.01; d=0.62) and VQ-OB (P<.001; d=0.87) scores compared to those older than 40 years in moderate ED and severe ED. Additionally, it was found that men younger than 40 years with moderate to severe ED had significantly higher CFQ (P=.01; d=0.42) and VQ-OB (P=.02; d=0.38) scores compared to men younger than 40 years without ED. On the other hand, no interaction was found for AAQ-II (P=.16) scores. Conclusions: To the best of our knowledge, this web-based cross-sectional study is the first to examine the relationship between psychological inflexibility and ED. We conclude that men with moderate and severe ED younger than 40 years have higher psychological inflexibility and might be eligible for ACT. UR - https://formative.jmir.org/2024/1/e45998 UR - http://dx.doi.org/10.2196/45998 UR - http://www.ncbi.nlm.nih.gov/pubmed/38170587 ID - info:doi/10.2196/45998 ER - TY - JOUR AU - Zhang, Pin AU - Wu, Lei AU - Zou, Ting-Ting AU - Zou, ZiXuan AU - Tu, JiaXin AU - Gong, Ren AU - Kuang, Jie PY - 2024/1/3 TI - Machine Learning for Early Prediction of Major Adverse Cardiovascular Events After First Percutaneous Coronary Intervention in Patients With Acute Myocardial Infarction: Retrospective Cohort Study JO - JMIR Form Res SP - e48487 VL - 8 KW - acute myocardial infarction KW - percutaneous coronary intervention KW - machine learning KW - early prediction KW - cardiovascular event N2 - Background: The incidence of major adverse cardiovascular events (MACEs) remains high in patients with acute myocardial infarction (AMI) who undergo percutaneous coronary intervention (PCI), and early prediction models to guide their clinical management are lacking. Objective: This study aimed to develop machine learning?based early prediction models for MACEs in patients with newly diagnosed AMI who underwent PCI. Methods: A total of 1531 patients with AMI who underwent PCI from January 2018 to December 2019 were enrolled in this consecutive cohort. The data comprised demographic characteristics, clinical investigations, laboratory tests, and disease-related events. Four machine learning models?artificial neural network (ANN), k-nearest neighbors, support vector machine, and random forest?were developed and compared with the logistic regression model. Our primary outcome was the model performance that predicted the MACEs, which was determined by accuracy, area under the receiver operating characteristic curve, and F1-score. Results: In total, 1362 patients were successfully followed up. With a median follow-up of 25.9 months, the incidence of MACEs was 18.5% (252/1362). The area under the receiver operating characteristic curve of the ANN, random forest, k-nearest neighbors, support vector machine, and logistic regression models were 80.49%, 72.67%, 79.80%, 77.20%, and 71.77%, respectively. The top 5 predictors in the ANN model were left ventricular ejection fraction, the number of implanted stents, age, diabetes, and the number of vessels with coronary artery disease. Conclusions: The ANN model showed good MACE prediction after PCI for patients with AMI. The use of machine learning?based prediction models may improve patient management and outcomes in clinical practice. UR - https://formative.jmir.org/2024/1/e48487 UR - http://dx.doi.org/10.2196/48487 UR - http://www.ncbi.nlm.nih.gov/pubmed/38170581 ID - info:doi/10.2196/48487 ER - TY - JOUR AU - Légaré, France AU - G V Mochcovitch, Diogo AU - de Carvalho Corôa, Roberta AU - Gogovor, Amédé AU - Ben Charif, Ali AU - Cameron, Cynthia AU - Plamondon, Annie AU - Cimon, Marie AU - Guay-Bélanger, Sabrina AU - Roch, Geneviève AU - Dumas Pilon, Maxine AU - Paquette, Jean-Sébastien AU - McLean, D. Robert K. AU - Milat, Andrew PY - 2023/12/28 TI - Spontaneous Scaling of a Primary Care Innovation in Real-Life Conditions: Protocol for a Case Study JO - JMIR Res Protoc SP - e54855 VL - 12 KW - scaling KW - spread KW - primary care KW - spontaneous KW - knowledge translation KW - implementation science KW - scaling science N2 - Background: Scaling effective primary care innovations to benefit more people is of interest to decision makers. However, we know little about how promising innovations are being scaled ?spontaneously,? that is, without deliberate guidance. Objective: We aim to observe, document, and analyze how, in real-life conditions, 1 primary care innovation spontaneously scales up across Quebec, Canada. Methods: We will conduct a participative study using a descriptive single-case study. It will be guided by the McLean and Gargani principles for scaling and reported according to the COREQ (Consolidated Criteria for Reporting Qualitative Research) guidelines. Informed by an integrated knowledge translation approach, our steering committee will include patient users throughout the project. Inspired by the Quebec College of Family Physician?s ?Dragons? Den? primary care program, we will identify a promising primary care innovation that is being or will be scaled spontaneously. We will interview the innovation team about their scaling experiences every month for 1 year. We will conduct interviews and focus groups with decision makers, health care providers, and end users in the innovation team and the target site about their experience of both scaling and receiving the scaled innovation and document meetings as nonparticipant observers. Interview transcripts and documentary data will be analyzed to (1) compare the spontaneous scaling plan and implementation with the McLean and Gargani principles for scaling and (2) determine how it was consistent with or diverged from the 4 McLean and Gargani guiding principles: justification, optimal scale, coordination, and dynamic evaluation. Results: This study was funded in March 2020 by the Canadian Institutes of Health Research. Recruitment began in November 2023 and data collection began in December 2023. Results are expected to be published in the first quarter of 2024. Conclusions: Our study will advance the science of scaling by providing practical evidence?based material about scaling health and social care innovations in real-world settings using the 4 guiding principles of McLean and Gargani. International Registered Report Identifier (IRRID): PRR1-10.2196/54855 UR - https://www.researchprotocols.org/2023/1/e54855 UR - http://dx.doi.org/10.2196/54855 UR - http://www.ncbi.nlm.nih.gov/pubmed/38032757 ID - info:doi/10.2196/54855 ER - TY - JOUR AU - Valvi, Nimish AU - McFarlane, Timothy AU - Allen, S. Katie AU - Gibson, Joseph P. AU - Dixon, Edward Brian PY - 2023/12/27 TI - Identification of Hypertension in Electronic Health Records Through Computable Phenotype Development and Validation for Use in Public Health Surveillance: Retrospective Study JO - JMIR Form Res SP - e46413 VL - 7 KW - computable phenotypes KW - electronic health records KW - health information exchange KW - hypertension KW - population surveillance KW - public health informatics N2 - Background: Electronic health record (EHR) systems are widely used in the United States to document care delivery and outcomes. Health information exchange (HIE) networks, which integrate EHR data from the various health care providers treating patients, are increasingly used to analyze population-level data. Existing methods for population health surveillance of essential hypertension by public health authorities may be complemented using EHR data from HIE networks to characterize disease burden at the community level. Objective: We aimed to derive and validate computable phenotypes (CPs) to estimate hypertension prevalence for population-based surveillance using an HIE network. Methods: Using existing data available from an HIE network, we developed 6 candidate CPs for essential (primary) hypertension in an adult population from a medium-sized Midwestern metropolitan area in the United States. A total of 2 independent clinician reviewers validated the phenotypes through a manual chart review of 150 randomly selected patient records. We assessed the precision of CPs by calculating sensitivity, specificity, positive predictive value (PPV), F1-score, and validity of chart reviews using prevalence-adjusted bias-adjusted ?. We further used the most balanced CP to estimate the prevalence of hypertension in the population. Results: Among a cohort of 548,232 adults, 6 CPs produced PPVs ranging from 71% (95% CI 64.3%-76.9%) to 95.7% (95% CI 84.9%-98.9%). The F1-score ranged from 0.40 to 0.91. The prevalence-adjusted bias-adjusted ? revealed a high percentage agreement of 0.88 for hypertension. Similarly, interrater agreement for individual phenotype determination demonstrated substantial agreement (range 0.70-0.88) for all 6 phenotypes examined. A phenotype based solely on diagnostic codes possessed reasonable performance (F1-score=0.63; PPV=95.1%) but was imbalanced with low sensitivity (47.6%). The most balanced phenotype (F1-score=0.91; PPV=83.5%) included diagnosis, blood pressure measurements, and medications and identified 210,764 (38.4%) individuals with hypertension during the study period (2014-2015). Conclusions: We identified several high-performing phenotypes to identify essential hypertension prevalence for local public health surveillance using EHR data. Given the increasing availability of EHR systems in the United States and other nations, leveraging EHR data has the potential to enhance surveillance of chronic disease in health systems and communities. Yet given variability in performance, public health authorities will need to decide whether to seek optimal balance or declare a preference for algorithms that lean toward sensitivity or specificity to estimate population prevalence of disease. UR - https://formative.jmir.org/2023/1/e46413 UR - http://dx.doi.org/10.2196/46413 UR - http://www.ncbi.nlm.nih.gov/pubmed/38150296 ID - info:doi/10.2196/46413 ER - TY - JOUR AU - Zheng, Yifan AU - Rowell, Brigid AU - Chen, Qiyuan AU - Kim, Yong Jin AU - Kontar, Al Raed AU - Yang, Jessie X. AU - Lester, A. Corey PY - 2023/12/25 TI - Designing Human-Centered AI to Prevent Medication Dispensing Errors: Focus Group Study With Pharmacists JO - JMIR Form Res SP - e51921 VL - 7 KW - artificial intelligence KW - communication KW - design methods KW - design KW - development KW - engineering KW - focus groups KW - human-computer interaction KW - medication errors KW - morbidity KW - mortality KW - patient safety KW - safety KW - SEIPS KW - Systems Engineering Initiative for Patient Safety KW - tool KW - user-centered design methods KW - user-centered KW - visualization N2 - Background: Medication errors, including dispensing errors, represent a substantial worldwide health risk with significant implications in terms of morbidity, mortality, and financial costs. Although pharmacists use methods like barcode scanning and double-checking for dispensing verification, these measures exhibit limitations. The application of artificial intelligence (AI) in pharmacy verification emerges as a potential solution, offering precision, rapid data analysis, and the ability to recognize medications through computer vision. For AI to be embraced, it must be designed with the end user in mind, fostering trust, clear communication, and seamless collaboration between AI and pharmacists. Objective: This study aimed to gather pharmacists? feedback in a focus group setting to help inform the initial design of the user interface and iterative designs of the AI prototype. Methods: A multidisciplinary research team engaged pharmacists in a 3-stage process to develop a human-centered AI system for medication dispensing verification. To design the AI model, we used a Bayesian neural network that predicts the dispensed pills? National Drug Code (NDC). Discussion scripts regarding how to design the system and feedback in focus groups were collected through audio recordings and professionally transcribed, followed by a content analysis guided by the Systems Engineering Initiative for Patient Safety and Human-Machine Teaming theoretical frameworks. Results: A total of 8 pharmacists participated in 3 rounds of focus groups to identify current challenges in medication dispensing verification, brainstorm solutions, and provide feedback on our AI prototype. Participants considered several teaming scenarios, generally favoring a hybrid teaming model where the AI assists in the verification process and a pharmacist intervenes based on medication risk level and the AI?s confidence level. Pharmacists highlighted the need for improving the interpretability of AI systems, such as adding stepwise checkmarks, probability scores, and details about drugs the AI model frequently confuses with the target drug. Pharmacists emphasized the need for simplicity and accessibility. They favored displaying only essential information to prevent overwhelming users with excessive data. Specific design features, such as juxtaposing pill images with their packaging for quick comparisons, were requested. Pharmacists preferred accept, reject, or unsure options. The final prototype interface included (1) checkmarks to compare pill characteristics between the AI-predicted NDC and the prescription?s expected NDC, (2) a histogram showing predicted probabilities for the AI-identified NDC, (3) an image of an AI-provided ?confused? pill, and (4) an NDC match status (ie, match, unmatched, or unsure). Conclusions: In partnership with pharmacists, we developed a human-centered AI prototype designed to enhance AI interpretability and foster trust. This initiative emphasized human-machine collaboration and positioned AI as an augmentative tool rather than a replacement. This study highlights the process of designing a human-centered AI for dispensing verification, emphasizing its interpretability, confidence visualization, and collaborative human-machine teaming styles. UR - https://formative.jmir.org/2023/1/e51921 UR - http://dx.doi.org/10.2196/51921 UR - http://www.ncbi.nlm.nih.gov/pubmed/38145475 ID - info:doi/10.2196/51921 ER - TY - JOUR AU - Folkvord, Frans AU - Bol, Nadine AU - Stazi, Giacomo AU - Peschke, Lutz AU - Lupiáñez-Villanueva, Francisco PY - 2023/12/25 TI - Preferences in the Willingness to Download an mHealth App: Discrete Choice Experimental Study in Spain, Germany, and the Netherlands JO - JMIR Form Res SP - e48335 VL - 7 KW - mHealth adoption KW - discrete choice task KW - mobile apps KW - self-monitoring KW - willingness KW - mobile health app KW - mobile app KW - mobile health KW - mHealth KW - adoption KW - mHealth tools KW - health care cost KW - effectiveness KW - mobile phone N2 - Background: Despite the worldwide growth in mobile health (mHealth) tools and the possible benefits for both patients and health care providers, the adoption of mHealth is low, and only a limited number of studies have examined the intention to download mHealth apps. Objective: In this study, we investigated individuals? preferences in the adoption of a health app. Methods: We conducted a discrete choice experimental study in 3 countries (Spain: n=800, Germany: n=800, and the Netherlands: n=416) with 4 different attributes and levels (ie, price: ?1.99 vs ?4.99 [a currency exchange rate of ?1=US $1.09 is applicable] vs for free, data protection: data protection vs no information, recommendation: patients? association vs doctors, and manufacturer: medical association vs pharmaceutical company). Participants were randomly assigned. For the analyses, we used the conditional logistic model separately for each country. Results: The results showed that price and data protection were considered important factors that significantly increased the probability to download an mHealth app. In general, the source of the recommendation and the manufacturer affected the probability to download the mHealth app less. However, in Germany and the Netherlands, we found that if the app was manufactured by a pharmaceutical company, the probability to download the mHealth app decreased. Conclusions: mHealth tools are highly promising to reduce health care costs and increase the effectiveness of traditional health interventions and therapies. Improving data protection, reducing costs, and creating sound business models are the major driving forces to increase the adoption of mHealth apps in the future. It is thereby essential to create trustworthy standards for mobile apps, whereby prices, legislation concerning data protection, and health professionals can have a leading role to inform the potential consumers. UR - https://formative.jmir.org/2023/1/e48335 UR - http://dx.doi.org/10.2196/48335 UR - http://www.ncbi.nlm.nih.gov/pubmed/38145470 ID - info:doi/10.2196/48335 ER - TY - JOUR AU - Ulin, Lindsey AU - Bernstein, A. Simone AU - Nunes, C. Julio AU - Gu, Alex AU - Hammoud, M. Maya AU - Gold, A. Jessica AU - Mirza, M. Kamran PY - 2023/12/25 TI - Improving Transparency in the Residency Application Process: Survey Study JO - JMIR Form Res SP - e45919 VL - 7 KW - data elaboration KW - information transparency KW - medical school KW - residency application KW - residency programs KW - resident N2 - Background: Increasing numbers of residency applications create challenges for applicants and residency programs to assess if they are a good fit during the residency application and match process. Applicants face limited or conflicting information as they assess programs, leading to overapplying. A holistic review of residency applications is considered a gold standard for programs, but the current volumes and associated time constraints leave programs relying on numerical filters, which do not predict success in residency. Applicants could benefit from increased transparency in the residency application process. Objective: This study aims to determine the information applicants find most beneficial from residency programs when deciding where to apply, by type of medical school education background. Methods: Match 2023 applicants voluntarily completed an anonymous survey through the Twitter and Instagram social media platforms. We asked the respondents to select 3 top factors from a multiple-choice list of what information they would like from residency programs to help determine if the characteristics of their application align with program values. We examined differences in helpful factors selected by medical school backgrounds using ANOVA. Results: There were 4649 survey respondents. When responses were analyzed by United States-allopathic (US-MD), doctor of osteopathic medicine (DO), and international medical graduate (IMG) educational backgrounds, respondents chose different factors as most helpful: minimum United States Medical Licensing Examination (USMLE) or Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Step 2 scores (565/3042, 18.57% US-MD; 485/3042, 15.9% DO; and 1992/3042, 65.48% IMG; P<.001), resident hometown region (281/1132, 24.82% US-MD; 189/1132, 16.7% DO; and 662/1132, 58.48% IMG; P=.02), resident medical school region (476/2179, 22% US-MD; 250/2179, 11.5% DO; and 1453/2179, 66.7% IMG; P=.002), and percent of residents or attendings underrepresented in medicine (417/1815, 22.98% US-MD; 158/1815, 8.71% DO; and 1240/1815, 68.32% IMG; P<.001). Conclusions: When applying to residency programs, this study found that the factors that respondents consider most helpful from programs in deciding where to apply differ by educational background. Across all educational groups, respondents want transparency around standardized exam scores, geography, and the racial or ethnic backgrounds of residents and attendings. UR - https://formative.jmir.org/2023/1/e45919 UR - http://dx.doi.org/10.2196/45919 UR - http://www.ncbi.nlm.nih.gov/pubmed/38145482 ID - info:doi/10.2196/45919 ER - TY - JOUR AU - Schapranow, Matthieu-P AU - Bayat, Mozhgan AU - Rasheed, Aadil AU - Naik, Marcel AU - Graf, Verena AU - Schmidt, Danilo AU - Budde, Klemens AU - Cardinal, Héloïse AU - Sapir-Pichhadze, Ruth AU - Fenninger, Franz AU - Sherwood, Karen AU - Keown, Paul AU - Günther, P. Oliver AU - Pandl, D. Konstantin AU - Leiser, Florian AU - Thiebes, Scott AU - Sunyaev, Ali AU - Niemann, Matthias AU - Schimanski, Andreas AU - Klein, Thomas PY - 2023/12/22 TI - NephroCAGE?German-Canadian Consortium on AI for Improved Kidney Transplantation Outcome: Protocol for an Algorithm Development and Validation Study JO - JMIR Res Protoc SP - e48892 VL - 12 KW - posttransplant risks KW - kidney transplantation KW - federated learning infrastructure KW - clinical prediction model KW - donor-recipient matching KW - multinational transplant data set N2 - Background: Recent advances in hardware and software enabled the use of artificial intelligence (AI) algorithms for analysis of complex data in a wide range of daily-life use cases. We aim to explore the benefits of applying AI to a specific use case in transplant nephrology: risk prediction for severe posttransplant events. For the first time, we combine multinational real-world transplant data, which require specific legal and technical protection measures. Objective: The German-Canadian NephroCAGE consortium aims to develop and evaluate specific processes, software tools, and methods to (1) combine transplant data of more than 8000 cases over the past decades from leading transplant centers in Germany and Canada, (2) implement specific measures to protect sensitive transplant data, and (3) use multinational data as a foundation for developing high-quality prognostic AI models. Methods: To protect sensitive transplant data addressing the first and second objectives, we aim to implement a decentralized NephroCAGE federated learning infrastructure upon a private blockchain. Our NephroCAGE federated learning infrastructure enables a switch of paradigms: instead of pooling sensitive data into a central database for analysis, it enables the transfer of clinical prediction models (CPMs) to clinical sites for local data analyses. Thus, sensitive transplant data reside protected in their original sites while the comparable small algorithms are exchanged instead. For our third objective, we will compare the performance of selected AI algorithms, for example, random forest and extreme gradient boosting, as foundation for CPMs to predict severe short- and long-term posttransplant risks, for example, graft failure or mortality. The CPMs will be trained on donor and recipient data from retrospective cohorts of kidney transplant patients. Results: We have received initial funding for NephroCAGE in February 2021. All clinical partners have applied for and received ethics approval as of 2022. The process of exploration of clinical transplant database for variable extraction has started at all the centers in 2022. In total, 8120 patient records have been retrieved as of August 2023. The development and validation of CPMs is ongoing as of 2023. Conclusions: For the first time, we will (1) combine kidney transplant data from nephrology centers in Germany and Canada, (2) implement federated learning as a foundation to use such real-world transplant data as a basis for the training of CPMs in a privacy-preserving way, and (3) develop a learning software system to investigate population specifics, for example, to understand population heterogeneity, treatment specificities, and individual impact on selected posttransplant outcomes. International Registered Report Identifier (IRRID): DERR1-10.2196/48892 UR - https://www.researchprotocols.org/2023/1/e48892 UR - http://dx.doi.org/10.2196/48892 UR - http://www.ncbi.nlm.nih.gov/pubmed/38133915 ID - info:doi/10.2196/48892 ER - TY - JOUR AU - Choi, JiYeon AU - Choi, Seongmi AU - Song, Kijun AU - Baek, Jiwon AU - Kim, Heejung AU - Choi, Mona AU - Kim, Yesol AU - Chu, Hui Sang AU - Shin, Jiyoung PY - 2023/12/14 TI - Everyday Digital Literacy Questionnaire for Older Adults: Instrument Development and Validation Study JO - J Med Internet Res SP - e51616 VL - 25 KW - aging KW - older adults KW - digital literacy KW - instrument KW - validation KW - psychometrics KW - European Commission?s Digital Competence framework N2 - Background: The need for digital literacy in aging populations is increasing in the digitalizing society. Digital literacy involves the identification, evaluation, and communication of information through various digital devices or relevant programs. Objective: The aims of this study were to develop an Everyday Digital Literacy Questionnaire (EDLQ), a digital literacy assessment scale, and subsequently evaluate its psychometric properties using a population of community-dwelling older adults in South Korea. Methods: The EDLQ was developed using an instrument development design. A nationwide survey was conducted, and the study included 1016 community-dwelling older adults (age ?60 years). To evaluate the psychometric properties, the participants were randomly divided into 2 groups (n=508 each), and the internal consistency (Cronbach ? and McDonald ?), structural validity (exploratory factor analysis and confirmatory factor analysis), hypothesis-testing construct validity using the eHealth Literacy Scale (eHEALS), and measurement invariance were analyzed. Results: Among the initial 30 items of the EDLQ, 22 items with a 3-factor solution had a total explained variance of 77%. The domains included ?information and communication? (9 items), ?content creation and management? (4 items), and ?safety and security? (9 items). Confirmatory factor analysis was conducted with this 3-factor solution (?2206=345.1; normed ?2206=1.7; comparative fit index=0.997; Tucker-Lewis index=0.997; root-mean-square error of approximation=0.036; standardized root-mean-square residual=0.050; composite reliability=0.903-0.959; average variance extracted=0.699-0.724; R2=0.616-0.773). Hypothesis-testing construct validity with the eHEALS revealed a strong correlation (r=0.75). Cronbach ? and McDonald ? coefficients were .98 and 0.98, respectively. The fit indices for measurement invariance, including the configural, metric, and scalar invariance models, demonstrated a satisfactory fit to the data. Our findings suggest that the psychometric properties of the 22-item EDLQ are valid and reliable for assessing digital literacy among older Korean adults. Conclusions: In this study, we developed a digital literacy measure with strong psychometric properties that made it suitable for assessing the digital literacy of community-dwelling older adults in Korea. To broaden its applicability, however, further assessment of its feasibility for use with different languages and cultures is necessary. Moreover, more empirical research on digital literacy and related factors in older adults can facilitate the development of personalized digital health care services and educational interventions in the digital society. UR - https://www.jmir.org/2023/1/e51616 UR - http://dx.doi.org/10.2196/51616 UR - http://www.ncbi.nlm.nih.gov/pubmed/38095999 ID - info:doi/10.2196/51616 ER - TY - JOUR AU - Rahmadhan, Putra Muhamad Adhytia Wana AU - Handayani, Wuri Putu PY - 2023/12/14 TI - Integrated Immunization Information System in Indonesia: Prototype Design Using Quantitative and Qualitative Data JO - JMIR Form Res SP - e53132 VL - 7 KW - vaccination KW - immunization KW - immunization information system KW - high-fidelity prototype KW - Indonesia N2 - Background: As the volume of immunization records increases, problems with fragmented records arise, especially since the majority of records in developing countries, including Indonesia, remain paper based. Implementing an immunization information system (IIS) offers a solution to this problem. Objective: In this study, we designed an integrated IIS prototype in Indonesia using the design science research (DSR) methodology. Methods: The stages of the DSR methodology followed in this study included identifying problems and motivating and defining objectives for a solution, design and development, demonstration, evaluation, communication, and drawing conclusions and suggestions. Specifically, this study began with problem formulation and a literature review. We then applied quantitative (questionnaire with 305 members of the public) and qualitative (interviews with 15 health workers including nurses, midwives, and doctors) data collection approaches. Results: The resulting high-fidelity prototype follows the 8 golden rules. There are 2 IIS designs, one for the public as immunization recipients and another for health workers. The functionalities include immunization history, schedule, recommendations, verification, certificates, reminders and recalls, coverage, monitoring, news, and reports of adverse events. Evaluation of the prototype was carried out through interviews and a questionnaire designed according to the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). The SUS value was 72.5 or ?Good (Acceptable),? while the system usefulness, information quality, interface quality, and overall value on the PSSUQ were 2.65, 2.94, 2.48, and 2.71, respectively, which indicate it has an effective design. Conclusions: This provides a guide for health facilities, health regulators, and health application developers on how to implement an integrated IIS in Indonesia. UR - https://formative.jmir.org/2023/1/e53132/ UR - http://dx.doi.org/10.2196/53132 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/53132 ER - TY - JOUR AU - Persson, Inger AU - Grünwald, Adam AU - Morvan, Ludivine AU - Becedas, David AU - Arlbrandt, Martin PY - 2023/12/14 TI - A Machine Learning Algorithm Predicting Acute Kidney Injury in Intensive Care Unit Patients (NAVOY Acute Kidney Injury): Proof-of-Concept Study JO - JMIR Form Res SP - e45979 VL - 7 KW - acute kidney injury KW - AKI KW - algorithm KW - early detection KW - electronic health records KW - ICU KW - intensive care unit KW - machine learning KW - nephrology KW - prediction KW - software as a medical device N2 - Background: Acute kidney injury (AKI) represents a significant global health challenge, leading to increased patient distress and financial health care burdens. The development of AKI in intensive care unit (ICU) settings is linked to prolonged ICU stays, a heightened risk of long-term renal dysfunction, and elevated short- and long-term mortality rates. The current diagnostic approach for AKI is based on late indicators, such as elevated serum creatinine and decreased urine output, which can only detect AKI after renal injury has transpired. There are no treatments to reverse or restore renal function once AKI has developed, other than supportive care. Early prediction of AKI enables proactive management and may improve patient outcomes. Objective: The primary aim was to develop a machine learning algorithm, NAVOY Acute Kidney Injury, capable of predicting the onset of AKI in ICU patients using data routinely collected in ICU electronic health records. The ultimate goal was to create a clinical decision support tool that empowers ICU clinicians to proactively manage AKI and, consequently, enhance patient outcomes. Methods: We developed the NAVOY Acute Kidney Injury algorithm using a hybrid ensemble model, which combines the strengths of both a Random Forest (Leo Breiman and Adele Cutler) and an XGBoost model (Tianqi Chen). To ensure the accuracy of predictions, the algorithm used 22 clinical variables for hourly predictions of AKI as defined by the Kidney Disease: Improving Global Outcomes guidelines. Data for algorithm development were sourced from the Massachusetts Institute of Technology Lab for Computational Physiology Medical Information Mart for Intensive Care IV clinical database, focusing on ICU patients aged 18 years or older. Results: The developed algorithm, NAVOY Acute Kidney Injury, uses 4 hours of input and can, with high accuracy, predict patients with a high risk of developing AKI 12 hours before onset. The prediction performance compares well with previously published prediction algorithms designed to predict AKI onset in accordance with Kidney Disease: Improving Global Outcomes diagnosis criteria, with an impressive area under the receiver operating characteristics curve (AUROC) of 0.91 and an area under the precision-recall curve (AUPRC) of 0.75. The algorithm?s predictive performance was externally validated on an independent hold-out test data set, confirming its ability to predict AKI with exceptional accuracy. Conclusions: NAVOY Acute Kidney Injury is an important development in the field of critical care medicine. It offers the ability to predict the onset of AKI with high accuracy using only 4 hours of data routinely collected in ICU electronic health records. This early detection capability has the potential to strengthen patient monitoring and management, ultimately leading to improved patient outcomes. Furthermore, NAVOY Acute Kidney Injury has been granted Conformite Europeenne (CE)?marking, marking a significant milestone as the first CE-marked AKI prediction algorithm for commercial use in European ICUs. UR - https://formative.jmir.org/2023/1/e45979 UR - http://dx.doi.org/10.2196/45979 UR - http://www.ncbi.nlm.nih.gov/pubmed/38096015 ID - info:doi/10.2196/45979 ER - TY - JOUR AU - Crespi, Elizabeth AU - Heller, Johanna AU - Hardesty, J. Jeffrey AU - Nian, Qinghua AU - Sinamo, K. Joshua AU - Welding, Kevin AU - Kennedy, David Ryan AU - Cohen, E. Joanna PY - 2023/12/13 TI - Exploring Different Incentive Structures Among US Adults Who Use e-Cigarettes to Optimize Retention in Longitudinal Web-Based Surveys: Case Study JO - J Med Internet Res SP - e49354 VL - 25 KW - incentive KW - conditional incentive KW - web-based survey KW - longitudinal study KW - follow-up KW - nicotine KW - e-cigarettes KW - tobacco KW - survey KW - retention KW - demographics KW - case study KW - optimization KW - adults N2 - Background: Longitudinal cohort studies are critical for understanding the evolution of health-influencing behaviors, such as e-cigarette use, over time. Optimizing follow-up rates in longitudinal studies is necessary for ensuring high-quality data with sufficient power for analyses. However, achieving high rates of follow-up in web-based longitudinal studies can be challenging, even when monetary incentives are provided. Objective: This study compares participant progress through a survey and demographics for 2 incentive structures (conditional and hybrid unconditional-conditional) among US adults using e-cigarettes to understand the optimal incentive structure. Methods: The data used in this study are from a web-based longitudinal cohort study (wave 4; July to September 2022) of US adults (aged 21 years or older) who use e-cigarettes ?5 days per week. Participants (N=1804) invited to the follow-up survey (median completion time=16 minutes) were randomly assigned into 1 of 2 incentive structure groups (n=902 each): (1) conditional (US $30 gift code upon survey completion) and (2) hybrid unconditional-conditional (US $15 gift code prior to survey completion and US $15 gift code upon survey completion). Chi-square tests assessed group differences in participant progress through 5 sequential stages of the survey (started survey, completed screener, deemed eligible, completed survey, and deemed valid) and demographics. Results: Of the 902 participants invited to the follow-up survey in each group, a higher proportion of those in the conditional (662/902, 73.4%) than the hybrid (565/902, 62.6%) group started the survey (P<.001). Of those who started the survey, 643 (97.1%) participants in the conditional group and 548 (97%) participants in the hybrid group completed the screener (P=.89), which was used each wave to ensure participants remained eligible. Of those who completed the screener, 555 (86.3%) participants in the conditional group and 446 (81.4%) participants in the hybrid group were deemed eligible for the survey (P=.02). Of those eligible, 514 (92.6%) participants from the conditional group and 401 (89.9%) participants from the hybrid group completed the survey and were deemed valid after data review (P=.14). Overall, more valid completions were yielded from the conditional (514/902, 57%) than the hybrid group (401/902, 44.5%; P<.001). Among those who validly completed the survey, no significant differences were found by group for gender, income, race, ethnicity, region, e-cigarette use frequency, past 30-day cigarette use, or number of waves previously completed. Conclusions: Providing a US $30 gift code upon survey completion yielded higher rates of survey starts and completions than providing a US $15 gift code both before and after survey completion. These 2 methods yielded participants with similar demographics, suggesting that one approach is not superior in obtaining a balanced sample. Based on this case study, future web-based surveys examining US adults using e-cigarettes could consider providing the full incentive upon completion of the survey. International Registered Report Identifier (IRRID): RR2-10.2196/38732 UR - https://www.jmir.org/2023/1/e49354 UR - http://dx.doi.org/10.2196/49354 UR - http://www.ncbi.nlm.nih.gov/pubmed/38090793 ID - info:doi/10.2196/49354 ER - TY - JOUR AU - Long, Chen AU - Zheng, Lin AU - Liu, Runhua AU - Duan, Zhongxian PY - 2023/12/13 TI - Structural Validation and Measurement Invariance Testing of the Chinese Version of the eHealth Literacy Scale Among Undergraduates: Cross-Sectional Study JO - J Med Internet Res SP - e48838 VL - 25 KW - eHealth literacy KW - eHEALS KW - factor structure KW - measurement invariance KW - undergraduates KW - health literacy KW - cross-sectional survey KW - digital health literacy KW - measurement N2 - Background: The eHealth Literacy Scale (eHEALS) was introduced in China in 2013 as one of the most important electronic health literacy measurement instruments. After a decade of development in China, it has received widespread attention, although its theoretical underpinnings have been challenged, thus demanding more robust research evidence of factorial validity and multigroup measurement properties. Objective: This study aimed to evaluate the Chinese version of the eHEALS in terms of its measurement properties. Methods: A cross-sectional survey was conducted in a university setting in China. Item statistics were checked for response distributions and floor and ceiling effects. Internal consistency reliability was confirmed with Cronbach ?, split-half reliability, Cronbach ? if an item was deleted, and item-total correlation. A total of 5 representative eHEALS factor structures were examined and contrasted using confirmatory factor analysis. The study used the item-level content validity index (I-CVI) and the average of the I-CVI scores of all items on the scale to assess the content validity of the dominance model. Furthermore, the validated dominance model was subsequently used to evaluate the relevance and representation of elements in the instrument and to assess measurement invariance across genders. Results: A total of 972 respondents were identified, with a Cronbach ? of .92, split-half reliability of 0.88, and item-total score correlation coefficients ranging from 0.715 to 0.781. Cronbach ? if an item was deleted showed that all items should be retained. Acceptable content validity was supported by I-CVIs ?0.80. The confirmatory factor analysis confirmed that the 3-factor model was acceptable. The measurement model met all relevant fit indices: average variance extracted from 0.663 to 0.680, composite reliability from 0.810 to 0.857, chi-square divided by the df of 4.768, root mean square error of approximation of 0.062, standardized root mean squared residual of 0.020, comparative fit index (CFI) of 0.987, and Tucker-Lewis index of 0.979. In addition, the scale demonstrated error variance invariance (?normed fit index=?0.016, ?incremental fit index=?0.012, ?Tucker-Lewis index=0.005, ?comparative fit index=?0.012, ?relative fit index=0.005, and ?root mean square error of approximation=0.005). Conclusions: A 3-factor model of the Chinese version of the eHEALS fits best, and our findings provide evidence for the strict measurement invariance of the instrument regarding gender. UR - https://www.jmir.org/2023/1/e48838 UR - http://dx.doi.org/10.2196/48838 UR - http://www.ncbi.nlm.nih.gov/pubmed/37990370 ID - info:doi/10.2196/48838 ER - TY - JOUR AU - Deschênes, Marie-France AU - Fernandez, Nicolas AU - Lechasseur, Kathleen AU - Caty, Marie-Ève AU - Azimzadeh, Dina AU - Mai, Tue-Chieu AU - Lavoie, Patrick PY - 2023/12/13 TI - Transformation and Articulation of Clinical Data to Understand Students? and Health Professionals? Clinical Reasoning: Protocol for a Scoping Review JO - JMIR Res Protoc SP - e50797 VL - 12 KW - clinical reasoning KW - semantic qualifiers KW - discourse KW - linguistics KW - education KW - natural language processing KW - scoping review KW - clinical data KW - educational strategy KW - student KW - health care professional KW - semantic transformation N2 - Background: There are still unanswered questions regarding effective educational strategies to promote the transformation and articulation of clinical data while teaching and learning clinical reasoning. Additionally, understanding how this process can be analyzed and assessed is crucial, particularly considering the rapid growth of natural language processing in artificial intelligence. Objective: The aim of this study is to map educational strategies to promote the transformation and articulation of clinical data among students and health care professionals and to explore the methods used to assess these individuals? transformation and articulation of clinical data. Methods: This scoping review follows the Joanna Briggs Institute framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist for the analysis. A literature search was performed in November 2022 using 5 databases: CINAHL (EBSCOhost), MEDLINE (Ovid), Embase (Ovid), PsycINFO (Ovid), and Web of Science (Clarivate). The protocol was registered on the Open Science Framework in November 2023. The scoping review will follow the 9-step framework proposed by Peters and colleagues of the Joanna Briggs Institute. A data extraction form has been developed using key themes from the research questions. Results: After removing duplicates, the initial search yielded 6656 results, and study selection is underway. The extracted data will be qualitatively analyzed and presented in a diagrammatic or tabular form alongside a narrative summary. The review will be completed by February 2024. Conclusions: By synthesizing the evidence on semantic transformation and articulation of clinical data during clinical reasoning education, this review aims to contribute to the refinement of educational strategies and assessment methods used in academic and continuing education programs. The insights gained from this review will help educators develop more effective semantic approaches for teaching or learning clinical reasoning, as opposed to fragmented, purely symptom-based or probabilistic approaches. Besides, the results may suggest some ways to address challenges related to the assessment of clinical reasoning and ensure that the assessment tasks accurately reflect learners? developing competencies and educational progress. International Registered Report Identifier (IRRID): DERR1-10.2196/50797 UR - https://www.researchprotocols.org/2023/1/e50797 UR - http://dx.doi.org/10.2196/50797 UR - http://www.ncbi.nlm.nih.gov/pubmed/38090795 ID - info:doi/10.2196/50797 ER - TY - JOUR AU - Scheibein, Florian AU - Caballeria, Elsa AU - Taher, Abu Md AU - Arya, Sidharth AU - Bancroft, Angus AU - Dannatt, Lisa AU - De Kock, Charlotte AU - Chaudhary, Idrees Nazish AU - Gayo, Perez Roberto AU - Ghosh, Abhishek AU - Gelberg, Lillian AU - Goos, Cees AU - Gordon, Rebecca AU - Gual, Antoni AU - Hill, Penelope AU - Jeziorska, Iga AU - Kurcevi?, Eliza AU - Lakhov, Aleksey AU - Maharjan, Ishwor AU - Matrai, Silvia AU - Morgan, Nirvana AU - Paraskevopoulos, Ilias AU - Puhari?, Zrinka AU - Sibeko, Goodman AU - Stola, Jan AU - Tiburcio, Marcela AU - Tay Wee Teck, Joseph AU - Tsereteli, Zaza AU - López-Pelayo, Hugo PY - 2023/12/12 TI - Optimizing Digital Tools for the Field of Substance Use and Substance Use Disorders: Backcasting Exercise JO - JMIR Hum Factors SP - e46678 VL - 10 KW - substance use KW - substance use disorders KW - addictions KW - telemedicine KW - eHealth KW - digital tools KW - backcasting exercise KW - drug addiction KW - ethical frameworks KW - digital health N2 - Background: Substance use trends are complex; they often rapidly evolve and necessitate an intersectional approach in research, service, and policy making. Current and emerging digital tools related to substance use are promising but also create a range of challenges and opportunities. Objective: This paper reports on a backcasting exercise aimed at the development of a roadmap that identifies values, challenges, facilitators, and milestones to achieve optimal use of digital tools in the substance use field by 2030. Methods: A backcasting exercise method was adopted, wherein the core elements are identifying key values, challenges, facilitators, milestones, cornerstones and a current, desired, and future scenario. A structured approach was used by means of (1) an Open Science Framework page as a web-based collaborative working space and (2) key stakeholders? collaborative engagement during the 2022 Lisbon Addiction Conference. Results: The identified key values were digital rights, evidence-based tools, user-friendliness, accessibility and availability, and person-centeredness. The key challenges identified were ethical funding, regulations, commercialization, best practice models, digital literacy, and access or reach. The key facilitators identified were scientific research, interoperable infrastructure and a culture of innovation, expertise, ethical funding, user-friendly designs, and digital rights and regulations. A range of milestones were identified. The overarching identified cornerstones consisted of creating ethical frameworks, increasing access to digital tools, and continuous trend analysis. Conclusions: The use of digital tools in the field of substance use is linked to a range of risks and opportunities that need to be managed. The current trajectories of the use of such tools are heavily influenced by large multinational for-profit companies with relatively little involvement of key stakeholders such as people who use drugs, service providers, and researchers. The current funding models are problematic and lack the necessary flexibility associated with best practice business approaches such as lean and agile principles to design and execute customer discovery methods. Accessibility and availability, digital rights, user-friendly design, and person-focused approaches should be at the forefront in the further development of digital tools. Global legislative and technical infrastructures by means of a global action plan and strategy are necessary and should include ethical frameworks, accessibility of digital tools for substance use, and continuous trend analysis as cornerstones. UR - https://humanfactors.jmir.org/2023/1/e46678 UR - http://dx.doi.org/10.2196/46678 UR - http://www.ncbi.nlm.nih.gov/pubmed/38085569 ID - info:doi/10.2196/46678 ER - TY - JOUR AU - Liu, Jiaxing AU - Gupta, Shalini AU - Chen, Aipeng AU - Wang, Chen-Kai AU - Mishra, Pratik AU - Dai, Hong-Jie AU - Wong, Shui-Yee Zoie AU - Jonnagaddala, Jitendra PY - 2023/12/6 TI - OpenDeID Pipeline for Unstructured Electronic Health Record Text Notes Based on Rules and Transformers: Deidentification Algorithm Development and Validation Study JO - J Med Internet Res SP - e48145 VL - 25 KW - deidentification KW - scrubbing KW - anonymization KW - surrogate generation KW - unstructured EHRs KW - electronic health records KW - BERT KW - Bidirectional Encoder Representations from Transformers N2 - Background: Electronic health records (EHRs) in unstructured formats are valuable sources of information for research in both the clinical and biomedical domains. However, before such records can be used for research purposes, sensitive health information (SHI) must be removed in several cases to protect patient privacy. Rule-based and machine learning?based methods have been shown to be effective in deidentification. However, very few studies investigated the combination of transformer-based language models and rules. Objective: The objective of this study is to develop a hybrid deidentification pipeline for Australian EHR text notes using rules and transformers. The study also aims to investigate the impact of pretrained word embedding and transformer-based language models. Methods: In this study, we present a hybrid deidentification pipeline called OpenDeID, which is developed using an Australian multicenter EHR-based corpus called OpenDeID Corpus. The OpenDeID corpus consists of 2100 pathology reports with 38,414 SHI entities from 1833 patients. The OpenDeID pipeline incorporates a hybrid approach of associative rules, supervised deep learning, and pretrained language models. Results: The OpenDeID achieved a best F1-score of 0.9659 by fine-tuning the Discharge Summary BioBERT model and incorporating various preprocessing and postprocessing rules. The OpenDeID pipeline has been deployed at a large tertiary teaching hospital and has processed over 8000 unstructured EHR text notes in real time. Conclusions: The OpenDeID pipeline is a hybrid deidentification pipeline to deidentify SHI entities in unstructured EHR text notes. The pipeline has been evaluated on a large multicenter corpus. External validation will be undertaken as part of our future work to evaluate the effectiveness of the OpenDeID pipeline. UR - https://www.jmir.org/2023/1/e48145 UR - http://dx.doi.org/10.2196/48145 UR - http://www.ncbi.nlm.nih.gov/pubmed/38055317 ID - info:doi/10.2196/48145 ER - TY - JOUR AU - Lai, Byron AU - Young, Raven AU - Craig, Mary AU - Chaviano, Kelli AU - Swanson-Kimani, Erin AU - Wozow, Cynthia AU - Davis, Drew AU - Rimmer, H. James PY - 2023/12/6 TI - Improving Social Isolation and Loneliness Among Adolescents With Physical Disabilities Through Group-Based Virtual Reality Gaming: Feasibility Pre-Post Trial Study JO - JMIR Form Res SP - e47630 VL - 7 KW - therapy KW - mindfulness KW - play KW - friend KW - friends KW - friendship KW - lonely KW - loneliness KW - psychotherapy KW - peer KW - peers KW - recreation KW - disability KW - adolescent KW - adolescents KW - disabled KW - physical disability KW - digital mental health intervention KW - youth KW - young adult KW - virtual reality KW - VR KW - gaming KW - depression KW - depressive KW - mental health KW - social KW - isolated KW - isolation KW - socialize KW - socializing KW - socialization KW - interaction KW - interactions KW - acceptability KW - game KW - games KW - exergame KW - exergames KW - exergaming N2 - Background: Adolescents with disabilities experience alarmingly higher rates of depression and isolation than peers without disabilities. There is a need to identify interventions that can improve mental health and isolation among this underserved population. Innovations in virtual reality (VR) gaming ?standalone? headsets allow greater access to immersive high-quality digital experiences, due to their relatively low cost. Objective: This study had three purposes, which were to (1) examine the preliminary effects of a low-cost, home-based VR multiplayer recreation and socialization on depression, socialization, and loneliness; (2) quantify the acceptability of the program as measured by participant adherence, total play time, and exercise time; and (3) identify and describe behavioral mechanisms that affected participant engagement. Methods: This was a single-group, pre- to postdesign trial. The intervention was conducted at home. Participants were recruited from a children?s hospital. The intervention lasted 4 weeks and included 2×1-hour sessions per week of supervised peer-to-peer gaming. Participants used the Meta Quest 2 headset to meet peers and 2 coaches in a private party held digitally. Aim 1 was evaluated with the Children?s Depression Inventory 2 Short Form and the University of California, Los Angeles Loneliness Scale 20 items, which are measures of social isolation and loneliness, respectively. Aim 2 was evaluated through the following metrics: participant adherence, the types of games played, friendship building and playtime, and program satisfaction and enjoyment. Results: In total, 12 people enrolled (mean age 16.6, SD 1.8 years; male: n=9 and female: n=3), and 8 people completed the program. Mean attendance for the 8 participants was 77% (49 sessions of 64 total possible sessions; mean 6, SD 2 sessions). A trend was observed for improved Children?s Depression Inventory 2 Short Form scores (mean preintervention score 7.25, SD 4.2; mean postintervention score 5.38, SD 4.1; P=.06; effect size=0.45, 95% CI ?0.15 to 3.9), but this was not statistically significant; no difference was observed for University of California, Los Angeles Loneliness Scale 20 items scores. Most participants (7/8, 88%) stated that they became friends with a peer in class; 50% (4/8) reported that they played with other people. Participants reported high levels of enjoyment and satisfaction with how the program was implemented. Qualitative analysis resulted in 4 qualitative themes that explained behavioral mechanisms that determined engagement in the program. Conclusions: The study findings demonstrated that a brief VR group program could be valuable for potentially improving mental health among adolescents with physical disabilities. Participants built friendships with peers and other players on the web, using low-cost consumer equipment that provided easy access and strong scale-up potential. Study findings identified factors that can be addressed to enhance the program within a larger clinical trial. Trial Registration: ClinicalTrials.gov NCT05259462; https://clinicaltrials.gov/study/NCT05259462 International Registered Report Identifier (IRRID): RR2-10.2196/42651 UR - https://formative.jmir.org/2023/1/e47630 UR - http://dx.doi.org/10.2196/47630 UR - http://www.ncbi.nlm.nih.gov/pubmed/38055309 ID - info:doi/10.2196/47630 ER - TY - JOUR AU - Petcu, Catalina AU - Boukhelif, Ikram AU - Davis, Veena AU - Shamsi, Hamda AU - Al-Assi, Marwa AU - Miladi, Anis AU - Khaled, M. Salma PY - 2023/11/27 TI - Design and Implementation of Survey Quality Control System for Qatar?s First National Mental Health Survey: Case Study JO - JMIR Form Res SP - e37653 VL - 7 KW - World Mental Health Survey KW - quality control indicators KW - Middle East KW - phone interview KW - case study KW - COVID-19 N2 - Background: All World Mental Health (WMH) Surveys apply high standards of data quality. To date, most of the published quality control (QC) procedures for these surveys were in relation to face-to-face interviews. However, owing to the social restrictions that emerged from the COVID-19 pandemic, telephone interviews are the most effective alternative for conducting complex probability-based large-scale surveys. Objective: In this paper, we present the QC system implemented in the WMH Qatar Survey, the first WMH Survey conducted during the COVID-19 pandemic in the Middle East. The objective of the QC process was to acquire high data quality through the reduction of random errors and bias in data collection. Methods: The QC design and procedures in this study were adapted to the telephone survey mode in response to the COVID-19 pandemic. We focus on the design of the QC indicator system and its implementation, including the investigation process, monitoring interviewers? performance during survey fielding and applying quality-informed interventions. Results: The study team investigated 11,035 flags triggered during the 2 waves of the survey data collection. The most triggered flags were related to short question administration duration and multiple visits to the same survey questions or responses. Live monitoring of the interviews helped in understanding why certain duration-related flags were triggered and the interviewing patterns of the interviewers. Corrective and preventive actions were taken against interviewers? behaviors based on the investigation of triggered flags per interviewer and live call monitoring of interviews. Although, in most cases, the interviewers required refresher training sessions and feedback to improve their performance, several interviewers discontinued work because of low productivity and a high number of triggered flags. Conclusions: The specific QC procedures implemented in the course of the WMH Qatar Survey were essential for successfully meeting the target number of interviews (N=5000). The QC strategies and the new indicators customized for telephone interviews contributed to the flag investigation and verification process. The QC data presented in this study shed light on the rigorous methods and quality monitoring processes in the course of conducting a large-scale national survey on sensitive topics during the COVID-19 pandemic. UR - https://formative.jmir.org/2023/1/e37653 UR - http://dx.doi.org/10.2196/37653 UR - http://www.ncbi.nlm.nih.gov/pubmed/37906213 ID - info:doi/10.2196/37653 ER - TY - JOUR AU - Estevez, Mégane AU - Domecq, Sandrine AU - Montagni, Ilaria AU - Ramel, Viviane PY - 2023/11/24 TI - Evaluating a Public Health Information Service According to Users? Socioeconomic Position and Health Status: Protocol for a Cross-Sectional Study JO - JMIR Res Protoc SP - e51123 VL - 12 KW - internet KW - health information seeking KW - literacy KW - social health inequalities KW - evaluation KW - digital health KW - public health KW - socioeconomic position KW - health status KW - user KW - empowerment KW - social inequality KW - mobile phone N2 - Background: The increasing use of information technology in the field of health is supposed to promote users? empowerment but can also reinforce social inequalities. Some health authorities in various countries have developed mechanisms to offer accurate and relevant information to health care system users, often through health websites. However, the evaluation of these sociotechnical tools is inadequate, particularly with respect to differences and inequalities in use by social groups. Objective: Our study aims to evaluate the access, understanding, appraisal, and use of the French website Santé.fr by users according to their socioeconomic position and perceived health status. Methods: This cross-sectional study involves the entire French population to which Santé.fr is offered. Data will be collected through mixed methods, including a web-based questionnaire for quantitative data and interviews and focus groups for qualitative data. Collected data will cover users? access, understanding, appraisal, and use of Santé.fr, as well as sociodemographic and socioeconomic characteristics, health status, and digital health literacy. A validation of the dimensions of access, understanding, appraisal, and use of Santé.fr will be conducted, followed by principal component analysis and ascendant hierarchical classification based on the 2 main components of principal component analysis to characterize homogeneous users? profiles. Regression models will be used to investigate the relationships between each dimension and socioeconomic position and health status variables. NVivo 11 software (Lumivero) will be used to categorize interviewees? comments into preidentified themes or themes emerging from the discourse and compare them with the comments of various types of interviewees to understand the factors influencing people?s access, understanding, appraisal, and use of Santé.fr. Results: Recruitment is scheduled to begin in January 2024 and will conclude when the required number of participants is reached. Data collection is expected to be finalized approximately 7 months after recruitment, with the final data analysis programmed to be completed around December 2024. Conclusions: This study would be the first in France and in Europe to evaluate a public health information service, in this case the Santé.fr website (the official website of the French Ministry of Health), according to users? socioeconomic position and health status. The study could discover issues related to inequalities in access to, and the use of, digital technologies for obtaining health information on the internet. Given that access to health information on the internet is crucial for health decision-making and empowerment, inequalities in access may have subsequent consequences on health inequalities among social categories. Therefore, it is important to ensure that all social categories have access to Santé.fr. International Registered Report Identifier (IRRID): PRR1-10.2196/51123 UR - https://www.researchprotocols.org/2023/1/e51123 UR - http://dx.doi.org/10.2196/51123 UR - http://www.ncbi.nlm.nih.gov/pubmed/37999943 ID - info:doi/10.2196/51123 ER - TY - JOUR AU - Weermeijer, Merlijn Jeroen Dennis AU - Wampers, Martien AU - de Thurah, Lena AU - Bonnier, Rafaël AU - Piot, Maarten AU - Kuppens, Peter AU - Myin-Germeys, Inez AU - Kiekens, Glenn PY - 2023/11/21 TI - Usability of the Experience Sampling Method in Specialized Mental Health Care: Pilot Evaluation Study JO - JMIR Form Res SP - e48821 VL - 7 KW - experience sampling KW - ecological momentary assessment KW - implementation KW - digital mental health KW - mobile phone N2 - Background: Mental health problems occur in interactions in daily life. Yet, it is challenging to bring contextual information into the therapy room. The experience sampling method (ESM) may facilitate this by assessing clients? thoughts, feelings, symptoms, and behaviors as they are experienced in everyday life. However, the ESM is still primarily used in research settings, with little uptake in clinical practice. One aspect that may facilitate clinical implementation concerns the use of ESM protocols, which involves providing practitioners with ready-to-use ESM questionnaires, sampling schemes, visualizations, and training. Objective: This pilot study?s objective was to evaluate the usability of an ESM protocol for using the ESM in a specialized mental health care setting. Methods: We created the ESM protocol using the m-Path software platform and tested its usability in clinical practice. The ESM protocol consists of a dashboard for practitioners (ie, including the setup of the template and data visualizations) and an app for clients (ie, for completing the ESM questionnaires). A total of 8 practitioners and 17 clients used the ESM in practice between December 1, 2020, and July 31, 2021. Usability was assessed using questionnaires, ESM compliance rates, and semistructured interviews. Results: The usability was overall rated reasonable to good by practitioners (mean scores of usability items ranging from 5.33, SD 0.91, to 6.06, SD 0.73, on a scale ranging from 1 to 7). However, practitioners expressed difficulty in personalizing the template and reported insufficient guidelines on how to use the ESM in clinical practice. On average, clients completed 55% (SD 25%) of the ESM questionnaires. They rated the usability as reasonable to good, but their scores were slightly lower and more variable than those of the practitioners (mean scores of usability items ranging from 4.18, SD 1.70, to 5.94, SD 1.50 on a scale ranging from 1 to 7). Clients also voiced several concerns over the piloted ESM template, with some indicating no interest in the continued use of the ESM. Conclusions: The findings suggest that using an ESM protocol may facilitate the implementation of the ESM as a mobile health assessment tool in psychiatry. However, additional adaptions should be made before further implementation. Adaptions include providing training on personalizing questionnaires, adding additional sampling scheme formats as well as an open-text field, and creating a dynamic data visualization interface. Future studies should also identify factors determining the suitability of the ESM for specific treatment goals among different client populations. UR - https://formative.jmir.org/2023/1/e48821 UR - http://dx.doi.org/10.2196/48821 UR - http://www.ncbi.nlm.nih.gov/pubmed/37988137 ID - info:doi/10.2196/48821 ER - TY - JOUR AU - Yahya, Gezan AU - O'Keefe, B. James AU - Moore, A. Miranda PY - 2023/11/16 TI - Comparing a Data Entry Tool to Provider Insights Alone for Assessment of COVID-19 Hospitalization Risk: Pilot Matched Cohort Comparison Study JO - JMIR Form Res SP - e44250 VL - 7 KW - COVID-19 KW - risk assessment KW - hospitalization KW - outpatient KW - telemedicine KW - data KW - tool KW - risk KW - assessment KW - utilization KW - algorithm KW - symptoms KW - disease KW - community KW - patient KW - decision making tool KW - risk algorithm N2 - Background: In March 2020, the World Health Organization declared COVID-19 a global pandemic, necessitating an understanding of factors influencing severe disease outcomes. High COVID-19 hospitalization rates underscore the need for robust risk prediction tools to determine estimated risk for future hospitalization for outpatients with COVID-19. We introduced the ?COVID-19 Risk Tier Assessment Tool? (CRTAT), designed to enhance clinical decision-making for outpatients. Objective: We investigated whether CRTAT offers more accurate risk tier assignments (RTAs) than medical provider insights alone. Methods: We assessed COVID-19?positive patients enrolled at Emory Healthcare's Virtual Outpatient Management Clinic (VOMC)?a telemedicine monitoring program, from May 27 through August 24, 2020?who were not hospitalized at the time of enrollment. The primary analysis included patients from this program, who were later hospitalized due to COVID-19. We retroactively formed an age-, gender-, and risk factor?matched group of nonhospitalized patients for comparison. Data extracted from clinical notes were entered into CRTAT. We used descriptive statistics to compare RTAs reported by algorithm?trained health care providers and those produced by CRTAT. Results: Our patients were primarily younger than 60 years (67% hospitalized and 71% nonhospitalized). Moderate risk factors were prevalent (hospitalized group: 1 among 11, 52% patients; 2 among 2, 10% patients; and ?3 among 4, 19% patients; nonhospitalized group: 1 among 11, 52% patients, 2 among 5, 24% patients, and ?3 among 4, 19% patients). High risk factors were prevalent in approximately 45% (n=19) of the sample (hospitalized group: 11, 52% patients; nonhospitalized: 8, 38% patients). Approximately 83% (n=35) of the sample reported nonspecific symptoms, and the symptoms were generally mild (hospitalized: 12, 57% patients; nonhospitalized: 14, 67% patients). Most patient visits were seen within the first 1-6 days of their illness (n=19, 45%) with symptoms reported as stable over this period (hospitalized: 7, 70% patients; nonhospitalized: 3, 33% patients). Of 42 matched patients (hospitalized: n=21; nonhospitalized: n=21), 26 had identical RTAs and 16 had discrepancies between VOMC providers and CRTAT. Elements that led to different RTAs were as follows: (1) the provider ?missed? comorbidity (n=6), (2) the provider noted comorbidity but undercoded risk (n=10), and (3) the provider miscoded symptom severity and course (n=7). Conclusions: CRTAT, a point-of-care data entry tool, more accurately categorized patients into risk tiers (particularly those hospitalized), underscored by its ability to identify critical factors in patient history and clinical status. Clinical decision-making regarding patient management, resource allocation, and treatment plans could be enhanced by using similar risk assessment data entry tools for other disease states, such as influenza and community-acquired pneumonia. The COVID-19 pandemic has accelerated the adoption of telemedicine, enabling remote patient tools such as CRTAT. Future research should explore the long-term impact of outpatient clinical risk assessment tools and their contribution to better patient care. UR - https://formative.jmir.org/2023/1/e44250 UR - http://dx.doi.org/10.2196/44250 UR - http://www.ncbi.nlm.nih.gov/pubmed/37903299 ID - info:doi/10.2196/44250 ER - TY - JOUR AU - von Wulffen, Clemens AU - Marciniak, Anna Marta AU - Rohde, Judith AU - Kalisch, Raffael AU - Binder, Harald AU - Tuescher, Oliver AU - Kleim, Birgit PY - 2023/11/13 TI - German Version of the Mobile Agnew Relationship Measure: Translation and Validation Study JO - J Med Internet Res SP - e43368 VL - 25 KW - therapeutic alliance KW - digital therapeutic alliance KW - mental health apps KW - mHealth KW - mobile health KW - translation KW - validation KW - mobile phone N2 - Background: The mobile Agnew Relationship Measure (mARM) is a self-report questionnaire for the evaluation of digital mental health interventions and their interactions with users. With the global increase in digital mental health intervention research, translated measures are required to conduct research with local populations. Objective: The aim of this study was to translate and validate the original English version of the mARM into a German version (mARM-G). Methods: A total of 2 native German speakers who spoke English as their second language conducted forward translation of the original items. This version was then back translated by 2 native German speakers with a fluent knowledge of English. An independent bilingual reviewer then compared these drafts and created a final German version. The mARM-G was validated by 15 experts in the field of mobile app development and 15 nonexperts for content validity and face validity; 144 participants were recruited to conduct reliability testing as well as confirmatory factor analysis. Results: The content validity index of the mARM-G was 0.90 (expert ratings) and 0.79 (nonexperts). The face validity index was 0.89 (experts) and 0.86 (nonexperts). Internal consistency for the entire scale was Cronbach ?=.91. Confirmatory factor analysis results were as follows: the chi-square statistic to df ratio was 1.66. Comparative Fit Index was 0.87 and the Tucker-Lewis Index was 0.86. The root mean square error of approximation was 0.07. Conclusions: The mARM-G is a valid and reliable tool that can be used for future studies in German-speaking countries. UR - https://www.jmir.org/2023/1/e43368 UR - http://dx.doi.org/10.2196/43368 UR - http://www.ncbi.nlm.nih.gov/pubmed/37955952 ID - info:doi/10.2196/43368 ER - TY - JOUR AU - Mora, Nallely AU - Arvanitakis, Zoe AU - Thomas, Merly AU - Kramer, Holly AU - Morrato, H. Elaine AU - Markossian, W. Talar PY - 2023/11/13 TI - Applying Customer Discovery Method to a Chronic Disease Self-Management Mobile App: Qualitative Study JO - JMIR Form Res SP - e50334 VL - 7 KW - mobile app KW - disease management KW - customer discovery KW - customer segment KW - value proposition KW - chronic disease management KW - self-management KW - chronic disease KW - digital health KW - ehealth KW - mobile application KW - mhealth KW - customer delivery N2 - Background: A significant health challenge is evident in the United States, with 6 in 10 adults having a chronic disease and 4 in 10 adults having 2 or more. Chronic disease self-management aims to prevent or delay disease progression and disability and reduce mortality risk. The evidence to support the use of information technology tools, including mobile apps, web-based portals, and web-based educational interventions, that support disease self-management and improve clinical outcomes is growing. Customer discovery and value proposition design methodology is a form of stakeholder engagement and is based on marketing and lean start-up business methods. As applied in health care, customer discovery and value proposition methodology can be used to understand the clinical problem and articulate the product?s hypothesized unique value proposition relative to alternative options that are available to end users. Objective: This study aims to describe the experience and findings of academic researchers applying the customer discovery and value proposition methodology to identify stakeholders, needs, adaptability, and sustainability of a chronic disease self-management mobile app (CDapp). The motivation of the work is to make mobile health app interventions accessible and acceptable for all segments of patients? chronic diseases. Methods: Data were obtained through key informant interviews and analyzed using rapid qualitative analysis techniques. The value proposition framework was used to build the interview guide. The aim was to identify the needs, challenges (pains), and potential benefits (gains) of the CDapp for our stakeholders. Results: Our results showed that the primary consumers (end users) of a CDapp were the patients. The app adopters (decision makers) can be medical center leaders including population health department managers or insurance providers, while the consumer adoption influencers (influencers or saboteurs) are clinicians and patient caregivers. We developed an ecosystem map to visualize the clinical practice workflow and how an app for chronic disease management might integrate within an academic health care center or system. A value proposition for the identified customer segments was generated. Each stakeholder segment was working within a different framework to improve patient self-management. Patients needed help to adhere to self-care activities and they needed tailored health education. Health care leaders aim to improve the quality of care while reducing costs and workload. Clinicians wanted to improve patient education and care while reducing the time burden. Our results also showed that within academic medical centers, there were variations regarding patients? self-reported abilities to manage their diseases. Conclusions: Customer discovery is a useful form of stakeholder engagement when designing studies that seek to implement, adapt, and sustain an intervention. The customer discovery and value proposition methodology can be used as an alternative or complementary approach to formative research to generate valuable information in a brief period. UR - https://formative.jmir.org/2023/1/e50334 UR - http://dx.doi.org/10.2196/50334 UR - http://www.ncbi.nlm.nih.gov/pubmed/37955947 ID - info:doi/10.2196/50334 ER - TY - JOUR AU - Ambros-Antemate, Fernando Jorge AU - Beristain-Colorado, Pilar María del AU - Vargas-Treviño, Marciano AU - Gutiérrez-Gutiérrez, Jaime AU - Hernández-Cruz, Antonio Pedro AU - Gallegos-Velasco, Belem Itandehui AU - Moreno-Rodríguez, Adriana PY - 2023/11/10 TI - Improving Adherence to Physical Therapy in the Development of Serious Games: Conceptual Framework Design Study JO - JMIR Form Res SP - e39838 VL - 7 KW - conceptual framework KW - serious game KW - Flow Theory KW - adherence KW - gamification KW - physical rehabilitation N2 - Background: Insufficient levels of treatment adherence can have adverse effects on the outcomes of physical rehabilitation. To address this issue, alternative approaches to traditional therapies, such as serious games, have been designed to enhance adherence. Nevertheless, there remain gaps in the development of serious games concerning the effective implementation of motivation, engagement, and the enhancement of treatment adherence. Objective: This study aims to design a conceptual framework for the development of serious games that incorporate essential adherence factors to enhance patient compliance with physical rehabilitation programs. Methods: We formulated a conceptual framework using iterative techniques inspired by a conceptual framework analysis. Initially, we conducted a comprehensive literature review, concentrating on the critical adherence factors in physical rehabilitation. Subsequently, we identified, categorized, integrated, and synthesized the concepts derived from the literature review to construct the conceptual framework. Results: The framework resembles a road map, comprising 3 distinct phases. In the initial phase, the patient?s characteristics are identified through an initial exploration. The second phase involves the development of a serious game, with a focus on enhancing treatment adherence by integrating the key adherence factors identified. The third phase revolves around the evaluation of the serious game. These phases are underpinned by 2 overarching themes, namely, a user-centered design and the GameFlow model. Conclusions: The conceptual framework offers a detailed, step-by-step guide for creating serious games that incorporate essential adherence factors, thereby contributing to improved adherence in the physical rehabilitation process. To establish its validity, further evaluations of this framework across various physical rehabilitation programs and user groups are necessary. UR - https://formative.jmir.org/2023/1/e39838 UR - http://dx.doi.org/10.2196/39838 UR - http://www.ncbi.nlm.nih.gov/pubmed/37948110 ID - info:doi/10.2196/39838 ER - TY - JOUR AU - El Dahr, Yola AU - Perquier, Florence AU - Moloney, Madison AU - Woo, Guyyunge AU - Dobrin-De Grace, Roksana AU - Carvalho, Daniela AU - Addario, Nicole AU - Cameron, E. Emily AU - Roos, E. Leslie AU - Szatmari, Peter AU - Aitken, Madison PY - 2023/11/9 TI - Feasibility of Using Research Electronic Data Capture (REDCap) to Collect Daily Experiences of Parent-Child Dyads: Ecological Momentary Assessment Study JO - JMIR Form Res SP - e42916 VL - 7 KW - ambulatory assessment KW - children KW - ecological momentary assessment KW - longitudinal KW - parents KW - survey N2 - Background: Intensive longitudinal data collection, including ecological momentary assessment (EMA), has the potential to reduce recall biases, collect more ecologically valid data, and increase our understanding of dynamic associations between variables. EMA is typically administered using an application that is downloaded on participants? devices, which presents cost and privacy concerns that may limit its use. Research Electronic Data Capture (REDCap), a web-based survey application freely available to nonprofit organizations, may allow researchers to overcome these barriers; however, at present, little guidance is available to researchers regarding the setup of EMA in REDCap, especially for those who are new to using REDCap or lack advanced programming expertise. Objective: We provide an example of a simplified EMA setup in REDCap. This study aims to demonstrate the feasibility of this approach. We provide information on survey completion and user behavior in a sample of parents and children recruited across Canada. Methods: We recruited 66 parents and their children (aged 9-13 years old) from an existing longitudinal cohort study to participate in a study on risk and protective factors for children?s mental health. Parents received survey prompts (morning and evening) by email or SMS text message for 14 days, twice daily. Each survey prompt contained 2 sections, one for parents and one for children to complete. Results: The completion rates were good (mean 82%, SD 8%) and significantly higher on weekdays than weekends and in dyads with girls than dyads with boys. Children were available to respond to their own survey questions most of the time (in 1134/1498, 75.7% of surveys submitted). The number of assessments submitted was significantly higher, and response times were significantly faster among participants who selected SMS text message survey notifications compared to email survey notifications. The average response time was 47.0 minutes after the initial survey notification, and the use of reminder messages increased survey completion. Conclusions: Our results support the feasibility of using REDCap for EMA studies with parents and children. REDCap also has features that can accommodate EMA studies by recruiting participants across multiple time zones and providing different survey delivery methods. Offering the option of SMS text message survey notifications and reminders may be an important way to increase completion rates and the timeliness of responses. REDCap is a potentially useful tool for researchers wishing to implement EMA in settings in which cost or privacy are current barriers. Researchers should weigh these benefits with the potential limitations of REDCap and this design, including staff time to set up, monitor, and clean the data outputs of the project. UR - https://formative.jmir.org/2023/1/e42916 UR - http://dx.doi.org/10.2196/42916 UR - http://www.ncbi.nlm.nih.gov/pubmed/37943593 ID - info:doi/10.2196/42916 ER - TY - JOUR AU - Zhang, Yue Xi AU - Arata Found, Anelyse AU - Butler, Sheila PY - 2023/11/8 TI - Effects of Distance-Learning Strategies in Dental Fixed Prosthodontics Amidst the COVID-19 Pandemic: Cross-Sectional Questionnaire Study on Preclinical Dental Students? Perspective JO - JMIR Form Res SP - e45311 VL - 7 KW - dental education KW - dental KW - dentist KW - dentistry KW - technology-based learning KW - online learning KW - pre-clinical training KW - distance learning KW - transmissibility KW - dental school KW - mental health KW - COVID-19 KW - student perception KW - online teaching KW - survey KW - teaching methods KW - training KW - isolation KW - teaching KW - module KW - education N2 - Background: COVID-19?s high transmissibility led to gathering restrictions where dental schools experienced disruptions due to restrictions on attending in-person lectures and limitations placed on applied preclinical and clinical activities. Students not only had to rapidly switch to digital technology-based learning (TB-learning) modules but also experienced high levels of social isolation and anxiety around virus transmission. Objective: This study aims to evaluate the preclinical students? perception of switching TB-learning modules amidst the COVID-19 pandemic, identifying which module parameters were associated with strong student outcomes. Methods: A web-based survey of 39 Likert scale questions was delivered to preclinical dental students (Western University) to evaluate students? perceptions concerning TB-learning, fear amidst the COVID-19 pandemic, and the impact on their preclinical training. A Spearman rank correlation coefficient was determined to estimate the relationship between 2 variables in isolation (P=.01). An ordinal regression analysis was performed on variables of interest to determine how module variables (typically within the instructor?s control) influenced the student outcomes (P=.05). Results: The response rate was 30% (n=39). TB-learning was considered vital (34/39, 87.2%) as the students? education improved (18/39, 46.2%). However, 53.8% (n=21) of students showed increased difficulties in retaining, visualizing, or understanding the materials using TB-learning, and 64.1% (n=25) found it more difficult to concentrate than in in-person classes. In total, 79.5% (n=31) of students showed different levels of agreement about feeling fatigued from TB-learning. Through Spearman ? correlation analysis, the quality of questions in quizzes (?=0.514; P<.001), relevant handouts (?=0.729; P<.001), and high-quality audiovisuals (?=0.585; P<.001) were positively correlated with students responding that the modules were useful to preclinical training. Similarly, good organization (?=0.512; P<.001), high-quality questions in quizzes (?=0.431; P=.01), and relevant handouts (?=0.551; P<.001) were positively correlated with web-based classes as an effective way to learn. In total, 91.6% (n=36) of the students agreed that COVID-19 was a dangerous disease, whereas 53.8% (n=21) showed different levels of agreement that they were afraid to be infected personally, and 69.2% (n=27) feared passing COVID-19 along to family and friends. A total of 82.1% (n=32) of the students showed that COVID-19 impacted their overall learning process and had a negative impact on their practical preclinical training (31/39, 79.5%). Conclusions: The students found a difference between TB-learning and face-to-face learning methods, where the students perceived fatigue toward the web-based method with difficulty concentrating and visualizing the subject. Moreover, there was a consensus that COVID-19 itself affected the students? overall learning process and preclinical training. As dental schools continue implementing TB-learning into their curriculum, this investigation identifies the students? struggles with the paradigm shift. In an effort to improve TB-learning, this work highlights 4 variables (organization, quizzes, quality handouts, and quality audiovisuals) within the control of instructors that can help improve content deliverance, improving the students? experience. UR - https://formative.jmir.org/2023/1/e45311 UR - http://dx.doi.org/10.2196/45311 UR - http://www.ncbi.nlm.nih.gov/pubmed/37938882 ID - info:doi/10.2196/45311 ER - TY - JOUR AU - Moldestad, Megan AU - Petrova, V. Valentina AU - Tirtanadi, Katie AU - Mishra, R. Sonali AU - Rajan, Suparna AU - Sayre, George AU - Fortney, C. John AU - Reisinger, Schacht Heather PY - 2023/11/6 TI - Improving the Usability of Written Exposure Therapy for Therapists in the Department of Veterans Affairs Telemental Health: Formative Study Using Qualitative and User-Centered Design Methods JO - JMIR Form Res SP - e47189 VL - 7 KW - evidence-based psychosocial interventions KW - telehealth KW - qualitative KW - user-centered design KW - implementation science KW - Department of Veterans Affairs health care system N2 - Background: User modifications are common in evidence-based psychosocial interventions (EBPIs) for mental health disorders. Often, EBPIs fit poorly into clinical workflows, require extensive resources, or pose considerable burden to patients and therapists. Implementation science is increasingly researching ways to improve the usability of EBPIs before implementation. A user-centered design can be used to support implementation methods to prioritize user needs and solutions to improve EBPI usability. Objective: Trauma-focused EBPIs are a first-line treatment for patients with posttraumatic stress disorder (PTSD) in the Department of Veterans Affairs. Written exposure therapy (WET) is a brief, trauma-focused EBPI wherein patients handwrite about trauma associated with their PTSD. Initially developed for in-person delivery, WET is increasingly being delivered remotely, and outcomes appear to be equivalent to in-person delivery. However, there are logistical issues in delivering WET via video. In this evaluation, we explored usability issues related to WET telehealth delivery via videoconferencing software and designed a solution for therapist-facing challenges to systematize WET telehealth delivery. Methods: The Discover, Design and Build, and Test framework guided this formative evaluation and served to inform a larger Virtual Care Quality Enhancement Research Initiative. We used qualitative descriptive methods in the Discover phase to understand the experiences and needs of 2 groups of users providing care within the Department of Veterans Affairs: in-person therapists delivering WET via video because of the COVID-19 pandemic and telehealth therapists who regularly deliver PTSD therapies. We then used user-centered design methods in the Design and Build phase to brainstorm, develop, and iteratively refine potential workflows to address identified usability issues. All procedures were conducted remotely. Results: In the Discover phase, both groups had challenges delivering WET and other PTSD therapies via telehealth because of technology issues with videoconferencing software, environmental distractions, and workflow disruptions. Narrative transfer (ie, patients sending handwritten trauma accounts to therapists) was the first target for design solution development as it was deemed most critical to WET delivery. In the Design and Build phase, we identified design constraints and brainstormed solution ideas. This led to the development of 3 solution workflows that were presented to a subgroup of therapist users through cognitive walkthroughs. Meetings with this subgroup allowed workflow refinement to improve narrative transfers. Finally, to facilitate using these workflows, we developed PDF manuals that are being refined in subsequent phases of the implementation project (not mentioned in this paper). Conclusions: The Discover, Design and Build, and Test framework can be a useful tool for understanding user needs in complex EBPI interventions and designing solutions to user-identified usability issues. Building on this work, an iterative evaluation of the 3 solution workflows and accompanying manuals with therapists and patients is underway as part of a nationwide WET implementation in telehealth settings. UR - https://formative.jmir.org/2023/1/e47189 UR - http://dx.doi.org/10.2196/47189 UR - http://www.ncbi.nlm.nih.gov/pubmed/37930747 ID - info:doi/10.2196/47189 ER - TY - JOUR AU - Cross, Shane AU - Nicholas, Jennifer AU - Mangelsdorf, Shaminka AU - Valentine, Lee AU - Baker, Simon AU - McGorry, Patrick AU - Gleeson, John AU - Alvarez-Jimenez, Mario PY - 2023/11/3 TI - Developing a Theory of Change for a Digital Youth Mental Health Service (Moderated Online Social Therapy): Mixed Methods Knowledge Synthesis Study JO - JMIR Form Res SP - e49846 VL - 7 KW - adolescence KW - adolescent KW - blended care KW - blended KW - co-design KW - development KW - digital health KW - digital intervention KW - digital mental health KW - framework KW - hybrid KW - mental health KW - model KW - platform KW - self-determination theory KW - service KW - services KW - theory of change KW - therapy KW - youth mental health KW - youth N2 - Background: Common challenges in the youth mental health system include low access, poor uptake, poor adherence, and limited overall effectiveness. Digital technologies offer promise, yet challenges in real-world integration and uptake persist. Moderated Online Social Therapy (MOST) aims to overcome these problems by integrating a comprehensive digital platform into existing youth mental health services. Theory of change (ToC) frameworks can help articulate how and why complex interventions work and what conditions are required for success. Objective: The objective of this study is to create a ToC for MOST to explain how it works, why it works, who benefits and how, and what conditions are required for its success. Methods: We used a multimethod approach to construct a ToC for MOST. The synthesis aimed to assess the real-world impact of MOST, a digital platform designed to enhance face-to-face youth mental health services, and to guide its iterative refinement. Data were gathered from 2 completed and 4 ongoing randomized controlled trials, 11 pilot studies, and over 1000 co-design sessions using MOST. Additionally, published qualitative findings from diverse clinical contexts and a review of related digital mental health literature were included. The study culminated in an updated ToC framework informed by expert feedback. The final ToC was produced in both narrative and table form and captured components common in program logic and ToC frameworks. Results: The MOST ToC captured several assumptions about digital mental health adoption, including factors such as the readiness of young people and service providers to embrace digital platforms. External considerations included high service demand and a potential lack of infrastructure to support integration. Young people and service providers face several challenges and pain points MOST seeks to address, such as limited accessibility, high demand, poor engagement, and a lack of personalized support. Self-determination theory, transdiagnostic psychological treatment approaches, and evidence-based implementation theories and their associated mechanisms are drawn upon to frame the intervention components that make up the platform. Platform usage data are captured and linked to short-, medium-, and long-term intended outcomes, such as reductions in mental health symptoms, improvements in functioning and quality of life, reductions in hospital visits, and reduced overall mental health care costs. Conclusions: The MOST ToC serves as a strategic framework for refining MOST over time. The creation of the ToC helped guide the development of therapeutic content personalization, user engagement enhancement, and clinician adoption through specialized implementation frameworks. While powerful, the ToC approach has its limitations, such as a lack of standardized methodology and the amount of resourcing required for its development. Nonetheless, it provides an invaluable roadmap for iterative development, evaluation, and scaling of MOST and offers a replicable model for other digital health interventions aiming for targeted, evidence-based impact. UR - https://formative.jmir.org/2023/1/e49846 UR - http://dx.doi.org/10.2196/49846 UR - http://www.ncbi.nlm.nih.gov/pubmed/37921858 ID - info:doi/10.2196/49846 ER - TY - JOUR AU - Gregory, E. Megan AU - Sova, N. Lindsey AU - Huerta, R. Timothy AU - McAlearney, Scheck Ann PY - 2023/11/1 TI - Implications for Electronic Surveys in Inpatient Settings Based on Patient Survey Response Patterns: Cross-Sectional Study JO - J Med Internet Res SP - e48236 VL - 25 KW - surveys KW - patient satisfaction KW - patient experience KW - patient surveys KW - electronic survey KW - cross-sectional study KW - quality of life KW - mental health KW - symptoms KW - data quality KW - hospitalization N2 - Background:  Surveys of hospitalized patients are important for research and learning about unobservable medical issues (eg, mental health, quality of life, and symptoms), but there has been little work examining survey data quality in this population whose capacity to respond to survey items may differ from the general population. Objective:  The aim of this study is to determine what factors drive response rates, survey drop-offs, and missing data in surveys of hospitalized patients. Methods:  Cross-sectional surveys were distributed on an inpatient tablet to patients in a large, midwestern US hospital. Three versions were tested: 1 with 174 items and 2 with 111 items; one 111-item version had missing item reminders that prompted participants when they did not answer items. Response rate, drop-off rate (abandoning survey before completion), and item missingness (skipping items) were examined to investigate data quality. Chi-square tests, Kaplan-Meyer survival curves, and distribution charts were used to compare data quality among survey versions. Response duration was computed for each version. Results: Overall, 2981 patients responded. Response rate did not differ between the 174- and 111-item versions (81.7% vs 83%, P=.53). Drop-off was significantly reduced when the survey was shortened (65.7% vs 20.2% of participants dropped off, P<.001). Approximately one-quarter of participants dropped off by item 120, with over half dropping off by item 158. The percentage of participants with missing data decreased substantially when missing item reminders were added (77.2% vs 31.7% of participants, P<.001). The mean percentage of items with missing data was reduced in the shorter survey (40.7% vs 20.3% of items missing); with missing item reminders, the percentage of items with missing data was further reduced (20.3% vs 11.7% of items missing). Across versions, for the median participant, each item added 24.6 seconds to a survey?s duration. Conclusions:  Hospitalized patients may have a higher tolerance for longer surveys than the general population, but surveys given to hospitalized patients should have a maximum of 120 items to ensure high rates of completion. Missing item prompts should be used to reduce missing data. Future research should examine generalizability to nonhospitalized individuals. UR - https://www.jmir.org/2023/1/e48236 UR - http://dx.doi.org/10.2196/48236 UR - http://www.ncbi.nlm.nih.gov/pubmed/37910163 ID - info:doi/10.2196/48236 ER - TY - JOUR AU - Kunitsu, Yuki PY - 2023/10/30 TI - The Potential of GPT-4 as a Support Tool for Pharmacists: Analytical Study Using the Japanese National Examination for Pharmacists JO - JMIR Med Educ SP - e48452 VL - 9 KW - natural language processing KW - generative pretrained transformer KW - GPT-4 KW - ChatGPT KW - artificial intelligence KW - AI KW - chatbot KW - pharmacy KW - pharmacist N2 - Background: The advancement of artificial intelligence (AI), as well as machine learning, has led to its application in various industries, including health care. AI chatbots, such as GPT-4, developed by OpenAI, have demonstrated potential in supporting health care professionals by providing medical information, answering examination questions, and assisting in medical education. However, the applicability of GPT-4 in the field of pharmacy remains unexplored. Objective: This study aimed to evaluate GPT-4?s ability to answer questions from the Japanese National Examination for Pharmacists (JNEP) and assess its potential as a support tool for pharmacists in their daily practice. Methods: The question texts and answer choices from the 107th and 108th JNEP, held in February 2022 and February 2023, were input into GPT-4. As GPT-4 cannot process diagrams, questions that included diagram interpretation were not analyzed and were initially given a score of 0. The correct answer rates were calculated and compared with the passing criteria of each examination to evaluate GPT-4?s performance. Results: For the 107th and 108th JNEP, GPT-4 achieved an accuracy rate of 64.5% (222/344) and 62.9% (217/345), respectively, for all questions. When considering only the questions that GPT-4 could answer, the accuracy rates increased to 78.2% (222/284) and 75.3% (217/287), respectively. The accuracy rates tended to be lower for physics, chemistry, and calculation questions. Conclusions: Although GPT-4 demonstrated the potential to answer questions from the JNEP and support pharmacists? capabilities, it also showed limitations in handling highly specialized questions, calculation questions, and questions requiring diagram recognition. Further evaluation is necessary to explore its applicability in real-world clinical settings, considering the complexities of patient scenarios and collaboration with health care professionals. By addressing these limitations, GPT-4 could become a more reliable tool for pharmacists in their daily practice. UR - https://mededu.jmir.org/2023/1/e48452 UR - http://dx.doi.org/10.2196/48452 UR - http://www.ncbi.nlm.nih.gov/pubmed/37837968 ID - info:doi/10.2196/48452 ER - TY - JOUR AU - Underly, Robert AU - Dull, M. Gary AU - Nudi, Evan AU - Pionk, Timothy AU - Prevette, Kristen AU - Smith, Jeffrey PY - 2023/10/30 TI - Using a Novel Connected Device for the Collection of Puffing Topography Data for the Vuse Solo Electronic Nicotine Delivery System in a Real-World Setting: Prospective Ambulatory Clinical Study JO - JMIR Form Res SP - e49876 VL - 7 KW - topography KW - electronic cigarette KW - e-cigarette KW - electronic nicotine delivery system KW - ENDS KW - ambulatory puffing KW - use behavior KW - sessions KW - mobile phone N2 - Background: Over the last decade, the use of electronic nicotine delivery systems (ENDSs) has risen, whereas studies that describe how consumers use these products have been limited. Most studies related to ENDS use have involved study designs focused on use in a central location environment or attempted to measure use outcomes through subjective self-reported end points. The development of accurate and reliable tools to collect data in a naturalistic real-world environment is necessary to capture the complexities of ENDS use. Using connected devices in a real-world setting provides a convenient and objective approach to collecting behavioral outcomes with ENDS. Objective: The Product Use and Behavior instrument was developed and used to capture the use of the Vuse Solo ENDS in an ambulatory setting to best replicate real-world use behavior. This study aims to determine overall mean values for topography outcomes while also providing a definition for an ENDS use session. Methods: A prospective ambulatory clinical study was performed with the Product Use and Behavior instrument. Participants (n=75) were aged between 21 and 60 years, considered in good health, and were required to be established regular users of ENDSs. To better understand use behavior within the population, the sample was sorted into percentiles with bins based on daily puff counts. To frame these data in the relevant context, they were binned into low-, moderate-, and high-use categories (10th to 40th, 40th to 70th, and 70th to 100th percentiles, respectively), with the low-use group representing the nonintense category, the high-use group representing the intense category, and the moderate-use group being reflective of the average consumer. Results: Participants with higher daily use took substantially more puffs per use session (6.71 vs 4.40) and puffed more frequently (interpuff interval: 32.78 s vs 61.66 s) than participants in the low-use group. Puff duration remained consistent across the low-, moderate?, and high-use groups (2.10 s, 2.18 s, and 2.19 s, respectively). The moderate-use group had significantly shorter session lengths (P<.001) than the high- and low-use groups, which did not differ significantly from each other (P=.16). Conclusions: Using connected devices allows for a convenient and robust approach to the collection of behavioral outcomes related to product use in an ambulatory setting. By using the variables captured with these tools, it becomes possible to move away from predefined periods of use to better understand topography outcomes and define use sessions. The data presented here offer a possible method to define these sessions. These data also begin to frame international standards used for the analytical assessments of ENDSs in the correct context and begin to shed light on the differences between standardized testing regimens and actual use behavior. Trial Registration: Clinicaltrials.gov NCT04226404; https://clinicaltrials.gov/study/NCT04226404 UR - https://formative.jmir.org/2023/1/e49876 UR - http://dx.doi.org/10.2196/49876 UR - http://www.ncbi.nlm.nih.gov/pubmed/37902830 ID - info:doi/10.2196/49876 ER - TY - JOUR AU - Sükei, Emese AU - Romero-Medrano, Lorena AU - de Leon-Martinez, Santiago AU - Herrera López, Jesús AU - Campaña-Montes, José Juan AU - Olmos, M. Pablo AU - Baca-Garcia, Enrique AU - Artés, Antonio PY - 2023/10/30 TI - Continuous Assessment of Function and Disability via Mobile Sensing: Real-World Data-Driven Feasibility Study JO - JMIR Form Res SP - e47167 VL - 7 KW - WHODAS KW - functional limitations KW - mobile sensing KW - passive ecological momentary assessment KW - predictive modeling KW - interpretable machine learning KW - machine learning KW - disability KW - clinical outcome N2 - Background: Functional limitations are associated with poor clinical outcomes, higher mortality, and disability rates, especially in older adults. Continuous assessment of patients? functionality is important for clinical practice; however, traditional questionnaire-based assessment methods are very time-consuming and infrequently used. Mobile sensing offers a great range of sources that can assess function and disability daily. Objective: This work aims to prove the feasibility of an interpretable machine learning pipeline for predicting function and disability based on the World Health Organization Disability Assessment Schedule (WHODAS) 2.0 outcomes of clinical outpatients, using passively collected digital biomarkers. Methods: One-month-long behavioral time-series data consisting of physical and digital activity descriptor variables were summarized using statistical measures (minimum, maximum, mean, median, SD, and IQR), creating 64 features that were used for prediction. We then applied a sequential feature selection to each WHODAS 2.0 domain (cognition, mobility, self-care, getting along, life activities, and participation) in order to find the most descriptive features for each domain. Finally, we predicted the WHODAS 2.0 functional domain scores using linear regression using the best feature subsets. We reported the mean absolute errors and the mean absolute percentage errors over 4 folds as goodness-of-fit statistics to evaluate the model and allow for between-domain performance comparison. Results: Our machine learning?based models for predicting patients? WHODAS functionality scores per domain achieved an average (across the 6 domains) mean absolute percentage error of 19.5%, varying between 14.86% (self-care domain) and 27.21% (life activities domain). We found that 5-19 features were sufficient for each domain, and the most relevant being the distance traveled, time spent at home, time spent walking, exercise time, and vehicle time. Conclusions: Our findings show the feasibility of using machine learning?based methods to assess functional health solely from passively sensed mobile data. The feature selection step provides a set of interpretable features for each domain, ensuring better explainability to the models? decisions?an important aspect in clinical practice. UR - https://formative.jmir.org/2023/1/e47167 UR - http://dx.doi.org/10.2196/47167 UR - http://www.ncbi.nlm.nih.gov/pubmed/37902823 ID - info:doi/10.2196/47167 ER - TY - JOUR AU - Zhang, Xiaobo AU - Gu, Ying AU - Yin, Jie AU - Zhang, Yuejie AU - Jin, Cheng AU - Wang, Weibing AU - Li, Martin Albert AU - Wang, Yingwen AU - Su, Ling AU - Xu, Hong AU - Ge, Xiaoling AU - Ye, Chengjie AU - Tang, Liangfeng AU - Shen, Bing AU - Fang, Jinwu AU - Wang, Daoyang AU - Feng, Rui PY - 2023/10/26 TI - Development, Reliability, and Structural Validity of the Scale for Knowledge, Attitude, and Practice in Ethics Implementation Among AI Researchers: Cross-Sectional Study JO - JMIR Form Res SP - e42202 VL - 7 KW - medical artificial intelligence KW - ethics implementation KW - Knowledge-Attitude-Practice model KW - reliability KW - validity KW - measure KW - artificial intelligence KW - development KW - attitude KW - ethics N2 - Background: Medical artificial intelligence (AI) has significantly contributed to decision support for disease screening, diagnosis, and management. With the growing number of medical AI developments and applications, incorporating ethics is considered essential to avoiding harm and ensuring broad benefits in the lifecycle of medical AI. One of the premises for effectively implementing ethics in Medical AI research necessitates researchers' comprehensive knowledge, enthusiastic attitude, and practical experience. However, there is currently a lack of an available instrument to measure these aspects. Objective: The aim of this study was to develop a comprehensive scale for measuring the knowledge, attitude, and practice of ethics implementation among medical AI researchers, and to evaluate its measurement properties. Methods: The construct of the Knowledge-Attitude-Practice in Ethics Implementation (KAP-EI) scale was based on the Knowledge-Attitude-Practice (KAP) model, and the evaluation of its measurement properties was in compliance with the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) reporting guidelines for studies on measurement instruments. The study was conducted in 2 phases. The first phase involved scale development through a systematic literature review, qualitative interviews, and item analysis based on a cross-sectional survey. The second phase involved evaluation of structural validity and reliability through another cross-sectional study. Results: The KAP-EI scale had 3 dimensions including knowledge (10 items), attitude (6 items), and practice (7 items). The Cronbach ? for the whole scale reached .934. Confirmatory factor analysis showed that the goodness-of-fit indices of the scale were satisfactory (?2/df ratio:=2.338, comparative fit index=0.949, Tucker Lewis index=0.941, root-mean-square error of approximation=0.064, and standardized root-mean-square residual=0.052). Conclusions: The results show that the scale has good reliability and structural validity; hence, it could be considered an effective instrument. This is the first instrument developed for this purpose. UR - https://formative.jmir.org/2023/1/e42202 UR - http://dx.doi.org/10.2196/42202 UR - http://www.ncbi.nlm.nih.gov/pubmed/37883175 ID - info:doi/10.2196/42202 ER - TY - JOUR AU - Ranusch, Allison AU - Lin, Ying-Jen AU - Dorsch, P. Michael AU - Allen, L. Arthur AU - Spoutz, Patrick AU - Seagull, Jacob F. AU - Sussman, B. Jeremy AU - Barnes, D. Geoffrey PY - 2023/10/24 TI - Role of Individual Clinician Authority in the Implementation of Informatics Tools for Population-Based Medication Management: Qualitative Semistructured Interview Study JO - JMIR Hum Factors SP - e49025 VL - 10 KW - direct oral anticoagulant KW - population management KW - implementation science KW - medical informatics KW - individual clinician authority KW - electronic health record KW - health records KW - EHR KW - EHRs KW - implementation KW - clotting KW - clot KW - clots KW - anticoagulant KW - anticoagulants KW - dashboard KW - DOAC KW - satisfaction KW - interview KW - interviews KW - pharmacist KW - pharmacy KW - pharmacology KW - medication KW - prescribe KW - prescribing N2 - Background: Direct oral anticoagulant (DOAC) medications are frequently associated with inappropriate prescribing and adverse events. To improve the safe use of DOACs, health systems are implementing population health tools within their electronic health record (EHR). While EHR informatics tools can help increase awareness of inappropriate prescribing of medications, a lack of empowerment (or insufficient empowerment) of nonphysicians to implement change is a key barrier. Objective: This study examined how the individual authority of clinical pharmacists and anticoagulation nurses is impacted by and changes the implementation success of an EHR DOAC Dashboard for safe DOAC medication prescribing. Methods: We conducted semistructured interviews with pharmacists and nurses following the implementation of the EHR DOAC Dashboard at 3 clinical sites. Interview transcripts were coded according to the key determinants of implementation success. The intersections between individual clinician authority and other determinants were examined to identify themes. Results: A high level of individual clinician authority was associated with high levels of key facilitators for effective use of the DOAC Dashboard (communication, staffing and work schedule, job satisfaction, and EHR integration). Conversely, a lack of individual authority was often associated with key barriers to effective DOAC Dashboard use. Positive individual authority was sometimes present with a negative example of another determinant, but no evidence was found of individual authority co-occurring with a positive instance of another determinant. Conclusions: Increased individual clinician authority is a necessary antecedent to the effective implementation of an EHR DOAC Population Management Dashboard and positively affects other aspects of implementation. International Registered Report Identifier (IRRID): RR2-10.1186/s13012-020-01044-5 UR - https://humanfactors.jmir.org/2023/1/e49025 UR - http://dx.doi.org/10.2196/49025 UR - http://www.ncbi.nlm.nih.gov/pubmed/37874636 ID - info:doi/10.2196/49025 ER - TY - JOUR AU - Amigó-Vega, Joaquín AU - Ottenhoff, C. Maarten AU - Verwoert, Maxime AU - Kubben, Pieter AU - Herff, Christian PY - 2023/10/24 TI - The Easy and Versatile Neural Recording Platform (T-REX): Design and Development Study JO - JMIR Neurotech SP - e47881 VL - 2 KW - recording KW - platform KW - flexible KW - data recording KW - neurotechnology KW - experiments N2 - Background: Recording time in invasive neuroscientific research is limited and must be used as efficiently as possible. Time is often lost due to a long setup time and errors by the researcher, driven by the number of manually performed steps. Currently, recording solutions that automate experimental overhead are either custom-made by researchers or provided as a submodule in comprehensive neuroscientific toolboxes, and there are no platforms focused explicitly on recording. Objective: Minimizing the number of manual actions may reduce error rates and experimental overhead. However, automation should avoid reducing the flexibility of the system. Therefore, we developed a software package named T-REX (Standalone Recorder of Experiments) that specifically simplifies the recording of experiments while focusing on retaining flexibility. Methods: The proposed solution is a standalone webpage that the researcher can provide without an active internet connection. It is built using Bootstrap5 for the frontend and the Python package Flask for the backend. Only Python 3.7+ and a few dependencies are required to start the different experiments. Data synchronization is implemented using Lab Streaming Layer, an open-source networked synchronization ecosystem, enabling all major programming languages and toolboxes to be used for developing and executing the experiments. Additionally, T-REX runs on Windows, Linux, and macOS. Results: The system reduces experimental overhead during recordings to a minimum. Multiple experiments are centralized in a simple local web interface that reduces an experiment?s setup, start, and stop to a single button press. In principle, any type of experiment, regardless of the scientific field (eg, behavioral or cognitive sciences, and electrophysiology), can be executed with the platform. T-REX includes an easy-to-use interface that can be adjusted to specific recording modalities, amplifiers, and participants. Because of the automated setup, easy recording, and easy-to-use interface, participants may even start and stop experiments by themselves, thus potentially providing data without the researcher?s presence. Conclusions: We developed a new recording platform that is operating system independent, user friendly, and robust. We provide researchers with a solution that can greatly increase the time spent on recording instead of setting up (with its possible errors). UR - https://neuro.jmir.org/2023/1/e47881 UR - http://dx.doi.org/10.2196/47881 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/47881 ER - TY - JOUR AU - Kim, Young Se AU - Park, Jinseok AU - Choi, Hojin AU - Loeser, Martin AU - Ryu, Hokyoung AU - Seo, Kyoungwon PY - 2023/10/20 TI - Digital Marker for Early Screening of Mild Cognitive Impairment Through Hand and Eye Movement Analysis in Virtual Reality Using Machine Learning: First Validation Study JO - J Med Internet Res SP - e48093 VL - 25 KW - Alzheimer disease KW - biomarkers KW - dementia KW - digital markers KW - eye movement KW - hand movement KW - machine learning KW - mild cognitive impairment KW - screening KW - virtual reality N2 - Background: With the global rise in Alzheimer disease (AD), early screening for mild cognitive impairment (MCI), which is a preclinical stage of AD, is of paramount importance. Although biomarkers such as cerebrospinal fluid amyloid level and magnetic resonance imaging have been studied, they have limitations, such as high cost and invasiveness. Digital markers to assess cognitive impairment by analyzing behavioral data collected from digital devices in daily life can be a new alternative. In this context, we developed a ?virtual kiosk test? for early screening of MCI by analyzing behavioral data collected when using a kiosk in a virtual environment. Objective: We aimed to investigate key behavioral features collected from a virtual kiosk test that could distinguish patients with MCI from healthy controls with high statistical significance. Also, we focused on developing a machine learning model capable of early screening of MCI based on these behavioral features. Methods: A total of 51 participants comprising 20 healthy controls and 31 patients with MCI were recruited by 2 neurologists from a university hospital. The participants performed a virtual kiosk test?developed by our group?where we recorded various behavioral data such as hand and eye movements. Based on these time series data, we computed the following 4 behavioral features: hand movement speed, proportion of fixation duration, time to completion, and the number of errors. To compare these behavioral features between healthy controls and patients with MCI, independent-samples 2-tailed t tests were used. Additionally, we used these behavioral features to train and validate a machine learning model for early screening of patients with MCI from healthy controls. Results: In the virtual kiosk test, all 4 behavioral features showed statistically significant differences between patients with MCI and healthy controls. Compared with healthy controls, patients with MCI had slower hand movement speed (t49=3.45; P=.004), lower proportion of fixation duration (t49=2.69; P=.04), longer time to completion (t49=?3.44; P=.004), and a greater number of errors (t49=?3.77; P=.001). All 4 features were then used to train a support vector machine to distinguish between healthy controls and patients with MCI. Our machine learning model achieved 93.3% accuracy, 100% sensitivity, 83.3% specificity, 90% precision, and 94.7% F1-score. Conclusions: Our research preliminarily suggests that analyzing hand and eye movements in the virtual kiosk test holds potential as a digital marker for early screening of MCI. In contrast to conventional biomarkers, this digital marker in virtual reality is advantageous as it can collect ecologically valid data at an affordable cost and in a short period (5-15 minutes), making it a suitable means for early screening of MCI. We call for further studies to confirm the reliability and validity of this approach. UR - https://www.jmir.org/2023/1/e48093 UR - http://dx.doi.org/10.2196/48093 UR - http://www.ncbi.nlm.nih.gov/pubmed/37862101 ID - info:doi/10.2196/48093 ER - TY - JOUR AU - Edney, Sarah AU - Goh, Marie Claire AU - Chua, Hui Xin AU - Low, Alicia AU - Chia, Janelle AU - S Koek, Daphne AU - Cheong, Karen AU - van Dam, Rob AU - Tan, Seng Chuen AU - Müller-Riemenschneider, Falk PY - 2023/10/19 TI - Evaluating the Effects of Rewards and Schedule Length on Response Rates to Ecological Momentary Assessment Surveys: Randomized Controlled Trials JO - J Med Internet Res SP - e45764 VL - 25 KW - experience sampling KW - ambulatory assessment KW - compliance KW - mobile phone N2 - Background: Ecological momentary assessments (EMAs) are short, repeated surveys designed to collect information on experiences in real-time, real-life contexts. Embedding periodic bursts of EMAs within cohort studies enables the study of experiences on multiple timescales and could greatly enhance the accuracy of self-reported information. However, the burden on participants may be high and should be minimized to optimize EMA response rates. Objective: We aimed to evaluate the effects of study design features on EMA response rates. Methods: Embedded within an ongoing cohort study (Health@NUS), 3 bursts of EMAs were implemented over a 7-month period (April to October 2021). The response rate (percentage of completed EMA surveys from all sent EMA surveys; 30-42 individual EMA surveys sent/burst) for each burst was examined. Following a low response rate in burst 1, changes were made to the subsequent implementation strategy (SMS text message announcements instead of emails). In addition, 2 consecutive randomized controlled trials were conducted to evaluate the efficacy of 4 different reward structures (with fixed and bonus components) and 2 different schedule lengths (7 or 14 d) on changes to the EMA response rate. Analyses were conducted from 2021 to 2022 using ANOVA and analysis of covariance to examine group differences and mixed models to assess changes across all 3 bursts. Results: Participants (N=384) were university students (n=232, 60.4% female; mean age 23, SD 1.3 y) in Singapore. Changing the reward structure did not significantly change the response rate (F3,380=1.75; P=.16). Changing the schedule length did significantly change the response rate (F1,382=6.23; P=.01); the response rate was higher for the longer schedule (14 d; mean 48.34%, SD 33.17%) than the shorter schedule (7 d; mean 38.52%, SD 33.44%). The average response rate was higher in burst 2 and burst 3 (mean 50.56, SD 33.61 and mean 48.34, SD 33.17, respectively) than in burst 1 (mean 25.78, SD 30.12), and the difference was statistically significant (F2,766=93.83; P<.001). Conclusions: Small changes to the implementation strategy (SMS text messages instead of emails) may have contributed to increasing the response rate over time. Changing the available rewards did not lead to a significant difference in the response rate, whereas changing the schedule length did lead to a significant difference in the response rate. Our study provides novel insights on how to implement EMA surveys in ongoing cohort studies. This knowledge is essential for conducting high-quality studies using EMA surveys. Trial Registration: ClinicalTrials.gov NCT05154227; https://clinicaltrials.gov/ct2/show/NCT05154227 UR - https://www.jmir.org/2023/1/e45764 UR - http://dx.doi.org/10.2196/45764 UR - http://www.ncbi.nlm.nih.gov/pubmed/37856188 ID - info:doi/10.2196/45764 ER - TY - JOUR AU - Glavin, Darragh AU - Grua, Martino Eoin AU - Nakamura, Akemi Carina AU - Scazufca, Marcia AU - Ribeiro dos Santos, Edinilza AU - Wong, Y. Gloria H. AU - Hollingworth, William AU - Peters, J. Tim AU - Araya, Ricardo AU - Van de Ven, Pepijn PY - 2023/10/19 TI - Patient Health Questionnaire-9 Item Pairing Predictiveness for Prescreening Depressive Symptomatology: Machine Learning Analysis JO - JMIR Ment Health SP - e48444 VL - 10 KW - Patient Health Questionnaire-2 KW - PHQ-2 KW - Patient Health Questionnaire-9 KW - PHQ-9 items KW - depressive symptomatology KW - ultrabrief questionnaires KW - prescreening KW - machine learning KW - cardinal symptoms KW - low energy KW - psychomotor dysfunction KW - depressed mood N2 - Background: Anhedonia and depressed mood are considered the cardinal symptoms of major depressive disorder. These are the first 2 items of the Patient Health Questionnaire (PHQ)?9 and comprise the ultrabrief PHQ-2 used for prescreening depressive symptomatology. The prescreening performance of alternative PHQ-9 item pairings is rarely compared with that of the PHQ-2. Objective: This study aims to use machine learning (ML) with the PHQ-9 items to identify and validate the most predictive 2-item depressive symptomatology ultrabrief questionnaire and to test the generalizability of the best pairings found on the primary data set, with 6 external data sets from different populations to validate their use as prescreening instruments. Methods: All 36 possible PHQ-9 item pairings (each yielding scores of 0-6) were investigated using ML-based methods with logistic regression models. Their performances were evaluated based on the classification of depressive symptomatology, defined as PHQ-9 scores ?10. This gave each pairing an equal opportunity and avoided any bias in item pairing selection. Results: The ML-based PHQ-9 items 2 and 4 (phq2&4), the depressed mood and low-energy item pairing, and PHQ-9 items 2 and 8 (phq2&8), the depressed mood and psychomotor retardation or agitation item pairing, were found to be the best on the primary data set training split. They generalized well on the primary data set test split with area under the curves (AUCs) of 0.954 and 0.946, respectively, compared with an AUC of 0.942 for the PHQ-2. The phq2&4 had a higher AUC than the PHQ-2 on all 6 external data sets, and the phq2&8 had a higher AUC than the PHQ-2 on 3 data sets. The phq2&4 had the highest Youden index (an unweighted average of sensitivity and specificity) on 2 external data sets, and the phq2&8 had the highest Youden index on another 2. The PHQ-2?2 cutoff also had the highest Youden index on 2 external data sets, joint highest with the phq2&4 on 1, but its performance fluctuated the most. The PHQ-2?3 cutoff had the highest Youden index on 1 external data set. The sensitivity and specificity achieved by the phq2&4 and phq2&8 were more evenly balanced than the PHQ-2?2 and ?3 cutoffs. Conclusions: The PHQ-2 did not prove to be a more effective prescreening instrument when compared with other PHQ-9 item pairings. Evaluating all item pairings showed that, compared with alternative partner items, the anhedonia item underperformed alongside the depressed mood item. This suggests that the inclusion of anhedonia as a core symptom of depression and its presence in ultrabrief questionnaires may be incompatible with the empirical evidence. The use of the PHQ-2 to prescreen for depressive symptomatology could result in a greater number of misclassifications than alternative item pairings. UR - https://mental.jmir.org/2023/1/e48444 UR - http://dx.doi.org/10.2196/48444 UR - http://www.ncbi.nlm.nih.gov/pubmed/37856186 ID - info:doi/10.2196/48444 ER - TY - JOUR AU - Yang, Jun AU - Dong, Hang AU - Yu, Chao AU - Li, Bixia AU - Lin, Guozhen AU - Chen, Sujuan AU - Cai, Dongjie AU - Huang, Lin AU - Wang, Boguang AU - Li, Mengmeng PY - 2023/10/9 TI - Mortality Risk and Burden From a Spectrum of Causes in Relation to Size-Fractionated Particulate Matters: Time Series Analysis JO - JMIR Public Health Surveill SP - e41862 VL - 9 KW - size-fractionated particulate matter KW - cause-specific mortality KW - cardiovascular disease KW - respiratory disease KW - neoplasm KW - attributable burden N2 - Background: There is limited evidence regarding the adverse impact of particulate matters (PMs) on multiple body systems from both epidemiological and mechanistic studies. The association between size-fractionated PMs and mortality risk, as well as the burden of a whole spectrum of causes of death, remains poorly characterized. Objective: We aimed to examine the wide range of susceptible diseases affected by different sizes of PMs. We also assessed the association between PMs with an aerodynamic diameter less than 1 µm (PM1), 2.5 µm (PM2.5), and 10 µm (PM10) and deaths from 36 causes in Guangzhou, China. Methods: Daily data were obtained on cause-specific mortality, PMs, and meteorology from 2014 to 2016. A time-stratified case-crossover approach was applied to estimate the risk and burden of cause-specific mortality attributable to PMs after adjusting for potential confounding variables, such as long-term trend and seasonality, relative humidity, temperature, air pressure, and public holidays. Stratification analyses were further conducted to explore the potential modification effects of season and demographic characteristics (eg, gender and age). We also assessed the reduction in mortality achieved by meeting the new air quality guidelines set by the World Health Organization (WHO). Results: Positive and monotonic associations were generally observed between PMs and mortality. For every 10 ?g/m3 increase in 4-day moving average concentrations of PM1, PM2.5, and PM10, the risk of all-cause mortality increased by 2.00% (95% CI 1.08%-2.92%), 1.54% (95% CI 0.93%-2.16%), and 1.38% (95% CI 0.95%-1.82%), respectively. Significant effects of size-fractionated PMs were observed for deaths attributed to nonaccidental causes, cardiovascular disease, respiratory disease, neoplasms, chronic rheumatic heart diseases, hypertensive diseases, cerebrovascular diseases, stroke, influenza, and pneumonia. If daily concentrations of PM1, PM2.5, and PM10 reached the WHO target levels of 10, 15, and 45 ?g/m3, 7921 (95% empirical CI [eCI] 4454-11,206), 8303 (95% eCI 5063-11,248), and 8326 (95% eCI 5980-10690) deaths could be prevented, respectively. The effect estimates of PMs were relatively higher during hot months, among female individuals, and among those aged 85 years and older, although the differences between subgroups were not statistically significant. Conclusions: We observed positive and monotonical exposure-response curves between PMs and deaths from several diseases. The effect of PM1 was stronger on mortality than that of PM2.5 and PM10. A substantial number of premature deaths could be preventable by adhering to the WHO?s new guidelines for PMs. Our findings highlight the importance of a size-based strategy in controlling PMs and managing their health impact. UR - https://publichealth.jmir.org/2023/1/e41862 UR - http://dx.doi.org/10.2196/41862 UR - http://www.ncbi.nlm.nih.gov/pubmed/37812487 ID - info:doi/10.2196/41862 ER - TY - JOUR AU - Lyon, Matthieu AU - Fehlmann, Alain Christophe AU - Augsburger, Marc AU - Schaller, Thomas AU - Zimmermann-Ivol, Catherine AU - Celi, Julien AU - Gartner, Andrea Birgit AU - Lorenzon, Nicolas AU - Sarasin, François AU - Suppan, Laurent PY - 2023/10/6 TI - Evaluation of a Portable Blood Gas Analyzer for Prehospital Triage in Carbon Monoxide Poisoning: Instrument Validation Study JO - JMIR Form Res SP - e48057 VL - 7 KW - carbon monoxide poisoning KW - carbon monoxide intoxication KW - prehospital triage KW - Avoximeter 4000 KW - CO-oximetry KW - blood gas KW - blood work KW - pulse oximeter KW - cohort study KW - carbon monoxide KW - poisoning KW - sensor KW - triage tool KW - triage KW - oximeter KW - pilot study KW - medical device N2 - Background: Carbon monoxide (CO) poisoning is an important cause of morbidity and mortality worldwide. Symptoms are mostly aspecific, making it hard to identify, and its diagnosis is usually made through blood gas analysis. However, the bulkiness of gas analyzers prevents them from being used at the scene of the incident, thereby leading to the unnecessary transport and admission of many patients. While multiple-wavelength pulse oximeters have been developed to discriminate carboxyhemoglobin (COHb) from oxyhemoglobin, their reliability is debatable, particularly in the hostile prehospital environment. Objective: The main objective of this pilot study was to assess whether the Avoximeter 4000, a transportable blood gas analyzer, could be considered for prehospital triage. Methods: This was a monocentric, prospective, pilot evaluation study. Blood samples were analyzed sequentially with 2 devices: the Avoximeter 4000 (experimental), which performs direct measurements on blood samples of about 50 µL by analyzing light absorption at 5 different wavelengths; and the ABL827 FLEX (control), which measures COHb levels through an optical system composed of a 128-wavelength spectrophotometer. The blood samples belonged to 2 different cohorts: the first (clinical cohort) was obtained in an emergency department and consisted of 68 samples drawn from patients admitted for reasons other than CO poisoning. These samples were used to determine whether the Avoximeter 4000 could properly exclude the diagnosis. The second (forensic) cohort was derived from the regional forensic center, which provided 12 samples from documented CO poisoning. Results: The mean COHb level in the clinical cohort was 1.7% (SD 1.8%; median 1.2%, IQR 0.7%-1.9%) with the ABL827 FLEX versus 3.5% (SD 2.3%; median 3.1%, IQR 2.2%-4.1%) with the Avoximeter 4000. Therefore, the Avoximeter 4000 overestimated COHb levels by a mean difference of 1.8% (95% CI 1.5%-2.1%). The consistency of COHb readings by the Avoximeter 4000 was excellent, with an intraclass correlation coefficient of 0.97 (95% CI 0.93-0.99) when the same blood sample was analyzed repeatedly. Using prespecified cutoffs (5% in nonsmokers and 10% in smokers), 3 patients (4%) had high COHb levels according to the Avoximeter 4000, while their values were within the normal range according to the ABL827 FLEX. Therefore, the specificity of the Avoximeter 4000 in this cohort was 95.6% (95% CI 87%-98.6%), and the overtriage rate would have been 4.4% (95% CI 1.4%-13%). Regarding the forensic samples, 10 of 12 (83%) samples were positive with both devices, while the 2 remaining samples were negative with both devices. Conclusions: The limited difference in COHb level measurements between the Avoximeter 4000 and the control device, which erred on the side of safety, and the relatively low overtriage rate warrant further exploration of this device as a prehospital triage tool. UR - https://formative.jmir.org/2023/1/e48057 UR - http://dx.doi.org/10.2196/48057 UR - http://www.ncbi.nlm.nih.gov/pubmed/37801355 ID - info:doi/10.2196/48057 ER - TY - JOUR AU - Francis-Oliviero, Florence AU - Loubières, Céline AU - Grové, Christine AU - Marinucci, Alexandra AU - Shankland, Rebecca AU - Salamon, Réda AU - Perez, Emmanuelle AU - Garancher, Laure AU - Galera, Cédric AU - Gaillard, Elsa AU - Orri, Massimiliano AU - González-Caballero, Luis Juan AU - Montagni, Ilaria PY - 2023/10/5 TI - Improving Children?s Mental Health Literacy Through the Cocreation of an Intervention and Scale Validation: Protocol for the CHILD-Mental Health Literacy Research Study JO - JMIR Res Protoc SP - e51096 VL - 12 KW - child KW - mental health KW - literacy KW - intervention KW - scale N2 - Background: Children?s mental health is a public health priority, with 1 in 5 European children younger than 12 years having a behavioral, developmental, or psychological disorder. Mental health literacy (MHL) is a modifiable determinant of mental health, promoting psychological well-being and reducing mental health problems. Despite its significance, no interventions or scales currently exist for increasing and measuring MHL in this population. Objective: This study has dual objectives: (1) cocreating and evaluating an intervention on children?s MHL, and (2) developing and validating a scale that measures children?s MHL. Methods: Our study focuses on children aged 9-11 years attending primary school classes in various settings, including urban and rural areas, and priority education zones within a French department. Using a participatory research approach, we will conduct workshops involving children, parents, teachers, and 1 artist to cocreate an intervention comprising multiple tools (eg, a pedagogical kit and videos). This intervention will undergo initial evaluation in 4 classes through observations, interviews, and satisfaction questionnaires to assess its viability. Concurrently, the artist will collaborate with children to create the initial version of the CHILD-MHL scale, which will then be administered to 300 children. Psychometric analyses will validate the scale. Subsequently, we will conduct a cluster randomized controlled trial involving a minimum of 20 classes, using the CHILD-MHL scale scores as the primary end point to evaluate the intervention?s efficacy. Additional interviews will complement this mixed methods evaluation. Both the intervention and the scale are grounded in the Child-Focused MHL model. Results: The first tool of the intervention is the pedagogical kit Le Jardin du Dedans, supported by the public organization Psycom Santé Mentale Info and endorsed by UNICEF (United Nations Children?s Fund) France. The second tool is a handbook by the Pan American Health Organization and the World Health Organization, which is addressed to teachers to sensitize them to children?s mental health problems. The third is a 5-page supplementary leaflet produced by the nongovernmental organization The Ink Link, which teaches children the notion of MHL. Finally, we produced 56 items of the MHL Scale and listed existing education policies for children?s mental health. Conclusions: After its robust evaluation, the intervention could be extended to several schools in France. The scale will be the first in the world to measure children?s MHL. It will be used not only to evaluate interventions but also to provide data for decision makers to include MHL in all educational policies. Both the intervention and the scale could be translated into other languages. International Registered Report Identifier (IRRID): PRR1-10.2196/51096 UR - https://www.researchprotocols.org/2023/1/e51096 UR - http://dx.doi.org/10.2196/51096 UR - http://www.ncbi.nlm.nih.gov/pubmed/37796588 ID - info:doi/10.2196/51096 ER - TY - JOUR AU - Kim, Hyeonseong AU - Jeong, Seohyun AU - Hwang, Inae AU - Sung, Kiyoung AU - Moon, Woori AU - Shin, Min-Sup PY - 2023/9/29 TI - Validation of a Brief Internet-Based Self-Report Measure of Maladaptive Personality and Interpersonal Schema: Confirmatory Factor Analysis JO - Interact J Med Res SP - e48425 VL - 12 KW - maladaptive schema KW - measure of schema KW - self-report measure KW - internet-based measure KW - digital mental health care KW - interpersonal schema N2 - Background: Existing digital mental health interventions mainly focus on the symptoms of specific mental disorders, but do not focus on Maladaptive Personalities and Interpersonal Schemas (MPISs). As an initial step toward considering personalities and schemas in intervention programs, there is a need for the development of tools for measuring core personality traits and interpersonal schemas known to cause psychological discomfort among potential users of digital mental health interventions. Thus, the MPIS was developed. Objective: The objectives of this study are to validate the MPIS by comparing 2 models of the MPIS factor structure and to understand the characteristics of the MPIS by assessing its correlations with other measures. Methods: Data were collected from 234 participants who were using web-based community sites in South Korea, including university students, graduate students, working professionals, and homemakers. All the data were gathered through web-based surveys. Confirmatory factor analysis was used to compare a single-factor model with a 5-factor model. Reliability and correlation analyses with other scales were performed. Results: The results of confirmatory factor analysis indicated that the 5-factor model (?2550=1278.1; Tucker-Lewis index=0.80; comparative fit index=0.81; and Root Mean Square Error of Approximation=0.07) was more suitable than the single-factor model (?2560=2341.5; Tucker-Lewis index=0.52; comparative fit index=0.54; and Root Mean Square Error of Approximation=0.11) for measuring maladaptive personality traits and interpersonal relationship patterns. The internal consistency of each factor of the MPIS was good (Cronbach ?=.71-.88), and the correlations with existing measures were statistically significant. The MPIS is a validated 35-item tool for measuring 5 essential personality traits and interpersonal schemas in adults aged 18-39 years. Conclusions: This study introduced the MPIS, a concise and effective questionnaire capable of measuring maladaptive personality traits and interpersonal relationship schemas. Through analysis, the MPIS was shown to reliably assess these psychological constructs and validate them. Its web-based accessibility and reduced item count make it a valuable tool for mental health assessment. Future applications include its integration into digital mental health care services, allowing easy web-based administration and aiding in the classification of psychological therapy programs based on the obtained results. Trial Registration: ClinicalTrials.gov NCT05952063; https://www.clinicaltrials.gov/study/NCT05952063 UR - https://www.i-jmr.org/2023/1/e48425 UR - http://dx.doi.org/10.2196/48425 UR - http://www.ncbi.nlm.nih.gov/pubmed/37773606 ID - info:doi/10.2196/48425 ER - TY - JOUR AU - Sideropoulos, Vassilis AU - Vangeli, Eleni AU - Naughton, Felix AU - Cox, Sharon AU - Frings, Daniel AU - Notley, Caitlin AU - Brown, Jamie AU - Kimber, Catherine AU - Dawkins, Lynne PY - 2023/9/27 TI - Mobile Phone Text Messages to Support People to Stop Smoking by Switching to Vaping: Codevelopment, Coproduction, and Initial Testing Study JO - JMIR Form Res SP - e49668 VL - 7 KW - coproduction KW - SMS text messages KW - e-cigarette KW - smoking KW - eHealth KW - vaping KW - mobile phone KW - codevelopment KW - text message N2 - Background: SMS text messages are affordable, scalable, and effective smoking cessation interventions. However, there is little research on SMS text message interventions specifically designed to support people who smoke to quit by switching to vaping. Objective: Over 3 phases, with vapers and smokers, we codeveloped and coproduced a mobile phone SMS text message program. The coproduction paradigm allowed us to collaborate with researchers and the community to develop a more relevant, acceptable, and equitable SMS text message program. Methods: In phase 1, we engaged people who vape via Twitter and received 167 responses to our request to write SMS text messages for people who wish to quit smoking by switching to vaping. We screened, adjusted, refined, and themed the messages, resulting in a set of 95 that were mapped against the Capability, Opportunity, and Motivation?Behavior constructs. In phase 2, we evaluated the 95 messages from phase 1 via a web survey where participants (66/202, 32.7% woman) rated up to 20 messages on 7-point Likert scales on 9 constructs: being understandable, clear, believable, helpful, interesting, inoffensive, positive, and enthusiastic and how happy they would be to receive the messages. In phase 3, we implemented the final set of SMS text messages as part of a larger randomized optimization trial, in which 603 participants (mean age 38.33, SD 12.88 years; n=369, 61.2% woman) received SMS text message support and then rated their usefulness and frequency and provided free-text comments at the 12-week follow-up. Results: For phase 2, means and SDs were calculated for each message across the 9 constructs. Those with means below the neutral anchor of 4 or with unfavorable comments were discussed with vapers and further refined or removed. This resulted in a final set of 78 that were mapped against early, mid-, or late stages of quitting to create an order for the messages. For phase 3, a total of 38.5% (232/603) of the participants provided ratings at the 12-week follow-up. In total, 69.8% (162/232) reported that the SMS text messages had been useful, and a significant association between quit rates and usefulness ratings was found (?21=9.6; P=.002). A content analysis of free-text comments revealed that the 2 most common positive themes were helpful (13/47, 28%) and encouraging (6/47, 13%) and the 2 most common negative themes were too frequent (9/47, 19%) and annoying (4/47, 9%). Conclusions: In this paper, we describe the initial coproduction and codevelopment of a set of SMS text messages to help smokers stop smoking by transitioning to vaping. We encourage researchers to use, further develop, and evaluate the set of SMS text messages and adapt it to target populations and relevant contexts. UR - https://formative.jmir.org/2023/1/e49668 UR - http://dx.doi.org/10.2196/49668 UR - http://www.ncbi.nlm.nih.gov/pubmed/37756034 ID - info:doi/10.2196/49668 ER - TY - JOUR AU - Karystianis, George AU - Simpson, Paul AU - Lukmanjaya, Wilson AU - Ginnivan, Natasha AU - Nenadic, Goran AU - Buchan, Iain AU - Butler, Tony PY - 2023/9/22 TI - Automatic Extraction of Research Themes in Epidemiological Criminology From PubMed Abstracts From 1946 to 2020: Text Mining Study JO - JMIR Form Res SP - e49721 VL - 7 KW - epidemiology KW - study determinant KW - study outcome KW - PubMed KW - research priorities KW - epidemiological criminology KW - criminology KW - open research N2 - Background: The emerging field of epidemiological criminology studies the intersection between public health and justice systems. To increase the value of and reduce waste in research activities in this area, it is important to perform transparent research priority setting considering the needs of research beneficiaries and end users along with a systematic assessment of the existing research activities to address gaps and harness opportunities. Objective: In this study, we aimed to examine published research outputs in epidemiological criminology to assess gaps between published outputs and current research priorities identified by prison stakeholders. Methods: A rule-based method was applied to 23,904 PubMed epidemiological criminology abstracts to extract the study determinants and outcomes (ie, ?themes?). These were mapped against the research priorities identified by Australian prison stakeholders to assess the differences from research outputs. The income level of the affiliation country of the first authors was also identified to compare the ranking of research priorities in countries categorized by income levels. Results: On an evaluation set of 100 abstracts, the identification of themes returned an F1-score of 90%, indicating reliable performance. More than 53.3% (11,927/22,361) of the articles had at least 1 extracted theme; the most common was substance use (1533/11,814, 12.97%), followed by HIV (1493/11,814, 12.64%). The infectious disease category (2949/11,814, 24.96%) was the most common research priority category, followed by mental health (2840/11,814, 24.04%) and alcohol and other drug use (2433/11,814, 20.59%). A comparison between the extracted themes and the stakeholder priorities showed an alignment for mental health, infectious diseases, and alcohol and other drug use. Although behavior- and juvenile-related themes were common, they did not feature as prison priorities. Most studies were conducted in high-income countries (10,083/11,814, 85.35%), while countries with the lowest income status focused half of their research on infectious diseases (47/91, 52%). Conclusions: The identification of research themes from PubMed epidemiological criminology research abstracts is possible through the application of a rule-based text mining method. The frequency of the investigated themes may reflect historical developments concerning disease prevalence, treatment advances, and the social understanding of illness and incarcerated populations. The differences between income status groups are likely to be explained by local health priorities and immediate health risks. Notable gaps between stakeholder research priorities and research outputs concerned themes that were more focused on social factors and systems and may reflect publication bias or self-publication selection, highlighting the need for further research on prison health services and the social determinants of health. Different jurisdictions, countries, and regions should undertake similar systematic and transparent research priority?setting processes. UR - https://formative.jmir.org/2023/1/e49721 UR - http://dx.doi.org/10.2196/49721 UR - http://www.ncbi.nlm.nih.gov/pubmed/37738080 ID - info:doi/10.2196/49721 ER - TY - JOUR AU - Condron, Claire AU - Power, Mide AU - Mathew, Midhun AU - Lucey, M. Siobhan PY - 2023/9/20 TI - Gender Equality Training for Students in Higher Education: Protocol for a Scoping Review JO - JMIR Res Protoc SP - e44584 VL - 12 KW - gender equality KW - student leaders KW - simulation-based education KW - communications skills KW - training KW - higher education KW - sustainable goal KW - scoping KW - review method KW - PRISMA KW - Preferred Reporting Items for Systematic Reviews and Meta-Analyses KW - search strategy KW - gender KW - equality KW - equalities KW - inclusion KW - diversity KW - postsecondary KW - student KW - teaching KW - coaching KW - teacher KW - educator N2 - Background: The principles of gender equality are integral to the goals, targets, and indicators of all sustainable development goals. Higher education institutes can be powerful agents for promoting gender equality, diversity, and inclusion not only in the higher education context but also in society as a whole. To address and overcome gender inequality in the higher education environment, experts posit that change needs to occur from day 1 of the student?s academic experience. To this end, training is required. A preliminary review of the literature indicates that multiple gender equality?based training programs or initiatives for students have been designed and evaluated in second and third-level education settings. Examples of educational activities undertaken include delivery of didactic teaching, participation in a face-to-face collaboration project, site visits, case studies, and coaching. Yet, our initial search indicated that, to date, a comprehensive review collating the available evidence on gender equality training for third-level students has not yet been carried out. Objective: Our review seeks to identify and explore the existing literature on gender equality training interventions for third-level students, with a particular emphasis on training content, methodology, and outcome evaluation. Methods: This scoping review will be structured using the Arskey and O?Malley?s 5-stage framework and will consider empirical research and other relevant published works that address gender equality training. Systematic searches will be carried out in 6 research databases and the gray literature using key search terms. Inclusion and exclusion criteria have been defined, and a data charting tool created to methodically extract information from selected literature. The free web software Rayyan will be used for primary screening where each reference will be screened in duplicate first by title, then abstract, and finally by full text. Results: This review forms part of the LIBRA (Balance) study and has received peer-reviewed grant funding from the Irish Higher Education Authority. LIBRA aims to use simulation-based education to develop a gender equality leadership training program for student leaders in higher education. The findings will be summarized in tabular form, and a narrative synthesis produced to inform curriculum development. Conclusions: This review seeks to inform curriculum design by reporting on the gender equality?enabling skills and leadership skills necessary to foster gender equality. This paper should inform recommendations for training and catalyze future research in this rapidly evolving area. International Registered Report Identifier (IRRID): DERR1-10.2196/44584 UR - https://www.researchprotocols.org/2023/1/e44584 UR - http://dx.doi.org/10.2196/44584 UR - http://www.ncbi.nlm.nih.gov/pubmed/37728987 ID - info:doi/10.2196/44584 ER - TY - JOUR AU - Børøsund, Elin AU - Meland, Anders AU - Eriksen, R. Hege AU - Rygg, M. Christine AU - Ursin, Giske AU - Solberg Nes, Lise PY - 2023/9/19 TI - Digital Cognitive Behavioral- and Mindfulness-Based Stress-Management Interventions for Survivors of Breast Cancer: Development Study JO - JMIR Form Res SP - e48719 VL - 7 KW - cancer KW - stress management KW - mindfulness KW - cognitive behavioral therapy KW - digital KW - eHealth KW - mHealth KW - app KW - user-driven development KW - usability N2 - Background: Psychosocial stress-management interventions can reduce stress and distress and improve the quality of life for survivors of cancer. As these in-person interventions are not always offered or accessible, evidence-informed digital stress-management interventions may have the potential to improve outreach of psychosocial support for survivors of cancer. Few such digital interventions exist so far, few if any have been developed specifically for survivors of breast cancer, and few if any have attempted to explore more than 1 distinct type of intervention framework. Objective: This study aimed to develop 2 digital psychosocial stress-management interventions for survivors of breast cancer; 1 cognitive behavioral therapy-based intervention (CBI), and 1 mindfulness-based intervention (MBI). Methods: The development of the CBI and MBI interventions originated from the existing StressProffen program, a digital stress-management intervention program for survivors of cancer, based on a primarily cognitive behavioral therapeutic concept. Development processes entailed a multidisciplinary design approach and were iteratively conducted in close collaboration between key stakeholders, including experts within psychosocial oncology, cancer epidemiology, stress-management, and eHealth as well as survivors of breast cancer and health care providers. Core psychosocial oncology stress-management and cancer epidemiology experts first conducted a series of workshops to identify cognitive behavioral and mindfulness specific StressProffen content, overlapping psychoeducational content, and areas where development and incorporation of new material were needed. Following the program content adaptation and development phase, phases related to user testing of new content and technical, privacy, security, and ethical aspects and adjustments ensued. Intervention content for the distinct CBI and MBI interventions was refined in iterative user-centered design processes and adjusted to electronic format through stakeholder-centered iterations. Results: For the CBI version, the mindfulness-based content of the original StressProffen was removed, and for the MBI version, cognitive behavioral content was removed. Varying degrees of new content were created for both versions, using a similar layout as for the original StressProffen program. New content and new exercises in particular were tested by survivors of breast cancer and a project-related editorial team, resulting in subsequent user centered adjustments, including ensuring auditory versions and adequate explanations before less intuitive sections. Other improvements included implementing a standard closing sentence to round off every exercise, and allowing participants to choose the length of some of the mindfulness exercises. A legal disclaimer and a description of data collection, user rights and study contact information were included to meet ethical, privacy, and security requirements. Conclusions: This study shows how theory specific (ie, CBI and MBI) digital stress-management interventions for survivors of breast cancer can be developed through extensive collaborations between key stakeholders, including scientists, health care providers, and survivors of breast cancer. Offering a variety of evidence-informed stress-management approaches may potentially increase interest for outreach and impact of psychosocial interventions for survivors of cancer. International Registered Report Identifier (IRRID): RR2-10.2196/47195 UR - https://formative.jmir.org/2023/1/e48719 UR - http://dx.doi.org/10.2196/48719 UR - http://www.ncbi.nlm.nih.gov/pubmed/37725424 ID - info:doi/10.2196/48719 ER - TY - JOUR AU - Wu, Y. Danny T. AU - Hanauer, David AU - Murdock, Paul AU - Vydiswaran, Vinod V. G. AU - Mei, Qiaozhu AU - Zheng, Kai PY - 2023/9/15 TI - Developing a Semantically Based Query Recommendation for an Electronic Medical Record Search Engine: Query Log Analysis and Design Implications JO - JMIR Form Res SP - e45376 VL - 7 KW - electronic health records KW - information retrieval KW - user-centered evaluation KW - query recommendation KW - query log analysis KW - clinical research informatics N2 - Background: An effective and scalable information retrieval (IR) system plays a crucial role in enabling clinicians and researchers to harness the valuable information present in electronic health records. In a previous study, we developed a prototype medical IR system, which incorporated a semantically based query recommendation (SBQR) feature. The system was evaluated empirically and demonstrated high perceived performance by end users. To delve deeper into the factors contributing to this perceived performance, we conducted a follow-up study using query log analysis. Objective: One of the primary challenges faced in IR is that users often have limited knowledge regarding their specific information needs. Consequently, an IR system, particularly its user interface, needs to be thoughtfully designed to assist users through the iterative process of refining their queries as they encounter relevant documents during their search. To address these challenges, we incorporated ?query recommendation? into our Electronic Medical Record Search Engine (EMERSE), drawing inspiration from the success of similar features in modern IR systems for general purposes. Methods: The query log data analyzed in this study were collected during our previous experimental study, where we developed EMERSE with the SBQR feature. We implemented a logging mechanism to capture user query behaviors and the output of the IR system (retrieved documents). In this analysis, we compared the initial query entered by users with the query formulated with the assistance of the SBQR. By examining the results of this comparison, we could examine whether the use of SBQR helped in constructing improved queries that differed from the original ones. Results: Our findings revealed that the first query entered without SBQR and the final query with SBQR assistance were highly similar (Jaccard similarity coefficient=0.77). This suggests that the perceived positive performance of the system was primarily attributed to the automatic query expansion facilitated by the SBQR rather than users manually manipulating their queries. In addition, through entropy analysis, we observed that search results converged in scenarios of moderate difficulty, and the degree of convergence correlated strongly with the perceived system performance. Conclusions: The study demonstrated the potential contribution of the SBQR in shaping participants' positive perceptions of system performance, contingent upon the difficulty of the search scenario. Medical IR systems should therefore consider incorporating an SBQR as a user-controlled option or a semiautomated feature. Future work entails redesigning the experiment in a more controlled manner and conducting multisite studies to demonstrate the effectiveness of EMERSE with SBQR for patient cohort identification. By further exploring and validating these findings, we can enhance the usability and functionality of medical IR systems in real-world settings. UR - https://formative.jmir.org/2023/1/e45376 UR - http://dx.doi.org/10.2196/45376 UR - http://www.ncbi.nlm.nih.gov/pubmed/37713239 ID - info:doi/10.2196/45376 ER - TY - JOUR AU - Oyedele, K. Natasha AU - Lansey, G. Dina AU - Chiew, Calvin AU - Chan, Cupid AU - Quon, Harry AU - Dean, T. Lorraine PY - 2023/9/14 TI - Development and Testing of a Mobile App to Collect Social Determinants of Health Data in Cancer Settings: Interview Study JO - JMIR Form Res SP - e48737 VL - 7 KW - social determinants of health KW - mobile apps KW - medical oncology KW - mobile phone N2 - Background: Social determinants of health (SDOH) such as lack of basic resources, housing, transportation, and social isolation play an important role for patients on the cancer care continuum. Health systems? current technological solutions for identifying and managing patients? SDOH data largely focus on information recorded in the electronic health record by providers, which is often inaccessible to patients to contribute to or modify. Objective: We developed and tested a patient-centric SDOH screening tool designed for use on patients? personal mobile phone that preserves patient privacy and confidentiality, collects information about the unmet social needs of patients with cancer, and communicates them to the provider. Methods: We interviewed 22 patients with cancer, oncologists, and social workers associated with a US-based comprehensive cancer center to better understand how patients? SDOH information is collected and reported. After triangulating data obtained from thematic analysis of interviews, an environmental scan, and a literature search of validated tools to collect SDOH data, we developed an SDOH screening tool mobile app and conducted a pilot study of 16 dyadic pairs of patients and cancer care team members at the same cancer center. We collected patient SDOH data using 36 survey items covering 7 SDOH domains and used validated scales and follow-up interviews to assess the app?s usability and acceptability among patients and cancer care team members. Results: Formative interviews with patients and care team members revealed that transportation, financial challenges, food insecurity, and low health literacy were common SDOH challenges and that a mobile app that collected those data, shared those data with care team members, and offered supportive resources could be useful and valuable. In the pilot study, 25% (4/16) of app-using patients reported having at least one of the abovementioned social needs; the most common social need was social isolation (7/16, 44%). Patients rated the mobile app as easy to use, accurately capturing their SDOH, and preserving their privacy but suggested that the app could be more helpful by connecting patients to actual resources. Providers reported high acceptability and usability of the app. Conclusions: Use of a brief, patient-centric, mobile app?based SDOH screening tool can effectively capture SDOH of patients with cancer for care team members in a way that preserves patient privacy and that is acceptable and usable for patients and care team members. However, only collecting SDOH information is not sufficient; usefulness can be increased by connecting patients directly to resources to address their unmet social needs. UR - https://formative.jmir.org/2023/1/e48737 UR - http://dx.doi.org/10.2196/48737 UR - http://www.ncbi.nlm.nih.gov/pubmed/37707880 ID - info:doi/10.2196/48737 ER - TY - JOUR AU - Matsuda, Shinichi AU - Ohtomo, Takumi AU - Okuyama, Masaru AU - Miyake, Hiraku AU - Aoki, Kotonari PY - 2023/9/14 TI - Estimating Patient Satisfaction Through a Language Processing Model: Model Development and Evaluation JO - JMIR Form Res SP - e48534 VL - 7 KW - breast cancer KW - internet KW - machine learning KW - natural language processing KW - natural language-processing model KW - neural network KW - NLP KW - patient satisfaction KW - textual data N2 - Background: Measuring patient satisfaction is a crucial aspect of medical care. Advanced natural language processing (NLP) techniques enable the extraction and analysis of high-level insights from textual data; nonetheless, data obtained from patients are often limited. Objective: This study aimed to create a model that quantifies patient satisfaction based on diverse patient-written textual data. Methods: We constructed a neural network?based NLP model for this cross-sectional study using the textual content from disease blogs written in Japanese on the Internet between 1994 and 2020. We extracted approximately 20 million sentences from 56,357 patient-authored disease blogs and constructed a model to predict the patient satisfaction index (PSI) using a regression approach. After evaluating the model?s effectiveness, PSI was predicted before and after cancer notification to examine the emotional impact of cancer diagnoses on 48 patients with breast cancer. Results: We assessed the correlation between the predicted and actual PSI values, labeled by humans, using the test set of 169 sentences. The model successfully quantified patient satisfaction by detecting nuances in sentences with excellent effectiveness (Spearman correlation coefficient [?]=0.832; root-mean-squared error [RMSE]=0.166; P<.001). Furthermore, the PSI was significantly lower in the cancer notification period than in the preceding control period (?0.057 and ?0.012, respectively; 2-tailed t47=5.392, P<.001), indicating that the model quantifies the psychological and emotional changes associated with the cancer diagnosis notification. Conclusions: Our model demonstrates the ability to quantify patient dissatisfaction and identify significant emotional changes during the disease course. This approach may also help detect issues in routine medical practice. UR - https://formative.jmir.org/2023/1/e48534 UR - http://dx.doi.org/10.2196/48534 UR - http://www.ncbi.nlm.nih.gov/pubmed/37707946 ID - info:doi/10.2196/48534 ER - TY - JOUR AU - Yordanov, Stefan AU - Yang, Xiaoyu AU - Mowforth, Oliver AU - K Demetriades, Andreas AU - Ivanov, Marcel AU - Vergara, Pierluigi AU - Gardner, Adrian AU - Pereira, Erlick AU - Bateman, Antony AU - Alamri, Alexander AU - Francis, Jibin AU - Trivedi, Rikin AU - Kotter, Mark AU - Davies, Benjamin AU - Budu, Alexandru AU - PY - 2023/9/12 TI - Factors Influencing Surgical Decision-Making in the Posterior Laminectomy With Fixation for Degenerative Cervical Myelopathy (POLYFIX-DCM) Trial: Survey Study JO - JMIR Form Res SP - e48321 VL - 7 KW - cervical myelopathy KW - spondylosis KW - spondylotic stenosis KW - disc herniation KW - ossification posterior longitudinal ligament KW - degeneration KW - disability KW - recovery KW - questionnaire KW - decision-making KW - surgeons KW - myelopathy KW - stress KW - spinal cord KW - surgery KW - decompression KW - laminectomy N2 - Background: Degenerative cervical myelopathy (DCM) is estimated to affect 2% of the adult population. DCM occurs when degenerative processes cause compression and injure the spinal cord. Surgery to remove the stress caused by the compression of the spinal cord is the mainstay of treatment, with a range of techniques in use. Although various factors are described to inform the selection of these techniques, there needs to be more consensus and limited comparative evidence. Objective: The main objective of this survey was to explore the variation of practice and decision-making, with a focus on laminectomy versus laminectomy and fusion in posterior surgery of the cervical spine. We present the results of a survey conducted among the principal investigators (PIs) of the National Institute for Health and Care Research (NIHR) randomized controlled trial on posterior laminectomy with fixation for degenerative cervical myelopathy (POLYFIX-DCM). Methods: A series of 7 cases were shared with 24 PIs using SurveyMonkey. Each case consisted of a midsagittal T2-weighted magnetic resonance imaging and lateral cervical x-rays in flexion and extension. Surgeons were asked if their preferred approach was anterior or posterior. If posterior, they were asked whether they preferred to instrument and whether they had the equipoise to randomize in the NIHR POLYFIX-DCM trial. Variability in decision-making was then explored using factors reported to inform decision-making, such as alignment, location of compression, number of levels operated, presence of mobile spondylolisthesis, and patient age. Results: The majority of PIs (16/30, 53%) completed the survey. Overall, PIs favored a posterior approach (12/16, 75%) with instrumentation (75/112, average 66%) and would randomize (67/112, average 62%) most cases. Factors reported to inform decision-making poorly explained variability in responses in both univariate testing and with a multivariate model (R2=0.1). Only surgeon experience of more than 5 years and orthopedic specialty training background were significant predictors, both associated with an anterior approach (odds ratio [OR] 1.255; P=.02 and OR 1.344; P=.007, respectively) and fusion for posterior procedures (OR 0.628; P<.001 and OR 1.344; P<.001, respectively). Surgeon experience also significantly affected the openness to randomize, with those with more than 5 years of experience less likely to randomize (OR ?0.68; P<.001). Conclusions: In this representative sample of spine surgeons participating in the POLYFIX-DCM trial as investigators, there is no consensus on surgical strategy, including the role of instrumented fusion following posterior decompression. Overall, this study supports the view that there appears to be a clinical equipoise, and conceptually, a randomized controlled trial appears feasible, which sets the scene for the NIHR POLYFIX-DCM trial. UR - https://formative.jmir.org/2023/1/e48321 UR - http://dx.doi.org/10.2196/48321 UR - http://www.ncbi.nlm.nih.gov/pubmed/37698903 ID - info:doi/10.2196/48321 ER - TY - JOUR AU - Bin, Jia Kaio AU - De Pretto, Ramos Lucas AU - Sanchez, Beltrame Fábio AU - De Souza e Castro, Muniz Fabio Pacheco AU - Ramos, Delgado Vinicius AU - Battistella, Rizzo Linamara PY - 2023/9/12 TI - Digital Platform for Continuous Monitoring of Patients Using a Smartwatch: Longitudinal Prospective Cohort Study JO - JMIR Form Res SP - e47388 VL - 7 KW - smartwatch KW - digital health KW - telemedicine KW - wearable KW - telemonitoring KW - mobile health KW - General Data Protection Regulation KW - GDPR KW - Lei Geral de Proteção de Dados KW - LGPD KW - digital platform KW - clinical intervention KW - sensitive data KW - clinical trial KW - mobile phone N2 - Background: Since the COVID-19 pandemic, there has been a boost in the digital transformation of the human society, where wearable devices such as a smartwatch can already measure vital signs in a continuous and naturalistic way; however, the security and privacy of personal data is a challenge to expanding the use of these data by health professionals in clinical follow-up for decision-making. Similar to the European General Data Protection Regulation, in Brazil, the Lei Geral de Proteção de Dados established rules and guidelines for the processing of personal data, including those used for patient care, such as those captured by smartwatches. Thus, in any telemonitoring scenario, there is a need to comply with rules and regulations, making this issue a challenge to overcome. Objective: This study aimed to build a digital solution model for capturing data from wearable devices and making them available in a safe and agile manner for clinical and research use, following current laws. Methods: A functional model was built following the Brazilian Lei Geral de Proteção de Dados (2018), where data captured by smartwatches can be transmitted anonymously over the Internet of Things and be identified later within the hospital. A total of 80 volunteers were selected for a 24-week follow-up clinical trial divided into 2 groups, one group with a previous diagnosis of COVID-19 and a control group without a previous diagnosis of COVID-19, to measure the synchronization rate of the platform with the devices and the accuracy and precision of the smartwatch in out-of-hospital conditions to simulate remote monitoring at home. Results: In a 35-week clinical trial, >11.2 million records were collected with no system downtime; 66% of continuous beats per minute were synchronized within 24 hours (79% within 2 days and 91% within a week). In the limit of agreement analysis, the mean differences in oxygen saturation, diastolic blood pressure, systolic blood pressure, and heart rate were ?1.280% (SD 5.679%), ?1.399 (SD 19.112) mm Hg, ?1.536 (SD 24.244) mm Hg, and 0.566 (SD 3.114) beats per minute, respectively. Furthermore, there was no difference in the 2 study groups in terms of data analysis (neither using the smartwatch nor the gold-standard devices), but it is worth mentioning that all volunteers in the COVID-19 group were already cured of the infection and were highly functional in their daily work life. Conclusions: On the basis of the results obtained, considering the validation conditions of accuracy and precision and simulating an extrahospital use environment, the functional model built in this study is capable of capturing data from the smartwatch and anonymously providing it to health care services, where they can be treated according to the legislation and be used to support clinical decisions during remote monitoring. UR - https://formative.jmir.org/2023/1/e47388 UR - http://dx.doi.org/10.2196/47388 UR - http://www.ncbi.nlm.nih.gov/pubmed/37698916 ID - info:doi/10.2196/47388 ER - TY - JOUR AU - Almansour, Amal AU - Montague, Enid AU - Furst, Jacob AU - Raicu, Daniela PY - 2023/9/8 TI - Evaluation of Eye Gaze Dynamics During Physician-Patient-Computer Interaction in Federally Qualified Health Centers: Systematic Analysis JO - JMIR Hum Factors SP - e46120 VL - 10 KW - patient-physician-computer interaction KW - nonverbal communication KW - Federally Qualified Health Centers KW - primary care encounter N2 - Background: Understanding the communication between physicians and patients can identify areas where they can improve and build stronger relationships. This led to better patient outcomes including increased engagement, enhanced adherence to treatment plan, and a boost in trust. Objective: This study investigates eye gaze directions of physicians, patients, and computers in naturalistic medical encounters at Federally Qualified Health Centers to understand communication patterns given different patients? diverse backgrounds. The aim is to support the building and designing of health information technologies, which will facilitate the improvement of patient outcomes. Methods: Data were obtained from 77 videotaped medical encounters in 2014 from 3 Federally Qualified Health Centers in Chicago, Illinois, that included 11 physicians and 77 patients. Self-reported surveys were collected from physicians and patients. A systematic analysis approach was used to thoroughly examine and analyze the data. The dynamics of eye gazes during interactions between physicians, patients, and computers were evaluated using the lag sequential analysis method. The objective of the study was to identify significant behavior patterns from the 6 predefined patterns initiated by both physicians and patients. The association between eye gaze patterns was examined using the Pearson chi-square test and the Yule Q test. Results: The results of the lag sequential method showed that 3 out of 6 doctor-initiated gaze patterns were followed by patient-response gaze patterns. Moreover, 4 out of 6 patient-initiated patterns were significantly followed by doctor-response gaze patterns. Unlike the findings in previous studies, doctor-initiated eye gaze behavior patterns were not leading patients? eye gaze. Moreover, patient-initiated eye gaze behavior patterns were significant in certain circumstances, particularly when interacting with physicians. Conclusions: This study examined several physician-patient-computer interaction patterns in naturalistic settings using lag sequential analysis. The data indicated a significant influence of the patients? gazes on physicians. The findings revealed that physicians demonstrated a higher tendency to engage with patients by reciprocating the patient?s eye gaze when the patient looked at them. However, the reverse pattern was not observed, suggesting a lack of reciprocal gaze from patients toward physicians and a tendency to not direct their gaze toward a specific object. Furthermore, patients exhibited a preference for the computer when physicians directed their eye gaze toward it. UR - https://humanfactors.jmir.org/2023/1/e46120 UR - http://dx.doi.org/10.2196/46120 UR - http://www.ncbi.nlm.nih.gov/pubmed/37682590 ID - info:doi/10.2196/46120 ER - TY - JOUR AU - Delmonaco, Daniel AU - Li, Shannon AU - Paneda, Christian AU - Popoff, Elliot AU - Hughson, Luna AU - Jadwin-Cakmak, Laura AU - Alferio, Jack AU - Stephenson, Christian AU - Henry, Angelique AU - Powdhar, Kiandra AU - Gierlinger, Isabella AU - Harper, W. Gary AU - Haimson, L. Oliver PY - 2023/9/7 TI - Community-Engaged Participatory Methods to Address Lesbian, Gay, Bisexual, Transgender, Queer, and Questioning Young People?s Health Information Needs With a Resource Website: Participatory Design and Development Study JO - JMIR Form Res SP - e41682 VL - 7 KW - lesbian, gay, bisexual, transgender, and queer health KW - LGBTQ+ health KW - information seeking KW - participatory design KW - community-based research KW - web-based health resources KW - lesbian, gay, bisexual, transgender, and queer young people KW - LGBTQ+ young people KW - mobile phone N2 - Background: Lesbian, gay, bisexual, transgender, queer, and questioning (LGBTQ+) young people (aged 15 to 25 years) face unique health challenges and often lack resources to adequately address their health information needs related to gender and sexuality. Beyond information access issues, LGBTQ+ young people may need information resources to be designed and organized differently compared with their cisgender and heterosexual peers and, because of identity exploration, may have different information needs related to gender and sexuality than older people. Objective: The objective of our study was to work with a community partner to develop an inclusive and comprehensive new website to address LGBTQ+ young people?s health information needs. To design this resource website using a community-engaged approach, our objective required working with and incorporating content and design recommendations from young LGBTQ+ participants. Methods: We conducted interviews (n=17) and participatory design sessions (n=11; total individual participants: n=25) with LGBTQ+ young people to understand their health information needs and elicit design recommendations for the new website. We involved our community partner in all aspects of the research and design process. Results: We present participants? desired resources, health topics, and technical website features that can facilitate information seeking for LGBTQ+ young people exploring their sexuality and gender and looking for health resources. We describe how filters can allow people to find information related to intersecting marginalized identities and how dark mode can be a privacy measure to avoid unwanted identity disclosure. We reflect on our design process and situate the website development in previous critical reflections on participatory research with marginalized communities. We suggest recommendations for future LGBTQ+ health websites based on our research and design experiences and final website design, which can enable LGBTQ+ young people to access information, find the right information, and navigate identity disclosure concerns. These design recommendations include filters, a reduced number of links, conscientious choice of graphics, dark mode, and resources tailored to intersecting identities. Conclusions: Meaningful collaboration with community partners throughout the design process is vital for developing technological resources that meet community needs. We argue for community partner leadership rather than just involvement in community-based research endeavors at the intersection of human-computer interaction and health. UR - https://formative.jmir.org/2023/1/e41682 UR - http://dx.doi.org/10.2196/41682 UR - http://www.ncbi.nlm.nih.gov/pubmed/37676709 ID - info:doi/10.2196/41682 ER - TY - JOUR AU - Spooner, Caitlin AU - Vivat, Bella AU - White, Nicola AU - Stone, Patrick PY - 2023/9/1 TI - Developing a Core Outcome Set for Prognostic Research in Palliative Cancer Care: Protocol for a Mixed Methods Study JO - JMIR Res Protoc SP - e49774 VL - 12 KW - core outcome set KW - palliative care KW - end-of-life KW - prognosis KW - advanced cancer KW - systematic review KW - interviews KW - Delphi study N2 - Background: Studies exploring the impact of receiving end-of-life prognoses in patients with advanced cancer use a variety of different measures to evaluate the outcomes, and thus report often conflicting findings. The standardization of outcomes reported in studies of prognostication in palliative cancer care could enable uniform assessment and reporting, as well as intertrial comparisons. A core outcome set promotes consistency in outcome selection and reporting among studies within a particular population. We aim to develop a set of core outcomes to be used to measure the impact of end-of-life prognostication in palliative cancer care. Objective: This protocol outlines the proposed methodology to develop a core outcome set for measuring the impact of end-of-life prognostication in palliative cancer care. Methods: We will adopt a mixed methods approach consisting of 3 phases using methodology recommended by the Core Outcome Measure in Effectiveness Trials (COMET) initiative. In phase I, we will conduct a systematic review to identify existing outcomes that prognostic studies have previously used, so as to inform the development of items and domains for the proposed core outcome set. Phase II will consist of semistructured interviews with patients with advanced cancer who are receiving palliative care, informal caregivers, and clinicians, to explore their perceptions and experiences of end-of-life prognostication. Outcomes identified in the interviews will be combined with those found in existing literature and taken forward to phase III, a Delphi survey, in which we will ask patients, informal caregivers, clinicians, and relevant researchers to rate these outcomes until consensus is achieved as to which are considered to be the most important for inclusion in the core outcome set. The resulting, prioritized outcomes will be discussed in a consensus meeting to agree and endorse the final core outcome set. Results: Ethical approval was received for this study in September 2022. As of July 2023, we have completed and published the systematic review (phase I) and have started recruitment for phase II. Data analysis for phase II has not yet started. We expect to complete the study by October 2024. Conclusions: This protocol presents the stepwise approach that will be taken to develop a core outcome set for measuring the impact of end-of-life prognostication in palliative cancer care. The final core outcome set has the potential for translation into clinical practice, allowing for consistent evaluation of emerging prognostic algorithms and improving communication of end-of-life prognostication. This study will also potentially facilitate the design of future clinical trials of the impact of end-of-life prognostication in palliative care that are acceptable to key stakeholders. Trial Registration: Core Outcome Measures in Effectiveness Trials 2136; https://www.comet-initiative.org/Studies/Details/2136 International Registered Report Identifier (IRRID): DERR1-10.2196/49774 UR - https://www.researchprotocols.org/2023/1/e49774 UR - http://dx.doi.org/10.2196/49774 UR - http://www.ncbi.nlm.nih.gov/pubmed/37656505 ID - info:doi/10.2196/49774 ER - TY - JOUR AU - Marlow, Nicholas AU - Eckert, Marion AU - Sharplin, Greg AU - Gwilt, Ian AU - Carson-Chahhoud, Kristin PY - 2023/8/31 TI - Graphical User Interface Development for a Hospital-Based Predictive Risk Tool: Protocol for a Co-Design Study JO - JMIR Res Protoc SP - e47717 VL - 12 KW - co-design KW - health information technology KW - health professional KW - user-centered design KW - user interface N2 - Background: This co-design research method details the iterative process developed to identify health professional recommendations for the graphical user interface (GUI) of an artificial intelligence (AI)?enabled risk prediction tool. Driving the decision to include a co-design process is the belief that choices regarding the aesthetic and functionality of an intervention are best made by its intended users and that engaging these users in its design will promote the tool?s adoption and use. Objective: The aim of this research is to identify health professional design and uptake recommendations for the GUI of an AI-enabled predictive risk tool. Methods: We will hold 3 research phases, each consisting of 2 workshops with health professionals, between mid-2023 and mid-2024. A total of 6 health professionals will be sought per workshop, resulting in a total enrollment of 36 health professionals at the conclusion of the research. A total of 7 workshop activities have been scheduled across the 3 workshops; these include context of use, notifiers, format, AI survey?Likert, prototype, AI survey?written, and testing. The first 6 of these activities will be repeated in each workshop to enable the iterative development and refinement of GUI. The last activity (testing) will be performed in the final workshop to examine health professionals? thoughts on the final GUI iteration. Qualitative and quantitative results data will be produced from tasks in each research activity. Qualitative data will be examined through inductive thematic analysis or deductive thematic analysis in accordance with the Nonadoption, Abandonment, and Challenges to the Scale-up, Spread, and Sustainability (NASSS) framework; visual data will be examined in accordance with ?framework of interactivity;? and quantitative data will be examined using descriptive statistics. Results: Project registration with the Australia and New Zealand Clinical Trial Registry has been requested (#384098). Finalized design recommendations are expected in early to mid-2024, with a results manuscript to be submitted in mid-2024. This research method has human research ethics approval from the South Australian Department of Health and Wellbeing (#2022/HRE00131) as well as from the Human Research Ethics Committee of the University of South Australia (application ID#204143). Conclusions: Understanding whether an intervention is needed in a particular situation is just the start; designing an intervention so that it is used within that situation is paramount. This co-design process engages end users to create a GUI that includes the aesthetic and functional details they need in a manner that aligns with their existing work practices. Indeed, interventions that fail to do this may be disliked, and at worst, they may be dangerous. International Registered Report Identifier (IRRID): PRR1-10.2196/47717 UR - https://www.researchprotocols.org/2023/1/e47717 UR - http://dx.doi.org/10.2196/47717 UR - http://www.ncbi.nlm.nih.gov/pubmed/37651166 ID - info:doi/10.2196/47717 ER - TY - JOUR AU - Sari, Kencana Puspita AU - Handayani, Wuri Putu AU - Hidayanto, Nizar Achmad PY - 2023/8/24 TI - Demographic Comparison of Information Security Behavior Toward Health Information System Protection: Survey Study JO - JMIR Form Res SP - e49439 VL - 7 KW - behavioral research KW - health information system KW - human activities KW - information security KW - mobile security N2 - Background: The health information system (HIS) functions are getting wider with more diverse users. Information security in the health industry is crucial because it involves comprehensive and strategic information that might harm human life. The human factor is one of the biggest security threats to HIS. Objective: This study aims to investigate the information security behavior (ISB) of HIS users using a comprehensive assessment scale suited to the information security concerns in health care. Patients are increasingly being asked to submit their own data into HIS systems. As a result, this study examines the security behavior of health workers and patients, as well as their demographic variables. Methods: We used a quantitative approach using surveys of health workers and patients. We created a research instrument from 4 existing measurement scales to measure prosecurity and antisecurity behavior. We analyzed statistical differences to test the hypotheses, that is, the Kruskal-Wallis test and the Mann-Whitney test. The descriptive analysis was used to determine whether the group exhibited exemplary behavior when processing the survey results. A correlational test using the Spearman correlation coefficient was performed to establish the significance of the relationship between ISB and age as well as level of education. Results: We analyzed 421 responses from the survey. According to demographic factors, the hypotheses tested for full and partial security behavior reveal substantial differences. Education levels most significantly affect security behavior differences, followed by user type, gender, and age. The health workers? ISB is higher than that of the patients. Women are more likely than men to engage in prosecurity actions while avoiding antisecurity behaviors. The older the HIS user, the more likely it is that they will participate in prosecurity behavior and the less probable it is that they will engage in antisecurity behavior. According to this study, differences in prosecurity behavior are mostly impacted by education level. Higher education, on the other hand, does not guarantee improved ISB for HIS users. All demographic characteristics, particularly concerning user type, show discrepancies that are caused mainly by antisecurity behavior rather than prosecurity behavior. Conclusions: Since patients engage in antisecurity behavior more frequently than health workers and may pose security risks, health care facilities should start to consider information security education for patients. More comprehensive research on ISB in health care facilities is required to better understand the patient?s perspective, which is currently understudied. UR - https://formative.jmir.org/2023/1/e49439 UR - http://dx.doi.org/10.2196/49439 UR - http://www.ncbi.nlm.nih.gov/pubmed/37616025 ID - info:doi/10.2196/49439 ER - TY - JOUR AU - Akahane, Manabu AU - Kanagawa, Yoshiyuki AU - Takahata, Yoshihisa AU - Nakanishi, Yasuhiro AU - Akahane, Takemi AU - Imamura, Tomoaki PY - 2023/8/24 TI - Consumer Awareness of Food Defense Measures at Food Delivery Service Providers and Food Manufacturers: Web-Based Consumer Survey Study JO - JMIR Form Res SP - e44150 VL - 7 KW - food defense KW - health hazards KW - intentional contamination KW - foreign substances KW - food delivery service N2 - Background: Various stages of the food chain, from production to processing to distribution, can impact food safety. The concept of ?food defense? has emerged as a countermeasure against intentional contamination of food with foreign substances. Although knowledge of food hygiene is common among consumers, there are currently no reports of consumer surveys on food defense. Objective: This study aims to investigate consumer awareness of food defense and food safety. We analyzed the results focusing on how consumers behave when they find abnormalities in food to further our knowledge on promoting food defense measures. Methods: Participants completed a web-based questionnaire that included items related to awareness of food safety and food defense, as well as actions to be taken in cases of food abnormalities, such as contamination by foreign substances, the presence of a bad smell in purchased food, and the inclusion of extra items not selected by the individual. The participants were asked to indicate their preference among the 5 suggested actions in each case using a 6-point Likert scale. Data analysis involved aggregating responses into binary values. Stepwise linear regression analysis was conducted to examine the relationship between selected actions and questionnaire items, such as sex, age, and personality. Results: A total of 1442 respondents completed the survey, and the majority of participants placed importance on food safety when making food purchases. The recognition of each term was as follows: 95.2% (n=1373) for ?food security and safety,? 95.6% (n=1379) for ?food hygiene,? and 17.1% (n=247) for ?food defense.? The percentages of those who answered that they would ?eat without worrying? in the case of ?contamination by foreign substances,? ?bad smell,? or ?including unpurchased product? in the frozen food they purchased were 9.1% (n=131), 4.8% (n=69), and 30.7% (n=443), respectively. The results showed that contacting the manufacturer was the most common action when faced with contaminated food or food with a bad smell. Interestingly, a significant percentage of respondents indicated they would upload the issue on social networking sites. Logistic regression analysis revealed that male participants and the younger generation were more likely to choose the option of eating contaminated food without worrying. Additionally, the tendency to upload the issue on social networking sites was higher among respondents who were sociable and brand-conscious. Conclusions: The findings of this study indicate that if food intentionally contaminated with a foreign substance is sold and delivered to consumers, it is possible consumers may eat it and experience health problems. Therefore, it is crucial for not only food manufacturers but also food delivery service providers to consider food defense measures such as protecting food from intentional contamination. Additionally, promoting consumer education and awareness regarding food defense can contribute to enhancing food safety throughout the food chain. UR - https://formative.jmir.org/2023/1/e44150 UR - http://dx.doi.org/10.2196/44150 UR - http://www.ncbi.nlm.nih.gov/pubmed/37616047 ID - info:doi/10.2196/44150 ER - TY - JOUR AU - LaMonica, M. Haley AU - Crouse, J. Jacob AU - Song, C. Yun J. AU - Alam, Mafruha AU - Wilson, E. Chloe AU - Hindmarsh, Gabrielle AU - Yoon, Adam AU - Boulton, A. Kelsie AU - Ekambareshwar, Mahalakshmi AU - Loblay, Victoria AU - Troy, Jakelin AU - Torwali, Mujahid AU - Guastella, J. Adam AU - Banati, B. Richard AU - Hickie, B. Ian PY - 2023/8/23 TI - Developing Culturally Appropriate Content for a Child-Rearing App to Support Young Children?s Socioemotional and Cognitive Development in Afghanistan: Co-Design Study JO - JMIR Form Res SP - e44267 VL - 7 KW - child development KW - digital technology KW - global health KW - co-design KW - participatory research KW - stakeholder participation KW - mobile app KW - smartphone KW - mobile phone KW - Afghanistan N2 - Background: Optimal child-rearing practices can help mitigate the consequences of detrimental social determinants of health in early childhood. Given the ubiquity of personal digital technologies worldwide, the direct delivery of evidence-based information about early childhood development holds great promise. However, to make the content of these novel systems effective, it is crucial to incorporate place-based cultural beliefs, traditions, circumstances, and value systems of end users. Objective: This paper describes the iterative approach used to develop the Thrive by Five child-rearing app in collaboration with Afghan parents, caregivers (eg, grandparents, aunts, and nannies), and subject matter experts (SMEs). We outline how co-design methodologies informed the development and cultural contextualization of content to meet the specific needs of Afghan parents and the content was tested and refined in collaboration with key Afghan stakeholders. Methods: The preliminary content was developed based on a comprehensive literature review of the historical and sociocultural contexts in Afghanistan, including factors that influence child-rearing practices and early childhood development. After an initial review and refinement based on feedback from SMEs, this content was populated into a beta app for testing. Overall, 8 co-design workshops were conducted in July and August 2021 and February 2022 with 39 Afghan parents and caregivers and 6 SMEs to collect their feedback on the app and its content. The workshops were audio recorded and transcribed; detailed field notes were taken by 2 scribes. A theoretical thematic analysis using semantic codes was conducted to inform the refinement of existing content and development of new content to fulfill the needs identified by participants. Results: The following 4 primary themes were identified: child-rearing in the Afghan sociocultural context, safety concerns, emotion and behavior management, and physical health and nutrition. Overall, participants agreed that the app had the potential to deliver valuable information to Afghan parents; however, owing to the volatility in the country, participants recommended including more activities that could be safely done indoors, as mothers and children are required to spend most of their time at home. Additionally, restrictions on public engagement in music required the removal of activities referencing singing that might be performed outside the home. Further, activities to help parents reduce their children?s screen time, promote empathy, manage emotions, regulate behavior, and improve physical health and nutrition were requested. Conclusions: Direct engagement with Afghan parents, caregivers, and SMEs through co-design workshops enabled the development and refinement of evidence-based, localized, and contextually relevant child-rearing activities promoting healthy social, emotional, and cognitive development during the first 5 years of children?s lives. Importantly, the content was adapted for the ongoing conflict in Afghanistan with the aim of empowering Afghan parents and caregivers to support their children?s developmental potential despite the security concerns and situational stressors. UR - https://formative.jmir.org/2023/1/e44267 UR - http://dx.doi.org/10.2196/44267 UR - http://www.ncbi.nlm.nih.gov/pubmed/37610805 ID - info:doi/10.2196/44267 ER - TY - JOUR AU - Goueth, Rose AU - Holt, Kelsey AU - Eden, B. Karen AU - Hoffman, Aubri PY - 2023/8/21 TI - Clinicians? Perspectives and Proposed Solutions to Improve Contraceptive Counseling in the United States: Qualitative Semistructured Interview Study With Clinicians From the Society of Family Planning JO - JMIR Form Res SP - e47298 VL - 7 KW - contraceptive counseling KW - qualitative study KW - decision making KW - decision aids KW - clinician engagement KW - user-centered design KW - contraceptive KW - birth control KW - clinicians? perspectives KW - patient-centered counseling KW - sexual health KW - family planning N2 - Background: Contraceptive care is a key element of reproductive health, yet only 12%-30% of women report being able to access and receive the information they need to make these complex, personal health care decisions. Current guidelines recommend implementing shared decision-making approaches; and tools such as patient decision aid (PtDA) applications have been proposed to improve patients? access to information, contraceptive knowledge, decisional conflict, and engagement in decision-making and contraception use. To inform the design of meaningful, effective, elegant, and feasible PtDA applications, studies are needed of all users? current experiences, needs, and barriers. While multiple studies have explored patients? experiences, needs, and barriers, little is known about clinicians? experiences, perspectives, and barriers to delivering contraceptive counseling. Objective: This study focused on assessing clinicians? experiences, including their perspectives of patients? needs and barriers. It also explored clinicians? suggestions for improving contraceptive counseling and the feasibility of a contraceptive PtDA. Methods: Following the decisional needs assessment approach, we conducted semistructured interviews with clinicians recruited from the Society of Family Planning. The Ottawa Decision Support Framework informed the interview guide and initial codebook, with a specific focus on decision support and decisional needs as key elements that should be assessed from the clinicians? perspective. An inductive content approach was used to analyze data and identify primary themes and suggestions for improvement. Results: Fifteen clinicians (12 medical doctors and 3 nurse practitioners) participated, with an average of 19 years of experience in multiple regions of the United States. Analyses identified 3 primary barriers to the provision of quality contraceptive counseling: gaps in patients? underlying sexual health knowledge, biases that impede decision-making, and time constraints. All clinicians supported the development of contraceptive PtDAs as a feasible solution to these main barriers. Multiple suggestions for improvement were provided, including clinician- and system-level training, tools, and changes that could support successful implementation. Conclusions: Clinicians and developers interested in improving contraceptive counseling and decision-making may wish to incorporate approaches that assess and address upstream factors, such as sexual health knowledge and existing heuristics and biases. Clinical leaders and administrators may also wish to prioritize solutions that improve equity and accessibility, including PtDAs designed to provide education and support in advance of the time-constrained consultations, and strategic training opportunities that support cultural awareness and shared decision-making skills. Future studies can then explore whether well-designed, user-centered shared decision-making programs lead to successful and sustainable uptake and improve patients? reproductive health contraceptive decision-making. UR - https://formative.jmir.org/2023/1/e47298 UR - http://dx.doi.org/10.2196/47298 UR - http://www.ncbi.nlm.nih.gov/pubmed/37603407 ID - info:doi/10.2196/47298 ER - TY - JOUR AU - Chen, Yifei AU - Li, Xiaoying AU - Li, Aihua AU - Li, Yongjie AU - Yang, Xuemei AU - Lin, Ziluo AU - Yu, Shirui AU - Tang, Xiaoli PY - 2023/8/18 TI - A Deep Learning Model for the Normalization of Institution Names by Multisource Literature Feature Fusion: Algorithm Development Study JO - JMIR Form Res SP - e47434 VL - 7 KW - multisource literature KW - institution name normalization KW - deep learning KW - bidirectional encoder representations from transformers KW - BERT N2 - Background: The normalization of institution names is of great importance for literature retrieval, statistics of academic achievements, and evaluation of the competitiveness of research institutions. Differences in authors? writing habits and spelling mistakes lead to various names of institutions, which affects the analysis of publication data. With the development of deep learning models and the increasing maturity of natural language processing methods, training a deep learning?based institution name normalization model can increase the accuracy of institution name normalization at the semantic level. Objective: This study aimed to train a deep learning?based model for institution name normalization based on the feature fusion of affiliation data from multisource literature, which would realize the normalization of institution name variants with the help of authority files and achieve a high specification accuracy after several rounds of training and optimization. Methods: In this study, an institution name normalization?oriented model was trained based on bidirectional encoder representations from transformers (BERT) and other deep learning models, including the institution classification model, institutional hierarchical relation extraction model, and institution matching and merging model. The model was then trained to automatically learn institutional features by pretraining and fine-tuning, and institution names were extracted from the affiliation data of 3 databases to complete the normalization process: Dimensions, Web of Science, and Scopus. Results: It was found that the trained model could achieve at least 3 functions. First, the model could identify the institution name that is consistent with the authority files and associate the name with the files through the unique institution ID. Second, it could identify the nonstandard institution name variants, such as singular forms, plural changes, and abbreviations, and update the authority files. Third, it could identify the unregistered institutions and add them to the authority files, so that when the institution appeared again, the model could identify and regard it as a registered institution. Moreover, the test results showed that the accuracy of the normalization model reached 93.79%, indicating the promising performance of the model for the normalization of institution names. Conclusions: The deep learning?based institution name normalization model trained in this study exhibited high accuracy. Therefore, it could be widely applied in the evaluation of the competitiveness of research institutions, analysis of research fields of institutions, and construction of interinstitutional cooperation networks, among others, showing high application value. UR - https://formative.jmir.org/2023/1/e47434 UR - http://dx.doi.org/10.2196/47434 UR - http://www.ncbi.nlm.nih.gov/pubmed/37594844 ID - info:doi/10.2196/47434 ER - TY - JOUR AU - Xing, Zhaoquan AU - Ji, Meng AU - Shan, Yi AU - Dong, Zhaogang AU - Xu, Xiaofei PY - 2023/8/16 TI - Using the Multidimensional Health Locus of Control Scale Form C to Investigate Health Beliefs About Bladder Cancer Prevention and Treatment Among Male Patients: Cross-Sectional Study JO - JMIR Form Res SP - e43345 VL - 7 KW - health beliefs KW - Multidimensional Health Locus of Control KW - Chinese translation KW - bladder cancer prevention and treatment KW - male patients KW - latent class analysis N2 - Background: Bladder cancer is a leading cause of death among Chinese male populations in recent years. The health locus of control construct can mediate health status and outcomes, and it has proven helpful in predicting and explaining specific health-related behaviors. However, it has never been used to investigate health beliefs about bladder cancer prevention and treatment. Objective: This study aimed to classify male patients into different latent groups according to their beliefs about bladder cancer prevention and treatment and to identify associated factors to provide implications for the delivery of tailored education and interventions and the administration of targeted prevention and treatment. Methods: First, we designed a four-section questionnaire to solicit data: section 1?age, gender, and education; section 2?the communicative subscale of the All Aspects of Health Literacy Scale; section 3?the eHealth Literacy Scale; and section 4?health beliefs about bladder cancer prevention and treatment measured by the Multidimensional Health Locus of Control Scale Form C. We hypothesized that the participants? health beliefs about bladder cancer prevention and treatment measured in section 4 could be closely associated with information collected through sections 1 to 3. We recruited 718 Chinese male patients from Qilu Hospital of Shandong University, China, and invited them to participate in a web-based questionnaire survey. Finally, we used latent class analysis to identify subgroups of men based on their categorical responses to the items on the Multidimensional Health Locus of Control Scale Form C and ascertained factors contributing to the low self-efficacy group identified. Results: We identified 2 subgroups defined as low and moderate self-efficacy groups representing 75.8% (544/718) and 24.2% (174/718) of the total sample, respectively. Men in the low self-efficacy cluster (cluster 1: 544/718, 75.8%) were less likely to believe in their own capability or doctors? advice to achieve optimal outcomes in bladder cancer prevention and treatment. Men in the moderate self-efficacy cluster (cluster 2: 174/718, 24.2%) had distinct psychological traits. They had stronger beliefs in their own capability to manage their health with regard to bladder cancer prevention and treatment and moderate to high levels of trust in health and medical professionals and their advice to achieve better prevention and treatment outcomes. Four factors contributing to low self-efficacy were identified, including limited education (Year 6 to Year 12), aged ?44 years, limited communicative health literacy, and limited digital health literacy. Conclusions: This was the first study investigating beliefs about bladder cancer prevention and treatment among Chinese male patients. Given that bladder cancer represents a leading cause of death among Chinese male populations in recent years, the low self-efficacy cluster and associated contributing factors identified in this study can provide implications for clinical practice, health education, medical research, and health policy-making. UR - https://formative.jmir.org/2023/1/e43345 UR - http://dx.doi.org/10.2196/43345 UR - http://www.ncbi.nlm.nih.gov/pubmed/37585255 ID - info:doi/10.2196/43345 ER - TY - JOUR AU - Pigat, Sandrine AU - Soshina, Mariya AU - Berezhnaya, Yulia AU - Kryzhanovskaya, Ekaterina PY - 2023/8/16 TI - Web-Based 24-Hour Dietary Recall Tool for Russian Adults and School-Aged Children: Validation Study JO - JMIR Form Res SP - e41774 VL - 7 KW - dietary assessment KW - 24-hour dietary recall KW - extent of agreement KW - energy and nutrient intake KW - Russian diet KW - interviewer-administered KW - web-based self-administered KW - diet KW - food intake KW - dietary recall KW - energy intake KW - nutrient intake N2 - Background: Data on dietary intakes in Russian adults and children are assessed very infrequently primarily due to the time, cost, and burden to the participants for assessing dietary patterns. To overcome some of those challenges, the use of web-based 24-hour recall methods can be successfully used. Objective: The study objective is to assess the extent of agreement between a self-administered and an interviewer-administered 24-hour dietary recall in Russian adults and school-aged children using an adaptation of a web-based 24-hour recall tool. Methods: This web-based dietary assessment tool is based on a previously validated tool, which has been adapted to the Russian diet and language. A randomized 50% (n=97) of 194 participants initially completed a self-administered web-based dietary recall, followed by an interviewer-administered 24-hour dietary recall later that same day, and vice versa for the other 50% (n=97) of participants. Following at least 1 week wash-out period, during visit 2, participant groups completed the 2 dietary recalls in the opposite order. Statistical analysis was carried out on the intake results from both methods for the 2 recalls. Finally, an evaluation questionnaire on ease-of-use of the tool was also completed. Results: In total, intakes of 28 nutrients and energy were analyzed in this study. The Bland-Altman analysis showed that between 98.4% and 90.5% of data points were within the limits of agreement among all age groups and nutrients analyzed. A ?moderate to excellent? reliability between the 2 methods was observed in younger children. In older children, a ?moderate to good? reliability was observed, with the exception of sodium. In adults, ?moderate to excellent? reliability between both methods was observed with the exception of vitamins B1, B2, and B6, and pantothenic acid. The level of agreement between the categorization of estimates into thirds of the intake distribution for the average of the 2 days was satisfactory, since the percentages of participants categorized into the same tertile of intake were ?50%, and the percentages of participants categorized into the opposite tertile of intake were <10%. The majority of respondents were very positive in their evaluation of the web-based dietary assessment tool. Conclusions: Overall, the web-based dietary assessment tool performs well when compared with a face-to-face, interviewer-administered 24-hour dietary recall and provides comparable estimates of energy and nutrient intakes in Russian adults and children. Trial Registration: ClinicalTrials.gov NCT04372160; https://clinicaltrials.gov/ct2/show/NCT04372160 UR - https://formative.jmir.org/2023/1/e41774 UR - http://dx.doi.org/10.2196/41774 UR - http://www.ncbi.nlm.nih.gov/pubmed/37585243 ID - info:doi/10.2196/41774 ER - TY - JOUR AU - Golec, Marcin AU - Kamdar, Maulik AU - Barteit, Sandra PY - 2023/8/11 TI - Comprehensive Ontology of Fibroproliferative Diseases: Protocol for a Semantic Technology Study JO - JMIR Res Protoc SP - e48645 VL - 12 KW - fibroproliferative disease KW - fibrosis KW - fibrotic disease KW - ontology KW - OWL KW - semantic technology KW - Web Ontology Language N2 - Background: Fibroproliferative or fibrotic diseases (FDs), which represent a significant proportion of age-related pathologies and account for over 40% of mortality in developed nations, are often underrepresented in focused research. Typically, these conditions are studied individually, such as chronic obstructive pulmonary disease or idiopathic pulmonary fibrosis (IPF), rather than as a collective entity, thereby limiting the holistic understanding and development of effective treatments. To address this, we propose creating and publicizing a comprehensive fibroproliferative disease ontology (FDO) to unify the understanding of FDs. Objective: This paper aims to delineate the study protocol for the creation of the FDO, foster transparency and high quality standards during its development, and subsequently promote its use once it becomes publicly available. Methods: We aim to establish an ontology encapsulating the broad spectrum of FDs, constructed in the Web Ontology Language format using the Protégé ontology editor, adhering to ontology development life cycle principles. The modeling process will leverage Protégé in accordance with a methodologically defined process, involving targeted scoping reviews of MEDLINE and PubMed information, expert knowledge, and an ontology development process. A hybrid top-down and bottom-up strategy will guide the identification of core concepts and relations, conducted by a team of domain experts based on systematic iterations of scientific literature reviews. Results: The result will be an exhaustive FDO accommodating a wide variety of crucial biomedical concepts, augmented with synonyms, definitions, and references. The FDO aims to encapsulate diverse perspectives on the FD domain, including those of clinicians, health informaticians, medical researchers, and public health experts. Conclusions: The FDO is expected to stimulate broader and more in-depth FD research by enabling reasoning, inference, and the identification of relationships between concepts for application in multiple contexts, such as developing specialized software, fostering research communities, and enhancing domain comprehension. A common vocabulary and understanding of relationships among medical professionals could potentially expedite scientific progress and the discovery of innovative solutions. The publicly available FDO will form the foundation for future research, technological advancements, and public health initiatives. International Registered Report Identifier (IRRID): PRR1-10.2196/48645 UR - https://www.researchprotocols.org/2023/1/e48645 UR - http://dx.doi.org/10.2196/48645 UR - http://www.ncbi.nlm.nih.gov/pubmed/37566458 ID - info:doi/10.2196/48645 ER - TY - JOUR AU - Kamdje Wabo, Gaetan AU - Prasser, Fabian AU - Gierend, Kerstin AU - Siegel, Fabian AU - Ganslandt, Thomas PY - 2023/8/11 TI - Data Quality? and Utility-Compliant Anonymization of Common Data Model?Harmonized Electronic Health Record Data: Protocol for a Scoping Review JO - JMIR Res Protoc SP - e46471 VL - 12 KW - EHR KW - electronic health record KW - data quality KW - common data model KW - data standard KW - data privacy models KW - data anonymization N2 - Background: The anonymization of Common Data Model (CDM)?converted EHR data is essential to ensure the data privacy in the use of harmonized health care data. However, applying data anonymization techniques can significantly affect many properties of the resulting data sets and thus biases research results. Few studies have reviewed these applications with a reflection of approaches to manage data utility and quality concerns in the context of CDM-formatted health care data. Objective: Our intended scoping review aims to identify and describe (1) how formal anonymization methods are carried out with CDM-converted health care data, (2) how data quality and utility concerns are considered, and (3) how the various CDMs differ in terms of their suitability for recording anonymized data. Methods: The planned scoping review is based on the framework of Arksey and O'Malley. By using this, only articles published in English will be included. The retrieval of literature items should be based on a literature search string combining keywords related to data anonymization, CDM standards, and data quality assessment. The proposed literature search query should be validated by a librarian, accompanied by manual searches to include further informal sources. Eligible articles will first undergo a deduplication step, followed by the screening of titles. Second, a full-text reading will allow the 2 reviewers involved to reach the final decision about article selection, while a domain expert will support the resolution of citation selection conflicts. Additionally, key information will be extracted, categorized, summarized, and analyzed by using a proposed template into an iterative process. Tabular and graphical analyses should be addressed in alignment with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist. We also performed some tentative searches on Web of Science for estimating the feasibility of reaching eligible articles. Results: Tentative searches on Web of Science resulted in 507 nonduplicated matches, suggesting the availability of (potential) relevant articles. Further analysis and selection steps will allow us to derive a final literature set. Furthermore, the completion of this scoping review study is expected by the end of the fourth quarter of 2023. Conclusions: Outlining the approaches of applying formal anonymization methods on CDM-formatted health care data while taking into account data quality and utility concerns should provide useful insights to understand the existing approaches and future research direction based on identified gaps. This protocol describes a schedule to perform a scoping review, which should support the conduction of follow-up investigations. International Registered Report Identifier (IRRID): PRR1-10.2196/46471 UR - https://www.researchprotocols.org/2023/1/e46471 UR - http://dx.doi.org/10.2196/46471 UR - http://www.ncbi.nlm.nih.gov/pubmed/37566443 ID - info:doi/10.2196/46471 ER - TY - JOUR AU - Wang, Bin AU - Lai, Junkai AU - Liao, Xiwen AU - Jin, Feifei AU - Yao, Chen PY - 2023/8/10 TI - Challenges and Solutions in Implementing eSource Technology for Real-World Studies in China: Qualitative Study Among Different Stakeholders JO - JMIR Form Res SP - e48363 VL - 7 KW - electronic medical record KW - electronic source KW - eSource KW - challenge KW - real-world data KW - interoperability N2 - Background: eSources consist of data that were initially documented in an electronic structure. Typically, an eSource encompasses the direct acquisition, compilation, and retention of electronic information (such as electronic health records [EHRs] or wearable devices), which serves to streamline clinical research. eSources have the potential to enhance the accuracy of data, promote patient safety, and minimize expenses associated with clinical trials. An opinion study published in September 2020 by TransCelerate outlined a road map for the future application of eSource technology and identified 5 key areas of challenges. The background of this study concerns the use of eSource technology in clinical research. Objective: The aim of this study was to present challenges and possible solutions for the implementation of eSource technology in real-world studies by summarizing team experiences and lessons learned from an eSource record (ESR) project. Methods: After initially developing a simple prototype of the ESR software that can be demonstrated systematically, the researchers conducted in-depth interviews and interacted with different stakeholders to obtain guidance and suggestions. The researchers selected 5 different roles for interviewees: regulatory authorities, pharmaceutical company representatives, hospital information department employees, medical system providers, and clinicians. Results: After screening all consultants, the researchers concluded that there were 25 representative consultants. The hospital information department needs to implement many demands from various stakeholders, which makes the existing EHR system unable to meet all the demands of eSources. The emergence of an ESR is intended to divert the burden of the hospital information department from the enormous functional requirements of the outdated EHR system. The entire research process emphasizes multidisciplinary and multibackground expert opinions and considers the complexity of the knowledge backgrounds of personnel involved in the chain of clinical source data collection, processing, quality control, and application in real-world scenarios. To increase the readability of the results, the researchers classified the main results in accordance with the paragraph titles in ?Use of Electronic Health Record Data in Clinical Investigations,? a guide released by the US Food and Drug Administration. Conclusions: This study introduces the requirement dependencies of different stakeholders and the challenges and recommendations for designing ESR software when implementing eSource technology in China. Experiences based on ESR projects will provide new insights into the disciplines that advance the eSource research field. Future studies should engage patients directly to understand their experiences, concerns, and preferences regarding the implementation of eSource technology. Moreover, involving additional stakeholders, including community health care providers and social workers, will provide valuable insights into the challenges and potential solutions across various health care settings. UR - https://formative.jmir.org/2023/1/e48363 UR - http://dx.doi.org/10.2196/48363 UR - http://www.ncbi.nlm.nih.gov/pubmed/37561551 ID - info:doi/10.2196/48363 ER - TY - JOUR AU - Nghiem, Jodie AU - Adler, A. Daniel AU - Estrin, Deborah AU - Livesey, Cecilia AU - Choudhury, Tanzeem PY - 2023/8/10 TI - Understanding Mental Health Clinicians? Perceptions and Concerns Regarding Using Passive Patient-Generated Health Data for Clinical Decision-Making: Qualitative Semistructured Interview Study JO - JMIR Form Res SP - e47380 VL - 7 KW - digital technology KW - clinical decision support KW - mobile health KW - mHealth KW - qualitative research KW - mental health KW - clinician KW - perception KW - patient-generated health data KW - mobile app KW - digital app KW - wearables KW - mobile phone N2 - Background: Digital health-tracking tools are changing mental health care by giving patients the ability to collect passively measured patient-generated health data (PGHD; ie, data collected from connected devices with little to no patient effort). Although there are existing clinical guidelines for how mental health clinicians should use more traditional, active forms of PGHD for clinical decision-making, there is less clarity on how passive PGHD can be used. Objective: We conducted a qualitative study to understand mental health clinicians? perceptions and concerns regarding the use of technology-enabled, passively collected PGHD for clinical decision-making. Our interviews sought to understand participants? current experiences with and visions for using passive PGHD. Methods: Mental health clinicians providing outpatient services were recruited to participate in semistructured interviews. Interview recordings were deidentified, transcribed, and qualitatively coded to identify overarching themes. Results: Overall, 12 mental health clinicians (n=11, 92% psychiatrists and n=1, 8% clinical psychologist) were interviewed. We identified 4 overarching themes. First, passive PGHD are patient driven?we found that current passive PGHD use was patient driven, not clinician driven; participating clinicians only considered passive PGHD for clinical decision-making when patients brought passive data to clinical encounters. The second theme was active versus passive data as subjective versus objective data?participants viewed the contrast between active and passive PGHD as a contrast between interpretive data on patients? mental health and objective information on behavior. Participants believed that prioritizing passive over self-reported, active PGHD would reduce opportunities for patients to reflect upon their mental health, reducing treatment engagement and raising questions about how passive data can best complement active data for clinical decision-making. Third, passive PGHD must be delivered at appropriate times for action?participants were concerned with the real-time nature of passive PGHD; they believed that it would be infeasible to use passive PGHD for real-time patient monitoring outside clinical encounters and more feasible to use passive PGHD during clinical encounters when clinicians can make treatment decisions. The fourth theme was protecting patient privacy?participating clinicians wanted to protect patient privacy within passive PGHD-sharing programs and discussed opportunities to refine data sharing consent to improve transparency surrounding passive PGHD collection and use. Conclusions: Although passive PGHD has the potential to enable more contextualized measurement, this study highlights the need for building and disseminating an evidence base describing how and when passive measures should be used for clinical decision-making. This evidence base should clarify how to use passive data alongside more traditional forms of active PGHD, when clinicians should view passive PGHD to make treatment decisions, and how to protect patient privacy within passive data?sharing programs. Clear evidence would more effectively support the uptake and effective use of these novel tools for both patients and their clinicians. UR - https://formative.jmir.org/2023/1/e47380 UR - http://dx.doi.org/10.2196/47380 UR - http://www.ncbi.nlm.nih.gov/pubmed/37561561 ID - info:doi/10.2196/47380 ER - TY - JOUR AU - Wu, Chao-Yi AU - Tibbitts, Deanne AU - Beattie, Zachary AU - Dodge, Hiroko AU - Shannon, Jackilen AU - Kaye, Jeffrey AU - Winters-Stone, Kerri PY - 2023/8/10 TI - Using Continuous Passive Assessment Technology to Describe Health and Behavior Patterns Preceding and Following a Cancer Diagnosis in Older Adults: Proof-of-Concept Case Series Study JO - JMIR Form Res SP - e45693 VL - 7 KW - sensor KW - quality of life KW - physical activity KW - medication KW - monitoring KW - function KW - mobile phone N2 - Background: Describing changes in health and behavior that precede and follow a sentinel health event, such as a cancer diagnosis, is challenging because of the lack of longitudinal, objective measurements that are collected frequently enough to capture varying trajectories of change leading up to and following the event. A continuous passive assessment system that continuously monitors older adults? physical activity, weight, medication-taking behavior, pain, health events, and mood could enable the identification of more specific health and behavior patterns leading up to a cancer diagnosis and whether and how patterns change thereafter. Objective: In this study, we conducted a proof-of-concept retrospective analysis, in which we identified new cancer diagnoses in older adults and compared trajectories of change in health and behaviors before and after cancer diagnosis. Methods: Participants were 10 older adults (mean age 71.8, SD 4.9 years; 3/10, 30% female) with various self-reported cancer types from a larger prospective cohort study of older adults. A technology-agnostic assessment platform using multiple devices provided continuous data on daily physical activity via wearable sensors (actigraphy); weight via a Wi-Fi?enabled digital scale; daily medication-taking behavior using electronic Bluetooth-enabled pillboxes; and weekly pain, health events, and mood with online, self-report surveys. Results: Longitudinal linear mixed-effects models revealed significant differences in the pre- and postcancer trajectories of step counts (P<.001), step count variability (P=.004), weight (P<.001), pain severity (P<.001), hospitalization or emergency room visits (P=.03), days away from home overnight (P=.01), and the number of pillbox door openings (P<.001). Over the year preceding a cancer diagnosis, there were gradual reductions in step counts and weight and gradual increases in pain severity, step count variability, hospitalization or emergency room visits, and days away from home overnight compared with 1 year after the cancer diagnosis. Across the year after the cancer diagnosis, there was a gradual increase in the number of pillbox door openings compared with 1 year before the cancer diagnosis. There was no significant trajectory change from the pre? to post?cancer diagnosis period in terms of low mood (P=.60) and loneliness (P=.22). Conclusions: A home-based, technology-agnostic, and multidomain assessment platform could provide a unique approach to monitoring different types of behavior and health markers in parallel before and after a life-changing health event. Continuous passive monitoring that is ecologically valid, less prone to bias, and limits participant burden could greatly enhance research that aims to improve early detection efforts, clinical care, and outcomes for people with cancer. UR - https://formative.jmir.org/2023/1/e45693 UR - http://dx.doi.org/10.2196/45693 UR - http://www.ncbi.nlm.nih.gov/pubmed/37561574 ID - info:doi/10.2196/45693 ER - TY - JOUR AU - Kim, Hyunsoo AU - Jang, Jin Seong AU - Lee, Dong Hee AU - Ko, Hoon Jae AU - Lim, Young Jee PY - 2023/8/7 TI - Smart Floor Mats for a Health Monitoring System Based on Textile Pressure Sensing: Development and Usability Study JO - JMIR Form Res SP - e47325 VL - 7 KW - analysis KW - auto-mapping KW - monitoring KW - healthcare KW - health-monitoring KW - online KW - piezo-resistance sensor KW - pressure mat KW - real-time KW - sensing mats KW - smart home technology KW - smart home KW - spatial map KW - technology KW - textile N2 - Background: The rise in single-person households has resulted in social problems like loneliness and isolation, commonly known as ?death by loneliness.? Various factors contribute to this increase, including a desire for independent living and communication challenges within families due to societal changes. Older individuals living alone are particularly susceptible to loneliness and isolation due to limited family communication and a lack of social activities. Addressing these issues is crucial, and proactive solutions are needed. It is important to explore diverse measures to tackle the challenges of single-person households and prevent deaths due to loneliness in our society. Objective: Non?face-to-face health care service systems have gained widespread interest owing to the rapid development of smart home technology. Particularly, a health monitoring system must be developed to manage patients? health status and send alerts for dangerous situations based on their activity. Therefore, in this study, we present a novel health monitoring system based on the auto-mapping method, which uses real-time position sensing mats. Methods: The smart floor mats are operated as piezo-resistive devices, which are composed of a carbon nanotube?based conductive textile, electrodes, main processor circuit, and a mat. The developed smart floor system acquires real-time position information using a multiconnection method between the modules based on the auto-mapping algorithm, which automatically creates a spatial map. The auto-mapping algorithm allows the user to freely set various activity areas through floor mapping. Then, the monitoring system was evaluated in a room with an area of 41.3 m2, which is embedded with the manufactured floor mats and monitoring application. Results: This monitoring system automatically acquires information on the total number, location, and direction of the mats and creates a spatial map. The position sensing mats can be easily configured with a simple structure by using a carbon nanotube?based piezo-resistive textile. The mats detect the activity in real time and record location information since they are connected through auto-mapping technology. Conclusions: This system allows for the analysis of patients? behavior patterns and the management of health care on the web by providing important basic information for activity patterns in the monitoring system. The proposed smart floor system can serve as the foundation for smart home applications in the future, which include health care, intelligent automation, and home security, owing to its advantages of low cost, large area, and high reliability. UR - https://formative.jmir.org/2023/1/e47325 UR - http://dx.doi.org/10.2196/47325 UR - http://www.ncbi.nlm.nih.gov/pubmed/37548993 ID - info:doi/10.2196/47325 ER - TY - JOUR AU - Spadaro, Benedetta AU - Martin-Key, A. Nayra AU - Funnell, Erin AU - Bená?ek, Ji?í AU - Bahn, Sabine PY - 2023/8/7 TI - Opportunities for the Implementation of a Digital Mental Health Assessment Tool in the United Kingdom: Exploratory Survey Study JO - JMIR Form Res SP - e43271 VL - 7 KW - assessment KW - digital mental health KW - development KW - implementation KW - mental health KW - provision KW - support KW - mobile phone N2 - Background: Every year, one-fourth of the people in the United Kingdom experience diagnosable mental health concerns, yet only a proportion receive a timely diagnosis and treatment. With novel developments in digital technologies, the potential to increase access to mental health assessments and triage is promising. Objective: This study aimed to investigate the current state of mental health provision in the United Kingdom and understand the utility of, and interest in, digital mental health technologies. Methods: A web-based survey was generated using Qualtrics XM. Participants were recruited via social media. Data were explored using descriptive statistics. Results: The majority of the respondents (555/618, 89.8%) had discussed their mental health with a general practitioner. More than three-fourths (503/618, 81.4%) of the respondents had been diagnosed with a mental health disorder, with the most common diagnoses being depression and generalized anxiety disorder. Diagnostic waiting times from first contact with a health care professional varied by diagnosis. Neurodevelopmental disorders (30/56, 54%), bipolar disorder (25/52, 48%), and personality disorders (48/101, 47.5%) had the longest waiting times, with almost half (103/209, 49.3%) of these diagnoses taking >6 months. Participants stated that waiting times resulted in symptoms worsening (262/353, 74.2%), lower quality of life (166/353, 47%), and the necessity to seek emergency care (109/353, 30.9%). Of the 618 participants, 386 (62.5%) stated that they felt that their mental health symptoms were not always taken seriously by their health care provider and 297 (48.1%) were not given any psychoeducational information. The majority of the respondents (416/595, 77.5%) did not have the chance to discuss mental health support and treatment options. Critically, 16.1% (96/595) did not find any treatment or support provided at all helpful, with 63% (48/76) having discontinued treatment with no effective alternatives. Furthermore, 88.3% (545/617) of the respondents) had sought help on the web regarding mental health symptoms, and 44.4% (272/612) had used a web application or smartphone app for their mental health. Psychoeducation (364/596, 61.1%), referral to a health care professional (332/596, 55.7%), and symptom monitoring (314/596, 52.7%) were the most desired app features. Only 6.8% (40/590) of the participants said that they would not be interested in using a mental health assessment app. Respondents were the most interested to receive an overall severity score of their mental health symptoms (441/546, 80.8%) and an indication of whether they should seek mental health support (454/546, 83.2%). Conclusions: Key gaps in current UK mental health care provision are highlighted. Assessment and treatment waiting times together with a lack of information regarding symptoms and treatment options translated into poor care experiences. The participants? responses provide proof-of-concept support for the development of a digital mental health assessment app and valuable recommendations regarding desirable app features. UR - https://formative.jmir.org/2023/1/e43271 UR - http://dx.doi.org/10.2196/43271 UR - http://www.ncbi.nlm.nih.gov/pubmed/37549003 ID - info:doi/10.2196/43271 ER - TY - JOUR AU - Amiri, Maryam AU - Li, Juan AU - Hasan, Wordh PY - 2023/8/3 TI - Personalized Flexible Meal Planning for Individuals With Diet-Related Health Concerns: System Design and Feasibility Validation Study JO - JMIR Form Res SP - e46434 VL - 7 KW - diabetes KW - fuzzy logic KW - meal planning KW - multicriteria decision-making KW - optimization N2 - Background: Chronic diseases such as heart disease, stroke, diabetes, and hypertension are major global health challenges. Healthy eating can help people with chronic diseases manage their condition and prevent complications. However, making healthy meal plans is not easy, as it requires the consideration of various factors such as health concerns, nutritional requirements, tastes, economic status, and time limits. Therefore, there is a need for effective, affordable, and personalized meal planning that can assist people in choosing food that suits their individual needs and preferences. Objective: This study aimed to design an artificial intelligence (AI)?powered meal planner that can generate personalized healthy meal plans based on the user?s specific health conditions, personal preferences, and status. Methods: We proposed a system that integrates semantic reasoning, fuzzy logic, heuristic search, and multicriteria analysis to produce flexible, optimized meal plans based on the user?s health concerns, nutrition needs, as well as food restrictions or constraints, along with other personal preferences. Specifically, we constructed an ontology-based knowledge base to model knowledge about food and nutrition. We defined semantic rules to represent dietary guidelines for different health concerns and built a fuzzy membership of food nutrition based on the experience of experts to handle vague and uncertain nutritional data. We applied a semantic rule-based filtering mechanism to filter out food that violate mandatory health guidelines and constraints, such as allergies and religion. We designed a novel, heuristic search method that identifies the best meals among several candidates and evaluates them based on their fuzzy nutritional score. To select nutritious meals that also satisfy the user?s other preferences, we proposed a multicriteria decision-making approach. Results: We implemented a mobile app prototype system and evaluated its effectiveness through a use case study and user study. The results showed that the system generated healthy and personalized meal plans that considered the user?s health concerns, optimized nutrition values, respected dietary restrictions and constraints, and met the user?s preferences. The users were generally satisfied with the system and its features. Conclusions: We designed an AI-powered meal planner that helps people create healthy and personalized meal plans based on their health conditions, preferences, and status. Our system uses multiple techniques to create optimized meal plans that consider multiple factors that affect food choice. Our evaluation tests confirmed the usability and feasibility of the proposed system. However, some limitations such as the lack of dynamic and real-time updates should be addressed in future studies. This study contributes to the development of AI-powered personalized meal planning systems that can support people?s health and nutrition goals. UR - https://formative.jmir.org/2023/1/e46434 UR - http://dx.doi.org/10.2196/46434 UR - http://www.ncbi.nlm.nih.gov/pubmed/37535413 ID - info:doi/10.2196/46434 ER - TY - JOUR AU - Zhao, Liang AU - Shen, Caiyi AU - Liu, Ming AU - Zhang, Jiaoyan AU - Cheng, Luying AU - Li, Yuanyuan AU - Yuan, Lanbin AU - Zhang, Junhua AU - Tian, Jinhui PY - 2023/8/2 TI - Comparison of Reporting and Transparency in Published Protocols and Publications in Umbrella Reviews: Scoping Review JO - J Med Internet Res SP - e43299 VL - 25 KW - umbrella reviews KW - protocol KW - publication KW - inconsistency KW - transparency N2 - Background: Inconsistencies between a protocol and its umbrella review (UR) may mislead readers about the importance of findings or lead to false-positive results. Furthermore, not documenting and explaining inconsistencies in the UR could reduce its transparency. To our knowledge, no study has examined the methodological consistency of the protocols with their URs and assessed the transparency of the URs when generating evidence. Objective: This study aimed to investigate the inconsistency of protocols with their URs in the methodology and assess the transparency of the URs. Methods: We searched medical-related electronic databases from their inception to January 1, 2022. We investigated inconsistencies between protocols and their publications and transparencies in the search strategy, inclusion criteria, methods of screening and data extraction, quality assessment, and statistical analysis. Results: We included 31 protocols and 35 publications. For the search strategy, 39 inconsistencies between the protocols and their publications were found in 26 of the 35 (74%) URs, and 16 of these inconsistencies were indicated and explained. There were 84 inconsistencies between the protocols and their URs regarding the inclusion criteria in 31 of the 35 (89%) URs, and 29 of the inconsistencies were indicated and explained. Deviations from their protocols were found in 12 of the 32 (38%) URs reporting the methods of screening, 14 of the 30 (47%) URs reporting the methods of data extraction, and 11 of the 32 (34%) URs reporting the methods for quality assessment. Of the 35 URs, 6 (17%) were inconsistent with their protocols in terms of the tools for quality assessment; one-half (3/6, 50%) of them indicated and explained the deviations. As for the statistical analysis, 31 of the 35 (89%) URs generated 61 inconsistencies between the publications and their protocols, and 16 inconsistencies were indicated and explained. Conclusions: There was a high prevalence of inconsistencies between protocols and publications of URs, and more than one-half of the inconsistencies were not indicated and explained in the publications. Therefore, how to promote the transparency of URs will be a major part of future work. UR - https://www.jmir.org/2023/1/e43299 UR - http://dx.doi.org/10.2196/43299 UR - http://www.ncbi.nlm.nih.gov/pubmed/37531172 ID - info:doi/10.2196/43299 ER - TY - JOUR AU - Murray, Aja AU - Ushakova, Anastasia AU - Zhu, Xinxin AU - Yang, Yi AU - Xiao, Zhuoni AU - Brown, Ruth AU - Speyer, Lydia AU - Ribeaud, Denis AU - Eisner, Manuel PY - 2023/8/2 TI - Predicting Participation Willingness in Ecological Momentary Assessment of General Population Health and Behavior: Machine Learning Study JO - J Med Internet Res SP - e41412 VL - 25 KW - ecological momentary assessment KW - experience sampling KW - machine learning KW - recruitment KW - sampling N2 - Background: Ecological momentary assessment (EMA) is widely used in health research to capture individuals? experiences in the flow of daily life. The majority of EMA studies, however, rely on nonprobability sampling approaches, leaving open the possibility of nonrandom participation concerning the individual characteristics of interest in EMA research. Knowledge of the factors that predict participation in EMA research is required to evaluate this possibility and can also inform optimal recruitment strategies. Objective: This study aimed to examine the extent to which being willing to participate in EMA research is related to respondent characteristics and to identify the most critical predictors of participation. Methods: We leveraged the availability of comprehensive data on a general young adult population pool of potential EMA participants and used and compared logistic regression, classification and regression trees, and random forest approaches to evaluate respondents? characteristic predictors of willingness to participate in the Decades-to-Minutes EMA study. Results: In unadjusted logistic regression models, gender, migration background, anxiety, attention deficit hyperactivity disorder symptoms, stress, and prosociality were significant predictors of participation willingness; in logistic regression models, mutually adjusting for all predictors, migration background, tobacco use, and social exclusion were significant predictors. Tree-based approaches also identified migration status, tobacco use, and prosociality as prominent predictors. However, overall, willingness to participate in the Decades-to-Minutes EMA study was only weakly predictable from respondent characteristics. Cross-validation areas under the curve for the best models were only in the range of 0.56 to 0.57. Conclusions: Results suggest that migration background is the single most promising target for improving EMA participation and sample representativeness; however, more research is needed to improve prediction of participation in EMA studies in health. UR - https://www.jmir.org/2023/1/e41412 UR - http://dx.doi.org/10.2196/41412 UR - http://www.ncbi.nlm.nih.gov/pubmed/37531181 ID - info:doi/10.2196/41412 ER - TY - JOUR AU - Harada, Yukinori AU - Tomiyama, Shusaku AU - Sakamoto, Tetsu AU - Sugimoto, Shu AU - Kawamura, Ren AU - Yokose, Masashi AU - Hayashi, Arisa AU - Shimizu, Taro PY - 2023/8/2 TI - Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence?Driven Automated History?Taking System: Pilot Cross-Sectional Study JO - JMIR Form Res SP - e49034 VL - 7 KW - collective intelligence KW - differential diagnosis generator KW - diagnostic accuracy KW - automated medical history taking system KW - artificial intelligence KW - AI N2 - Background: Low diagnostic accuracy is a major concern in automated medical history?taking systems with differential diagnosis (DDx) generators. Extending the concept of collective intelligence to the field of DDx generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple people than when accepting a diagnosis list from a single person may be a possible solution. Objective: The purpose of this study is to assess whether the combined use of several DDx generators improves the diagnostic accuracy of DDx lists. Methods: We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)?driven automated medical history?taking system from 103 patients with confirmed diagnoses. Two research physicians independently created the other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other 2 DDx generators based on the medical history generated by the automated medical history?taking system without reading the index lists generated by the automated medical history?taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (1) simply combining DDx lists from the index, second, and third lists; (2) creating a new top 10 DDx list using a 1/n weighting rule; and (3) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by 2 research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using 2 additional lists was 206 (103 cases × 2 physicians? input). Results: The diagnostic accuracy of the index lists was 46% (47/103). Diagnostic accuracy was improved by simply combining the other 2 DDx lists (133/206, 65%, P<.001), whereas the other 2 combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 52%, P=.05 in the collective list with the 1/n weighting rule and 29/206, 14%, P<.001 in the only shared diagnoses among the 3 DDx lists). Conclusions: Simply adding each of the top 10 DDx lists from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial. UR - https://formative.jmir.org/2023/1/e49034 UR - http://dx.doi.org/10.2196/49034 UR - http://www.ncbi.nlm.nih.gov/pubmed/37531164 ID - info:doi/10.2196/49034 ER - TY - JOUR AU - Lu, Qi Kevin Jia AU - Meaney, Christopher AU - Guo, Elaine AU - Leung, Fok-Han PY - 2023/7/27 TI - Evaluating the Applicability of Existing Lexicon-Based Sentiment Analysis Techniques on Family Medicine Resident Feedback Field Notes: Retrospective Cohort Study JO - JMIR Med Educ SP - e41953 VL - 9 KW - medical education KW - medical resident KW - feedback KW - field note KW - text mining KW - data mining KW - sentiment analysis KW - lexicon KW - lexical KW - dictionary KW - dictionaries KW - vocabulary KW - resident KW - medical student KW - medical trainee KW - residency KW - utility KW - feasibility N2 - Background: Field notes, a form for resident-preceptor clinical encounter feedback, are widely adopted across Canadian medical residency training programs for documenting residents? performance. This process generates a sizeable cumulative collection of feedback text, which is difficult for medical education faculty to navigate. As sentiment analysis is a subfield of text mining that can efficiently synthesize the polarity of a text collection, sentiment analysis may serve as an innovative solution. Objective: This study aimed to examine the feasibility and utility of sentiment analysis using 3 popular sentiment lexicons on medical resident field notes. Methods: We used a retrospective cohort design, curating text data from University of Toronto medical resident field notes gathered over 2 years (from July 2019 to June 2021). Lexicon-based sentiment analysis was applied using 3 standardized dictionaries, modified by removing ambiguous words as determined by a medical subject matter expert. Our modified lexicons assigned words from the text data a sentiment score, and we aggregated the word-level scores to a document-level polarity score. Agreement between dictionaries was assessed, and the document-level polarity was correlated with the overall preceptor rating of the clinical encounter under assessment. Results: Across the 3 original dictionaries, approximately a third of labeled words in our field note corpus were deemed ambiguous and were removed to create modified dictionaries. Across the 3 modified dictionaries, the mean sentiment for the ?Strengths? section of the field notes was mildly positive, while it was slightly less positive in the ?Areas of Improvement? section. We observed reasonable agreement between dictionaries for sentiment scores in both field note sections. Overall, the proportion of positively labeled documents increased with the overall preceptor rating, and the proportion of negatively labeled documents decreased with the overall preceptor rating. Conclusions: Applying sentiment analysis to systematically analyze field notes is feasible. However, the applicability of existing lexicons is limited in the medical setting, even after the removal of ambiguous words. Limited applicability warrants the need to generate new dictionaries specific to the medical education context. Additionally, aspect-based sentiment analysis may be applied to navigate the more nuanced structure of texts when identifying sentiments. Ultimately, this will allow for more robust inferences to discover opportunities for improving resident teaching curriculums. UR - https://mededu.jmir.org/2023/1/e41953 UR - http://dx.doi.org/10.2196/41953 UR - http://www.ncbi.nlm.nih.gov/pubmed/37498660 ID - info:doi/10.2196/41953 ER - TY - JOUR AU - McClure, A. Erin AU - Baker, Nathaniel AU - Walters, J. Kyle AU - Tomko, L. Rachel AU - Carpenter, J. Matthew AU - Bradley, Elizabeth AU - Squeglia, M. Lindsay AU - Gray, M. Kevin PY - 2023/7/27 TI - Monitoring Cigarette Smoking and Relapse in Young Adults With and Without Remote Biochemical Verification: Randomized Brief Cessation Study JO - JMIR Form Res SP - e47662 VL - 7 KW - technology KW - mHealth KW - young adults KW - cessation KW - relapse KW - biochemical verification KW - cigarette KW - smoking KW - monitoring KW - abstinence KW - mobile phone N2 - Background: Technological advancements to study young adult smoking, relapse, and to deliver interventions remotely offer conceptual appeal, but the incorporation of technological enhancement must demonstrate benefit over traditional methods without adversely affecting outcomes. Further, integrating remote biochemical verification of smoking and abstinence may yield value in the confirmation of self-reported smoking, in addition to ecologically valid, real-time assessments. Objective: The goal of this study was to evaluate the impact of remote biochemical verification on 24-hour self-reported smoking and biochemical verification agreement, retention, compliance with remote sessions, and abstinence during a brief, 5-week cessation attempt and relapse monitoring phase. Methods: Participants (N=39; aged 18-25 years; mean age 21.6, SD 2.1 years; n=22, 56% male; n=29, 74% White) who smoked cigarettes daily engaged in a 5-week cessation and monitoring study (including a 48-hour quit attempt and provision of tobacco treatment in the form of nicotine replacement therapy, brief cessation counseling, and financial incentives for abstinence during the 2-day quit attempt only). Smoking (cigarettes per day) was self-reported through ecological momentary assessment (EMA) procedures, and participants were randomized to either (1) the inclusion of remote biochemical verification (EMA + remote carbon monoxide [rCO]) 2× per day or (2) in-person, weekly CO (wCO). Groups were compared on the following outcomes: (1) agreement in self-reported smoking and breath carbon monoxide (CO) at common study time points, (2) EMA session compliance, (3) retention in study procedures, and (4) abstinence from smoking during the 2-day quit attempt and at the end of the 5-week study. Results: No significant differences were demonstrated between the rCO group and the wCO (weekly in-person study visit) group on agreement between 24-hour self-reported smoking and breath CO (moderate to poor), compliance with remote sessions, or retention, though these outcomes numerically favored the wCO group. Abstinence was numerically higher in the wCO group after the 2-day quit attempt and significantly different at the end of treatment (day 35), favoring the wCO group. Conclusions: Though study results should be interpreted with caution given the small sample size, findings suggest that the inclusion of rCO breath added to EMA compared to EMA with weekly, in-person CO collection in young adults did not yield benefit and may have even adversely affected outcomes. Our results suggest that technological advancements may improve data accuracy through objective measurement but may also introduce barriers and burdens and could result in higher rates of missing data. The inclusion of technology to inform smoking cessation research and intervention delivery among young adults should consider (1) the research question and necessity of biochemical verification and then (2) how to seamlessly incorporate monitoring into personalized and dynamic systems to avoid the added burden and detrimental effects to compliance and honesty in self-report. UR - https://formative.jmir.org/2023/1/e47662 UR - http://dx.doi.org/10.2196/47662 UR - http://www.ncbi.nlm.nih.gov/pubmed/37498643 ID - info:doi/10.2196/47662 ER - TY - JOUR AU - Tang, Mitchell AU - Sharma, Yashoda AU - Goldsack, C. Jennifer AU - Stern, Dora Ariel PY - 2023/7/26 TI - Building the Business Case for an Inclusive Approach to Digital Health Measurement With a Web App (Market Opportunity Calculator): Instrument Development Study JO - JMIR Form Res SP - e45713 VL - 7 KW - inclusion KW - digital health KW - digital product development KW - health equity KW - public health KW - Digital Health Measurement Collaborative Community KW - DATAcc N2 - Background: The use of digital health measurement tools has grown substantially in recent years. However, there are concerns that the promised benefits from these products will not be shared equitably. Underserved populations, such as those with lower education and income, racial and ethnic minorities, and those with disabilities, may find such tools poorly suited for their needs. Because underserved populations shoulder a disproportionate share of the US disease burden, they also represent a substantial share of digital health companies? target markets. Incorporating inclusive principles into the product development process can help ensure that the resulting tools are broadly accessible and effective. In this context, inclusivity not only maximizes societal benefit but also leads to greater commercial success. Objective: A critical element in fostering inclusive product development is building the business case for why it is worthwhile. The Digital Health Measurement Collaborative Community (DATAcc) Market Opportunity Calculator was developed as an open-access resource to enable digital health measurement product developers to build a business case for incorporating inclusive practices into their research and development processes. Methods: The DATAcc Market Opportunity Calculator combines data on population demographics and disease prevalence and health status from the US Census Bureau and the US Centers for Disease Control and Prevention (CDC). Together, these data are used to calculate the share of US adults with specific conditions (eg, diabetes) falling into various population segments along key ?inclusion vectors? (eg, race and ethnicity). Results: A free and open resource, the DATAcc Market Opportunity Calculator can be accessed from the DATAcc website. Users first select the target health condition addressed by their product, and then an inclusion vector to segment the patient population. The calculator displays each segment as a share of the overall US adult population and its share specifically among adults with the target condition, quantifying the importance of underserved patient segments to the target market. The calculator also estimates the value of improvements to product inclusivity by modeling the downstream impact on the accessible market size. For example, simplifying prompts on a hypertension-focused product to make it more accessible for adults with lower educational attainment is shown by the calculator to increase the target market by 2 million people and the total addressable market opportunity by US $200 million. Conclusions: Digital health measurement is still in its infancy. Now is the time to establish a precedent for inclusive product development to maximize societal benefit and build sustainable commercial returns. The Market Opportunity Calculator can help build the business case for ?why??showing how inclusivity can translate to financial opportunity. Once the decision has been made to pursue inclusive design, other components of the broader DATAcc toolkit for inclusive product development can support the ?how.? UR - https://formative.jmir.org/2023/1/e45713 UR - http://dx.doi.org/10.2196/45713 UR - http://www.ncbi.nlm.nih.gov/pubmed/37494108 ID - info:doi/10.2196/45713 ER - TY - JOUR AU - Greene, Brittney AU - Bernardo, Leah AU - Thompson, Morgan AU - Loughead, James AU - Ashare, Rebecca PY - 2023/7/21 TI - Behavioral Economic Strategies to Improve Enrollment Rates in Clinical Research: Embedded Recruitment Pilot Trial JO - JMIR Form Res SP - e47121 VL - 7 KW - behavior change KW - behavioral economics KW - clinical trials KW - contingency management KW - evidence based KW - information provision KW - recruitment KW - retention KW - SMS text messaging KW - study within a trial KW - SWAT N2 - Background: Nearly 1 in 3 clinical trials end prematurely due to underenrollment. Strategies to enhance recruitment are often implemented without scientific rigor to evaluate efficacy. Evidence-based, cost-effective behavioral economic strategies designed to influence decision-making may be useful to promote clinical trial enrollment. Objective: This study evaluated 2 behavioral economic strategies to improve enrollment and retention rates across 4 clinical trials: information provision (IP) and contingency management (CM; ie, lottery). IP targets descriptive and injunctive norms about participating in research and CM provides participants incentives to reinforce a target behavior. Methods: A sample of 212 participants was enrolled across 4 clinical trials focused on tobacco use: 2 focused on HIV and 2 focused on neuroimaging. The CM condition included a lottery: for each study visit completed, participants received 5 ?draws? from a bowl containing 500 ?chips? valued at US $0, US $1, US $5, or US $100. In the IP condition, text messages that targeted injunctive norms about research (eg, ?Many find it a rewarding way to advance science and be part of a community?) were sent through the Way to Health platform before all study visits. Participants were randomized to 1 of 4 conditions: IP, CM, IP+CM, or standard recruitment (SR). We performed logistic regression, controlling for sex and study, with condition as a between-subject predictor. Outcomes were the percentage of participants who attended a final eligibility visit (primary), met intent-to-treat (ITT) criteria (secondary), and completed the study (secondary). Recruitment was evaluated by the percentage of participants who attended a final eligibility visit, enrollment by ITT status, and retention by the percentage of participants who completed the study. Results: Rates of attending the eligibility visit and meeting ITT status were 58.9% (33/56) and 33.9% (19/56) for IP+CM; 45.5% (25/55) and 18.2% (10/55) for IP only; 41.5% (22/53) and 18.9% (10/53) for CM only; and 37.5% (18/48) and 12.5% (6/48) for SR, respectively. In the logistic regression, females were more likely to meet ITT status than males (odds ratio [OR] 2.7, 95% CI 1.2-5.7; P=.01). The IP+CM group was twice as likely to attend the final eligibility visit than the SR group (OR 2.4, 95% CI 1.1-5.2; P=.04). The IP+CM group was also significantly more likely to reach ITT status than the SR condition (OR 3.9, 95% CI 1.3-11.1; P=.01). Those who received any active intervention (IP, CM, or IP+CM) had a higher study completion rate (33/53, 63.5%) compared to those who received SR (5/12, 41.7%), but this difference was not significant (P=.26). Conclusions: Combining IP and CM strategies may motivate participants to participate in research and improve recruitment and retention rates. Evidence from this study provides preliminary support for the utility of behavioral economics strategies to improve enrollment and reduce attrition in clinical trials. UR - https://formative.jmir.org/2023/1/e47121 UR - http://dx.doi.org/10.2196/47121 UR - http://www.ncbi.nlm.nih.gov/pubmed/37477975 ID - info:doi/10.2196/47121 ER - TY - JOUR AU - Potter, H. Thomas B. AU - Pratap, Sharmila AU - Nicolas, Carlos Juan AU - Khan, S. Osman AU - Pan, P. Alan AU - Bako, T. Abdulaziz AU - Hsu, Enshuo AU - Johnson, Carnayla AU - Jefferson, N. Imory AU - Adegbindin, K. Sofiat AU - Baig, Eman AU - Kelly, R. Hannah AU - Jones, L. Stephen AU - Britz, W. Gavin AU - Tannous, Jonika AU - Vahidy, S. Farhaan PY - 2023/7/21 TI - A Neuro-Informatics Pipeline for Cerebrovascular Disease: Research Registry Development JO - JMIR Form Res SP - e40639 VL - 7 KW - clinical outcome KW - intracerebral hemorrhage KW - acute ischemic stroke KW - transient ischemic attack KW - subarachnoid hemorrhage KW - cerebral amyloid angiopathy KW - learning health system KW - electronic health record KW - data curation KW - database N2 - Background: Although stroke is well recognized as a critical disease, treatment options are often limited. Inpatient stroke encounters carry critical information regarding the mechanisms of stroke and patient outcomes; however, these data are typically formatted to support administrative functions instead of research. To support improvements in the care of patients with stroke, a substantive research data platform is needed. Objective: To advance a stroke-oriented learning health care system, we sought to establish a comprehensive research repository of stroke data using the Houston Methodist electronic health record (EHR) system. Methods: Dedicated processes were developed to import EHR data of patients with primary acute ischemic stroke, intracerebral hemorrhage (ICH), transient ischemic attack, and subarachnoid hemorrhage under a review board?approved protocol. Relevant patients were identified from discharge diagnosis codes and assigned registry patient identification numbers. For identified patients, extract, transform, and load processes imported EHR data of primary cerebrovascular disease admissions and available data from any previous or subsequent admissions. Data were loaded into patient-focused SQL objects to enable cross-sectional and longitudinal analyses. Primary data domains (admission details, comorbidities, laboratory data, medications, imaging data, and discharge characteristics) were loaded into separate relational tables unified by patient and encounter identification numbers. Computed tomography, magnetic resonance, and angiography images were retrieved. Imaging data from patients with ICH were assessed for hemorrhage characteristics and cerebral small vessel disease markers. Patient information needed to interface with other local and national databases was retained. Prospective patient outreach was established, with patients contacted via telephone to assess functional outcomes 30, 90, 180, and 365 days after discharge. Dashboards were constructed to provide investigators with data summaries to support access. Results: The Registry of Neurological Endpoint Assessments among Patients with Ischemic and Hemorrhagic Stroke (REINAH) database was constructed as a series of relational category-specific SQL objects. Encounter summaries and dashboards were constructed to draw from these objects, providing visual data summaries for investigators seeking to build studies based on REINAH data. As of June 2022, the database contains 18,061 total patients, including 1809 (10.02%) with ICH, 13,444 (74.43%) with acute ischemic stroke, 1221 (6.76%) with subarachnoid hemorrhage, and 3165 (17.52%) with transient ischemic attack. Depending on the cohort, imaging data from computed tomography are available for 85.83% (1048/1221) to 98.4% (1780/1809) of patients, with magnetic resonance imaging available for 27.85% (340/1221) to 85.54% (11,500/13,444) of patients. Outcome assessment has successfully contacted 56.1% (240/428) of patients after ICH, with 71.3% (171/240) of responders providing consent for assessment. Responders reported a median modified Rankin Scale score of 3 at 90 days after discharge. Conclusions: A highly curated and clinically focused research platform for stroke data will establish a foundation for future research that may fundamentally improve poststroke patient care and outcomes. UR - https://formative.jmir.org/2023/1/e40639 UR - http://dx.doi.org/10.2196/40639 UR - http://www.ncbi.nlm.nih.gov/pubmed/37477961 ID - info:doi/10.2196/40639 ER - TY - JOUR AU - Godfrey, M. Emily AU - Schwartz, R. Malaika AU - Stukovsky, Hinckley Karen D. AU - Woodward, Danielle AU - Magaret, S. Amalia AU - Aitken, L. Moira PY - 2023/7/18 TI - Web-Based Survey Piloting Process as a Model for Developing and Testing Past Contraceptive Use and Pregnancy History: Cystic Fibrosis Case Study JO - JMIR Form Res SP - e46395 VL - 7 KW - contraception KW - chronic disease KW - genetic disease KW - questionnaire reliability KW - surveys and questionnaires KW - contraceptive KW - birth control KW - cystic fibrosis KW - reproductive health KW - pregnancy KW - electronic survey KW - obstetrical history N2 - Background: Individuals with complex, chronic diseases are now living longer, making reproductive health an important topic to address in the health care setting. Self-respondent surveys are a feasible way to collect past contraceptive use and pregnancy history to assess contraceptive safety and effectiveness. Because sensitive topics, such as contraception and pregnancy outcomes, can vary across social groups or cultures, piloting questions and evaluating survey administration procedures in the target population are necessary for precise and reliable responses before wide distribution. Objective: This study aimed to develop a precise and reliable survey instrument and related procedures among individuals with cystic fibrosis regarding contraceptive use and obstetrical history. Methods: We piloted and tested web-based questions related to contraceptive use and pregnancy history among 50 participants with and those without cystic fibrosis aged 18 to 45 years using a 3-tier process. Findings from each tier informed changes to the questionnaire before testing in the subsequent tier. Tier 1 used cognitive pretesting to assess question understanding and the need for memory prompts. In tier 2, we used test-retest self- and interviewer-administered approaches to assess question reliability, evaluate response missingness, and determine confidence between 2 types of survey administration approaches. In tier 3, we tested the questionnaire for clarity, time to complete, and whether additional prompts were necessary. Results: In tier 1, respondents suggested improvements to the web-based survey questions and to the written and visual prompts for better recall regarding past contraceptive use. In tier 2, the test-retest reliability between self- and interviewer-administrative procedures of ?ever use? contraceptive method questions was similar, with percent absolute agreement ranging between 84% and 100%. When the survey was self-administered, the percentage of missing responses was higher and respondent confidence about month and year when contraceptive methods were used was lower. Most respondents reported that they preferred the self-administered survey because it was more convenient and faster to complete. Conclusions: Our 3-tier process to pilot web-based survey questions related to contraceptive and obstetrical history in our complex disease population helped us tailor content and format questions before wide dissemination to our target population. Results from this pilot study informed the subsequent larger study design to include a 10% respondent test-retest self- and interviewer-administered quality control component to better inform imputation procedures of missing data. UR - https://formative.jmir.org/2023/1/e46395 UR - http://dx.doi.org/10.2196/46395 UR - http://www.ncbi.nlm.nih.gov/pubmed/37463015 ID - info:doi/10.2196/46395 ER - TY - JOUR AU - Yang, Dan AU - Su, Zihan AU - Mu, Runqing AU - Diao, Yingying AU - Zhang, Xin AU - Liu, Yusi AU - Wang, Shuo AU - Wang, Xu AU - Zhao, Lei AU - Wang, Hongyi AU - Zhao, Min PY - 2023/7/17 TI - Effects of Using Different Indirect Techniques on the Calculation of Reference Intervals: Observational Study JO - J Med Internet Res SP - e45651 VL - 25 KW - comparative study KW - data transformation KW - indirect method KW - outliers KW - reference interval KW - clinical decision-making KW - complete blood count KW - red blood cells KW - white blood cells KW - platelets KW - laboratory KW - clinical N2 - Background: Reference intervals (RIs) play an important role in clinical decision-making. However, due to the time, labor, and financial costs involved in establishing RIs using direct means, the use of indirect methods, based on big data previously obtained from clinical laboratories, is getting increasing attention. Different indirect techniques combined with different data transformation methods and outlier removal might cause differences in the calculation of RIs. However, there are few systematic evaluations of this. Objective: This study used data derived from direct methods as reference standards and evaluated the accuracy of combinations of different data transformation, outlier removal, and indirect techniques in establishing complete blood count (CBC) RIs for large-scale data. Methods: The CBC data of populations aged ?18 years undergoing physical examination from January 2010 to December 2011 were retrieved from the First Affiliated Hospital of China Medical University in northern China. After exclusion of repeated individuals, we performed parametric, nonparametric, Hoffmann, Bhattacharya, and truncation points and Kolmogorov?Smirnov distance (kosmic) indirect methods, combined with log or BoxCox transformation, and Reed?Dixon, Tukey, and iterative mean (3SD) outlier removal methods in order to derive the RIs of 8 CBC parameters and compared the results with those directly and previously established. Furthermore, bias ratios (BRs) were calculated to assess which combination of indirect technique, data transformation pattern, and outlier removal method is preferrable. Results: Raw data showed that the degrees of skewness of the white blood cell (WBC) count, platelet (PLT) count, mean corpuscular hemoglobin (MCH), mean corpuscular hemoglobin concentration (MCHC), and mean corpuscular volume (MCV) were much more obvious than those of other CBC parameters. After log or BoxCox transformation combined with Tukey or iterative mean (3SD) processing, the distribution types of these data were close to Gaussian distribution. Tukey-based outlier removal yielded the maximum number of outliers. The lower-limit bias of WBC (male), PLT (male), hemoglobin (HGB; male), MCH (male/female), and MCV (female) was greater than that of the corresponding upper limit for more than half of 30 indirect methods. Computational indirect choices of CBC parameters for males and females were inconsistent. The RIs of MCHC established by the direct method for females were narrow. For this, the kosmic method was markedly superior, which contrasted with the RI calculation of CBC parameters with high |BR| qualification rates for males. Among the top 10 methodologies for the WBC count, PLT count, HGB, MCV, and MCHC with a high-BR qualification rate among males, the Bhattacharya, Hoffmann, and parametric methods were superior to the other 2 indirect methods. Conclusions: Compared to results derived by the direct method, outlier removal methods and indirect techniques markedly influence the final RIs, whereas data transformation has negligible effects, except for obviously skewed data. Specifically, the outlier removal efficiency of Tukey and iterative mean (3SD) methods is almost equivalent. Furthermore, the choice of indirect techniques depends more on the characteristics of the studied analyte itself. This study provides scientific evidence for clinical laboratories to use their previous data sets to establish RIs. UR - https://www.jmir.org/2023/1/e45651 UR - http://dx.doi.org/10.2196/45651 UR - http://www.ncbi.nlm.nih.gov/pubmed/37459170 ID - info:doi/10.2196/45651 ER - TY - JOUR AU - Duran, T. Andrea AU - Keener-DeNoia, Adrianna AU - Stavrolakes, Kimberly AU - Fraser, Adina AU - Blanco, V. Luis AU - Fleisch, Emily AU - Pieszchata, Nicole AU - Cannone, Diane AU - Keys McKay, Charles AU - Whittman, Emma AU - Edmondson, Donald AU - Shelton, C. Rachel AU - Moise, Nathalie PY - 2023/7/13 TI - Applying User-Centered Design and Implementation Science to the Early-Stage Development of a Telehealth-Enhanced Hybrid Cardiac Rehabilitation Program: Quality Improvement Study JO - JMIR Form Res SP - e47264 VL - 7 KW - user-centered design KW - implementation science KW - cardiac rehabilitation KW - telemedicine KW - remote patient monitoring KW - behavioral intervention development KW - hybrid N2 - Background: Cardiac rehabilitation (CR) is an evidence-based intervention that improves event-free survival in patients with cardiac conditions, yet <27% of all eligible patients use CR in the United States. CR is traditionally delivered in clinic-based settings where implementation barriers abound. Innovative nontraditional program designs and strategies are needed to support widespread CR uptake. Objective: This study aimed to demonstrate how user-centered design (UCD) and implementation science (IS) principles and methods can be integrated into the early-stage development of nontraditional CR interventions. Methods: As part of a NewYork-Presbyterian Hospital (NYPH) quality improvement initiative (March 2020-February 2022), we combined UCD and IS principles and methods to design a novel home- and clinic-based telehealth-enhanced hybrid CR (THCR) program. We co-designed this program with multilevel stakeholders using an iterative 3-step UCD process to identify user and contextual barriers and facilitators to CR uptake (using semistructured interviews and contextual inquiry [step 1]), design an intervention prototype that targets contextual and user factors and emulates the evidence-based practice (through design workshops and journey mapping [step 2]), and review and refine the prototype (according to real-world usability testing and feedback [step 3]). The UCD process was informed by the Theoretical Domains Framework and Consolidated Framework for Implementation Research. Results: At step 1, we conducted semistructured interviews with 9 provider- and system-level stakeholders (female: n=6, 67%) at 3 geographically diverse academic medical centers, which revealed behavioral (eg, self-efficacy and knowledge) and contextual (eg, social distancing guidelines, physical space, staffing, and reimbursement) barriers to uptake; hybrid delivery was a key facilitator. Step 2 involved conducting 20 design workshops and 3 journey-mapping sessions with multidisciplinary NYPH stakeholders (eg, digital health team, CR clinicians, and creative director) where we identified key design elements (eg, mix of clinic- and home-based CR and synchronous remote patient monitoring), yielding an initial THCR prototype that leveraged NYPH?s telehealth infrastructure. At step 3, we conducted usability testing with 2 CR clinicians (both female) administering home-based sessions to 3 CR patients (female: n=1, 33%), which revealed usability themes (eg, ease of using remote patient monitoring devices or a telehealth platform, technology disruptions, and confidence in using the telehealth platform to safely monitor patients) and design solutions (eg, onboarding sessions, safety surveys, and fully supervised remote sessions) to be included in the final THCR prototype. Conclusions: Combining UCD and IS methods while engaging multidisciplinary stakeholders in an iterative process yielded a theory-informed THCR program targeting user and contextual barriers to real-world CR implementation. We provide a detailed summary of the process and guidance for incorporating UCD and IS principles and methods into the early-stage development of a nontraditional CR intervention. The feasibility, acceptability, appropriateness, and usability of the final THCR prototype is being evaluated in an ongoing study. UR - https://formative.jmir.org/2023/1/e47264 UR - http://dx.doi.org/10.2196/47264 UR - http://www.ncbi.nlm.nih.gov/pubmed/37440285 ID - info:doi/10.2196/47264 ER - TY - JOUR AU - Rosen, Barthlow Claire AU - Roberts, Eugene Sanford AU - Syvyk, Solomiya AU - Finn, Caitlin AU - Tong, Jason AU - Wirtalla, Christopher AU - Spinks, Hunter AU - Kelz, Rapaport Rachel PY - 2023/7/13 TI - A Novel Mobile App to Identify Patients With Multimorbidity in the Emergency Setting: Development of an App and Feasibility Trial JO - JMIR Form Res SP - e42970 VL - 7 KW - clinical operationalization KW - delphi KW - development KW - emergency KW - general surgery KW - mHealth KW - mobile app KW - mobile health KW - morbidity KW - multimorbidity KW - qualifying comorbidity set KW - surgery KW - usability N2 - Background: Multimorbidity is associated with an increased risk of poor surgical outcomes among older adults; however, identifying multimorbidity in the clinical setting can be a challenge. Objective: We created the Multimorbid Patient Identifier App (MMApp) to easily identify patients with multimorbidity identified by the presence of a Qualifying Comorbidity Set and tested its feasibility for use in future clinical research, validation, and eventually to guide clinical decision-making. Methods: We adapted the Qualifying Comorbidity Sets? claims-based definition of multimorbidity for clinical use through a modified Delphi approach and developed MMApp. A total of 10 residents input 5 hypothetical emergency general surgery patient scenarios, common among older adults, into the MMApp and examined MMApp test characteristics for a total of 50 trials. For MMApp, comorbidities selected for each scenario were recorded, along with the number of comorbidities correctly chosen, incorrectly chosen, and missed for each scenario. The sensitivity and specificity of identifying a patient as multimorbid using MMApp were calculated using composite data from all scenarios. To assess model feasibility, we compared the mean task completion by scenario to that of the American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator (ACS-NSQIP-SRC) using paired t tests. Usability and satisfaction with MMApp were assessed using an 18-item questionnaire administered immediately after completing all 5 scenarios. Results: There was no significant difference in the task completion time between the MMApp and the ACS-NSQIP-SRC for scenarios A (86.3 seconds vs 74.3 seconds, P=.85) or C (58.4 seconds vs 68.9 seconds,P=.064), MMapp took less time for scenarios B (76.1 seconds vs 87.4 seconds, P=.03) and E (20.7 seconds vs 73 seconds, P<.001), and more time for scenario D (78.8 seconds vs 58.5 seconds, P=.02). The MMApp identified multimorbidity with 96.7% (29/30) sensitivity and 95% (19/20) specificity. User feedback was positive regarding MMApp?s usability, efficiency, and usefulness. Conclusions: The MMApp identified multimorbidity with high sensitivity and specificity and did not require significantly more time to complete than a commonly used web-based risk-stratification tool for most scenarios. Mean user times were well under 2 minutes. Feedback was overall positive from residents regarding the usability and usefulness of this app, even in the emergency general surgery setting. It would be feasible to use MMApp to identify patients with multimorbidity in the emergency general surgery setting for validation, research, and eventual clinical use. This type of mobile app could serve as a template for other research teams to create a tool to easily screen participants for potential enrollment. UR - https://formative.jmir.org/2023/1/e42970 UR - http://dx.doi.org/10.2196/42970 UR - http://www.ncbi.nlm.nih.gov/pubmed/37440310 ID - info:doi/10.2196/42970 ER - TY - JOUR AU - Besculides, Melanie AU - Mazumdar, Madhu AU - Phlegar, Sydney AU - Freeman, Robert AU - Wilson, Sara AU - Joshi, Himanshu AU - Kia, Arash AU - Gorbenko, Ksenia PY - 2023/7/13 TI - Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning?Based Clinical Decision Support Systems JO - JMIR Form Res SP - e42262 VL - 7 KW - machine learning KW - AI KW - CDSS KW - evaluation KW - nutrition KW - screening KW - clinical KW - usability KW - effectiveness KW - treatment KW - malnutrition KW - decision-making KW - tool KW - data KW - acceptability N2 - Background: Machine learning (ML)?based clinical decision support systems (CDSS) are popular in clinical practice settings but are often criticized for being limited in usability, interpretability, and effectiveness. Evaluating the implementation of ML-based CDSS is critical to ensure CDSS is acceptable and useful to clinicians and helps them deliver high-quality health care. Malnutrition is a common and underdiagnosed condition among hospital patients, which can have serious adverse impacts. Early identification and treatment of malnutrition are important. Objective: This study aims to evaluate the implementation of an ML tool, Malnutrition Universal Screening Tool (MUST)?Plus, that predicts hospital patients at high risk for malnutrition and identify best implementation practices applicable to this and other ML-based CDSS. Methods: We conducted a qualitative postimplementation evaluation using in-depth interviews with registered dietitians (RDs) who use MUST-Plus output in their everyday work. After coding the data, we mapped emergent themes onto select domains of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework. Results: We interviewed 17 of the 24 RDs approached (71%), representing 37% of those who use MUST-Plus output. Several themes emerged: (1) enhancements to the tool were made to improve accuracy and usability; (2) MUST-Plus helped identify patients that would not otherwise be seen; perceived usefulness was highest in the original site; (3) perceived accuracy varied by respondent and site; (4) RDs valued autonomy in prioritizing patients; (5) depth of tool understanding varied by hospital and level; (6) MUST-Plus was integrated into workflows and electronic health records; and (7) RDs expressed a desire to eventually have 1 automated screener. Conclusions: Our findings suggest that continuous involvement of stakeholders at new sites given staff turnover is vital to ensure buy-in. Qualitative research can help identify the potential bias of ML tools and should be widely used to ensure health equity. Ongoing collaboration among CDSS developers, data scientists, and clinical providers may help refine CDSS for optimal use and improve the acceptability of CDSS in the clinical context. UR - https://formative.jmir.org/2023/1/e42262 UR - http://dx.doi.org/10.2196/42262 UR - http://www.ncbi.nlm.nih.gov/pubmed/37440303 ID - info:doi/10.2196/42262 ER - TY - JOUR AU - Shan, Yi AU - Ji, Meng AU - Xing, Zhaoquan AU - Dong, Zhaogang AU - Wang, Ding AU - Cao, Xiangting PY - 2023/7/7 TI - The Simplified Chinese Version of the Suitability Assessment of Materials for the Evaluation of Health-Related Information for Adults: Translation and Validation Study JO - JMIR Form Res SP - e41609 VL - 7 KW - simplified Chinese version of the SAM KW - translation KW - validation KW - suitability KW - health education materials N2 - Background: Suitable health education materials can educate people about the potential harms of high-risk factors, leading to expected behavior changes and improved health outcomes. However, most patient education materials were not suitable in terms of content, structure, design, composition, and language, as stated in the literature. There is a pressing need to use well-designed scales to assess the suitability of health education materials. Although such assessment is a common practice in English-speaking communities, few assessment tools are available in mainland China. Objective: This study aimed to translate the Suitability Assessment of Materials (SAM) for the evaluation of health-related information for adults into a simplified Chinese version (S-C-SAM) and validate its reliability for evaluating the suitability of health education materials written in simplified Chinese in mainland China. Methods: The SAM was translated into an S-C-SAM in three steps: (1) translating the SAM into an S-C-SAM, (2) translating the S-C-SAM back into an English version, and (3) testing the translation equivalence between the 2 English versions (original and back-translated) of the SAM linguistically and culturally. Any differences between these 2 English versions were resolved through a panel discussion. The validity of the S-C-SAM was determined by measuring its content validity index. The final version of the S-C-SAM was used by 3 native Chinese-speaking health educators to assess 15 air pollution?related health education materials. The Cohen ? coefficient and Cronbach ? were calculated to determine the interrater agreement and internal consistency of the S-C-SAM. Results: We agreed on the final version of the S-C-SAM after settling the discrepancies between the 2 English versions (original and back-translated) and revising 2 items (sentences) rated negatively in content validation. The S-C-SAM was proven valid and reliable: the content validity index was 0.95 both in clarity and in relevance, the Cohen ? coefficient for the interrater agreement was 0.61 (P<.05), and Cronbach ? for the internal consistency of the whole scale was .71. Conclusions: The S-C-SAM is the first simplified Chinese version of the SAM. It has been proven valid and reliable for evaluating the suitability of air pollution?related health education materials written in simplified Chinese in mainland China. It has the potential to be used for assessing the suitability of health education materials specifically selected for other health education purposes. UR - https://formative.jmir.org/2023/1/e41609 UR - http://dx.doi.org/10.2196/41609 UR - http://www.ncbi.nlm.nih.gov/pubmed/37418296 ID - info:doi/10.2196/41609 ER - TY - JOUR AU - Brady, Brooke AU - Zhou, Shally AU - Ashworth, Daniel AU - Zheng, Lidan AU - Eramudugolla, Ranmalee AU - Huque, Hamidul Md AU - Anstey, Jane Kaarin PY - 2023/7/6 TI - A Technology-Enriched Approach to Studying Microlongitudinal Aging Among Adults Aged 18 to 85 Years: Protocol for the Labs Without Walls Study JO - JMIR Res Protoc SP - e47053 VL - 12 KW - life-course aging KW - longitudinal research KW - subjective age KW - gender KW - cognition KW - sensory function KW - app KW - mobile app KW - eHealth KW - mobile health KW - mHealth KW - measurement burst design KW - ecological momentary assessment KW - health information technology KW - personalized health KW - mobile phone N2 - Background: Traditional longitudinal aging research involves studying the same individuals over a long period, with measurement intervals typically several years apart. App-based studies have the potential to provide new insights into life-course aging by improving the accessibility, temporal specificity, and real-world integration of data collection. We developed a new research app for iOS named Labs Without Walls to facilitate the study of life-course aging. Combined with data collected using paired smartwatches, the app collects complex data including data from one-time surveys, daily diary surveys, repeated game-like cognitive and sensory tasks, and passive health and environmental data. Objective: The aim of this protocol is to describe the research design and methods of the Labs Without Walls study conducted between 2021 and 2023 in Australia. Methods: Overall, 240 Australian adults will be recruited, stratified by age group (18-25, 26-35, 36-45, 46-55, 56-65, 66-75, and 76-85 years) and sex at birth (male and female). Recruitment procedures include emails to university and community networks, as well as paid and unpaid social media advertisements. Participants will be invited to complete the study onboarding either in person or remotely. Participants who select face-to-face onboarding (n=approximately 40) will be invited to complete traditional in-person cognitive and sensory assessments to be cross-validated against their app-based counterparts. Participants will be sent an Apple Watch and headphones for use during the study period. Participants will provide informed consent within the app and then begin an 8-week study protocol, which includes scheduled surveys, cognitive and sensory tasks, and passive data collection using the app and a paired watch. At the conclusion of the study period, participants will be invited to rate the acceptability and usability of the study app and watch. We hypothesize that participants will be able to successfully provide e-consent, input survey data through the Labs Without Walls app, and have passive data collected over 8 weeks; participants will rate the app and watch as user-friendly and acceptable; the app will allow for the study of daily variability in self-perceptions of age and gender; and data will allow for the cross-validation of app- and laboratory-based cognitive and sensory tasks. Results: Recruitment began in May 2021, and data collection was completed in February 2023. The publication of preliminary results is anticipated in 2023. Conclusions: This study will provide evidence regarding the acceptability and usability of the research app and paired watch for studying life-course aging processes on multiple timescales. The feedback obtained will be used to improve future iterations of the app, explore preliminary evidence for intraindividual variability in self-perceptions of aging and gender expression across the life span, and explore the associations between performance on app-based cognitive and sensory tests and that on similar traditional cognitive and sensory tests. International Registered Report Identifier (IRRID): DERR1-10.2196/47053 UR - https://www.researchprotocols.org/2023/1/e47053 UR - http://dx.doi.org/10.2196/47053 UR - http://www.ncbi.nlm.nih.gov/pubmed/37410527 ID - info:doi/10.2196/47053 ER - TY - JOUR AU - Idigbe, Ifeoma AU - Gbaja-Biamila, Titilola AU - Asuquo, Sarah AU - Nwaozuru, Ucheoma AU - Obiezu-Umeh, Chisom AU - Tahlil, M. Kadija AU - Musa, Zaidat Adesola AU - Oladele, David AU - Kapogiannis, Bill AU - Tucker, Joseph AU - Iwelunmor, Juliet AU - Ezechi, Oliver PY - 2023/6/29 TI - Using a Designathon to Develop an HIV Self-Testing Intervention to Improve Linkage to Care Among Youths in Nigeria: Qualitative Approach Based on a Participatory Research Action Framework JO - JMIR Form Res SP - e38528 VL - 7 KW - crowdsourcing KW - youth-led strategies KW - linkage to care KW - HIV KW - HIV testing KW - public health KW - participatory medicine KW - youth N2 - Background: UNAIDS (Joint United Nations Programme on HIV and AIDS) and the Nigeria National HIV/AIDS Strategic Framework recommend HIV self-testing and youth-friendly services to enhance HIV testing, linkage to health services, and prevention. However, the voices of youths are seldom incorporated into interventions. We examined qualitative data generated from a series of participatory events in partnership with Nigerian youths focused on enhancing linkage to care. Objective: The aim of this study was to assess youth-initiated interventions developed during a designathon to improve linkage to care and sexually transmitted infection services. Methods: This study conducted a designathon informed by crowdsourcing principles and the participatory research action framework. A designathon is a multistage process including an open call, a sprint event, and follow-up activities. The open call solicited Nigerian youths (14-24 years old) to develop intervention strategies for linkage to care and youth-friendly health services. A total of 79 entries were received; from this, a subset of 13 teams responded to the open call and was invited to participate in a sprint event over 72 hours. Narratives from the open-call proposals were analyzed using grounded theory to identify emergent themes focused on youth-proposed interventions for linkage to care and youth-friendly services. Results: A total of 79 entries (through the web=26; offline=53) were submitted. Women or girls submitted 40 of the 79 (51%) submissions. The average age of participants was 17 (SD 2.7) years, and 64 of 79 (81%) participants had secondary education or less. Two main themes highlighted strategies for enhancing youths? HIV linkage to care: digital interventions and collaboration with youth influencers. A total of 76 participants suggested digital interventions that would facilitate anonymous web-based counseling, text prompt referrals, and related services. In addition, 16 participants noted that collaboration with youth influencers would be useful. This could involve working in partnership with celebrities, gatekeepers, or others who have a large youth audience to enhance the promotion of messages on HIV self-testing and linkage. The facilitators of youths? linkage included health facility restructuring, dedicated space for youths, youth-trained staff, youth-friendly amenities, and subsidized fees. Barriers to HIV linkage to care among youths included a lack of privacy at clinics and concerns about the potential for breaching confidentiality. Conclusions: Our data suggest specific strategies that may be useful for enhancing HIV linkage to care for Nigerian youths, but further research is needed to assess the feasibility and implementation of these strategies. Designathons are an effective way to generate ideas from youths. UR - https://formative.jmir.org/2023/1/e38528 UR - http://dx.doi.org/10.2196/38528 UR - http://www.ncbi.nlm.nih.gov/pubmed/37384385 ID - info:doi/10.2196/38528 ER - TY - JOUR AU - Chaturvedi, Jaya AU - Chance, Natalia AU - Mirza, Luwaiza AU - Vernugopan, Veshalee AU - Velupillai, Sumithra AU - Stewart, Robert AU - Roberts, Angus PY - 2023/6/26 TI - Development of a Corpus Annotated With Mentions of Pain in Mental Health Records: Natural Language Processing Approach JO - JMIR Form Res SP - e45849 VL - 7 KW - pain KW - mental health KW - natural language processing KW - annotation KW - information extraction N2 - Background: Pain is a widespread issue, with 20% of adults (1 in 5) experiencing it globally. A strong association has been demonstrated between pain and mental health conditions, and this association is known to exacerbate disability and impairment. Pain is also known to be strongly related to emotions, which can lead to damaging consequences. As pain is a common reason for people to access health care facilities, electronic health records (EHRs) are a potential source of information on this pain. Mental health EHRs could be particularly beneficial since they can show the overlap of pain with mental health. Most mental health EHRs contain the majority of their information within the free-text sections of the records. However, it is challenging to extract information from free text. Natural language processing (NLP) methods are therefore required to extract this information from the text. Objective: This research describes the development of a corpus of manually labeled mentions of pain and pain-related entities from the documents of a mental health EHR database, for use in the development and evaluation of future NLP methods. Methods: The EHR database used, Clinical Record Interactive Search, consists of anonymized patient records from The South London and Maudsley National Health Service Foundation Trust in the United Kingdom. The corpus was developed through a process of manual annotation where pain mentions were marked as relevant (ie, referring to physical pain afflicting the patient), negated (ie, indicating absence of pain), or not relevant (ie, referring to pain affecting someone other than the patient, or metaphorical and hypothetical mentions). Relevant mentions were also annotated with additional attributes such as anatomical location affected by pain, pain character, and pain management measures, if mentioned. Results: A total of 5644 annotations were collected from 1985 documents (723 patients). Over 70% (n=4028) of the mentions found within the documents were annotated as relevant, and about half of these mentions also included the anatomical location affected by the pain. The most common pain character was chronic pain, and the most commonly mentioned anatomical location was the chest. Most annotations (n=1857, 33%) were from patients who had a primary diagnosis of mood disorders (International Classification of Diseases?10th edition, chapter F30-39). Conclusions: This research has helped better understand how pain is mentioned within the context of mental health EHRs and provided insight into the kind of information that is typically mentioned around pain in such a data source. In future work, the extracted information will be used to develop and evaluate a machine learning?based NLP application to automatically extract relevant pain information from EHR databases. UR - https://formative.jmir.org/2023/1/e45849 UR - http://dx.doi.org/10.2196/45849 UR - http://www.ncbi.nlm.nih.gov/pubmed/37358897 ID - info:doi/10.2196/45849 ER - TY - JOUR AU - Rao, Kaushal AU - Speier, William AU - Meng, Yiwen AU - Wang, Jinhan AU - Ramesh, Nidhi AU - Xie, Fenglong AU - Su, Yujie AU - Nowell, Benjamin W. AU - Curtis, R. Jeffrey AU - Arnold, Corey PY - 2023/6/26 TI - Machine Learning Approaches to Classify Self-Reported Rheumatoid Arthritis Health Scores Using Activity Tracker Data: Longitudinal Observational Study JO - JMIR Form Res SP - e43107 VL - 7 KW - rheumatoid arthritis KW - rheumatic KW - rheumatism KW - Fitbit KW - classification KW - physical data KW - digital health KW - activity tracker KW - mobile health KW - machine learning KW - model KW - patient reported KW - outcome measure KW - PROMIS KW - nonclinical monitoring KW - mHealth KW - tracker KW - wearable KW - arthritis KW - mobile phone N2 - Background: The increasing use of activity trackers in mobile health studies to passively collect physical data has shown promise in lessening participation burden to provide actively contributed patient-reported outcome (PRO) information. Objective: The aim of this study was to develop machine learning models to classify and predict PRO scores using Fitbit data from a cohort of patients with rheumatoid arthritis. Methods: Two different models were built to classify PRO scores: a random forest classifier model that treated each week of observations independently when making weekly predictions of PRO scores, and a hidden Markov model that additionally took correlations between successive weeks into account. Analyses compared model evaluation metrics for (1) a binary task of distinguishing a normal PRO score from a severe PRO score and (2) a multiclass task of classifying a PRO score state for a given week. Results: For both the binary and multiclass tasks, the hidden Markov model significantly (P<.05) outperformed the random forest model for all PRO scores, and the highest area under the curve, Pearson correlation coefficient, and Cohen ? coefficient were 0.750, 0.479, and 0.471, respectively. Conclusions: While further validation of our results and evaluation in a real-world setting remains, this study demonstrates the ability of physical activity tracker data to classify health status over time in patients with rheumatoid arthritis and enables the possibility of scheduling preventive clinical interventions as needed. If patient outcomes can be monitored in real time, there is potential to improve clinical care for patients with other chronic conditions. UR - https://formative.jmir.org/2023/1/e43107 UR - http://dx.doi.org/10.2196/43107 UR - http://www.ncbi.nlm.nih.gov/pubmed/37017471 ID - info:doi/10.2196/43107 ER - TY - JOUR AU - Tonkin, Sarah AU - Gass, Julie AU - Wray, Jennifer AU - Maguin, Eugene AU - Mahoney, Martin AU - Colder, Craig AU - Tiffany, Stephen AU - Hawk Jr, W. Larry PY - 2023/6/22 TI - Evaluating Declines in Compliance With Ecological Momentary Assessment in Longitudinal Health Behavior Research: Analyses From a Clinical Trial JO - J Med Internet Res SP - e43826 VL - 25 KW - ecological momentary assessment KW - compliance KW - health behavior KW - methodology KW - longitudinal KW - smoking KW - smoker KW - cessation KW - quit KW - adherence KW - dropout KW - RCT KW - cigar KW - retention N2 - Background: Ecological momentary assessment (EMA) is increasingly used to evaluate behavioral health processes over extended time periods. The validity of EMA for providing representative, real-world data with high temporal precision is threatened to the extent that EMA compliance drops over time. Objective: This research builds on prior short-term studies by evaluating the time course of EMA compliance over 9 weeks and examines predictors of weekly compliance rates among cigarette-using adults. Methods: A total of 257 daily cigarette-using adults participating in a randomized controlled trial for smoking cessation completed daily smartphone EMA assessments, including 1 scheduled morning assessment and 4 random assessments per day. Weekly EMA compliance was calculated and multilevel modeling assessed the rate of change in compliance over the 9-week assessment period. Participant and study characteristics were examined as predictors of overall compliance and changes in compliance rates over time. Results: Compliance was higher for scheduled morning assessments (86%) than for random assessments (58%) at the beginning of the EMA period (P<.001). EMA compliance declined linearly across weeks, and the rate of decline was greater for morning assessments (2% per week) than for random assessments (1% per week; P<.001). Declines in compliance were stronger for younger participants (P<.001), participants who were employed full-time (P=.03), and participants who subsequently dropped out of the study (P<.001). Overall compliance was higher among White participants compared to Black or African American participants (P=.001). Conclusions: This study suggests that EMA compliance declines linearly but modestly across lengthy EMA protocols. In general, these data support the validity of EMA for tracking health behavior and hypothesized treatment mechanisms over the course of several months. Future work should target improving compliance among subgroups of participants and investigate the extent to which rapid declines in EMA compliance might prove useful for triggering interventions to prevent study dropout. Trial Registration: ClinicalTrials.gov NCT03262662; https://clinicaltrials.gov/ct2/show/NCT03262662 UR - https://www.jmir.org/2023/1/e43826 UR - http://dx.doi.org/10.2196/43826 UR - http://www.ncbi.nlm.nih.gov/pubmed/37347538 ID - info:doi/10.2196/43826 ER - TY - JOUR AU - Reid, C. Sean AU - Wang, Vania AU - Assaf, D. Ryan AU - Kaloper, Sofia AU - Murray, T. Alan AU - Shoptaw, Steven AU - Gorbach, Pamina AU - Cassels, Susan PY - 2023/6/22 TI - Novel Location-Based Survey Using Cognitive Interviews to Assess Geographic Networks and Hotspots of Sex and Drug Use: Implementation and Validation Study JO - JMIR Form Res SP - e45188 VL - 7 KW - networks KW - sexual network geography KW - activity space KW - HIV KW - survey design KW - risk hotspots KW - cognitive interviews KW - health interventions KW - mobile phone N2 - Background: The Ending the HIV Epidemic initiative in the United States relies on HIV hotspots to identify where to geographically target new resources, expertise, and technology. However, interventions targeted at places with high HIV transmission and infection risk, not just places with high HIV incidence, may be more effective at reducing HIV incidence and achieving health equity. Objective: We described the implementation and validation of a web-based activity space survey on HIV risk behaviors. The survey was intended to collect geographic information that will be used to map risk behavior hotspots as well as the geography of sexual networks in Los Angeles County. Methods: The survey design team developed a series of geospatial questions that follow a 3-level structure that becomes more geographically precise as participants move through the levels. The survey was validated through 9 cognitive interviews and iteratively updated based on participant feedback until the saturation of topics and technical issues was reached. Results: In total, 4 themes were identified through the cognitive interviews: functionality of geospatial questions, representation and accessibility, privacy, and length and understanding of the survey. The ease of use for the geospatial questions was critical as many participants were not familiar with mapping software. The inclusion of well-known places, landmarks, and road networks was critical for ease of use. The addition of a Google Maps interface, which was familiar to many participants, aided in collecting accurate and precise location information. The geospatial questions increased the length of the survey and warranted the inclusion of features to simplify it and speed it up. Using nicknames to refer to previously entered geographic locations limited the number of geospatial questions that appeared in the survey and reduced the time taken to complete it. The long-standing relationship between participants and the research team improved comfort to disclose sensitive geographic information related to drug use and sex. Participants in the cognitive interviews highlighted how trust and inclusive and validating language in the survey alleviated concerns related to privacy and representation. Conclusions: This study provides promising results regarding the feasibility of using a web-based mapping survey to collect sensitive location information relevant to ending the HIV epidemic. Data collection at several geographic levels will allow for insights into spatial recall of behaviors as well as future sensitivity analysis of the spatial scale of hotspots and network characteristics. This design also promotes the privacy and comfort of participants who provide location information for sensitive topics. Key considerations for implementing this type of survey include trust from participants, community partners, or research teams to overcome concerns related to privacy and comfort. The implementation of similar surveys should consider local characteristics and knowledge when crafting the geospatial components. UR - https://formative.jmir.org/2023/1/e45188 UR - http://dx.doi.org/10.2196/45188 UR - http://www.ncbi.nlm.nih.gov/pubmed/37347520 ID - info:doi/10.2196/45188 ER - TY - JOUR AU - Gresenz, Roan Carole AU - Singh, Lisa AU - Wang, Yanchen AU - Haber, Jaren AU - Liu, Yaguang PY - 2023/6/13 TI - Development and Assessment of a Social Media?Based Construct of Firearm Ownership: Computational Derivation and Benchmark Comparison JO - J Med Internet Res SP - e45187 VL - 25 KW - criterion validity KW - firearms ownership KW - gun violence KW - machine learning KW - social media data N2 - Background: Gun violence research is characterized by a dearth of data available for measuring key constructs. Social media data may offer a potential opportunity to significantly reduce that gap, but developing methods for deriving firearms-related constructs from social media data and understanding the measurement properties of such constructs are critical precursors to their broader use. Objective: This study aimed to develop a machine learning model of individual-level firearm ownership from social media data and assess the criterion validity of a state-level construct of ownership. Methods: We used survey responses to questions on firearm ownership linked with Twitter data to construct different machine learning models of firearm ownership. We externally validated these models using a set of firearm-related tweets hand-curated from the Twitter Streaming application programming interface and created state-level ownership estimates using a sample of users collected from the Twitter Decahose application programming interface. We assessed the criterion validity of state-level estimates by comparing their geographic variance to benchmark measures from the RAND State-Level Firearm Ownership Database. Results: We found that the logistic regression classifier for gun ownership performs the best with an accuracy of 0.7 and an F1-score of 0.69. We also found a strong positive correlation between Twitter-based estimates of gun ownership and benchmark ownership estimates. For states meeting a threshold requirement of a minimum of 100 labeled Twitter users, the Pearson and Spearman correlation coefficients are 0.63 (P<.001) and 0.64 (P<.001), respectively. Conclusions: Our success in developing a machine learning model of firearm ownership at the individual level with limited training data as well as a state-level construct that achieves a high level of criterion validity underscores the potential of social media data for advancing gun violence research. The ownership construct is an important precursor for understanding the representativeness of and variability in outcomes that have been the focus of social media analyses in gun violence research to date, such as attitudes, opinions, policy stances, sentiments, and perspectives on gun violence and gun policy. The high criterion validity we achieved for state-level gun ownership suggests that social media data may be a useful complement to traditional sources of information on gun ownership such as survey and administrative data, especially for identifying early signals of changes in geographic patterns of gun ownership, given the immediacy of the availability of social media data, their continuous generation, and their responsiveness. These results also lend support to the possibility that other computationally derived, social media?based constructs may be derivable, which could lend additional insight into firearm behaviors that are currently not well understood. More work is needed to develop other firearms-related constructs and to assess their measurement properties. UR - https://www.jmir.org/2023/1/e45187 UR - http://dx.doi.org/10.2196/45187 UR - http://www.ncbi.nlm.nih.gov/pubmed/37310779 ID - info:doi/10.2196/45187 ER - TY - JOUR AU - Hibi, Masanobu AU - Katada, Shun AU - Kawakami, Aya AU - Bito, Kotatsu AU - Ohtsuka, Mayumi AU - Sugitani, Kei AU - Muliandi, Adeline AU - Yamanaka, Nami AU - Hasumura, Takahiro AU - Ando, Yasutoshi AU - Fushimi, Takashi AU - Fujimatsu, Teruhisa AU - Akatsu, Tomoki AU - Kawano, Sawako AU - Kimura, Ren AU - Tsuchiya, Shigeki AU - Yamamoto, Yuuki AU - Haneoka, Mai AU - Kushida, Ken AU - Hideshima, Tomoki AU - Shimizu, Eri AU - Suzuki, Jumpei AU - Kirino, Aya AU - Tsujimura, Hisashi AU - Nakamura, Shun AU - Sakamoto, Takashi AU - Tazoe, Yuki AU - Yabuki, Masayuki AU - Nagase, Shinobu AU - Hirano, Tamaki AU - Fukuda, Reiko AU - Yamashiro, Yukari AU - Nagashima, Yoshinao AU - Ojima, Nobutoshi AU - Sudo, Motoki AU - Oya, Naoki AU - Minegishi, Yoshihiko AU - Misawa, Koichi AU - Charoenphakdee, Nontawat AU - Gao, Zhengyan AU - Hayashi, Kohei AU - Oono, Kenta AU - Sugawara, Yohei AU - Yamaguchi, Shoichiro AU - Ono, Takahiro AU - Maruyama, Hiroshi PY - 2023/6/9 TI - Assessment of Multidimensional Health Care Parameters Among Adults in Japan for Developing a Virtual Human Generative Model: Protocol for a Cross-sectional Study JO - JMIR Res Protoc SP - e47024 VL - 12 KW - bacterial profiles KW - body odor KW - joint probability distribution model KW - chiral amino acid KW - skin surface lipid KW - multidimensional data KW - mobile phone N2 - Background: Human health status can be measured on the basis of many different parameters. Statistical relationships among these different health parameters will enable several possible health care applications and an approximation of the current health status of individuals, which will allow for more personalized and preventive health care by informing the potential risks and developing personalized interventions. Furthermore, a better understanding of the modifiable risk factors related to lifestyle, diet, and physical activity will facilitate the design of optimal treatment approaches for individuals. Objective: This study aims to provide a high-dimensional, cross-sectional data set of comprehensive health care information to construct a combined statistical model as a single joint probability distribution and enable further studies on individual relationships among the multidimensional data obtained. Methods: In this cross-sectional observational study, data were collected from a population of 1000 adult men and women (aged ?20 years) matching the age ratio of the typical adult Japanese population. Data include biochemical and metabolic profiles from blood, urine, saliva, and oral glucose tolerance tests; bacterial profiles from feces, facial skin, scalp skin, and saliva; messenger RNA, proteome, and metabolite analyses of facial and scalp skin surface lipids; lifestyle surveys and questionnaires; physical, motor, cognitive, and vascular function analyses; alopecia analysis; and comprehensive analyses of body odor components. Statistical analyses will be performed in 2 modes: one to train a joint probability distribution by combining a commercially available health care data set containing large amounts of relatively low-dimensional data with the cross-sectional data set described in this paper and another to individually investigate the relationships among the variables obtained in this study. Results: Recruitment for this study started in October 2021 and ended in February 2022, with a total of 997 participants enrolled. The collected data will be used to build a joint probability distribution called a Virtual Human Generative Model. Both the model and the collected data are expected to provide information on the relationships between various health statuses. Conclusions: As different degrees of health status correlations are expected to differentially affect individual health status, this study will contribute to the development of empirically justified interventions based on the population. International Registered Report Identifier (IRRID): DERR1-10.2196/47024 UR - https://www.researchprotocols.org/2023/1/e47024 UR - http://dx.doi.org/10.2196/47024 UR - http://www.ncbi.nlm.nih.gov/pubmed/37294611 ID - info:doi/10.2196/47024 ER - TY - JOUR AU - Guven, Emine PY - 2023/6/6 TI - Decision of the Optimal Rank of a Nonnegative Matrix Factorization Model for Gene Expression Data Sets Utilizing the Unit Invariant Knee Method: Development and Evaluation of the Elbow Method for Rank Selection JO - JMIR Bioinform Biotech SP - e43665 VL - 4 KW - gene expression data KW - nonnegative matrix factorization KW - rank factorization KW - optimal rank KW - unit invariant knee method KW - elbow method KW - consensus matrix N2 - Background: There is a great need to develop a computational approach to analyze and exploit the information contained in gene expression data. The recent utilization of nonnegative matrix factorization (NMF) in computational biology has demonstrated the capability to derive essential details from a high amount of data in particular gene expression microarrays. A common problem in NMF is finding the proper number rank (r) of factors of the degraded demonstration, but no agreement exists on which technique is most appropriate to utilize for this purpose. Thus, various techniques have been suggested to select the optimal value of rank factorization (r). Objective: In this work, a new metric for rank selection is proposed based on the elbow method, which was methodically compared against the cophenetic metric. Methods: To decide the optimum number rank (r), this study focused on the unit invariant knee (UIK) method of the NMF on gene expression data sets. Since the UIK method requires an extremum distance estimator that is eventually employed for inflection and identification of a knee point, the proposed method finds the first inflection point of the curvature of the residual sum of squares of the proposed algorithms using the UIK method on gene expression data sets as a target matrix. Results: Computation was conducted for the UIK task using gene expression data of acute lymphoblastic leukemia and acute myeloid leukemia samples. Consequently, the distinct results of NMF were subjected to comparison on different algorithms. The proposed UIK method is easy to perform, fast, free of a priori rank value input, and does not require initial parameters that significantly influence the model?s functionality. Conclusions: This study demonstrates that the elbow method provides a credible prediction for both gene expression data and for precisely estimating simulated mutational processes data with known dimensions. The proposed UIK method is faster than conventional methods, including metrics utilizing the consensus matrix as a criterion for rank selection, while achieving significantly better computational efficiency without visual inspection on the curvatives. Finally, the suggested rank tuning method based on the elbow method for gene expression data is arguably theoretically superior to the cophenetic measure. UR - https://bioinform.jmir.org/2023/1/e43665 UR - http://dx.doi.org/10.2196/43665 UR - http://www.ncbi.nlm.nih.gov/pubmed/ ID - info:doi/10.2196/43665 ER - TY - JOUR AU - Crocker, Bradley AU - Feng, Olivia AU - Duncan, R. Lindsay PY - 2023/6/2 TI - Performance-Based Measurement of eHealth Literacy: Systematic Scoping Review JO - J Med Internet Res SP - e44602 VL - 25 KW - eHealth literacy KW - measurement KW - performance-based KW - eHealth Literacy Scale KW - eHEALS KW - scoping review KW - review method KW - health literacy KW - library science KW - librarian KW - search strategy N2 - Background: eHealth literacy describes the ability to locate, comprehend, evaluate, and apply web-based health information to a health problem. In studies of eHealth literacy, researchers have primarily assessed participants? perceived eHealth literacy using a short self-report instrument, for which ample research has shown little to no association with actual performed eHealth-related skills. Performance-based measures of eHealth literacy may be more effective at assessing actual eHealth skills, yet such measures seem to be scarcer in the literature. Objective: The primary purpose of this study was to identify tools that currently exist to measure eHealth literacy based on objective performance. A secondary purpose of this study was to characterize the prevalence of performance-based measurement of eHealth literacy in the literature compared with subjective measurement. Methods: We conducted a systematic scoping review of the literature, aligning with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist, in 3 stages: conducting the search, screening articles, and extracting data into a summary table. The summary table includes terminology for eHealth literacy, description of participants, instrument design, health topics used, and a brief note on the evidence of validity for each performance-based measurement tool. A total of 1444 unique articles retrieved from 6 relevant databases (MEDLINE; PsycINFO; CINAHL; Library and Information Science Abstracts [LISA]; Library, Information Science & Technology Abstracts [LISTA]; and Education Resources Information Center [ERIC]) were considered for inclusion, of which 313 (21.68%) included a measure of eHealth literacy. Results: Among the 313 articles that included a measure of eHealth literacy, we identified 33 (10.5%) that reported on 29 unique performance-based eHealth literacy measurement tools. The types of tools ranged from having participants answer health-related questions using the internet, having participants engage in simulated internet tasks, and having participants evaluate website quality to quizzing participants on their knowledge of health and the web-based health information?seeking process. In addition, among the 313 articles, we identified 280 (89.5%) that measured eHealth literacy using only a self-rating tool. Conclusions: This study is the first research synthesis looking specifically at performance-based measures of eHealth literacy and may direct researchers toward existing performance-based measurement tools to be applied in future projects. We discuss some of the key benefits and drawbacks of different approaches to performance-based measurement of eHealth literacy. Researchers with an interest in gauging participants? actual eHealth literacy (as opposed to perceived eHealth literacy) should make efforts to incorporate tools such as those identified in this systematic scoping review. UR - https://www.jmir.org/2023/1/e44602 UR - http://dx.doi.org/10.2196/44602 UR - http://www.ncbi.nlm.nih.gov/pubmed/37266975 ID - info:doi/10.2196/44602 ER - TY - JOUR AU - Kusejko, Katharina AU - Smith, Daniel AU - Scherrer, Alexandra AU - Paioni, Paolo AU - Kohns Vasconcelos, Malte AU - Aebi-Popp, Karoline AU - Kouyos, D. Roger AU - Günthard, F. Huldrych AU - Kahlert, R. Christian AU - PY - 2023/5/31 TI - Migrating a Well-Established Longitudinal Cohort Database From Oracle SQL to Research Electronic Data Entry (REDCap): Data Management Research and Design Study JO - JMIR Form Res SP - e44567 VL - 7 KW - REDCap KW - cohort study KW - data collection KW - electronic case report forms KW - eCRF KW - software KW - digital solution KW - electronic data entry KW - HIV N2 - Background: Providing user-friendly electronic data collection tools for large multicenter studies is key for obtaining high-quality research data. Research Electronic Data Capture (REDCap) is a software solution developed for setting up research databases with integrated graphical user interfaces for electronic data entry. The Swiss Mother and Child HIV Cohort Study (MoCHiV) is a longitudinal cohort study with around 2 million data entries dating back to the early 1980s. Until 2022, data collection in MoCHiV was paper-based. Objective: The objective of this study was to provide a user-friendly graphical interface for electronic data entry for physicians and study nurses reporting MoCHiV data. Methods: MoCHiV collects information on obstetric events among women living with HIV and children born to mothers living with HIV. Until 2022, MoCHiV data were stored in an Oracle SQL relational database. In this project, R and REDCap were used to develop an electronic data entry platform for MoCHiV with migration of already collected data. Results: The key steps for providing an electronic data entry option for MoCHiV were (1) design, (2) data cleaning and formatting, (3) migration and compliance, and (4) add-on features. In the first step, the database structure was defined in REDCap, including the specification of primary and foreign keys, definition of study variables, and the hierarchy of questions (termed ?branching logic?). In the second step, data stored in Oracle were cleaned and formatted to adhere to the defined database structure. Systematic data checks ensured compliance to all branching logic and levels of categorical variables. REDCap-specific variables and numbering of repeated events for enabling a relational data structure in REDCap were generated using R. In the third step, data were imported to REDCap and then systematically compared to the original data. In the last step, add-on features, such as data access groups, redirections, and summary reports, were integrated to facilitate data entry in the multicenter MoCHiV study. Conclusions: By combining different software tools?Oracle SQL, R, and REDCap?and building a systematic pipeline for data cleaning, formatting, and comparing, we were able to migrate a multicenter longitudinal cohort study from Oracle SQL to REDCap. REDCap offers a flexible way for developing customized study designs, even in the case of longitudinal studies with different study arms (ie, obstetric events, women, and mother-child pairs). However, REDCap does not offer built-in tools for preprocessing large data sets before data import. Additional software is needed (eg, R) for data formatting and cleaning to achieve the predefined REDCap data structure. UR - https://formative.jmir.org/2023/1/e44567 UR - http://dx.doi.org/10.2196/44567 UR - http://www.ncbi.nlm.nih.gov/pubmed/37256686 ID - info:doi/10.2196/44567 ER - TY - JOUR AU - Hernandez, Raymond AU - Hoogendoorn, Claire AU - Gonzalez, S. Jeffrey AU - Jin, Haomiao AU - Pyatak, A. Elizabeth AU - Spruijt-Metz, Donna AU - Junghaenel, U. Doerte AU - Lee, Pey-Jiuan AU - Schneider, Stefan PY - 2023/5/30 TI - Reliability and Validity of Noncognitive Ecological Momentary Assessment Survey Response Times as an Indicator of Cognitive Processing Speed in People?s Natural Environment: Intensive Longitudinal Study JO - JMIR Mhealth Uhealth SP - e45203 VL - 11 KW - cognitive performance KW - processing speed KW - ecological momentary assessment KW - ambulatory assessment KW - type 1 diabetes KW - survey response times KW - paradata KW - chronic illness KW - smartphone KW - mobile health KW - mHealth KW - mobile phone N2 - Background: Various populations with chronic conditions are at risk for decreased cognitive performance, making assessment of their cognition important. Formal mobile cognitive assessments measure cognitive performance with greater ecological validity than traditional laboratory-based testing but add to participant task demands. Given that responding to a survey is considered a cognitively demanding task itself, information that is passively collected as a by-product of ecological momentary assessment (EMA) may be a means through which people?s cognitive performance in their natural environment can be estimated when formal ambulatory cognitive assessment is not feasible. We specifically examined whether the item response times (RTs) to EMA questions (eg, mood) can serve as approximations of cognitive processing speed. Objective: This study aims to investigate whether the RTs from noncognitive EMA surveys can serve as approximate indicators of between-person (BP) differences and momentary within-person (WP) variability in cognitive processing speed. Methods: Data from a 2-week EMA study investigating the relationships among glucose, emotion, and functioning in adults with type 1 diabetes were analyzed. Validated mobile cognitive tests assessing processing speed (Symbol Search task) and sustained attention (Go-No Go task) were administered together with noncognitive EMA surveys 5 to 6 times per day via smartphones. Multilevel modeling was used to examine the reliability of EMA RTs, their convergent validity with the Symbol Search task, and their divergent validity with the Go-No Go task. Other tests of the validity of EMA RTs included the examination of their associations with age, depression, fatigue, and the time of day. Results: Overall, in BP analyses, evidence was found supporting the reliability and convergent validity of EMA question RTs from even a single repeatedly administered EMA item as a measure of average processing speed. BP correlations between the Symbol Search task and EMA RTs ranged from 0.43 to 0.58 (P<.001). EMA RTs had significant BP associations with age (P<.001), as expected, but not with depression (P=.20) or average fatigue (P=.18). In WP analyses, the RTs to 16 slider items and all 22 EMA items (including the 16 slider items) had acceptable (>0.70) WP reliability. After correcting for unreliability in multilevel models, EMA RTs from most combinations of items showed moderate WP correlations with the Symbol Search task (ranged from 0.29 to 0.58; P<.001) and demonstrated theoretically expected relationships with momentary fatigue and the time of day. The associations between EMA RTs and the Symbol Search task were greater than those between EMA RTs and the Go-No Go task at both the BP and WP levels, providing evidence of divergent validity. Conclusions: Assessing the RTs to EMA items (eg, mood) may be a method of approximating people?s average levels of and momentary fluctuations in processing speed without adding tasks beyond the survey questions. UR - https://mhealth.jmir.org/2023/1/e45203 UR - http://dx.doi.org/10.2196/45203 UR - http://www.ncbi.nlm.nih.gov/pubmed/37252787 ID - info:doi/10.2196/45203 ER - TY - JOUR AU - Islam, Rezbaul A. B. M. AU - Khan, M. Khalid AU - Scarbrough, Amanda AU - Zimpfer, Jade Mariah AU - Makkena, Navya AU - Omogunwa, Adebola AU - Ahamed, Iqbal Sheikh PY - 2023/5/30 TI - An Artificial Intelligence?Based Smartphone App for Assessing the Risk of Opioid Misuse in Working Populations Using Synthetic Data: Pilot Development Study JO - JMIR Form Res SP - e45434 VL - 7 KW - opioid overused disorder KW - OUD KW - mobile health KW - mHealth KW - artificial intelligence KW - smartphone app KW - opioids KW - application KW - caregivers KW - mobile app N2 - Background: Opioid use disorder (OUD) is an addiction crisis in the United States. As recent as 2019, more than 10 million people have misused or abused prescription opioids, making OUD one of the leading causes of accidental death in the United States. Workforces that are physically demanding and laborious in the transportation, construction and extraction, and health care industries are prime targets for OUD due to high-risk occupational activities. Because of this high prevalence of OUD among working populations in the United States, elevated workers? compensation and health insurance costs, absenteeism, and declined productivity in workplaces have been reported. Objective: With the emergence of new smartphone technologies, health interventions can be widely used outside clinical settings via mobile health tools. The major objective of our pilot study was to develop a smartphone app that can track work-related risk factors leading to OUD with a specific focus on high-risk occupational groups. We used synthetic data analyzed by applying a machine learning algorithm to accomplish our objective. Methods: To make the OUD assessment process more convenient and to motivate potential patients with OUD, we developed a smartphone-based app through a step-by-step process. First, an extensive literature survey was conducted to list a set of critical risk assessment questions that can capture high-risk behaviors leading to OUD. Next, a review panel short-listed 15 questions after careful evaluation with specific emphasis on physically demanding workforces?9 questions had two, 5 questions had five, and 1 question had three response options. Instead of human participant data, synthetic data were used as user responses. Finally, an artificial intelligence algorithm, naive Bayes, was used to predict the OUD risk, trained with the synthetic data collected. Results: The smartphone app we have developed is functional as tested with synthetic data. Using the naive Bayes algorithm on collected synthetic data, we successfully predicted the risk of OUD. This would eventually create a platform to test the functionality of the app further using human participant data. Conclusions: The use of mobile health techniques, such as our mobile app, is highly promising in predicting and offering mitigation plans for disease detection and prevention. Using a naive Bayes algorithm model along with a representational state transfer (REST) application programming interface and cloud-based data encryption storage, respondents can guarantee their privacy and accuracy in estimating their risk. Our app offers a tailored mitigation strategy for specific workforces (eg, transportation and health care workers) that are most impacted by OUD. Despite the limitations of the study, we have developed a robust methodology and believe that our app has the potential to help reduce the opioid crisis. UR - https://formative.jmir.org/2023/1/e45434 UR - http://dx.doi.org/10.2196/45434 UR - http://www.ncbi.nlm.nih.gov/pubmed/37252763 ID - info:doi/10.2196/45434 ER - TY - JOUR AU - Darnell, Doyanne AU - Ranna-Stewart, Minu AU - Psaros, Christina AU - Filipowicz, R. Teresa AU - Grimes, LaKendra AU - Henderson, Savannah AU - Parman, Mariel AU - Gaddis, Kathy AU - Gaynes, Neil Bradley AU - Mugavero, J. Michael AU - Dorsey, Shannon AU - Pence, W. Brian PY - 2023/5/30 TI - Using Principles of an Adaptation Framework to Adapt a Transdiagnostic Psychotherapy for People With HIV to Improve Mental Health and HIV Treatment Engagement: Focus Groups and Formative Research Study JO - JMIR Form Res SP - e45106 VL - 7 KW - adaptation KW - transdiagnostic psychotherapy KW - people with HIV KW - trauma KW - comorbidity N2 - Background: HIV treatment engagement is critical for people with HIV; however, behavioral health comorbidities and HIV-related stigma are key barriers to engagement. Treatments that address these barriers and can be readily implemented in HIV care settings are needed. Objective: We presented the process for adapting transdiagnostic cognitive behavioral psychotherapy, the Common Elements Treatment Approach (CETA), for people with HIV receiving HIV treatment at a Southern US HIV clinic. Behavioral health targets included posttraumatic stress, depression, anxiety, substance use, and safety concerns (eg, suicidality). The adaptation also included ways to address HIV-related stigma and a component based on Life-Steps, a brief cognitive behavioral intervention to support patient HIV treatment engagement. Methods: We applied principles of the Assessment, Decision, Administration, Production, Topical Experts, Integration, Training, Testing model, a framework for adapting evidence-based HIV interventions, and described our adaptation process, which included adapting the CETA manual based on expert input; conducting 3 focus groups, one with clinic social workers (n=3) and 2 with male (n=3) and female (n=4) patients to obtain stakeholder input for the adapted therapy; revising the manual according to this input; and training 2 counselors on the adapted protocol, including a workshop held over the internet followed by implementing the therapy with 3 clinic patients and receiving case-based consultation for them. For the focus groups, all clinic social workers were invited to participate, and patients were referred by clinic social workers if they were adults receiving services at the clinic and willing to provide written informed consent. Social worker focus group questions elicited reactions to the adapted therapy manual and content. Patient focus group questions elicited experiences with behavioral health conditions and HIV-related stigma and their impacts on HIV treatment engagement. Transcripts were reviewed by 3 team members to catalog participant commentary according to themes relevant to adapting CETA for people with HIV. Coauthors independently identified themes and met to discuss and reach a consensus on them. Results: We successfully used principles of the Assessment, Decision, Administration, Production, Topical Experts, Integration, Training, Testing framework to adapt CETA for people with HIV. The focus group with social workers indicated that the adapted therapy made conceptual sense and addressed common behavioral health concerns and practical and cognitive behavioral barriers to HIV treatment engagement. Key considerations for CETA for people with HIV obtained from social worker and patient focus groups were related to stigma, socioeconomic stress, and instability experienced by the clinic population and some patients? substance use, which can thwart the stability needed to engage in care. Conclusions: The resulting brief, manualized therapy is designed to help patients build skills that promote HIV treatment engagement and reduce symptoms of common behavioral health conditions that are known to thwart HIV treatment engagement. UR - https://formative.jmir.org/2023/1/e45106 UR - http://dx.doi.org/10.2196/45106 UR - http://www.ncbi.nlm.nih.gov/pubmed/37252786 ID - info:doi/10.2196/45106 ER - TY - JOUR AU - Kumwichar, Ponlagrit PY - 2023/5/29 TI - Enhancing Learning About Epidemiological Data Analysis Using R for Graduate Students in Medical Fields With Jupyter Notebook: Classroom Action Research JO - JMIR Med Educ SP - e47394 VL - 9 KW - learning KW - Jupyter KW - R KW - epidemiology KW - data analysis KW - medical education KW - graduate student KW - longitudinal data analysis KW - graduate education KW - implementation N2 - Background: Graduate students in medical fields must learn about epidemiology and data analysis to conduct their research. R is a software environment used to develop and run packages for statistical analysis; it can be challenging for students to learn because of compatibility with their computers and problems with package installations. Jupyter Notebook was used to run R, which enhanced the graduate students? ability to learn epidemiological data analysis by providing an interactive and collaborative environment that allows for more efficient and effective learning. Objective: This study collected class reflections from students and their lecturer in the class ?Longitudinal Data Analysis Using R,? identified problems that occurred, and illustrated how Jupyter Notebook can solve those problems. Methods: The researcher analyzed issues encountered in the previous class and devised solutions using Jupyter Notebook. These solutions were then implemented and applied to a new group of students. Reflections from the students were regularly collected and documented in an electronic form. The comments were then thematically analyzed and compared to those of the prior cohort. Results: Improvements that were identified included the ease of using Jupyter R for data analysis without needing to install packages, increased student questioning due to curiosity, and students having the ability to immediately use all code functions. After using Jupyter Notebook, the lecturer could stimulate interest more effectively and challenge students. Furthermore, they highlighted that students responded to questions. The student feedback shows that learning R with Jupyter Notebook was effective in stimulating their interest. Based on the feedback received, it can be inferred that using Jupyter Notebook to learn R is an effective approach for equipping students with an all-encompassing comprehension of longitudinal data analysis. Conclusions: The use of Jupyter Notebook can improve graduate students? learning experience for epidemiological data analysis by providing an interactive and collaborative environment that is not affected by compatibility issues with different operating systems and computers. UR - https://mededu.jmir.org/2023/1/e47394 UR - http://dx.doi.org/10.2196/47394 UR - http://www.ncbi.nlm.nih.gov/pubmed/37247206 ID - info:doi/10.2196/47394 ER - TY - JOUR AU - Shan, Yi AU - Ji, Meng AU - Xing, Zhaoquan AU - Dong, Zhaogang PY - 2023/5/24 TI - Factors Associated With Limited Cancer Health Literacy Among Chinese People: Cross-sectional Survey Study JO - JMIR Form Res SP - e42666 VL - 7 KW - factor KW - limited cancer health literacy KW - Chinese people KW - logistic regression N2 - Background: Limited cancer health literacy may be attributed to various factors. Although these factors play decisive roles in identifying individuals with limited cancer health literacy, they have not been sufficiently investigated, especially in China. There is a pressing need to ascertain the factors that effectively identify Chinese people with poor cancer health literacy. Objective: This study aimed to identify the factor associated with limited cancer health literacy among Chinese people based on the 6-Item Cancer Health Literacy Test (CHLT-6). Methods: We first categorized Chinese study participants according to the answers provided for cancer health literacy as follows: people who provided ?3 correct answers were labeled as having limited cancer health literacy, whereas those who provided between 4 and 6 correct answers were labeled as having adequate cancer health literacy. We then adopted logistic regression to analyze the factors that were closely related to limited cancer health literacy among at-risk study participants. Results: The logistic regression analysis identified the following factors that effectively predicted limited cancer health literacy: (1) male gender, (2) low education attainment, (3) age, (4) high levels of self-assessed general disease knowledge, (5) low levels of digital health literacy, (6) limited communicative health literacy, (7) low general health numeracy, and (8) high levels of mistrust in health authorities. Conclusions: Using regression analysis, we successfully identified 8 factors that could be used as predictors of limited cancer health literacy among Chinese populations. These findings have important clinical implications for supporting Chinese people with limited cancer health literacy through the development of more targeted health educational programs and resources that better align with their actual skill levels. UR - https://formative.jmir.org/2023/1/e42666 UR - http://dx.doi.org/10.2196/42666 UR - http://www.ncbi.nlm.nih.gov/pubmed/37223982 ID - info:doi/10.2196/42666 ER - TY - JOUR AU - Kanzow, Friederike Amelie AU - Schmidt, Dennis AU - Kanzow, Philipp PY - 2023/5/19 TI - Scoring Single-Response Multiple-Choice Items: Scoping Review and Comparison of Different Scoring Methods JO - JMIR Med Educ SP - e44084 VL - 9 KW - alternate-choice KW - best-answer KW - education KW - education system KW - educational assessment KW - educational measurement KW - examination KW - multiple choice KW - results KW - scoring KW - scoring system KW - single choice KW - single response KW - scoping review KW - test KW - testing KW - true/false KW - true-false KW - Type A N2 - Background: Single-choice items (eg, best-answer items, alternate-choice items, single true-false items) are 1 type of multiple-choice items and have been used in examinations for over 100 years. At the end of every examination, the examinees? responses have to be analyzed and scored to derive information about examinees? true knowledge. Objective: The aim of this paper is to compile scoring methods for individual single-choice items described in the literature. Furthermore, the metric expected chance score and the relation between examinees? true knowledge and expected scoring results (averaged percentage score) are analyzed. Besides, implications for potential pass marks to be used in examinations to test examinees for a predefined level of true knowledge are derived. Methods: Scoring methods for individual single-choice items were extracted from various databases (ERIC, PsycInfo, Embase via Ovid, MEDLINE via PubMed) in September 2020. Eligible sources reported on scoring methods for individual single-choice items in written examinations including but not limited to medical education. Separately for items with n=2 answer options (eg, alternate-choice items, single true-false items) and best-answer items with n=5 answer options (eg, Type A items) and for each identified scoring method, the metric expected chance score and the expected scoring results as a function of examinees? true knowledge using fictitious examinations with 100 single-choice items were calculated. Results: A total of 21 different scoring methods were identified from the 258 included sources, with varying consideration of correctly marked, omitted, and incorrectly marked items. Resulting credit varied between ?3 and +1 credit points per item. For items with n=2 answer options, expected chance scores from random guessing ranged between ?1 and +0.75 credit points. For items with n=5 answer options, expected chance scores ranged between ?2.2 and +0.84 credit points. All scoring methods showed a linear relation between examinees? true knowledge and the expected scoring results. Depending on the scoring method used, examination results differed considerably: Expected scoring results from examinees with 50% true knowledge ranged between 0.0% (95% CI 0% to 0%) and 87.5% (95% CI 81.0% to 94.0%) for items with n=2 and between ?60.0% (95% CI ?60% to ?60%) and 92.0% (95% CI 86.7% to 97.3%) for items with n=5. Conclusions: In examinations with single-choice items, the scoring result is not always equivalent to examinees? true knowledge. When interpreting examination scores and setting pass marks, the number of answer options per item must usually be taken into account in addition to the scoring method used. UR - https://mededu.jmir.org/2023/1/e44084 UR - http://dx.doi.org/10.2196/44084 UR - http://www.ncbi.nlm.nih.gov/pubmed/37001510 ID - info:doi/10.2196/44084 ER - TY - JOUR AU - Shan, Yi AU - Ji, Meng AU - Dong, Zhaogang AU - Xing, Zhaoquan AU - Wang, Ding AU - Cao, Xiangting PY - 2023/5/18 TI - The Chinese Version of the Patient Education Materials Assessment Tool for Printable Materials: Translation, Adaptation, and Validation Study JO - J Med Internet Res SP - e39808 VL - 25 KW - actionability KW - adaptation KW - Chinese version of the PEMAT-P KW - comprehensibility KW - health education materials translation KW - validation N2 - Background: Providing people with understandable and actionable health information can considerably promote healthy behaviors and outcomes. To this end, some valid and reliable scales assessing the patient-friendliness of health education materials, like the PEMAT-P (Patient Education Materials Assessment Tool for printable materials), have been well developed in English-speaking countries. However, the English version of the PEMAT-P has not been translated and adapted into simplified Chinese and validated in mainland China. Objective: This study sought to translate the PEMAT-P tool into a simplified Chinese (Mandarin) version (C-PEMAT-P, a Chinese version of the Patient Education Materials Assessment Tool for printable materials) and verify its validity and reliability for assessing the comprehensibility and actionability of health education resources written in simplified Chinese. As a result, the validated C-PEMAT-P could be used to guide health researchers and educators to design more comprehensible and actionable materials for more tailored and targeted health education and interventions. Methods: We translated the PEMAT-P into simplified Chinese in the following three steps: (1) forward-translating the PEMAT-P into simplified Chinese, (2) back-translating the simplified Chinese version into English, and (3) testing translation equivalence linguistically and culturally by examining the original English version of the PEMAT-P and the back-translated English version of the tool. Any discrepancies between the original English tool and the back-translated English tool were resolved through a panel discussion among the research team of all authors to produce a revised forward-translated Chinese version (C-PEMAT-P). We then evaluated the clarity of construction and wording as well as the content relevance of the C-PEMAT-P using a 4-point ordinal scale to determine its content validity. After that, 2 native Chinese speakers (health educators) used the C-PEMAT-P to rate 15 health education handouts concerning air pollution and health to validate their reliability. We calculated the Cohen coefficient and Cronbach ? to determine the interrater agreement and internal consistency of the C-PEMAT-P, respectively. Results: We finalized the translated Chinese tool after discussing the differences between the 2 English versions (original and back-translated) of the PEMAT-P, producing the final Chinese version of the PEMAT-P (C-PEMAT-P). The content validity index of the C-PEMAT-P version was 0.969, the Cohen coefficient for the interrater scoring agreement was 0.928, and the Cronbach ? for internal consistency was .897. These values indicated the high validity and reliability of the C-PEMAT-P. Conclusions: The C-PEMAT-P has been proven valid and reliable. It is the first Chinese scale for assessing the comprehensibility and actionability of Chinese health education materials. It can be used as an assessment tool to evaluate health education materials currently available and a guide to help health researchers and educators design more comprehensible and actionable materials for more tailored and targeted health education and interventions. UR - https://www.jmir.org/2023/1/e39808 UR - http://dx.doi.org/10.2196/39808 UR - http://www.ncbi.nlm.nih.gov/pubmed/37200085 ID - info:doi/10.2196/39808 ER - TY - JOUR AU - Bojic, Iva AU - Mammadova, Maleyka AU - Ang, Chin-Siang AU - Teo, Lung Wei AU - Diordieva, Cristina AU - Pienkowska, Anita AU - Ga?evi?, Dragan AU - Car, Josip PY - 2023/5/17 TI - Empowering Health Care Education Through Learning Analytics: In-depth Scoping Review JO - J Med Internet Res SP - e41671 VL - 25 KW - distance education and web-based learning KW - distributed learning environments KW - data science applications in education KW - 21st century abilities KW - cooperative and collaborative learning KW - COVID-19 KW - education KW - digital KW - data KW - student N2 - Background: Digital education has expanded since the COVID-19 pandemic began. A substantial amount of recent data on how students learn has become available for learning analytics (LA). LA denotes the ?measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.? Objective: This scoping review aimed to examine the use of LA in health care professions education and propose a framework for the LA life cycle. Methods: We performed a comprehensive literature search of 10 databases: MEDLINE, Embase, Web of Science, ERIC, Cochrane Library, PsycINFO, CINAHL, ICTP, Scopus, and IEEE Explore. In total, 6 reviewers worked in pairs and performed title, abstract, and full-text screening. We resolved disagreements on study selection by consensus and discussion with other reviewers. We included papers if they met the following criteria: papers on health care professions education, papers on digital education, and papers that collected LA data from any type of digital education platform. Results: We retrieved 1238 papers, of which 65 met the inclusion criteria. From those papers, we extracted some typical characteristics of the LA process and proposed a framework for the LA life cycle, including digital education content creation, data collection, data analytics, and the purposes of LA. Assignment materials were the most popular type of digital education content (47/65, 72%), whereas the most commonly collected data types were the number of connections to the learning materials (53/65, 82%). Descriptive statistics was mostly used in data analytics in 89% (58/65) of studies. Finally, among the purposes for LA, understanding learners? interactions with the digital education platform was cited most often in 86% (56/65) of papers and understanding the relationship between interactions and student performance was cited in 63% (41/65) of papers. Far less common were the purposes of optimizing learning: the provision of at-risk intervention, feedback, and adaptive learning was found in 11, 5, and 3 papers, respectively. Conclusions: We identified gaps for each of the 4 components of the LA life cycle, with the lack of an iterative approach while designing courses for health care professions being the most prevalent. We identified only 1 instance in which the authors used knowledge from a previous course to improve the next course. Only 2 studies reported that LA was used to detect at-risk students during the course?s run, compared with the overwhelming majority of other studies in which data analysis was performed only after the course was completed. UR - https://www.jmir.org/2023/1/e41671 UR - http://dx.doi.org/10.2196/41671 UR - http://www.ncbi.nlm.nih.gov/pubmed/37195746 ID - info:doi/10.2196/41671 ER - TY - JOUR AU - Peretz, Gal AU - Taylor, Barr C. AU - Ruzek, I. Josef AU - Jefroykin, Samuel AU - Sadeh-Sharvit, Shiri PY - 2023/5/15 TI - Machine Learning Model to Predict Assignment of Therapy Homework in Behavioral Treatments: Algorithm Development and Validation JO - JMIR Form Res SP - e45156 VL - 7 KW - deep learning KW - empirically-based practice KW - natural language processing KW - behavioral treatment KW - machine learning KW - homework KW - treatment fidelity KW - artificial intelligence KW - intervention KW - therapy KW - mental health KW - mHealth N2 - Background: Therapeutic homework is a core element of cognitive and behavioral interventions, and greater homework compliance predicts improved treatment outcomes. To date, research in this area has relied mostly on therapists? and clients? self-reports or studies carried out in academic settings, and there is little knowledge on how homework is used as a treatment intervention in routine clinical care. Objective: This study tested whether a machine learning (ML) model using natural language processing could identify homework assignments in behavioral health sessions. By leveraging this technology, we sought to develop a more objective and accurate method for detecting the presence of homework in therapy sessions. Methods: We analyzed 34,497 audio-recorded treatment sessions provided in 8 behavioral health care programs via an artificial intelligence (AI) platform designed for therapy provided by Eleos Health. Therapist and client utterances were captured and analyzed via the AI platform. Experts reviewed the homework assigned in 100 sessions to create classifications. Next, we sampled 4000 sessions and labeled therapist-client microdialogues that suggested homework to train an unsupervised sentence embedding model. This model was trained on 2.83 million therapist-client microdialogues. Results: An analysis of 100 random sessions found that homework was assigned in 61% (n=61) of sessions, and in 34% (n=21) of these cases, more than one homework assignment was provided. Homework addressed practicing skills (n=34, 37%), taking action (n=26, 28.5%), journaling (n=17, 19%), and learning new skills (n=14, 15%). Our classifier reached a 72% F1-score, outperforming state-of-the-art ML models. The therapists reviewing the microdialogues agreed in 90% (n=90) of cases on whether or not homework was assigned. Conclusions: The findings of this study demonstrate the potential of ML and natural language processing to improve the detection of therapeutic homework assignments in behavioral health sessions. Our findings highlight the importance of accurately capturing homework in real-world settings and the potential for AI to support therapists in providing evidence-based care and increasing fidelity with science-backed interventions. By identifying areas where AI can facilitate homework assignments and tracking, such as reminding therapists to prescribe homework and reducing the charting associated with homework, we can ultimately improve the overall quality of behavioral health care. Additionally, our approach can be extended to investigate the impact of homework assignments on therapeutic outcomes, providing insights into the effectiveness of specific types of homework. UR - https://formative.jmir.org/2023/1/e45156 UR - http://dx.doi.org/10.2196/45156 UR - http://www.ncbi.nlm.nih.gov/pubmed/37184927 ID - info:doi/10.2196/45156 ER - TY - JOUR AU - Elvsaas, Orjasaeter Ida-Kristin AU - Garnweidner-Holme, Lisa AU - Habib, Laurence AU - Molin, Marianne PY - 2023/5/11 TI - Development and Evaluation of a Serious Game Application to Engage University Students in Critical Thinking About Health Claims: Mixed Methods Study JO - JMIR Form Res SP - e44831 VL - 7 KW - game application KW - critical thinking KW - critical health literacy KW - Informed Health Choices KW - evaluation KW - mixed methods study KW - serious games KW - behind the headlines KW - mobile phone N2 - Background: Misleading health claims are widespread in the media, and making choices based on such claims can negatively affect health. Thus, developing effective learning resources to enable people to think critically about health claims is of great value. Serious games can become an effective learning resource in this respect, as they can affect motivation and learning. Objective: This study aims to document how user insights and input can inform the concept and development of a serious game application in critical thinking about health claims in addition to gathering user experiences with the game application. Methods: This was a mixed methods study in 4 successive phases with both qualitative and quantitative data collected in the period from 2020-2022. Qualitative data on design and development were obtained from 4 unrecorded discussions, and qualitative evaluation data were obtained from 1 recorded focus group interview and 3 open-ended questions in the game application. The quantitative data originate from user statistics. The qualitative data were analyzed thematically, and user data were analyzed using nonparametric tests. Results: The first unrecorded discussion revealed that the students? (3 participants?) assessment of whether a claim was reliable or not was limited to performing Google searches when faced with an ad for a health intervention. On the basis of the acquired knowledge of the target group, the game?s prerequisites, and the technical possibilities, a pilot of the game was created and reviewed question by question in 3 unrecorded discussions (6 participants). After adjustments, the game was advertised at the Oslo Metropolitan University, and 193 students tested the game. A correlation (r=0.77; P<.001) was found between the number of replays and total points achieved in the game. There was no demonstrable difference (P=.07) between the total scores of students from different faculties. Overall, 36.3% (70/193) of the students answered the evaluation questions in the game. They used words such as ?fun? and ?educational? about the experiences with the game, and words such as ?motivating? and ?engaging? related to the learning experience. The design was described as ?varied? and ?user-friendly.? Suggested improvements include adding references, more games and modules, more difficult questions, and an introductory text explaining the game. The results from the focus group interview (4 participants) corresponded to a large extent with the results of the open-ended questions in the game. Conclusions: We found that user insights and inputs can be successfully used in the concept and development of a serious game that aims to engage students to think critically about health claims. The mixed methods evaluation revealed that the users experienced the game as educational and fun. Future research may focus on assessing the effect of the serious game on learning outcomes and health choices in randomized trials. UR - https://formative.jmir.org/2023/1/e44831 UR - http://dx.doi.org/10.2196/44831 UR - http://www.ncbi.nlm.nih.gov/pubmed/37166972 ID - info:doi/10.2196/44831 ER - TY - JOUR AU - Dechsling, Anders AU - Cogo-Moreira, Hugo AU - Gangestad, Spydevold Jonathan AU - Johannessen, Nettum Sandra AU - Nordahl-Hansen, Anders PY - 2023/5/11 TI - Evaluating the Feasibility of Emotion Expressions in Avatars Created From Real Person Photos: Pilot Web-Based Survey of Virtual Reality Software JO - JMIR Form Res SP - e44632 VL - 7 KW - avatar KW - emotion recognition KW - emotion KW - face KW - facial expression KW - facial KW - images KW - real images KW - software KW - virtual reality N2 - Background: The availability and potential of virtual reality (VR) has led to an increase of its application. VR is suggested to be helpful in training elements of social competence but with an emphasis on interventions being tailored. Recognizing facial expressions is an important social skill and thus a target for training. Using VR in training these skills could have advantages over desktop alternatives. Children with autism, for instance, appear to prefer avatars over real images when assessing facial expressions. Available software provides the opportunity to transform profile pictures into avatars, thereby giving the possibility of tailoring according to an individual?s own environment. However, the emotions provided by such software should be validated before application. Objective: Our aim was to investigate whether available software is a quick, easy, and viable way of providing emotion expressions in avatars transformed from real images. Methods: A total of 401 participants from a general population completed a survey on the web containing 27 different images of avatars transformed, using a software, from real images. We calculated the reliability of each image and their level of difficulty using a structural equation modeling approach. We used Bayesian confirmatory factor analysis testing under a multidimensional first-order correlated factor structure where faces showing the same emotions represented a latent variable. Results: Few emotions were correctly perceived and rated as higher than other emotions. The factor loadings indicating the discrimination of the image were around 0.7, which means 49% shared variance with the latent factor that the face is linked with. The standardized thresholds indicating the difficulty level of the images are mostly around average, and the highest correlation is between faces showing happiness and anger. Conclusions: Only using a software to transform profile pictures to avatars is not sufficient to provide valid emotion expressions. Adjustments are needed to increase faces? discrimination (eg, increasing reliabilities). The faces showed average levels of difficulty, meaning that they are neither very difficult nor very easy to perceive, which fits a general population. Adjustments should be made for specific populations and when applying this technology in clinical practice. UR - https://formative.jmir.org/2023/1/e44632 UR - http://dx.doi.org/10.2196/44632 UR - http://www.ncbi.nlm.nih.gov/pubmed/37166970 ID - info:doi/10.2196/44632 ER - TY - JOUR AU - Ataya, Jawdat AU - Jamous, Issam AU - Dashash, Mayssoon PY - 2023/5/2 TI - Measurement of Humanity Among Health Professionals: Development and Validation of the Medical Humanity Scale Using the Delphi Method JO - JMIR Form Res SP - e44241 VL - 7 KW - medical humanity KW - Medical Humanitarian Scale KW - scale KW - humanity KW - humanitarian KW - humane KW - Hippocratic oath KW - Delphi KW - development KW - patient centered KW - compassion KW - ethic KW - empathy KW - empathetic KW - validity KW - validation KW - person centered KW - the humanitarian aspect KW - students of medical colleges KW - Syria N2 - Background: Despite the importance of humanism in providing health care, there is a lack of valid and reliable tool for assessing humanity among health professionals. Objective: The aim of this study was to design a new humanism scale and to assess the validity of this scale in measuring humanism among Syrian health professional students. Methods: The Medical Humanity Scale (MHS) was designed. It consists of 27 items categorized into 7 human values including patient-oriented care, respect, empathy, ethics, altruism, and compassion. The scale was tested for internal consistency and reliability using Cronbach ? and test-retest methods. The construct validity of the scale was also tested to assess the ability of the scale in differentiating between groups of health professional students with different levels of medical humanity. A 7-point Likert scale was adopted. The study included 300 participants including 97 medical, 78 dental, 82 pharmacy, and 43 preparatory-year students from Syrian universities. The Delphi method was used and factors analysis was performed. Bartlett test of sphericity and the Kaiser-Meyer-Olkin measure of sample adequacy were used. The number of components was extracted using principal component analysis. Results: The mean score of the MHS was 158.7 (SD 11.4). The MHS mean score of female participants was significantly higher than the mean score of male participants (159.59, SD 10.21 vs 155.48, SD 14.35; P=.008). The MHS mean score was significantly lower in dental students (154.12, SD 1.45; P=.005) than the mean scores of medical students (159.77, SD 1.02), pharmacy students (161.40, SD 1.05), and preparatory-year students (159.05, SD 1.94). However, no significant relationship was found between humanism and academic year (P=.32), university type (P=.34), marital status (P=.64), or financial situation (P=.16). The Kaiser-Meyer-Olkin test (0.730) and Bartlett test of sphericity (1201.611, df=351; P=.01) were performed. Factor analysis indicated that the proportion of variables between the first and second factors was greater than 10%, confirming that the scale was a single group. The Cronbach ? for the overall scale was 0.735, indicating that the scale had acceptable reliability and validity. Conclusions: The results of this study suggest that the MHS is a reliable and valid tool for measuring humanity among health professional students and the development of patient-centered care. UR - https://formative.jmir.org/2023/1/e44241 UR - http://dx.doi.org/10.2196/44241 UR - http://www.ncbi.nlm.nih.gov/pubmed/37129940 ID - info:doi/10.2196/44241 ER - TY - JOUR AU - Liyanage, Ravihari Chandreen AU - Mago, Vijay AU - Schiff, Rebecca AU - Ranta, Ken AU - Park, Aaron AU - Lovato-Day, Kristyn AU - Agnor, Elise AU - Gokani, Ravi PY - 2023/5/2 TI - Understanding Why Many People Experiencing Homelessness Reported Migrating to a Small Canadian City: Machine Learning Approach With Augmented Data JO - JMIR Form Res SP - e43511 VL - 7 KW - data augmentation KW - feature selection KW - homelessness KW - machine learning KW - migrants N2 - Background: Over the past years, homelessness has become a substantial issue around the globe. The largest social services organization in Thunder Bay, Ontario, Canada, has observed that a majority of the people experiencing homelessness in the city were from outside of the city or province. Thus, to improve programming and resource allocation for people experiencing homelessness in the city, including shelter use, it was important to investigate the trends associated with homelessness and migration. Objective: This study aimed to address 3 research questions related to homelessness and migration in Thunder Bay: What factors predict whether a person who migrated to the city and is experiencing homelessness stays or leaves shelters? If an individual stays, how long are they likely to stay? What factors predict stay duration? Methods: We collected the required data from 2 sources: a survey conducted with people experiencing homelessness at 3 homeless shelters in Thunder Bay and the database of a homeless information management system. The records of 110 migrants were used for the analysis. Two feature selection techniques were used to address the first and third research questions, and 8 machine learning models were used to address the second research question. In addition, data augmentation was performed to improve the size of the data set and to resolve the class imbalance problem. The area under the receiver operating characteristic curve value and cross-validation accuracy were used to measure the models? performances while avoiding possible model overfitting. Results: Factors predicting an individual?s stay duration included home or previous district, highest educational qualification, recent receipt of mental health support, migrating to visit family or friends, and finding employment upon arrival. For research question 2, among the classification models developed for predicting the stay duration of migrants, the random forest and gradient boosting tree models presented better results with area under the receiver operating characteristic curve values of 0.91 and 0.93, respectively. Finally, home district, band membership, status card, previous district, and recent support for drug and/or alcohol use were recognized as the factors predicting stay duration. Conclusions: Applying machine learning enables researchers to make predictions related to migrants? homelessness and investigate how various factors become determinants of the predictions. We hope that the findings of this study will aid future policy making and resource allocation to better serve people experiencing homelessness. However, further improvements in the data set size and interpretation of the identified factors in decision-making are required. UR - https://formative.jmir.org/2023/1/e43511 UR - http://dx.doi.org/10.2196/43511 UR - http://www.ncbi.nlm.nih.gov/pubmed/37129936 ID - info:doi/10.2196/43511 ER - TY - JOUR AU - Wen, Bingyang AU - Wang, Ning AU - Subbalakshmi, Koduvayur AU - Chandramouli, Rajarathnam PY - 2023/5/2 TI - Revealing the Roles of Part-of-Speech Taggers in Alzheimer Disease Detection: Scientific Discovery Using One-Intervention Causal Explanation JO - JMIR Form Res SP - e36590 VL - 7 KW - explainable machine learning KW - Alzheimer disease KW - natural language processing KW - causal inference N2 - Background: Recently, rich computational methods that use deep learning or machine learning have been developed using linguistic biomarkers for the diagnosis of early-stage Alzheimer disease (AD). Moreover, some qualitative and quantitative studies have indicated that certain part-of-speech (PoS) features or tags could be good indicators of AD. However, there has not been a systematic attempt to discover the underlying relationships between PoS features and AD. Moreover, there has not been any attempt to quantify the relative importance of PoS features in detecting AD. Objective: Our goal was to disclose the underlying relationship between PoS features and AD, understand whether PoS features are useful in AD diagnosis, and explore which PoS features play a vital role in the diagnosis. Methods: The DementiaBank, containing 1049 transcripts from 208 patients with AD and 243 transcripts from 104 older control individuals, was used. A total of 27 PoS features were extracted from each record. Then, the relationship between AD and each of the PoS features was explored. A transformer-based deep learning model for AD prediction using PoS features was trained. Then, a global explainable artificial intelligence method was proposed and used to discover which PoS features were the most important in AD diagnosis using the transformer-based predictor. A global (model-level) feature importance measure was derived as a summary from the local (example-level) feature importance metric, which was obtained using the proposed causally aware counterfactual explanation method. The unique feature of this method is that it considers causal relations among PoS features and can, hence, preclude counterfactuals that are improbable and result in more reliable explanations. Results: The deep learning?based AD predictor achieved an accuracy of 92.2% and an F1-score of 0.955 when distinguishing patients with AD from healthy controls. The proposed explanation method identified 12 PoS features as being important for distinguishing patients with AD from healthy controls. Of these 12 features, 3 (25%) have been identified by other researchers in previous works in psychology and natural language processing. The remaining 75% (9/12) of PoS features have not been previously identified. We believe that this is an interesting finding that can be used in creating tests that might aid in the diagnosis of AD. Note that although our method is focused on PoS features, it should be possible to extend it to more types of features, perhaps even those derived from other biomarkers, such as syntactic features. Conclusions: The high classification accuracy of the proposed deep learner indicates that PoS features are strong clues in AD diagnosis. There are 12 PoS features that are strongly tied to AD, and because language is a noninvasive and potentially cheap method for detecting AD, this work shows some promising directions in this field. UR - https://formative.jmir.org/2023/1/e36590 UR - http://dx.doi.org/10.2196/36590 UR - http://www.ncbi.nlm.nih.gov/pubmed/37129944 ID - info:doi/10.2196/36590 ER - TY - JOUR AU - Shiffman, Saul AU - McCaffrey, A. Stacey AU - Hannon, J. Michael AU - Goldenson, I. Nicholas AU - Black, A. Ryan PY - 2023/4/14 TI - A New Questionnaire to Assess Respiratory Symptoms (The Respiratory Symptom Experience Scale): Quantitative Psychometric Assessment and Validation Study JO - JMIR Form Res SP - e44036 VL - 7 KW - measure development KW - respiratory symptoms KW - COPD KW - e-cigarettes KW - electronic nicotine delivery system KW - ENDS KW - smoking KW - respiratory disease KW - respiratory health KW - health intervention KW - questionnaire KW - validation KW - validate KW - development KW - respiratory KW - pulmonary KW - lung KW - smoker N2 - Background: Smokers often experience respiratory symptoms (eg, morning cough), and those who stop smoking, including those who do so by switching completely to electronic nicotine delivery systems (ENDS), may experience reductions in symptoms. Existing respiratory symptom questionnaires may not be suitable for studying these changes, as they are intended for patient populations, such as those with chronic obstructive pulmonary disease (COPD). Objective: This study aimed to develop a respiratory symptom questionnaire appropriate for current smokers and for assessing changes when smokers stop smoking. Methods: The Respiratory Symptom Experience Scale (RSES) was derived from existing instruments and subject matter expert input and refined through cognitive debriefing interviews (n=49). Next, for purposes of the quantitative psychometric evaluation, the RSES was administered to smokers (n=202), former smokers (no tobacco use in >6 months; n=200), and switchers (n=208, smokers who switched to ENDS for >6 months), all of whom had smoked for at least 10 years (mean age 33 years). Participants, who averaged 62 (SD 12) years of age, included 28% (173/610) with respiratory allergy symptoms and 17% (104/610) with COPD. Test-retest reliability was assessed by repeat assessment after 1 week in 128 participants. Results: A generalized partial credit model confirmed that the response options were ordered, and a parallel analysis using principal components confirmed that the scale was unidimensional. With allowance for 2 sets of correlated errors between pairs of items, a 1-factor graded response model fit the data. Discrimination parameters were approximately 1 or greater for all items. Scale reliability was 0.80 or higher across a broad range of severity (standardized scores ?0.40 to 3.00). Test-retest reliability (absolute intraclass correlation) was good, at 0.89. RSES convergent validity was supported by substantial differences (Cohen d=0.74) between those with and without a diagnosis of respiratory disease (averaging 0.57 points, indicating that differences of this size or smaller represent meaningful differences). RSES scores also strongly differentiated those with and without COPD (d=1.52). Smokers? RSES scores were significantly higher than former smokers? scores (P<.001). Switchers? RSES scores were significantly lower than smokers? scores (P<.001) and no different from former smokers? scores (P=.34). Conclusions: The RSES fills an important gap in the existing toolkit of respiratory symptom questionnaires; it is a reliable and valid tool to assess respiratory symptoms in adult current and former smokers, including those who have switched to noncombusted nicotine products. This suggests that the scale is sensitive to respiratory symptoms that develop in smokers and to their remission when smokers quit or switch to noncombusted nicotine products intended to reduce the harm of smoking. The findings also suggest that switching from cigarettes to ENDS may improve respiratory health. UR - https://formative.jmir.org/2023/1/e44036 UR - http://dx.doi.org/10.2196/44036 UR - http://www.ncbi.nlm.nih.gov/pubmed/37058347 ID - info:doi/10.2196/44036 ER - TY - JOUR AU - Yip, Jason AU - Wong, Kelly AU - Oh, Isabella AU - Sultan, Farisha AU - Roldan, Wendy AU - Lee, Jin Kung AU - Huh, Jimi PY - 2023/4/14 TI - Co-design Tensions Between Parents, Children, and Researchers Regarding Mobile Health Technology Design Needs and Decisions: Case Study JO - JMIR Form Res SP - e41726 VL - 7 KW - just-in-time adaptive intervention KW - JITAI KW - mobile health KW - mHealth KW - participatory design KW - co-design KW - children and families KW - Black, Indigenous, and people of color KW - BIPOC KW - child-computer interaction KW - design KW - children KW - mobile intervention KW - intervention KW - development KW - mobile phone N2 - Background: Just-in-time adaptive interventions (JITAIs) in mobile health are an intervention design that provides behavior change support based on an individual?s changing and dynamic contextual state. However, few studies have documented how end users of JITAI technologies are involved in their development, particularly from historically marginalized families and children. Less is known for public health researchers and designers of the tensions that occur as families negotiate their needs. Objective: We aimed to broaden our understanding of how historically marginalized families are included in co-design from a public health perspective. We sought to address research questions surrounding JITAIs; co-design; and working with historically marginalized families, including Black, Indigenous, and people of color (BIPOC) children and adults, regarding improving sun protection behaviors. We sought to better understand value tensions in parents? and children?s needs regarding mobile health technologies and how design decisions are made. Methods: We examined 2 sets of co-design data (local and web-based) pertaining to a larger study on mobile SunSmart JITAI technologies with families in Los Angeles, California, United States, who were predominantly of Latinx and multiracial backgrounds. In these co-design sessions, we conducted stakeholder analysis through perceptions of harms and benefits and an assessment of stakeholder views and values. We open coded the data and compared the developed themes using a value-sensitive design framework by examining value tensions to help organize our qualitative data. Our study is formatted through a narrative case study that captures the essential meanings and qualities that are difficult to present, such as quotes in isolation. Results: We presented 3 major themes from our co-design data: different experiences with the sun and protection, misconceptions about the sun and sun protection, and technological design and expectations. We also provided value flow (opportunities for design), value dam (challenges to design), or value flow or dam (a hybrid problem) subthemes. For each subtheme, we provided a design decision and a response we ended up making based on what was presented and the kinds of value tensions we observed. Conclusions: We provide empirical data to show what it is like to work with multiple BIPOC stakeholders in the roles of families and children. We demonstrate the use of the value tension framework to explain the different needs of multiple stakeholders and technology development. Specifically, we demonstrate that the value tension framework helps sort our participants? co-design responses into clear and easy-to-understand design guidelines. Using the value tension framework, we were able to sort the tensions between children and adults, family socioeconomic and health wellness needs, and researchers and participants while being able to make specific design decisions from this organized view. Finally, we provide design implications and guidance for the development of JITAI mobile interventions for BIPOC families. UR - https://formative.jmir.org/2023/1/e41726 UR - http://dx.doi.org/10.2196/41726 UR - http://www.ncbi.nlm.nih.gov/pubmed/37058350 ID - info:doi/10.2196/41726 ER - TY - JOUR AU - Faber, S. Jasper AU - Poot, C. Charlotte AU - Dekkers, Tessa AU - Romero Herrera, Natalia AU - Chavannes, H. Niels AU - Meijer, Eline AU - Visch, T. V. PY - 2023/4/12 TI - Developing a Digital Medication Adherence Intervention for and With Patients With Asthma and Low Health Literacy: Protocol for a Participatory Design Approach JO - JMIR Form Res SP - e35112 VL - 7 KW - participatory design KW - low health literacy KW - eHealth KW - medication adherence KW - asthma KW - mHealth KW - health literacy KW - participatory medicine KW - health care KW - medication N2 - Background: Current eHealth interventions are poorly adopted by people with low health literacy (LHL) as they often fail to meet their needs, skills, and preferences. A major reason for this poor adoption is the generic, one-size-fits-all approach taken by designers of these interventions, without addressing the needs, skills, and preferences of disadvantaged groups. Participatory design approaches are effective for developing interventions that fit the needs of specific target groups; yet, very little is known about the practical implications of executing a participatory design project for and with people with LHL. Objective: This study aimed to demonstrate the application of participatory design activities specifically selected to fit the needs and skills of people with LHL and how these were manifested within an overarching eHealth design process. In addition, the study aims to present reflections and implications of these activities that could support future designers to engage people with LHL in their design processes. Methods: We used the design process of a smart asthma inhaler for people with asthma and LHL to demonstrate participatory design activities. The study was framed under 5 stages of design thinking: empathize, define, ideate, prototype, and test within 2 major iteration cycles. We integrated 3 participatory design activities deemed specifically appropriate for people with LHL: co-constructing stories, experience prototype exhibition, and video prototype evaluation. Results: Co-constructing stories was found to deepen the understanding of the participant?s motivation to use or not to use maintenance medication. This understanding informed and facilitated the subsequent development of diverse preliminary prototypes of possible interventions. Discussing these prototypes in the experience prototype exhibition helped provoke reactions, thoughts, and feelings about the interventions, and potential scenarios of use. Through the video prototype evaluation, we were able to clearly communicate the goal and functionality of the final version of our intervention and gather appropriate responses from our participants. Conclusions: This study demonstrates a participatory design approach for and with patients with asthma and LHL. We demonstrated that careful consideration and selection of activities can result in participants that are engaged and feel understood. This paper provides insight into the practical implications of participatory activities with people with LHL and supports and inspires future designers to engage with this disadvantaged target group. UR - https://formative.jmir.org/2023/1/e35112 UR - http://dx.doi.org/10.2196/35112 UR - http://www.ncbi.nlm.nih.gov/pubmed/37043260 ID - info:doi/10.2196/35112 ER - TY - JOUR AU - Li, Dan AU - Shelby, Tyler AU - Brault, Marie AU - Manohar, Rajit AU - Vermund, Sten AU - Hagaman, Ashley AU - Forastiere, Laura AU - Caruthers, Tyler AU - Egger, Emilie AU - Wang, Yizhou AU - Manohar, Nathan AU - Manohar, Peter AU - Davis, Lucian J. AU - Zhou, Xin PY - 2023/4/7 TI - Implementation of a Hardware-Assisted Bluetooth-Based COVID-19 Tracking Device in a High School: Mixed Methods Study JO - JMIR Form Res SP - e39765 VL - 7 KW - contact tracing KW - COVID-19 KW - digital contact tracing KW - Bluetooth device KW - school health KW - secondary school KW - implementation science KW - mixed methods N2 - Background: Contact tracing is a vital public health tool used to prevent the spread of infectious diseases. However, traditional interview-format contact tracing (TCT) is labor-intensive and time-consuming and may be unsustainable for large-scale pandemics such as COVID-19. Objective: In this study, we aimed to address the limitations of TCT. The Yale School of Engineering developed a Hardware-Assisted Bluetooth-based Infection Tracking (HABIT) device. Following the successful implementation of HABIT in a university setting, this study sought to evaluate the performance and implementation of HABIT in a high school setting using an embedded mixed methods design. Methods: In this pilot implementation study, we first assessed the performance of HABIT using mock case simulations in which we compared contact tracing data collected from mock case interviews (TCT) versus Bluetooth devices (HABIT). For each method, we compared the number of close contacts identified and identification of unique contacts. We then conducted an embedded mixed methods evaluation of the implementation outcomes of HABIT devices using pre- and postimplementation quantitative surveys and qualitative focus group discussions with users and implementers according to the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Results: In total, 17 students and staff completed mock case simulations in which 161 close contact interactions were detected by interview or Bluetooth devices. We detected significant differences in the number of close contacts detected by interview versus Bluetooth devices (P<.001), with most (127/161, 78.9%) contacts being reported by interview only. However, a significant number (26/161, 16.1%; P<.001) of contacts were uniquely identified by Bluetooth devices. The interface, ease of use, coherence, and appropriateness were highly rated by both faculty and students. HABIT provided emotional security to users. However, the prototype design and technical difficulties presented barriers to the uptake and sustained use of HABIT. Conclusions: Implementation of HABIT in a high school was impeded by technical difficulties leading to decreased engagement and adherence. Nonetheless, HABIT identified a significant number of unique contacts not reported by interview, indicating that electronic technologies may augment traditional contact tracing once user preferences are accommodated and technical glitches are overcome. Participants indicated a high degree of acceptance, citing emotional reassurance and a sense of security with the device. UR - https://formative.jmir.org/2023/1/e39765 UR - http://dx.doi.org/10.2196/39765 UR - http://www.ncbi.nlm.nih.gov/pubmed/36525333 ID - info:doi/10.2196/39765 ER - TY - JOUR AU - Ramos, Giovanni AU - Montoya, Kay Amanda AU - Hammons, Renee Hayley AU - Smith, Danielle AU - Chavira, April Denise AU - Rith-Najarian, Rose Leslie PY - 2023/4/6 TI - Digital Intervention Barriers Scale?7 (DIBS-7): Development, Evaluation, and Preliminary Validation JO - JMIR Form Res SP - e40509 VL - 7 KW - barriers KW - development KW - digital mental health intervention KW - measure KW - psychometrics KW - scale KW - validation N2 - Background: The translation of mental health services into digital formats, deemed digital mental health interventions (DMHIs), has the potential to address long-standing obstacles to accessing care. However, DMHIs have barriers of their own that impact enrollment, adherence, and attrition in these programs. Unlike in traditional face-to-face therapy, there is a paucity of standardized and validated measures of barriers in DMHIs. Objective: In this study, we describe the preliminary development and evaluation of such a scale, the Digital Intervention Barriers Scale-7 (DIBS-7). Methods: Following an iterative QUAN ? QUAL mixed methods approach, item generation was guided by qualitative analysis of feedback from participants (n=259) who completed a DMHI trial for anxiety and depression and identified barriers related to self-motivation, ease of use, acceptability, and comprehension of tasks. Item refinement was achieved through DMHI expert review. A final item pool was administered to 559 treatment completers (mean age 23.02 years; 438/559, 78.4% female; 374/559, 69.9% racially or ethnically minoritized). Exploratory factor analyses and confirmatory factor analyses were estimated to determine the psychometric properties of the measure. Finally, criterion-related validity was examined by estimating partial correlations between the DIBS-7 mean score and constructs related to treatment engagement in DMHIs. Results: Statistical analyses estimated a 7-item unidimensional scale with high internal consistency (?=.82, ?=0.89). Preliminary criterion-related validity was supported by significant partial correlations between the DIBS-7 mean score and treatment expectations (pr=?0.25), number of modules with activity (pr=?0.55), number of weekly check-ins (pr=?0.28), and treatment satisfaction (pr=?0.71). Conclusions: Overall, these results provide preliminary support for the use of the DIBS-7 as a potentially useful short scale for clinicians and researchers interested in measuring an important variable often associated with treatment adherence and outcomes in DMHIs. UR - https://formative.jmir.org/2023/1/e40509 UR - http://dx.doi.org/10.2196/40509 UR - http://www.ncbi.nlm.nih.gov/pubmed/37023417 ID - info:doi/10.2196/40509 ER - TY - JOUR AU - Cunningham-Erves, Jennifer AU - Wilkins, H. Consuelo AU - Dempsey, F. Amanda AU - Jones, L. Jessica AU - Thompson, Chris AU - Edwards, Kathryn AU - Davis, Megan AU - Mayberry, S. Lindsay AU - Landsittal, Douglas AU - Hull, C. Pamela PY - 2023/4/4 TI - Development of a Tailored Mobile Phone?Based Intervention to Facilitate Parent-Child Communication and Build Human Papillomavirus Vaccine Confidence: Formative Qualitative Study JO - JMIR Form Res SP - e43041 VL - 7 KW - human papillomavirus KW - HPV KW - vaccine KW - hesitancy KW - parent-child communication KW - theory KW - mobile health KW - mHealth KW - adolescents KW - patient education N2 - Background: Human papillomavirus (HPV) vaccine hesitancy is on the rise, and provider communication is a first-line strategy to address parental concerns. The use of the presumptive approach and motivational interviewing by providers may not be enough to influence parental decision-making owing to the providers? limited time, self-efficacy, and skills to implement these strategies. Interventions to enhance provider communication and build parental HPV vaccine confidence have been undertested. Delivering tailored patient education to parents via mobile phones before they visit the health care provider may address time constraints during clinic visits and positively affect vaccine uptake. Objective: This study aimed to describe the development and evaluate the acceptability of a mobile phone?based, family-focused intervention guided by theory to address concerns of HPV vaccine?hesitant parents before the clinic visit, as well as explore intervention use to facilitate parent-child communication. Methods: The health belief model and theory of reasoned action guided intervention content development. A multilevel stakeholder engagement process was used to iteratively develop the HPVVaxFacts intervention, including a community advisory board review, a review by an advisory panel comprising HPV vaccine?hesitant parents, a health communications expert review, semistructured qualitative interviews with HPV vaccine?hesitant parents (n=31) and providers (n=15), and a content expert review. Inductive thematic analysis was used to identify themes in the interview data. Results: The qualitative interviews yielded 4 themes: overall views toward mobile device use for health information, acceptability of HPVVaxFacts, facilitators of HPVVaxFacts use, and barriers to HPVVaxFacts use. In parent interviews after reviewing HPVVaxFacts prototypes, almost all parents (29/31, 94%) stated they intended to have their child vaccinated. Most of the parents stated that they liked the added adolescents? corner to engage in optional parent-child communication (ie, choice to share and discuss information with their child; 27/31, 87%) and shared decision-making in some cases (8/31, 26%). After incorporating all input, the final intervention consisted of a 10-item survey to identify the top 3 concerns of parents, followed by tailored education that was mapped to each of the following concerns: evidential messages, images or graphics to enhance comprehension and address low literacy, links to credible websites, a provider video, suggested questions to ask their child?s physician, and an optional adolescents? corner to educate the patient and support parent-child communication. Conclusions: The multilevel stakeholder-engaged process used to iteratively develop this novel intervention for HPV vaccine?hesitant families can be used as a model to develop future mobile health interventions. This intervention is currently being pilot-tested in preparation for a randomized controlled trial aiming to increase HPV vaccination among adolescent children of vaccine-hesitant parents in a clinic setting. Future research can adapt HPVVaxFacts for other vaccines and use in other settings (eg, health departments and pharmacies). UR - https://formative.jmir.org/2023/1/e43041 UR - http://dx.doi.org/10.2196/43041 UR - http://www.ncbi.nlm.nih.gov/pubmed/37014680 ID - info:doi/10.2196/43041 ER - TY - JOUR AU - Kiddle, Adam AU - Barham, Helen AU - Wegerif, Simon AU - Petronzio, Connie PY - 2023/3/30 TI - Dynamic Region of Interest Selection in Remote Photoplethysmography: Proof-of-Concept Study JO - JMIR Form Res SP - e44575 VL - 7 KW - vital sign KW - vital sign measurement KW - remote photoplethysmography KW - contactless vital sign measurement KW - region of interest (ROI) KW - biomedical sensing KW - facial camera PPG KW - signal processing KW - machine learning KW - smart device KW - mobile app KW - algorithm KW - skin tone N2 - Background: Remote photoplethysmography (rPPG) can record vital signs (VSs) by detecting subtle changes in the light reflected from the skin. Lifelight (Xim Ltd) is a novel software being developed as a medical device for the contactless measurement of VSs using rPPG via integral cameras on smart devices. Research to date has focused on extracting the pulsatile VS from the raw signal, which can be influenced by factors such as ambient light, skin thickness, facial movements, and skin tone. Objective: This preliminary proof-of-concept study outlines a dynamic approach to rPPG signal processing wherein green channel signals from the most relevant areas of the face (the midface, comprising the cheeks, nose, and top of the lip) are optimized for each subject using tiling and aggregation (T&A) algorithms. Methods: High-resolution 60-second videos were recorded during the VISION-MD study. The midface was divided into 62 tiles of 20×20 pixels, and the signals from multiple tiles were evaluated using bespoke algorithms through weighting according to signal-to-noise ratio in the frequency domain (SNR-F) score or segmentation. Midface signals before and after T&A were categorized by a trained observer blinded to the data processing as 0 (high quality, suitable for algorithm training), 1 (suitable for algorithm testing), or 2 (inadequate quality). On secondary analysis, observer categories were compared for signals predicted to improve categories following T&A based on the SNR-F score. Observer ratings and SNR-F scores were also compared before and after T&A for Fitzpatrick skin tones 5 and 6, wherein rPPG is hampered by light absorption by melanin. Results: The analysis used 4310 videos recorded from 1315 participants. Category 2 and 1 signals had lower mean SNR-F scores than category 0 signals. T&A improved the mean SNR-F score using all algorithms. Depending on the algorithm, 18% (763/4212) to 31% (1306/4212) of signals improved by at least one category, with up to 10% (438/4212) improving into category 0, and 67% (2834/4212) to 79% (3337/4212) remaining in the same category. Importantly, 9% (396/4212) to 21% (875/4212) improved from category 2 (not usable) into category 1. All algorithms showed improvements. No more than 3% (137/4212) of signals were assigned to a lower-quality category following T&A. On secondary analysis, 62% of signals (32/52) were recategorized, as predicted from the SNR-F score. T&A improved SNR-F scores in darker skin tones; 41% of signals (151/369) improved from category 2 to 1 and 12% (44/369) from category 1 to 0. Conclusions: The T&A approach to dynamic region of interest selection improved signal quality, including in dark skin tones. The method was verified by comparison with a trained observer?s rating. T&A could overcome factors that compromise whole-face rPPG. This method?s performance in estimating VS is currently being assessed. Trial Registration: ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746 UR - https://formative.jmir.org/2023/1/e44575 UR - http://dx.doi.org/10.2196/44575 UR - http://www.ncbi.nlm.nih.gov/pubmed/36995742 ID - info:doi/10.2196/44575 ER - TY - JOUR AU - Palmer, Abigail AU - Johns, Gemma AU - Ahuja, Alka AU - Gartner, Daniel PY - 2023/3/28 TI - Optimizing an Adolescent Hybrid Telemedical Mental Health Service Through Staff Scheduling Using Mathematical Programming: Model Development Study JO - JMIR Form Res SP - e43222 VL - 7 KW - linear Programming KW - telemedicine KW - remote consultation KW - mental health KW - teen KW - adolescent KW - mental disorder KW - disorder KW - disease KW - youth KW - decision KW - support KW - tool KW - model N2 - Background: According to the World Health Organization, globally, one in seven 10- to 19-year-olds experiences a mental disorder, accounting for 13% of the global burden of disease in this age group. Half of all mental illnesses begin by the age of 14 years and some teenagers with severe presentations must be admitted to the hospital and assessed by highly skilled mental health care practitioners. Digital telehealth solutions can be useful for the assessment of young individuals remotely. Ultimately, this technology can save travel costs for the health service rather than assessing adolescents in person at the corresponding hospital. Especially in rural regions, where travel times can be high, this innovative approach can make a difference to patients by providing quicker assessments. Objective: The aim of this study is to share insights on how we developed a decision support tool to assign staff to days and locations where adolescent mental health patients are assessed face to face. Where possible, patients are seen through video consultation. The model not only seeks to reduce travel times and consequently carbon emissions but also can be used to find a minimum number of staff to run the service. Methods: To model the problem, we used integer linear programming, a technique that is used in mathematical modeling. The model features 2 objectives: first, we aim to find a minimum coverage of staff to provide the service and second, to reduce travel time. The constraints that are formulated algebraically are used to ensure the feasibility of the schedule. The model is implemented using an open-source solver backend. Results: In our case study, we focus on real-world demand coming from different hospital sites in the UK National Health Service (NHS). We incorporate our model into a decision support tool and solve a realistic test instance. Our results reveal that the tool is not only capable of solving this problem efficiently but also shows the benefits of using mathematical modeling in health services. Conclusions: Our approach can be used by NHS managers to better match capacity and location-dependent demands within an increasing need for hybrid telemedical services, and the aims to reduce traveling and the carbon footprint within health care organizations. UR - https://formative.jmir.org/2023/1/e43222 UR - http://dx.doi.org/10.2196/43222 UR - http://www.ncbi.nlm.nih.gov/pubmed/36976622 ID - info:doi/10.2196/43222 ER - TY - JOUR AU - Iacobelli, Francisco AU - Yang, Anna AU - Tom, Laura AU - Leung, S. Ivy AU - Crissman, John AU - Salgado, Rufino AU - Simon, Melissa PY - 2023/3/28 TI - Predicting Social Determinants of Health in Patient Navigation: Case Study JO - JMIR Form Res SP - e42683 VL - 7 KW - patient navigation KW - machine learning KW - social determinants of health KW - health care disparities KW - health equity KW - case study N2 - Background: Patient navigation (PN) programs have demonstrated efficacy in improving health outcomes for marginalized populations across a range of clinical contexts by addressing barriers to health care, including social determinants of health (SDoHs). However, it can be challenging for navigators to identify SDoHs by asking patients directly because of many factors, including patients? reluctance to disclose information, communication barriers, and the variable resources and experience levels of patient navigators. Navigators could benefit from strategies that augment their ability to gather SDoH data. Machine learning can be leveraged as one of these strategies to identify SDoH-related barriers. This could further improve health outcomes, particularly in underserved populations. Objective: In this formative study, we explored novel machine learning?based approaches to predict SDoHs in 2 Chicago area PN studies. In the first approach, we applied machine learning to data that include comments and interaction details between patients and navigators, whereas the second approach augmented patients? demographic information. This paper presents the results of these experiments and provides recommendations for data collection and the application of machine learning techniques more generally to the problem of predicting SDoHs. Methods: We conducted 2 experiments to explore the feasibility of using machine learning to predict patients? SDoHs using data collected from PN research. The machine learning algorithms were trained on data collected from 2 Chicago area PN studies. In the first experiment, we compared several machine learning algorithms (logistic regression, random forest, support vector machine, artificial neural network, and Gaussian naive Bayes) to predict SDoHs from both patient demographics and navigator?s encounter data over time. In the second experiment, we used multiclass classification with augmented information, such as transportation time to a hospital, to predict multiple SDoHs for each patient. Results: In the first experiment, the random forest classifier achieved the highest accuracy among the classifiers tested. The overall accuracy to predict SDoHs was 71.3%. In the second experiment, multiclass classification effectively predicted a few patients? SDoHs based purely on demographic and augmented data. The best accuracy of these predictions overall was 73%. However, both experiments yielded high variability in individual SDoH predictions and correlations that become salient among SDoHs. Conclusions: To our knowledge, this study is the first approach to applying PN encounter data and multiclass learning algorithms to predict SDoHs. The experiments discussed yielded valuable lessons, including the awareness of model limitations and bias, planning for standardization of data sources and measurement, and the need to identify and anticipate the intersectionality and clustering of SDoHs. Although our focus was on predicting patients? SDoHs, machine learning can have a broad range of applications in the field of PN, from tailoring intervention delivery (eg, supporting PN decision-making) to informing resource allocation for measurement, and PN supervision. UR - https://formative.jmir.org/2023/1/e42683 UR - http://dx.doi.org/10.2196/42683 UR - http://www.ncbi.nlm.nih.gov/pubmed/36976634 ID - info:doi/10.2196/42683 ER - TY - JOUR AU - Liu, Yuxuan AU - Lyu, Xiaoguang AU - Yang, Bo AU - Fang, Zhixiang AU - Hu, Dejun AU - Shi, Lei AU - Wu, Bisheng AU - Tian, Yong AU - Zhang, Enli AU - Yang, YuanChao PY - 2023/3/21 TI - Early Triage of Critically Ill Adult Patients With Mushroom Poisoning: Machine Learning Approach JO - JMIR Form Res SP - e44666 VL - 7 KW - mushroom poisoning KW - triage KW - model KW - machine learning KW - XGBoost KW - extreme gradient boosting N2 - Background: Early triage of patients with mushroom poisoning is essential for administering precise treatment and reducing mortality. To our knowledge, there has been no established method to triage patients with mushroom poisoning based on clinical data. Objective: The purpose of this work was to construct a triage system to identify patients with mushroom poisoning based on clinical indicators using several machine learning approaches and to assess the prediction accuracy of these strategies. Methods: In all, 567 patients were collected from 5 primary care hospitals and facilities in Enshi, Hubei Province, China, and divided into 2 groups; 322 patients from 2 hospitals were used as the training cohort, and 245 patients from 3 hospitals were used as the test cohort. Four machine learning algorithms were used to construct the triage model for patients with mushroom poisoning. Performance was assessed using the area under the receiver operating characteristic curve (AUC), decision curve, sensitivity, specificity, and other representative statistics. Feature contributions were evaluated using Shapley additive explanations. Results: Among several machine learning algorithms, extreme gradient boosting (XGBoost) showed the best discriminative ability in 5-fold cross-validation (AUC=0.83, 95% CI 0.77-0.90) and the test set (AUC=0.90, 95% CI 0.83-0.96). In the test set, the XGBoost model had a sensitivity of 0.93 (95% CI 0.81-0.99) and a specificity of 0.79 (95% CI 0.73-0.85), whereas the physicians? assessment had a sensitivity of 0.86 (95% CI 0.72-0.95) and a specificity of 0.66 (95% CI 0.59-0.73). Conclusions: The 14-factor XGBoost model for the early triage of mushroom poisoning can rapidly and accurately identify critically ill patients and will possibly serve as an important basis for the selection of treatment options and referral of patients, potentially reducing patient mortality and improving clinical outcomes. UR - https://formative.jmir.org/2023/1/e44666 UR - http://dx.doi.org/10.2196/44666 UR - http://www.ncbi.nlm.nih.gov/pubmed/36943366 ID - info:doi/10.2196/44666 ER - TY - JOUR AU - Rebhi, Mahmoud AU - Ben Aissa, Mohamed AU - Tannoubi, Amayra AU - Saidane, Mouna AU - Guelmami, Noomen AU - Puce, Luca AU - Chen, Wen AU - Chalghaf, Nasr AU - Azaiez, Fairouz AU - Zghibi, Makrem AU - Bragazzi, Luigi Nicola PY - 2023/3/20 TI - Reliability and Validity of the Arabic Version of the Game Experience Questionnaire: Pilot Questionnaire Study JO - JMIR Form Res SP - e42584 VL - 7 KW - Arab countries KW - game experience KW - reliability KW - scale KW - validity N2 - Background: Nowadays, digital gaming occupies a central position in the entertainment industry where it has developed into a cherished kind of entertainment in markets all over the world. In addition, it provides other sectors with various social and economic benefits. The Game Experience Questionnaire (GEQ) is a free, quantitative, and comprehensive self-report measure that was developed to assess the player game experience. Despite having been widely used by many research projects in the past, it has not been adapted into Arabic. Furthermore, several components of the scale proved problematic from a psychometric point of view. Therefore, a modified version of the scale is needed to measure the gaming experience of the Arab population. Objective: The aim of this study was to validate and examine the psychometrics of an adapted Arabic version of the GEQ in Tunisia. Methods: A total of 771 volunteer participants completed an online survey, which included an Arabic version of the GEQ, gaming data, and a sociodemographic questionnaire. Subjects were randomized in order to complete two phases of the study: exploratory and confirmatory. The exploratory data were acquired from 360 respondents whose mean age was 23.89 (SD 2.29) years. Out of 360 respondents, 111 (30.8%) were female and 249 (69.2%) were male. Confirmatory data were obtained from the remaining 411 subjects whose mean age was 21.94 (SD 1.80) years. Out of 411 subjects, 169 (41.1%) were female and 242 (58.9%) were male. Results: After the elimination of two items, the exploratory and the confirmatory factor analyses provided an adequate factor structure of the Arabic version of the GEQ. In addition, the internal consistency coefficients suggested the reliability of the instrument. Significant differences were revealed for three subcomponents: flow by age (?2=0.013, P=.002), gender (?2=0.007, P=.02), and game type (?2=0.03, P<.001). For competence (?2=0.01, P=.03) and immersion (?2=0.02, P=.01), significant differences were highlighted by the type of game. The discriminant and convergent validities of the instrument were supported by calculating the average variance extracted (AVE) and comparing the square roots of the AVE values to the correlation coefficients, respectively. Conclusions: The Arabic adapted version of the GEQ is valid and reliable and can be administered to measure the game experience in Arab countries. UR - https://formative.jmir.org/2023/1/e42584 UR - http://dx.doi.org/10.2196/42584 UR - http://www.ncbi.nlm.nih.gov/pubmed/36482747 ID - info:doi/10.2196/42584 ER - TY - JOUR AU - Bjertnæs, Øyvind AU - Iversen, Hestad Hilde AU - Norman, Rebecka AU - Valderas, M. Jose PY - 2023/3/17 TI - Web-Based Public Ratings of General Practitioners in Norway: Validation Study JO - JMIR Form Res SP - e38932 VL - 7 KW - web-based rating KW - questionnaire KW - psychometric KW - patient-reported experiences and satisfaction KW - survey KW - health care KW - practitioner KW - doctor rating KW - physician rating KW - patient provider KW - patient experience KW - patient satisfaction N2 - Background: Understanding the complex relationships among multiple strategies for gathering users? perspectives in the evaluation of the performance of services is crucial for the interpretation of user-reported measures. Objective: The main objectives were to (1) evaluate the psychometric performance of an 11-item web-based questionnaire of ratings of general practitioners (GPs) currently used in Norway (Legelisten.no) and (2) assess the association between web-based and survey-based patient experience indicators. Methods: We included all published ratings on GPs and practices on Legelisten.no in the period of May 5, 2012, to December 15, 2021 (N=76,521). The questionnaire consists of 1 mandatory item and 10 voluntary items with 5 response categories (1 to 5 stars), alongside an open-ended review question and background variables. Questionnaire dimensionality and internal consistency were assessed with Cronbach ?, exploratory factor, and item response theory analyses, and a priori hypotheses were developed for assessing construct validity (chi-square analysis). We calculated Spearman correlations between web-based ratings and reference patient experience indicators based on survey data using the patient experiences with the GP questionnaire (n=5623 respondents for a random sample of 50 GPs). Results: Web-based raters were predominantly women (n=32,074, 64.0%), in the age range of 20-50 years (n=35,113, 74.6%), and reporting 5 or fewer consultations with the GP each year (n=28,798, 64.5%). Ratings were missing for 18.9% (n=14,500) to 27.4% (n=20,960) of nonmandatory items. A total of 4 of 11 rating items showed a U-shaped distribution, with >60% reporting 5 stars. Factor analysis and internal consistency testing identified 2 rating scales: ?GP? (5 items; ?=.98) and ?practice? (6 items; ?=.85). Some associations were not consistent with a priori hypotheses and allowed only partial confirmation of the construct validity of ratings. Item response theory analysis results were adequate for the ?practice? scale but not for the ?GP? scale, with items with inflated discrimination (>5) distributed over a narrow interval of the scale. The correlations between the web-based ratings GP scale and GP reference indicators ranged from 0.34 (P=.021) to 0.44 (P=.002), while the correlation between the web-based ratings practice scale and reference indicators ranged from 0.17 (not significant) to 0.49 (P<.001). The strongest correlations between web-based and survey scores were found for items measuring practice-related experiences: phone availability (?=0.51), waiting time in the office (?=0.62), other staff (?=0.54-0.58; P<.001). Conclusions: The practice scale of the web-based ratings has adequate psychometric performance, while the GP suffers from important limitations. The associations with survey-based patient experience indicators were accordingly mostly weak to modest. Our study underlines the importance of interpreting web-based ratings with caution and the need to further develop rating sites. UR - https://formative.jmir.org/2023/1/e38932 UR - http://dx.doi.org/10.2196/38932 UR - http://www.ncbi.nlm.nih.gov/pubmed/36930207 ID - info:doi/10.2196/38932 ER - TY - JOUR AU - Sezgin, Emre AU - Hussain, Syed-Amad AU - Rust, Steve AU - Huang, Yungui PY - 2023/3/7 TI - Extracting Medical Information From Free-Text and Unstructured Patient-Generated Health Data Using Natural Language Processing Methods: Feasibility Study With Real-world Data JO - JMIR Form Res SP - e43014 VL - 7 KW - patient-generated health data KW - natural language processing KW - named entity recognition KW - patient health records KW - text notes KW - voice KW - audio real-world data N2 - Background: Patient-generated health data (PGHD) captured via smart devices or digital health technologies can reflect an individual health journey. PGHD enables tracking and monitoring of personal health conditions, symptoms, and medications out of the clinic, which is crucial for self-care and shared clinical decisions. In addition to self-reported measures and structured PGHD (eg, self-screening, sensor-based biometric data), free-text and unstructured PGHD (eg, patient care note, medical diary) can provide a broader view of a patient?s journey and health condition. Natural language processing (NLP) is used to process and analyze unstructured data to create meaningful summaries and insights, showing promise to improve the utilization of PGHD. Objective: Our aim is to understand and demonstrate the feasibility of an NLP pipeline to extract medication and symptom information from real-world patient and caregiver data. Methods: We report a secondary data analysis, using a data set collected from 24 parents of children with special health care needs (CSHCN) who were recruited via a nonrandom sampling approach. Participants used a voice-interactive app for 2 weeks, generating free-text patient notes (audio transcription or text entry). We built an NLP pipeline using a zero-shot approach (adaptive to low-resource settings). We used named entity recognition (NER) and medical ontologies (RXNorm and SNOMED CT [Systematized Nomenclature of Medicine Clinical Terms]) to identify medication and symptoms. Sentence-level dependency parse trees and part-of-speech tags were used to extract additional entity information using the syntactic properties of a note. We assessed the data; evaluated the pipeline with the patient notes; and reported the precision, recall, and F1 scores. Results: In total, 87 patient notes are included (audio transcriptions n=78 and text entries n=9) from 24 parents who have at least one CSHCN. The participants were between the ages of 26 and 59 years. The majority were White (n=22, 92%), had more than one child (n=16, 67%), lived in Ohio (n=22, 92%), had mid- or upper-mid household income (n=15, 62.5%), and had higher level education (n=24, 58%). Out of 87 notes, 30 were drug and medication related, and 46 were symptom related. We captured medication instances (medication, unit, quantity, and date) and symptoms satisfactorily (precision >0.65, recall >0.77, F1>0.72). These results indicate the potential when using NER and dependency parsing through an NLP pipeline on information extraction from unstructured PGHD. Conclusions: The proposed NLP pipeline was found to be feasible for use with real-world unstructured PGHD to accomplish medication and symptom extraction. Unstructured PGHD can be leveraged to inform clinical decision-making, remote monitoring, and self-care including medical adherence and chronic disease management. With customizable information extraction methods using NER and medical ontologies, NLP models can feasibly extract a broad range of clinical information from unstructured PGHD in low-resource settings (eg, a limited number of patient notes or training data). UR - https://formative.jmir.org/2023/1/e43014 UR - http://dx.doi.org/10.2196/43014 UR - http://www.ncbi.nlm.nih.gov/pubmed/36881467 ID - info:doi/10.2196/43014 ER - TY - JOUR AU - Harakeh, Zeena AU - de Hoogh, M. Iris AU - van Keulen, Hilde AU - Kalkman, Gino AU - van Someren, Eugene AU - van Empelen, Pepijn AU - Otten, Wilma PY - 2023/3/7 TI - 360° Diagnostic Tool to Personalize Lifestyle Advice in Primary Care for People With Type 2 Diabetes: Development and Usability Study JO - JMIR Form Res SP - e37305 VL - 7 KW - type 2 diabetes KW - diagnostic tool KW - holistic approach KW - personalized advice KW - shared decision-making KW - health professionals N2 - Background: Various multifaceted factors need to be addressed to improve the health and quality of life of people with type 2 diabetes (T2D). Therefore, we developed a web-based decision support tool that comprises a more holistic diagnosis (including 4 domains: body, thinking and feeling, behavior, and environment) and personalized advice. This 360° diagnostic tool enables people with T2D and health care professionals at the general practice to obtain an overview of the most important T2D-related issues and, subsequently, determine the most suitable intervention for the person with T2D. Objective: This study aimed to describe the systematic and iterative development and evaluation of the web-based 360° diagnostic tool. Methods: We defined the requirements for the web-based 360° diagnostic tool based on previously developed tools, a literature review, and inputs from a multidisciplinary team of experts. As part of the conceptualization, we defined 3 requirements: diagnostics; feedback; and advice, consultation, and follow-up. Next, we developed and designed the content for each of these requirements. We evaluated the diagnostic part of the tool (ie, measurement instruments and visualization) with a qualitative design, in a usability study with a think-aloud strategy and interview questions, among 8 people with T2D at a Dutch general practice. Results: For each of the 4 domains, specific parameters and underlying elements were selected, and measurement instruments (including clinical data and questionnaires) were chosen. Cutoff values were defined to identify high-, middle-, and low-ranking scores, and decision rules were developed and implemented using R scripts and algorithms. A traffic light color visual design was created (profile wheel) to provide an overview of the scores per domain. We mapped the interventions that could be added to the tool and developed a protocol designed as a card deck with motivational interview steps. Furthermore, the usability study showed that people with T2D perceived the tool as easy to use, useful, easy to understand, and insightful. Conclusions: Preliminary evaluation of the 360° diagnostic tool by experts, health care professionals, and people with T2D showed that the tool was considered relevant, clear, and practical. The iterative process provided insights into the areas of improvement, which were implemented. The strengths, shortcomings, future use, and challenges are also discussed. UR - https://formative.jmir.org/2023/1/e37305 UR - http://dx.doi.org/10.2196/37305 UR - http://www.ncbi.nlm.nih.gov/pubmed/36881463 ID - info:doi/10.2196/37305 ER - TY - JOUR AU - van Barneveld, Esther AU - Lim, Arianne AU - van Hanegem, Nehalennia AU - van Osch, Frits AU - Vork, Lisa AU - Kruimel, Joanna AU - Bongers, Marlies AU - Leue, Carsten PY - 2023/3/3 TI - Real-time Symptom Assessment in Patients With Endometriosis: Psychometric Evaluation of an Electronic Patient-Reported Outcome Measure, Based on the Experience Sampling Method JO - JMIR Form Res SP - e29480 VL - 7 KW - endometriosis KW - pelvic pain KW - positive affect KW - negative affect KW - patient-reported outcome measure KW - momentary symptom assessment KW - experience sampling method KW - pain KW - PROM KW - outcome KW - patient-reported KW - assessment KW - symptom KW - sampling KW - method KW - evaluation KW - psychometric KW - real-time KW - prospective N2 - Background: The experience sampling method (ESM) holds advantages over traditional retrospective questionnaires including a high ecological validity, no recall bias, the ability to assess fluctuation of symptoms, and the ability to analyze the temporal relationship between variables. Objective: This study aimed to evaluate the psychometric properties of an endometriosis-specific ESM tool. Methods: This is a short-term follow-up prospective study, including patients with premenopausal endometriosis aged ?18 years who reported dysmenorrhea, chronic pelvic pain, or dyspareunia between December 2019 and November 2020. An ESM-based questionnaire was sent out by a smartphone application 10 times a day during 1 week on randomly chosen moments. Additionally, patients completed questionnaires concerning demographics, end-of-day pain scores, and end-of-week symptom scores. The psychometric evaluation included compliance, concurrent validity, and internal consistency. Results: Twenty-eight patients with endometriosis completed the study. Compliance for answering the ESM questions was as high as 52%. End-of-week pain scores were higher than ESM mean scores and showed peak reporting. ESM scores showed strong concurrent validity when compared with symptoms scored by the Gastrointestinal Symptom Rating Scale?Irritable Bowel Syndrome, 7-item Generalized Anxiety Disorders Scale, 9-question Patient Health Questionnaire, and the majority of questions of the 30-item Endometriosis Health Profile. Cronbach ? coefficients demonstrated a good internal consistency for abdominal symptoms, general somatic symptoms, and positive affect, and an excellent internal consistency for negative affect. Conclusions: This study supports the validity and reliability of a newly developed electronic instrument for the measurement of symptoms in women with endometriosis, based on momentary assessments. This ESM patient-reported outcome measure has the advantage of providing a more detailed view on individual symptom patterns and offers the possibility for patients to have insight in their symptomatology, leading to more individualized treatment strategies that can improve the quality of life of women with endometriosis. UR - https://formative.jmir.org/2023/1/e29480 UR - http://dx.doi.org/10.2196/29480 UR - http://www.ncbi.nlm.nih.gov/pubmed/36867439 ID - info:doi/10.2196/29480 ER - TY - JOUR AU - O'Connor, Antonia AU - Sharrad, Kelsey AU - King, Charmaine AU - Carson-Chahhoud, Kristin PY - 2023/3/2 TI - An Augmented Reality Technology to Provide Demonstrative Inhaler Technique Education for Patients With Asthma: Interview Study Among Patients, Health Professionals, and Key Community Stakeholders JO - JMIR Form Res SP - e34958 VL - 7 KW - augmented reality KW - asthma KW - disease management KW - smartphone KW - inhaler technique KW - mobile phone N2 - Background: Many people with asthma use incorrect inhaler technique, resulting in suboptimal disease management and increased health service use. Novel ways of delivering appropriate instructions are needed. Objective: This study explored stakeholder perspectives on the potential use of augmented reality (AR) technology to improve asthma inhaler technique education. Methods: On the basis of existing evidence and resources, an information poster displaying the images of 22 asthma inhaler devices was developed. Using AR technology via a free smartphone app, the poster launched video demonstrations of correct inhaler technique for each device. In total, 21 semistructured, one?on?one interviews with health professionals, people with asthma, and key community stakeholders were conducted, and data were analyzed thematically using the Triandis model of interpersonal behavior. Results: A total of 21 participants were recruited into the study, and data saturation was achieved. People with asthma were confident with inhaler technique (mean score 9.17, SD 1.33, out of 10). However, health professionals and key community stakeholders identified that this perception was misguided (mean 7.25, SD 1.39, and mean 4.5, SD 0.71, for health professionals and key community stakeholders, respectively) and facilitates persistent incorrect inhaler use and suboptimal disease management. Delivering inhaler technique education using AR was favored by all participants (21/21, 100%), particularly around ease of use, with the ability to visually display inhaler techniques for each device. There was a strongly held belief that the technology has the capacity for improving inhaler technique across all participant groups (mean 9.25, SD 0.89, for participants; mean 9.83, SD 0.41, for health professionals; and mean 9.5, SD 0.71, for key community stakeholders). However, all participants (21/21, 100%) identified some barriers, particularly regarding access and appropriateness of AR for older people. Conclusions: AR technology may be a novel means to address poor inhaler technique among certain cohorts of patients with asthma and serve as a prompt for health professionals to initiate review of inhaler devices. A randomized controlled trial design is needed to evaluate the efficacy of this technology for use in the clinical care setting. UR - https://formative.jmir.org/2023/1/e34958 UR - http://dx.doi.org/10.2196/34958 UR - http://www.ncbi.nlm.nih.gov/pubmed/36862496 ID - info:doi/10.2196/34958 ER - TY - JOUR AU - Ayre, Julie AU - Bonner, Carissa AU - Muscat, M. Danielle AU - Dunn, G. Adam AU - Harrison, Eliza AU - Dalmazzo, Jason AU - Mouwad, Dana AU - Aslani, Parisa AU - Shepherd, L. Heather AU - McCaffery, J. Kirsten PY - 2023/2/14 TI - Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1 JO - JMIR Form Res SP - e40645 VL - 7 KW - health literacy KW - comprehension KW - health education KW - health communication KW - medicine information KW - readability UR - https://formative.jmir.org/2023/1/e40645 UR - http://dx.doi.org/10.2196/40645 UR - http://www.ncbi.nlm.nih.gov/pubmed/36787164 ID - info:doi/10.2196/40645 ER - TY - JOUR AU - Chorney, Jill AU - Johnson Emberly, Debbie AU - Jeffrey, Jennifer AU - Hundert, Amos AU - Pakkanlilar, Onur AU - Abidi, Sabina AU - Bagnell, Alexa AU - Brennan, Maureen AU - Campbell, Anne Leslie AU - Clark, Sharon AU - Bradley, Kristina AU - Ross, Olivia PY - 2023/2/6 TI - Implementation of a Knowledge Management System in Mental Health and Addictions: Mixed Methods Case Study JO - JMIR Form Res SP - e39334 VL - 7 KW - mental health KW - knowledge management KW - information KW - technology KW - capacity building N2 - Background: Mental health and addictions (MHA) care is complex and individualized and requires coordination across providers and areas of care. Knowledge management is an essential facilitator and common challenge in MHA services. Objective: This paper aimed to describe the development of a knowledge management system (KMS) and the associated processes in 1 MHA program. We also aimed to examine the uptake and use, satisfaction, and feedback on implementation among a group of pilot testers. Methods: This project was conducted as a continuous quality-improvement initiative. Integrated stakeholder engagement was used to scope the content and design the information architecture to be implemented using a commercially available knowledge management platform. A group of 30 clinical and administrative staff were trained and tested with the KMS over a period of 10 weeks. Feedback was collected via surveys and focus groups. System analytics were used to characterize engagement. The content, design, and full-scale implementation planning of the KMS were refined based on the results. Results: Satisfaction with accessing the content increased from baseline to after the pilot. Most testers indicated that they would recommend the KMS to a colleague, and satisfaction with KMS functionalities was high. A median of 7 testers was active each week, and testers were active for a median of 4 days over the course of the pilot. Focus group themes included the following: the KMS was a solution to problems for staff members, functionality of the KMS was important, quality content matters, training was helpful and could be improved, and KMS access was required to be easy and barrier free. Conclusions: Knowledge management is an ongoing need in MHA services, and KMSs hold promise in addressing this need. Testers in 1 MHA program found a KMS that is easy to use and would recommend it to colleagues. Opportunities to improve implementation and increase uptake were identified. Future research is needed to understand the impact of KMSs on quality of care and organizational efficiency. UR - https://formative.jmir.org/2023/1/e39334 UR - http://dx.doi.org/10.2196/39334 UR - http://www.ncbi.nlm.nih.gov/pubmed/36745489 ID - info:doi/10.2196/39334 ER - TY - JOUR AU - Zale, D. Andrew AU - Abusamaan, S. Mohammed AU - McGready, John AU - Mathioudakis, Nestoras PY - 2023/1/31 TI - Prediction of Next Glucose Measurement in Hospitalized Patients by Comparing Various Regression Methods: Retrospective Cohort Study JO - JMIR Form Res SP - e41577 VL - 7 KW - hospital KW - glucose KW - inpatient KW - prediction KW - regression KW - machine learning N2 - Background: Continuous glucose monitors have shown great promise in improving outpatient blood glucose (BG) control; however, continuous glucose monitors are not routinely used in hospitals, and glucose management is driven by point-of-care (finger stick) and serum glucose measurements in most patients. Objective: This study aimed to evaluate times series approaches for prediction of inpatient BG using only point-of-care and serum glucose observations. Methods: Our data set included electronic health record data from 184,320 admissions, from patients who received at least one unit of subcutaneous insulin, had at least 4 BG measurements, and were discharged between January 1, 2015, and May 31, 2019, from 5 Johns Hopkins Health System hospitals. A total of 2,436,228 BG observations were included after excluding measurements obtained in quick succession, from patients who received intravenous insulin, or from critically ill patients. After exclusion criteria, 2.85% (3253/113,976), 32.5% (37,045/113,976), and 1.06% (1207/113,976) of admissions had a coded diagnosis of type 1, type 2, and other diabetes, respectively. The outcome of interest was the predicted value of the next BG measurement (mg/dL). Multiple time series predictors were created and analyzed by comparing those predictors and the index BG measurement (sample-and-hold technique) with next BG measurement. The population was classified by glycemic variability based on the coefficient of variation. To compare the performance of different time series predictors among one another, R2, root mean squared error, and Clarke Error Grid were calculated and compared with the next BG measurement. All these time series predictors were then used together in Cubist, linear, random forest, partial least squares, and k-nearest neighbor methods. Results: The median number of BG measurements from 113,976 admissions was 12 (IQR 5-24). The R2 values for the sample-and-hold, 2-hour, 4-hour, 16-hour, and 24-hour moving average were 0.529, 0.504, 0.481, 0.467, and 0.459, respectively. The R2 values for 4-hour moving average based on glycemic variability were 0.680, 0.480, 0.290, and 0.205 for low, medium, high, and very high glucose variability, respectively. The proportion of BG predictions in zone A of the Clarke Error Grid analysis was 61%, 59%, 27%, and 53% for 4-hour moving average, 24-hour moving average, 3 observation rolling regression, and recursive regression predictors, respectively. In a fully adjusted Cubist, linear, random forest, partial least squares, and k-nearest neighbor model, the R2 values were 0.563, 0.526, 0.538, and 0.472, respectively. Conclusions: When analyzing time series predictors independently, increasing variability in a patient?s BG decreased predictive accuracy. Similarly, inclusion of older BG measurements decreased predictive accuracy. These relationships become weaker as glucose variability increases. Machine learning techniques marginally augmented the performance of time series predictors for predicting a patient?s next BG measurement. Further studies should determine the potential of using time series analyses for prediction of inpatient dysglycemia. UR - https://formative.jmir.org/2023/1/e41577 UR - http://dx.doi.org/10.2196/41577 UR - http://www.ncbi.nlm.nih.gov/pubmed/36719713 ID - info:doi/10.2196/41577 ER - TY - JOUR AU - Varriale, Pasquale AU - Müller, Borna AU - Katz, Grégory AU - Dallas, Lorraine AU - Aguaron, Alfonso AU - Azoulai, Marion AU - Girard, Nicolas PY - 2023/1/26 TI - Patient Perspectives on Value Dimensions of Lung Cancer Care: Cross-sectional Web-Based Survey JO - JMIR Form Res SP - e37190 VL - 7 KW - lung KW - cancer KW - health quality of life KW - patient reported outcome KW - PROM KW - economic burden KW - cost KW - economic KW - burden KW - perspective KW - survey KW - QoL KW - quality of life KW - questionnaire KW - caregiver KW - caregiving KW - physical well-being KW - end of life KW - palliative KW - physical function KW - independence KW - distress N2 - Background: While the lung cancer (LC) treatment landscape has rapidly evolved in recent years, easing symptom burden and treatment side effects remain central considerations in disease control. Objective: The aim of this study was to assess the relative importance of dimensions of LC care to patients, and to explore the disease burden, including socioeconomic aspects not commonly covered in patient-reported outcomes instruments. Methods: A questionnaire was sent to patients with LC and their caregivers to rate the value of a diverse set of quality of life dimensions in care, to evaluate communication between health care professionals (HCPs) and patients, and to explore the economic impact on respondents. The survey included questions on the dimensions of care covered by patient-reported outcomes instruments for quality-of-life evaluation (Functional Assessment of Cancer Therapy?Lung scale, EQ-5D, the European Organization for Research and Treatment of Cancer?s Core Quality of Life questionnaire, and the European Organization for Research and Treatment of Cancer?s Core Quality of Life in lung cancer 13-item questionnaire), as well as the International Consortium for Health Outcomes Measurement (ICHOM) standard set of patient-centered outcomes for LC. The survey respondents were participants on Carenity?s patient community platform, living either in France, the United Kingdom, Germany, Italy, or Spain. Results: The survey included 150 respondents (115 patients and 35 caregivers). ?Physical well-being? and ?end-of-life care? (median scores of 9.6, IQR 7.7-10, and 9.7, IQR 8.0-10, on a 10-point scale) were rated highest among the different value dimensions assessed. ?Physical well-being and functioning? was the dimension most frequently discussed with health care professionals (82/150, 55%), while only (17/100, 17%) reported discussing ?end-of-life care.? After diagnosis, 43% (49/112) of patients younger than 65 years stopped working. Among respondents who indicated their monthly household income before and after diagnosis, 55% (38/69) reported a loss of income. Conclusions: Our results showed the relevance of a broad range of aspects of care for the quality of life of patients with LC. End-of-life care was the dimension of care rated highest by patients with LC, irrespective of stage at diagnosis; however, this aspect is least frequently discussed with HCPs. The results also highlight the considerable socioeconomic impact of the disease, despite insurance coverage of direct costs. UR - https://formative.jmir.org/2023/1/e37190 UR - http://dx.doi.org/10.2196/37190 UR - http://www.ncbi.nlm.nih.gov/pubmed/36416499 ID - info:doi/10.2196/37190 ER - TY - JOUR AU - Cuomo, Raphael AU - Purushothaman, Vidya AU - Calac, J. Alec AU - McMann, Tiana AU - Li, Zhuoran AU - Mackey, Tim PY - 2023/1/25 TI - Estimating County-Level Overdose Rates Using Opioid-Related Twitter Data: Interdisciplinary Infodemiology Study JO - JMIR Form Res SP - e42162 VL - 7 KW - overdose KW - mortality KW - geospatial analysis KW - social media KW - drug overuse KW - substance use KW - social media data KW - mortality estimates KW - real-time data KW - public health data KW - demographic variables KW - county-level N2 - Background: There were an estimated 100,306 drug overdose deaths between April 2020 and April 2021, a three-quarter increase from the prior 12-month period. There is an approximate 6-month reporting lag for provisional counts of drug overdose deaths from the National Vital Statistics System, and the highest level of geospatial resolution is at the state level. By contrast, public social media data are available close to real-time and are often accessible with precise coordinates. Objective: The purpose of this study is to assess whether county-level overdose mortality burden could be estimated using opioid-related Twitter data. Methods: International Classification of Diseases (ICD) codes for poisoning or exposure to overdose at the county level were obtained from CDC WONDER. Demographics were collected from the American Community Survey. The Twitter Application Programming Interface was used to obtain tweets that contained any of the 36 terms with drug names. An unsupervised classification approach was used for clustering tweets. Population-normalized variables and polynomial population-normalized variables were produced. Furthermore, z scores of the Getis Ord Gi clustering statistic were produced, and both these scores and their polynomial counterparts were explored in regression modeling of county-level overdose mortality burden. A series of linear regression models were used for predictive modeling to explore the interpretability of the analytical output. Results: Modeling overdose mortality with normalized demographic variables alone explained only 7.4% of the variability in county-level overdose mortality, whereas this was approximately doubled by the use of specific demographic and Twitter data covariates based on a backward selection approach. The highest adjusted R2 and lowest AIC (Akaike Info Criterion) were obtained for the model with normalized demographic variables, normalized z scores from geospatial analyses, and normalized topic counts (adjusted R2=0.133, AIC=8546.8). The z scores of the Getis Ord Gi statistic appeared to have improved utility over population-normalization alone. In this model, median age, female population, and tweets about web-based drug sales were positively associated with opioid mortality. Asian race and Hispanic ethnicity were significantly negatively associated with county-level burdens of overdose mortality. Conclusions: Social media data, when transformed using certain statistical approaches, may add utility to the goal of producing closer to real-time county-level estimates of overdose mortality. Prediction of opioid-related outcomes can be advanced to inform prevention and treatment decisions. This interdisciplinary approach can facilitate evidence-based funding decisions for various substance use disorder prevention and treatment programs. UR - https://formative.jmir.org/2023/1/e42162 UR - http://dx.doi.org/10.2196/42162 UR - http://www.ncbi.nlm.nih.gov/pubmed/36548118 ID - info:doi/10.2196/42162 ER - TY - JOUR AU - Rodoreda-Pallàs, Berta AU - Lumillo-Gutiérrez, Iris AU - Miró Catalina, Queralt AU - Torra Escarrer, Eva AU - Sanahuja Juncadella, Jaume AU - Morin Fraile, Victoria PY - 2023/1/25 TI - Recording of Social Determinants in Computerized Medical Records in Primary Care Consultations: Quasi-Experimental Study JO - JMIR Form Res SP - e41706 VL - 7 KW - recording of social determinants of health KW - computerised medical records KW - electronic health record coding KW - non-clinical diagnoses KW - Z-coding KW - primary care KW - medical records KW - intervention KW - medical KW - treatment N2 - Background:  Social determinants of health may be more important than medical or lifestyle choices in influencing people's health. Even so, there is a deficit in recording these in patients' computerized medical histories. The Spanish administration and the World Health Organization are promoting the recording of diagnoses in computerized clinical histories with the aim of benefiting the individual, the professional, and the community. In most cases, professionals tend to record only clinical diagnoses despite evidence in the literature documenting that addressing the social determinants of health can lead to improvements in health and reductions in social disparities in disease. Objective:  This study aims to develop and evaluate the effectiveness of a mixed intervention (face-to-face-digital) aimed at improving the quantity and quality of the records of the social determinants of health in computerized medical records at primary care clinics. Methods:  A quasi-experimental, nonrandomized, controlled, multicenter study with 2 parallel study arms was conducted in the area of Central Catalonia (Spain) with primary care professionals of the Institut Català de la Salut (ICS), working from September 23, 2019, to March 31, 2020. All interested professionals were accepted. In total, 22 basic health areas were involved in the study. In Spain and Catalonia, the International Classification of Diseases is used, in which there is a coding of the social determinants of health. Five social determinants were selected by a physician, a nurse, and a social worker; these professionals had experience in primary care and were experts in community health. The choice was made taking into account the ease of use, benefit, and existing terminology. The intervention, based on the integration of a checklist, was integrated as part of the usual multidisciplinary clinical workflow in primary care consultations to influence the recording of these determinants in the patient's computerized medical record. Results:  After 6 months of implementing the intervention, the volume and quantity of records of 5 social determinants of health were compared, and a significant increase in the median number of pre- and postintervention diagnoses was observed (P?.001). There was also an increase in the diversity of selected social determinants. Using the linear regression model, the significant mean increase of the experimental group with respect to the control group was estimated with a coefficient of 8.18 (95% CI 5.11-11.26). Conclusions:  The intervention described in this study is an effective tool for coding the social determinants of health designed by a multidisciplinary team to be incorporated into the workflow of primary care practices. The effectiveness of its usability and the description of the intervention described here should be generalizable to any environment. Trial Registration: ClinicalTrials.gov NCT04151056; https://clinicaltrials.gov/ct2/show/NCT04151056 UR - https://formative.jmir.org/2023/1/e41706 UR - http://dx.doi.org/10.2196/41706 UR - http://www.ncbi.nlm.nih.gov/pubmed/36696168 ID - info:doi/10.2196/41706 ER - TY - JOUR AU - Claessens, Janneau AU - van Egmond, Juultje AU - Wanten, Joukje AU - Bauer, Noël AU - Nuijts, Rudy AU - Wisse, Robert PY - 2023/1/25 TI - The Accuracy of a Web-Based Visual Acuity Self-assessment Tool Performed Independently by Eye Care Patients at Home: Method Comparison Study JO - JMIR Form Res SP - e41045 VL - 7 KW - eHealth KW - telemonitoring KW - telemedicine KW - telehealth KW - visual acuity KW - eye care KW - ophthalmology N2 - Background:  Telehealth solutions can play an important role in increasing access to eye care. Web-based eye tests can enable individuals to self-assess their visual function remotely without the assistance of an eye care professional. A web-based tool for self-assessing visual acuity (VA) has previously been studied in controlled, supervised conditions. The accuracy of this tool when performed independently by patients in their home environment, using their own devices, has not yet been examined. Objective:  The objective of this paper was to examine the accuracy of a web-based tool with respect to measuring VA in ophthalmic patients in their home environment, compared with a conventional in-hospital assessment using a Snellen chart (the gold standard). Methods:  From April through September 2020, consecutive adult patients with uveitis at the University Medical Center Utrecht, the Netherlands, performed the web-based VA test at home (the index test) before their upcoming conventional VA assessment at the hospital (the reference test). The agreement between the 2 tests was assessed by the Bland-Altman analysis. Additional analyses were performed to investigate associations between clinical characteristics and the accuracy of the web-based test. Results:  A total of 98 eyes in 59 patients were included in the study. The difference in VA between the index and reference tests was not significant, with a mean difference of 0.02 (SD 0.12) logMAR (P=.09) and 95% limits of agreement of ?0.21 to 0.26 logMAR. The majority of the differences (77%) fell within the predetermined acceptable deviation limit of 0.15 logMAR. In addition, no patient characteristics or clinical parameters were found to significantly affect the accuracy of the web-based test. Conclusions:  This web-based test for measuring VA is a valid tool for remotely assessing VA, also when performed independently by patients at home. Implementation of validated web-based tools like this in the health care system may represent a valuable step forward in revolutionizing teleconsultations and can provide individual patients with the opportunity to self-monitor changes in VA. This is particularly relevant when the patient?s access to ophthalmic care is limited. Future developments should focus on optimizing the testing conditions at home to reduce outliers. UR - https://formative.jmir.org/2023/1/e41045 UR - http://dx.doi.org/10.2196/41045 UR - http://www.ncbi.nlm.nih.gov/pubmed/36696171 ID - info:doi/10.2196/41045 ER - TY - JOUR AU - Hoogendoorn, Petra AU - Versluis, Anke AU - van Kampen, Sanne AU - McCay, Charles AU - Leahy, Matt AU - Bijlsma, Marlou AU - Bonacina, Stefano AU - Bonten, Tobias AU - Bonthuis, Marie-José AU - Butterlin, Anouk AU - Cobbaert, Koen AU - Duijnhoven, Thea AU - Hallensleben, Cynthia AU - Harrison, Stuart AU - Hastenteufel, Mark AU - Holappa, Terhi AU - Kokx, Ben AU - Morlion, Birgit AU - Pauli, Norbert AU - Ploeg, Frank AU - Salmon, Mark AU - Schnoor, Kyma AU - Sharp, Mary AU - Sottile, Angelo Pier AU - Värri, Alpo AU - Williams, Patricia AU - Heidenreich, Georg AU - Oughtibridge, Nicholas AU - Stegwee, Robert AU - Chavannes, H. Niels PY - 2023/1/23 TI - What Makes a Quality Health App?Developing a Global Research-Based Health App Quality Assessment Framework for CEN-ISO/TS 82304-2: Delphi Study JO - JMIR Form Res SP - e43905 VL - 7 KW - health app KW - wellness app KW - mobile health KW - mHealth KW - Delphi technique KW - quality assessment KW - assessment framework KW - standard KW - standardization KW - COVID-19 N2 - Background: The lack of an international standard for assessing and communicating health app quality and the lack of consensus about what makes a high-quality health app negatively affect the uptake of such apps. At the request of the European Commission, the international Standard Development Organizations (SDOs), European Committee for Standardization, International Organization for Standardization, and International Electrotechnical Commission have joined forces to develop a technical specification (TS) for assessing the quality and reliability of health and wellness apps. Objective: This study aimed to create a useful, globally applicable, trustworthy, and usable framework to assess health app quality. Methods: A 2-round Delphi technique with 83 experts from 6 continents (predominantly Europe) participating in one (n=42, 51%) or both (n=41, 49%) rounds was used to achieve consensus on a framework for assessing health app quality. Aims included identifying the maximum 100 requirement questions for the uptake of apps that do or do not qualify as medical devices. The draft assessment framework was built on 26 existing frameworks, the principles of stringent legislation, and input from 20 core experts. A follow-up survey with 28 respondents informed a scoring mechanism for the questions. After subsequent alignment with related standards, the quality assessment framework was tested and fine-tuned with manufacturers of 11 COVID-19 symptom apps. National mirror committees from the 52 countries that participated in the SDO technical committees were invited to comment on 4 working drafts and subsequently vote on the TS. Results: The final quality assessment framework includes 81 questions, 67 (83%) of which impact the scores of 4 overarching quality aspects. After testing with people with low health literacy, these aspects were phrased as ?Healthy and safe,? ?Easy to use,? ?Secure data,? and ?Robust build.? The scoring mechanism enables communication of the quality assessment results in a health app quality score and label, alongside a detailed report. Unstructured interviews with stakeholders revealed that evidence and third-party assessment are needed for health app uptake. The manufacturers considered the time needed to complete the assessment and gather evidence (2-4 days) acceptable. Publication of CEN-ISO/TS 82304-2:2021 Health software ? Part 2: Health and wellness apps ? Quality and reliability was approved in May 2021 in a nearly unanimous vote by 34 national SDOs, including 6 of the 10 most populous countries worldwide. Conclusions: A useful and usable international standard for health app quality assessment was developed. Its quality, approval rate, and early use provide proof of its potential to become the trusted, commonly used global framework. The framework will help manufacturers enhance and efficiently demonstrate the quality of health apps, consumers, and health care professionals to make informed decisions on health apps. It will also help insurers to make reimbursement decisions on health apps. UR - https://formative.jmir.org/2023/1/e43905 UR - http://dx.doi.org/10.2196/43905 UR - http://www.ncbi.nlm.nih.gov/pubmed/36538379 ID - info:doi/10.2196/43905 ER - TY - JOUR AU - Gao, Xiang AU - Fewx, Melody AU - Sprock, John AU - Jiang, Huanhuan AU - Gao, Yong AU - Liu, Yatao PY - 2023/1/16 TI - A Novel Puff Recording Electronic Nicotine Delivery System for Assessing Naturalistic Puff Topography and Nicotine Consumption During Ad Libitum Use: Ancillary Study JO - JMIR Form Res SP - e42544 VL - 7 KW - electronic nicotine delivery system KW - electronic cigarettes KW - puff topography KW - nicotine consumption KW - ad libitum use N2 - Background: Assessing the naturalistic puff topography and associated nicotine consumption during e-cigarette use is important as such information will not only unveil how these products are being consumed in real-world conditions, but also enable investigators and regulatory bodies to conduct quantitative, accurate, and realistic harmful exposure and nicotine abuse liability risk assessments based on actual e-cigarette use. Conventional approaches cannot accurately, conveniently, and noninvasively determine e-cigarette puff topography in a natural use environment. Thus, novel technology-enabled systems that do not primarily rely on self-report mechanisms or intrusive measurements to monitor e-cigarette product use behaviors are highly desired. Objective: This study aimed to explore and demonstrate the feasibility of a novel puff recording electronic nicotine delivery system (PR-ENDS) device for measuring naturalistic puff topography and estimating nicotine consumption during the ad libitum use of products among smokers and vapers. Methods: An ancillary data analysis based on a completed parent study was conducted. The parent study was a 1-way randomized controlled open-label puff topography and nicotine pharmacokinetic assessment carried out in 24 healthy adults (12 smokers and 12 vapers). Participants were assigned a randomized product use sequence of a PR-ENDS device within 5 site visits for both controlled and ad libitum product use sessions. Blood samples were obtained for plasma nicotine analysis, and questionnaires were administered at various time points. During the ad libitum use session, puff topography was measured using a Clinical Research Support System (CReSS) device as a benchmark, as well as the PR-ENDS device with a built-in puff recording feature. Results: There were no significant differences in representative puff topography parameters (number of puffs, total puff duration, and average puff duration) between the PR-ENDS and CReSS devices at the populational level across different device powers, e-liquid nicotine strengths, and flavors. The nicotine consumption estimated by the PR-ENDS device suggested that this device can be employed as a convenient monitoring tool for estimating nicotine use without measuring e-liquid weight loss between puffs. The linear relationship between nicotine consumption estimated by the PR-ENDS device and the pharmacokinetic parameter AUCad lib (plasma concentration-time curve for 1-hour ad libitum use) substantiated the potential of using this device as a pragmatic, noninvasive, and convenient means for estimating nicotine intake in the human body without blood collection. Conclusions: The novel PR-ENDS device was feasible for assessing naturalistic puff topography and estimating nicotine consumption and intake in the human body during ad libitum use. Several key factors can influence users? puff topography, including device power levels, e-liquid nicotine strengths, and flavors. The study results pave the way for further research in the real-time measurement of naturalistic puff topography and puffing behaviors in the real world. UR - https://formative.jmir.org/2023/1/e42544 UR - http://dx.doi.org/10.2196/42544 UR - http://www.ncbi.nlm.nih.gov/pubmed/36542679 ID - info:doi/10.2196/42544 ER - TY - JOUR AU - Foerster, Milena AU - Dufour, Lucas AU - Bäumler, Wolfgang AU - Schreiver, Ines AU - Goldberg, Marcel AU - Zins, Marie AU - Ezzedine, Khaled AU - Schüz, Joachim PY - 2023/1/11 TI - Development and Validation of the Epidemiological Tattoo Assessment Tool to Assess Ink Exposure and Related Factors in Tattooed Populations for Medical Research: Cross-sectional Validation Study JO - JMIR Form Res SP - e42158 VL - 7 KW - tattoos KW - cancer KW - questionnaire development KW - exposure assessment KW - cohort studies KW - epidemiology KW - polycyclic aromatic hydrocarbons KW - primary aromatic amines KW - metals KW - digital surface analysis N2 - Background: Tattooing, whose popularity is growing worldwide, is an invasive body art that involves the injection of chemical mixtures, the tattoo ink, into the upper layer of the dermis. Although these inks may contain environmental toxins, including known human carcinogens, their long-term health effects are poorly studied. To conduct the urgently required epidemiological studies on tattoos and their long-term health effects, a validated method for assessing the complex tattoo exposure is needed. Objective: We aimed to develop and validate the Epidemiological Tattoo Assessment Tool (EpiTAT), a questionnaire to self-assess tattoo ink exposure in tattooed populations suitable for application in large epidemiological cohort studies. Methods: One of 3 preliminary versions of the EpiTAT using one of the alternative tattoo measurement units hand surface, credit card, or body schemes was randomly filled in by tattooed volunteers in Lyon, France. To identify the most suitable unit of tattoo self-assessment, a validation study was conducted with the selected respondents (N=97) to compare the self-assessments of tattoo surface, color, and coverage with validation measurements made by trained study personnel. Intraclass correlation, the Kendall rank correlation, and 2-tailed t tests were used to statistically compare tattoo size, color area, and tattoo coverage separately for each questionnaire version. Participants? opinions on the alternative measurement units were also considered in the overall evaluation. For quality control of the validation measures, digital surface analysis of 62 photographs of selected tattoos was performed using Fiji/ImageJ. Results: In general, the results revealed overestimation of self-assessed measures compared with validation measures (eg, mean tattooed body surface 1768, SD 1547, cm2 vs 930, SD 1047, cm2, respectively, for hand surface; P<.001) and validation measures compared with digital image analysis (mean individual tattoo surface 147, SD 303.9, cm2 vs 101, SD 154.7, cm2, respectively; P=.05). Although the measurement unit credit card yielded the most accurate measures for all variables of interest, it had a much lower completion rate (78/129, 60.5%) than hand surface (89/104, 85.6%) and body schemes (90/106, 84.9%). Hand surface measured total tattoo size more accurately than body schemes (absolute agreement intraclass correlation coefficient: 0.71 vs 0.64, respectively). Conclusions: The final version of the EpiTAT contains 21 items and uses hand surface as a visual unit of measurement. Likert scales are used to assess color and coverage as a proportion of the total tattoo area. The overestimation of tattoo size by self-reporting merits further research to identify potential influential factors or predictive patterns that could be considered when calculating exposure. UR - https://formative.jmir.org/2023/1/e42158 UR - http://dx.doi.org/10.2196/42158 UR - http://www.ncbi.nlm.nih.gov/pubmed/36630184 ID - info:doi/10.2196/42158 ER - TY - JOUR AU - Lin, Chen AU - Yousefi, Safoora AU - Kahoro, Elvis AU - Karisani, Payam AU - Liang, Donghai AU - Sarnat, Jeremy AU - Agichtein, Eugene PY - 2022/12/19 TI - Detecting Elevated Air Pollution Levels by Monitoring Web Search Queries: Algorithm Development and Validation JO - JMIR Form Res SP - e23422 VL - 6 IS - 12 KW - nowcasting of air pollution KW - web-based public health surveillance KW - neural network sequence modeling KW - search engine log analysis KW - air pollution exposure assessment KW - mobile phone N2 - Background: Real-time air pollution monitoring is a valuable tool for public health and environmental surveillance. In recent years, there has been a dramatic increase in air pollution forecasting and monitoring research using artificial neural networks. Most prior work relied on modeling pollutant concentrations collected from ground-based monitors and meteorological data for long-term forecasting of outdoor ozone (O3), oxides of nitrogen, and fine particulate matter (PM2.5). Given that traditional, highly sophisticated air quality monitors are expensive and not universally available, these models cannot adequately serve those not living near pollutant monitoring sites. Furthermore, because prior models were built based on physical measurement data collected from sensors, they may not be suitable for predicting the public health effects of pollution exposure. Objective: This study aimed to develop and validate models to nowcast the observed pollution levels using web search data, which are publicly available in near real time from major search engines. Methods: We developed novel machine learning?based models using both traditional supervised classification methods and state-of-the-art deep learning methods to detect elevated air pollution levels at the US city level by using generally available meteorological data and aggregate web-based search volume data derived from Google Trends. We validated the performance of these methods by predicting 3 critical air pollutants (O3, nitrogen dioxide, and PM2.5) across 10 major US metropolitan statistical areas in 2017 and 2018. We also explore different variations of the long short-term memory model and propose a novel search term dictionary learner-long short-term memory model to learn sequential patterns across multiple search terms for prediction. Results: The top-performing model was a deep neural sequence model long short-term memory, using meteorological and web search data, and reached an accuracy of 0.82 (F1-score 0.51) for O3, 0.74 (F1-score 0.41) for nitrogen dioxide, and 0.85 (F1-score 0.27) for PM2.5, when used for detecting elevated pollution levels. Compared with using only meteorological data, the proposed method achieved superior accuracy by incorporating web search data. Conclusions: The results show that incorporating web search data with meteorological data improves the nowcasting performance for all 3 pollutants and suggest promising novel applications for tracking global physical phenomena using web search data. UR - https://formative.jmir.org/2022/12/e23422 UR - http://dx.doi.org/10.2196/23422 UR - http://www.ncbi.nlm.nih.gov/pubmed/36534457 ID - info:doi/10.2196/23422 ER - TY - JOUR AU - Rosenfeld, A. Eve AU - Lyman, Cassondra AU - Roberts, E. John PY - 2022/12/13 TI - Development of an mHealth App?Based Intervention for Depressive Rumination (RuminAid): Mixed Methods Focus Group Evaluation JO - JMIR Form Res SP - e40045 VL - 6 IS - 12 KW - depression KW - rumination KW - mobile health KW - mHealth KW - evidence-based treatment KW - focus group KW - mental health KW - mobile app KW - mobile phone N2 - Background: Depression is a common mental health condition that poses a significant public health burden. Effective treatments for depression exist; however, access to evidence-based care remains limited. Mobile health (mHealth) apps offer an avenue for improving access. However, few mHealth apps are informed by evidence-based treatments and even fewer are empirically evaluated before dissemination. To address this gap, we developed RuminAid, an mHealth app that uses evidence-based treatment components to reduce depression by targeting a single key depressogenic process?rumination. Objective: The primary objective of this study was to collect qualitative and quantitative feedback that could be used to improve the design of RuminAid before the software development phase. Methods: We reviewed empirically supported interventions for depression and rumination and used the key aspects of each to create a storyboard version of RuminAid. We distributed an audio-guided presentation of the RuminAid storyboard to 22 individuals for viewing and solicited user feedback on app content, design, and perceived functionality across 7 focus group sessions. Results: The consumer-rated quality of the storyboard version of RuminAid was in the acceptable to good range. Indeed, most participants reported that they thought RuminAid would be an engaging, functional, and informational app. Likewise, they endorsed overwhelming positive beliefs about the perceived impact of RuminAid; specifically, 96% (21/22) believed that RuminAid will help depressed ruminators with depression and rumination. Nevertheless, the results highlighted the need for improved app aesthetics (eg, a more appealing color scheme and modern design). Conclusions: Focus group members reported that the quality of information was quite good and had the potential to help adults who struggle with depression and rumination but expressed concern that poor aesthetics would interfere with users? desire to continue using the app. To address these comments, we hired a graphic designer and redesigned each screen to improve visual appeal. We also removed time gating from the app based on participant feedback and findings from related research. These changes helped elevate RuminAid and informed its initial software build for a pilot trial that focused on evaluating its feasibility and acceptability. UR - https://formative.jmir.org/2022/12/e40045 UR - http://dx.doi.org/10.2196/40045 UR - http://www.ncbi.nlm.nih.gov/pubmed/36512400 ID - info:doi/10.2196/40045 ER - TY - JOUR AU - Braitman, L. Abby AU - Strowger, Megan AU - Shipley, L. Jennifer AU - Ortman, Jordan AU - MacIntyre, I. Rachel AU - Bauer, A. Elizabeth PY - 2022/12/9 TI - Data Quality and Study Compliance Among College Students Across 2 Recruitment Sources: Two Study Investigation JO - JMIR Form Res SP - e39488 VL - 6 IS - 12 KW - data quality KW - attention checks KW - recruitment KW - retention KW - college students KW - mobile phone N2 - Background: Models of satisficing suggest that study participants may not fully process survey items and provide accurate responses when survey burden is higher and when participant motivation is lower. Participants who do not fully process survey instructions can reduce a study?s power and hinder generalizability. Common concerns among researchers using self-report measures are data quality and participant compliance. Similarly, attrition can hurt the power and generalizability of a study. Objective: Given that college students comprise most samples in psychological studies, especially examinations of student issues and psychological health, it is critical to understand how college student recruitment sources impact data quality (operationalized as attention check items with directive instructions and correct answers) and retention (operationalized as the completion of follow-up surveys over time). This examination aimed to examine the following: whether data quality varies across recruitment sources, whether study retention varies across recruitment sources, the impact of data quality on study variable associations, the impact of data quality on measures of internal consistency, and whether the demographic qualities of participants significantly vary across those who failed attention checks versus those who did not. Methods: This examination was a follow-up analysis of 2 previously published studies to explore data quality and study compliance. Study 1 was a cross-sectional, web-based survey examining college stressors and psychological health (282/407, 69.3% female; 230/407, 56.5% White, 113/407, 27.8% Black; mean age 22.65, SD 6.73 years). Study 2 was a longitudinal college drinking intervention trial with an in-person baseline session and 2 web-based follow-up surveys (378/528, 71.6% female; 213/528, 40.3% White, 277/528, 52.5% Black; mean age 19.85, SD 1.65 years). Attention checks were included in both studies to assess data quality. Participants for both studies were recruited from a psychology participation pool (a pull-in method; for course credit) and the general student body (a push-out method; for monetary payment or raffle entry). Results: A greater proportion of participants recruited through the psychology pool failed attention checks in both studies, suggesting poorer data quality. The psychology pool was also associated with lower retention rates over time. After screening out those who failed attention checks, some correlations among the study variables were stronger, some were weaker, and some were fairly similar, potentially suggesting bias introduced by including these participants. Differences among the indicators of internal consistency for the study measures were negligible. Finally, attention check failure was not significantly associated with most demographic characteristics but varied across some racial identities. This suggests that filtering out data from participants who failed attention checks may not limit sample diversity. Conclusions: Investigators conducting college student research should carefully consider recruitment and include attention checks or other means of detecting poor quality data. Recommendations for researchers are discussed. UR - https://formative.jmir.org/2022/12/e39488 UR - http://dx.doi.org/10.2196/39488 UR - http://www.ncbi.nlm.nih.gov/pubmed/36485020 ID - info:doi/10.2196/39488 ER - TY - JOUR AU - Patel, Piyush Sharvil AU - Sun, Elizabeth AU - Reinhardt, Alec AU - Geevarghese, Sanjaly AU - He, Simon AU - Gazmararian, A. Julie PY - 2022/12/6 TI - Social Determinants of Digital Health Adoption: Pilot Cross-sectional Survey JO - JMIR Form Res SP - e39647 VL - 6 IS - 12 KW - digital health KW - health accessibility KW - utilization KW - mobile health KW - mHealth KW - telemedicine N2 - Background: Interest in and funding for digital health interventions have rapidly grown in recent years. Despite the increasing familiarity with mobile health from regulatory bodies, providers, and patients, overarching research on digital health adoption has been primarily limited to morbidity-specific and non-US samples. Consequently, there is a limited understanding of what personal factors hold statistically significant relationships with digital health uptake. Moreover, this limits digital health communities? knowledge of equity along digital health use patterns. Objective: This study aims to identify the social determinants of digital health tool adoption in Georgia. Methods: Web-based survey respondents in Georgia 18 years or older were recruited from mTurk to answer primarily closed-ended questions within the following domains: participant demographics and health consumption background, telehealth, digital health education, prescription management tools, digital mental health services, and doctor finder tools. Participants spent around 15 to 20 minutes on a survey to provide demographic and personal health care consumption data. This data was analyzed with multivariate linear and logistic regressions to identify which of these determinants, if any, held statistically significant relationships with the total number of digital health tool categories adopted and which of these determinants had absolute relationships with specific categories. Results: A total of 362 respondents completed the survey. Private insurance, residence in an urban area, having a primary care provider, fewer urgent emergency room (ER) visits, more ER visits leading to inpatient stays, and chronic condition presence were significantly associated with the number of digital health tool categories adopted. The separate logistic regressions exhibited substantial variability, with 3.5 statistically significant predictors per model, on average. Age, federal poverty level, number of primary care provider visits in the past 12 months, number of nonurgent ER visits in the past 12 months, number of urgent ER visits in the past 12 months, number of ER visits leading to inpatient stays in the past 12 months, race, gender, ethnicity, insurance, education, residential area, access to the internet, difficulty accessing health care, usual source of care, status of primary care provider, and status of chronic condition all had at least one statistically significant relationship with the use of a specific digital health category. Conclusions: The results demonstrate that persons who are socioeconomically disadvantaged may not adopt digital health tools at disproportionately higher rates. Instead, digital health tools may be adopted along social determinants of health, providing strong evidence for the digital health divide. The variability of digital health adoption necessitates investing in and building a common framework to increase mobile health access. With a common framework and a paradigm shift in the design, evaluation, and implementation strategies around digital health, disparities can be further mitigated and addressed. This likely will begin with a coordinated effort to determine barriers to adopting digital health solutions. UR - https://formative.jmir.org/2022/12/e39647 UR - http://dx.doi.org/10.2196/39647 UR - http://www.ncbi.nlm.nih.gov/pubmed/36472905 ID - info:doi/10.2196/39647 ER - TY - JOUR AU - Barzegari, Saeed AU - Sharifi Kia, Ali AU - Bardus, Marco AU - Stoyanov, R. Stoyan AU - GhaziSaeedi, Marjan AU - Rafizadeh, Mouna PY - 2022/12/5 TI - The Persian Version of the Mobile Application Rating Scale (MARS-Fa): Translation and Validation Study JO - JMIR Form Res SP - e42225 VL - 6 IS - 12 KW - mobile application rating scale KW - Farsi KW - mobile apps KW - validation KW - smartphone addiction KW - Persian KW - Iran KW - development KW - mobile health KW - mHealth KW - scale KW - validate KW - reliability KW - measurement tool KW - assessment tool N2 - Background: Approximately 110 million Farsi speakers worldwide have access to a growing mobile app market. Despite restrictions and international sanctions, Iran?s internal mobile health app market is growing, especially for Android-based apps. However, there is a need for guidelines for developing health apps that meet international quality standards. There are also no tools in Farsi that assess health app quality. Developers and researchers who operate in Farsi could benefit from such quality assessment tools to improve their outputs. Objective: This study aims to translate and culturally adapt the Mobile Application Rating Scale in Farsi (MARS-Fa). This study also evaluates the validity and reliability of the newly developed MARS-Fa tool. Methods: We used a well-established method to translate and back translate the MARS-Fa tool with a group of Iranian and international experts in Health Information Technology and Psychology. The final translated version of the tool was tested on a sample of 92 apps addressing smartphone addiction. Two trained reviewers completed an independent assessment of each app in Farsi and English. We reported reliability and construct validity estimates for the objective scales (engagement, functionality, aesthetics, and information quality). Reliability was based on the evaluation of intraclass correlation coefficients, Cronbach ? and Spearman-Brown split-half reliability indicators (for internal consistency), as well as Pearson correlations for test-retest reliability. Construct validity included convergent and discriminant validity (through item-total correlations within the objective scales) and concurrent validity using Pearson correlations between the objective and subjective scores. Results: After completing the translation and cultural adaptation, the MARS-Fa tool was used to assess the selected apps for smartphone addiction. The MARS-Fa total scale showed good interrater reliability (intraclass correlation coefficient=0.83, 95% CI 0.74-0.89) and good internal consistency (Cronbach ?=.84); Spearman-Brown split-half reliability for both raters was 0.79 to 0.93. The instrument showed excellent test-retest reliability (r=0.94). The correlations among the MARS-Fa subdomains and the total score were all significant and above r=0.40, suggesting good convergent and discriminant validity. The MARS-Fa was positively and significantly correlated with subjective quality (r=0.90, P<.001), and so were the objective subdomains of engagement (r=0.85, P<.001), information quality (r=0.80, P<.001), aesthetics (r=0.79, P<.001), and functionality (r=0.57, P<.001), indicating concurrent validity. Conclusions: The MARS-Fa is a reliable and valid instrument to assess mobile health apps. This instrument could be adopted by Farsi-speaking researchers and developers who want to evaluate the quality of mobile apps. While we tested the tool with a sample of apps addressing smartphone addiction, the MARS-Fa could assess other domains or issues since the Mobile App Rating Scale has been used to rate apps in different contexts and languages. UR - https://formative.jmir.org/2022/12/e42225 UR - http://dx.doi.org/10.2196/42225 UR - http://www.ncbi.nlm.nih.gov/pubmed/36469402 ID - info:doi/10.2196/42225 ER - TY - JOUR AU - Lyu, Chen Joanne AU - Afolabi, Aliyyat AU - White, S. Justin AU - Ling, M. Pamela PY - 2022/12/1 TI - Perceptions and Aspirations Toward Peer Mentoring in Social Media?Based Electronic Cigarette Cessation Interventions for Adolescents and Young Adults: Focus Group Study JO - JMIR Form Res SP - e42538 VL - 6 IS - 12 KW - peer mentoring KW - electronic nicotine delivery systems KW - cessation KW - social media KW - adolescents and young adults N2 - Background: Social media offer a promising channel to deliver e-cigarette cessation interventions to adolescents and young adults (AYAs); however, interventions delivered on social media face challenges of low participant retention and decreased engagement over time. Peer mentoring has the potential to ameliorate these challenges. Objective: The aim of this study was to understand, from both the mentee and potential mentor perspective, the needs, expectations, and concerns of AYAs regarding peer mentoring to inform the development of social media?based peer mentoring interventions for e-cigarette cessation among AYAs. Methods: Seven focus groups, including four mentee groups and three potential mentor groups, were conducted with 26 AYAs who had prior experience with e-cigarette use and attempts to quit in the context of a social media?based e-cigarette cessation intervention. Discussion focused on preferred characteristics of peer mentors, expectations about peer mentoring, mentoring mode, mentor training, incentives for peer mentors, preferred social media platforms for intervention delivery, supervision, and concerns. Focus group transcripts were coded and analyzed using a thematic analysis approach. Results: Overall, participants were receptive to peer mentoring in social media?based cessation interventions and believed they could be helpful in assisting e-cigarette cessation. Participants identified the most important characteristics of peer mentors to be of similar age and to be abstinent from e-cigarette use. Participants expected peer mentors would share personal experiences, provide emotional support, and send check-ins and reminders. Peer mentors supporting a group of mentees in combination with one-on-one mentoring as needed was the preferred mentoring mode. A group of 10 mentees with a mentor:mentee ratio of 1:3-5 was deemed acceptable for most participants. Participants expressed that mentor training should include emotional intelligence, communication skills, and the scientific evidence about e-cigarettes. Although monetary incentives were not the main motivating factor for being a peer mentor, they were viewed as a good way to compensate mentors? time. Instagram was considered an appropriate social media platform to deliver a peer-mentored intervention due to its functionality. Participants did not express many privacy concerns about social media?based peer mentoring, but mentioned that boundaries and community agreements should be set to keep relationships professional. Conclusions: This study reflects the needs and preferences of young people for a peer mentoring intervention to complement a social media program to support e-cigarette cessation. The next step will be to establish the feasibility, acceptability, and preliminary efficacy of such a peer mentoring program. UR - https://formative.jmir.org/2022/12/e42538 UR - http://dx.doi.org/10.2196/42538 UR - http://www.ncbi.nlm.nih.gov/pubmed/36454628 ID - info:doi/10.2196/42538 ER - TY - JOUR AU - Chalghaf, Nasr AU - Chen, Wen AU - Tannoubi, Amayra AU - Guelmami, Noomen AU - Puce, Luca AU - Ben Said, Noureddine AU - Ben Khalifa, Maher AU - Azaiez, Fairouz AU - Bragazzi, Luigi Nicola PY - 2022/12/1 TI - Job Disengagement Among Physical Education Teachers: Insights From a Cross-sectional Web-Based Survey With Path Modeling Analysis JO - JMIR Form Res SP - e29130 VL - 6 IS - 12 KW - Work Disengagement Scale KW - work KW - job KW - job satisfaction KW - family?work conflict KW - perceived stress KW - physical education KW - PLS-SEM KW - SmartPLS KW - teacher KW - engagement KW - Arab KW - stress KW - primary school KW - secondary school KW - development KW - measurement KW - scale KW - tool KW - fitness KW - educator KW - school KW - satisfaction KW - digital tool KW - mental health KW - family KW - cross-sectional KW - survey KW - modelling KW - psychology N2 - Background: Physical education teachers often experience stress and job disengagement. Objective: This study?s aims were as follows: (1) to adapt in the Arabic language and test the reliability and the validity of the work?family conflict (WFC) and family?work conflict (FWC) scales, (2) to develop and assess the psychometric properties of work disengagement among physical education teachers, and (3) to evaluate an explanatory model by presenting the mediating role of perceived stress as a major influencing factor in work disengagement and job satisfaction. Methods: A total of 303 primary and secondary school physical education teachers, comprising 165 (54.5%) men and 138 (45.5%) women participated voluntarily in our study. The measuring instruments are the Work Disengagement Scale, the Perceived Stress Scale, the WFC scale, the FWC scale, and the 9-item Teacher of Physical Education Job Satisfaction Inventory. Results: The Arabic language versions of the WFC and FWC scales had reasonably adequate psychometric properties, which were justified by confirmatory factor analyses and by the measurement of reliability, convergent, and discriminant validity through the measurement model using SmartPLS software. Similarly, the structural model established with SmartPLS confirmed strong links of the concepts of FWC, WFC, the job satisfaction questionnaire, and perceived stress with work disengagement among teachers of physical education. Conclusions: There is a growing interest in helping teachers cope with the daily pressures of work and family. A positive organizational context is a context with clear values regarding work priorities, which constitutes the basis of a feeling of shared responsibility and professional support. Good conditions can act as protective factors reducing work stress and positively influencing personal well-being, work attitudes, work commitment, and professional efficiency. Additional research on teachers is needed to examine the relationship between perceived work stress and the role of families, along with the extent to which this association can have a significant impact on teachers? commitment to work. UR - https://formative.jmir.org/2022/12/e29130 UR - http://dx.doi.org/10.2196/29130 UR - http://www.ncbi.nlm.nih.gov/pubmed/36084318 ID - info:doi/10.2196/29130 ER - TY - JOUR AU - Fenton, Alex AU - Heinze, Aleksej AU - Osborne, McVal AU - Ahmed, Wasim PY - 2022/11/25 TI - How to Use the Six-Step Digital Ethnography Framework to Develop Buyer Personas: The Case of Fan Fit JO - JMIR Form Res SP - e41489 VL - 6 IS - 11 KW - health tracking KW - digital KW - ethnography KW - apps KW - mobile app KW - customer KW - physical activity N2 - Background: One of the key features of digital marketing is customer centricity, which can be applied to the domain of health. This is expressed through the ability to target specific customer segments with relevant content using appropriate channels and having data to track and understand each interaction. In order to do this, marketers create buyer personas based on a wide spectrum of quantitative and qualitative data. Digital ethnography is another established method for studying web-based communities. However, for practitioners, the complexity, rigor, and time associated with ethnographical work are sometimes out of reach. Objective: This paper responds to the gaps in the practically focused method of using social media for digital ethnography to develop buyer personas. This paper aims to demonstrate how digital ethnography can be used as a way to create and refine buyer personas. Methods: Using a case study of the Fan Fit smartphone app, which aimed to increase physical activity, a digital ethnography was applied to create a better understanding of customers and to create and refine buyer personas. Results: We propose two buyer personas, and we develop a 6-step digital ethnography framework designed for the development of buyer personas. Conclusions: The key contribution of this work is the proposal of a 6-step digital ethnography framework designed for the development of buyer personas. We highlight that the 6-step digital ethnography could be a robust tool for practitioners and academicians to analyze digital communications for the process of creating and updating data-driven buyer personas to create deeper insights into digital and health marketing efforts. UR - https://formative.jmir.org/2022/11/e41489 UR - http://dx.doi.org/10.2196/41489 UR - http://www.ncbi.nlm.nih.gov/pubmed/36427232 ID - info:doi/10.2196/41489 ER - TY - JOUR AU - Henriksen, Berg Hege AU - Knudsen, Dines Markus AU - Carlsen, Hauger Monica AU - Hjartåker, Anette AU - Blomhoff, Rune PY - 2022/11/8 TI - A Short Digital Food Frequency Questionnaire (DIGIKOST-FFQ) Assessing Dietary Intake and Other Lifestyle Factors Among Norwegians: Qualitative Evaluation With Focus Group Interviews and Usability Testing JO - JMIR Form Res SP - e35933 VL - 6 IS - 11 KW - digital assessment tool KW - assessment tool KW - food frequency questionnaire KW - food KW - diet KW - nutrition KW - questionnaire KW - focus group KW - interview KW - usability KW - physical activity KW - lifestyle factor KW - dietary intake KW - digital health KW - chronic disease KW - chronic condition KW - health promotion KW - cancer KW - survivor KW - thematic analysis KW - research tool KW - measurement tool N2 - Background: In-person dietary counseling and interventions have shown promising results in changing habits toward healthier lifestyles, but they are costly to implement in large populations. Developing digital tools to assess individual dietary intake and lifestyle with integrated personalized feedback systems may help overcome this challenge. We developed a short digital food frequency questionnaire, known as the DIGIKOST-FFQ, to assess diet and other lifestyle factors based on the Norwegian Food-Based Dietary Guidelines. The DIGIKOST-FFQ includes a personalized feedback system, the DIGIKOST report, that benchmarks diet and lifestyle habits. We used qualitative focus group interviews and usability tests to test the feasibility and usability of the DIGIKOST application. Objective: We aimed to explore attitudes, perceptions, and challenges in completing the DIGIKOST-FFQ. We also investigated perceptions and understanding of the personalized feedback in the DIGIKOST report and the technical flow and usability of the DIGIKOST-FFQ and the DIGIKOST report. Methods: Healthy individuals and cancer survivors were invited to participate in the focus group interviews. The transcripts were analyzed using thematic analysis. Another group of healthy individuals completed the usability testing, which was administered individually by a moderator and 2 observers. The results were analyzed based on predefined assignments and discussion with the participants about the interpretation of the DIGIKOST report and technical flow of the DIGIKOST-FFQ. Results: A total of 20 individuals participated in the focus group interviews, divided into 3 groups of healthy individuals and 3 groups of cancer survivors. Each group consisted of 3 to 4 individuals. Five main themes were investigated: (1) completion time (on average 19.1, SD 8.3, minutes, an acceptable duration), (2) layout (participants reported the DIGIKOST-FFQ was easy to navigate and had clear questions but presented challenges in reporting dietary intake, sedentary time, and physical activity in the last year), (3) questions (the introductory questions on habitual intake worked well), (4) pictures (the pictures were very helpful, but some portion sizes were difficult to differentiate and adding weight in grams would have been helpful), and (5) motivation (users were motivated to obtain personalized feedback). Four individuals participated in the usability testing. The results showed that the users could seamlessly log in, give consent, fill in the DIGIKOST-FFQ, and receive, print, and read the DIGIKOST report. However, parts of the report were perceived as difficult to interpret. Conclusions: The DIGIKOST-FFQ was overall well received by participants, who found it feasible to use; however, some adjustments with regard to reporting dietary intake and lifestyle habits were suggested. The DIGIKOST report with personalized feedback was the main motivation to complete the questionnaire. The results from the usability testing revealed a need for adjustments and updates to make the report easier to read. UR - https://formative.jmir.org/2022/11/e35933 UR - http://dx.doi.org/10.2196/35933 UR - http://www.ncbi.nlm.nih.gov/pubmed/36346647 ID - info:doi/10.2196/35933 ER - TY - JOUR AU - Williams, Hants AU - Steinberg, Sarah AU - Leon, Kendall AU - O?Shea, Catherine AU - Berzin, Robin AU - Hagg, Heather PY - 2022/11/3 TI - Validity of the Parsley Symptom Index?an Electronic Patient-Reported Outcomes Measure Designed for Telehealth: Prospective Cohort Study JO - JMIR Form Res SP - e40063 VL - 6 IS - 11 KW - telemedicine KW - eHealth KW - mHealth KW - web-based N2 - Background: Electronic patient-reported outcomes measures (e-PROMs) are a valuable tool for the monitoring and management of chronic conditions over time. However, there are few validated tools available that capture symptoms across body systems in telehealth settings. The Parsley Symptom Index (PSI) is a recently developed symptom assessment for adults with chronic disease in telehealth settings. A previous study demonstrated the feasibility and acceptability of the PSI in a clinical telehealth setting. Objective: The purpose of this study was to assess convergent validity between the PSI and the self-rated health (SRH) item. Methods: This prospective cohort study took place from January 15, 2021, to December 15, 2021, among a sample of 10,519 adult patients at Parsley Health, a subscription-based holistic medical practice. The PSI and the SRH were completed by patients via an online portal. The association between the PSI and SRH was assessed via polyserial and polychoric correlations, while weighted ? scores provided information related to agreement between the PSI and SRH. Results: From 22,748 responses, there were moderate levels of association (polyserial r=0.51; polychoric r=0.52) and agreement (weighted ?=0.46) between the PSI and SRH. In total, 74.13% (n=16,865) of responses between the PSI and SRH were relatively congruent while 36.17% (n=8229) were literally congruent. Conclusions: The PSI demonstrates convergent validity with the SRH for adults with chronic disease in a telehealth setting. This finding further supports the validation of the PSI in a real-world clinical setting. Although it is conceptually similar to the 1-question SRH, the PSI is a 45-item PROM designed to capture quality of life and specific symptoms by body system. Future studies will compare the PSI to multi-item PROMs. UR - https://formative.jmir.org/2022/11/e40063 UR - http://dx.doi.org/10.2196/40063 UR - http://www.ncbi.nlm.nih.gov/pubmed/36326802 ID - info:doi/10.2196/40063 ER - TY - JOUR AU - Shibasaki, Sanchia AU - Watkin Lui, Felecia AU - Ah Mat, Lynda PY - 2022/11/1 TI - Knowledge Translation and Implementation Planning to Promote Research Governance in Nongovernment Organizations in the Torres Strait: Descriptive Study JO - Interact J Med Res SP - e39213 VL - 11 IS - 2 KW - knowledge translation KW - implementation planning KW - research governance KW - nongovernment organizations KW - nongovernment organisations KW - Aboriginal and Torres Strait Islander N2 - Background: Aboriginal and Torres Strait Islander people in Australia have participated in Western research for decades. When done well, research has resulted in significant benefits and positive impacts on society. However, the primary benefactor of this research has and continues to be researchers, with limited or no research knowledge mobilized for uptake and beneficial use by end users, such as individuals and communities. In 2021, the Torres Strait Islanders Research to Policy and Practice Hub (the Hub) at James Cook University designed and implemented several strategies, including a games-based interactive workshop with representatives from nongovernment organizations (NGOs). Feedback suggests the workshop and associated activities were a success. Objective: We describe knowledge translation (KT) and implementation planning to design and implement strategies to increase awareness and understanding of NGOs in research governance. Methods: This descriptive study involved representatives from NGOs on Thursday Island in the Torres Strait. We collected data from a literature review and informal discussions. We used several models and frameworks to guide our approach and underpin data collection and analysis. Results: Designing and implementing strategies to increase awareness and understanding of NGOs in the Torres Strait to govern research involved several key steps: (1) identifying and defining what needed to change and who needed to change, (2) identifying and mapping barriers and facilitators, (3) selecting the most appropriate strategies to support change, (4) implementing activities, and (5) monitoring and evaluating our approach. We developed a program logic to understand and communicate to others how we would implement activities and what resources would be required to support this process. We drew on several evidence-based KT and implementation models and frameworks to do this. First, a KT planning template was used to inform what evidence we wanted to mobilize, to whom, and for what purpose. Based on this step, we recognized we wanted to bring about change with the target audience, and as such, we drew on the previously mentioned implementation planning models and frameworks. We collated the outcomes from these initial steps. Conclusions: Our KT and implementation practice experience were successful. Encouraging researchers and end users to adopt similar practices requires investment in training and development of KT and implementation practice. This also entails modifying research standards and guidelines to include KT and implementation practice when working with Aboriginal and Torres Strait Islander people and other vulnerable groups, creating incentives for researchers and end users to embed KT and implementation practice in research, and recognizing and rewarding the benefits and impact beyond publication and presentation. UR - https://www.i-jmr.org/2022/2/e39213 UR - http://dx.doi.org/10.2196/39213 UR - http://www.ncbi.nlm.nih.gov/pubmed/36318255 ID - info:doi/10.2196/39213 ER - TY - JOUR AU - Pape, Magdalena AU - Färber, Tanja AU - Seiferth, Caroline AU - Roth, Tanja AU - Schroeder, Stefanie AU - Wolstein, Joerg AU - Herpertz, Stephan AU - Steins-Loeber, Sabine PY - 2022/10/27 TI - A Tailored Gender-Sensitive mHealth Weight Loss Intervention (I-GENDO): Development and Process Evaluation JO - JMIR Form Res SP - e38480 VL - 6 IS - 10 KW - mobile health KW - mHealth KW - eHealth KW - tailoring KW - gender KW - weight loss intervention KW - mobile phone N2 - Background: Given the increase in the prevalence of overweight and obesity worldwide, the number of digital weight loss interventions has also risen. However, these interventions often lack theoretical background and data on long-term effectiveness. The consideration of individual and gender differences in weight-related psychological parameters might enhance the efficacy and sustainability of mobile-based weight loss interventions. Objective: This paper presented an introduction to and the process evaluation of a 12-week gender-sensitive mobile health (mHealth) weight loss intervention (I-GENDO) combining computer-based and self-tailoring features. Methods: Between August 2020 and August 2021, individuals with overweight (BMI 25.0-29.9 kg/m²), those with obesity class I (BMI 30.0-34.9 kg/m²), and those with obesity class II (BMI 35.0-39.9 kg/m²) were recruited to the I-GENDO project, a multicenter study in Germany. The mHealth intervention aimed at targeting individual psychological factors associated with the development and persistence of overweight and obesity (eg, emotional eating) using computer-based tailoring. Moreover, the intervention took a gender-sensitive approach by implementing self-tailoring of gender-targeted module versions. The computer-based assignment of the main modules, self-selection of gender-targeted module versions, and use patterns were evaluated while considering gender. Moreover, gender differences in the usability assessment were analyzed. Results: Data from the intervention arm of the study were processed. A total of 116 individuals with overweight and obesity (77/116, 66.4% women; age mean 47.28, SD 11.66 years; BMI mean 33.58, SD 3.79 kg/m2) were included in the analyses. Overall, the compliance (90/109, 82.6%) and satisfaction with the app (mean 86% approval) were high and comparable with those of other mobile weight loss interventions. The usability of the intervention was rated with 71% (5.0/7.0 points) satisfaction. More women obtained the main module that focused on emotion regulation skills. Most men and women selected women-targeted versions of the main modules. Women used the app more frequently and longer than men. However, women and men did not differ in the progress of use patterns throughout the intervention. Conclusions: We developed a tailored gender-sensitive mHealth weight loss intervention. The usability of and engagement with the intervention were satisfactory, and the overall satisfaction with the intervention was also high. Gender differences must be considered in the evaluation of the effectiveness and sustainability of the intervention. UR - https://formative.jmir.org/2022/10/e38480 UR - http://dx.doi.org/10.2196/38480 UR - http://www.ncbi.nlm.nih.gov/pubmed/36301614 ID - info:doi/10.2196/38480 ER - TY - JOUR AU - Perez, Oriana AU - Kumar Vadathya, Anil AU - Beltran, Alicia AU - Barnett, Matthew R. AU - Hindera, Olivia AU - Garza, Tatyana AU - Musaad, M. Salma AU - Baranowski, Tom AU - Hughes, O. Sheryl AU - Mendoza, A. Jason AU - Sabharwal, Ashutosh AU - Veeraraghavan, Ashok AU - O'Connor, M. Teresia PY - 2022/10/21 TI - The Family Level Assessment of Screen Use?Mobile Approach: Development of an Approach to Measure Children?s Mobile Device Use JO - JMIR Form Res SP - e40452 VL - 6 IS - 10 KW - screen time KW - mobile media apps KW - children KW - mobile phone use KW - tablet use KW - mobile phone N2 - Background: There is a strong association between increased mobile device use and worse dietary habits, worse sleep outcomes, and poor academic performance in children. Self-report or parent-proxy report of children?s screen time has been the most common method of measuring screen time, which may be imprecise or biased. Objective: The objective of this study was to assess the feasibility of measuring the screen time of children on mobile devices using the Family Level Assessment of Screen Use (FLASH)?mobile approach, an innovative method that leverages the existing features of the Android platform. Methods: This pilot study consisted of 2 laboratory-based observational feasibility studies and 2 home-based feasibility studies in the United States. A total of 48 parent-child dyads consisting of a parent and child aged 6 to 11 years participated in the pilot study. The children had to have their own or shared Android device. The laboratory-based studies included a standardized series of tasks while using the mobile device or watching television, which were video recorded. Video recordings were coded by staff for a gold standard comparison. The home-based studies instructed the parent-child dyads to use their mobile device as they typically use it over 3 days. Parents received a copy of the use logs at the end of the study and completed an exit interview in which they were asked to review their logs and share their perceptions and suggestions for the improvement of the FLASH-mobile approach. Results: The final version of the FLASH-mobile approach resulted in user identification compliance rates of >90% for smartphones and >80% for tablets. For laboratory-based studies, a mean agreement of 73.6% (SD 16.15%) was achieved compared with the gold standard (human coding of video recordings) in capturing the target child?s mobile use. Qualitative feedback from parents and children revealed that parents found the FLASH-mobile approach useful for tracking how much time their child spends using the mobile device as well as tracking the apps they used. Some parents revealed concerns over privacy and provided suggestions for improving the FLASH-mobile approach. Conclusions: The FLASH-mobile approach offers an important new research approach to measure children?s use of mobile devices more accurately across several days, even when the child shares the device with other family members. With additional enhancement and validation studies, this approach can significantly advance the measurement of mobile device use among young children. UR - https://formative.jmir.org/2022/10/e40452 UR - http://dx.doi.org/10.2196/40452 UR - http://www.ncbi.nlm.nih.gov/pubmed/36269651 ID - info:doi/10.2196/40452 ER - TY - JOUR AU - Huang, Ching-Shui AU - Tai, Feng-Chuan AU - Lien, Heng-Hui AU - Wong, Jia-Uei AU - Huang, Chi-Cheng PY - 2022/10/19 TI - Long-term Follow-up of Patients With Hernia Using the Hernia-Specific Quality-of-Life Mobile App: Feasibility Questionnaire Study JO - JMIR Form Res SP - e39759 VL - 6 IS - 10 KW - hernia KW - mobile app KW - quality of life KW - Hernia-Specific Quality-of-Life (HERQL) KW - mobile health KW - mHealth KW - app KW - self-management N2 - Background: Hernia repair is one of the most common surgical procedures; however, the long-term outcomes are seldom reported due to incomplete follow-up. Objective: The aim of this study was to examine the use of a mobile app for the long-term follow-up of hernia recurrence, complication, and quality-of-life perception. Methods: A cloud-based corroborative system drove a mobile app with the HERQL (Hernia-Specific Quality-of-Life) questionnaire built in. Patients who underwent hernia repair were identified from medical records, and an invitation to participate in this study was sent through the post. Results: The response rate was 11.89% (311/2615) during the 1-year study period, whereas the recurrence rate was 1.0% (3/311). Causal relationships between symptomatic and functional domains of the HERQL questionnaire were indicated by satisfactory model fit indices and significant regression coefficients derived from structural equational modeling. Regarding patients? last hernia surgeries, 88.7% (276/311) of the patients reported them to be satisfactory or very satisfactory, 68.5% (213/311) of patients reported no discomfort, and 61.1% (190/311) of patients never experienced mesh foreign body sensation. Subgroup analysis for the most commonly used mesh repairs found that mesh plug repair inevitably resulted in worse symptoms and quality-of-life perception from the group with groin hernias. Conclusions: The mobile app has the potential to enhance the quality of care for patients with hernia and facilitate outcomes research with more complete follow-up. UR - https://formative.jmir.org/2022/10/e39759 UR - http://dx.doi.org/10.2196/39759 UR - http://www.ncbi.nlm.nih.gov/pubmed/36260390 ID - info:doi/10.2196/39759 ER - TY - JOUR AU - Agapie, Elena AU - Chang, Katherine AU - Patrachari, Sneha AU - Neary, Martha AU - Schueller, M. Stephen PY - 2022/10/18 TI - Understanding Mental Health Apps for Youth: Focus Group Study With Latinx Youth JO - JMIR Form Res SP - e40726 VL - 6 IS - 10 KW - mental health KW - mental health apps KW - youth KW - child KW - teenager KW - focus group KW - human-centered design KW - mobile health KW - mHealth KW - health app KW - cognitive behavioral therapy KW - CBT KW - perspective KW - qualitative KW - mindfulness KW - digital health tool KW - Latino KW - Latinx KW - mobile phone N2 - Background: An increasing number of mental health apps (MHapps) are being developed for youth. In addition, youth are high users of both technologies and MHapps. However, little is known about their perspectives on MHapps. MHapps might be particularly well suited to reach the youth underserved by traditional mental health resources, and incorporating their perspectives is especially critical to ensure such tools are useful to them. Objective: The goal of this study was to develop and pilot a process for eliciting youth perspectives on MHapps in a structured and collaborative way. We also sought to generate learnings on the perspectives of Latinx youth on MHapps and their use in ways that might facilitate discovery, activation, or engagement in MHapps, especially in Latinx populations. Methods: We created a series of focus groups consisting of 5 sessions. The groups introduced different categories of MHapps (cognitive behavioral therapy apps, mindfulness apps, and miscellaneous apps). Within each category, we selected 4 MHapps that participants chose to use for a week and provided feedback through both between-session and in-session activities. We recruited 5 youths ranging in age from 15 to 21 (mean 18, SD 2.2) years. All the participants identified as Hispanic or Latinx. After completing all 5 focus groups, the participants completed a brief questionnaire to gather their impressions of the apps they had used. Results: Our focus group methodology collected detailed and diverse information about youth perspectives on MHapps. However, we did identify some aspects of our methods that were less successful at engaging the youth, such as our between-session activities. The Latinx youth in our study wanted apps that were accessible, relatable, youth centric, and simple and could be integrated with their offline lives. We also found that the mindfulness apps were viewed most favorably but that the miscellaneous and cognitive behavioral therapy apps were viewed as more impactful. Conclusions: Eliciting youth feedback on MHapps is critical if these apps are going to serve a role in supporting their mental health and well-being. We refined a process for collecting feedback from the youth and identified factors that were important to a set of Latinx youth. Future work could be broader, that is, recruit larger samples of more diverse youth, or deeper, that is, collect more information from each youth around interests, needs, barriers, or facilitators or better understand the various impacts of MHapps by using qualitative and quantitative measures. Nevertheless, this study advances the formative understanding of how the youth, particularly Latinx youth, might be viewing these tools. UR - https://formative.jmir.org/2022/10/e40726 UR - http://dx.doi.org/10.2196/40726 UR - http://www.ncbi.nlm.nih.gov/pubmed/36256835 ID - info:doi/10.2196/40726 ER - TY - JOUR AU - Babrak, Marie Lmar AU - Smakaj, Erand AU - Agac, Teyfik AU - Asprion, Maria Petra AU - Grimberg, Frank AU - der Werf, Van Daan AU - van Ginkel, Willem Erwin AU - Tosoni, David Deniz AU - Clay, Ieuan AU - Degen, Markus AU - Brodbeck, Dominique AU - Natali, Noel Eriberto AU - Schkommodau, Erik AU - Miho, Enkelejda PY - 2022/10/18 TI - RWD-Cockpit: Application for Quality Assessment of Real-world Data JO - JMIR Form Res SP - e29920 VL - 6 IS - 10 KW - real-world data KW - real-world evidence KW - quality assessment KW - application KW - mobile phone N2 - Background: Digital technologies are transforming the health care system. A large part of information is generated as real-world data (RWD). Data from electronic health records and digital biomarkers have the potential to reveal associations between the benefits and adverse events of medicines, establish new patient-stratification principles, expose unknown disease correlations, and inform on preventive measures. The impact for health care payers and providers, the biopharmaceutical industry, and governments is massive in terms of health outcomes, quality of care, and cost. However, a framework to assess the preliminary quality of RWD is missing, thus hindering the conduct of population-based observational studies to support regulatory decision-making and real-world evidence. Objective: To address the need to qualify RWD, we aimed to build a web application as a tool to translate characterization of some quality parameters of RWD into a metric and propose a standard framework for evaluating the quality of the RWD. Methods: The RWD-Cockpit systematically scores data sets based on proposed quality metrics and customizable variables chosen by the user. Sleep RWD generated de novo and publicly available data sets were used to validate the usability and applicability of the web application. The RWD quality score is based on the evaluation of 7 variables: manageability specifies access and publication status; complexity defines univariate, multivariate, and longitudinal data; sample size indicates the size of the sample or samples; privacy and liability stipulates privacy rules; accessibility specifies how the data set can be accessed and to what granularity; periodicity specifies how often the data set is updated; and standardization specifies whether the data set adheres to any specific technical or metadata standard. These variables are associated with several descriptors that define specific characteristics of the data set. Results: To address the need to qualify RWD, we built the RWD-Cockpit web application, which proposes a framework and applies a common standard for a preliminary evaluation of RWD quality across data sets?molecular, phenotypical, and social?and proposes a standard that can be further personalized by the community retaining an internal standard. Applied to 2 different case studies?de novo?generated sleep data and publicly available data sets?the RWD-Cockpit could identify and provide researchers with variables that might increase quality. Conclusions: The results from the application of the framework of RWD metrics implemented in the RWD-Cockpit application suggests that multiple data sets can be preliminarily evaluated in terms of quality using the proposed metrics. The output scores?quality identifiers?provide a first quality assessment for the use of RWD. Although extensive challenges remain to be addressed to set RWD quality standards, our proposal can serve as an initial blueprint for community efforts in the characterization of RWD quality for regulated settings. UR - https://formative.jmir.org/2022/10/e29920 UR - http://dx.doi.org/10.2196/29920 UR - http://www.ncbi.nlm.nih.gov/pubmed/35266872 ID - info:doi/10.2196/29920 ER - TY - JOUR AU - Lanssens, Dorien AU - Thijs, M. Inge AU - Dreesen, Pauline AU - Van Hecke, Ann AU - Coorevits, Pascal AU - Gaethofs, Gitte AU - Derycke, Joyce AU - Tency, Inge PY - 2022/10/11 TI - Information Resources Among Flemish Pregnant Women: Cross-sectional Study JO - JMIR Form Res SP - e37866 VL - 6 IS - 10 KW - pregnancy app KW - mobile app KW - questionnaire KW - pregnancy KW - pregnant KW - mHealth KW - mobile health KW - cross-sectional KW - user need KW - user expectation KW - survey KW - maternal KW - maternity KW - user experience N2 - Background: There has been an exponential growth in the availability of apps, resulting in increased use of pregnancy apps. However, information on resources and use of apps among pregnant women is relatively limited. Objective: The aim of this study is to map the current information resources and the use of pregnancy apps among pregnant women in Flanders. Methods: A cross-sectional study was conducted, using a semistructured survey (April-June 2019) consisting of four different domains: (1) demographics; (2) use of devices; (3) sources of information; and (4) use of pregnancy apps. Women were recruited by social media, flyers, and paper questionnaires at prenatal consultations. Statistical analysis was mainly focused on descriptive statistics. Differences in continuous and categorical variables were tested using independent Student t tests and chi-square tests. Correlations were investigated between maternal characteristics and the women?s responses. Results: In total, 311 women completed the entire questionnaire. Obstetricians were the primary source of information (268/311, 86.2%) for pregnant women, followed by websites/internet (267/311, 85.9%) and apps (233/311, 74.9%). The information that was most searched for was information about the development of the baby (275/311, 88.5%), discomfort/complaints (251/311, 80.7%) and health during pregnancy (248/311, 79.7%), administrative/practical issues (233/311, 74.9%), and breastfeeding (176/311, 56.6%). About half of the women (172/311, 55.3%) downloaded a pregnancy app, and primarily searched app stores (133/311, 43.0%). Pregnant women who are single asked their mothers (22/30, 73.3%) or other family members (13/30, 43.3%) for significantly more information than did married women (mother [in law]: 82/160, 51.3%, P=.02; family members: 35/160, 21.9%, P=.01). Pregnant women with lower education were significantly more likely to have a PC or laptop than those with higher education (72/73, 98.6% vs 203/237, 85.5%; P=.008), and to consult other family members for pregnancy information (30/73, 41.1% vs 55/237, 23.1%; P<.001), but were less likely to consult a gynecologist (70/73, 95.9% vs 198/237, 83.5%; P=.001). They also followed more prenatal sessions (59/73, 80.8% vs 77/237, 32.5%; P=.04) and were more likely to search for information regarding discomfort/complaints during pregnancy (65/73, 89% vs 188/237, 79.5%; P=.02). Compared to multigravida, primigravida were more likely to solicit advice about their pregnancy from other women in their social networks (family members: primigravida 44/109, 40.4% vs multigravida 40/199, 20.1%; P<.001; other pregnant women: primigravida 58/109, 53.2% vs multigravida 80/199, 40.2%; P<.03). Conclusions: Health care professionals need to be aware that apps are important and are a growing source of information for pregnant women. Concerns rise about the quality and safety of those apps, as only a limited number of apps are subjected to an external quality check. Therefore, it is important that health care providers refer to high-quality digital resources and take the opportunity to discuss digital information with pregnant women. UR - https://formative.jmir.org/2022/10/e37866 UR - http://dx.doi.org/10.2196/37866 UR - http://www.ncbi.nlm.nih.gov/pubmed/36222794 ID - info:doi/10.2196/37866 ER - TY - JOUR AU - Lim, Renly AU - Thornton, Christopher AU - Stanek, Jan AU - Ellett, Kalisch Lisa AU - Thiessen, Myra PY - 2022/10/7 TI - Development of a Web-Based System to Report Medication-Related Adverse Effects: Design and Usability Study JO - JMIR Form Res SP - e37605 VL - 6 IS - 10 KW - adverse drug reaction KW - adverse drug event KW - digital health KW - eHealth KW - medication safety KW - mHealth KW - participatory design KW - patient reported outcomes KW - telehealth N2 - Background: Medicine use is the most common intervention in health care. The frequency with which medicines are used means medication-related problems are very common. One common type of medication-related problems is adverse drug events, which are unintended and harmful effects associated with use of medicines. Reporting of adverse drug events to regulatory authorities is important for evaluation of safety of medicines; however, these adverse effects are frequently unreported due to various factors, including lack of consumer-friendly reporting tools. Objective: The aim of this study was to develop a user-friendly digital tool for consumers to report medication-related adverse effects. Methods: The project consisted of 3 parts: (1) content development, including a systematic literature search; (2) iterative system development; and (3) usability testing. The project was guided by participatory design principles, which suggest involving key stakeholders throughout the design process. The first 2 versions were developed as a mobile app and were tested with end users in 2 workshops. The third version was developed as a web application and was tested with consumers who were taking regular medicines. Consumers were asked to complete a modified version of the mHealth app usability questionnaire (MAUQ), an 18-item questionnaire with each item scored using a 7-point Likert scale ranging from 0 (strongly disagree) to 7 (strongly agree). The MAUQ assessed 3 subscales including ease of use (5 items), interface and satisfaction (7 items), and usefulness (6 items). Continuous variables were reported as mean (SD) values, whereas categorical variables were presented as frequencies (percentages). Data analysis was conducted in Microsoft Excel. Results: The content for the system was based on a systematic literature search and short-listing of questions, followed by feedback from project team members and consumers. Feedback from consumers in the 2 workshops were incorporated to improve the functionality, visual design, and stability of the third (current) version. The third version of the system was tested with 26 consumers. A total of 79% (N=307/390) of all responses on the MAUQ were scored 6 or 7, indicating that users generally strongly agree with the usability of the system. When looking at the individual domains, the system had an average score of 6.3 (SD 0.9) for ?ease of use,? 6.3 (SD 0.8) for ?interface and satisfaction,? and 5.2 (SD 1.4) for ?usefulness.? Conclusions: The web-based system for medicine adverse effects reporting is a user-friendly tool developed using an iterative participatory design approach. Future research includes further improving the system, particularly the usefulness of the system, as well as testing the scalability and performance of the system in practice. UR - https://formative.jmir.org/2022/10/e37605 UR - http://dx.doi.org/10.2196/37605 UR - http://www.ncbi.nlm.nih.gov/pubmed/36206034 ID - info:doi/10.2196/37605 ER - TY - JOUR AU - Cluxton-Keller, Fallon AU - Hegel, T. Mark PY - 2022/10/4 TI - A Video-Delivered Family Therapeutic Intervention for Perinatal Women With Clinically Significant Depressive Symptoms and Family Conflict: Indicators of Feasibility and Acceptability JO - JMIR Form Res SP - e41697 VL - 6 IS - 10 KW - family intervention KW - perinatal KW - postnatal KW - depression KW - conflict KW - telehealth KW - family conflict KW - family therapy KW - family therapist KW - video conferencing KW - teleconference KW - teleconferencing KW - telemedicine KW - virtual care KW - mental health KW - psychological health KW - digital health intervention KW - parenting N2 - Background: Variation in family therapeutic intervention fidelity has an impact on outcomes. The use of video conferencing technology can strengthen therapist fidelity to family therapeutic interventions. Objective: This article explores indicators of feasibility and acceptability for a video-delivered family therapeutic intervention for perinatal women with depressive symptoms and family conflict. The objectives of this article are to describe indicators of feasibility, including therapist fidelity to the intervention and technological factors that relate to implementation of the intervention, as well as indicators of acceptability for participants of the intervention. Methods: The data included in this article are from an ongoing randomized trial of the Resilience Enhancement Skills Training (REST) video-delivered family therapeutic intervention. Participant recruitment and data collection are still underway for this clinical trial. Of the 106 participants who are currently enrolled in this study, 54 (51%) have been randomized to receive REST from May 2021 through July 2022. Currently, 2 therapists are delivering the intervention, and the training procedures for therapists are summarized herein. Therapist fidelity to the family therapeutic intervention was assessed in 67 audio recorded sessions. The training procedures were summarized for use of video conferencing technology by therapists and the 54 study participants. Knowledge of the video conferencing technology features was assessed in therapists and study participants by the number of attempts required to use the features. Participant responsiveness to the intervention was assessed by the percentage of attended sessions and percentage of complete homework assignments. Results: To date, both therapists have demonstrated high fidelity to the family therapeutic intervention and used all video conferencing technology features on their first attempt. The current participants required 1 to 3 attempts to use 1 or more of the video conferencing technology features. About 59% (n=32) of the current participants immediately accessed the features on the first attempt. Our results show that perinatal women attended all sessions, and their family members attended 80% of the sessions. To date, participants have completed 80% of the homework assignments. Conclusions: These early findings describe indicators of the feasibility and acceptability of the video-delivered family therapeutic intervention for use with this high priority population. Upon completion of recruitment and data collection, a subsequent article will include a mixed methods process evaluation of the feasibility and acceptability of the video-delivered family therapeutic intervention. Trial Registration: ClinicalTrials.gov NCT04741776; https://clinicaltrials.gov/ct2/show/NCT04741776 UR - https://formative.jmir.org/2022/10/e41697 UR - http://dx.doi.org/10.2196/41697 UR - http://www.ncbi.nlm.nih.gov/pubmed/36194458 ID - info:doi/10.2196/41697 ER - TY - JOUR AU - Munk, Niki AU - Daggy, K. Joanne AU - Evans, Erica AU - Kline, Matthew AU - Slaven, E. James AU - Laws, Brian AU - Foote, Trevor AU - Matthias, S. Marianne AU - Bair, J. Matthew PY - 2022/9/27 TI - Therapist-Delivered Versus Care Ally?Assisted Massage for Veterans With Chronic Neck Pain: Protocol for a Randomized Controlled Trial JO - JMIR Res Protoc SP - e38950 VL - 11 IS - 9 KW - Veterans KW - chronic neck pain KW - integrative medicine KW - whole health KW - modified trial design KW - therapist-delivered versus care ally?assisted massage for Veterans with chronic neck pain KW - TOMCATT N2 - Background: Chronic neck pain (CNP) is prevalent, and it reduces functional status and quality of life and is associated with deleterious psychological outcomes in affected individuals. Despite the desirability of massage and its demonstrated effectiveness in CNP treatment, multiple accessibility barriers exist. Caregiver-applied massage has demonstrated feasibility in various populations but has not been examined in Veterans with CNP or compared in parallel to therapist-delivered massage. Objective: This manuscript described the original study design, lessons learned, and resultant design modifications for the Trial Outcomes for Massage: Care Ally?Assisted Versus Therapist-Treated (TOMCATT) study. Methods: TOMCATT began as a 3-arm, randomized controlled trial of 2 massage delivery approaches for Veterans with CNP with measures collected at baseline, 1 and 3 months after intervention, and 6 months (follow-up). Arm I, care ally?assisted massage, consisted of an in-person, 3.5-hour training workshop, an instructional DVD, a printed treatment manual, and three 30-minute at-home care ally?assisted massage sessions weekly for 3 months. Arm II, therapist-treated massage, consisted of two 60-minute sessions tailored to individual pain experiences and treatments per week for 3 months. The treatments followed a standardized Swedish massage approach. Arm III consisted of wait-list control. Results: Retention and engagement challenges in the first 30 months were significant in the care ally?assisted massage study arm (63% attrition between randomization and treatment initiation) and prompted modification to a 2-arm trial, that is, removing arm I. Conclusions: The modified TOMCATT study successfully launched and exceeded recruitment goals 2.5 months before the necessary COVID-19 pause and is expected to be completed by early 2023. Trial Registration: ClinicalTrials.gov NCT03100539; https://clinicaltrials.gov/ct2/show/NCT03100539 International Registered Report Identifier (IRRID): DERR1-10.2196/38950 UR - https://www.researchprotocols.org/2022/9/e38950 UR - http://dx.doi.org/10.2196/38950 UR - http://www.ncbi.nlm.nih.gov/pubmed/36166287 ID - info:doi/10.2196/38950 ER - TY - JOUR AU - Hudson, Georgie AU - Negbenose, Esther AU - Neary, Martha AU - Jansli, M. Sonja AU - Schueller, M. Stephen AU - Wykes, Til AU - Jilka, Sagar PY - 2022/9/23 TI - Comparing Professional and Consumer Ratings of Mental Health Apps: Mixed Methods Study JO - JMIR Form Res SP - e39813 VL - 6 IS - 9 KW - well-being KW - apps KW - patient and public involvement KW - coproduction KW - mental health KW - service user KW - technology KW - mobile health KW - mHealth KW - digital KW - mobile phone N2 - Background: As the number of mental health apps has grown, increasing efforts have been focused on establishing quality tailored reviews. These reviews prioritize clinician and academic views rather than the views of those who use them, particularly those with lived experiences of mental health problems. Given that the COVID-19 pandemic has increased reliance on web-based and mobile mental health support, understanding the views of those with mental health conditions is of increasing importance. Objective: This study aimed to understand the opinions of people with mental health problems on mental health apps and how they differ from established ratings by professionals. Methods: A mixed methods study was conducted using a web-based survey administered between December 2020 and April 2021, assessing 11 mental health apps. We recruited individuals who had experienced mental health problems to download and use 3 apps for 3 days and complete a survey. The survey consisted of the One Mind PsyberGuide Consumer Review Questionnaire and 2 items from the Mobile App Rating Scale (star and recommendation ratings from 1 to 5). The consumer review questionnaire contained a series of open-ended questions, which were thematically analyzed and using a predefined protocol, converted into binary (positive or negative) ratings, and compared with app ratings by professionals and star ratings from app stores. Results: We found low agreement between the participants? and professionals? ratings. More than half of the app ratings showed disagreement between participants and professionals (198/372, 53.2%). Compared with participants, professionals gave the apps higher star ratings (3.58 vs 4.56) and were more likely to recommend the apps to others (3.44 vs 4.39). Participants? star ratings were weakly positively correlated with app store ratings (r=0.32, P=.01). Thematic analysis found 11 themes, including issues of user experience, ease of use and interactivity, privacy concerns, customization, and integration with daily life. Participants particularly valued certain aspects of mental health apps, which appear to be overlooked by professional reviewers. These included functions such as the ability to track and measure mental health and providing general mental health education. The cost of apps was among the most important factors for participants. Although this is already considered by professionals, this information is not always easily accessible. Conclusions: As reviews on app stores and by professionals differ from those by people with lived experiences of mental health problems, these alone are not sufficient to provide people with mental health problems with the information they desire when choosing a mental health app. App rating measures must include the perspectives of mental health service users to ensure ratings represent their priorities. Additional work should be done to incorporate the features most important to mental health service users into mental health apps. UR - https://formative.jmir.org/2022/9/e39813 UR - http://dx.doi.org/10.2196/39813 UR - http://www.ncbi.nlm.nih.gov/pubmed/36149733 ID - info:doi/10.2196/39813 ER - TY - JOUR AU - Oh, Taek Kyue AU - Ko, Jisu AU - Shin, Jaemyung AU - Ko, Minsam PY - 2022/9/21 TI - Using Wake-Up Tasks for Morning Behavior Change: Development and Usability Study JO - JMIR Form Res SP - e39497 VL - 6 IS - 9 KW - health app design KW - morning behavior change KW - wake-up task KW - mobile alarm KW - productivity N2 - Background: Early morning behaviors between waking up and beginning daily work can develop into productive habits. However, sleep inertia limits the level of human ability immediately after waking, lowering a person?s motivation and available time for productive morning behavior. Objective: This study explores a design for morning behavior change using a wake-up task, a simple assignment the user needs to finish before alarm dismissal. Specifically, we set two research objectives: (1) exploring key factors that relate to morning behavior performance, including the use of wake-up tasks in an alarm app and (2) understanding the general practice of affecting morning behavior change by implementing wake-up tasks. Methods: We designed and implemented an apparatus that provides wake-up task alarms and facilities for squat exercises. We recruited 36 participants to perform squat exercises in the early morning using the wake-up tasks for 2 weeks. First, we conducted a generalized estimating equation (GEE) analysis for the first research objective. Next, we conducted a thematic analysis of the postsurvey answers to identify key themes about morning behavior change with the wake-up tasks for the second objective. Results: The use of wake-up tasks was significantly associated with both the completion of the target behavior (math task: P=.005; picture task: P<.001) and the elapsed time (picture task: P=.08); the time to alarm dismissal was significantly related to the elapsed time to completion (P<.001). Moreover, the theory of planned behavior (TPB) variables, common factors for behavior change, were significant, but their magnitudes and directions differed slightly from the other domains. Furthermore, the survey results reveal how the participants used the wake-up tasks and why they were effective for morning behavior performance. Conclusions: The results reveal the effectiveness of wake-up tasks in accomplishing the target morning behavior and address key factors for morning behavior change, such as (1) waking up on time, (2) escaping from sleep inertia, and (3) quickly starting the desired target behavior. UR - https://formative.jmir.org/2022/9/e39497 UR - http://dx.doi.org/10.2196/39497 UR - http://www.ncbi.nlm.nih.gov/pubmed/36129742 ID - info:doi/10.2196/39497 ER - TY - JOUR AU - Bin, Jia Kaio AU - De Pretto, Ramos Lucas AU - Sanchez, Beltrame Fabio AU - Battistella, Rizzo Linamara PY - 2022/9/15 TI - Digital Platform to Continuously Monitor Patients Using a Smartwatch: Preliminary Report JO - JMIR Form Res SP - e40468 VL - 6 IS - 9 KW - smartwatch KW - digital health KW - telemedicine KW - wearable KW - telemonitoring KW - mobile health KW - digital platform KW - clinical intervention KW - sensitive data KW - clinical trial N2 - Background: Monitoring vital signs such as oximetry, blood pressure, and heart rate is important to follow the evolution of patients. Smartwatches are a revolution in medicine allowing the collection of such data in a continuous and organic way. However, it is still a challenge to make this information available to health care professionals to make decisions during clinical follow-up. Objective: This study aims to build a digital solution that displays vital sign data from smartwatches, collected remotely, continuously, reliably, and from multiple users, with trigger warnings when abnormal results are identified. Methods: This is a single-center prospective study following the guidelines ?Evaluating digital health products? from the UK Health Security Agency. A digital platform with 3 different applications was created to capture and display data from the mobile phones of volunteers with smartwatches. We selected 80 volunteers who were followed for 24 weeks each, and the synchronization interval between the smartwatch and digital solution was recorded for each vital sign collected. Results: In 14 weeks of project progress, we managed to recruit 80 volunteers, with 68 already registered in the digital solution. More than 2.8 million records have already been collected, without system downtime. Less than 5% of continuous heart rate measurements (bpm) were synchronized within 2 hours. However, approximately 70% were synchronized in less than 24 hours, and 90% were synchronized in less than 119 hours. Conclusions: The digital solution is working properly in its role of displaying data collected from smartwatches. Vital sign values are being monitored by the research team as part of the monitoring of volunteers. Although the digital solution proved unsuitable for monitoring urgent events, it is more than suitable for use in outpatient clinical use. This digital solution, which is based on cloud technology, can be applied in the future for telemonitoring in regions lacking health care professionals. Accuracy and reliability studies still need to be performed at the end of the 24-week follow-up. UR - https://formative.jmir.org/2022/9/e40468 UR - http://dx.doi.org/10.2196/40468 UR - http://www.ncbi.nlm.nih.gov/pubmed/36107471 ID - info:doi/10.2196/40468 ER - TY - JOUR AU - MacKinnon, Madison AU - Moloney, Max AU - Bullock, Emma AU - Morra, Alison AU - To, Teresa AU - Lemiere, Catherine AU - Lougheed, Diane M. PY - 2022/9/15 TI - Implementation of a Work-Related Asthma Screening Questionnaire in Clinical Settings: Multimethods Study JO - JMIR Form Res SP - e37503 VL - 6 IS - 9 KW - work-related asthma KW - asthma KW - dissemination KW - implementation KW - e-tools KW - barriers KW - limitations KW - electronic medical records KW - EMRs KW - knowledge translation KW - mobile phone N2 - Background: A work-related asthma (WRA) screening questionnaire is currently being validated for implementation in clinical settings. To minimize barriers to integrating tools into clinical practice, a discussion of strategies for the implementation of the questionnaire has begun. Objective: This study aimed to understand the benefits, feasibility, barriers, and limitations of implementing the Work-related Asthma Screening Questionnaire?Long version (WRASQ[L]) and asthma e-tools in clinical settings and propose dissemination and implementation strategies for the WRASQ(L). Methods: This study was conducted in Kingston, Ontario, Canada, from September 2019 to August 2021. A workshop and 2 questionnaires were used to understand the benefits of and barriers to implementing the questionnaire in clinical settings. An expert advisory committee was established to develop the implementation and dissemination strategies. Workshops were semistructured and used thematic qualitative analysis to identify themes that provided an understanding of the benefits and limitations of and barriers to using the WRASQ(L), and e-tools in general, in clinical settings. Workshop participants included patients and health care providers, including physicians, nurses, and asthma educators, who were implementation specialists and expert electronic medical record users. A questionnaire focusing on providers? knowledge and awareness of WRA and another focusing on WRASQ(L) feedback was administered at the workshops. Advisory committee members from relevant stakeholders met 3 times to strategize implementation opportunities. Results: A total of 6 themes were identified in the workshop: involving and addressing patient needs, novel data collection, knowledge translation, time considerations, functional and practical barriers, and human limitations. Questionnaire responses yielded positive feedback on the utility of the WRASQ(L) in clinical settings. All participants agreed that it is an easy way of collecting information on occupational and exposure history and could prompt a discussion between the health care provider and patient on how the workplace and exposures could affect one?s asthma, increase awareness of WRA in patients and providers, and increase awareness of exposures in the workplace. Implementation and dissemination strategies were generated with input from the advisory committee. Conclusions: Stakeholders and workshop participants consider the WRASQ(L) to be a useful tool that satisfies many provider needs in their clinical settings. Once validated, dissemination strategies will include developing educational materials that include the WRASQ(L), linking the questionnaire to stakeholder websites or e-toolkits, translation into other languages, leveraging health care and research networks, conference presentations, and peer-reviewed publications. Implementation strategies will include integration into electronic medical records; designing multifaceted interventions; and targeting nontraditional settings such as workplaces, pharmacies, and research settings. The WRASQ(L) addresses many benefits of and barriers to implementation, as identified in the workshop themes. These themes will guide future implementation and dissemination strategies, noting that human limitations identified in providers and patients will need to be overcome for successful implementation. UR - https://formative.jmir.org/2022/9/e37503 UR - http://dx.doi.org/10.2196/37503 UR - http://www.ncbi.nlm.nih.gov/pubmed/35964327 ID - info:doi/10.2196/37503 ER - TY - JOUR AU - Huang, Haley AU - Aschettino, Sofia AU - Lari, Nasim AU - Lee, Ting-Hsuan AU - Rosenberg, Stothers Sarah AU - Ng, Xinyi AU - Muthuri, Stella AU - Bakshi, Anirudh AU - Bishop, Korrin AU - Ezzeldin, Hussein PY - 2022/9/14 TI - A Versatile and Scalable Platform That Streamlines Data Collection for Patient-Centered Studies: Usability and Feasibility Study JO - JMIR Form Res SP - e38579 VL - 6 IS - 9 KW - mobile app KW - patient experience data KW - data-collection app KW - mobile phone KW - usability KW - mHealth app KW - feasibility KW - user centered KW - eHealth KW - patient-generated data N2 - Background: The Food and Drug Administration Center for Biologics Evaluation and Research (CBER) established the Biologics Effectiveness and Safety (BEST) Initiative with several objectives, including the expansion and enhancement of CBER?s access to fit-for-purpose data sources, analytics, tools, and infrastructures to improve the understanding of patient experiences with conditions related to CBER-regulated products. Owing to existing challenges in data collection, especially for rare disease research, CBER recognized the need for a comprehensive platform where study coordinators can engage with study participants and design and deploy studies while patients or caregivers could enroll, consent, and securely participate as well. Objective: This study aimed to increase awareness and describe the design, development, and novelty of the Survey of Health and Patient Experience (SHAPE) platform, its functionality and application, quality improvement efforts, open-source availability, and plans for enhancement. Methods: SHAPE is hosted in a Google Cloud environment and comprises 3 parts: the administrator application, participant app, and application programming interface. The administrator can build a study comprising a set of questionnaires and self-report entries through the app. Once the study is deployed, the participant can access the app, consent to the study, and complete its components. To build SHAPE to be scalable and flexible, we leveraged the open-source software development kit, Ionic Framework. This enabled the building and deploying of apps across platforms, including iOS, Android, and progressive web applications, from a single codebase by using standardized web technologies. SHAPE has been integrated with a leading Health Level 7 (HL7®) Fast Healthcare Interoperability Resources (FHIR®) application programming interface platform, 1upHealth, which allows participants to consent to 1-time data pull of their electronic health records. We used an agile-based process that engaged multiple stakeholders in SHAPE?s design and development. Results: SHAPE allows study coordinators to plan, develop, and deploy questionnaires to obtain important end points directly from patients or caregivers. Electronic health record integration enables access to patient health records, which can validate and enhance the accuracy of data-capture methods. The administrator can then download the study data into HL7® FHIR®?formatted JSON files. In this paper, we illustrate how study coordinators can use SHAPE to design patient-centered studies. We demonstrate its broad applicability through a hypothetical type 1 diabetes cohort study and an ongoing pilot study on metachromatic leukodystrophy to implement best practices for designing a regulatory-grade natural history study for rare diseases. Conclusions: SHAPE is an intuitive and comprehensive data-collection tool for a variety of clinical studies. Further customization of this versatile and scalable platform allows for multiple use cases. SHAPE can capture patient perspectives and clinical data, thereby providing regulators, clinicians, researchers, and patient advocacy organizations with data to inform drug development and improve patient outcomes. UR - https://formative.jmir.org/2022/9/e38579 UR - http://dx.doi.org/10.2196/38579 UR - http://www.ncbi.nlm.nih.gov/pubmed/36103218 ID - info:doi/10.2196/38579 ER - TY - JOUR AU - Maleki, Ghobad AU - Zhuparris, Ahnjili AU - Koopmans, Ingrid AU - Doll, J. Robert AU - Voet, Nicoline AU - Cohen, Adam AU - van Brummelen, Emilie AU - Groeneveld, Jan Geert AU - De Maeyer, Joris PY - 2022/9/13 TI - Objective Monitoring of Facioscapulohumeral Dystrophy During Clinical Trials Using a Smartphone App and Wearables: Observational Study JO - JMIR Form Res SP - e31775 VL - 6 IS - 9 KW - facioscapulohumeral dystrophy KW - FSHD KW - smartphone KW - wearables KW - machine learning KW - classification KW - mobile phone N2 - Background: Facioscapulohumeral dystrophy (FSHD) is a progressive muscle dystrophy disorder leading to significant disability. Currently, FSHD symptom severity is assessed by clinical assessments such as the FSHD clinical score and the Timed Up-and-Go test. These assessments are limited in their ability to capture changes continuously and the full impact of the disease on patients? quality of life. Real-world data related to physical activity, sleep, and social behavior could potentially provide additional insight into the impact of the disease and might be useful in assessing treatment effects on aspects that are important contributors to the functioning and well-being of patients with FSHD. Objective: This study investigated the feasibility of using smartphones and wearables to capture symptoms related to FSHD based on a continuous collection of multiple features, such as the number of steps, sleep, and app use. We also identified features that can be used to differentiate between patients with FSHD and non-FSHD controls. Methods: In this exploratory noninterventional study, 58 participants (n=38, 66%, patients with FSHD and n=20, 34%, non-FSHD controls) were monitored using a smartphone monitoring app for 6 weeks. On the first and last day of the study period, clinicians assessed the participants? FSHD clinical score and Timed Up-and-Go test time. Participants installed the app on their Android smartphones, were given a smartwatch, and were instructed to measure their weight and blood pressure on a weekly basis using a scale and blood pressure monitor. The user experience and perceived burden of the app on participants? smartphones were assessed at 6 weeks using a questionnaire. With the data collected, we sought to identify the behavioral features that were most salient in distinguishing the 2 groups (patients with FSHD and non-FSHD controls) and the optimal time window to perform the classification. Results: Overall, the participants stated that the app was well tolerated, but 67% (39/58) noticed a difference in battery life using all 6 weeks of data, we classified patients with FSHD and non-FSHD controls with 93% accuracy, 100% sensitivity, and 80% specificity. We found that the optimal time window for the classification is the first day of data collection and the first week of data collection, which yielded an accuracy, sensitivity, and specificity of 95.8%, 100%, and 94.4%, respectively. Features relating to smartphone acceleration, app use, location, physical activity, sleep, and call behavior were the most salient features for the classification. Conclusions: Remotely monitored data collection allowed for the collection of daily activity data in patients with FSHD and non-FSHD controls for 6 weeks. We demonstrated the initial ability to detect differences in features in patients with FSHD and non-FSHD controls using smartphones and wearables, mainly based on data related to physical and social activity. Trial Registration: ClinicalTrials.gov NCT04999735; https://www.clinicaltrials.gov/ct2/show/NCT04999735 UR - https://formative.jmir.org/2022/9/e31775 UR - http://dx.doi.org/10.2196/31775 UR - http://www.ncbi.nlm.nih.gov/pubmed/36098990 ID - info:doi/10.2196/31775 ER - TY - JOUR AU - Shuldiner, Jennifer AU - Srinivasan, Diya AU - Hall, N. Justin AU - May, R. Carl AU - Desveaux, Laura PY - 2022/9/12 TI - Implementing a Virtual Emergency Department: Qualitative Study Using the Normalization Process Theory JO - JMIR Hum Factors SP - e39430 VL - 9 IS - 3 KW - virtual care KW - emergency department KW - Normalization Process Theory N2 - Background: COVID-19 necessitated the rapid implementation and uptake of virtual health care; however, virtual care?s potential role remains unclear in the urgent care setting. In December 2020, the first virtual emergency department (ED) in the Greater Toronto Area was piloted at Sunnybrook Health Sciences Centre by connecting patients to emergency physicians through an online portal. Objective: This study aims to understand whether and how ED physicians were able to integrate a virtual ED alongside in-person operations. Methods: We conducted semistructured interviews with ED physicians guided by the Normalization Process Theory (NPT). The NPT provides a framework to understand how individuals and teams navigate the process of embedding new models of care as part of normal practice. All physicians who had worked within the virtual ED model were invited to participate. Data were analyzed using a combination of inductive and deductive techniques informed by the NPT. Results: A total of 14 physicians were interviewed. Participant experiences were categorized into 1 of 2 groups: 1 group moved to normalize the virtual ED in practice, while the other described barriers to routine adoption. These groups differed in their perception of the patient benefits as well as the perceived role in the virtual ED. The group that normalized the virtual ED model saw value for patients (coherence) and was motivated by patient satisfaction witnessed (reflexive monitoring) at the end of the virtual appointment. By contrast, the other group did not find virtual ED work reflective of the perceived role of urgent care (cognitive participation) and felt their skills as ED physicians were underutilized. The limited ability to examine patients and a sense that patient issues were not fully resolved at the end of the virtual appointment caused frustration among the second group. Conclusions: As further digital integration within the health care system occurs, it will be essential to support the evolution of staff skill sets to ensure physicians are satisfied with the care they are providing to their patients, while also ensuring the technology and process are efficient. UR - https://humanfactors.jmir.org/2022/3/e39430 UR - http://dx.doi.org/10.2196/39430 UR - http://www.ncbi.nlm.nih.gov/pubmed/36094801 ID - info:doi/10.2196/39430 ER - TY - JOUR AU - McMahan, M. Vanessa AU - Arenander, Justine AU - Matheson, Tim AU - Lambert, M. Audrey AU - Brennan, Sarah AU - Green, C. Traci AU - Walley, Y. Alexander AU - Coffin, O. Phillip PY - 2022/9/7 TI - ?There?s No Heroin Around Anymore. It?s All Fentanyl.? Adaptation of an Opioid Overdose Prevention Counseling Approach to Address Fentanyl Overdose: Formative Study JO - JMIR Form Res SP - e37483 VL - 6 IS - 9 KW - opioid overdose KW - fentanyl KW - motivational interviewing KW - naloxone KW - assessment, decision, adaptation, production, topical experts, integration, training, and testing KW - ADAPT-ITT KW - theater testing N2 - Background: Drug overdose mortality continues to increase, now driven by fentanyl. Prevention tools such as naloxone and medications to treat opioid use disorder are not sufficient to control overdose rates; additional strategies are urgently needed. Objective: We sought to adapt a behavioral intervention to prevent opioid overdose (repeated-dose behavioral intervention to reduce opioid overdose [REBOOT]) that had been successfully piloted in San Francisco, California, United States, to the setting of Boston, Massachusetts, United States, and the era of fentanyl for a full efficacy trial. Methods: We used the assessment, decision, adaptation, production, topical experts, integration, training, and testing (ADAPT-ITT) framework for intervention adaptation. We first identified opioid overdose survivors who were actively using opioids as the population of interest and REBOOT as the intervention to be adapted. We then performed theater testing and elicited feedback with 2 focus groups (n=10) in Boston in 2018. All participants had used opioids that were not prescribed to them in the past year and experienced an opioid overdose during their lifetime. We incorporated focus group findings into our initial draft of the adapted REBOOT intervention. The adapted intervention was reviewed by 3 topical experts, and their feedback was integrated into a subsequent draft. We trained study staff on the intervention and made final refinements based on internal piloting. This paper describes the overall ADAPT-ITT process for intervention adaptation, as well as a qualitative analysis of the focus groups. Working independently, 2 authors (VMM and JA) reviewed the focus group transcripts and coded them for salient and common themes using the constant comparison method, meeting to discuss any discrepancies until consensus was reached. Codes and themes were then mapped onto the REBOOT counseling steps. Results: Focus group findings contributed to substantial changes in the counseling intervention to better address fentanyl overdose risk. Participants described the widespread prevalence of fentanyl and said that, although they tried to avoid it, avoidance was becoming impossible. Using alone and lower opioid tolerance were identified as contributors to overdose risk. Slow shots or tester shots were acceptable and considered effective to reduce risk. Naloxone was considered an effective reversal strategy. Although calling emergency services was not ruled out, participants described techniques to prevent the arrival of police on the scene. Expert review and internal piloting improved the intervention manual through increased participant centeredness, clarity, and usability. Conclusions: We successfully completed the ADAPT-ITT approach for an overdose prevention intervention, using theater testing with people who use opioids to incorporate the perspectives of people who use drugs into a substance use intervention. In the current crisis, overdose prevention strategies must be adapted to the context of fentanyl, and innovative strategies must be deployed, including behavioral interventions. Trial Registration: ClinicalTrials.gov NCT03838510; https://clinicaltrials.gov/ct2/show/NCT03838510 UR - https://formative.jmir.org/2022/9/e37483 UR - http://dx.doi.org/10.2196/37483 UR - http://www.ncbi.nlm.nih.gov/pubmed/36069781 ID - info:doi/10.2196/37483 ER - TY - JOUR AU - Ferrell, J. Brian AU - Raskin, E. Sarah AU - Zimmerman, B. Emily AU - Timberline, H. David AU - McInnes, T. Bridget AU - Krist, H. Alex PY - 2022/9/6 TI - Attention-Based Models for Classifying Small Data Sets Using Community-Engaged Research Protocols: Classification System Development and Validation Pilot Study JO - JMIR Form Res SP - e32460 VL - 6 IS - 9 KW - data augmentation KW - BERT KW - transformer-based models KW - text classification KW - community engagement KW - prototype KW - IRB research KW - community-engaged research KW - participatory research KW - deep learning N2 - Background: Community-engaged research (CEnR) is a research approach in which scholars partner with community organizations or individuals with whom they share an interest in the study topic, typically with the goal of supporting that community?s well-being. CEnR is well-established in numerous disciplines including the clinical and social sciences. However, universities experience challenges reporting comprehensive CEnR metrics, limiting the development of appropriate CEnR infrastructure and the advancement of relationships with communities, funders, and stakeholders. Objective: We propose a novel approach to identifying and categorizing community-engaged studies by applying attention-based deep learning models to human participants protocols that have been submitted to the university?s institutional review board (IRB). Methods: We manually classified a sample of 280 protocols submitted to the IRB using a 3- and 6-level CEnR heuristic. We then trained an attention-based bidirectional long short-term memory unit (Bi-LSTM) on the classified protocols and compared it to transformer models such as Bidirectional Encoder Representations From Transformers (BERT), Bio + Clinical BERT, and Cross-lingual Language Model?Robustly Optimized BERT Pre-training Approach (XLM-RoBERTa). We applied the best-performing models to the full sample of unlabeled IRB protocols submitted in the years 2013-2019 (n>6000). Results: Although transfer learning is superior, receiving a 0.9952 evaluation F1 score for all transformer models implemented compared to the attention-based Bi-LSTM (between 48%-80%), there were key issues with overfitting. This finding is consistent across several methodological adjustments: an augmented data set with and without cross-validation, an unaugmented data set with and without cross-validation, a 6-class CEnR spectrum, and a 3-class one. Conclusions: Transfer learning is a more viable method than the attention-based bidirectional-LSTM for differentiating small data sets characterized by the idiosyncrasies and variability of CEnR descriptions used by principal investigators in research protocols. Despite these issues involving overfitting, BERT and the other transformer models remarkably showed an understanding of our data unlike the attention-based Bi-LSTM model, promising a more realistic path toward solving this real-world application. UR - https://formative.jmir.org/2022/9/e32460 UR - http://dx.doi.org/10.2196/32460 UR - http://www.ncbi.nlm.nih.gov/pubmed/36066925 ID - info:doi/10.2196/32460 ER - TY - JOUR AU - Chaniaud, Noémie AU - Sagnier, Camille AU - Loup-Escande, Emilie PY - 2022/8/31 TI - Translation and Validation Study of the French Version of the eHealth Literacy Scale: Web-Based Survey on a Student Population JO - JMIR Form Res SP - e36777 VL - 6 IS - 8 KW - eHealth Literacy Scale KW - eHEALS KW - eHealth literacy KW - transcultural validation process KW - Health Literacy Survey?Europe KW - HLS-EU N2 - Background: eHealth literacy is emerging as a crucial concept for promoting patient self-management in an overloaded hospital system. However, to the best of our knowledge, no tool currently exists to measure the level of eHealth literacy among French-speaking people. The eHealth Literacy Scale (eHEALS) is an easy-to-administer 8-item questionnaire (5-point Likert scale, ranging from strongly disagree to strongly agree) that has already been translated into many languages. Currently, it is the most cited questionnaire in the literature. Objective: The aim of this study was to translate eHEALS to French and validate the French version of eHEALS (F-eHEALS). Methods: The validation of the F-eHEALS scale followed the 5 steps of the transcultural validation method: double reverse translation, validation by a committee of experts (n=4), pretest measurement to check the clarity of the items (n=22), administration of the scale in French via a web-based quantitative study combined with two other questionnaires (Health Literacy Survey-Europe?16 and Patient Activation Measure?13; N=328 students), and finally test-retest (n=78) to check the temporal stability of the measurements obtained from the scale. Results: The results obtained for the measurement of factor structure, internal consistency, and temporal stability (intraclass correlation coefficient=0.84; 95% CI 0.76-0.9; F77,77=6.416; P<.001) prove the validity and fidelity of the proposed scale. The internal consistency of F-eHEALS was estimated by Cronbach ? of .89. The factor analysis with varimax rotation used to validate the construct showed a 2-factor scale. The effect of the construct was analyzed using 3 hypotheses related to the theory. The F-eHEALS score was correlated with the Health Literacy Survey-Europe?16 score (r=0.34; P<.001) and the Patient Activation Measure?13 score (r=0.31; P<.001). Conclusions: F-eHEALS is consistent with the original version. It presents adequate levels of validity and fidelity. This 2D scale will need to be generalized to other populations in a French-speaking context. Finally, a version taking into account collaborative applications (ie, Health 2.0; eg, Digital Health Literacy Instrument scale) should be considered on the basis of this study. UR - https://formative.jmir.org/2022/8/e36777 UR - http://dx.doi.org/10.2196/36777 UR - http://www.ncbi.nlm.nih.gov/pubmed/36044264 ID - info:doi/10.2196/36777 ER - TY - JOUR AU - Qasrawi, Radwan AU - Vicuna Polo, Paola Stephanny AU - Abu Al-Halawa, Diala AU - Hallaq, Sameh AU - Abdeen, Ziad PY - 2022/8/31 TI - Assessment and Prediction of Depression and Anxiety Risk Factors in Schoolchildren: Machine Learning Techniques Performance Analysis JO - JMIR Form Res SP - e32736 VL - 6 IS - 8 KW - machine learning KW - depression KW - anxiety KW - schoolchildren KW - school-age children KW - children KW - youth KW - young adult KW - transition-aged youth KW - early childhood education KW - prediction KW - random forest N2 - Background: Depression and anxiety symptoms in early childhood have a major effect on children?s mental health growth and cognitive development. The effect of mental health problems on cognitive development has been studied by researchers for the last 2 decades. Objective: In this paper, we sought to use machine learning techniques to predict the risk factors associated with schoolchildren?s depression and anxiety. Methods: The study sample consisted of 3984 students in fifth to ninth grades, aged 10-15 years, studying at public and refugee schools in the West Bank. The data were collected using the health behaviors schoolchildren questionnaire in the 2013-2014 academic year and analyzed using machine learning to predict the risk factors associated with student mental health symptoms. We used 5 machine learning techniques (random forest [RF], neural network, decision tree, support vector machine [SVM], and naive Bayes) for prediction. Results: The results indicated that the SVM and RF models had the highest accuracy levels for depression (SVM: 92.5%; RF: 76.4%) and anxiety (SVM: 92.4%; RF: 78.6%). Thus, the SVM and RF models had the best performance in classifying and predicting the students? depression and anxiety. The results showed that school violence and bullying, home violence, academic performance, and family income were the most important factors affecting the depression and anxiety scales. Conclusions: Overall, machine learning proved to be an efficient tool for identifying and predicting the associated factors that influence student depression and anxiety. The machine learning techniques seem to be a good model for predicting abnormal depression and anxiety symptoms among schoolchildren, so the deployment of machine learning within the school information systems might facilitate the development of health prevention and intervention programs that will enhance students? mental health and cognitive development. UR - https://formative.jmir.org/2022/8/e32736 UR - http://dx.doi.org/10.2196/32736 UR - http://www.ncbi.nlm.nih.gov/pubmed/35665695 ID - info:doi/10.2196/32736 ER - TY - JOUR AU - Tonn, Peter AU - Seule, Lea AU - Degani, Yoav AU - Herzinger, Shani AU - Klein, Amit AU - Schulze, Nina PY - 2022/8/30 TI - Digital Content-Free Speech Analysis Tool to Measure Affective Distress in Mental Health: Evaluation Study JO - JMIR Form Res SP - e37061 VL - 6 IS - 8 KW - mobile health KW - mHealth KW - depression KW - assessment KW - voice analysis KW - evaluation KW - speech KW - speech analysis KW - tool KW - distress KW - mental health KW - mood KW - diagnosis KW - measurement KW - questionnaire KW - mobile phone N2 - Background: Mood disorders and depression are pervasive and significant problems worldwide. These represent severe health and emotional impairments for individuals and a considerable economic and social burden. Therefore, fast and reliable diagnosis and appropriate treatment are of great importance. Verbal communication can clarify the speaker?s mental state?regardless of the content, via speech melody, intonation, and so on. In both everyday life and clinical conditions, a listener with appropriate previous knowledge or a trained specialist can grasp helpful knowledge about the speaker's psychological state. Using automated speech analysis for the assessment and tracking of patients with mental health issues opens up the possibility of remote, automatic, and ongoing evaluation when used with patients? smartphones, as part of the current trends toward the increasing use of digital and mobile health tools. Objective: The primary aim of this study is to evaluate the measurements of the presence or absence of depressive mood in participants by comparing the analysis of noncontentual speech parameters with the results of the Patient Health Questionnaire-9. Methods: This proof-of-concept study included participants in different affective phases (with and without depression). The inclusion criteria included a neurological or psychiatric diagnosis made by a specialist and fluent use of the German language. The measuring instrument was the VoiceSense digital voice analysis tool, which enables the analysis of 200 specific speech parameters based on machine learning and the assessment of the findings using Patient Health Questionnaire-9. Results: A total of 292 psychiatric and voice assessments were performed with 163 participants (males: n=47, 28.8%) aged 15 to 82 years. Of the 163 participants, 87 (53.3%) were not depressed at the time of assessment, and 88 (53.9%) participants had clinically mild to moderate depressive phases. Of the 163 participants, 98 (32.5%) showed subsyndromal symptoms, and 19 (11.7%) participants were severely depressed. In the speech analysis, a clear differentiation between the individual depressive levels, as seen in the Patient Health Questionnaire-9, was also shown, especially the clear differentiation between nondepressed and depressed participants. The study showed a Pearson correlation of 0.41 between clinical assessment and noncontentual speech analysis (P<.001). Conclusions: The use of speech analysis shows a high level of accuracy, not only in terms of the general recognition of a clinically relevant depressive state in the participants. Instead, there is a high degree of agreement regarding the extent of depressive impairment with the assessment of experienced clinical practitioners. From our point of view, the application of the noncontentual analysis system in everyday clinical practice makes sense, especially with the idea of a quick and unproblematic assessment of the state of mind, which can even be carried out without personal contact. Trial Registration: ClinicalTrials.gov NCT03700008; https://clinicaltrials.gov/ct2/show/NCT03700008 UR - https://formative.jmir.org/2022/8/e37061 UR - http://dx.doi.org/10.2196/37061 UR - http://www.ncbi.nlm.nih.gov/pubmed/36040767 ID - info:doi/10.2196/37061 ER - TY - JOUR AU - Tulsiani, Shreya AU - Ichimiya, Megumi AU - Gerard, Raquel AU - Mills, Sarah AU - Bingenheimer, B. Jeffrey AU - Hair, C. Elizabeth AU - Vallone, Donna AU - Evans, Douglas W. PY - 2022/8/29 TI - Assessing the Feasibility of Studying Awareness of a Digital Health Campaign on Facebook: Pilot Study Comparing Young Adult Subsamples JO - JMIR Form Res SP - e37856 VL - 6 IS - 8 KW - campaign evaluation KW - outcome evaluation KW - young adults KW - social marketing KW - health communications KW - tobacco control and policy KW - health campaign KW - youth KW - Facebook KW - digital media N2 - Background: Mass media campaigns for preventive health messaging have been shown to be effective through years of research. However, few studies have assessed the effectiveness of campaigns on digital media, which is currently how youths and young adults are primarily consuming media. In particular, a platform that can accurately assess exposure to digital messaging in a real-life setting has yet to be developed. Objective: This study examines the feasibility of a unique survey platform, Virtual Lab, to conduct a study on exposure to a media campaign within Facebook using a chatbot-style survey administration technique. Methods: Virtual Lab is a survey platform that was used to recruit and survey participants within Facebook and Facebook Messenger, respectively. We created a Facebook business account with 2 Facebook pages: one for recruitment and disseminating the survey and the other one for serving the target advertisements. Pre- and postexposure surveys were administered via Facebook Messenger using a chatbot-style questionnaire 1 week apart. During this time, the target advertisements were shown to participants who completed the pre-exposure survey. The total time from recruitment to completion of the postexposure survey was 13 days, and incentive costs were US $10 per participant. Survey data were compared between those who completed both pre- and postexposure surveys and those who only completed the pre-exposure survey; that is, those who were lost to follow-up. The demographics of the complete cases were also compared to the US census data. Results: A total of 375 Facebook users aged between 18 and 24 years met eligibility requirements and consented to the study, which consisted of complete cases (n=234) and participants lost to follow-up (n=141). A few differences between complete cases and participants lost to follow-up were observed. Regarding gender, complete cases comprised 40.2% males and 59.4% females, and among participants lost to follow-up, 44.0% were male and 50.4% were female (P=.003). Differences were also observed for e-cigarette use status, where a greater number of current users and fewer past and never users were lost to follow-up than complete cases (P=.01). Conclusions: The use of Virtual Lab yielded a diverse sample quickly and cost-effectively. Demographic characteristics of participants who completed the study and those who were lost to follow-up were similar, indicating that no biases were caused by the platform during recruitment or testing. This study suggests the feasibility of the Virtual Lab survey platform for studies of media campaign exposure within Facebook. This platform can advance health campaign research by providing more accurate data to inform digital messaging. UR - https://formative.jmir.org/2022/8/e37856 UR - http://dx.doi.org/10.2196/37856 UR - http://www.ncbi.nlm.nih.gov/pubmed/36036974 ID - info:doi/10.2196/37856 ER - TY - JOUR AU - Campbell, Chris AU - Feehan, Michael AU - Kanitscheider, Claudia AU - Makena, S. Patrudu AU - Cai, Jenny AU - Baxter, A. Sarah PY - 2022/8/19 TI - Designing Studies to Inform Tobacco Harm Reduction: Learnings From an Oral Nicotine Pouch Actual Use Pilot Study JO - JMIR Form Res SP - e37573 VL - 6 IS - 8 KW - harm reduction KW - pilot KW - nicotine pouch KW - actual use KW - electronic diary KW - smartphone KW - survey KW - combustible cigarette KW - smoking reduction KW - remote monitoring N2 - Background: Introduction of new tobacco products in the United States, including those that may be lower on the risk continuum than traditional combustible cigarettes, requires premarket authorization by the US Food and Drug Administration and information on the potential impact of the products on consumer behaviors. Efficient recruitment and data capture processes are needed to collect relevant information in a near-to-real-world environment. Objective: The aim of this pilot study was to develop and test a protocol for an actual use study of a new tobacco product. The product included in this study was a commercially available oral nicotine pouch. Through the process of study design and execution, learnings were garnered to inform the design, execution, analysis, and report writing of future full-scale actual use studies with tobacco products. Methods: A small sample (n=100) of healthy adult daily smokers of 7 or more cigarettes per day were recruited to participate in an 8-week prospective observational study conducted at 4 geographically dispersed sites in the United States. A smartphone-based customized electronic diary (eDiary) was employed to capture daily tobacco product use, including 1 week of baseline smoking and 6 weeks during which participants were provided with oral nicotine pouches for use as desired. Results: Online screening procedures with follow-up telephone interviews and on-site enrollment were successfully implemented. Of 100 participants, 97 completed the study, with more than half (59/99, 60%) identifying as dual- or poly-users of cigarettes and other types of tobacco products at baseline. There was more than 90% (91-93/99, 92%-94%) compliance with daily eDiary reporting, and the majority (92/99, 93%) of participants expressed satisfaction with the study processes. Product use data from the eDiary indicated that after an initial period of trial use, pouches per day increased among those continuing to use the products, while per day average cigarette consumption decreased for 82% (79/97) of all study participants. At the end of the week 6, 16% (15/97) of participants had reduced their cigarette consumption by more than half. Conclusions: The design of this study, including recruiting, enrollment, eDiary use, and oversight, was successfully implemented through the application of a detailed protocol, a user-friendly eDiary, electronically administered questionnaires, and remote monitoring procedures. High-resolution information was obtained on prospective changes in tobacco product use patterns in the context of availability of a new tobacco product. Future, larger actual use studies will provide important evidence supporting the role that alternatives to combustible cigarettes may play in smoking reduction and/or cessation and lowering the population health burden of tobacco and nicotine-containing products. UR - https://formative.jmir.org/2022/8/e37573 UR - http://dx.doi.org/10.2196/37573 UR - http://www.ncbi.nlm.nih.gov/pubmed/35984682 ID - info:doi/10.2196/37573 ER - TY - JOUR AU - Liu, Sam AU - La, Henry AU - Willms, Amanda AU - Rhodes, E. Ryan PY - 2022/8/18 TI - A ?No-Code? App Design Platform for Mobile Health Research: Development and Usability Study JO - JMIR Form Res SP - e38737 VL - 6 IS - 8 KW - app development KW - behavior change technique KW - health promotion KW - mobile health KW - mobile application KW - application development KW - design platform KW - platform development KW - no-code mHealth app KW - no-code app KW - no-code KW - end user KW - participatory research KW - Pathverse KW - agile KW - hybrid-agile KW - software design KW - software development KW - software developer KW - computer science KW - BCT KW - behavior change KW - research tool KW - research instrument KW - digital platform KW - mHealth KW - mobile app N2 - Background: A challenge facing researchers conducting mobile health (mHealth) research is the amount of resources required to develop mobile apps. This can be a barrier to generating relevant knowledge in a timely manner. The recent rise of ?no-code? software development platforms may overcome this challenge and enable researchers to decrease the cost and time required to develop mHealth research apps. Objective: We aimed to describe the development process and the lessons learned to build Pathverse, a no-code mHealth app design platform. Methods: The study took place between November 2019 and December 2021. We used a participatory research framework to develop the mHealth app design platform. In phase 1, we worked with researchers to gather key platform feature requirements and conducted an exploratory literature search to determine needs related to this platform. In phase 2, we used an agile software framework (Scrum) to develop the platform. Each development sprint cycle was 4 weeks in length. We created a minimum viable product at the end of 7 sprint cycles. In phase 3, we used a convenience sample of adults (n=5) to gather user feedback through usability and acceptability testing. In phase 4, we further developed the platform based on user feedback, following the V-model software development process. Results: Our team consulted end users (ie, researchers) and utilized behavior change technique taxonomy and behavior change models (ie, the multi-process action control framework) to guide the development of features. The first version of the Pathverse platform included features that allowed researchers to (1) design customized multimedia app content (eg, interactive lessons), (2) set content delivery logic (eg, only show new lessons when completing the previous lesson), (3) implement customized participant surveys, (4) provide self-monitoring tools, (5) set personalized goals, and (6) customize app notifications. Usability and acceptability testing revealed that researchers found the platform easy to navigate and that the features were intuitive to use. Potential improvements include the ability to deliver adaptive interventions and add features such as community group chat. Conclusions: To our knowledge, Pathverse is the first no-code mHealth app design platform for developing mHealth interventions for behavior. We successfully used behavior change models and the behavior change technique taxonomy to inform the feature requirements of Pathverse. Overall, the use of a participatory framework, combined with the agile and hybrid-agile software development process, enabled our team to successfully develop the Pathverse platform. UR - https://formative.jmir.org/2022/8/e38737 UR - http://dx.doi.org/10.2196/38737 UR - http://www.ncbi.nlm.nih.gov/pubmed/35980740 ID - info:doi/10.2196/38737 ER - TY - JOUR AU - Sadeh-Sharvit, Shiri AU - Rego, A. Simon AU - Jefroykin, Samuel AU - Peretz, Gal AU - Kupershmidt, Tomer PY - 2022/8/16 TI - A Comparison Between Clinical Guidelines and Real-World Treatment Data in Examining the Use of Session Summaries: Retrospective Study JO - JMIR Form Res SP - e39846 VL - 6 IS - 8 KW - Empirically based practices KW - natural language processing KW - psychotherapy KW - behavioral therapy KW - adherence KW - treatment fidelity KW - clinical training KW - real-world data KW - real-world study N2 - Background: Although behavioral interventions have been found to be efficacious and effective in randomized clinical trials for most mental illnesses, the quality and efficacy of mental health care delivery remains inadequate in real-world settings, partly owing to suboptimal treatment fidelity. This ?therapist drift? is an ongoing issue that ultimately reduces the effectiveness of treatments; however, until recently, there have been limited opportunities to assess adherence beyond large randomized controlled trials. Objective: This study explored therapists? use of a standard component that is pertinent across most behavioral treatments?prompting clients to summarize their treatment session as a means for consolidating and augmenting their understanding of the session and the treatment plan. Methods: The data set for this study comprised 17,607 behavioral treatment sessions administered by 322 therapists to 3519 patients in 37 behavioral health care programs across the United States. Sessions were captured by a therapy-specific artificial intelligence (AI) platform, and an automatic speech recognition system transcribed the treatment meeting and separated the data to the therapist and client utterances. A search for possible session summary prompts was then conducted, with 2 psychologists validating the text that emerged. Results: We found that despite clinical recommendations, only 54 (0.30%) sessions included a summary. Exploratory analyses indicated that session summaries mostly addressed relationships (n=27), work (n=20), change (n=6), and alcohol (n=5). Sessions with meeting summaries were also characterized by greater therapist interventions and included greater use of validation, complex reflections, and proactive problem-solving techniques. Conclusions: To the best of our knowledge, this is the first study to assess a large, diverse data set of real-world treatment practices. Our findings provide evidence that fidelity with the core components of empirically designed psychological interventions is a challenge in real-world settings. The results of this study can inform the development of machine learning and AI algorithms and offer nuanced, timely feedback to providers, thereby improving the delivery of evidence-based practices and quality of mental health care services and facilitating better clinical outcomes in real-world settings. UR - https://formative.jmir.org/2022/8/e39846 UR - http://dx.doi.org/10.2196/39846 UR - http://www.ncbi.nlm.nih.gov/pubmed/35972782 ID - info:doi/10.2196/39846 ER - TY - JOUR AU - Kamstra, M. Regina J. AU - Boorsma, André AU - Krone, Tanja AU - van Stokkum, M. Robin AU - Eggink, M. Hannah AU - Peters, Ton AU - Pasman, J. Wilrike PY - 2022/8/11 TI - Validation of the Mobile App Version of the EQ-5D-5L Quality of Life Questionnaire Against the Gold Standard Paper-Based Version: Randomized Crossover Study JO - JMIR Form Res SP - e37303 VL - 6 IS - 8 KW - quality of life assessment KW - EQ-5D-5L questionnaire KW - mobile app KW - test-retest reliability KW - mobile phone N2 - Background: Study participants and patients often perceive (long) questionnaires as burdensome. In addition, paper-based questionnaires are prone to errors such as (unintentionally) skipping questions or filling in a wrong type of answer. Such errors can be prevented with the emergence of mobile questionnaire apps. Objective: This study aimed to validate an innovative way to measure the quality of life using a mobile app based on the EQ-5D-5L questionnaire. This validation study compared the EQ-5D-5L questionnaire requested by a mobile app with the gold standard paper-based version of the EQ-5D-5L. Methods: This was a randomized, crossover, and open study. The main criteria for participation were participants should be aged ?18 years, healthy at their own discretion, in possession of a smartphone with at least Android version 4.1 or higher or iOS version 9 or higher, digitally skilled in downloading the mobile app, and able to read and answer questionnaires in Dutch. Participants were recruited by a market research company that divided them into 2 groups balanced for age, gender, and education. Each participant received a digital version of the EQ-5D-5L questionnaire via a mobile app and the EQ-5D-5L paper-based questionnaire by postal mail. In the mobile app, participants received, for 5 consecutive days, 1 question in the morning and 1 question in the afternoon; as such, all questions were asked twice (at time point 1 [App T1] and time point 2 [App T2]). The primary outcomes were the correlations between the answers (scores) of each EQ-5D-5L question answered via the mobile app compared with the paper-based questionnaire to assess convergent validity. Results: A total of 255 participants (healthy at their own discretion), 117 (45.9%) men and 138 (54.1%) women in the age range of 18 to 64 years, completed the study. To ensure randomization, the measured demographics were checked and compared between groups. To compare the results of the electronic and paper-based questionnaires, polychoric correlation analysis was performed. All questions showed a high correlation (0.64-0.92; P<.001) between the paper-based and the mobile app?based questions at App T1 and App T2. The scores and their variance remained similar over the questionnaires, indicating no clear difference in the answer tendency. In addition, the correlation between the 2 app-based questionnaires was high (>0.73; P<.001), illustrating a high test-retest reliability, indicating it to be a reliable replacement for the paper-based questionnaire. Conclusions: This study indicates that the mobile app is a valid tool for measuring the quality of life and is as reliable as the paper-based version of the EQ-5D-5L, while reducing the response burden. UR - https://formative.jmir.org/2022/8/e37303 UR - http://dx.doi.org/10.2196/37303 UR - http://www.ncbi.nlm.nih.gov/pubmed/35969437 ID - info:doi/10.2196/37303 ER - TY - JOUR AU - Dangerfield II, T. Derek AU - Anderson, N. Janeane AU - Wylie, Charleen AU - Arrington-Sanders, Renata AU - Bluthenthal, N. Ricky AU - Beyrer, Christopher AU - Farley, E. Jason PY - 2022/8/10 TI - Refining a Multicomponent Intervention to Increase Perceived HIV Risk and PrEP Initiation: Focus Group Study Among Black Sexual Minority Men JO - JMIR Form Res SP - e34181 VL - 6 IS - 8 KW - sexual health KW - life course theory KW - health belief KW - possible KW - HIV KW - preexposure prophylaxis KW - mHealth KW - smartphone KW - health app KW - digital health N2 - Background: Increased preexposure prophylaxis (PrEP) initiation is needed to substantially decrease HIV incidence among Black sexual minority men (BSMM). However, BSMM perceive others as PrEP candidates instead of themselves and are less likely than other groups to use PrEP if prescribed. Peers and smartphone apps are popular HIV prevention intervention tools typically used independently. However, they could be useful together in a multicomponent strategy to improve perceived HIV risk and PrEP initiation for this group. Information regarding attitudes and preferences toward this multicomponent strategy is limited. Objective: The goal of this study is to obtain attitudes and perspectives regarding the design of a multicomponent intervention that uses a smartphone app and a peer change agent (PCA) to increase perceived HIV risk and PrEP initiation. The intervention will be refined based on thematic findings for a culturally responsive approach. Methods: Data were obtained guided by life course theory and the health belief model using 12 focus groups and 1 in-depth interview among HIV-negative BSMM from Baltimore, MD, between October 2019 and May 2020 (n=39). Groups were stratified by the following ages: 18 to 24 years, 25 to 34 years, and 35 years and older. Participants were provided details regarding an existing mobile app diary to self-monitor sexual behaviors and a hypothetical PCA with whom to review the app. Facilitators posed questions regarding perceived HIV risk, attitudes toward the app, working with a PCA, and preferences for PCA characteristics and approaches. Results: Most participants identified as homosexual, gay, or same gender-loving (26/38, 68%), were employed (26/38, 69%), single (25/38, 66%), and interested in self-monitoring sexual behaviors (28/38, 68%). However, themes suggested that participants had low perceived HIV risk, that self-monitoring sexual behaviors using a mobile app diary was feasible but could trigger internalized stigma, and that an acceptable PCA should be a possible self for BSMM to aspire to but they still wanted clinicians to ?do their job.? Conclusions: HIV-negative BSMM have dissonant attitudes regarding perceived HIV risk and the utility of a mobile app and PCA to increase perceived HIV risk and PrEP initiation. Future research will explore the feasibility, acceptability, and preliminary impact of implementing the multicomponent intervention on perceived HIV risk and PrEP initiation among BSMM in a pilot study. UR - https://formative.jmir.org/2022/8/e34181 UR - http://dx.doi.org/10.2196/34181 UR - http://www.ncbi.nlm.nih.gov/pubmed/35947442 ID - info:doi/10.2196/34181 ER - TY - JOUR AU - Lei, Zhengdong AU - Martignetti, Lisa AU - Ridgway, Chelsea AU - Peacock, Simon AU - Sakata, T. Jon AU - Li-Jessen, K. Nicole Y. PY - 2022/8/5 TI - Wearable Neck Surface Accelerometers for Occupational Vocal Health Monitoring: Instrument and Analysis Validation Study JO - JMIR Form Res SP - e39789 VL - 6 IS - 8 KW - mechano-acoustic sensing KW - voice monitoring KW - wearable device KW - neck surface accelerometer N2 - Background: Neck surface accelerometer (NSA) wearable devices have been developed for voice and upper airway health monitoring. As opposed to acoustic sounds, NSA senses mechanical vibrations propagated from the vocal tract to neck skin, which are indicative of a person?s voice and airway conditions. NSA signals do not carry identifiable speech information and a speaker?s privacy is thus protected, which is important and necessary for continuous wearable monitoring. Our device was already tested for its durable endurance and signal processing algorithms in controlled laboratory conditions. Objective: This study aims to further evaluate both instrument and analysis validity in a group of occupational vocal users, namely, voice actors, who use their voices extensively at work in an ecologically valid setting. Methods: A total of 16 professional voice actors (age range 21-50 years; 11 females and 5 males) participated in this study. All participants were mounted with an NSA on their sternal notches during the voice acting and voice assessment sessions. The voice acting session was 4-hour long, directed by a voice director in a professional sound studio. Voice assessment sessions were conducted before, during, and 48 hours after the acting session. The assessment included phonation tasks of passage reading, sustained vowels, maximum vowel phonation, and pitch glides. Clinical acoustic metrics (eg, fundamental frequency, cepstral measures) and a vocal dose measure (ie, accumulated distance dose from acting) were computed from NSA signals. A commonly used online questionnaire (Self-Administered Voice Rating questionnaire) was also implemented to track participants? perception of vocal fatigue. Results: The NSA wearables stayed in place for all participants despite active body movements during the acting. The ensued body noise did not interfere with the NSA signal quality. All planned acoustic metrics were successfully derived from NSA signals and their numerical values were comparable with literature data. For a 4-hour long voice acting, the averaged distance dose was about 8354 m with no gender differences. Participants perceived vocal fatigue as early as 2 hours after the start of voice acting, with recovery 24-48 hours after the acting session. Among all acoustic metrics across phonation tasks, cepstral peak prominence and spectral tilt from the passage reading most closely mirrored trends in perceived fatigue. Conclusions: The ecological validity of an in-house NSA wearable was vetted in a workplace setting. One key application of this wearable is to prompt occupational voice users when their vocal safety limits are reached for duly protection. Signal processing algorithms can thus be further developed for near real-time estimation of clinically relevant metrics, such as accumulated distance dose, cepstral peak prominence, and spectral tilt. This functionality will enable continuous self-awareness of vocal behavior and protection of vocal safety in occupational voice users. UR - https://formative.jmir.org/2022/8/e39789 UR - http://dx.doi.org/10.2196/39789 UR - http://www.ncbi.nlm.nih.gov/pubmed/35930317 ID - info:doi/10.2196/39789 ER - TY - JOUR AU - Darko, Mirekuwaa Elizabeth AU - Kleib, Manal AU - Olson, Joanne PY - 2022/8/4 TI - Social Media Use for Research Participant Recruitment: Integrative Literature Review JO - J Med Internet Res SP - e38015 VL - 24 IS - 8 KW - advertisement KW - recruitment KW - research participants KW - social media KW - mobile phone N2 - Background: Social media tools have provided health researchers with the opportunity to engage with communities and groups in a nonconventional manner to recruit participants for health research. Using social media to advertise research opportunities and recruit participants facilitates accessibility to participants from broad geographical areas and diverse populations. However, little guidance is provided by ethics review boards for researchers to effectively use this recruitment method in their research. Objective: This study sought to explore the literature on the use of social media for participant recruitment for research studies and identify the best practices for recruiting participants using this method. Methods: An integrative review approach was used to synthesize the literature. A total of 5 health sciences databases, namely, EMBASE (Ovid), MEDLINE (Ovid and EBSCOhost), PsycINFO (Ovid), Scopus (Elsevier), and CINAHL Plus with Full Text (EBSCOhost), were searched using predefined keywords and inclusion and exclusion criteria. The initial search was conducted in October 2020 and was updated in February 2022. Descriptive and content analyses were applied to synthesize the results, and the findings are presented in a narrative and tabular format. Results: A total of 96 records were included in this review, 83 (86%) from the initial search and 13 (14%) from the updated search. The publication year ranged between 2011 and 2022, with most publications (63/96, 66%) being from the United States. Regarding recruitment strategy, 45% (43/96) of the studies exclusively used social media, whereas 51% (49/96) used social media in conjunction with other strategies. The remaining 4% (4/96) provided guidelines and recommendations for social media recruitment. Notably, 38% (36/96) of these studies involved hard-to-reach populations. The findings also revealed that the use of social media is a cost-effective and efficient strategy for recruiting research participants. Despite the expanded use across different populations, there is limited participation of older adults in social media recruitment. Conclusions: This review provides important insights into the current use of social media for health research participant recruitment. Ethics boards and research support services in academic institutions are encouraged to explicitly provide researchers with guidelines on the use of social media for health research participant recruitment. A preliminary guideline prepared based on the findings of this review is proposed to spark further development in this area. UR - https://www.jmir.org/2022/8/e38015 UR - http://dx.doi.org/10.2196/38015 UR - http://www.ncbi.nlm.nih.gov/pubmed/35925655 ID - info:doi/10.2196/38015 ER - TY - JOUR AU - Viola, S. Adrienne AU - Levonyan-Radloff, Kristine AU - Masterson, Margaret AU - Manne, L. Sharon AU - Hudson, V. Shawna AU - Devine, A. Katie PY - 2022/8/3 TI - Development of a Self-management and Peer-Mentoring Intervention to Improve Transition Readiness Among Young Adult Survivors of Pediatric Cancer: Formative Qualitative Research Study JO - JMIR Form Res SP - e36323 VL - 6 IS - 8 KW - self-management KW - peer mentoring KW - cancer survivorship KW - long-term follow-up care N2 - Background: Childhood cancer survivors require lifelong risk-based follow-up care. It should be noted that less than one-third of adult survivors of childhood cancer report any survivor-focused care, and fewer than 1 in 5 obtain risk-based follow-up care. It is thought that this may be due to inadequate transition readiness, including low levels of knowledge, skills, motivation, and resources to make the transition to independent self-management of follow-up care. Interventions that focus specifically on improving the transition from parent-managed to self-managed care are needed. Theory and prior research suggest that targeting self-management skills and using peer mentoring may be innovative strategies to improve transition readiness. Objective: This study aims to identify the content of a self-management intervention to improve transition readiness among adolescent and young adult (AYA) survivors. Methods: Intervention development occurred in 3 stages: formative research with AYA survivors to identify barriers and facilitators to obtaining risk-based survivorship care, content development using feedback from multiple stakeholders (AYA survivors, parents, and providers), and content refinement (usability testing) of the initial proposed educational modules for the program. Content analysis, guided by the social-ecological model of AYA readiness for transition, was used to identify themes and develop and refine the content for the intervention. Results: A total of 19 AYA survivors participated in the formative research stage, and 10 AYA survivors, parents, and health care providers participated in the content development and refinement stages. The major barrier and facilitator themes identified included knowledge of cancer history and risks; relationships with health care providers; relationships with family members involved in care; emotions about health, follow-up care, and transfer of care; and lifestyle behaviors and life transitions. These themes were translated into 5 self-management modules: understanding treatment history and the survivorship care plan, managing health care logistics and insurance, communicating with health care providers and family members involved in care, dealing with emotions, and staying healthy in the context of life transitions. Feedback from the key stakeholders indicated that the content was relevant but should include participative elements (videos and tailored feedback) to make the intervention more engaging. The AYA survivors were receptive to the idea of working with a peer mentor and expressed a preference for using SMS text messaging, telephone calls, or videoconference to communicate with their mentor. Conclusions: Incorporating AYA survivors, parents, and providers in the design was essential to developing the content of a self-management and peer-mentoring intervention. AYA survivors confirmed the important targets for the intervention and facilitated design decisions in line with our target users? preferences. The next step will be to conduct a single-arm trial to determine the feasibility and acceptability of the proposed intervention among AYA survivors of childhood cancer. UR - https://formative.jmir.org/2022/8/e36323 UR - http://dx.doi.org/10.2196/36323 UR - http://www.ncbi.nlm.nih.gov/pubmed/35921137 ID - info:doi/10.2196/36323 ER - TY - JOUR AU - Spreadbury, Henry John AU - Young, Alex AU - Kipps, Myles Christopher PY - 2022/7/28 TI - A Comprehensive Literature Search of Digital Health Technology Use in Neurological Conditions: Review of Digital Tools to Promote Self-management and Support JO - J Med Internet Res SP - e31929 VL - 24 IS - 7 KW - digital health technology KW - digital tools KW - neurology KW - patients KW - self-management N2 - Background: The use of digital health technology to promote and deliver postdiagnostic care in neurological conditions is becoming increasingly common. However, the range of digital tools available across different neurological conditions and how they facilitate self-management are unclear. Objective: This review aims to identify digital tools that promote self-management in neurological conditions and to investigate their underlying functionality and salient clinical outcomes. Methods: We conducted a search of 6 databases (ie, CINAHL, EMBASE, MEDLINE, PsycINFO, Web of Science, and the Cochrane Review) using free text and equivalent database-controlled vocabulary terms. Results: We identified 27 published articles reporting 17 self-management digital tools. Multiple sclerosis (MS) had the highest number of digital tools followed by epilepsy, stroke, and headache and migraine with a similar number, and then pain. The majority were aimed at patients with a minority for carers. There were 5 broad categories of functionality promoting self-management: (1) knowledge and understanding; (2) behavior modification; (3) self-management support; (4) facilitating communication; and (5) recording condition characteristics. Salient clinical outcomes included improvements in self-management, self-efficacy, coping, depression, and fatigue. Conclusions: There now exist numerous digital tools to support user self-management, yet relatively few are described in the literature. More research is needed to investigate their use, effectiveness, and sustainability, as well as how this interacts with increasing disability, and their integration within formal neurological care environments. UR - https://www.jmir.org/2022/7/e31929 UR - http://dx.doi.org/10.2196/31929 UR - http://www.ncbi.nlm.nih.gov/pubmed/35900822 ID - info:doi/10.2196/31929 ER - TY - JOUR AU - Cunningham-Erves, Jennifer AU - Brandt, M. Heather AU - Sanderson, Maureen AU - Clarkson, Kristin AU - Lee, Omaran AU - Schlundt, David AU - Bonnet, Kemberlee AU - Davis, Jamaine PY - 2022/7/28 TI - Development of a Theory-Based, Culturally Appropriate Message Library for Use in Interventions to Promote COVID-19 Vaccination Among African Americans: Formative Research JO - JMIR Form Res SP - e38781 VL - 6 IS - 7 KW - African American KW - Black American KW - Black KW - minority KW - ethnic KW - culturally sensitive KW - cultural sensitivity KW - inclusive KW - vulnerable KW - COVID-19 KW - vaccination KW - vaccine KW - health promotion KW - campaign KW - messaging KW - culturally appropriate KW - theory KW - adults KW - children KW - disparity KW - health belief model KW - community engagement KW - public engagement KW - public awareness KW - community-based KW - health information KW - health communication KW - health intervention KW - vulnerable population KW - community health KW - patient education N2 - Background: Disparities in COVID-19 incidence, hospitalization, and mortality rates among African Americans suggest the need for targeted interventions. Use of targeted, theory-driven messages in behavioral and communication interventions could empower African Americans to engage in behaviors that prevent COVID-19. Objective: To address this need, we performed a formative study that aimed to develop and design a culturally appropriate, theory-based library of messages targeting concerns around COVID-19 vaccines that could be used in behavioral and communication interventions for African Americans. Methods: Message development occurred between January 2021 and February 2022. Initial messages were designed by a multidisciplinary team of researchers, community leaders, and community members. Kreuter?s 5 strategies (ie, linguistic, peripheral, evidential, sociocultural, and constituent-involving strategies) were used to achieve cultural appropriateness. After forming a community-academic partnership, message development occurred in 4 phases: (1) adaptation of a message library using the literature, (2) review by 6 clinical and research experts for content validation, (3) input and review by a 6-member community advisory panel (CAP), and (4) message pretesting with African Americans via semistructured interviews in a qualitative study. Results: Themes from the semistructured interviews among 30 African Americans were as follows: (1) community reactions to the messages, (2) community questions and information needs, (3) suggestions for additional content, and (4) suggestions to improve comprehension, relevance, and trustworthiness. Feedback from the CAP, community members, and scientific experts was used by members of the community-academic partnership to iteratively update message content to maximize cultural appropriateness. The final message library had 18 message subsets for adults and 17 message subsets for parents and caregivers of children. These subsets were placed into 3 categories: (1) vaccine development, (2) vaccine safety, and (3) vaccine effectiveness. Conclusions: We used a 4-phase, systematic process using multiple community engagement approaches to create messages for African Americans to support interventions to improve COVID-19 vaccination rates among adults and children. The newly developed messages were deemed to be culturally appropriate according to experts and members of the African American community. Future research should evaluate the impact of these messages on COVID-19 vaccination rates among African Americans. UR - https://formative.jmir.org/2022/7/e38781 UR - http://dx.doi.org/10.2196/38781 UR - http://www.ncbi.nlm.nih.gov/pubmed/35781223 ID - info:doi/10.2196/38781 ER - TY - JOUR AU - Hoepper, B. Bettina AU - Siegel, R. Kaitlyn AU - Carlon, A. Hannah AU - Kahler, W. Christopher AU - Park, R. Elyse AU - Taylor, Trevor Steven AU - Simpson, V. Hazel AU - Hoeppner, S. Susanne PY - 2022/7/28 TI - Feature-Level Analysis of a Smoking Cessation Smartphone App Based on a Positive Psychology Approach: Prospective Observational Study JO - JMIR Form Res SP - e38234 VL - 6 IS - 7 KW - mHealth KW - smartphone KW - smartphone app KW - smoking KW - smoking cessation KW - nondaily smoking KW - positive psychology KW - happiness KW - positive affect KW - clinical trial KW - feasibility KW - acceptability KW - app usage KW - mobile health N2 - Background: Smoking cessation smartphone apps have emerged as highly accessible tools to support smoking cessation efforts. It is unknown how specific app features contribute to user engagement over time and relate to smoking outcomes. Objective: To provide a feature-level analysis of the Smiling Instead of Smoking app (version 2) and to link feature use to subsequent smoking cessation. Methods: Nondaily smokers (N=100) used the app for a period of 49 days (1 week before quitting and 6 weeks after quitting). Participants self-reported 30-day point-prevalence abstinence at the end of this period and at a 6-month follow up (the survey response rate was 94% and 89% at these points, respectively). Self-reported 30-day point prevalence abstinence rates were 40% at the end of treatment and 56% at the 6-month follow up. The app engaged users in both positive psychology content and traditional behavioral smoking cessation content. The app sent push notifications to prompt participants to complete prescribed content (ie, a ?happiness exercise? every day and a ?behavioral challenge? to use the app?s smoking cessation tools on 15 out of 49 days). Actions that participants took within the app were timestamped and recorded. Results: Participants used the app on 24.7 (SD 13.8) days out of the 49 prescribed days, interacting with the happiness content on more days than the smoking content (23.8, SD 13.8 days vs 17.8, SD 10.3 days; t99=9.28 [2-tailed]; P<.001). The prescribed content was frequently completed (45% of happiness exercises; 57% of behavioral challenges) and ad libitum tools were used on ?7 days. Most participants used each ad libitum smoking cessation tool at least once, with higher use of personalized content (?92% used ?strategies,? ?cigarette log,? ?smoke alarms,? and ?personal reasons?) than purely didactic content (79% viewed ?benefits of quitting smoking?). The number of days participants used the app significantly predicted 30-day point-prevalence abstinence at the end of treatment (odds ratio [OR] 1.05, 95% CI 1.02-1.09; P=.002) and at the 6-month follow up (OR 1.04, 95% CI 1.008-1.07; P=.01). The number of days participants engaged with the happiness content significantly predicted smoking abstinence at the end of treatment (OR 1.05, 95% CI 1.02-1.08; P=.002) and at the 6-month follow up (OR 1.04, 95% CI 1.007-1.07; P=.02). This effect was not significant for the number of days participants engaged with the smoking cessation content of the app, either at the end of treatment (OR 1.04, 95% CI 0.996-1.08, P=.08) or at the 6-month follow up (OR 1.02, 95% CI 0.98-1.06; P=.29). Conclusions: Greater app usage predicted greater odds of self-reported 30-day point-prevalence abstinence at both the end of treatment and over the long term, suggesting that the app had a therapeutic benefit. Positive psychology content and prescriptive clarity may promote sustained app engagement over time. Trial Registration: ClinicalTrials.gov NCT03951766; https://clinicaltrials.gov/ct2/show/NCT03951766 UR - https://formative.jmir.org/2022/7/e38234 UR - http://dx.doi.org/10.2196/38234 UR - http://www.ncbi.nlm.nih.gov/pubmed/35900835 ID - info:doi/10.2196/38234 ER - TY - JOUR AU - Johnson, K. Amy AU - Haider, Sadia AU - Nikolajuk, Katie AU - Kuhns, M. Lisa AU - Ott, Emily AU - Motley, Darnell AU - Hill, Brandon AU - Hirschhorn, Lisa PY - 2022/7/28 TI - An mHealth Intervention to Improve Pre-Exposure Prophylaxis Knowledge Among Young Black Women in Family Planning Clinics: Development and Usability Study JO - JMIR Form Res SP - e37738 VL - 6 IS - 7 KW - mHealth KW - adolescent health KW - young Black women KW - pre-exposure prophylaxis KW - HIV KW - mobile health KW - PrEP KW - mobile app N2 - Background: Young Black women between the ages of 18 and 24 years are disproportionately impacted by HIV, yet they have a low self-perception of HIV risk and limited exposure to prevention strategies. Pre-exposure prophylaxis (PrEP) is a safe and effective biomedical HIV prevention strategy for those at risk for HIV infection, but uptake has been slow among cisgender women. Family planning clinics are a primary source of health care access for young women, providing an ideal opportunity to integrate PrEP information and care into existing clinic practices. Objective: The aim of this study was to use a multistage, community-engaged process to develop a mobile health app and to evaluate the feasibility and acceptability of the app. Methods: Using user-centered design, the In the Loop app was developed in collaboration with a community advisory board of young Black women. This study employed a multistage design, which included community-engaged app development, user testing, and evaluation of the app?s feasibility and acceptability. A pre- and postdesign was used to assess the impact of the app on PrEP knowledge immediately after app use. Descriptive statistics (eg, mean, SD, and percentage values) were used to describe the sample, and Wilcoxon matched-pairs signed-ranks test was used to detect changes in PrEP knowledge before and immediately after using the app. Results: A total of 50 sexually active, young Black women, aged 18-24 (mean 21, SD 1.9) years, were enrolled in this study. Analysis comparing scores before and immediately after use of the app revealed a significant increase in PrEP content knowledge scores on a 7-item true or false scale (z=?6.04, P<.001). Overall, participants considered the In the Loop app feasible and acceptable to use while waiting for a family planning visit. The majority of participants (n=46, 92%) agreed that they would recommend In the Loop to friends to learn more about PrEP. Participants rated the overall quality of the app 4.3 on a 1-5 scale (1=very poor and 5=very good). Of 50 participants, 40 (80%) agreed that the app was easy to use, and 48 (96%) agreed that they found the information in the app easy to understand. Finally, 40 (80%) agreed that they had enjoyed using the app while waiting for their family planning visit. Conclusions: Our findings suggest that young Black women waiting for family planning visits found the In the Loop app to be feasible and acceptable. This study demonstrates the value of engaging young Black women in the app design process. As family planning clinics are a primary source of health care access for young women, they provide an ideal setting to integrate PrEP information and care into existing clinic practices. Next steps in the development of the In the Loop app include implementing user-suggested improvements and conducting efficacy testing in a randomized controlled trial to determine the app?s impact on PrEP uptake. UR - https://formative.jmir.org/2022/7/e37738 UR - http://dx.doi.org/10.2196/37738 UR - http://www.ncbi.nlm.nih.gov/pubmed/35900830 ID - info:doi/10.2196/37738 ER - TY - JOUR AU - Adejare, A. Adeboye AU - Duncan, J. Heather AU - Motz, Geoffrey R. AU - Shah, Silvi AU - Thakar, V. Charuhas AU - Eckman, H. Mark PY - 2022/7/28 TI - Implementing a Health Utility Assessment Platform to Acquire Health Utilities in a Hemodialysis Outpatient Setting: Feasibility Study JO - JMIR Form Res SP - e33562 VL - 6 IS - 7 KW - health utility assessment KW - patient reported outcomes KW - end-stage kidney disease KW - hemodialysis KW - hepatitis C N2 - Background: Patients with end-stage kidney disease (ESKD) wait roughly 4 years for a kidney transplant. A potential way to reduce wait times is using hepatitis C virus (HCV)?viremic kidneys. Objective: As preparation for developing a shared decision-making tool to assist patients with ESKD with the decision to accept an HCV-viremic kidney transplant, our initial goal was to assess the feasibility of using The Gambler II, a health utility assessment tool, in an ambulatory dialysis clinic setting. Our secondary goals were to collect health utilities for patients with ESKD and to explore whether the use of race-matched versus race-mismatched exemplars impacted the knowledge gained during the assessment process. Methods: We used The Gambler II to elicit utilities for the following ESKD-related health states: hemodialysis, kidney transplant with HCV-unexposed kidney, and transplantation with HCV-viremic kidney. We created race exemplar video clips describing these health states and randomly assigned patients into the race-matched or race-mismatched video arms. We obtained utilities for these 3 health states from each patient, and we evaluated knowledge about ESKD and HCV-associated health conditions with pre- and postintervention knowledge assessments. Results: A total of 63 patients with hemodialysis from 4 outpatient Dialysis Center Inc sites completed the study. Mean adjusted standard gamble utilities for hemodialysis, transplant with HCV-unexposed kidney, and transplantation with HCV-viremic kidney were 82.5, 89, and 75.5, respectively. General group knowledge assessment scores improved by 10 points (P<.05) following utility assessment process. The use of race-matched exemplars had little effect on the results of the knowledge assessment of patients. Conclusions: Using The Gambler II to collect utilities for patients with ESKD in an ambulatory dialysis clinic setting proved feasible. In addition, educational information about health states provided as part of the utility assessment process tool improved patients? knowledge and understanding about ESKD-related health states and implications of organ transplantation with HCV-viremic kidneys. A wide variation in patient health state utilities reinforces the importance of incorporating patients? preferences into decisions regarding use of HCV-viremic kidneys for transplantation. UR - https://formative.jmir.org/2022/7/e33562 UR - http://dx.doi.org/10.2196/33562 UR - http://www.ncbi.nlm.nih.gov/pubmed/35900828 ID - info:doi/10.2196/33562 ER - TY - JOUR AU - Rath, M. Jessica AU - Perks, N. Siobhan AU - Williams, N. Kenneshia AU - Budnik, Tracy AU - Geraci, John AU - Vallone, M. Donna AU - Hair, C. Elizabeth PY - 2022/7/26 TI - Developing the Message Assessment Scale for Tobacco Prevention Campaigns: Cross-sectional Validation Study JO - JMIR Form Res SP - e38156 VL - 6 IS - 7 KW - communication KW - youth/young adults KW - scales KW - message KW - behavior KW - health KW - campaign KW - tobacco KW - smoking cessation KW - prevention KW - youth KW - young adults KW - data KW - data analysis N2 - Background: Mass media campaigns are effective for influencing a broad range of health behaviors. Prior to launching a campaign, developers often conduct ad testing to help identify the strengths and weaknesses of the message executions among the campaign?s target audience. This process allows for changes to be made to ads, making them more relevant to or better received by the target audience before they are finalized. To assess the effectiveness of an ad?s message and execution, campaign ads are often rated using a single item or multiple items on a scale, and scores are calculated. Endorsement of a 6-item perceived message effectiveness (PME) scale, defined as the practice of using a target audience?s evaluative ratings to inform message selection, is one approach commonly used to select messages for antitobacco campaigns; however, the 6-item PME scale often does not produce enough specificity to make important decisions on ad optimization. In addition, the PME scale is typically used with adult populations for smoking cessation messages. Objective: This study includes the development of the Message Assessment Scale, a new tobacco prevention message testing scale for youth and young adults. Methods: Data were derived from numerous cross-sectional surveys designed to test the relevance and potential efficacy of antitobacco truth campaign ads. Participants aged 15-24 years (N=6108) responded to a set of 12 core attitudinal items, including relevance (both personal and cultural) as well as comprehension of the ad?s main message. Results: Analyses were completed in two phases. In phase I, mean scores were calculated for each of the 12 attitudinal items by ad type, with higher scores indicating more endorsement of the item. Next, all items were submitted to exploratory factor analysis. A four-factor model fit was revealed and verified with confirmatory factor analysis, resulting in the following constructs: personally relevant, culturally relevant, the strength of messaging, and negative attributes. In phase II, ads were categorized by performance (high/medium/low), and constructs identified in phase I were correlated with key campaign outcomes (ie, main fact agreement and likelihood to vape). Phase II confirmed that the four constructs identified in phase I were all significantly correlated with main fact agreement and vape intentions. Conclusions: Findings from this study advance the field by establishing an expanded set of validated items to comprehensively assess the potential effectiveness of advertising executions. This set of items expands the portfolio of ad testing measures for ads focused on tobacco use prevention. Findings can inform how best to optimize ad executions and message delivery for health behavior campaigns, particularly those focused on tobacco use prevention among youth and young adult populations. UR - https://formative.jmir.org/2022/7/e38156 UR - http://dx.doi.org/10.2196/38156 UR - http://www.ncbi.nlm.nih.gov/pubmed/35881429 ID - info:doi/10.2196/38156 ER - TY - JOUR AU - Adler, F. Rachel AU - Morales, Paulina AU - Sotelo, Jocelyn AU - Magasi, Susan PY - 2022/7/26 TI - Developing an mHealth App for Empowering Cancer Survivors With Disabilities: Co-design Study JO - JMIR Form Res SP - e37706 VL - 6 IS - 7 KW - user-centered design KW - co-design KW - mobile health KW - mHealth KW - cancer survivors KW - disabilities N2 - Background: The transition from active treatment to long-term cancer survivorship leaves the needs of many cancer survivors unaddressed as they struggle with physical, cognitive, psychological, and social consequences of cancer and its treatment. The lack of guidance after treatment has forced cancer survivors to manage long-term effects on their own, which has an impact on their overall health, quality of life, and social participation. Mobile health (mHealth) interventions can be used to promote self-management and evidence-informed education. Objective: This study aims to design an mHealth app for cancer survivors with disabilities that will offer interventions to improve their quality of life and increase their self-efficacy to manage cancer as a chronic condition. Methods: We organized 3 co-design workshops with cancer survivors (n=5). These workshops included persona development based on data from 25 interviews with cancer survivors with disabilities; prototype ideation, where we sketched ideas for the prototype; and prototype development, where participants critiqued, and suggested improvements for, the wireframes. Results: These workshops helped us to define the challenges that cancer survivors with disabilities face as well as important considerations when designing an mHealth app for cancer survivors with disabilities, such as the need for including flexibility, engagement, socialization, and a minimalistic design. We also outline guidelines for other researchers to follow when planning their own co-design workshops, which include allowing more time for discussion among participants, having small participant groups, keeping workshops engaging and inclusive, and letting participants dream big. Conclusions: Using a co-design process aided us in developing a prototype of an mHealth app for cancer survivors with disabilities as well as a list of guidelines that other researchers can use to develop their own co-design workshops and design their app. Furthermore, working together with cancer survivors ensured that the design team had a deeper sense of empathy toward the target users and kept the focus on our ultimate goal: creating something that cancer survivors would want to use and benefit from. Future work will include usability testing of a high-fidelity prototype based on the results of these workshops. UR - https://formative.jmir.org/2022/7/e37706 UR - http://dx.doi.org/10.2196/37706 UR - http://www.ncbi.nlm.nih.gov/pubmed/35881439 ID - info:doi/10.2196/37706 ER - TY - JOUR AU - Lakhtakia, Ritu AU - Otaki, Farah AU - Alsuwaidi, Laila AU - Zary, Nabil PY - 2022/7/22 TI - Assessment as Learning in Medical Education: Feasibility and Perceived Impact of Student-Generated Formative Assessments JO - JMIR Med Educ SP - e35820 VL - 8 IS - 3 KW - self-regulated learning KW - assessment as learning KW - student-generated assessments KW - lifelong learning KW - medical education N2 - Background: Self-regulated learning (SRL) is gaining widespread recognition as a vital competency that is desirable to sustain lifelong learning, especially relevant to health professions education. Contemporary educational practices emphasize this aspect of undergraduate medical education through innovative designs of teaching and learning, such as the flipped classroom and team-based learning. Assessment practices are less commonly deployed to build capacity for SRL. Assessment as learning (AaL) can be a unique way of inculcating SRL by enabling active learning habits. It charges students to create formative assessments, reinforcing student-centered in-depth learning and critical thinking. Objective: This study aimed to explore, from the learners? perspectives, the feasibility and perceived learning impact of student-generated formative assessments. Methods: This study relied on a convergent mixed methods approach. An educational intervention was deployed on a cohort of 54 students in the second year of a 6-year undergraduate medical program as part of a single-course curriculum. The AaL intervention engaged students in generating assessments using peer collaboration, tutor facilitation, and feedback. The outcomes of the intervention were measured through quantitative and qualitative data on student perceptions, which were collected through an anonymized web-based survey and in-person focus groups, respectively. Quantitative survey data were analyzed using SPSS (IBM), and qualitative inputs underwent thematic analysis. Results: The students? overall score of agreement with the AaL educational intervention was 84%, which was strongly correlated with scores for ease and impact on a 5-point Likert-type scale. The themes that emerged from the qualitative analysis included prominent characteristics, immediate gains, and expected long-term benefits of engagement. The prominent characteristics included individuals? engagement, effective interdependencies, novelty, and time requirements. The identified immediate gains highlighted increased motivation and acquisition of knowledge and skills. The expected long-term benefits included critical thinking, problem solving, and clinical reasoning. Conclusions: As a form of AaL, student-generated assessments were perceived as viable, constructive, and stimulating educational exercises by the student authors. In the short term, the activity provided students with a fun and challenging opportunity to dive deeply into the content, be creative in designing questions, and improve exam-taking skills. In the long term, students expected an enhancement of critical thinking and the inculcation of student-centered attributes of self-regulated lifelong learning and peer collaboration, which are vital to the practice of medicine. UR - https://mededu.jmir.org/2022/3/e35820 UR - http://dx.doi.org/10.2196/35820 UR - http://www.ncbi.nlm.nih.gov/pubmed/35867379 ID - info:doi/10.2196/35820 ER - TY - JOUR AU - Tran, V. Ha AU - Nong, T. Ha T. AU - Tran, T. Thuy T. AU - Filipowicz, R. Teresa AU - Landrum, R. Kelsey AU - Pence, W. Brian AU - Le, M. Giang AU - Nguyen, X. Minh AU - Chibanda, Dixon AU - Verhey, Ruth AU - Go, F. Vivian AU - Ho, T. Hien AU - Gaynes, N. Bradley PY - 2022/7/8 TI - Adaptation of a Problem-solving Program (Friendship Bench) to Treat Common Mental Disorders Among People Living With HIV and AIDS and on Methadone Maintenance Treatment in Vietnam: Formative Study JO - JMIR Form Res SP - e37211 VL - 6 IS - 7 KW - Friendship Bench KW - Vietnam KW - Assessment-Decision-Adaptation-Production-Topical Experts-Integration-Training-Testing KW - ADAPT-ITT KW - common mental disorders KW - people living with HIV KW - PWH KW - people who inject drugs KW - PWID KW - methadone maintenance treatment KW - MMT KW - depression KW - anxiety KW - stress disorder N2 - Background: The prevalence of common mental disorders (CMDs) among people living with HIV and people who inject drugs is high worldwide and in Vietnam. However, few evidence-informed CMD programs for people living with HIV who inject drugs have been adapted for use in Vietnam. We adapted the Friendship Bench (FB), a problem-solving therapy (PST)?based program that was successfully implemented among patients with CMDs in primary health settings in Zimbabwe and Malawi for use among people living with HIV on methadone maintenance treatment (MMT) with CMDs in Hanoi, Vietnam. Objective: This study aimed to describe the adaptation process with a detailed presentation of 4 phases from the third (adaptation) to the sixth (integration) of the Assessment-Decision-Adaptation-Production-Topical Experts-Integration-Training-Testing (ADAPT-ITT) framework. Methods: The adaptation phase followed a qualitative study design to explore symptoms of CMDs, facilitators, and barriers to conducting FB for people living with HIV on MMT in Vietnam, and patient, provider, and caretaker concerns about FB. In the production phase, we revised the original program manual and developed illustrated PST cases. In the topical expert and integration phases, 2 investigators (BNG and BWP) and 3 subject matter experts (RV, DC, and GML) reviewed the manual, with reviewer comments incorporated in the final, revised manual to be used in the training. The draft program will be used in the training and testing phases. Results: The study was methodologically aligned with the ADAPT-ITT goals as we chose a proven, effective program for adaptation. Insights from the adaptation phase addressed the who, where, when, and how of FB program implementation in the MMT clinics. The ADAPT-ITT framework guided the appropriate adaptation of the program manual while maintaining the core components of the PST of the original program throughout counseling techniques in all program sessions. The deliverable of this study was an adapted FB manual to be used for training and piloting to make a final program manual. Conclusions: This study successfully illustrated the process of operationalizing the ADAPT-ITT framework to adapt a mental health program in Vietnam. This study selected and culturally adapted an evidence-informed PST program to improve CMDs among people living with HIV on MMT in Vietnam. This adapted program has the potential to effectively address CMDs among people living with HIV on MMT in Vietnam. Trial Registration: ClinicalTrials.gov NCT04790201; https://clinicaltrials.gov/ct2/show/NCT04790201 UR - https://formative.jmir.org/2022/7/e37211 UR - http://dx.doi.org/10.2196/37211 UR - http://www.ncbi.nlm.nih.gov/pubmed/35802402 ID - info:doi/10.2196/37211 ER - TY - JOUR AU - Duan, Beibei AU - Liu, Zhe AU - Liu, Weiwei AU - Gou, Baohua PY - 2022/7/8 TI - Assessing the Views and Needs of People at High Risk of Gestational Diabetes Mellitus for the Development of Mobile Health Apps: Descriptive Qualitative Study JO - JMIR Form Res SP - e36392 VL - 6 IS - 7 KW - gestational diabetes mellitus KW - high-risk groups KW - mobile health KW - mHealth KW - applications KW - user-centered design KW - qualitative research N2 - Background: Early prevention of gestational diabetes mellitus (GDM) can reduce the incidence of not only GDM, but also adverse perinatal pregnancy outcomes. Moreover, it is of great significance to prevent or reduce the occurrence of type 2 diabetes. Mobile health (mHealth) apps can help pregnant women effectively prevent GDM by providing risk prediction, lifestyle support, peer support, professional support, and other functions. Before designing mHealth apps, developers must understand the views and needs of pregnant women, and closely combine users? needs to develop app functions, in order to better improve user experience and increase the usage rate of these apps in the future. Objective: The objective of this study was to understand the views of the high-risk population of gestational diabetes mellitus on the development of mobile health apps and the demand for app functions, so as to provide a basis for the development of gestational diabetes mellitus prevention apps. Methods: Fifteen pregnant women with at least one risk factor for gestational diabetes were recruited from July to September 2021, and were interviewed via a semistructured interview using the purpose sampling method. The transcribed data were analyzed by the traditional content analysis method, and themes were extracted. Results: Respondents wanted to develop user-friendly and fully functional mobile apps for the prevention of gestational diabetes mellitus. Pregnant women's requirements for app function development include: personalized customization, accurate information support, interactive design, practical tool support, visual presentation, convenient professional support, peer support, reasonable reminder function, appropriate maternal and infant auxiliary function, and differentiated incentive function.These function settings can encourage pregnant women to improve or maintain healthy living habits during their use of the app Conclusions: This study discusses the functional requirements of target users for gestational diabetes mellitus prevention apps, which can provide reference for the development of future applications. UR - https://formative.jmir.org/2022/7/e36392 UR - http://dx.doi.org/10.2196/36392 UR - http://www.ncbi.nlm.nih.gov/pubmed/35802414 ID - info:doi/10.2196/36392 ER - TY - JOUR AU - Tossaint-Schoenmakers, Rosian AU - Kasteleyn, J. Marise AU - Rauwerdink, Anneloek AU - Chavannes, Niels AU - Willems, Sofie AU - Talboom-Kamp, A. Esther P. W. PY - 2022/7/7 TI - Development of a Quality Management Model and Self-assessment Questionnaire for Hybrid Health Care: Concept Mapping Study JO - JMIR Form Res SP - e38683 VL - 6 IS - 7 KW - quality assessment KW - hybrid health care KW - blended health care KW - eHealth KW - digital health KW - structure KW - process KW - outcome KW - concept mapping N2 - Background: Working with eHealth requires health care organizations to make structural changes in the way they work. Organizational structure and process must be adjusted to provide high-quality care. This study is a follow-up study of a systematic literature review on optimally organizing hybrid health care (eHealth and face to face) using the Donabedian Structure-Process-Outcome (SPO) framework to translate the findings into a modus operandi for health care organizations. Objective: This study aimed to develop an SPO-based quality assessment model for organizing hybrid health care using an accompanying self-assessment questionnaire. Health care organizations can use this model and a questionnaire to manage and improve their hybrid health care. Methods: Concept mapping was used to enrich and validate evidence-based knowledge from a literature review using practice-based knowledge from experts. First, brainstorming was conducted. The participants listed all the factors that contributed to the effective organization of hybrid health care and the associated outcomes. Data from the brainstorming phase were combined with data from the literature study, and duplicates were removed. Next, the participants rated the factors on importance and measurability and grouped them into clusters. Finally, using multivariate statistical analysis (multidimensional scaling and hierarchical cluster analysis) and group interpretation, an SPO-based quality management model and an accompanying questionnaire were constructed. Results: All participants (n=39) were familiar with eHealth and were health care professionals, managers, researchers, patients, or eHealth suppliers. The brainstorming and literature review resulted in a list of 314 factors. After removing the duplicates, 78 factors remained. Using multivariate statistical analyses and group interpretations, a quality management model and questionnaire incorporating 8 clusters and 33 factors were developed. The 8 clusters included the following: Vision, strategy, and organization; Quality information technology infrastructure and systems; Quality eHealth application; Providing support to health care professionals; Skills, knowledge, and attitude of health care professionals; Attentiveness to the patient; Patient outcomes; and Learning system. The SPO categories were positioned as overarching themes to emphasize the interrelations between the clusters. Finally, a proposal was made to use the self-assessment questionnaire in practice, allowing measurement of the quality of each factor. Conclusions: The quality of hybrid care is determined by organizational, technological, process, and personal factors. The 33 most important factors were clustered in a quality management model and self-assessment questionnaire called the Hybrid Health Care Quality Assessment. The model visualizes the interrelations between the factors. Using a questionnaire, each factor can be assessed to determine how effectively it is organized and developed over time. Health care organizations can use the Hybrid Health Care Quality Assessment to identify improvement opportunities for solid and sustainable hybrid health care. UR - https://formative.jmir.org/2022/7/e38683 UR - http://dx.doi.org/10.2196/38683 UR - http://www.ncbi.nlm.nih.gov/pubmed/35797097 ID - info:doi/10.2196/38683 ER - TY - JOUR AU - Summers, Charlotte AU - Griffiths, Frances AU - Cave, Jonathan AU - Panesar, Arjun PY - 2022/7/7 TI - Understanding the Security and Privacy Concerns About the Use of Identifiable Health Data in the Context of the COVID-19 Pandemic: Survey Study of Public Attitudes Toward COVID-19 and Data-Sharing JO - JMIR Form Res SP - e29337 VL - 6 IS - 7 KW - COVID-19 KW - data KW - data ethics KW - privacy KW - sharing KW - ethics KW - attitude KW - perception KW - data sharing KW - survey KW - understanding KW - security KW - health data KW - willingness N2 - Background: The COVID-19 pandemic increased the availability and use of population and individual health data to optimize tracking and analysis of the spread of the virus. Many health care services have had to rapidly digitalize in order to maintain the continuity of care provision. Data collection and dissemination have provided critical support for defending against the spread of the virus since the beginning of the pandemic; however, little is known about public perceptions of and attitudes toward the use, privacy, and security of data. Objective: The goal of this study is to better understand people?s willingness to share data in the context of the COVID-19 pandemic. Methods: A web-based survey was conducted on individuals? use of and attitudes toward health data for individuals aged 18 years and older, and in particular, with a reported diagnosis of a chronic health condition placing them at the highest risk of severe COVID-19. Results: In total, 4764 individuals responded to this web-based survey, of whom 4674 (98.1%) reported a medical diagnosis of at least 1 health condition (3 per person on average), with type 2 diabetes (n=2974, 62.7%), hypertension (n=2147, 45.2%), and type 1 diabetes (n=1299, 27.4%) being most prominent in our sample. In general, more people are comfortable with sharing anonymized data than personally identifiable data. People reported feeling comfortable sharing data that were able to benefit others; 66% (3121 respondents) would share personal identifiable data if its primary purpose was deemed beneficial for the health of others. Almost two-thirds (n=3026; 63.9%) would consent to sharing personal, sensitive health data with government or health authority organizations. Conversely, over a quarter of respondents (n=1297, 27.8%) stated that they did not trust any organization to protect their data, and 54% (n=2528) of them reported concerns about the implications of sharing personal information. Almost two-thirds (n=3054, 65%) of respondents were concerned about the provisions of appropriate legislation that seeks to prevent data misuse and hold organizations accountable in the case of data misuse. Conclusions: Although our survey focused mainly on the views of those living with chronic health conditions, the results indicate that data sensitivity is highly contextual. More people are more comfortable with sharing anonymized data rather than personally identifiable data. Willingness to share data also depended on the receiving body, highlighting trust as a key theme, in particular who may have access to shared personal health data and how they may be used in the future. The nascency of legal guidance in this area suggests a need for humanitarian guidelines for data responsibility during disaster relief operations such as pandemics and for involving the public in their development. UR - https://formative.jmir.org/2022/7/e29337 UR - http://dx.doi.org/10.2196/29337 UR - http://www.ncbi.nlm.nih.gov/pubmed/35609306 ID - info:doi/10.2196/29337 ER - TY - JOUR AU - Chukwu, Emeka AU - Garg, Lalit AU - Obande-Ogbuinya, Nkiruka AU - Chattu, Kumar Vijay PY - 2022/7/7 TI - Standardizing Primary Health Care Referral Data Sets in Nigeria: Practitioners' Survey, Form Reviews, and Profiling of Fast Healthcare Interoperability Resources (FHIR) JO - JMIR Form Res SP - e28510 VL - 6 IS - 7 KW - FHIR KW - COVID-19 KW - digital health KW - eHealth KW - mHealth KW - BlockMom KW - Nigeria KW - primary health care KW - health information KW - health information exchange KW - interoperability N2 - Background: Referral linkages are crucial for efficient functioning of primary health care (PHC) systems. Fast Healthcare Interoperability Resource (FHIR) is an open global standard that facilitates structuring of health information for coordinated exchange among stakeholders. Objective: The objective of this study is to design FHIR profiles and present methodology and the profiled FHIR resource for Maternal and Child Health referral use cases in Ebonyi state, Nigeria?a typical low- and middle-income country (LMIC) setting. Methods: Practicing doctors, midwives, and nurses were purposefully sampled and surveyed. Different referral forms were reviewed. The union of data sets from surveys and forms was aggregated and mapped to base patient FHIR resource elements, and extensions were created for data sets not in the core FHIR specification. This study also introduced FHIR and its relation to the World Health Organization?s (WHO?s) International Classification of Diseases. Results: We found many different data elements from the referral forms and survey responses even in urban settings. The resulting FHIR standard profile is published on GitHub for adaptation or adoption as necessary to aid alignment with WHO recommendations. Understanding data sets used in health care and clinical practice for information sharing is crucial in properly standardizing information sharing, particularly during the management of COVID-19 and other infectious diseases. Development organizations and governments can use this methodology and profile to fast-track FHIR standards adoption for paper and electronic information sharing at PHC systems in LMICs. Conclusions: We presented our methodology for profiling the referral resource crucial for the standardized exchange of new and expectant moms? information. Using data from frontline providers and mapping to the FHIR profile helped contextualize the standardized profile. UR - https://formative.jmir.org/2022/7/e28510 UR - http://dx.doi.org/10.2196/28510 UR - http://www.ncbi.nlm.nih.gov/pubmed/35797096 ID - info:doi/10.2196/28510 ER - TY - JOUR AU - Titov, Nickolai AU - Dear, F. Blake AU - Bisby, A. Madelyne AU - Nielssen, Olav AU - Staples, G. Lauren AU - Kayrouz, Rony AU - Cross, Shane AU - Karin, Eyal PY - 2022/7/5 TI - Measures of Daily Activities Associated With Mental Health (Things You Do Questionnaire): Development of a Preliminary Psychometric Study and Replication Study JO - JMIR Form Res SP - e38837 VL - 6 IS - 7 KW - anxiety KW - depression KW - satisfaction with life KW - COVID-19 KW - behavior KW - habits KW - cognitions KW - survey KW - mechanisms KW - psychological well-being N2 - Background: A large body of research has identified modifiable cognitions and behaviors (actions) associated with psychological health. However, little is known regarding the actions that are most strongly associated with psychological health or the frequency with which they should be performed. Objective: This paper described 2 studies that used survey methodology to create the Things You Do Questionnaire (TYDQ), which aims to identify and rank actions (items) and domains of actions (factors) most strongly associated with psychological health. Methods: We used digital marketing strategies to recruit Australian adult participants, who were asked to complete 2 web-based surveys comprising versions of the TYDQ; validated measures of depression, anxiety, and satisfaction with life; and demographic questions. In study 1, a total of 3040 participants rated how often they performed each of the 96 items comprising the TYDQ. This design was replicated in study 2, in which a 59-item version of the TYDQ was completed by 3160 participants. In both studies, the factor structure and validity were examined, as were the associations between individual TYDQ items and 3 mental health outcomes: depression, anxiety, and satisfaction with life. Results: In study 1, factor analyses revealed that a 5-factor model comprising 27 items achieved an optimum balance between brevity and variance and accounted for 38.1%, 31.4%, and 33.2% of the variance in scores on measures of depression, anxiety, and satisfaction with life, respectively. The factors were interpreted as realistic thinking, meaningful activities, goals and plans, healthy habits, and social connections. These 5 factors were more strongly associated with psychological health than those such as practicing kindness, exercising gratitude, and practicing spirituality. This pattern of results was replicated across gender, age groups, and depression severity. The 5-factor solution found in study 1 was replicated in study 2. Analyses revealed that a 21-item version accounted for 46.8%, 38.2%, and 38.1% of the variance in scores on measures of depression, anxiety, and satisfaction with life, respectively. Conclusions: These findings indicate that some actions are more strongly associated with psychological health than others and that these activities fall within 5 broad domains, which represent skills often taught in psychological treatments. Subsequent studies are planned to explore the reliability of these items and results in other samples and to examine patterns of change in scores during treatment for anxiety and depression. If replicated, these efforts will assist in the development of new psychological interventions and provide an evidence base for public mental health campaigns designed to promote good mental health and prevent the emergence of common mental disorders. UR - https://formative.jmir.org/2022/7/e38837 UR - http://dx.doi.org/10.2196/38837 UR - http://www.ncbi.nlm.nih.gov/pubmed/35788101 ID - info:doi/10.2196/38837 ER - TY - JOUR AU - Pugmire, Juliana AU - Lever Taylor, Jessie AU - Wilkes, Matt AU - Wolfberg, Adam AU - Zahradka, Nicole PY - 2022/7/5 TI - Participant Experiences of a COVID-19 Virtual Clinical Study Using the Current Health Remote Monitoring Platform: Case Study and Qualitative Analysis JO - JMIR Form Res SP - e37567 VL - 6 IS - 7 KW - virtual trial designs KW - virtual enrollment KW - digitalized health KW - theoretical domains framework KW - thematic analysis KW - remote patient monitoring N2 - Background: During the COVID-19 pandemic, individuals with a positive viral test were enrolled in a study, within 48 hours, to remotely monitor their vital signs to characterize disease progression and recovery. A virtual trial design was adopted to reduce risks to participants and the research community in a study titled Risk Stratification and Early Alerting Regarding COVID-19 Hospitalization (RiskSEARCH). The Food and Drug Administration?cleared Current Health platform with a wearable device is a continuous remote patient monitoring technology that supports hospital-at-home care and is used as a data collection tool. Enrolled participants wore the Current Health wearable device continuously for up to 30 days and took a daily symptom survey via a tablet that was provided. A qualitative substudy was conducted in parallel to better understand virtual trial implementation, including barriers and facilitators for participants. Objective: This study aimed to understand the barriers and facilitators of the user experience of interacting with a virtual care platform and research team, while participating in a fully virtual study using qualitative and quantitative data. Methods: Semistructured interviews were conducted to understand participants? experience of participating in a virtual study during a global pandemic. The schedule included their experience of enrollment and their interactions with equipment and study staff. A total of 3 RiskSEARCH participants were interviewed over telephone, and transcriptions were inductively coded and analyzed using thematic analysis. Themes were mapped onto the Theoretical Domains Framework (TDF) to identify and describe the factors that influenced study adherence. Quantitative metrics, including adherence to wearable and scheduled tasks collected as part of the RiskSEARCH main study, were paired with the interviews to present an overall picture of participation. Results: All participants exceeded our definition of a fully adherent participant and reported that participation was feasible and had a low burden. The symptoms progressively resolved during the trial. Inductive thematic analysis identified 13 main themes from the interview data, which were deductively mapped onto 11 of the 14 TDF domains, highlighting barriers and facilitators for each. Conclusions: Participants in the RiskSEARCH substudy showed high levels of adherence and engagement throughout participation. Although participants experienced some challenges in setting up and maintaining the Current Health kit (eg, charging devices), they reported feeling that the requirements of participation were both reasonable and realistic. We demonstrated that the TDF can be used for inductive thematic analysis. We anticipate expanding this work in future virtual studies and trials to identify barriers and enabling factors for implementation. UR - https://formative.jmir.org/2022/7/e37567 UR - http://dx.doi.org/10.2196/37567 UR - http://www.ncbi.nlm.nih.gov/pubmed/35671408 ID - info:doi/10.2196/37567 ER - TY - JOUR AU - Mullick, Tahsin AU - Radovic, Ana AU - Shaaban, Sam AU - Doryab, Afsaneh PY - 2022/6/24 TI - Predicting Depression in Adolescents Using Mobile and Wearable Sensors: Multimodal Machine Learning?Based Exploratory Study JO - JMIR Form Res SP - e35807 VL - 6 IS - 6 KW - adolescent KW - depression KW - uHealth KW - machine learning KW - mobile phone N2 - Background: Depression levels in adolescents have trended upward over the past several years. According to a 2020 survey by the National Survey on Drug Use and Health, 4.1 million US adolescents have experienced at least one major depressive episode. This number constitutes approximately 16% of adolescents aged 12 to 17 years. However, only 32.3% of adolescents received some form of specialized or nonspecialized treatment. Identifying worsening symptoms earlier using mobile and wearable sensors may lead to earlier intervention. Most studies on predicting depression using sensor-based data are geared toward the adult population. Very few studies look into predicting depression in adolescents. Objective: The aim of our work was to study passively sensed data from adolescents with depression and investigate the predictive capabilities of 2 machine learning approaches to predict depression scores and change in depression levels in adolescents. This work also provided an in-depth analysis of sensor features that serve as key indicators of change in depressive symptoms and the effect of variation of data samples on model accuracy levels. Methods: This study included 55 adolescents with symptoms of depression aged 12 to 17 years. Each participant was passively monitored through smartphone sensors and Fitbit wearable devices for 24 weeks. Passive sensors collected call, conversation, location, and heart rate information daily. Following data preprocessing, 67% (37/55) of the participants in the aggregated data set were analyzed. Weekly Patient Health Questionnaire-9 surveys answered by participants served as the ground truth. We applied regression-based approaches to predict the Patient Health Questionnaire-9 depression score and change in depression severity. These approaches were consolidated using universal and personalized modeling strategies. The universal strategies consisted of Leave One Participant Out and Leave Week X Out. The personalized strategy models were based on Accumulated Weeks and Leave One Week One User Instance Out. Linear and nonlinear machine learning algorithms were trained to model the data. Results: We observed that personalized approaches performed better on adolescent depression prediction compared with universal approaches. The best models were able to predict depression score and weekly change in depression level with root mean squared errors of 2.83 and 3.21, respectively, following the Accumulated Weeks personalized modeling strategy. Our feature importance investigation showed that the contribution of screen-, call-, and location-based features influenced optimal models and were predictive of adolescent depression. Conclusions: This study provides insight into the feasibility of using passively sensed data for predicting adolescent depression. We demonstrated prediction capabilities in terms of depression score and change in depression level. The prediction results revealed that personalized models performed better on adolescents than universal approaches. Feature importance provided a better understanding of depression and sensor data. Our findings can help in the development of advanced adolescent depression predictions. UR - https://formative.jmir.org/2022/6/e35807 UR - http://dx.doi.org/10.2196/35807 UR - http://www.ncbi.nlm.nih.gov/pubmed/35749157 ID - info:doi/10.2196/35807 ER - TY - JOUR AU - Padhee, Swati AU - Nave Jr, K. Gary AU - Banerjee, Tanvi AU - Abrams, M. Daniel AU - Shah, Nirmish PY - 2022/6/23 TI - Improving Pain Assessment Using Vital Signs and Pain Medication for Patients With Sickle Cell Disease: Retrospective Study JO - JMIR Form Res SP - e36998 VL - 6 IS - 6 KW - pain management KW - pain medication KW - vital signs KW - sickle cell disease KW - machine learning N2 - Background: Sickle cell disease (SCD) is the most common inherited blood disorder affecting millions of people worldwide. Most patients with SCD experience repeated, unpredictable episodes of severe pain. These pain episodes are the leading cause of emergency department visits among patients with SCD and may last for several weeks. Arguably, the most challenging aspect of treating pain episodes in SCD is assessing and interpreting a patient?s pain intensity level. Objective: This study aims to learn deep feature representations of subjective pain trajectories using objective physiological signals collected from electronic health records. Methods: This study used electronic health record data collected from 496 Duke University Medical Center participants over 5 consecutive years. Each record contained measures for 6 vital signs and the patient?s self-reported pain score, with an ordinal range from 0 (no pain) to 10 (severe and unbearable pain). We also extracted 3 features related to medication: medication type, medication status (given or applied, or missed or removed or due), and total medication dosage (mg/mL). We used variational autoencoders for representation learning and designed machine learning classification algorithms to build pain prediction models. We evaluated our results using an accuracy and confusion matrix and visualized the qualitative data representations. Results: We designed a classification model using raw data and deep representational learning to predict subjective pain scores with average accuracies of 82.8%, 70.6%, 49.3%, and 47.4% for 2-point, 4-point, 6-point, and 11-point pain ratings, respectively. We observed that random forest classification models trained on deep represented features outperformed models trained on unrepresented data for all pain rating scales. We observed that at varying Likert scales, our models performed better when provided with medication data along with vital signs data. We visualized the data representations to understand the underlying latent representations, indicating neighboring representations for similar pain scores with a higher resolution of pain ratings. Conclusions: Our results demonstrate that medication information (the type of medication, total medication dosage, and whether the medication was given or missed) can significantly improve subjective pain prediction modeling compared with modeling with only vital signs. This study shows promise in data-driven estimated pain scores that will help clinicians with additional information about the patient?s condition, in addition to the patient?s self-reported pain scores. UR - https://formative.jmir.org/2022/6/e36998 UR - http://dx.doi.org/10.2196/36998 UR - http://www.ncbi.nlm.nih.gov/pubmed/35737453 ID - info:doi/10.2196/36998 ER - TY - JOUR AU - Schinle, Markus AU - Erler, Christina AU - Kaliciak, Mayumi AU - Milde, Christopher AU - Stock, Simon AU - Gerdes, Marius AU - Stork, Wilhelm PY - 2022/6/22 TI - Digital Health Apps in the Context of Dementia: Questionnaire Study to Assess the Likelihood of Use Among Physicians JO - JMIR Form Res SP - e35961 VL - 6 IS - 6 KW - digital health applications KW - likelihood of use KW - usability KW - adherence KW - dementia KW - screening KW - treatment KW - physician KW - eHealth KW - questionnaire KW - mobile phone N2 - Background: Age-related diseases such as dementia are playing an increasingly important role in global population development. Thus, prevention, diagnostics, and interventions require more accessibility, which can be realized through digital health apps. With the app on prescription, Germany made history by being the first country worldwide to offer physicians the possibility to prescribe and reimburse digital health apps as of the end of the year 2020. Objective: Considering the lack of knowledge about correlations with the likelihood of use among physicians, this study aimed to address the question of what makes the use of a digital health app by physicians more likely. Methods: We developed and validated a novel measurement tool?the Digital Health Compliance Questionnaire (DHCQ)?in an interdisciplinary collaboration of experts to assess the role of proposed factors in the likelihood of using a health app. Therefore, a web-based survey was conducted to evaluate the likelihood of using a digital app called DemPredict to screen for Alzheimer dementia. Within this survey, 5 latent dimensions (acceptance, attitude toward technology, technology experience, payment for time of use, and effort of collection), the dependent variable likelihood of use, and answers to exploratory questions were recorded and tested within directed correlations. Following a non?probability-sampling strategy, the study was completed by 331 physicians from Germany in the German language, of whom 301 (90.9%) fulfilled the study criteria (eg, being in regular contact with patients with dementia). These data were analyzed using a range of statistical methods to validate the dimensions of the DHCQ. Results: The DHCQ revealed good test theoretical measures?it showed excellent fit indexes (Tucker-Lewis index=0.98; comparative fit index=0.982; standardized root mean square residual=0.073; root mean square error of approximation=0.037), good internal consistency (Cronbach ?=.83), and signs of moderate to large correlations between the DHCQ dimensions and the dependent variable. The correlations between the variables acceptance, attitude toward technology, technology experience, and payment for the time of use and the dependent variable likelihood of use ranged from 0.29 to 0.79, and the correlation between effort of the collection and likelihood of use was ?0.80. In addition, we found high levels of skepticism regarding data protection, and the age of the participants was found to be negatively related to their technical experience and attitude toward technology. Conclusions: In the context of the results, increased communication between the medical and technology sectors and significantly more awareness raising are recommended to make the use of digital health apps more attractive to physicians as they can be adjusted to their everyday needs. Further research could explore the connection between areas such as adherence on the patient side and its impact on the likelihood of use by physicians. UR - https://formative.jmir.org/2022/6/e35961 UR - http://dx.doi.org/10.2196/35961 UR - http://www.ncbi.nlm.nih.gov/pubmed/35731567 ID - info:doi/10.2196/35961 ER - TY - JOUR AU - Cooper, R. Ian AU - Lindsay, Cameron AU - Fraser, Keaton AU - Hill, T. Tiffany AU - Siu, Andrew AU - Fletcher, Sarah AU - Klimas, Jan AU - Hamilton, Michee-Ana AU - Frazer, D. Amanda AU - Humphrys, Elka AU - Koepke, Kira AU - Hedden, Lindsay AU - Price, Morgan AU - McCracken, K. Rita PY - 2022/6/22 TI - Finding Primary Care?Repurposing Physician Registration Data to Generate a Regionally Accurate List of Primary Care Clinics: Development and Validation of an Open-Source Algorithm JO - JMIR Form Res SP - e34141 VL - 6 IS - 6 KW - physicians, primary care KW - primary health care KW - health services accessibility KW - practice patterns, physicians KW - physicians? offices KW - computing methodologies KW - algorithms N2 - Background: Some Canadians have limited access to longitudinal primary care, despite its known advantages for population health. Current initiatives to transform primary care aim to increase access to team-based primary care clinics. However, many regions lack a reliable method to enumerate clinics, limiting estimates of clinical capacity and ongoing access gaps. A region-based complete clinic list is needed to effectively describe clinic characteristics and to compare primary care outcomes at the clinic level. Objective: The objective of this study is to show how publicly available data sources, including the provincial physician license registry, can be used to generate a verifiable, region-wide list of primary care clinics in British Columbia, Canada, using a process named the Clinic List Algorithm (CLA). Methods: The CLA has 10 steps: (1) collect data sets, (2) develop clinic inclusion and exclusion criteria, (3) process data sets, (4) consolidate data sets, (5) transform from list of physicians to initial list of clinics, (6) add additional metadata, (7) create working lists, (8) verify working lists, (9) consolidate working lists, and (10) adjust processing steps based on learnings. Results: The College of Physicians and Surgeons of British Columbia Registry contained 13,726 physicians, at 2915 unique addresses, 6942 (50.58%) of whom were family physicians (FPs) licensed to practice in British Columbia. The CLA identified 1239 addresses where primary care was delivered by 4262 (61.39%) FPs. Of the included addresses, 84.50% (n=1047) were in urban locations, and there was a median of 2 (IQR 2-4, range 1-23) FPs at each unique address. Conclusions: The CLA provides a region-wide description of primary care clinics that improves on simple counts of primary care providers or self-report lists. It identifies the number and location of primary care clinics and excludes primary care providers who are likely not providing community-based primary care. Such information may be useful for estimates of capacity of primary care, as well as for policy planning and research in regions engaged in primary care evaluation or transformation. UR - https://formative.jmir.org/2022/6/e34141 UR - http://dx.doi.org/10.2196/34141 UR - http://www.ncbi.nlm.nih.gov/pubmed/35731556 ID - info:doi/10.2196/34141 ER - TY - JOUR AU - Poulton, Antoinette AU - Chen, Evelyn Li Peng AU - Dali, Gezelle AU - Fox, Michael AU - Hester, Robert PY - 2022/5/30 TI - Web-Based Independent Versus Laboratory-Based Stop-Signal Task Performance: Within-Subjects Counterbalanced Comparison Study JO - J Med Internet Res SP - e32922 VL - 24 IS - 5 KW - Stop-Signal Task KW - response inhibition KW - inhibitory control KW - online assessment KW - web-based assessment KW - cognition N2 - Background: Considered a facet of behavioral impulsivity, response inhibition facilitates adaptive and goal-directed behavior. It is often assessed using the Stop-Signal Task (SST), which is presented on stand-alone computers under controlled laboratory conditions. Sample size may consequently be a function of cost or time and sample diversity constrained to those willing or able to attend the laboratory. Statistical power and generalizability of results might, in turn, be impacted. Such limitations may potentially be overcome via the implementation of web-based testing. Objective: The aim of this study was to investigate if there were differences between variables derived from a web-based SST when it was undertaken independently?that is, outside the laboratory, on any computer, and in the absence of researchers?versus when it was performed under laboratory conditions. Methods: We programmed a web-based SST in HTML and JavaScript and employed a counterbalanced design. A total of 166 individuals (mean age 19.72, SD 1.85, range 18-36 years; 146/166, 88% female) were recruited. Of them, 79 undertook the independent task prior to visiting the laboratory and 78 completed the independent task following their laboratory visit. The average time between SST testing was 3.72 (SD 2.86) days. Dependent samples and Bayesian paired samples t tests were used to examine differences between laboratory-based and independent SST variables. Correlational analyses were conducted on stop-signal reaction times (SSRT). Results: After exclusions, 123 participants (mean age 19.73, SD 1.97 years) completed the SST both in the laboratory and independently. While participants were less accurate on go trials and exhibited reduced inhibitory control when undertaking the independent?compared to the laboratory-based?SST, there was a positive association between the SSRT of each condition (r=.48; P<.001; 95% CI 0.33-0.61). Conclusions: Findings suggest a web-based SST, which participants undertake on any computer, at any location, and in the absence of the researcher, is a suitable measure of response inhibition. UR - https://www.jmir.org/2022/5/e32922 UR - http://dx.doi.org/10.2196/32922 UR - http://www.ncbi.nlm.nih.gov/pubmed/35635745 ID - info:doi/10.2196/32922 ER - TY - JOUR AU - DeGuzman, Baker Pamela AU - Vogel, L. David AU - Bernacchi, Veronica AU - Scudder, A. Margaret AU - Jameson, J. Mark PY - 2022/5/19 TI - Self-reliance, Social Norms, and Self-stigma as Barriers to Psychosocial Help-Seeking Among Rural Cancer Survivors With Cancer-Related Distress: Qualitative Interview Study JO - JMIR Form Res SP - e33262 VL - 6 IS - 5 KW - cancer survivorship KW - cancer-related distress KW - rural health KW - self-stigma KW - help-seeking KW - psychosocial referral KW - support networks KW - self-reliance N2 - Background: Even when technology allows rural cancer survivors to connect with supportive care providers from a distance, uptake of psychosocial referrals is low. Fewer than one-third of participants in a telemedicine intervention for identifying rural survivors with high distress and connecting them with care accepted psychosocial referral. Objective: The purpose of this research was to examine the reasons for which rural cancer survivors did not accept a psychosocial referral. Methods: We utilized a qualitative design to address the research purpose. We interviewed participants who had been offered psychosocial referral. Semistructured interviews were conducted 6 weeks later (n=14), and structured interviews were conducted 9 months later (n=6). Data were analyzed descriptively using an inductive approach. Results: Ultimately, none of the rural cancer survivors (0/14, 0%) engaged with a psychosocial care provider, including those who had originally accepted referrals (0/4, 0%) for further psychosocial care. When explaining their decisions, survivors minimized their distress, emphasizing their self-reliance and the need to handle distress on their own. They expressed a preference for dealing with distress via informal support networks, which was often limited to close family members. No survivors endorsed public stigma as a barrier to accepting psychosocial help, but several suggested that self-stigma associated with not being able to handle their own distress was a reason for not seeking care. Conclusions: Rural cancer survivors? willingness to accept a psychosocial referral may be mediated by the rural cultural norm of self-reliance and by self-stigma. Interventions to address referral uptake may benefit from further illumination of these relationships as well as a strength-based approach that emphasizes positive aspects of the rural community and individual self-affirmation. UR - https://formative.jmir.org/2022/5/e33262 UR - http://dx.doi.org/10.2196/33262 UR - http://www.ncbi.nlm.nih.gov/pubmed/35588367 ID - info:doi/10.2196/33262 ER - TY - JOUR AU - Kamradt, Martina AU - Poß-Doering, Regina AU - Szecsenyi, Joachim PY - 2022/5/18 TI - Exploring Physician Perspectives on Using Real-world Care Data for the Development of Artificial Intelligence?Based Technologies in Health Care: Qualitative Study JO - JMIR Form Res SP - e35367 VL - 6 IS - 5 KW - artificial intelligence?based solutions KW - data donation KW - qualitative research KW - Germany KW - artificial intelligence KW - requirement analysis KW - physician perspective KW - real-world data KW - big data KW - data pool KW - interview KW - qualitative N2 - Background: Development of artificial intelligence (AI)?based technologies in health care is proceeding rapidly. The sharing and release of real-world data are key practical issues surrounding the implementation of AI solutions into existing clinical practice. However, data derived from daily patient care are necessary for initial training, and continued data supply is needed for the ongoing training, validation, and improvement of AI-based solutions. Data may need to be shared across multiple institutions and settings for the widespread implementation and high-quality use of these solutions. To date, solutions have not been widely implemented in Germany to meet the challenge of providing a sufficient data volume for the development of AI-based technologies for research and third-party entities. The Protected Artificial Intelligence Innovation Environment for Patient-Oriented Digital Health Solutions (pAItient) project aims to meet this challenge by creating a large data pool that feeds on the donation of data derived from daily patient care. Prior to building this data pool, physician perspectives regarding data donation for AI-based solutions should be studied. Objective: This study explores physician perspectives on providing and using real-world care data for the development of AI-based solutions in health care in Germany. Methods: As a part of the requirements analysis preceding the pAItient project, this qualitative study explored physician perspectives and expectations regarding the use of data derived from daily patient care in AI-based solutions. Semistructured, guide-based, and problem-centered interviews were audiorecorded, deidentified, transcribed verbatim, and analyzed inductively in a thematically structured approach. Results: Interviews (N=8) with a mean duration of 24 (SD 7.8) minutes were conducted with 6 general practitioners and 2 hospital-based physicians. The mean participant age was 54 (SD 14.1; range 30-74) years, with an average experience as a physician of 25 (SD 13.9; range 1-45) years. Self-rated affinity toward modern information technology varied from very high to low (5-point Likert scale: mean 3.75, SD 1.1). All participants reported they would support the development of AI-based solutions in research contexts by donating deidentified data derived from daily patient care if subsequent data use was made transparent to them and their patients and the benefits for patient care were clear. Contributing to care optimization and efficiency were cited as motivation for potential data donation. Concerns regarding workflow integration (time and effort), appropriate deidentification, and the involvement of third-party entities with economic interests were discussed. The donation of data in reference to psychosomatic treatment needs was viewed critically. Conclusions: The interviewed physicians reported they would agree to use real-world care data to support the development of AI-based solutions with a clear benefit for daily patient care. Joint ventures with third-party entities were viewed critically and should focus on care optimization and patient benefits rather than financial interests. UR - https://formative.jmir.org/2022/5/e35367 UR - http://dx.doi.org/10.2196/35367 UR - http://www.ncbi.nlm.nih.gov/pubmed/35583921 ID - info:doi/10.2196/35367 ER - TY - JOUR AU - Ahmad, Kashif AU - Alam, Firoj AU - Qadir, Junaid AU - Qolomany, Basheer AU - Khan, Imran AU - Khan, Talhat AU - Suleman, Muhammad AU - Said, Naina AU - Hassan, Zohaib Syed AU - Gul, Asma AU - Househ, Mowafa AU - Al-Fuqaha, Ala PY - 2022/5/11 TI - Global User-Level Perception of COVID-19 Contact Tracing Applications: Data-Driven Approach Using Natural Language Processing JO - JMIR Form Res SP - e36238 VL - 6 IS - 5 KW - COVID-19 KW - sentiment analysis KW - contact tracing applications KW - NLP KW - text classification KW - BERT KW - fastText KW - transformers KW - RoBerta N2 - Background: Contact tracing has been globally adopted in the fight to control the infection rate of COVID-19. To this aim, several mobile apps have been developed. However, there are ever-growing concerns over the working mechanism and performance of these applications. The literature already provides some interesting exploratory studies on the community?s response to the applications by analyzing information from different sources, such as news and users? reviews of the applications. However, to the best of our knowledge, there is no existing solution that automatically analyzes users? reviews and extracts the evoked sentiments. We believe such solutions combined with a user-friendly interface can be used as a rapid surveillance tool to monitor how effective an application is and to make immediate changes without going through an intense participatory design method. Objective: In this paper, we aim to analyze the efficacy of AI and NLP techniques for automatically extracting and classifying the polarity of users? sentiments by proposing a sentiment analysis framework to automatically analyze users? reviews on COVID-19 contact tracing mobile apps. We also aim to provide a large-scale annotated benchmark data set to facilitate future research in the domain. As a proof of concept, we also developed a web application based on the proposed solutions, which is expected to help the community quickly analyze the potential of an application in the domain. Methods: We propose a pipeline starting from manual annotation via a crowd-sourcing study and concluding with the development and training of artificial intelligence (AI) models for automatic sentiment analysis of users? reviews. In detail, we collected and annotated a large-scale data set of user reviews on COVID-19 contact tracing applications. We used both classical and deep learning methods for classification experiments. Results: We used 8 different methods on 3 different tasks, achieving up to an average F1 score of 94.8%, indicating the feasibility of the proposed solution. The crowd-sourcing activity resulted in a large-scale benchmark data set composed of 34,534 manually annotated reviews. Conclusions: The existing literature mostly relies on the manual or exploratory analysis of users? reviews on applications, which is tedious and time-consuming. In existing studies, generally, data from fewer applications are analyzed. In this work, we showed that AI and natural language processing techniques provide good results for analyzing and classifying users? sentiments? polarity and that automatic sentiment analysis can help to analyze users? responses more accurately and quickly. We also provided a large-scale benchmark data set. We believe the presented analysis, data set, and proposed solutions combined with a user-friendly interface can be used as a rapid surveillance tool to analyze and monitor mobile apps deployed in emergency situations leading to rapid changes in the applications without going through an intense participatory design method. UR - https://formative.jmir.org/2022/5/e36238 UR - http://dx.doi.org/10.2196/36238 UR - http://www.ncbi.nlm.nih.gov/pubmed/35389357 ID - info:doi/10.2196/36238 ER - TY - JOUR AU - Arigo, Danielle AU - Torous, John PY - 2022/5/2 TI - Development of a Mobile Assessment Tool for Understanding Social Comparison Processes Among Individuals With Schizophrenia: Two-Phase Survey Study JO - JMIR Form Res SP - e36541 VL - 6 IS - 5 KW - schizophrenia KW - social comparison KW - mobile assessment KW - smartphone app KW - variability N2 - Background: Digital tools may help to address social deficits in schizophrenia, particularly those that engage social comparison processes (ie, evaluating oneself relative to others). Yet, little is known about social comparison processes in schizophrenia or how best to capture between- versus within-person variability, which is critical to engaging comparisons in digital interventions. Objective: The goals of this pilot study were to (1) better understand affective responses to social comparisons among individuals with schizophrenia, relative to healthy controls, using a validated global self-report measure; and (2) test a new brief, mobile assessment of affective responses to social comparison among individuals with schizophrenia, relative to the full measure. This study was conducted in 2 phases. Methods: We first compared self-reported affective responses to social comparisons between individuals with schizophrenia (n=39) and healthy controls (n=38) using a traditional self-report measure, at 2 time points. We examined the temporal stability in responses and differences between groups. We then evaluated the performance of brief, mobile assessment of comparison responses among individuals with schizophrenia, completed over 12 weeks (n=31). Results: Individuals with schizophrenia showed greater variability in affective responses to social comparison than controls on traditional measures and completed an average of 7.46 mobile assessments over 12 weeks. Mobile assessments captured within-person variability in affective responses in the natural environment (intraclass correlation coefficients of 0.40-0.60). Average scores for mobile assessments were positively correlated with responses to traditional measures. Conclusions: Affective responses to social comparison vary both between and within individuals with schizophrenia and capturing this variability via smartphone surveys shows some evidence of feasibility. As affective variability is a potential indicator of poor outcomes among individuals with mental health conditions, in the future, a brief, mobile assessment of affective responses to social comparisons may be useful for screening among individuals with schizophrenia. Further research on this process is needed to identify when specific comparison messaging may be most effective in digital interventions and could suggest new therapeutic targets for illnesses such as schizophrenia. UR - https://formative.jmir.org/2022/5/e36541 UR - http://dx.doi.org/10.2196/36541 UR - http://www.ncbi.nlm.nih.gov/pubmed/35499856 ID - info:doi/10.2196/36541 ER - TY - JOUR AU - Lindhiem, Oliver AU - Goel, Mayank AU - Shaaban, Sam AU - Mak, J. Kristie AU - Chikersal, Prerna AU - Feldman, Jamie AU - Harris, L. Jordan PY - 2022/4/25 TI - Objective Measurement of Hyperactivity Using Mobile Sensing and Machine Learning: Pilot Study JO - JMIR Form Res SP - e35803 VL - 6 IS - 4 KW - assessment KW - machine learning KW - hyperactivity KW - attention-deficit/hyperactivity disorder KW - ADHD KW - wearables N2 - Background: Although hyperactivity is a core symptom of attention-deficit/hyperactivity disorder (ADHD), there are no objective measures that are widely used in clinical settings. Objective: We describe the development of a smartwatch app to measure hyperactivity in school-age children. The LemurDx prototype is a software system for smartwatches that uses wearable sensor technology and machine learning to measure hyperactivity. The goal is to differentiate children with ADHD combined presentation (a combination of inattentive and hyperactive/impulsive presentations) or predominantly hyperactive/impulsive presentation from children with typical levels of activity. Methods: In this pilot study, we recruited 30 children, aged 6 to 11 years, to wear a smartwatch with the LemurDx app for 2 days. Parents also provided activity labels for 30-minute intervals to help train the algorithm. Half of the participants had ADHD combined presentation or predominantly hyperactive/impulsive presentation (n=15), and half were in the healthy control group (n=15). Results: The results indicated high usability scores and an overall diagnostic accuracy of 0.89 (sensitivity=0.93; specificity=0.86) when the motion sensor output was paired with the activity labels. Conclusions: State-of-the-art sensors and machine learning may provide a promising avenue for the objective measurement of hyperactivity. UR - https://formative.jmir.org/2022/4/e35803 UR - http://dx.doi.org/10.2196/35803 UR - http://www.ncbi.nlm.nih.gov/pubmed/35468089 ID - info:doi/10.2196/35803 ER - TY - JOUR AU - Guglani, Sheena AU - Liddy, Clare AU - Afkham, Amir AU - Mitchell, Rhea AU - Keely, Erin PY - 2022/4/22 TI - The Ontario Electronic Consultation (eConsult) Service: Cross-sectional Analysis of Utilization Data for 2 Models JO - JMIR Form Res SP - e32101 VL - 6 IS - 4 KW - eConsult KW - access to care KW - utilization KW - consultation KW - primary care provider KW - direct-to-specialist KW - Ontario KW - healthcare system N2 - Background: The Ontario electronic consultation (eConsult) service allows a primary care provider (PCP) to access specialist advice through 2 models: the direct-to-specialist (DTS) model, where PCPs select a specialist from a directory, and the Building Access to Specialists Through eConsultation (BASE)?managed specialty service, where PCPs choose a specialty group and are assigned a specialist from a qualified pool based on availability. Objective: The aim of this study is to examine patterns of use between the 2 models of eConsult delivery. Methods: We conducted a cross-sectional analysis of utilization data collected from eConsults completed between October 2018 and September 2019. Cases were grouped based on the model used for submission (ie, BASE or DTS). Each model was assessed for the number of cases over time, specialty distribution, proportion resulting in new or additional information, impact on PCPs? decisions to refer, and billing time. Results: PCPs submitted 26,121 eConsults during the study period. The monthly case volume increased by 43% over the duration of the study, primarily in the BASE model (66% compared to 6% for DTS). PCPs were able to confirm a course of action that they originally had in mind in 41.4% (6373/15,376) of BASE cases and 41.3% (3363/8136) of DTS cases and received advice for a new or additional course of action in 54.7% (8418/15,376) of BASE cases and 56.3% (4582/8136) of DTS cases. A referral was originally contemplated but avoided in 51.3% (7887/15,376) of BASE cases and 53.3% (4336/8136) of DTS cases, originally contemplated and still needed in 19.4% (2986/15,376) of BASE cases and 17.7% (1438/8136) of DTS cases, and neither originally contemplated nor needed in 21.7% (3334/15,376) of BASE cases and 21.9% (1781/8136) of DTS cases. Conclusions: Both eConsult models had strong uptake. Use patterns varied between models, with the majority of growth occurring under BASE, but survey responses showed that both models provided similar outcomes in terms of new information offered and impact on decision to refer. UR - https://formative.jmir.org/2022/4/e32101 UR - http://dx.doi.org/10.2196/32101 UR - http://www.ncbi.nlm.nih.gov/pubmed/35451985 ID - info:doi/10.2196/32101 ER - TY - JOUR AU - Kim, Hyeoneui AU - Jung, Jinsun AU - Choi, Jisung PY - 2022/4/21 TI - Developing a Dietary Lifestyle Ontology to Improve the Interoperability of Dietary Data: Proof-of-Concept Study JO - JMIR Form Res SP - e34962 VL - 6 IS - 4 KW - dietary lifestyle data KW - person-generated health data KW - ontology KW - common data element KW - data interoperability KW - data standardization KW - dietary KW - health informatics N2 - Background: Dietary habits offer crucial information on one's health and form a considerable part of the patient-generated health data. Dietary data are collected through various channels and formats; thus, interoperability is a significant challenge to reusing this type of data. The vast scope of dietary concepts and the colloquial expression style add difficulty to standardizing the data. The interoperability issues of dietary data can be addressed through Common Data Elements with metadata annotation to some extent. However, making culture-specific dietary habits and questionnaire-based dietary assessment data interoperable still requires substantial efforts. Objective: The main goal of this study was to address the interoperability challenge of questionnaire-based dietary data from different cultural backgrounds by combining ontological curation and metadata annotation of dietary concepts. Specifically, this study aimed to develop a Dietary Lifestyle Ontology (DILON) and demonstrate the improved interoperability of questionnaire-based dietary data by annotating its main semantics with DILON. Methods: By analyzing 1158 dietary assessment data elements (367 in Korean and 791 in English), 515 dietary concepts were extracted and used to construct DILON. To demonstrate the utility of DILON in addressing the interoperability challenges of questionnaire-based multicultural dietary data, we developed 10 competency questions that asked to identify data elements sharing the same dietary topics and assessment properties. We instantiated 68 data elements on dietary habits selected from Korean and English questionnaires and annotated them with DILON to answer the competency questions. We translated the competency questions into Semantic Query-Enhanced Web Rule Language and reviewed the query results for accuracy. Results: DILON was built with 262 concept classes and validated with ontology validation tools. A small overlap (72 concepts) in the concepts extracted from the questionnaires in 2 languages indicates that we need to pay closer attention to representing culture-specific dietary concepts. The Semantic Query-Enhanced Web Rule Language queries reflecting the 10 competency questions yielded correct results. Conclusions: Ensuring the interoperability of dietary lifestyle data is a demanding task due to its vast scope and variations in expression. This study demonstrated that we could improve the interoperability of dietary data generated in different cultural contexts and expressed in various styles by annotating their core semantics with DILON. UR - https://formative.jmir.org/2022/4/e34962 UR - http://dx.doi.org/10.2196/34962 UR - http://www.ncbi.nlm.nih.gov/pubmed/35451991 ID - info:doi/10.2196/34962 ER - TY - JOUR AU - Teramoto, Kei AU - Kuwata, Shigeki PY - 2022/4/21 TI - Design and Evaluation of a Smartphone Medical Guidance App for Outpatients of Large-Scale Medical Institutions: Retrospective Observational Study JO - JMIR Form Res SP - e32990 VL - 6 IS - 4 KW - mHealth KW - outpatient clinics KW - electronic medical records KW - COVID-19 KW - EHR N2 - Background: The greatest stressor for outpatients is the waiting time before an examination. If the patient is able to use their smartphone to check in with reception, the patient can wait for their examination at any location, and the burden of waiting can be reduced. Objective: This study aimed to report the system design and postintroductory outcomes of the Tori RinRin (TR2) system that was developed to reduce outpatient burden imposed by wait times before examination. Methods: The TR2 system was introduced at Tottori University Hospital, a large medical facility that accepts a daily average of 1500 outpatients. The system, which links the hospital?s electronic medical record database with patients? mobile devices, has the following functions: (1) GPS-based examination check-in processing and (2) sending appointment notification messages via a cloud notification service. In order to evaluate the usefulness of the TR2 system, we surveyed the utilization rate of the TR2 system among outpatients, implemented a user questionnaire, and polled the average time required for patients to respond to call notifications about their turn. Results: The 3-month average of TR2 users 9 months after the TR 2 system introduction was 17.9% (14,536/81,066). In an investigation of 363 subjects, the mean examination call message response time using the TR2 system was 31 seconds (median 14 seconds). Among 166 subjects who responded to a user survey, 86.7% (144/166) said that the system helped reduce the burden of waiting time. Conclusions: The app allowed 17.9% of outpatients at a large medical facility to check in remotely and wait for examinations anywhere. Hence, it is effective in preventing the spread of infection, especially during pandemics such as that of coronavirus disease. The app reported in this study is beneficial for large medical facilities striving to reduce outpatient burden imposed by wait times. UR - https://formative.jmir.org/2022/4/e32990 UR - http://dx.doi.org/10.2196/32990 UR - http://www.ncbi.nlm.nih.gov/pubmed/34818208 ID - info:doi/10.2196/32990 ER - TY - JOUR AU - Rosenberg, Ellen Nora AU - Tembo, A. Tapiwa AU - Simon, R. Katherine AU - Mollan, Katie AU - Rutstein, E. Sarah AU - Mwapasa, Victor AU - Masiano, Steven AU - Huffstetler, E. Hanna AU - Go, Vivian AU - Kim, H. Maria PY - 2022/4/19 TI - Development of a Blended Learning Approach to Delivering HIV-Assisted Contact Tracing in Malawi: Applied Theory and Formative Research JO - JMIR Form Res SP - e32899 VL - 6 IS - 4 KW - HIV KW - e-learning KW - digitial learning KW - blended learning KW - digital KW - contact tracing KW - assisted partner services N2 - Background: Despite progress toward the Joint United Nations Programme on HIV/AIDS ?95-95-95? targets (95% of HIV-positive persons tested, 95% of tested persons on treatment, and 95% of treated persons virally suppressed), a gap remains in achieving the first 95% target. Assisted contact tracing (ACT), in which health workers support HIV-positive index clients to recruit their contacts (sexual partners and children) for HIV testing, efficiently identifies HIV-positive persons in need of treatment. Although many countries, including Malawi, began implementing ACT, testing outcomes in routine settings have been worse than those in trial settings. Objective: The aim of this paper is to use formative research and frameworks to develop and digitize an implementation package to bridge the gap between ACT research and practice. Methods: Semistructured qualitative research was conducted in 2019 in Malawi with key informants. Barriers and facilitators to intervention delivery were identified using the Consolidated Framework for Implementation Research. Approaches to digitization were examined using human-centered design principles. Results: Limited clinic coordination and health worker capacity to address the complexities of ACT were identified as barriers. Ongoing individual training consisting of learning, observing, practicing, and receiving feedback, as well as group problem-solving were identified as facilitators. Important features of digitization included (1) culturally relevant visual content, (2) capability of offline use, and (3) simple designs and basic editing to keep costs low. Conclusions: Formative research and frameworks played a key role in designing and digitizing an implementation package for ACT delivery in a low-income setting such as Malawi. UR - https://formative.jmir.org/2022/4/e32899 UR - http://dx.doi.org/10.2196/32899 UR - http://www.ncbi.nlm.nih.gov/pubmed/35438644 ID - info:doi/10.2196/32899 ER - TY - JOUR AU - Mackey, Rachel AU - Gleason, Ann AU - Ciulla, Robert PY - 2022/4/15 TI - A Novel Method for Evaluating Mobile Apps (App Rating Inventory): Development Study JO - JMIR Mhealth Uhealth SP - e32643 VL - 10 IS - 4 KW - mobile health apps KW - app rating KW - app analysis methodology KW - app market research KW - mobile phone N2 - Background: Selecting and integrating health-related apps into patient care is impeded by the absence of objective guidelines for identifying high-quality apps from the many thousands now available. Objective: This study aimed to evaluate the App Rating Inventory, which was developed by the Defense Health Agency?s Connected Health branch, to support clinical decisions regarding app selection and evaluate medical and behavioral apps. Methods: To enhance the tool?s performance, eliminate item redundancy, reduce scoring system subjectivity, and ensure a broad application of App Rating Inventory?derived results, inventory development included 3 rounds of validation testing and 2 trial periods conducted over a 6-month interval. The development focused on content validity testing, dimensionality (ie, whether the tool?s criteria performed as operationalized), factor and commonality analysis, and interrater reliability (reliability scores improved from 0.62 to 0.95 over the course of development). Results: The development phase culminated in a review of 248 apps for a total of 6944 data points and a final 28-item, 3-category app rating system. The App Rating Inventory produces scores for the following three categories: evidence (6 items), content (11 items), and customizability (11 items). The final (fourth) metric is the total score, which constitutes the sum of the 3 categories. All 28 items are weighted equally; no item is considered more (or less) important than any other item. As the scoring system is binary (either the app contains the feature or it does not), the ratings? results are not dependent on a rater?s nuanced assessments. Conclusions: Using predetermined search criteria, app ratings begin with an environmental scan of the App Store and Google Play. This first step in market research funnels hundreds of apps in a given disease category down to a manageable top 10 apps that are, thereafter, rated using the App Rating Inventory. The category and final scores derived from the rating system inform the clinician about whether an app is evidence informed and easy to use. Although a rating allows a clinician to make focused decisions about app selection in a context where thousands of apps are available, clinicians must weigh the following factors before integrating apps into a treatment plan: clinical presentation, patient engagement and preferences, available resources, and technology expertise. UR - https://mhealth.jmir.org/2022/4/e32643 UR - http://dx.doi.org/10.2196/32643 UR - http://www.ncbi.nlm.nih.gov/pubmed/35436227 ID - info:doi/10.2196/32643 ER - TY - JOUR AU - Nour, Radwa AU - Powell, Leigh AU - Alnakhi, K. Wafa AU - Mamdouh, Heba AU - Zidoun, Youness AU - Hussain, Y. Hamid AU - Al Suwaidi, Hanan AU - Zary, Nabil PY - 2022/4/12 TI - Adult Vaccine Hesitancy Scale in Arabic and French: Protocol for Translation and Validation in the World Health Organization Eastern Mediterranean Region JO - JMIR Res Protoc SP - e36928 VL - 11 IS - 4 KW - scale KW - instrument KW - vaccine hesitancy KW - COVID-19 KW - validation KW - translation KW - Arabic KW - French KW - EMRO N2 - Background: The world as we know it changed during the COVID-19 pandemic. Hope has emerged with the development of new vaccines against the disease. However, many factors hinder vaccine uptake and lead to vaccine hesitancy. Understanding the factors affecting vaccine hesitancy and how to assess its prevalence have become imperative amid the COVID-19 pandemic. The vaccine hesitancy scale (VHS), developed by the World Health Organization (WHO) Strategic Advisory Group of Experts on Immunization, has been modified to the adult VHS (aVHS) and validated in English and Chinese. To our knowledge, no available aVHS has been designed or validated in Arabic or French. Objective: The aim of this research is to translate the aVHS from its original English language to Arabic and French and validate the translations in the WHO Eastern Mediterranean region. Methods: The study will follow a cross-sectional design divided into 5 phases. In phase 1, the original aVHS will be forward-translated to Arabic and French, followed by backward translation to English. An expert committee will review and rate all versions of the translations. Expert agreement will then be measured using the Cohen kappa coefficient (k). In phase 2, the translated aVHS will be pilot-tested with 2 samples of participants (n=100): a group that speaks both Arabic and English and another that speaks French and English. Participants? responses to the English version will also be collected. In phase 3, responses will then be compared. Descriptive statistics and paired t tests or one-way analyses of variance (ANOVA) and Pearson correlation coefficient will be used in the preliminary validation. In phase 4, prefinal versions (Arabic and French) will be tested with larger sample sizes of Arabic speakers (n=1000) and French speakers (n=1000). Sociodemographic information and vaccination status will be collected and used for further analysis. In phase 5, the scale's statistical reliability and internal consistency will be measured using Cronbach alpha. An exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) will be used to examine the model fit resulting from the EFA. ANOVA and regression models will be constructed to control for confounders. All data will be electronically collected. Results: As of January 2022, the scale had been translated to Arabic and French and was undergoing the process of back translation. All data collection tools have been prepared (ie, sociodemographics, vaccination status, and open-ended questions) and are ready to go into their electronic formats. We expect to reach the desired sample size in this phase by June 2022. Conclusions: This study will provide researchers with a validated tool to assess adult vaccine hesitancy within populations that speak Arabic and/or French and provide a road map to scale translation and ensure cross-cultural adaptation. International Registered Report Identifier (IRRID): PRR1-10.2196/36928 UR - https://www.researchprotocols.org/2022/4/e36928 UR - http://dx.doi.org/10.2196/36928 UR - http://www.ncbi.nlm.nih.gov/pubmed/35247043 ID - info:doi/10.2196/36928 ER - TY - JOUR AU - Graziani, Grant AU - Kunkle, Sarah AU - Shih, Emily PY - 2022/3/31 TI - Resilience in 2021?Descriptive Analysis of Individuals Accessing Virtual Mental Health Services: Retrospective Observational Study JO - JMIR Form Res SP - e34283 VL - 6 IS - 3 KW - mental health KW - resilience KW - adaptability KW - measures KW - digital health KW - virtual health KW - psychiatry KW - demographic KW - depression KW - anxiety KW - symptom KW - support KW - treatment N2 - Background: Psychological resilience has been extensively studied by developmental researchers, and there is a growing body of literature regarding its role in psychiatry and psychopathology research and practice. This study contributes to this growing literature by providing real-world evidence on the relationship between resilience and clinical symptoms among a large sample of employed Americans. Objective: This study aimed to describe resilience levels in individuals accessing Ginger, a virtual mental health system, in addition to the association of resilience with demographic characteristics, baseline depression, and anxiety symptoms. Methods: We conducted a retrospective observational study of 9165 members who signed up for Ginger and completed a baseline survey between January 1 and August 5, 2021. We used multivariate regression models to test for associations between baseline resilience and other member characteristics. Results: Baseline resilience scores centered on a mean of 23.84 (SD 6.56) and median of 24 (IQR 8) out of 40, with 81.0% (7424/9165) of the sample having low resilience at baseline. Despite having relatively higher resilience scores, members with no or mild depression or anxiety still had low resilience scores on average. Self-reported suicidal ideation was associated with lower resilience. Conclusions: Overall, members had low baseline resilience, similar to resilience levels observed in trauma survivors in prior studies. Younger members and those with higher levels of depression and anxiety at intake reported lower levels of resilience at baseline. Notably, members with no or mild depression or anxiety still had low resilience scores on average, suggesting a need for mental health support among individuals who might not typically be recommended for treatment based on traditional clinical assessments, such as the 9-item Patient Health Questionnaire (PHQ-9) and the 7-item Generalized Anxiety Disorder scale (GAD-7). Two suggestions for topics of future research are to develop treatment recommendations based on the Connor-Davidson Resilience Scale and to understand the interaction between resilience levels and symptom-based outcome measures, such as the PHQ-9 and the GAD-7. UR - https://formative.jmir.org/2022/3/e34283 UR - http://dx.doi.org/10.2196/34283 UR - http://www.ncbi.nlm.nih.gov/pubmed/35357309 ID - info:doi/10.2196/34283 ER - TY - JOUR AU - Thorpe, Dan AU - Fouyaxis, John AU - Lipschitz, M. Jessica AU - Nielson, Amy AU - Li, Wenhao AU - Murphy, A. Susan AU - Bidargaddi, Niranjan PY - 2022/3/31 TI - Cost and Effort Considerations for the Development of Intervention Studies Using Mobile Health Platforms: Pragmatic Case Study JO - JMIR Form Res SP - e29988 VL - 6 IS - 3 KW - health informatics KW - human computer interaction KW - digital health KW - mobile health KW - ecological momentary assessment KW - ecological momentary intervention KW - behavioral activation KW - interventional research KW - mobile health costs N2 - Background: The research marketplace has seen a flood of open-source or commercial mobile health (mHealth) platforms that can collect and use user data in real time. However, there is a lack of practical literature on how these platforms are developed, integrated into study designs, and adopted, including important information around cost and effort considerations. Objective: We intend to build critical literacy in the clinician-researcher readership into the cost, effort, and processes involved in developing and operationalizing an mHealth platform, focusing on Intui, an mHealth platform that we developed. Methods: We describe the development of the Intui mHealth platform and general principles of its operationalization across sites. Results: We provide a worked example in the form of a case study. Intui was operationalized in the design of a behavioral activation intervention in collaboration with a mental health service provider. We describe the design specifications of the study site, the developed software, and the cost and effort required to build the final product. Conclusions: Study designs, researcher needs, and technical considerations can impact effort and costs associated with the use of mHealth platforms. Greater transparency from platform developers about the impact of these factors on practical considerations relevant to end users such as clinician-researchers is crucial to increasing critical literacy around mHealth, thereby aiding in the widespread use of these potentially beneficial technologies and building clinician confidence in these tools. UR - https://formative.jmir.org/2022/3/e29988 UR - http://dx.doi.org/10.2196/29988 UR - http://www.ncbi.nlm.nih.gov/pubmed/35357313 ID - info:doi/10.2196/29988 ER - TY - JOUR AU - Vuokko, Riikka AU - Vakkuri, Anne AU - Palojoki, Sari PY - 2022/3/29 TI - Preliminary Exploration of Main Elements for Systematic Classification Development: Case Study of Patient Safety Incidents JO - JMIR Form Res SP - e35474 VL - 6 IS - 3 KW - classification KW - qualitative research KW - methodology KW - patient safety KW - validation N2 - Background: Currently, there is no holistic theoretical approach available for guiding classification development. On the basis of our recent classification development research in the area of patient safety in health information technology, this focus area would benefit from a more systematic approach. Although some valuable theoretical and methodological approaches have been presented, classification development literature typically is limited to methodological development in a specific domain or is practically oriented. Objective: The main purposes of this study are to fill the methodological gap in classification development research by exploring possible elements of systematic development based on previous literature and to promote sustainable and well-grounded classification outcomes by identifying a set of recommended elements. Specifically, the aim is to answer the following question: what are the main elements for systematic classification development based on research evidence and our use case? Methods: This study applied a qualitative research approach. On the basis of previous literature, preliminary elements for classification development were specified, as follows: defining a concept model, documenting the development process, incorporating multidisciplinary expertise, validating results, and maintaining the classification. The elements were compiled as guiding principles for the research process and tested in the case of patient safety incidents (n=501). Results: The results illustrate classification development based on the chosen elements, with 4 examples of technology-induced errors. Examples from the use case regard usability, system downtime, clinical workflow, and medication section problems. The study results confirm and thus suggest that a more comprehensive and theory-based systematic approach promotes well-grounded classification work by enhancing transparency and possibilities for assessing the development process. Conclusions: We recommend further testing the preliminary main elements presented in this study. The research presented herein could serve as a basis for future work. Our recently developed classification and the use case presented here serve as examples. Data retrieved from, for example, other type of electronic health records and use contexts could refine and validate the suggested methodological approach. UR - https://formative.jmir.org/2022/3/e35474 UR - http://dx.doi.org/10.2196/35474 UR - http://www.ncbi.nlm.nih.gov/pubmed/35348463 ID - info:doi/10.2196/35474 ER - TY - JOUR AU - Merschel, Steve AU - Reinhardt, Lars PY - 2022/3/28 TI - Analyzability of Photoplethysmographic Smartwatch Data by the Preventicus Heartbeats Algorithm During Everyday Life: Feasibility Study JO - JMIR Form Res SP - e29479 VL - 6 IS - 3 KW - photoplethysmography KW - wearable KW - smartwatch KW - heart rate monitoring KW - cardiac arrhythmia screening KW - atrial fibrillation KW - signal quality KW - activity profile N2 - Background: Continuous heart rate monitoring via mobile health technologies based on photoplethysmography (PPG) has great potential for the early detection of sustained cardiac arrhythmias such as atrial fibrillation. However, PPG measurements are impaired by motion artifacts. Objective: The aim of this investigation was to evaluate the analyzability of smartwatch-derived PPG data during everyday life and to determine the relationship between the analyzability of the data and the activity level of the participant. Methods: A total of 41 (19 female and 22 male) adults in good cardiovascular health (aged 19-79 years) continuously wore a smartwatch equipped with a PPG sensor and a 3D accelerometer (Cardio Watch 287, Corsano Health BV) for a period of 24 hours that represented their individual daily routine. For each participant, smartwatch data were analyzed on a 1-minute basis by an algorithm designed for heart rhythm analysis (Preventicus Heartbeats, Preventicus GmbH). As outcomes, the percentage of analyzable data (PAD) and the mean acceleration (ACC) were calculated. To map changes of the ACC and PAD over the course of one day, the 24-hour period was divided into 8 subintervals comprising 3 hours each. Results: Univariate analysis of variance showed a large effect (?p2> 0.6; P<.001) of time interval (phase) on the ACC and PAD. The PAD ranged between 34% and 100%, with an average of 71.5% for the whole day, which is equivalent to a period of 17.2 hours. Between midnight and 6 AM, the mean values were the highest for the PAD (>94%) and the lowest for the ACC (<6×10-3 m/s2). Regardless of the time of the day, the correlation between the PAD and ACC was strong (r=?0.64). A linear regression analysis for the averaged data resulted in an almost perfect coefficient of determination (r2=0.99). Conclusions: This study showed a large relationship between the activity level and the analyzability of smartwatch-derived PPG data. Given the high yield of analyzable data during the nighttime, continuous arrhythmia screening seems particularly effective during sleep phases. UR - https://formative.jmir.org/2022/3/e29479 UR - http://dx.doi.org/10.2196/29479 UR - http://www.ncbi.nlm.nih.gov/pubmed/35343902 ID - info:doi/10.2196/29479 ER - TY - JOUR AU - Douze, Laura AU - Pelayo, Sylvia AU - Messaadi, Nassir AU - Grosjean, Julien AU - Kerdelhué, Gaétan AU - Marcilly, Romaric PY - 2022/3/25 TI - Designing Formulae for Ranking Search Results: Mixed Methods Evaluation Study JO - JMIR Hum Factors SP - e30258 VL - 9 IS - 1 KW - information retrieval KW - search engine KW - topical relevance KW - search result ranking KW - user testing KW - human factors N2 - Background: A major factor in the success of any search engine is the relevance of the search results; a tool should sort the search results to present the most relevant documents first. Assessing the performance of the ranking formula is an important part of search engine evaluation. However, the methods currently used to evaluate ranking formulae mainly collect quantitative data and do not gather qualitative data, which help to understand what needs to be improved to tailor the formulae to their end users. Objective: This study aims to evaluate 2 different parameter settings of the ranking formula of LiSSa (the French acronym for scientific literature in health care; Department of Medical Informatics and Information), a tool that provides access to health scientific literature in French, to adapt the formula to the needs of the end users. Methods: To collect quantitative and qualitative data, user tests were carried out with representative end users of LiSSa: 10 general practitioners and 10 registrars. Participants first assessed the relevance of the search results and then rated the ranking criteria used in the 2 formulae. Verbalizations were analyzed to characterize each criterion. Results: A formula that prioritized articles representing a consensus in the field was preferred. When users assess an article?s relevance, they judge its topic, methods, and value in clinical practice. Conclusions: Following the evaluation, several improvements were implemented to give more weight to articles that match the search topic and to downgrade articles that have less informative or scientific value for the reader. Applying a qualitative methodology generates valuable user inputs to improve the ranking formula and move toward a highly usable search engine. UR - https://humanfactors.jmir.org/2022/1/e30258 UR - http://dx.doi.org/10.2196/30258 UR - http://www.ncbi.nlm.nih.gov/pubmed/35333180 ID - info:doi/10.2196/30258 ER - TY - JOUR AU - Semborski, Sara AU - Henwood, Benjamin AU - Redline, Brian AU - Dzubur, Eldin AU - Mason, Tyler AU - Intille, Stephen PY - 2022/3/25 TI - Feasibility and Acceptability of Ecological Momentary Assessment With Young Adults Who Are Currently or Were Formerly Homeless: Mixed Methods Study JO - JMIR Form Res SP - e33387 VL - 6 IS - 3 KW - ecological momentary assessment KW - homelessness KW - young adults KW - reactivity KW - compliance KW - mobile phone N2 - Background: Ecological momentary assessment (EMA) has been used with young people experiencing homelessness to gather information on contexts associated with homelessness and risk behavior in real time and has proven feasible in this population. However, the extent to which EMA may affect the attitudes or behaviors of young adults who are currently or were formerly homeless and are residing in supportive housing has not been well investigated. Objective: This study aims to describe the feedback regarding EMA study participation from young adults who are currently or were formerly homeless and examine the reactivity to EMA participation and compliance. Methods: This mixed methods study used cross-sectional data collected before and after EMA, intensive longitudinal data from a 7-day EMA prompting period, and focus groups of young adults who are currently or were formerly homeless in Los Angeles, California, between 2017 and 2019. Results: Qualitative data confirmed the quantitative findings. Differences in the experience of EMA between young adults who are currently or were formerly homeless were found to be related to stress or anxiety, interference with daily life, difficulty charging, behavior change, and honesty in responses. Anxiety and depression symptomatology decreased from before to after EMA; however, compliance was not significantly associated with this decrease. Conclusions: The results point to special considerations when administering EMA to young adults who are currently or were formerly homeless. EMA appears to be slightly more burdensome for young adults who are currently homeless than for those residing in supportive housing, which are nuances to consider in the study design. The lack of a relationship between study compliance and symptomatology suggests low levels of reactivity. UR - https://formative.jmir.org/2022/3/e33387 UR - http://dx.doi.org/10.2196/33387 UR - http://www.ncbi.nlm.nih.gov/pubmed/35333187 ID - info:doi/10.2196/33387 ER - TY - JOUR AU - Timmermans, Lotte AU - Huybrechts, Ine AU - Decat, Peter AU - Foulon, Veerle AU - Van Hecke, Ann AU - Vermandere, Mieke AU - Schoenmakers, Birgitte PY - 2022/3/25 TI - Recommendations for Researchers on Synchronous, Online, Nominal Group Sessions in Times of COVID-19: Fishbone Analysis JO - JMIR Form Res SP - e34539 VL - 6 IS - 3 KW - COVID-19 KW - fishbone diagram KW - nominal group technique KW - video conferencing KW - primary health care KW - qualitative research N2 - Background: In times of COVID-19, we are challenged to experiment with alternative platforms or software to connect people. In particular, the struggle that arose in health research was how to interact with patients and care professionals. The latter is additionally faced with an extreme workload to fight the pandemic crisis. Creative strategies have been developed to continue research among patients and care professionals to improve quality of care. This paper addresses the issue of synchronous, online, nominal group sessions, a common consensus method used for group brainstorming. Objective: The purpose of this study was to share our experiences with performing online, nominal group sessions using the video conference software Microsoft Teams. In addition, we aimed to create a practical guide with recommendations for researchers. Methods: We critically analyzed the procedures for the online nominal group technique, according to the Fishbone methodology. Results: Performing synchronous, online, nominal group sessions is challenging but offers opportunities. Although interaction with and among the attendees complicates the process, the major advantage of online sessions is their accessibility and comfort because of reduced barriers to participation (eg, lower time investment). The role of the moderators is of major importance, and good preparation beforehand is required. Recommendations for future online, nominal research were formulated. Conclusions: Online, nominal group sessions seem to be a promising alternative for the real-life commonly used technique. Especially during the COVID-19 pandemic, the benefits must be highlighted. More expertise is needed to further refine the practical guide for using digital software in research and to achieve optimal performance. UR - https://formative.jmir.org/2022/3/e34539 UR - http://dx.doi.org/10.2196/34539 UR - http://www.ncbi.nlm.nih.gov/pubmed/35225814 ID - info:doi/10.2196/34539 ER - TY - JOUR AU - Kuleindiren, Narayan AU - Rifkin-Zybutz, Paul Raphael AU - Johal, Monika AU - Selim, Hamzah AU - Palmon, Itai AU - Lin, Aaron AU - Yu, Yizhou AU - Alim-Marvasti, Ali AU - Mahmud, Mohammad PY - 2022/3/22 TI - Optimizing Existing Mental Health Screening Methods in a Dementia Screening and Risk Factor App: Observational Machine Learning Study JO - JMIR Form Res SP - e31209 VL - 6 IS - 3 KW - depression KW - anxiety KW - screening KW - research method KW - questionnaire KW - precision KW - dementia KW - cognition KW - risk factors KW - machine learning KW - prediction N2 - Background: Mindstep is an app that aims to improve dementia screening by assessing cognition and risk factors. It considers important clinical risk factors, including prodromal symptoms, mental health disorders, and differential diagnoses of dementia. The 9-item Patient Health Questionnaire for depression (PHQ-9) and the 7-item Generalized Anxiety Disorder Scale (GAD-7) are widely validated and commonly used scales used in screening for depression and anxiety disorders, respectively. Shortened versions of both (PHQ-2/GAD-2) have been produced. Objective: We sought to develop a method that maintained the brevity of these shorter questionnaires while maintaining the better precision of the original questionnaires. Methods: Single questions were designed to encompass symptoms covered in the original questionnaires. Answers to these questions were combined with PHQ-2/GAD-2, and anonymized risk factors were collected by Mindset4Dementia from 2235 users. Machine learning models were trained to use these single questions in combination with data already collected by the app: age, response to a joke, and reporting of functional impairment to predict binary and continuous outcomes as measured using PHQ-9/GAD-7. Our model was developed with a training data set by using 10-fold cross-validation and a holdout testing data set and compared to results from using the shorter questionnaires (PHQ-2/GAD-2) alone to benchmark performance. Results: We were able to achieve superior performance in predicting PHQ-9/GAD-7 screening cutoffs compared to PHQ-2 (difference in area under the curve 0.04, 95% CI 0.00-0.08, P=.02) but not GAD-2 (difference in area under the curve 0.00, 95% CI ?0.02 to 0.03, P=.42). Regression models were able to accurately predict total questionnaire scores in PHQ-9 (R2=0.655, mean absolute error=2.267) and GAD-7 (R2=0.837, mean absolute error=1.780). Conclusions: We app-adapted PHQ-4 by adding brief summary questions about factors normally covered in the longer questionnaires. We additionally trained machine learning models that used the wide range of additional information already collected in Mindstep to make a short app-based screening tool for affective disorders, which appears to have superior or equivalent performance to well-established methods. UR - https://formative.jmir.org/2022/3/e31209 UR - http://dx.doi.org/10.2196/31209 UR - http://www.ncbi.nlm.nih.gov/pubmed/35315786 ID - info:doi/10.2196/31209 ER - TY - JOUR AU - Staffini, Alessio AU - Fujita, Kento AU - Svensson, Kishi Akiko AU - Chung, Ung-Il AU - Svensson, Thomas PY - 2022/3/18 TI - Statistical Methods for Item Reduction in a Representative Lifestyle Questionnaire: Pilot Questionnaire Study JO - Interact J Med Res SP - e28692 VL - 11 IS - 1 KW - item reduction KW - surveys and lifestyle questionnaires KW - feedback measures KW - questionnaire design KW - variance inflation factor KW - factor analysis KW - mobile phone N2 - Background: Reducing the number of items in a questionnaire while maintaining relevant information is important as it is associated with advantages such as higher respondent engagement and reduced response error. However, in health care, after the original design, an a posteriori check of the included items in a questionnaire is often overlooked or considered to be of minor importance. When conducted, this is often based on a single selected method. We argue that before finalizing any lifestyle questionnaire, a posteriori validation should always be conducted using multiple approaches to ensure the robustness of the results. Objective: The objectives of this study are to compare the results of two statistical methods for item reduction (variance inflation factor [VIF] and factor analysis [FA]) in a lifestyle questionnaire constructed by combining items from different sources and analyze the different results obtained from the 2 methods and the conclusions that can be made about the original items. Methods: Data were collected from 79 participants (heterogeneous in age and sex) with a high risk of metabolic syndrome working in a financial company based in Tokyo. The lifestyle questionnaire was constructed by combining items (asked with daily, weekly, and monthly frequency) from multiple validated questionnaires and other selected questions. Item reduction was conducted using VIF and exploratory FA. Adequacy tests were used to check the data distribution and sampling adequacy. Results: Among the daily and weekly questions, both VIF and FA identified redundancies in sleep-related items. Among the monthly questions, both approaches identified redundancies in stress-related items. However, the number of items suggested for reduction often differed: VIF suggested larger reductions than FA for daily questions but fewer reductions for weekly questions. Adequacy tests always confirmed that the structural detection was adequate for the considered items. Conclusions: As expected, our analyses showed that VIF and FA produced both similar and different findings, suggesting that questionnaire designers should consider using multiple methods for item reduction. Our findings using both methods indicate that many questions, especially those related to sleep, are redundant, indicating that the considered lifestyle questionnaire can be shortened. UR - https://www.i-jmr.org/2022/1/e28692 UR - http://dx.doi.org/10.2196/28692 UR - http://www.ncbi.nlm.nih.gov/pubmed/35302507 ID - info:doi/10.2196/28692 ER - TY - JOUR AU - Hopstock, Arnesdatter Laila AU - Medin, Christine Anine AU - Skeie, Guri AU - Henriksen, André AU - Lundblad, Wasmuth Marie PY - 2022/3/11 TI - Evaluation of a Web-Based Dietary Assessment Tool (myfood24) in Norwegian Women and Men Aged 60-74 Years: Usability Study JO - JMIR Form Res SP - e35092 VL - 6 IS - 3 KW - system usability score KW - older adults KW - measurements KW - nutrition KW - dietary intake KW - digital health KW - web tool N2 - Background: A healthy diet throughout the life course improves health and reduces the risk of disease. There is a need for new knowledge of the relation between diet and health, but existing methods to collect information on food and nutrient intake have their limitations. Evaluations of new tools to assess dietary intake are needed, especially in old people, where the introduction of new technology might impose challenges. Objective: We aimed to examine the usability of a new web-based dietary assessment tool in older adult women and men. Methods: A total of 60 women and men (participation 83%, 57% women) aged 60-74 years recruited by convenience and snowball sampling completed a 24-hour web-based dietary recall using the newly developed Norwegian version of Measure Your Food On One Day (myfood24). Total energy and nutrient intakes were calculated in myfood24, primarily on the basis of the Norwegian Food Composition Table. No guidance or support was provided to complete the recall. Usability was assessed using the system usability scale (SUS), where an SUS score of ?68 was considered satisfactory. We examined the responses to single SUS items and the mean (SD) SUS score in groups stratified by sex, age, educational level, and device used to complete the recall (smartphone, tablet device, or computer). Results: The mean total energy intake was 5815 (SD 3093) kJ. A total of 14% of participants had an energy intake of <2100 kJ (ie, 500 kilocalories) and none had an intake of >16,800 kJ (ie, 4000 kilocalories). Mean energy proportions from carbohydrates, fat, protein, alcohol, and fiber was within the national recommendations. The mean SUS score was 55.5 (SD 18.6), and 27% of participants had SUS scores above the satisfactory product cut-off. Higher SUS scores were associated with younger age and lower education, but not with the type of device used. Conclusions: We found the overall usability of a new web-based dietary assessment tool to be less than satisfactory in accordance with standard usability criteria in a sample of 60-74?year-old Norwegians. The observed total energy intakes suggest that several of the participants underreported their intake during the completion of the dietary recall. Implementing web-based dietary assessment tools in older adults is feasible, but guidance and support might be needed to ensure valid completion. UR - https://formative.jmir.org/2022/3/e35092 UR - http://dx.doi.org/10.2196/35092 UR - http://www.ncbi.nlm.nih.gov/pubmed/35275079 ID - info:doi/10.2196/35092 ER - TY - JOUR AU - Evans, Richard AU - Burns, Jennifer AU - Damschroder, Laura AU - Annis, Ann AU - Freitag, B. Michelle AU - Raffa, Susan AU - Wiitala, Wyndy PY - 2022/3/9 TI - Deriving Weight From Big Data: Comparison of Body Weight Measurement?Cleaning Algorithms JO - JMIR Med Inform SP - e30328 VL - 10 IS - 3 KW - veterans KW - weight KW - algorithms KW - obesity KW - measurement KW - electronic health record N2 - Background: Patient body weight is a frequently used measure in biomedical studies, yet there are no standard methods for processing and cleaning weight data. Conflicting documentation on constructing body weight measurements presents challenges for research and program evaluation. Objective: In this study, we aim to describe and compare methods for extracting and cleaning weight data from electronic health record databases to develop guidelines for standardized approaches that promote reproducibility. Methods: We conducted a systematic review of studies published from 2008 to 2018 that used Veterans Health Administration electronic health record weight data and documented the algorithms for constructing patient weight. We applied these algorithms to a cohort of veterans with at least one primary care visit in 2016. The resulting weight measures were compared at the patient and site levels. Results: We identified 496 studies and included 62 (12.5%) that used weight as an outcome. Approximately 48% (27/62) included a replicable algorithm. Algorithms varied from cutoffs of implausible weights to complex models using measures within patients over time. We found differences in the number of weight values after applying the algorithms (71,961/1,175,995, 6.12% to 1,175,177/1,175,995, 99.93% of raw data) but little difference in average weights across methods (93.3, SD 21.0 kg to 94.8, SD 21.8 kg). The percentage of patients with at least 5% weight loss over 1 year ranged from 9.37% (4933/52,642) to 13.99% (3355/23,987). Conclusions: Contrasting algorithms provide similar results and, in some cases, the results are not different from using raw, unprocessed data despite algorithm complexity. Studies using point estimates of weight may benefit from a simple cleaning rule based on cutoffs of implausible values; however, research questions involving weight trajectories and other, more complex scenarios may benefit from a more nuanced algorithm that considers all available weight data. UR - https://medinform.jmir.org/2022/3/e30328 UR - http://dx.doi.org/10.2196/30328 UR - http://www.ncbi.nlm.nih.gov/pubmed/35262492 ID - info:doi/10.2196/30328 ER - TY - JOUR AU - Zhao, Junqiang AU - Harvey, Gillian AU - Vandyk, Amanda AU - Gifford, Wendy PY - 2022/3/9 TI - Social Media for ImpLementing Evidence (SMILE): Conceptual Framework JO - JMIR Form Res SP - e29891 VL - 6 IS - 3 KW - social media KW - research use KW - knowledge translation KW - implementation science KW - conceptual framework N2 - Background: Social media has become widely used by individual researchers and professional organizations to translate research evidence into health care practice. Despite its increasing popularity, few social media initiatives consider the theoretical perspectives of how social media works as a knowledge translation strategy to affect research use. Objective: The purpose of this paper is to propose a conceptual framework to understand how social media works as a knowledge translation strategy for health care providers, policy makers, and patients to inform their health care decision-making. Methods: We developed this framework using an integrative approach that first involved reviewing 5 long-standing social media initiatives. We then drafted the initial framework using a deductive approach by referring to 5 theories on social media studies and knowledge translation. A total of 58 empirical studies on factors that influenced the use of social media and its messages and strategies for promoting the use of research evidence via social media were further integrated to substantiate and fine-tune our initial framework. Through an iterative process, we developed the Social Media for ImpLementing Evidence (SMILE) framework. Results: The SMILE framework has six key constructs: developers, messages and delivery strategies, recipients, context, triggers, and outcomes. For social media to effectively enable recipients to use research evidence in their decision-making, the framework proposes that social media content developers respond to target recipients? needs and context and develop relevant messages and appropriate delivery strategies. The recipients? use of social media messages is influenced by the virtual?technical, individual, organizational, and system contexts and can be activated by three types of triggers: sparks, facilitators, and signals. Conclusions: The SMILE framework maps the factors that are hypothesized to influence the use of social media messages by recipients and offers a heuristic device for social media content developers to create interventions for promoting the use of evidence in health care decision-making. Empirical studies are now needed to test the propositions of this framework. UR - https://formative.jmir.org/2022/3/e29891 UR - http://dx.doi.org/10.2196/29891 UR - http://www.ncbi.nlm.nih.gov/pubmed/35262488 ID - info:doi/10.2196/29891 ER - TY - JOUR AU - Jackson, N. Devlon AU - Sehgal, Neil AU - Baur, Cynthia PY - 2022/3/9 TI - Benefits of mHealth Co-design for African American and Hispanic Adults: Multi-Method Participatory Research for a Health Information App JO - JMIR Form Res SP - e26764 VL - 6 IS - 3 KW - mHealth app design KW - health literacy KW - health disparities KW - health equity KW - African Americans KW - Hispanics KW - mobile phone N2 - Background: Participatory research methodologies can provide insight into the use of mobile health (mHealth) apps, cultural preferences and needs, and health literacy issues for racial and ethnic groups, such as African Americans and Hispanics who experience health disparities. Objective: This methodological paper aims to describe a 1-year multi-method participatory research process that directly engaged English-speaking African American and bilingual or Spanish-speaking Hispanic adults in designing a prevention-focused, personalized mHealth, information-seeking smartphone app. We report design team participants? experiences with the methods to show why our approach is valuable in producing apps that are more aligned with their needs. Methods: Three design sessions were conducted to inform the iteration of a prevention-focused, personalized mHealth, information-seeking app. The research team led sessions with 2 community member design teams. Design team participants described their goals, motives, and interests regarding prevention information using different approaches, such as collage and card sorting (design session 1), interaction with the app prototype (design session 2), and rating of cultural appropriateness strategies (design session 3). Results: Each design team had 5 to 6 participants: 2 to 3 male participants and 3 female participants aged between 30 and 76 years. Design team participants shared their likes and dislikes about the sessions and the overall experience of the design sessions. Both African American and Hispanic teams reported positive participation experience. The primary reasons included the opportunity for their views to be heard, collectively working together in the design process, having their apprehension about mHealth reduced, and an opportunity to increase their knowledge of how they could manage their health through mHealth. The feedback from each session informed the following design sessions and a community-engaged process. In addition, the specific findings for each design session informed the design of the app for both communities. Conclusions: This multi-method participatory research process revealed 4 key lessons learned and recommendations for future research in mHealth app design for African Americans and Hispanics. Lesson 1?community partnerships are key because they provide the chain of trust that helps African American and Hispanic participants feel comfortable participating in app research. Lesson 2?community-based participatory research principles continue to yield promising results to engage these populations in mHealth research. Lesson 3?interactive design sessions uncover participants? needs and development opportunities for mHealth tools. Lesson 4?multiple design sessions with different methods provide an in-depth understanding of participants? mHealth preferences and needs. Future developers should consider these methods and lessons to ensure health apps in the marketplace contribute to eliminating health disparities and achieving health equity. UR - https://formative.jmir.org/2022/3/e26764 UR - http://dx.doi.org/10.2196/26764 UR - http://www.ncbi.nlm.nih.gov/pubmed/35262496 ID - info:doi/10.2196/26764 ER - TY - JOUR AU - Choi, H. Edmond P. AU - Duan, Wenjie AU - Fong, T. Daniel Y. AU - Lok, W. Kris Y. AU - Ho, Mandy AU - Wong, H. Janet Y. AU - Lin, Chia-Chin PY - 2022/3/2 TI - Psychometric Evaluation of a Fear of COVID-19 Scale in China: Cross-sectional Study JO - JMIR Form Res SP - e31992 VL - 6 IS - 3 KW - Chinese KW - COVID-19 KW - fear KW - psychometric KW - validation KW - scale KW - mental health KW - information KW - cross-sectional KW - validity KW - reliability KW - support N2 - Background: At the very beginning of the COVID-19 pandemic, information about fear of COVID-19 was very limited in Chinese populations, and there was no standardized and validated scale to measure the fear associated with the pandemic. Objective: This cross-sectional study aimed to adapt and validate a fear scale to determine the levels of fear of COVID-19 among the general population in mainland China and Hong Kong. Methods: A web-based questionnaire platform was developed for data collection; the study instruments were an adapted version of the 8-item Breast Cancer Fear Scale (?Fear Scale?) and the 4-item Patient Health Questionnaire. The internal construct validity, convergent validity, known group validity, and reliability of the adapted Fear Scale were assessed, and descriptive statistics were used to summarize the participants? fear levels. Results: A total of 2822 study participants aged 18 years or older were included in the analysis. The reliability of the adapted scale was satisfactory, with a Cronbach ? coefficient of .93. The item-total correlations corrected for overlap were >0.4, confirming their internal construct validity. Regarding convergent validity, a small-to-moderate correlation between the Fear Scale and the 4-item Patient Health Questionnaire scores was found. Regarding known group validity, we found that the study participants who were recruited from Hong Kong had a higher level of fear than the study participants from mainland China. Older adults had a higher level of fear compared with younger adults. Furthermore, having hypertension, liver disease, heart disease, cancer, anxiety, and insomnia were associated with a higher fear level. The descriptive analysis found that more than 40% of the study participants reported that the thought of COVID-19 scared them. About one-third of the study participants reported that when they thought about COVID-19, they felt nervous, uneasy, and depressed. Conclusions: The psychometric properties of the adapted Fear Scale are acceptable to measure the fear of COVID-19 among Chinese people. Our study stresses the need for more psychosocial support and care to help this population cope with their fears during the pandemic. UR - https://formative.jmir.org/2022/3/e31992 UR - http://dx.doi.org/10.2196/31992 UR - http://www.ncbi.nlm.nih.gov/pubmed/35072632 ID - info:doi/10.2196/31992 ER - TY - JOUR AU - Nissen, Michael AU - Slim, Syrine AU - Jäger, Katharina AU - Flaucher, Madeleine AU - Huebner, Hanna AU - Danzberger, Nina AU - Fasching, A. Peter AU - Beckmann, W. Matthias AU - Gradl, Stefan AU - Eskofier, M. Bjoern PY - 2022/3/1 TI - Heart Rate Measurement Accuracy of Fitbit Charge 4 and Samsung Galaxy Watch Active2: Device Evaluation Study JO - JMIR Form Res SP - e33635 VL - 6 IS - 3 KW - wearable validation KW - heart rate validation KW - Fitbit Charge 4 KW - Samsung Galaxy Watch Active2 KW - heart rate accuracy KW - fitness tracker accuracy KW - wearable accuracy KW - wearable KW - Fitbit KW - heart rate KW - fitness tracker KW - fitness KW - cardiovascular N2 - Background: Fitness trackers and smart watches are frequently used to collect data in longitudinal medical studies. They allow continuous recording in real-life settings, potentially revealing previously uncaptured variabilities of biophysiological parameters and diseases. Adequate device accuracy is a prerequisite for meaningful research. Objective: This study aims to assess the heart rate recording accuracy in two previously unvalidated devices: Fitbit Charge 4 and Samsung Galaxy Watch Active2. Methods: Participants performed a study protocol comprising 5 resting and sedentary, 2 low-intensity, and 3 high-intensity exercise phases, lasting an average of 19 minutes 27 seconds. Participants wore two wearables simultaneously during all activities: Fitbit Charge 4 and Samsung Galaxy Watch Active2. Reference heart rate data were recorded using a medically certified Holter electrocardiogram. The data of the reference and evaluated devices were synchronized and compared at 1-second intervals. The mean, mean absolute error, mean absolute percentage error, Lin concordance correlation coefficient, Pearson correlation coefficient, and Bland-Altman plots were analyzed. Results: A total of 23 healthy adults (mean age 24.2, SD 4.6 years) participated in our study. Overall, and across all activities, the Fitbit Charge 4 slightly underestimated the heart rate, whereas the Samsung Galaxy Watch Active2 overestimated it (?1.66 beats per minute [bpm]/3.84 bpm). The Fitbit Charge 4 achieved a lower mean absolute error during resting and sedentary activities (seated rest: 7.8 vs 9.4; typing: 8.1 vs 11.6; laying down [left]: 7.2 vs 9.4; laying down [back]: 6.0 vs 8.6; and walking slowly: 6.8 vs 7.7 bpm), whereas the Samsung Galaxy Watch Active2 performed better during and after low- and high-intensity activities (standing up: 12.3 vs 9.0; walking fast: 6.1 vs 5.8; stairs: 8.8 vs 6.9; squats: 15.7 vs 6.1; resting: 9.6 vs 5.6 bpm). Conclusions: Device accuracy varied with activity. Overall, both devices achieved a mean absolute percentage error of just <10%. Thus, they were considered to produce valid results based on the limits established by previous work in the field. Neither device reached sufficient accuracy during seated rest or keyboard typing. Thus, both devices may be eligible for use in respective studies; however, researchers should consider their individual study requirements. UR - https://formative.jmir.org/2022/3/e33635 UR - http://dx.doi.org/10.2196/33635 UR - http://www.ncbi.nlm.nih.gov/pubmed/35230250 ID - info:doi/10.2196/33635 ER - TY - JOUR AU - Zhang, Jonathan Nasen AU - Rameau, Philippe AU - Julemis, Marsophia AU - Liu, Yan AU - Solomon, Jeffrey AU - Khan, Sundas AU - McGinn, Thomas AU - Richardson, Safiya PY - 2022/2/28 TI - Automated Pulmonary Embolism Risk Assessment Using the Wells Criteria: Validation Study JO - JMIR Form Res SP - e32230 VL - 6 IS - 2 KW - health informatics KW - pulmonary embolism KW - electronic health record KW - quality improvement KW - clinical decision support systems N2 - Background: Computed tomography pulmonary angiography (CTPA) is frequently used in the emergency department (ED) for the diagnosis of pulmonary embolism (PE), while posing risk for contrast-induced nephropathy and radiation-induced malignancy. Objective: We aimed to create an automated process to calculate the Wells score for pulmonary embolism for patients in the ED, which could potentially reduce unnecessary CTPA testing. Methods: We designed an automated process using electronic health records data elements, including using a combinatorial keyword search method to query free-text fields, and calculated automated Wells scores for a sample of all adult ED encounters that resulted in a CTPA study for PE at 2 tertiary care hospitals in New York, over a 2-month period. To validate the automated process, the scores were compared to those derived from a 2-clinician chart review. Results: A total of 202 ED encounters resulted in a completed CTPA to form the retrospective study cohort. Patients classified as ?PE likely? by the automated process (126/202, 62%) had a PE prevalence of 15.9%, whereas those classified as ?PE unlikely? (76/202, 38%; Wells score >4) had a PE prevalence of 7.9%. With respect to classification of the patient as ?PE likely,? the automated process achieved an accuracy of 92.1% when compared with the chart review, with sensitivity, specificity, positive predictive value, and negative predictive value of 93%, 90.5%, 94.4%, and 88.2%, respectively. Conclusions: This was a successful development and validation of an automated process using electronic health records data elements, including free-text fields, to classify risk for PE in ED visits. UR - https://formative.jmir.org/2022/2/e32230 UR - http://dx.doi.org/10.2196/32230 UR - http://www.ncbi.nlm.nih.gov/pubmed/35225812 ID - info:doi/10.2196/32230 ER - TY - JOUR AU - Ponnada, Aditya AU - Wang, Shirlene AU - Chu, Daniel AU - Do, Bridgette AU - Dunton, Genevieve AU - Intille, Stephen PY - 2022/2/9 TI - Intensive Longitudinal Data Collection Using Microinteraction Ecological Momentary Assessment: Pilot and Preliminary Results JO - JMIR Form Res SP - e32772 VL - 6 IS - 2 KW - intensive longitudinal data KW - ecological momentary assessment KW - experience sampling KW - microinteractions KW - smartwatch KW - health behavior research KW - mobile phone N2 - Background: Ecological momentary assessment (EMA) uses mobile technology to enable in situ self-report data collection on behaviors and states. In a typical EMA study, participants are prompted several times a day to answer sets of multiple-choice questions. Although the repeated nature of EMA reduces recall bias, it may induce participation burden. There is a need to explore complementary approaches to collecting in situ self-report data that are less burdensome yet provide comprehensive information on an individual?s behaviors and states. A new approach, microinteraction EMA (?EMA), restricts EMA items to single, cognitively simple questions answered on a smartwatch with single-tap assessments using a quick, glanceable microinteraction. However, the viability of using ?EMA to capture behaviors and states in a large-scale longitudinal study has not yet been demonstrated. Objective: This paper describes the ?EMA protocol currently used in the Temporal Influences on Movement & Exercise (TIME) Study conducted with young adults, the interface of the ?EMA app used to gather self-report responses on a smartwatch, qualitative feedback from participants after a pilot study of the ?EMA app, changes made to the main TIME Study ?EMA protocol and app based on the pilot feedback, and preliminary ?EMA results from a subset of active participants in the TIME Study. Methods: The TIME Study involves data collection on behaviors and states from 246 individuals; measurements include passive sensing from a smartwatch and smartphone and intensive smartphone-based hourly EMA, with 4-day EMA bursts every 2 weeks. Every day, participants also answer a nightly EMA survey. On non?EMA burst days, participants answer ?EMA questions on the smartwatch, assessing momentary states such as physical activity, sedentary behavior, and affect. At the end of the study, participants describe their experience with EMA and ?EMA in a semistructured interview. A pilot study was used to test and refine the ?EMA protocol before the main study. Results: Changes made to the ?EMA study protocol based on pilot feedback included adjusting the single-question selection method and smartwatch vibrotactile prompting. We also added sensor-triggered questions for physical activity and sedentary behavior. As of June 2021, a total of 81 participants had completed at least 6 months of data collection in the main study. For 662,397 ?EMA questions delivered, the compliance rate was 67.6% (SD 24.4%) and the completion rate was 79% (SD 22.2%). Conclusions: The TIME Study provides opportunities to explore a novel approach for collecting temporally dense intensive longitudinal self-report data in a sustainable manner. Data suggest that ?EMA may be valuable for understanding behaviors and states at the individual level, thus possibly supporting future longitudinal interventions that require within-day, temporally dense self-report data as people go about their lives. UR - https://formative.jmir.org/2022/2/e32772 UR - http://dx.doi.org/10.2196/32772 UR - http://www.ncbi.nlm.nih.gov/pubmed/35138253 ID - info:doi/10.2196/32772 ER - TY - JOUR AU - Scheder-Bieschin, Justus AU - Blümke, Bibiana AU - de Buijzer, Erwin AU - Cotte, Fabienne AU - Echterdiek, Fabian AU - Nacsa, Júlia AU - Ondresik, Marta AU - Ott, Matthias AU - Paul, Gregor AU - Schilling, Tobias AU - Schmitt, Anne AU - Wicks, Paul AU - Gilbert, Stephen PY - 2022/2/7 TI - Improving Emergency Department Patient-Physician Conversation Through an Artificial Intelligence Symptom-Taking Tool: Mixed Methods Pilot Observational Study JO - JMIR Form Res SP - e28199 VL - 6 IS - 2 KW - symptom assessment application KW - anamnesis KW - health care system KW - patient history taking KW - diagnosis KW - emergency department N2 - Background: Establishing rapport and empathy between patients and their health care provider is important but challenging in the context of a busy and crowded emergency department (ED). Objective: We explore the hypotheses that rapport building, documentation, and time efficiency might be improved in the ED by providing patients a digital tool that uses Bayesian reasoning?based techniques to gather relevant symptoms and history for handover to clinicians. Methods: A 2-phase pilot evaluation was carried out in the ED of a German tertiary referral and major trauma hospital that treats an average of 120 patients daily. Phase 1 observations guided iterative improvement of the digital tool, which was then further evaluated in phase 2. All patients who were willing and able to provide consent were invited to participate, excluding those with severe injury or illness requiring immediate treatment, with traumatic injury, incapable of completing a health assessment, and aged <18 years. Over an 18-day period with 1699 patients presenting to the ED, 815 (47.96%) were eligible based on triage level. With available recruitment staff, 135 were approached, of whom 81 (60%) were included in the study. In a mixed methods evaluation, patients entered information into the tool, accessed by clinicians through a dashboard. All users completed evaluation Likert-scale questionnaires rating the tool?s performance. The feasibility of a larger trial was evaluated through rates of recruitment and questionnaire completion. Results: Respondents strongly endorsed the tool for facilitating conversation (61/81, 75% of patients, 57/78, 73% of physician ratings, and 10/10, 100% of nurse ratings). Most nurses judged the tool as potentially time saving, whereas most physicians only agreed for a subset of medical specialties (eg, surgery). Patients reported high usability and understood the tool?s questions. The tool was recommended by most patients (63/81, 78%), in 53% (41/77) of physician ratings, and in 76% (61/80) of nurse ratings. Questionnaire completion rates were 100% (81/81) by patients and 96% (78/81 enrolled patients) by physicians. Conclusions: This pilot confirmed that a larger study in the setting would be feasible. The tool has clear potential to improve patient?health care provider interaction and could also contribute to ED efficiency savings. Future research and development will extend the range of patients for whom the history-taking tool has clinical utility. Trial Registration: German Clinical Trials Register DRKS00024115; https://drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00024115 UR - https://formative.jmir.org/2022/2/e28199 UR - http://dx.doi.org/10.2196/28199 UR - http://www.ncbi.nlm.nih.gov/pubmed/35129452 ID - info:doi/10.2196/28199 ER - TY - JOUR AU - Hadian, Kimia AU - Fernie, Geoff AU - Roshan Fekr, Atena PY - 2022/2/2 TI - A New Performance Metric to Estimate the Risk of Exposure to Infection in a Health Care Setting: Descriptive Study JO - JMIR Form Res SP - e32384 VL - 6 IS - 2 KW - hand hygiene KW - health care-acquired KW - infection control KW - compliance KW - electronic monitoring KW - exposure KW - risk KW - hygiene KW - monitoring KW - surveillance KW - performance KW - metric KW - method KW - estimate KW - predict KW - development N2 - Background: Despite several measures to monitor and improve hand hygiene (HH) in health care settings, health care-acquired infections (HAIs) remain prevalent. The measures used to calculate HH performance are not able to fully benefit from the high-resolution data collected using electronic monitoring systems. Objective: This study proposes a novel parameter for quantifying the HAI exposure risk of individual patients by considering temporal and spatial features of health care workers? HH adherence. Methods: Patient exposure risk is calculated as a function of the number of consecutive missed HH opportunities, the number of unique rooms visited by the health care professional, and the time duration that the health care professional spends inside and outside the patient?s room without performing HH. The patient exposure risk is compared to the entrance compliance rate (ECR) defined as the ratio of the number of HH actions performed at a room entrance to the total number of entrances into the room. The compliance rate is conventionally used to measure HH performance. The ECR and the patient exposure risk are analyzed using the data collected from an inpatient nursing unit for 12 weeks. Results: The analysis of data collected from 59 nurses and more than 25,600 records at a musculoskeletal rehabilitation unit at the Toronto Rehabilitation Institute, KITE, showed that there is no strong linear relation between the ECR and patient exposure risk (r=0.7, P<.001). Since the ECR is calculated based on the number of missed HH actions upon room entrance, this parameter is already included in the patient exposure risk. Therefore, there might be scenarios that these 2 parameters are correlated; however, in several cases, the ECR contrasted with the reported patient exposure risk. Generally, the patients in rooms with a significantly high ECR can be potentially exposed to a considerable risk of infection. By contrast, small ECRs do not necessarily result in a high patient exposure risk. The results clearly explained the important role of the factors incorporated in patient exposure risk for quantifying the risk of infection for the patients. Conclusions: Patient exposure risk might provide a more reliable estimation of the risk of developing HAIs compared to ECR by considering both the temporal and spatial aspects of HH records. UR - https://formative.jmir.org/2022/2/e32384 UR - http://dx.doi.org/10.2196/32384 UR - http://www.ncbi.nlm.nih.gov/pubmed/35107424 ID - info:doi/10.2196/32384 ER - TY - JOUR AU - Fortuna, L. Karen AU - Marceau, R. Skyla AU - Kadakia, Arya AU - Pratt, I. Sarah AU - Varney, Joy AU - Walker, Robert AU - Myers, L. Amanda AU - Thompson, Shavon AU - Carter, Katina AU - Greene, Kaycie AU - Pringle, Willie PY - 2022/2/1 TI - Peer Support Specialists? Perspectives of a Standard Online Research Ethics Training: Qualitative Study JO - JMIR Form Res SP - e29073 VL - 6 IS - 2 KW - peer support specialists KW - community engagement KW - research ethics KW - mental health KW - peer support KW - codebook KW - online health KW - online training KW - education KW - ethics N2 - Background: Certified peer support specialists (CPS) have a mental health condition and are trained and certified by their respective state to offer Medicaid reimbursable peer support services. CPS are increasingly involved as partners in research studies. However, most research ethics training in the protection of human subjects is designed for people who, unlike CPS, have had exposure to prior formal research training. Objective: The aim of this study is to explore the perspectives of CPS in completing the Collaborative Institutional Training Initiative Social and Behavioral Responsible Conduct of Research online training. Methods: A total of 5 CPS were recruited using a convenience sample framework through the parent study, a patient-centered outcomes research study that examined the comparative effectiveness of two chronic health disease management programs for people with serious mental illness. Participants independently completed the Collaborative Institutional Training Initiative Social and Behavioral Responsible Conduct of Research online training. All participants completed 15 online modules in approximately 7-9 hours and also filled out a self-report measure of executive functioning (the Adult Executive Functioning Inventory [ADEXI]). Qualitative data were collected from a 1-hour focus group and qualitative analysis was informed by the grounded theory approach. The codebook consisted of codes inductively derived from the data. Codes were independently assigned to text, grouped, and checked for themes. Thematic analysis was used to organize themes. Results: Passing scores for each module ranged from 81%-89%, with an average of 85.4% and a median of 86%. The two themes that emerged from the focus group were the following: comprehension (barrier) and opportunity (facilitator). Participants had a mean score of 27.4 on the ADEXI. Conclusions: The CPS perceived the research ethics online training as an opportunity to share their lived experience expertise to enhance current research efforts by nonpeer scientists. Although the CPS completed the online research ethics training, the findings indicate CPS experienced difficulty with comprehension of the research ethics online training materials. Adaptations may be needed to facilitate uptake of research ethics online training by CPS and create a workforce of CPS to offer their lived experience expertise alongside peer and nonpeer researchers. UR - https://formative.jmir.org/2022/2/e29073 UR - http://dx.doi.org/10.2196/29073 UR - http://www.ncbi.nlm.nih.gov/pubmed/35103606 ID - info:doi/10.2196/29073 ER - TY - JOUR AU - Gilbert, M. Rose AU - Sumodhee, Dayyanah AU - Pontikos, Nikolas AU - Hollyhead, Catherine AU - Patrick, Angus AU - Scarles, Samuel AU - Van Der Smissen, Sabrina AU - Young, M. Rodrigo AU - Nettleton, Nick AU - Webster, R. Andrew AU - Cammack, Jocelyn PY - 2022/1/31 TI - Collaborative Research and Development of a Novel, Patient-Centered Digital Platform (MyEyeSite) for Rare Inherited Retinal Disease Data: Acceptability and Feasibility Study JO - JMIR Form Res SP - e21341 VL - 6 IS - 1 KW - MyEyeSite KW - inherited retinal diseases (IRD) KW - rare diseases KW - genetics KW - ophthalmology KW - digital health KW - eye data KW - GDPR KW - subject access request (SAR) KW - mobile phone N2 - Background: Inherited retinal diseases (IRDs) are a leading cause of blindness in children and working age adults in the United Kingdom and other countries, with an appreciable socioeconomic impact. However, by definition, IRD data are individually rare, and as a result, this patient group has been underserved by research. Researchers need larger amounts of these rare data to make progress in this field, for example, through the development of gene therapies. The challenge has been how to find and make these data available to researchers in the most productive way. MyEyeSite is a research collaboration aiming to design and develop a digital platform (the MyEyeSite platform) for people with rare IRDs that will enable patients, doctors, and researchers to aggregate and share specialist eye health data. A crucial component of this platform is the MyEyeSite patient application, which will provide the means for patients with IRD to interact with the system and, in particular, to collate, manage, and share their personal specialist IRD data both for research and their own health care. Objective: This study aims to test the acceptability and feasibility of the MyEyeSite platform in the target IRD population through a collaborative patient-centered study. Methods: Qualitative data were generated through focus groups and workshops, and quantitative data were obtained through a survey of patients with IRD. Participants were recruited through clinics at Moorfields Eye Hospital National Health Service (NHS) Foundation Trust and the National Institute for Health Research (NIHR) Moorfields Biomedical Research Centre through their patient and public involvement databases. Results: Our IRD focus group sample (n=50) highlighted the following themes: frustration with the current system regarding data sharing within the United Kingdom?s NHS; positive expectations of the potential benefits of the MyEyeSite patient application, resulting from increased access to this specialized data; and concerns regarding data security, including potentially unethical use of the data outside the NHS. Of the surveyed 80 participants, 68 (85%) were motivated to have a more active role in their eye care and share their data for research purposes using a secure technology, such as a web application or mobile app. Conclusions: This study demonstrates that patients with IRD are highly motivated to be actively involved in managing their own data for research and their own eye care. It demonstrates the feasibility of involving patients with IRD in the detailed design of the MyEyeSite platform exemplar, with input from the patient with IRD workshops playing a key role in determining both the functionality and accessibility of the designs and prototypes. The development of a user-centered technological solution to the problem of rare health data has the potential to benefit not only the patient with IRD community but also others with rare diseases. UR - https://formative.jmir.org/2022/1/e21341 UR - http://dx.doi.org/10.2196/21341 UR - http://www.ncbi.nlm.nih.gov/pubmed/35099396 ID - info:doi/10.2196/21341 ER - TY - JOUR AU - Salihu, M. Hamisu AU - Yusuf, Zenab AU - Dongarwar, Deepa AU - Aliyu, H. Sani AU - Yusuf, A. Rafeek AU - Aliyu, H. Muktar AU - Aliyu, Gambo PY - 2022/1/28 TI - Development of a Quality Assurance Score for the Nigeria AIDS Indicator and Impact Survey (NAIIS) Database: Validation Study JO - JMIR Form Res SP - e25752 VL - 6 IS - 1 KW - database quality assurance KW - Delphi method KW - quality assurance tool KW - Nigeria AIDS Indicator and Impact Survey N2 - Background: In 2018, Nigeria implemented the world?s largest HIV survey, the Nigeria AIDS Indicator and Impact Survey (NAIIS), with the overarching goal of obtaining more reliable metrics regarding the national scope of HIV epidemic control in Nigeria. Objective: This study aimed to (1) describe the processes involved in the development of a new database evaluation tool (Database Quality Assurance Score [dQAS]) and (2) assess the application of the dQAS in the evaluation and validation of the NAIIS database. Methods: The dQAS tool was created using an online, electronic Delphi (e-Delphi) methodology with the assistance of expert review panelists. Thematic categories were developed to form superordinate categories that grouped themes together. Subordinate categories were then created that decomposed themes for more specificity. A validation score using dQAS was employed to assess the technical performance of the NAIIS database. Results: The finalized dQAS tool was composed of 34 items, with a total score of 81. The tool had 2 sections: validation item section, which contains 5 subsections, and quality assessment score section, with a score of ?1? for ?Yes? to indicate that the performance measure item was present and ?0? for ?No? to indicate that the measure was absent. There were also additional scaling scores ranging from ?0? to a maximum of ?4? depending on the measure. The NAIIS database achieved 78 out of the maximum total score of 81, yielding an overall technical performance score of 96.3%, which placed it in the highest category denoted as ?Exceptional.? Conclusions: This study showed the feasibility of remote internet-based collaboration for the development of dQAS?a tool to assess the validity of a locally created database infrastructure for a resource-limited setting. Using dQAS, the NAIIS database was found to be valid, reliable, and a valuable source of data for future population-based, HIV-related studies. UR - https://formative.jmir.org/2022/1/e25752 UR - http://dx.doi.org/10.2196/25752 UR - http://www.ncbi.nlm.nih.gov/pubmed/35089143 ID - info:doi/10.2196/25752 ER - TY - JOUR AU - Chen, Yu-Chi AU - Cheng, Christina AU - Osborne, H. Richard AU - Kayser, Lars AU - Liu, Chieh-Yu AU - Chang, Li-Chun PY - 2022/1/19 TI - Validity Testing and Cultural Adaptation of the eHealth Literacy Questionnaire (eHLQ) Among People With Chronic Diseases in Taiwan: Mixed Methods Study JO - J Med Internet Res SP - e32855 VL - 24 IS - 1 KW - chronic illness KW - eHealth literacy questionnaire KW - eHLQ KW - validation KW - cultural adaptation KW - eHealth N2 - Background: Advancements in digital technologies seek to promote health and access to services. However, people lacking abilities and confidence to use technology are likely to be left behind, leading to health disparities. In providing digital health services, health care providers need to be aware of users? diverse electronic health (eHealth) literacy to address their particular needs and ensure equitable uptake and use of digital services. To understand such needs, an instrument that captures users? knowledge, skills, trust, motivation, and experiences in relation to technology is required. The eHealth Literacy Questionnaire (eHLQ) is a multidimensional tool with 7 scales covering diverse dimensions of eHealth literacy. The tool was simultaneously developed in English and Danish using a grounded and validity-driven approach and has been shown to have strong psychometric properties. Objective: This study aims to translate and culturally adapt the eHLQ for application among Mandarin-speaking people with chronic diseases in Taiwan and then undertake a rigorous set of validity-testing procedures. Methods: The cross-cultural adaptation of the eHLQ included translation and evaluation of the translations. The measurement properties were assessed using classical test theory and item response theory (IRT) approaches. Content validity, known-group validity, and internal consistency were explored, as well as item characteristic curves (ICCs), item discrimination, and item location/difficulty. Results: The adapted version was reviewed, and a recommended forward translation was confirmed through consensus. The tool exhibited good content validity. A total of 420 people with 1 or more chronic diseases participated in a validity-testing survey. The eHLQ showed good internal consistency (Cronbach ?=.75-.95). For known-group validity, all 7 eHLQ scales showed strong associations with education. Unidimensionality and local independence assumptions were met except for scale 2. IRT analysis showed that all items demonstrated good discrimination (range 0.27-12.15) and a good range of difficulty (range 0.59-1.67) except for 2 items in scale 7. Conclusions: Using a rigorous process, the eHLQ was translated from English into a culturally appropriate tool for use in the Mandarin language. Validity testing provided evidence of satisfactory-to-strong psychometric properties of the eHLQ. The 7 scales are likely to be useful research tools for evaluating digital health interventions and for informing the development of health technology products and interventions that equitably suit diverse users? needs. UR - https://www.jmir.org/2022/1/e32855 UR - http://dx.doi.org/10.2196/32855 UR - http://www.ncbi.nlm.nih.gov/pubmed/35044310 ID - info:doi/10.2196/32855 ER - TY - JOUR AU - Cihoric, Nikola AU - Badra, Vlaskou Eugenia AU - Stenger-Weisser, Anna AU - Aebersold, M. Daniel AU - Pavic, Matea PY - 2022/1/19 TI - Toward Data-Driven Radiation Oncology Using Standardized Terminology as a Starting Point: Cross-sectional Study JO - JMIR Form Res SP - e27550 VL - 6 IS - 1 KW - terminology KW - semantic interoperability KW - radiation oncology KW - informatics KW - medical informatics KW - oncology KW - lexical analysis KW - eHealth N2 - Background: The inability to seamlessly exchange information across radiation therapy ecosystems is a limiting factor in the pursuit of data-driven clinical practice. The implementation of semantic interoperability is a prerequisite for achieving the full capacity of the latest developments in personalized and precision medicine, such as mathematical modeling, advanced algorithmic information processing, and artificial intelligence approaches. Objective: This study aims to evaluate the state of terminology resources (TRs) dedicated to radiation oncology as a prerequisite for an oncology semantic ecosystem. The goal of this cross-sectional analysis is to quantify the state of the art in radiation therapy specific terminology. Methods: The Unified Medical Language System (UMLS) was searched for the following terms: radio oncology, radiation oncology, radiation therapy, and radiotherapy. We extracted 6509 unique concepts for further analysis. We conducted a quantitative analysis of available source vocabularies (SVs) and analyzed all UMLS SVs according to the route source, number, author, location of authors, license type, the lexical density of TR, and semantic types. Descriptive data are presented as numbers and percentages. Results: The concepts were distributed across 35 SVs. The median number of unique concepts per SV was 5 (range 1-5479), with 14% (5/35) of SVs containing 94.59% (6157/6509) of the concepts. The SVs were created by 29 authors, predominantly legal entities registered in the United States (25/35, 71%), followed by international organizations (6/35, 17%), legal entities registered in Australia (2/35, 6%), and the Netherlands and the United Kingdom with 3% (1/35) of authors each. Of the total 35 SVs, 16 (46%) did not have any restrictions on use, whereas for 19 (54%) of SVs, some level of restriction was required. Overall, 57% (20/35) of SVs were updated within the last 5 years. All concepts found within radiation therapy SVs were labeled with one of the 29 semantic types represented within UMLS. After removing the stop words, the total number of words for all SVs together was 56,219, with a median of 25 unique words per SV (range 3-50,682). The total number of unique words in all SVs was 1048, with a median of 19 unique words per vocabulary (range 3-406). The lexical density for all concepts within all SVs was 0 (0.02 rounded to 2 decimals). Median lexical density per unique SV was 0.7 (range 0.0-1.0). There were no dedicated radiation therapy SVs. Conclusions: We did not identify any dedicated TRs for radiation oncology. Current terminologies are not sufficient to cover the need of modern radiation oncology practice and research. To achieve a sufficient level of interoperability, of the creation of a new, standardized, universally accepted TR dedicated to modern radiation therapy is required. UR - https://formative.jmir.org/2022/1/e27550 UR - http://dx.doi.org/10.2196/27550 UR - http://www.ncbi.nlm.nih.gov/pubmed/35044315 ID - info:doi/10.2196/27550 ER - TY - JOUR AU - Feldman, G. Amy AU - Moore, Susan AU - Bull, Sheana AU - Morris, A. Megan AU - Wilson, Kumanan AU - Bell, Cameron AU - Collins, M. Margaret AU - Denize, M. Kathryn AU - Kempe, Allison PY - 2022/1/13 TI - A Smartphone App to Increase Immunizations in the Pediatric Solid Organ Transplant Population: Development and Initial Usability Study JO - JMIR Form Res SP - e32273 VL - 6 IS - 1 KW - vaccinations KW - transplantation KW - mobile app KW - agile development KW - immunization KW - mHealth KW - mobile health KW - children KW - transplant recipients KW - pediatric transplant recipients KW - pediatrics N2 - Background: Vaccine-preventable infections result in significant morbidity, mortality, and costs in pediatric transplant recipients. However, at the time of transplant, less than 20% of children are up-to-date for age-appropriate immunizations that could prevent these diseases. Smartphone apps have the potential to increase immunization rates through their ability to provide vaccine education, send vaccine reminders, and facilitate communication between parents and a multidisciplinary medical group. Objective: The aim of this study was to describe the development of a smartphone app, Immunize PediatricTransplant, to promote pretransplant immunization and to report on app functionality and usability when applied to the target population. Methods: We used a mixed methods study design guided by the Mobile Health Agile Development and Evaluation Lifecycle. We first completed a formative research including semistructured interviews with transplant stakeholders (12 primary care physicians, 40 parents or guardians of transplant recipients, 11 transplant nurse coordinators, and 19 transplant subspecialists) to explore the acceptability of an immunization app to be used in the pretransplant period. Based on these findings, CANImmunize Inc developed the Immunize PediatricTransplant app. We next held 2 focus group discussions with 5-6 transplant stakeholders/group (n=11; 5 parents of transplant recipients, 2 primary care physicians, 2 transplant nurse coordinators, and 2 transplant subspecialists) to receive feedback on the app. After the app modifications were made, alpha testing was conducted on the functional prototype. We then implemented beta testing with 12 stakeholders (6 parents of transplant recipients, 2 primary care doctors, 2 transplant nurse coordinators, and 2 transplant subspecialists) to refine the app through an iterative process. Finally, the stakeholders completed the user version of the Mobile Application Rating Scale (uMARS) to assess the functionality and quality of the app. Results: A new Android- and Apple-compatible app, Immunize PediatricTransplant, was developed to improve immunization delivery in the pretransplant period. The app contains information about vaccine use in the pretransplant period, houses a complete immunization record for each child, includes a communication tool for parents and care providers, and sends automated reminders to parents and care providers when immunizations are due. During usability testing, the stakeholders were able to enter a mock vaccine record containing 16 vaccines in an average of 8.1 minutes (SD 1.8) with 87% accuracy. The stakeholders rated engagement, functionality, aesthetics, and information quality of the app as 4.2/5, 4.5/5, 4.6/5, and 4.8/5, respectively. All participants reported that they would recommend this app to families and care teams with a child awaiting solid organ transplant. Conclusions: Through a systematic, user-centered, agile, iterative approach, the Immunize PediatricTransplant app was developed to improve immunization delivery in the pretransplant period. The app tested well with end users. Further testing and agile development among patients awaiting transplant are needed to understand real-world acceptability and effectiveness in improving immunization rates in children awaiting transplant. UR - https://formative.jmir.org/2022/1/e32273 UR - http://dx.doi.org/10.2196/32273 UR - http://www.ncbi.nlm.nih.gov/pubmed/35023840 ID - info:doi/10.2196/32273 ER - TY - JOUR AU - Farr, E. Deeonna AU - Battle, A. Darian AU - Hall, B. Marla PY - 2022/1/12 TI - Using Facebook Advertisements for Women?s Health Research: Methodology and Outcomes of an Observational Study JO - JMIR Form Res SP - e31759 VL - 6 IS - 1 KW - social media KW - surveys KW - questionnaires KW - advertising KW - patient selection KW - methodology KW - ethnic groups KW - health research KW - healthcare KW - health care KW - women?s health N2 - Background: Recruitment of diverse populations for health research studies remains a challenge. The COVID-19 pandemic has exacerbated these challenges by limiting in-person recruitment efforts and placing additional demands on potential participants. Social media, through the use of Facebook advertisements, has the potential to address recruitment challenges. However, existing reports are inconsistent with regard to the success of this strategy. Additionally, limited information is available about processes that can be used to increase the diversity of study participants. Objective: A Qualtrics survey was fielded to ascertain women?s knowledge of and health care experiences related to breast density. This paper describes the process of using Facebook advertisements for recruitment and the effectiveness of various advertisement strategies. Methods: Facebook advertisements were placed in 2 rounds between June and July 2020. During round 1, multiple combinations of headlines and interest terms were tested to determine the most cost-effective advertisement. The best performing advertisement was used in round 2 in combination with various strategies to enhance the diversity of the survey sample. Advertisement performance, cost, and survey respondent data were collected and examined. Results: In round 1, a total of 45 advertisements with 5 different headlines were placed, and the average cost per link click for each headline ranged from US $0.12 to US $0.79. Of the 164 women recruited in round 1, in total 91.62% were eligible to complete the survey. Advertisements used during recruitment in round 2 resulted in an average cost per link click of US $0.11. During the second round, 478 women attempted the survey, and 87.44% were eligible to participate. The majority of survey respondents were White (80.41%), over the age of 55 years (63.94%), and highly educated (63.71%). Conclusions: Facebook advertisements can be used to recruit respondents for health research quickly, but this strategy may yield participants who are less racially diverse, more educated, and older than the general population. Researchers should consider recruiting participants through other methods in addition to creating Facebook advertisements targeting underrepresented populations. UR - https://formative.jmir.org/2022/1/e31759 UR - http://dx.doi.org/10.2196/31759 UR - http://www.ncbi.nlm.nih.gov/pubmed/35019843 ID - info:doi/10.2196/31759 ER - TY - JOUR AU - Shalaby, Reham AU - Vuong, Wesley AU - Eboreime, Ejemai AU - Surood, Shireen AU - Greenshaw, J. Andrew AU - Agyapong, Opoku Vincent Israel PY - 2022/1/11 TI - Patients? Expectations and Experiences With a Mental Health?Focused Supportive Text Messaging Program: Mixed Methods Evaluation JO - JMIR Form Res SP - e33438 VL - 6 IS - 1 KW - supportive text messages KW - patients? experience KW - mental health KW - mixed methods N2 - Background: Web-based services are an economical and easily scalable means of support that uses existing technology. Text4Support is a supportive, complementary text messaging service that supports people with different mental health conditions after they are discharged from inpatient psychiatric care. Objective: In this study, we aim to assess user satisfaction with the Text4Support service to gain a better understanding of subscribers? experiences. Methods: This was a mixed methods study using secondary data from a pilot observational controlled trial. The trial included 181 patients discharged from acute psychiatric care and distributed into 4 randomized groups. Out of the 4 study groups in the initial study, 2 groups who received supportive text messages (89/181, 49.2% of patients), either alone or alongside a peer support worker, were included. Thematic and descriptive analyses were also performed. Differences in feedback based on sex at birth and primary diagnosis were determined using univariate analysis. The study was registered with ClinicalTrials.gov (trial registration number: NCT03404882). Results: Out of 89 participants, 36 (40%) completed the follow-up survey. The principal findings were that Text4Support was well perceived with a high satisfaction rate either regarding the feedback of the messages or their perceived impact. Meanwhile, there was no statistically significant difference between satisfactory items based on the subscriber?s sex at birth or primary diagnosis. The patients? initial expectations were either neutral or positive in relation to the expected nature or the impact of the text messages received on their mental well-being. In addition, the subscribers were satisfied with the frequency of the messages, which were received once daily for 6 consecutive months. The participants recommended more personalized messages or mutual interaction with health care personnel. Conclusions: Text4Support was generally well perceived by patients after hospital discharge, regardless of their sex at birth or mental health diagnosis. Further personalization and interactive platforms were recommended by participants that may need to be considered when designing similar future services. UR - https://formative.jmir.org/2022/1/e33438 UR - http://dx.doi.org/10.2196/33438 UR - http://www.ncbi.nlm.nih.gov/pubmed/35014972 ID - info:doi/10.2196/33438 ER - TY - JOUR AU - Braekman, Elise AU - Demarest, Stefaan AU - Charafeddine, Rana AU - Drieskens, Sabine AU - Berete, Finaba AU - Gisle, Lydia AU - Van der Heyden, Johan AU - Van Hal, Guido PY - 2022/1/7 TI - Unit Response and Costs in Web Versus Face-To-Face Data Collection: Comparison of Two Cross-sectional Health Surveys JO - J Med Internet Res SP - e26299 VL - 24 IS - 1 KW - health interview surveys KW - data collection mode KW - face-to-face KW - web KW - unit response KW - response rate KW - nonresponse KW - data collection costs KW - web data KW - health surveys KW - internet penetration KW - web survey KW - costs N2 - Background: Potential is seen in web data collection for population health surveys due to its combined cost-effectiveness, implementation ease, and increased internet penetration. Nonetheless, web modes may lead to lower and more selective unit response than traditional modes, and this may increase bias in the measured indicators. Objective: This research assesses the unit response and costs of a web study versus face-to-face (F2F) study. Methods: Alongside the Belgian Health Interview Survey by F2F edition 2018 (BHISF2F; net sample used: 3316), a web survey (Belgian Health Interview Survey by Web [BHISWEB]; net sample used: 1010) was organized. Sociodemographic data on invited individuals was obtained from the national register and census linkages. Unit response rates considering the different sampling probabilities of both surveys were calculated. Logistic regression analyses examined the association between mode system and sociodemographic characteristics for unit nonresponse. The costs per completed web questionnaire were compared with the costs for a completed F2F questionnaire. Results: The unit response rate is lower in BHISWEB (18.0%) versus BHISF2F (43.1%). A lower response rate was observed for the web survey among all sociodemographic groups, but the difference was higher among people aged 65 years and older (15.4% vs 45.1%), lower educated people (10.9% vs 38.0%), people with a non-Belgian European nationality (11.4% vs 40.7%), people with a non-European nationality (7.2% vs 38.0%), people living alone (12.6% vs 40.5%), and people living in the Brussels-Capital (12.2% vs 41.8%) region. The sociodemographic characteristics associated with nonresponse are not the same in the 2 studies. Having another European (OR 1.60, 95% CI 1.20-2.13) or non-European nationality (OR 2.57, 95% CI 1.79-3.70) compared to a Belgian nationality and living in the Brussels-Capital (OR 1.72, 95% CI 1.41-2.10) or Walloon (OR 1.47, 95% CI 1.15-1.87) regions compared to the Flemish region are associated with a higher nonresponse only in the BHISWEB study. In BHISF2F, younger people (OR 1.31, 95% CI 1.11-1.54) are more likely to be nonrespondents than older people, and this was not the case in BHISWEB. In both studies, lower educated people have a higher probability of being nonrespondent, but this effect is more pronounced in BHISWEB (low vs high education level: Web, OR 2.71, 95% CI 2.21-3.39 and F2F OR 1.70, 95% CI 1.48-1.95). The BHISWEB study had a considerable advantage; the cost per completed questionnaire was almost 3 times lower (?41 [US $48]) compared with F2F data collection (?111 [US $131]). Conclusions: The F2F unit response rate was generally higher, yet for certain groups the difference between web and F2F was more limited. Web data collection has a considerable cost advantage. It is therefore worth experimenting with adaptive mixed-mode designs to optimize financial resources without increasing selection bias (eg, only inviting sociodemographic groups who are keener to participate online for web surveys while continuing to focus on increasing F2F response rates for other groups). UR - https://www.jmir.org/2022/1/e26299 UR - http://dx.doi.org/10.2196/26299 UR - http://www.ncbi.nlm.nih.gov/pubmed/34994701 ID - info:doi/10.2196/26299 ER - TY - JOUR AU - Hsu, Wan-Yu AU - Rowles, William AU - Anguera, A. Joaquin AU - Anderson, Annika AU - Younger, W. Jessica AU - Friedman, Samuel AU - Gazzaley, Adam AU - Bove, Riley PY - 2021/12/30 TI - Assessing Cognitive Function in Multiple Sclerosis With Digital Tools: Observational Study JO - J Med Internet Res SP - e25748 VL - 23 IS - 12 KW - cognition KW - digital health KW - mHealth KW - multiple sclerosis KW - cognitive assessment N2 - Background: Cognitive impairment (CI) is one of the most prevalent symptoms of multiple sclerosis (MS). However, it is difficult to include cognitive assessment as part of MS standard care since the comprehensive neuropsychological examinations are usually time-consuming and extensive. Objective: To improve access to CI assessment, we evaluated the feasibility and potential assessment sensitivity of a tablet-based cognitive battery in patients with MS. Methods: In total, 53 participants with MS (24 [45%] with CI and 29 [55%] without CI) and 24 non-MS participants were assessed with a tablet-based cognitive battery (Adaptive Cognitive Evaluation [ACE]) and standard cognitive measures, including the Symbol Digit Modalities Test (SDMT) and the Paced Auditory Serial Addition Test (PASAT). Associations between performance in ACE and the SDMT/PASAT were explored, with group comparisons to evaluate whether ACE modules can capture group-level differences. Results: Correlations between performance in ACE and the SDMT (R=?0.57, P<.001), as well as PASAT (R=?0.39, P=.01), were observed. Compared to non-MS and non-CI MS groups, the CI MS group showed a slower reaction time (CI MS vs non-MS: P<.001; CI MS vs non-CI MS: P=.004) and a higher attention cost (CI MS vs non-MS: P=.02; CI MS vs non-CI MS: P<.001). Conclusions: These results provide preliminary evidence that ACE, a tablet-based cognitive assessment battery, provides modules that could potentially serve as a digital cognitive assessment for people with MS. Trial Registration: ClinicalTrials.gov NCT03569618; https://clinicaltrials.gov/ct2/show/NCT03569618 UR - https://www.jmir.org/2021/12/e25748 UR - http://dx.doi.org/10.2196/25748 UR - http://www.ncbi.nlm.nih.gov/pubmed/34967751 ID - info:doi/10.2196/25748 ER - TY - JOUR AU - Liu, Dianbo AU - Zheng, Ming AU - Sepulveda, Andres Nestor PY - 2021/12/8 TI - Using Artificial Neural Network Condensation to Facilitate Adaptation of Machine Learning in Medical Settings by Reducing Computational Burden: Model Design and Evaluation Study JO - JMIR Form Res SP - e20767 VL - 5 IS - 12 KW - artificial neural network KW - electronic medical records KW - parameter pruning KW - machine learning KW - computational burden KW - N2 - Background: Machine learning applications in the health care domain can have a great impact on people?s lives. At the same time, medical data is usually big, requiring a significant number of computational resources. Although this might not be a problem for the wide adoption of machine learning tools in high-income countries, the availability of computational resources can be limited in low-income countries and on mobile devices. This can limit many people from benefiting from the advancement in machine learning applications in the field of health care. Objective: In this study, we explore three methods to increase the computational efficiency and reduce model sizes of either recurrent neural networks (RNNs) or feedforward deep neural networks (DNNs) without compromising their accuracy. Methods: We used inpatient mortality prediction as our case analysis upon review of an intensive care unit dataset. We reduced the size of RNN and DNN by applying pruning of ?unused? neurons. Additionally, we modified the RNN structure by adding a hidden layer to the RNN cell but reducing the total number of recurrent layers to accomplish a reduction of the total parameters used in the network. Finally, we implemented quantization on DNN by forcing the weights to be 8 bits instead of 32 bits. Results: We found that all methods increased implementation efficiency, including training speed, memory size, and inference speed, without reducing the accuracy of mortality prediction. Conclusions: Our findings suggest that neural network condensation allows for the implementation of sophisticated neural network algorithms on devices with lower computational resources. UR - https://formative.jmir.org/2021/12/e20767 UR - http://dx.doi.org/10.2196/20767 UR - http://www.ncbi.nlm.nih.gov/pubmed/34889747 ID - info:doi/10.2196/20767 ER - TY - JOUR AU - Alanazi, M. Eman AU - Abdou, Aalaa AU - Luo, Jake PY - 2021/12/2 TI - Predicting Risk of Stroke From Lab Tests Using Machine Learning Algorithms: Development and Evaluation of Prediction Models JO - JMIR Form Res SP - e23440 VL - 5 IS - 12 KW - stroke KW - lab tests KW - machine learning technology KW - predictive analytics N2 - Background: Stroke, a cerebrovascular disease, is one of the major causes of death. It causes significant health and financial burdens for both patients and health care systems. One of the important risk factors for stroke is health-related behavior, which is becoming an increasingly important focus of prevention. Many machine learning models have been built to predict the risk of stroke or to automatically diagnose stroke, using predictors such as lifestyle factors or radiological imaging. However, there have been no models built using data from lab tests. Objective: The aim of this study was to apply computational methods using machine learning techniques to predict stroke from lab test data. Methods: We used the National Health and Nutrition Examination Survey data sets with three different data selection methods (ie, without data resampling, with data imputation, and with data resampling) to develop predictive models. We used four machine learning classifiers and six performance measures to evaluate the performance of the models. Results: We found that accurate and sensitive machine learning models can be created to predict stroke from lab test data. Our results show that the data resampling approach performed the best compared to the other two data selection techniques. Prediction with the random forest algorithm, which was the best algorithm tested, achieved an accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under the curve of 0.96, 0.97, 0.96, 0.75, 0.99, and 0.97, respectively, when all of the attributes were used. Conclusions: The predictive model, built using data from lab tests, was easy to use and had high accuracy. In future studies, we aim to use data that reflect different types of stroke and to explore the data to build a prediction model for each type. UR - https://formative.jmir.org/2021/12/e23440 UR - http://dx.doi.org/10.2196/23440 UR - http://www.ncbi.nlm.nih.gov/pubmed/34860663 ID - info:doi/10.2196/23440 ER - TY - JOUR AU - Parker, N. Jayelin AU - Hunter, S. Alexis AU - Bauermeister, A. Jose AU - Bonar, E. Erin AU - Carrico, Adam AU - Stephenson, Rob PY - 2021/12/1 TI - Comparing Social Media and In-Person Recruitment: Lessons Learned From Recruiting Substance-Using, Sexual and Gender Minority Adolescents and Young Adults for a Randomized Control Trial JO - JMIR Public Health Surveill SP - e31657 VL - 7 IS - 12 KW - HIV testing KW - substance use KW - recruitment KW - sexual and gender minorities KW - youth N2 - Background: Recruiting large samples of diverse sexual and gender minority adolescent and young adults (AYAs) into HIV intervention research is critical to the development and later dissemination of interventions that address the risk factors for HIV transmission among substance-using, sexual and gender minority AYAs. Objective: This paper aimed to describe the characteristics of the samples recruited via social media and in-person methods and makes recommendations for strategies to recruit substance-using, sexual and gender minority AYAs, a hardly reached population that is a priority for HIV prevention research. Methods: Using data from a randomized control trial of an HIV and substance use intervention with sexual and gender minority AYAs, aged 15 to 29 years in southeastern Michigan (n=414), we examined demographic and behavioral characteristics associated with successful recruitment from a range of virtual and physical venues. Results: We found that paid advertisements on Facebook, Instagram, and Grindr offered the largest quantity of eligible participants willing to enroll in the trial. Instagram offered the largest proportion of transgender masculine participants, and Grindr offered the largest proportion of Black/African American individuals. Although we attempted venue-based recruitment at clubs, bars, community centers, and AIDS service organizations, we found it to be unsuccessful for this specific hardly reached population. Social media and geobased dating applications offered the largest pool of eligible participants. Conclusions: Understanding factors associated with successful recruitment has the potential to inform effective and efficient strategies for HIV prevention research with substance-using, sexual and gender AYAs. Trial Registration: ClinicalTrials.gov NCT02945436; https://clinicaltrials.gov/ct2/show/NCT02945436 International Registered Report Identifier (IRRID): RR2-10.2196/resprot.9414 UR - https://publichealth.jmir.org/2021/12/e31657 UR - http://dx.doi.org/10.2196/31657 UR - http://www.ncbi.nlm.nih.gov/pubmed/34855613 ID - info:doi/10.2196/31657 ER - TY - JOUR AU - Gleason, Neil AU - Serrano, A. Pedro AU - Muñoz, Alejandro AU - French, Audrey AU - Hosek, Sybil PY - 2021/11/26 TI - Limited Interaction Targeted Epidemiology of HIV in Sexual and Gender Minority American Adolescents and Adults: Feasibility of the Keeping it LITE Study JO - JMIR Form Res SP - e30761 VL - 5 IS - 11 KW - social epidemiology KW - adolescents and young adults KW - sexual and gender minorities KW - HIV testing N2 - Background: HIV infection rates among sexual minority men and transgender individuals, particularly adolescents and young adults, remain elevated in the United States despite continued improvement in the HIV public health response. However, there remains a knowledge gap in understanding the barriers faced by this community in receiving HIV care and prevention resources. To address this, the Keeping it LITE study was conducted to assess HIV risk factors and barriers to preventive treatment in a large national cohort of young sexual minority men and transgender individuals at high risk of HIV infection. Objective: This study aims to evaluate the feasibility of enrolling a large remote cohort, challenges encountered in recruitment, and adjustments made to address these challenges. Methods: A large national cohort (n=3444) of young sexual minority men and transgender individuals were recruited. Participants were recruited via advertisements on social media; social apps for lesbian, gay, bisexual, transgender, and queer individuals; print advertising; and word-of-mouth. Before enrolling, participants verified their HIV status using an at-home HIV test or by providing their own testing documentation. Descriptive statistics were generated, and a series of logistic regressions were conducted to evaluate demographic differences between recruitment methods, HIV testing methods, and enrollment status. Results: The Keeping it LITE study was particularly successful in recruiting participants via social media, with over half of the participants recruited from advertisements on social media platforms such as Facebook, Instagram, and Snapchat. Participants were also recruited via word-of-mouth; lesbian, gay, bisexual, transgender, and queer apps (ie, Grindr, Scruff); and print advertisements, and participants recruited from these sources tended to be older and have a higher risk profile. The study was also successful in recruiting a large sample of transgender youth, particularly transgender men and nonbinary individuals. At-home HIV testing was acceptable and more heavily used by younger participants, although several barriers were encountered and overcome in the implementation of this testing. The study had more limited success in recruiting participants aged 13-17 years because of lower enrollment rates and barriers to advertising on social media platforms. The implications of these findings for the future development of HIV research and intervention protocols among sexual minorities and trans youth are discussed. Conclusions: The methods used in the Keeping it LITE study, particularly recruitment via social media, were found to be feasible and acceptable to participants. UR - https://formative.jmir.org/2021/11/e30761 UR - http://dx.doi.org/10.2196/30761 UR - http://www.ncbi.nlm.nih.gov/pubmed/34346403 ID - info:doi/10.2196/30761 ER - TY - JOUR AU - Holub, Karl AU - Hardy, Nicole AU - Kallmes, Kevin PY - 2021/11/24 TI - Toward Automated Data Extraction According to Tabular Data Structure: Cross-sectional Pilot Survey of the Comparative Clinical Literature JO - JMIR Form Res SP - e33124 VL - 5 IS - 11 KW - table structure KW - systematic review KW - automated data extraction KW - data reporting conventions KW - clinical comparative data KW - data elements KW - statistic formats N2 - Background: Systematic reviews depend on time-consuming extraction of data from the PDFs of underlying studies. To date, automation efforts have focused on extracting data from the text, and no approach has yet succeeded in fully automating ingestion of quantitative evidence. However, the majority of relevant data is generally presented in tables, and the tabular structure is more amenable to automated extraction than free text. Objective: The purpose of this study was to classify the structure and format of descriptive statistics reported in tables in the comparative medical literature. Methods: We sampled 100 published randomized controlled trials from 2019 based on a search in PubMed; these results were imported to the AutoLit platform. Studies were excluded if they were nonclinical, noncomparative, not in English, protocols, or not available in full text. In AutoLit, tables reporting baseline or outcome data in all studies were characterized based on reporting practices. Measurement context, meaning the structure in which the interventions of interest, patient arm breakdown, measurement time points, and data element descriptions were presented, was classified based on the number of contextual pieces and metadata reported. The statistic formats for reported metrics (specific instances of reporting of data elements) were then classified by location and broken down into reporting strategies for continuous, dichotomous, and categorical metrics. Results: We included 78 of 100 sampled studies, one of which (1.3%) did not report data elements in tables. The remaining 77 studies reported baseline and outcome data in 174 tables, and 96% (69/72) of these tables broke down reporting by patient arms. Fifteen structures were found for the reporting of measurement context, which were broadly grouped into: 1×1 contexts, where two pieces of context are reported in total (eg, arms in columns, data elements in rows); 2×1 contexts, where two pieces of context are given on row headers (eg, time points in columns, arms nested in data elements on rows); and 1×2 contexts, where two pieces of context are given on column headers. The 1×1 contexts were present in 57% of tables (99/174), compared to 20% (34/174) for 2×1 contexts and 15% (26/174) for 1×2 contexts; the remaining 8% (15/174) used unique/other stratification methods. Statistic formats were reported in the headers or descriptions of 84% (65/74) of studies. Conclusions: In this cross-sectional pilot review, we found a high density of information in tables, but with major heterogeneity in presentation of measurement context. The highest-density studies reported both baseline and outcome measures in tables, with arm-level breakout, intervention labels, and arm sizes present, and reported both the statistic formats and units. The measurement context formats presented here, broadly classified into three classes that cover 92% (71/78) of studies, form a basis for understanding the frequency of different reporting styles, supporting automated detection of the data format for extraction of metrics. UR - https://formative.jmir.org/2021/11/e33124 UR - http://dx.doi.org/10.2196/33124 UR - http://www.ncbi.nlm.nih.gov/pubmed/34821562 ID - info:doi/10.2196/33124 ER - TY - JOUR AU - Beckers, B. Abraham AU - Snijkers, W. Johanna T. AU - Weerts, M. Zsa Zsa R. AU - Vork, Lisa AU - Klaassen, Tim AU - Smeets, M. Fabienne G. AU - Masclee, M. Ad A. AU - Keszthelyi, Daniel PY - 2021/11/24 TI - Digital Instruments for Reporting of Gastrointestinal Symptoms in Clinical Trials: Comparison of End-of-Day Diaries Versus the Experience Sampling Method JO - JMIR Form Res SP - e31678 VL - 5 IS - 11 KW - irritable bowel syndrome KW - functional dyspepsia KW - digital diary KW - experience sampling method KW - smartphone app KW - mobile phone application KW - mHealth KW - eHealth KW - compliance KW - patient-reported outcome measures N2 - Background: Questionnaires are necessary tools for assessing symptoms of disorders of the brain-gut interaction in clinical trials. We previously reported on the excellent adherence to a smartphone app used as symptom diary in a randomized clinical trial on irritable bowel syndrome (IBS). Other sampling methods, such as the experience sampling method (ESM), are better equipped to measure symptom variability over time and provide useful information regarding possible symptom triggers, and they are free of ecological and recall bias. The high frequency of measurements, however, could limit the feasibility of ESM in clinical trials. Objective: This study aimed to compare the adherence rates of a smartphone-based end-of-day diary and ESM for symptom assessment in IBS and functional dyspepsia (FD). Methods: Data from 4 separate studies were included. Patients with IBS participated in a randomized controlled trial, which involved a smartphone end-of-day diary for a 2+8-week (pretreatment + treatment) period, and an observational study in which patients completed ESM assessments using a smartphone app for 1 week. Patients with FD participated in a randomized controlled trial, which involved a smartphone end-of-day diary for a 2+12-week (pretreatment + treatment) period, and an observational study in which patients completed ESM assessments using a smartphone app for 1 week. Adherence rates were compared between these 2 symptom sampling methods. Results: In total, 25 patients with IBS and 15 patients with FD were included. Overall adherence rates for the end-of-day diaries were significantly higher than those for ESM (IBS: 92.7% vs 69.8%, FD: 90.1% vs 61.4%, respectively). Conclusions: This study demonstrates excellent adherence rates for smartphone app?based end-of-day diaries as used in 2 separate clinical trials. Overall adherence rates for ESM were significantly lower, rendering it more suitable for intermittent sampling periods rather than continuous sampling during longer clinical trials. UR - https://formative.jmir.org/2021/11/e31678 UR - http://dx.doi.org/10.2196/31678 UR - http://www.ncbi.nlm.nih.gov/pubmed/34821561 ID - info:doi/10.2196/31678 ER - TY - JOUR AU - Sasaki, Natsu AU - Obikane, Erika AU - Vedanthan, Rajesh AU - Imamura, Kotaro AU - Cuijpers, Pim AU - Shimazu, Taichi AU - Kamada, Masamitsu AU - Kawakami, Norito AU - Nishi, Daisuke PY - 2021/11/23 TI - Implementation Outcome Scales for Digital Mental Health (iOSDMH): Scale Development and Cross-sectional Study JO - JMIR Form Res SP - e24332 VL - 5 IS - 11 KW - implementation outcomes KW - acceptability KW - appropriateness KW - feasibility KW - harm N2 - Background: Digital mental health interventions are being used more than ever for the prevention and treatment of psychological problems. Optimizing the implementation aspects of digital mental health is essential to deliver the program to populations in need, but there is a lack of validated implementation outcome measures for digital mental health interventions. Objective: The primary aim of this study is to develop implementation outcome scales of digital mental health for different levels of stakeholders involved in the implementation process: users, providers, and managers or policy makers. The secondary aim is to validate the developed scale for users. Methods: We developed English and Japanese versions of the implementation outcome scales for digital mental health (iOSDMH) based on the literature review and panel discussions with experts in implementation research and web-based psychotherapy. The study developed acceptability, appropriateness, feasibility, satisfaction, and harm as the outcome measures for users, providers, and managers or policy makers. We conducted evidence-based interventions via the internet using UTSMeD, a website for mental health information (N=200). Exploratory factor analysis (EFA) was conducted to assess the structural validity of the iOSDMH for users. Satisfaction, which consisted of a single item, was not included in the EFA. Results: The iOSDMH was developed for users, providers, and managers or policy makers. The iOSDMH contains 19 items for users, 11 items for providers, and 14 items for managers or policy makers. Cronbach ? coefficients indicated intermediate internal consistency for acceptability (?=.665) but high consistency for appropriateness (?=.776), feasibility (?=.832), and harm (?=.777) of the iOSDMH for users. EFA revealed 3-factor structures, indicating acceptability and appropriateness as close concepts. Despite the similarity between these 2 concepts, we inferred that acceptability and appropriateness should be used as different factors, following previous studies. Conclusions: We developed iOSDMH for users, providers, and managers. Psychometric assessment of the scales for users demonstrated acceptable reliability and validity. Evaluating the components of digital mental health implementation is a major step forward in implementation science. UR - https://formative.jmir.org/2021/11/e24332 UR - http://dx.doi.org/10.2196/24332 UR - http://www.ncbi.nlm.nih.gov/pubmed/34817391 ID - info:doi/10.2196/24332 ER - TY - JOUR AU - Wang, Huan AU - Wu, Wei AU - Han, Chunxia AU - Zheng, Jiaqi AU - Cai, Xinyu AU - Chang, Shimin AU - Shi, Junlong AU - Xu, Nan AU - Ai, Zisheng PY - 2021/11/19 TI - Prediction Model of Osteonecrosis of the Femoral Head After Femoral Neck Fracture: Machine Learning?Based Development and Validation Study JO - JMIR Med Inform SP - e30079 VL - 9 IS - 11 KW - femoral neck fracture KW - osteonecrosis of the femoral head KW - machine learning KW - interpretability N2 - Background: The absolute number of femoral neck fractures (FNFs) is increasing; however, the prediction of traumatic femoral head necrosis remains difficult. Machine learning algorithms have the potential to be superior to traditional prediction methods for the prediction of traumatic femoral head necrosis. Objective: The aim of this study is to use machine learning to construct a model for the analysis of risk factors and prediction of osteonecrosis of the femoral head (ONFH) in patients with FNF after internal fixation. Methods: We retrospectively collected preoperative, intraoperative, and postoperative clinical data of patients with FNF in 4 hospitals in Shanghai and followed up the patients for more than 2.5 years. A total of 259 patients with 43 variables were included in the study. The data were randomly divided into a training set (181/259, 69.8%) and a validation set (78/259, 30.1%). External data (n=376) were obtained from a retrospective cohort study of patients with FNF in 3 other hospitals. Least absolute shrinkage and selection operator regression and the support vector machine algorithm were used for variable selection. Logistic regression, random forest, support vector machine, and eXtreme Gradient Boosting (XGBoost) were used to develop the model on the training set. The validation set was used to tune the model hyperparameters to determine the final prediction model, and the external data were used to compare and evaluate the model performance. We compared the accuracy, discrimination, and calibration of the models to identify the best machine learning algorithm for predicting ONFH. Shapley additive explanations and local interpretable model-agnostic explanations were used to determine the interpretability of the black box model. Results: A total of 11 variables were selected for the models. The XGBoost model performed best on the validation set and external data. The accuracy, sensitivity, and area under the receiver operating characteristic curve of the model on the validation set were 0.987, 0.929, and 0.992, respectively. The accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve of the model on the external data were 0.907, 0.807, 0.935, and 0.933, respectively, and the log-loss was 0.279. The calibration curve demonstrated good agreement between the predicted probability and actual risk. The interpretability of the features and individual predictions were realized using the Shapley additive explanations and local interpretable model-agnostic explanations algorithms. In addition, the XGBoost model was translated into a self-made web-based risk calculator to estimate an individual?s probability of ONFH. Conclusions: Machine learning performs well in predicting ONFH after internal fixation of FNF. The 6-variable XGBoost model predicted the risk of ONFH well and had good generalization ability on the external data, which can be used for the clinical prediction of ONFH after internal fixation of FNF. UR - https://medinform.jmir.org/2021/11/e30079 UR - http://dx.doi.org/10.2196/30079 UR - http://www.ncbi.nlm.nih.gov/pubmed/34806984 ID - info:doi/10.2196/30079 ER - TY - JOUR AU - Sylcott, Brian AU - Lin, Chia-Cheng AU - Williams, Keith AU - Hinderaker, Mark PY - 2021/11/15 TI - Investigating the Use of Virtual Reality Headsets for Postural Control Assessment: Instrument Validation Study JO - JMIR Rehabil Assist Technol SP - e24950 VL - 8 IS - 4 KW - postural sway KW - virtual reality KW - force plate KW - center of pressure N2 - Background: Accurately measuring postural sway is an important part of balance assessment and rehabilitation. Although force plates give accurate measurements, their costs and space requirements make their use impractical in many situations. Objective: The work presented in this paper aimed to address this issue by validating a virtual reality (VR) headset as a relatively low-cost alternative to force plates for postural sway measurement. The HTC Vive (HTC Corporation) VR headset has built-in sensors that allow for position and orientation tracking, making it a potentially e?ective tool for balance assessments. Methods: Participants in this study were asked to stand upright on a force plate (NeuroCom; Natus Medical Incorporated) while wearing the HTC Vive. Position data were collected from the headset and force plate simultaneously as participants experienced a custom-built VR environment that covered their entire field of view. The intraclass correlation coefficient (ICC) was used to examine the test-retest reliability of the postural control variables, which included the normalized path length, root mean square (RMS), and peak-to-peak (P2P) value. These were computed from the VR position output data and the center of pressure (COP) data from the force plate. Linear regression was used to investigate the correlations between the VR and force plate measurements. Results: Our results showed that the test-retest reliability of the RMS and P2P value of VR headset outputs (ICC: range 0.285-0.636) was similar to that of the RMS and P2P value of COP outputs (ICC: range 0.228-0.759). The linear regression between VR and COP measures showed significant correlations in RMSs and P2P values. Conclusions: Based on our results, the VR headset has the potential to be used for postural control measurements. However, the further development of software and testing protocols for balance assessments is needed. UR - https://rehab.jmir.org/2021/4/e24950 UR - http://dx.doi.org/10.2196/24950 UR - http://www.ncbi.nlm.nih.gov/pubmed/34779789 ID - info:doi/10.2196/24950 ER - TY - JOUR AU - Glöggler, Michael AU - Ammenwerth, Elske PY - 2021/10/5 TI - Improvement and Evaluation of the TOPCOP Taxonomy of Patient Portals: Taxonomy-Evaluation-Delphi (TED) Approach JO - J Med Internet Res SP - e30701 VL - 23 IS - 10 KW - taxonomy KW - classification system KW - patient portal KW - EHR portal KW - online EHR access KW - evaluation KW - Delphi study KW - electronic health records KW - digital health KW - health information KW - information management KW - user perspectives N2 - Background: Patient portals have been introduced in many countries over the last 10 years, but many health information managers still feel they have too little knowledge of patient portals. A taxonomy can help them to better compare and select portals. This has led us to develop the TOPCOP taxonomy for classifying and comparing patient portals. However, the taxonomy has not been evaluated by users. Objective: This study aimed to evaluate the taxonomy?s usefulness to support health information managers in comparing, classifying, defining a requirement profile for, and selecting patient portals and to improve the taxonomy where needed. Methods: We used a modified Delphi approach. We sampled a heterogeneous panel of 13 health information managers from 3 countries using the criterion sampling strategy. We conducted 4 anonymous survey rounds with qualitative and quantitative questions. In round 1, the panelists assessed the appropriateness of each dimension, and we collected new ideas to improve the dimensions. In rounds 2 and 3, the panelists iteratively evaluated the taxonomy that was revised based on round 1. In round 4, the panelists assessed the need for a taxonomy and the appropriateness of patient engagement as a distinguishing concept. Then, they compared 2 real portals with the final taxonomy and evaluated its usefulness for comparing portals, creating an initial requirement profile, and selecting patient portals. To determine group consensus, we applied the RAND/UCLA Appropriateness Method. Results: The final taxonomy consists of 25 dimensions with 65 characteristics. Five new dimensions were added to the original taxonomy, with 8 characteristics added to already existing dimensions. Group consensus was achieved on the need for such a taxonomy to compare portals, on patient engagement as an appropriate distinguishing concept, and on the comprehensibility of the taxonomy?s form. Further, consensus was achieved on the taxonomy?s usefulness for classifying and comparing portals, assisting users in better understanding portals, creating a requirement profile, and selecting portals. This allowed us to test the usefulness of the final taxonomy with the intended users. Conclusions: The TOPCOP taxonomy aims to support health information managers in comparing and selecting patient portals. By providing a standardized terminology to describe various aspects of patient portals independent of clinical setting or country, the taxonomy will also be useful for advancing research and evaluation of patient portals. UR - https://www.jmir.org/2021/10/e30701 UR - http://dx.doi.org/10.2196/30701 UR - http://www.ncbi.nlm.nih.gov/pubmed/34403354 ID - info:doi/10.2196/30701 ER - TY - JOUR AU - Lenaerts, Gerlinde AU - Bekkering, E. Geertruida AU - Goossens, Martine AU - De Coninck, Leen AU - Delvaux, Nicolas AU - Cordyn, Sam AU - Adriaenssens, Jef AU - Aertgeerts, Bert AU - Vankrunkelsven, Patrik PY - 2021/10/5 TI - A Tool to Assess the Trustworthiness of Evidence-Based Point-of-Care Information for Health Care Professionals (CAPOCI): Design and Validation Study JO - J Med Internet Res SP - e27174 VL - 23 IS - 10 KW - evidence-based medicine KW - evidence-based practice KW - point-of-care systems KW - health care quality KW - information science KW - practice guidelines as a topic N2 - Background: User-friendly information at the point of care for health care professionals should be well structured, rapidly accessible, comprehensive, and trustworthy. The reliability of information and the associated methodological process must be clear. There is no standard tool to evaluate the trustworthiness of such point-of-care (POC) information. Objective: We aim to develop and validate a new tool for assessment of trustworthiness of evidence-based POC resources to enhance the quality of POC resources and facilitate evidence-based practice. Methods: We designed the Critical Appraisal of Point-of-Care Information (CAPOCI) tool based on the criteria important for assessment of trustworthiness of POC information, reported in a previously published review. A group of health care professionals and methodologists (the authors of this paper) defined criteria for the CAPOCI tool in an iterative process of discussion and pilot testing until consensus was reached. In the next step, all criteria were subject to content validation with a Delphi study. We invited an international panel of 10 experts to rate their agreement with the relevance and wording of the criteria and to give feedback. Consensus was reached when 70% of the experts agreed. When no consensus was reached, we reformulated the criteria based on the experts? comments for a next round of the Delphi study. This process was repeated until consensus was reached for each criterion. In a last step, the interrater reliability of the CAPOCI tool was calculated with a 2-tailed Kendall tau correlation coefficient to quantify the agreement between 2 users who piloted the CAPOCI tool on 5 POC resources. Two scoring systems were tested: a 3-point ordinal scale and a 7-point Likert scale. Results: After validation, the CAPOCI tool was designed with 11 criteria that focused on methodological quality and author-related information. The criteria assess authorship, literature search, use of preappraised evidence, critical appraisal of evidence, expert opinions, peer review, timeliness and updating, conflict of interest, and commercial support. Interrater agreement showed substantial agreement between 2 users for scoring with the 3-point ordinal scale (?=.621, P<.01) and scoring with the 7-point Likert scale (?=.677, P<.01). Conclusions: The CAPOCI tool may support validation teams in the assessment of trustworthiness of POC resources. It may also provide guidance for producers of POC resources. UR - https://www.jmir.org/2021/10/e27174 UR - http://dx.doi.org/10.2196/27174 UR - http://www.ncbi.nlm.nih.gov/pubmed/34609314 ID - info:doi/10.2196/27174 ER - TY - JOUR AU - Snow-Hill, L. Nyssa AU - Donenberg, Geri AU - Feil, G. Edward AU - Smith, R. David AU - Floyd, R. Brenikki AU - Leve, Craig PY - 2021/9/30 TI - A Technology-Based Training Tool for a Health Promotion and Sex Education Program for Justice-Involved Youth: Development and Usability Study JO - JMIR Form Res SP - e31185 VL - 5 IS - 9 KW - health education KW - sexual behavior KW - juvenile delinquency KW - feasibility studies KW - evidence-based practice KW - adolescent health services KW - inservice training KW - implementation science KW - organizational innovation KW - technology KW - risk reduction behavior KW - mobile phone KW - health technology KW - health promotion KW - sexual health N2 - Background: Justice-involved youth are especially vulnerable to mental health distress, substance misuse, and risky sexual activity, amplifying the need for evidence-based programs (EBPs). Yet, uptake of EBPs in the justice system is challenging because staff training is costly in time and effort. Hence, justice-involved youth experience increasing health disparities despite the availability of EBPs. Objective: To counter these challenges, this study develops and pilot-tests a prototype of a technology-based training tool that teaches juvenile justice staff to deliver a uniquely tailored EBP for justice-involved youth?PHAT (Preventing HIV/AIDS Among Teens) Life. PHAT Life is a comprehensive sex education, mental health, and substance use EBP collaboratively designed and tested with guidance from key stakeholders and community members. The training tool addresses implementation barriers that impede uptake and sustainment of EBPs, including staff training and support and implementation costs. Methods: Staff (n=11) from two juvenile justice settings pilot-tested the technology-based training tool, which included five modules. Participants completed measures of HIV and sexually transmitted infection (STI) knowledge, sex education confidence, and implementation outcomes such as training satisfaction, adoption, implementation, acceptability, appropriateness, and sustainability. PHAT Life trainers assessed fidelity through two activity role plays participants submitted upon completing the training modules. Results: Participants demonstrated increases in HIV and STI knowledge (t10=3.07; P=.01), and were very satisfied (mean 4.42, SD 0.36) with the training tool and the PHAT Life curriculum. They believed that the training tool and curriculum could be adopted, implemented, and sustained within their settings as an appropriate and acceptable intervention and training. Conclusions: Overall, the results from this pilot test demonstrate feasibility and support continuing efforts toward completing the training tool and evaluating it within a fully powered randomized controlled trial. Ultimately, this study will provide a scalable option for disseminating an EBP and offers a more cost-effective and sustainable way to train staff in an EBP. UR - https://formative.jmir.org/2021/9/e31185 UR - http://dx.doi.org/10.2196/31185 UR - http://www.ncbi.nlm.nih.gov/pubmed/34591028 ID - info:doi/10.2196/31185 ER - TY - JOUR AU - Kim, Bokyung AU - Hong, Seoyeon AU - Kim, Sungwook PY - 2021/9/29 TI - Introducing an Integrated Model of Adults? Wearable Activity Tracker Use and Obesity Information?Seeking Behaviors From a National Quota Sample Survey JO - JMIR Form Res SP - e23237 VL - 5 IS - 9 KW - wearable activity tracker KW - wearable health technology KW - obesity KW - health belief KW - health belief model KW - Technology Acceptance Model KW - online information seeking N2 - Background: Research from multiple perspectives to investigate adults? use of wearable activity-tracking devices is limited. We offer a multiperspective model and provide empirical evidence of what leads to frequent usage of wearable health technologies from a large, nationally representative survey sample. Objective: This study aims to explore factors affecting the use of wearable activity-tracking devices among health consumers from the perspectives of individual health beliefs (perceived severity, perceived susceptibility, perceived benefits, and self-efficacy) and information-seeking behaviors. Methods: Our Integrated Model of Wearable Activity Tracker (IMWAT) use and proposed hypotheses were validated and tested with data collected from a telephone survey with a national quota sample. The data were analyzed using a variety of statistical techniques, including structural equation analysis. Results: The sample comprised 2006 participants. Our results showed that the perceived benefits of physical activity, perceived susceptibility, and self-efficacy toward obesity were significant predictors of information-seeking behaviors, which, in turn, mediated their effects on the use of wearable activity trackers. Perceptions of obesity severity directly promoted wearable device usage. Conclusions: This study provided a new and powerful theoretical model that combined the health beliefs and information-seeking behaviors behind the use of wearable activity trackers in the adult population. The findings provide meaningful implications for developers and designers of wearable health technology products and will assist health informatics practitioners and obesity prevention communicators. UR - https://formative.jmir.org/2021/9/e23237 UR - http://dx.doi.org/10.2196/23237 UR - http://www.ncbi.nlm.nih.gov/pubmed/34586076 ID - info:doi/10.2196/23237 ER - TY - JOUR AU - An, Jisun AU - Kwak, Haewoon AU - Qureshi, M. Hanya AU - Weber, Ingmar PY - 2021/9/24 TI - Precision Public Health Campaign: Delivering Persuasive Messages to Relevant Segments Through Targeted Advertisements on Social Media JO - JMIR Form Res SP - e22313 VL - 5 IS - 9 KW - precision public health KW - tailored health communication KW - social media advertising KW - Facebook advertising KW - public health campaigns KW - effectiveness of campaigns KW - public health KW - advertising UR - https://formative.jmir.org/2021/9/e22313 UR - http://dx.doi.org/10.2196/22313 UR - http://www.ncbi.nlm.nih.gov/pubmed/34559055 ID - info:doi/10.2196/22313 ER - TY - JOUR AU - Schreiweis, Björn AU - Brandner, Antje AU - Bergh, Björn PY - 2021/9/21 TI - Applicability of Different Electronic Record Types for Use in Patient Recruitment Support Systems: Comparative Analysis JO - JMIR Form Res SP - e13790 VL - 5 IS - 9 KW - clinical trials KW - patient recruitment support system KW - PRSS KW - electronic medical record KW - EMR KW - electronic health record KW - EHR KW - personal health record KW - PHR KW - personal enterprise health record KW - PEHR KW - clinical trial recruitment support system KW - CTRSS. N2 - Background: Clinical trials constitute an important pillar in medical research. It is beneficial to support recruitment for clinical trials using software tools, so-called patient recruitment support systems; however, such information technology systems have not been frequently used to date. Because medical information systems' underlying data collection methods strongly influence the benefits of implementing patient recruitment support systems, we investigated patient recruitment support system requirements and corresponding electronic record types such as electronic medical record, electronic health record, electronic medical case record, personal health record, and personal cross-enterprise health record. Objective: The aim of this study was to (1) define requirements for successful patient recruitment support system deployment and (2) differentiate and compare patient recruitment support system?relevant properties of different electronic record types. Methods: In a previous study, we gathered requirements for patient recruitment support systems from literature and unstructured interviews with stakeholders (15 patients, 3 physicians, 5 data privacy experts, 4 researchers, and 5 staff members of hospital administration). For this investigation, the requirements were amended and categorized based on input from scientific sessions. Based on literature with a focus on patient recruitment support system?relevant properties, different electronic record types (electronic medical record, electronic health record, electronic medical case record, personal health record and personal cross-enterprise health record) were described in detail. We also evaluated which patient recruitment support system requirements can be achieved for each electronic record type. Results: Patient recruitment support system requirements (n=16) were grouped into 4 categories (consent management, patient recruitment management, trial management, and general requirements). All 16 requirements could be partially met by at least 1 type of electronic record. Only 1 requirement was fully met by all 5 types. According to our analysis, personal cross-enterprise health records fulfill most requirements for patient recruitment support systems. They demonstrate advantages especially in 2 domains (1) supporting patient empowerment and (2) granting access to the complete medical history of patients. Conclusions: In combination with patient recruitment support systems, personal cross-enterprise health records prove superior to other electronic record types, and therefore, this integration approach should be further investigated. UR - https://formative.jmir.org/2021/9/e13790 UR - http://dx.doi.org/10.2196/13790 UR - http://www.ncbi.nlm.nih.gov/pubmed/34546175 ID - info:doi/10.2196/13790 ER - TY - JOUR AU - Naeim, Arash AU - Dry, Sarah AU - Elashoff, David AU - Xie, Zhuoer AU - Petruse, Antonia AU - Magyar, Clara AU - Johansen, Liliana AU - Werre, Gabriela AU - Lajonchere, Clara AU - Wenger, Neil PY - 2021/9/8 TI - Electronic Video Consent to Power Precision Health Research: A Pilot Cohort Study JO - JMIR Form Res SP - e29123 VL - 5 IS - 9 KW - biobanking KW - precision medicine KW - electronic consent KW - privacy KW - pilot study KW - video KW - consent KW - precision KW - innovation KW - efficient KW - cancer KW - education KW - barrier KW - engagement KW - participation N2 - Background: Developing innovative, efficient, and institutionally scalable biospecimen consent for remnant tissue that meets the National Institutes of Health consent guidelines for genomic and molecular analysis is essential for precision medicine efforts in cancer. Objective: This study aims to pilot-test an electronic video consent that individuals could complete largely on their own. Methods: The University of California, Los Angeles developed a video consenting approach designed to be comprehensive yet fast (around 5 minutes) for providing universal consent for remnant biospecimen collection for research. The approach was piloted in 175 patients who were coming in for routine services in laboratory medicine, radiology, oncology, and hospital admissions. The pilot yielded 164 completed postconsent surveys. The pilot assessed the usefulness, ease, and trustworthiness of the video consent. In addition, we explored drivers for opting in or opting out. Results: The pilot demonstrated that the electronic video consent was well received by patients, with high scores for usefulness, ease, and trustworthiness even among patients that opted out of participation. The revised more animated video pilot test in phase 2 was better received in terms of ease of use (P=.005) and the ability to understand the information (P<.001). There were significant differences between those who opted in and opted out in their beliefs concerning the usefulness of tissue, trusting researchers, the importance of contributing to science, and privacy risk (P<.001). The results showed that ?I trust researchers to use leftover biological specimens to promote the public?s health? and ?Sharing a biological sample for research is safe because of the privacy protections in place? discriminated opt-in statuses were the strongest predictors (both areas under the curve were 0.88). Privacy concerns seemed universal in individuals who opted out. Conclusions: Efforts to better educate the community may be needed to help overcome some of the barriers in engaging individuals to participate in precision health initiatives. UR - https://formative.jmir.org/2021/9/e29123 UR - http://dx.doi.org/10.2196/29123 UR - http://www.ncbi.nlm.nih.gov/pubmed/34313247 ID - info:doi/10.2196/29123 ER - TY - JOUR AU - Ribanszki, Robert AU - Saez Fonseca, Andres Jose AU - Barnby, Matthew Joseph AU - Jano, Kimberly AU - Osmani, Fatima AU - Almasi, Soma AU - Tsakanikos, Elias PY - 2021/8/27 TI - Preferences for Digital Smartphone Mental Health Apps Among Adolescents: Qualitative Interview Study JO - JMIR Form Res SP - e14004 VL - 5 IS - 8 KW - qualitative KW - adolescents KW - mental health KW - digital smartphone app KW - digital mental health KW - mobile phone N2 - Background: Mental health digital apps hold promise for providing scalable solutions to individual self-care, education, and illness prevention. However, a problem with these apps is that they lack engaging user interfaces and experiences and thus potentially result in high attrition. Although guidelines for new digital interventions for adults have begun to examine engagement, there is a paucity of evidence on how to best address digital interventions for adolescents. As adolescence is a period of transition, during which the onset of many potentially lifelong mental health conditions frequently occurs, understanding how best to engage this population is crucial. Objective: The study aims to detect potential barriers to engagement and to gather feedback on the current elements of app design regarding user experience, user interface, and content. Methods: This study used a qualitative design. A sample of 14 adolescents was asked to use the app for 1 week and was interviewed using a semistructured interview schedule. The interviews were transcribed and analyzed using thematic analysis. Results: Overall, 13 participants completed the interviews. The authors developed 6 main themes and 20 subthemes based on the data that influenced engagement with and the perceived usefulness of the app. Our main themes were timing, stigma, perception, congruity, usefulness, and user experience. Conclusions: In line with previous research, we suggest how these aspects of app development should be considered for future apps that aim to prevent and manage mental health conditions. UR - https://formative.jmir.org/2021/8/e14004 UR - http://dx.doi.org/10.2196/14004 UR - http://www.ncbi.nlm.nih.gov/pubmed/34128814 ID - info:doi/10.2196/14004 ER - TY - JOUR AU - Noriega, Alejandro AU - Meizner, Daniela AU - Camacho, Dalia AU - Enciso, Jennifer AU - Quiroz-Mercado, Hugo AU - Morales-Canton, Virgilio AU - Almaatouq, Abdullah AU - Pentland, Alex PY - 2021/8/26 TI - Screening Diabetic Retinopathy Using an Automated Retinal Image Analysis System in Independent and Assistive Use Cases in Mexico: Randomized Controlled Trial JO - JMIR Form Res SP - e25290 VL - 5 IS - 8 KW - diabetic retinopathy KW - automated diagnosis KW - retina KW - fundus image analysis N2 - Background: The automated screening of patients at risk of developing diabetic retinopathy represents an opportunity to improve their midterm outcome and lower the public expenditure associated with direct and indirect costs of common sight-threatening complications of diabetes. Objective: This study aimed to develop and evaluate the performance of an automated deep learning?based system to classify retinal fundus images as referable and nonreferable diabetic retinopathy cases, from international and Mexican patients. In particular, we aimed to evaluate the performance of the automated retina image analysis (ARIA) system under an independent scheme (ie, only ARIA screening) and 2 assistive schemes (ie, hybrid ARIA plus ophthalmologist screening), using a web-based platform for remote image analysis to determine and compare the sensibility and specificity of the 3 schemes. Methods: A randomized controlled experiment was performed where 17 ophthalmologists were asked to classify a series of retinal fundus images under 3 different conditions. The conditions were to (1) screen the fundus image by themselves (solo); (2) screen the fundus image after exposure to the retina image classification of the ARIA system (ARIA answer); and (3) screen the fundus image after exposure to the classification of the ARIA system, as well as its level of confidence and an attention map highlighting the most important areas of interest in the image according to the ARIA system (ARIA explanation). The ophthalmologists? classification in each condition and the result from the ARIA system were compared against a gold standard generated by consulting and aggregating the opinion of 3 retina specialists for each fundus image. Results: The ARIA system was able to classify referable vs nonreferable cases with an area under the receiver operating characteristic curve of 98%, a sensitivity of 95.1%, and a specificity of 91.5% for international patient cases. There was an area under the receiver operating characteristic curve of 98.3%, a sensitivity of 95.2%, and a specificity of 90% for Mexican patient cases. The ARIA system performance was more successful than the average performance of the 17 ophthalmologists enrolled in the study. Additionally, the results suggest that the ARIA system can be useful as an assistive tool, as sensitivity was significantly higher in the experimental condition where ophthalmologists were exposed to the ARIA system?s answer prior to their own classification (93.3%), compared with the sensitivity of the condition where participants assessed the images independently (87.3%; P=.05). Conclusions: These results demonstrate that both independent and assistive use cases of the ARIA system present, for Latin American countries such as Mexico, a substantial opportunity toward expanding the monitoring capacity for the early detection of diabetes-related blindness. UR - https://formative.jmir.org/2021/8/e25290 UR - http://dx.doi.org/10.2196/25290 UR - http://www.ncbi.nlm.nih.gov/pubmed/34435963 ID - info:doi/10.2196/25290 ER - TY - JOUR AU - Manabe, Masae AU - Liew, Kongmeng AU - Yada, Shuntaro AU - Wakamiya, Shoko AU - Aramaki, Eiji PY - 2021/8/12 TI - Estimation of Psychological Distress in Japanese Youth Through Narrative Writing: Text-Based Stylometric and Sentiment Analyses JO - JMIR Form Res SP - e29500 VL - 5 IS - 8 KW - psychological distress KW - youth KW - narratives KW - natural language processing KW - Japan KW - mental health KW - stress KW - distress KW - young adult KW - teenager KW - sentiment N2 - Background: Internalizing mental illnesses associated with psychological distress are often underdetected. Text-based detection using natural language processing (NLP) methods is increasingly being used to complement conventional detection efforts. However, these approaches often rely on self-disclosure through autobiographical narratives that may not always be possible, especially in the context of the collectivistic Japanese culture. Objective: We propose the use of narrative writing as an alternative resource for mental illness detection in youth. Accordingly, in this study, we investigated the textual characteristics of narratives written by youth with psychological distress; our research focuses on the detection of psychopathological tendencies in written imaginative narratives. Methods: Using NLP tools such as stylometric measures and lexicon-based sentiment analysis, we examined short narratives from 52 Japanese youth (mean age 19.8 years, SD 3.1) obtained through crowdsourcing. Participants wrote a short narrative introduction to an imagined story before completing a questionnaire to quantify their tendencies toward psychological distress. Based on this score, participants were categorized into higher distress and lower distress groups. The written narratives were then analyzed using NLP tools and examined for between-group differences. Although outside the scope of this study, we also carried out a supplementary analysis of narratives written by adults using the same procedure. Results: Youth demonstrating higher tendencies toward psychological distress used significantly more positive (happiness-related) words, revealing differences in valence of the narrative content. No other significant differences were observed between the high and low distress groups. Conclusions: Youth with tendencies toward mental illness were found to write more positive stories that contained more happiness-related terms. These results may potentially have widespread implications on psychological distress screening on online platforms, particularly in cultures such as Japan that are not accustomed to self-disclosure. Although the mechanisms that we propose in explaining our results are speculative, we believe that this interpretation paves the way for future research in online surveillance and detection efforts. UR - https://formative.jmir.org/2021/8/e29500 UR - http://dx.doi.org/10.2196/29500 UR - http://www.ncbi.nlm.nih.gov/pubmed/34387556 ID - info:doi/10.2196/29500 ER - TY - JOUR AU - Oxholm, Christina AU - Christensen, Soendergaard Anne-Marie AU - Christiansen, Regina AU - Wiil, Kock Uffe AU - Nielsen, Søgaard Anette PY - 2021/8/9 TI - Attitudes of Patients and Health Professionals Regarding Screening Algorithms: Qualitative Study JO - JMIR Form Res SP - e17971 VL - 5 IS - 8 KW - screening KW - algorithms KW - alcohol KW - qualitative study KW - attitudes KW - opinions KW - patients KW - health professionals N2 - Background: As a preamble to an attempt to develop a tool that can aid health professionals at hospitals in identifying whether the patient may have an alcohol abuse problem, this study investigates opinions and attitudes among both health professionals and patients about using patient data from electronic health records (EHRs) in an algorithm screening for alcohol problems. Objective: The aim of this study was to investigate the attitudes and opinions of patients and health professionals at hospitals regarding the use of previously collected data in developing and implementing an algorithmic helping tool in EHR for screening inexpedient alcohol habits; in addition, the study aims to analyze how patients would feel about asking and being asked about alcohol by staff, based on a notification in the EHR from such a tool. Methods: Using semistructured interviews, we interviewed 9 health professionals and 5 patients to explore their opinions and attitudes about an algorithm-based helping tool and about asking and being asked about alcohol usage when being given a reminder from this type of tool. The data were analyzed using an ad hoc method consistent with a close reading and meaning condensing. Results: The health professionals were both positive and negative about a helping tool grounded in algorithms. They were optimistic about the potential of such a tool to save some time by providing a quick overview if it was easy to use but, on the negative side, noted that this type of helping tool might take away the professionals? instinct. The patients were overall positive about the helping tool, stating that they would find this tool beneficial for preventive care. Some of the patients expressed concerns that the information provided by the tool could be misused. Conclusions: When developing and implementing an algorithmic helping tool, the following aspects should be considered: (1) making the helping tool as transparent in its recommendations as possible, avoiding black boxing, and ensuring room for professional discretion in clinical decision making; and (2) including and taking into account the attitudes and opinions of patients and health professionals in the design and development process of such an algorithmic helping tool. UR - https://formative.jmir.org/2021/8/e17971 UR - http://dx.doi.org/10.2196/17971 UR - http://www.ncbi.nlm.nih.gov/pubmed/34383666 ID - info:doi/10.2196/17971 ER - TY - JOUR AU - Davidson, C. Jennifer AU - Karadzhov, Dimitar AU - Wilson, Graham PY - 2021/7/29 TI - Practitioners? and Policymakers? Successes, Challenges, Innovations, and Learning in Promoting Children?s Well-being During COVID-19: Protocol for a Multinational Smartphone App Survey JO - JMIR Res Protoc SP - e31013 VL - 10 IS - 7 KW - mobile phones KW - smartphone app KW - qualitative KW - mixed method KW - international KW - survey KW - service providers KW - policy KW - practice KW - children?s rights KW - well-being KW - COVID-19 KW - pandemic KW - app KW - mHealth KW - children N2 - Background: The advent of COVID-19 abruptly thrust the health and safety of children and families into greater risk around the world. As regional and local governments, nongovernmental organizations, communities, families, and children grapple with the immediate public health impact of COVID-19, the rights and well-being of children, especially those who are already marginalized, have been overlooked. Those working with children have likely encountered unprecedented challenges and responded in innovative ways in efforts to address the needs and rights of all children. Objective: This paper presents a protocol for a large-scale, multinational study using a new smartphone app to capture the real-time experiences and perspectives of practitioners and policymakers supporting children and families during the COVID-19 pandemic around the globe in relation to a children?s human rights 4P framework of protection, provision, prevention, and participation. Methods: This protocol describes a mixed methods survey utilizing a custom-built iOS and Android smartphone app called the COVID 4P Log for Children?s Wellbeing, which was developed in close consultation with 17 international key partner organizations. Practitioners and policymakers working with and for children?s well-being across 29 countries and 5 continents were invited to download the app and respond to questions over the course of 8 weeks. The anticipated large amount of qualitative and quantitative response data will be analyzed using content analysis, descriptive statistics, and word frequencies. Results: Formal data collection took place from October 2020 until March 2021. Data analysis was completed in July 2021. Conclusions: The findings will directly inform the understanding of the ways in which COVID-19 has impacted practitioners?, managers?, and policymakers? efforts to support children?s well-being in their practices, services, and policies, respectively. Innovative and ambitious in its scope and use of smartphone technology, this project also aims to inform and inspire future multinational research using app-based methodologies?the demand for which is likely to continue to dramatically rise in the COVID-19 era. Mitigating the risks of longitudinal remote data collection will help maximize the acceptability of the app, respondents? sustained engagement, and data quality. International Registered Report Identifier (IRRID): DERR1-10.2196/31013 UR - https://www.researchprotocols.org/2021/7/e31013 UR - http://dx.doi.org/10.2196/31013 UR - http://www.ncbi.nlm.nih.gov/pubmed/34323850 ID - info:doi/10.2196/31013 ER - TY - JOUR AU - Canales, Lea AU - Menke, Sebastian AU - Marchesseau, Stephanie AU - D?Agostino, Ariel AU - del Rio-Bermudez, Carlos AU - Taberna, Miren AU - Tello, Jorge PY - 2021/7/23 TI - Assessing the Performance of Clinical Natural Language Processing Systems: Development of an Evaluation Methodology JO - JMIR Med Inform SP - e20492 VL - 9 IS - 7 KW - natural language processing KW - clinical natural language processing KW - electronic health records KW - gold standard KW - reference standard KW - sample size N2 - Background: Clinical natural language processing (cNLP) systems are of crucial importance due to their increasing capability in extracting clinically important information from free text contained in electronic health records (EHRs). The conversion of a nonstructured representation of a patient?s clinical history into a structured format enables medical doctors to generate clinical knowledge at a level that was not possible before. Finally, the interpretation of the insights gained provided by cNLP systems has a great potential in driving decisions about clinical practice. However, carrying out robust evaluations of those cNLP systems is a complex task that is hindered by a lack of standard guidance on how to systematically approach them. Objective: Our objective was to offer natural language processing (NLP) experts a methodology for the evaluation of cNLP systems to assist them in carrying out this task. By following the proposed phases, the robustness and representativeness of the performance metrics of their own cNLP systems can be assured. Methods: The proposed evaluation methodology comprised five phases: (1) the definition of the target population, (2) the statistical document collection, (3) the design of the annotation guidelines and annotation project, (4) the external annotations, and (5) the cNLP system performance evaluation. We presented the application of all phases to evaluate the performance of a cNLP system called ?EHRead Technology? (developed by Savana, an international medical company), applied in a study on patients with asthma. As part of the evaluation methodology, we introduced the Sample Size Calculator for Evaluations (SLiCE), a software tool that calculates the number of documents needed to achieve a statistically useful and resourceful gold standard. Results: The application of the proposed evaluation methodology on a real use-case study of patients with asthma revealed the benefit of the different phases for cNLP system evaluations. By using SLiCE to adjust the number of documents needed, a meaningful and resourceful gold standard was created. In the presented use-case, using as little as 519 EHRs, it was possible to evaluate the performance of the cNLP system and obtain performance metrics for the primary variable within the expected CIs. Conclusions: We showed that our evaluation methodology can offer guidance to NLP experts on how to approach the evaluation of their cNLP systems. By following the five phases, NLP experts can assure the robustness of their evaluation and avoid unnecessary investment of human and financial resources. Besides the theoretical guidance, we offer SLiCE as an easy-to-use, open-source Python library. UR - https://medinform.jmir.org/2021/7/e20492 UR - http://dx.doi.org/10.2196/20492 UR - http://www.ncbi.nlm.nih.gov/pubmed/34297002 ID - info:doi/10.2196/20492 ER - TY - JOUR AU - Pratt-Chapman, Mandi AU - Moses, Jenna AU - Arem, Hannah PY - 2021/7/16 TI - Strategies for the Identification and Prevention of Survey Fraud: Data Analysis of a Web-Based Survey JO - JMIR Cancer SP - e30730 VL - 7 IS - 3 KW - cancer survivors KW - pandemic KW - COVID-19 KW - fraudulent responses KW - survey KW - research methods KW - cancer patients KW - fraud KW - CAPTCHA KW - data integrity KW - online surveys N2 - Background: To assess the impact of COVID-19 on cancer survivors, we fielded a survey promoted via email and social media in winter 2020. Examination of the data showed suspicious patterns that warranted serious review. Objective: The aim of this paper is to review the methods used to identify and prevent fraudulent survey responses. Methods: As precautions, we included a Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA), a hidden question, and instructions for respondents to type a specific word. To identify likely fraudulent data, we defined a priori indicators that warranted elimination or suspicion. If a survey contained two or more suspicious indicators, the survey was eliminated. We examined differences between the retained and eliminated data sets. Results: Of the total responses (N=1977), nearly three-fourths (n=1408) were dropped and one-fourth (n=569) were retained after data quality checking. Comparisons of the two data sets showed statistically significant differences across almost all demographic characteristics. Conclusions: Numerous precautions beyond the inclusion of a CAPTCHA are needed when fielding web-based surveys, particularly if a financial incentive is offered. UR - https://cancer.jmir.org/2021/3/e30730 UR - http://dx.doi.org/10.2196/30730 UR - http://www.ncbi.nlm.nih.gov/pubmed/34269685 ID - info:doi/10.2196/30730 ER - TY - JOUR AU - Chen, Ciao-Sin AU - Kim, Judith AU - Garg, Noemi AU - Guntupalli, Harsha AU - Jagsi, Reshma AU - Griggs, J. Jennifer AU - Sabel, Michael AU - Dorsch, P. Michael AU - Callaghan, C. Brian AU - Hertz, L. Daniel PY - 2021/7/5 TI - Chemotherapy-Induced Peripheral Neuropathy Detection via a Smartphone App: Cross-sectional Pilot Study JO - JMIR Mhealth Uhealth SP - e27502 VL - 9 IS - 7 KW - chemotherapy-induced peripheral neuropathy KW - smartphone KW - mobile health KW - gait KW - balance KW - 9-Hole Peg Test N2 - Background: Severe chemotherapy-induced peripheral neuropathy (CIPN) can cause long-term dysfunction of the hands and feet, interfere with activities of daily living, and diminish the quality of life. Monitoring to identify CIPN and adjust treatment before it progressing to a life-altering severity relies on patients self-reporting subjective symptoms to their clinical team. Objective assessment is not a standard component of CIPN monitoring due to the requirement for specially trained health care professionals and equipment. Smartphone apps have the potential to conveniently collect both subjective and objective CIPN data directly from patients, which could improve CIPN monitoring. Objective: The objective of this cross-sectional pilot study was to assess the feasibility of functional CIPN assessment via a smartphone app in patients with cancer that have received neurotoxic chemotherapy. Methods: A total of 26 patients who had completed neurotoxic chemotherapy were enrolled and classified as CIPN cases (n=17) or controls (n=9) based on self-report symptoms. All participants completed CIPN assessments within the NeuroDetect app a single time, including patient-reported surveys (CIPN20 [European Organization for Research and Treatment of Cancer Quality of Life Questionnaire for Chemotherapy-induced Peripheral Neuropathy 20-item scale] and PRO-CTCAE [Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events]) and functional assessments (Gait and Balance and 9-Hole Peg Test). Functional assessment data were decomposed into features. The primary analysis was done to identify features indicative of the difference between CIPN cases and controls using partial least squares analyses. Exploratory analyses were performed to test if any features were associated with specific symptom subtypes or patient-reported survey scores. Patient interviews were also conducted to understand the challenges they experienced with the app. Results: Comparisons between CIPN cases and controls indicate that CIPN cases had shorter step length (P=.007), unique swaying acceleration patterns during a walking task, and shorter hand moving distance in the dominant hands during a manual dexterity task (variable importance in projection scores ?2). Exploratory analyses showed similar signatures associated with symptoms subtypes, CIPN20, and PRO-CTCAE. The interview results showed that some patients had difficulties due to technical issues, which indicated a need for additional training or oversight during the initial app download. Conclusions: Our results supported the feasibility of remote CIPN assessment via a smartphone app and suggested that functional assessments may indicate CIPN manifestations in the hands and feet. Additional work is needed to determine which functional assessments are most indicative of CIPN and could be used for CIPN monitoring within clinical care. UR - https://mhealth.jmir.org/2021/7/e27502 UR - http://dx.doi.org/10.2196/27502 UR - http://www.ncbi.nlm.nih.gov/pubmed/36260403 ID - info:doi/10.2196/27502 ER - TY - JOUR AU - Domingos, Célia AU - Costa, Soares Patrício AU - Santos, Correia Nadine AU - Pêgo, Miguel José PY - 2021/6/29 TI - European Portuguese Version of the User Satisfaction Evaluation Questionnaire (USEQ): Transcultural Adaptation and Validation Study JO - JMIR Mhealth Uhealth SP - e19245 VL - 9 IS - 6 KW - satisfaction KW - usability KW - reliability KW - validity KW - seniors KW - elderly KW - technology KW - wearables N2 - Background: Wearable activity trackers have the potential to encourage users to adopt healthier lifestyles by tracking daily health information. However, usability is a critical factor in technology adoption. Older adults may be more resistant to accepting novel technologies. Understanding the difficulties that older adults face when using activity trackers may be useful for implementing strategies to promote their use. Objective: The purpose of this study was to conduct a transcultural adaptation of the User Satisfaction Evaluation Questionnaire (USEQ) into European Portuguese and validate the adapted questionnaire. Additionally, we aimed to provide information about older adults? satisfaction regarding the use of an activity tracker (Xiaomi Mi Band 2). Methods: The USEQ was translated following internationally accepted guidelines. The psychometric evaluation of the final version of the translated USEQ was assessed based on structural validity using exploratory and confirmatory factor analyses. Construct validity was examined using divergent and discriminant validity analysis, and internal consistency was evaluated using Cronbach ? and McDonald ? coefficients. Results: A total of 110 older adults completed the questionnaire. Confirmatory factor analysis supported the conceptual unidimensionality of the USEQ (?24=7.313, P=.12, comparative fit index=0.973, Tucker-Lewis index=0.931, goodness of fit index=0.977, root mean square error of approximation=0.087, standardized root mean square residual=0.038). The internal consistency showed acceptable reliability (Cronbach ?=.677, McDonald ?=0.722). Overall, 90% of the participants reported excellent satisfaction with the Xiaomi Mi Band 2. Conclusions: The findings support the use of this translated USEQ as a valid and reliable tool for measuring user satisfaction with wearable activity trackers in older adults, with psychometric properties consistent with the original version. UR - https://mhealth.jmir.org/2021/6/e19245 UR - http://dx.doi.org/10.2196/19245 UR - http://www.ncbi.nlm.nih.gov/pubmed/34185018 ID - info:doi/10.2196/19245 ER - TY - JOUR AU - Hwang, Sung Ho AU - Choi, Seong-Youl PY - 2021/6/17 TI - Development of an Android-Based Self-Report Assessment for Elderly Driving Risk (SAFE-DR) App: Mixed Methods Study JO - JMIR Mhealth Uhealth SP - e25310 VL - 9 IS - 6 KW - Android driving app KW - driving safety KW - reliability KW - self-assessment KW - validity KW - mHealth KW - driving N2 - Background: Self-report assessments for elderly drivers are used in various countries for accessible, widespread self-monitoring of driving ability in the elderly population. Likewise, in South Korea, a paper-based Self-Report Assessment for Elderly Driving Risk (SAFE-DR) has been developed. Here, we implemented the SAFE-DR through an Android app, which provides the advantages of accessibility, convenience, and provision of diverse information, and verified its reliability and validity. Objective: This study tested the validity and reliability of a mobile app-based version of a self-report assessment for elderly persons contextualized to the South Korean culture and compared it with a paper-based test. Methods: In this mixed methods study, we recruited and interviewed 567 elderly drivers (aged 65 years and older) between August 2018 and May 2019. For participants who provided consent, the app-based test was repeated after 2 weeks and an additional paper-based test (Driver 65 Plus test) was administered. Using the collected data, we analyzed the reliability and validity of the app-based SAFE-DR. The internal consistency of provisional items in each subdomain of the SAFE-DR and the test-retest stability were analyzed to examine reliability. Exploratory factor analysis was performed to examine the validity of the subdomain configuration. To verify the appropriateness of using an app-based test for older drivers possibly unfamiliar with mobile technology, the correlation between the results of the SAFE-DR app and the paper-based offline test was also analyzed. Results: In the reliability analysis, Cronbach ? for all items was 0.975 and the correlation of each item with the overall score ranged from r=0.520 to r=0.823; 4 items with low correlations were removed from each of the subdomains. In the retest after 2 weeks, the mean correlation coefficient across all items was r=0.951, showing very high reliability. Exploratory factor analysis on 40 of the 44 items established 5 subdomains: on-road (8 items), coping (16 items), cognitive functions (5 items), general conditions (8 items), and medical health (3 items). A very strong negative correlation of ?0.864 was observed between the total score for the app-based SAFE-DR and the paper-based Driver 65 Plus with decorrelation scales. The app-based test was found to be reliable. Conclusions: In this study, we developed an app-based self-report assessment tool for elderly drivers and tested its reliability and validity. This app can help elderly individuals easily assess their own driving skills. Therefore, this assessment can be used to educate drivers and for preventive screening for elderly drivers who want to renew their driver?s licenses in South Korea. In addition, the app can contribute to safe driving among elderly drivers. UR - https://mhealth.jmir.org/2021/6/e25310 UR - http://dx.doi.org/10.2196/25310 UR - http://www.ncbi.nlm.nih.gov/pubmed/33934068 ID - info:doi/10.2196/25310 ER - TY - JOUR AU - Ferreira, Ferraz Gabriel AU - Quiles, Gonçalves Marcos AU - Nazaré, Santana Tiago AU - Rezende, Oliveira Solange AU - Demarzo, Marcelo PY - 2021/6/15 TI - Automation of Article Selection Process in Systematic Reviews Through Artificial Neural Network Modeling and Machine Learning: Protocol for an Article Selection Model JO - JMIR Res Protoc SP - e26448 VL - 10 IS - 6 KW - deep learning KW - machine learning KW - systematic review KW - mindfulness N2 - Background: A systematic review can be defined as a summary of the evidence found in the literature via a systematic search in the available scientific databases. One of the steps involved is article selection, which is typically a laborious task. Machine learning and artificial intelligence can be important tools in automating this step, thus aiding researchers. Objective: The aim of this study is to create models based on an artificial neural network system to automate the article selection process in systematic reviews related to ?Mindfulness and Health Promotion.? Methods: The study will be performed using Python programming software. The system will consist of six main steps: (1) data import, (2) exclusion of duplicates, (3) exclusion of non-articles, (4) article reading and model creation using artificial neural network, (5) comparison of the models, and (6) system sharing. We will choose the 10 most relevant systematic reviews published in the fields of ?Mindfulness and Health Promotion? and ?Orthopedics? (control group) to serve as a test of the effectiveness of the article selection. Results: Data collection will begin in July 2021, with completion scheduled for December 2021, and final publication available in March 2022. Conclusions: An automated system with a modifiable sensitivity will be created to select scientific articles in systematic review that can be expanded to various fields. We will disseminate our results and models through the ?Observatory of Evidence? in public health, an open and online platform that will assist researchers in systematic reviews. International Registered Report Identifier (IRRID): PRR1-10.2196/26448 UR - https://www.researchprotocols.org/2021/6/e26448 UR - http://dx.doi.org/10.2196/26448 UR - http://www.ncbi.nlm.nih.gov/pubmed/34128820 ID - info:doi/10.2196/26448 ER - TY - JOUR AU - Williams, Hants AU - Steinberg, Sarah AU - Berzin, Robin PY - 2021/6/11 TI - The Development of a Digital Patient-Reported Outcome Measurement for Adults With Chronic Disease (The Parsley Symptom Index): Prospective Cohort Study JO - JMIR Form Res SP - e29122 VL - 5 IS - 6 KW - patient-reported outcomes KW - PROMs KW - chronic diseases KW - symptom management KW - Parsley Symptom Index KW - Review of Symptoms N2 - Background: The monitoring and management of chronic illness has always been a challenge. Patient-reported outcome measures (PROMs) can be powerful tools for monitoring symptoms and guiding treatment of chronic diseases, but the available PROM tools are either too broad or too disease specific for the needs of a primary care practice focused on longitudinal care. Objective: In this study we describe the development and preliminary validation of the Parsley Symptom Index (PSI). Methods: This prospective cohort study took place from January 5, 2018, to June 05, 2020, among a sample of 4621 adult patients at Parsley Health. After a review of literature, followed by binning and winnowing of potential items, a 45-item PROM that also served as a review of systems (ROS) was developed. The PSI was deployed and completed by patients via an online portal. Construct and face validity was performed by clinicians, tested on patients, and feasibility was measured by response rate, completion rate, and percentage of missing data. Results: The response rate for 12,175 collected PSIs was 93.72% (4331/4621) with a 100% item completion rate. A confirmatory factor analysis confirmed the model structure was satisfactory by a Comparative Fit Index of 0.943, Tucker?Lewis index of 0.938, and root mean square error of approximation of 0.028. Conclusions: A 45-item ROS-style PROM designed to capture chronic disease symptoms was developed, and preliminary validation suggests that the PSI can be deployed, completed, and helpful to both patients and clinicians. UR - https://formative.jmir.org/2021/6/e29122 UR - http://dx.doi.org/10.2196/29122 UR - http://www.ncbi.nlm.nih.gov/pubmed/33999007 ID - info:doi/10.2196/29122 ER - TY - JOUR AU - Fritz, Jessica AU - Stochl, Jan AU - Kievit, A. Rogier AU - van Harmelen, Anne-Laura AU - Wilkinson, O. Paul PY - 2021/6/8 TI - Tracking Stress, Mental Health, and Resilience Factors in Medical Students Before, During, and After a Stress-Inducing Exam Period: Protocol and Proof-of-Principle Analyses for the RESIST Cohort Study JO - JMIR Form Res SP - e20128 VL - 5 IS - 6 KW - exam stress KW - perceived stress KW - mental distress KW - student mental health KW - mental health resilience KW - protective factors KW - resilience factors N2 - Background: Knowledge of mental distress and resilience factors over the time span from before to after a stressor is important to be able to leverage the most promising resilience factors and promote mental health at the right time. To shed light on this topic, we designed the RESIST (Resilience Study) study, in which we assessed medical students before, during, and after their yearly exam period. Exam time is generally a period of notable stress among medical students, and it has been suggested that exam time triggers mental distress. Objective: In this paper, we aim to describe the study protocol and to examine whether the exam period indeed induces higher perceived stress and mental distress. We also aim to explore whether perceived stress and mental distress coevolve in response to exams. Methods: RESIST is a cohort study in which exam stress functions as a within-subject natural stress manipulation. In this paper, we outline the sample (N=451), procedure, assessed measures (including demographics, perceived stress, mental distress, 13 resilience factors, and adversity), and ethical considerations. Moreover, we conducted a series of latent growth models and bivariate latent change score models to analyze perceived stress and mental distress changes over the 3 time points. Results: We found that perceived stress and mental distress increased from the time before the exams to the exam period and decreased after the exams to a lower level than before the exams. Our findings further suggest that higher mental distress before exams increased the risk of developing more perceived stress during exams. Higher perceived stress during exams, in turn, increased the risk of experiencing a less successful (or quick) recovery of mental distress after exams. Conclusions: As expected, the exam period caused a temporary increase in perceived stress and mental distress. Therefore, the RESIST study lends itself well to exploring resilience factors in response to naturally occurring exam stress. Such knowledge will eventually help researchers to find out which resilience factors lend themselves best as prevention targets and which lend themselves best as treatment targets for the mitigation of mental health problems that are triggered or accelerated by natural exam stress. The findings from the RESIST study may therefore inform student support services, mental health services, and resilience theory. UR - https://formative.jmir.org/2021/6/e20128 UR - http://dx.doi.org/10.2196/20128 UR - http://www.ncbi.nlm.nih.gov/pubmed/34100761 ID - info:doi/10.2196/20128 ER - TY - JOUR AU - Buck, Benjamin AU - Chander, Ayesha AU - Brian, M. Rachel AU - Wang, Weichen AU - Campbell, T. Andrew AU - Ben-Zeev, Dror PY - 2021/6/3 TI - Expanding the Reach of Research: Quantitative Evaluation of a Web-Based Approach for Remote Recruitment of People Who Hear Voices JO - JMIR Form Res SP - e23118 VL - 5 IS - 6 KW - digital health KW - research procedures KW - recruitment KW - mobile phone N2 - Background: Similar to other populations with highly stigmatized medical or psychiatric conditions, people who hear voices (ie, experience auditory verbal hallucinations [AVH]) are often difficult to identify and reach for research. Technology-assisted remote research strategies reduce barriers to research recruitment; however, few studies have reported on the efficiency and effectiveness of these approaches. Objective: This study introduces and evaluates the efficacy of technology-assisted remote research designed for people who experience AVH. Methods: Our group developed an integrated, automated and human complementary web-based recruitment and enrollment apparatus that incorporated Google Ads, web-based screening, identification verification, hybrid automation, and interaction with live staff. We examined the efficacy of that apparatus by examining the number of web-based advertisement impressions (ie, number of times the web-based advertisement was viewed); clicks on that advertisement; engagement with web-based research materials; and the extent to which it succeeded in representing a broad sample of individuals with AVH, assessed through the self-reported AVH symptom severity and demographic representativeness (relative to the US population) of the sample recruited. Results: Over an 18-month period, our Google Ads advertisement was viewed 872,496 times and clicked on 11,183 times. A total amount of US $4429.25 was spent on Google Ads, resulting in 772 individuals who experience AVH providing consent to participate in an entirely remote research study (US $0.40 per click on the advertisement and US $5.73 per consented participant) after verifying their phone number, passing a competency screening questionnaire, and providing consent. These participants reported high levels of AVH frequency (666/756, 88.1% daily or more), distress (689/755, 91.3%), and functional interference (697/755, 92.4%). They also represented a broad sample of diversity that mirrored the US population demographics. Approximately one-third (264/756, 34.9%) of the participants had never received treatment for their AVH and, therefore, were unlikely to be identified via traditional clinic-based research recruitment strategies. Conclusions: Web-based procedures allow for time saving, cost-efficient, and representative recruitment of individuals with AVH and can serve as a model for future studies focusing on hard-to-reach populations. UR - https://formative.jmir.org/2021/6/e23118 UR - http://dx.doi.org/10.2196/23118 UR - http://www.ncbi.nlm.nih.gov/pubmed/34081011 ID - info:doi/10.2196/23118 ER - TY - JOUR AU - Espinosa-Gonzalez, Belen Ana AU - Neves, Luisa Ana AU - Fiorentino, Francesca AU - Prociuk, Denys AU - Husain, Laiba AU - Ramtale, Christian Sonny AU - Mi, Emma AU - Mi, Ella AU - Macartney, Jack AU - Anand, N. Sneha AU - Sherlock, Julian AU - Saravanakumar, Kavitha AU - Mayer, Erik AU - de Lusignan, Simon AU - Greenhalgh, Trisha AU - Delaney, C. Brendan PY - 2021/5/25 TI - Predicting Risk of Hospital Admission in Patients With Suspected COVID-19 in a Community Setting: Protocol for Development and Validation of a Multivariate Risk Prediction Tool JO - JMIR Res Protoc SP - e29072 VL - 10 IS - 5 KW - COVID-19 severity KW - risk prediction tool KW - early warning score KW - hospital admission KW - primary care KW - electronic health records N2 - Background: During the pandemic, remote consultations have become the norm for assessing patients with signs and symptoms of COVID-19 to decrease the risk of transmission. This has intensified the clinical uncertainty already experienced by primary care clinicians when assessing patients with suspected COVID-19 and has prompted the use of risk prediction scores, such as the National Early Warning Score (NEWS2), to assess severity and guide treatment. However, the risk prediction tools available have not been validated in a community setting and are not designed to capture the idiosyncrasies of COVID-19 infection. Objective: The objective of this study is to produce a multivariate risk prediction tool, RECAP-V1 (Remote COVID-19 Assessment in Primary Care), to support primary care clinicians in the identification of those patients with COVID-19 that are at higher risk of deterioration and facilitate the early escalation of their treatment with the aim of improving patient outcomes. Methods: The study follows a prospective cohort observational design, whereby patients presenting in primary care with signs and symptoms suggestive of COVID-19 will be followed and their data linked to hospital outcomes (hospital admission and death). Data collection will be carried out by primary care clinicians in four arms: North West London Clinical Commissioning Groups (NWL CCGs), Oxford-Royal College of General Practitioners (RCGP) Research and Surveillance Centre (RSC), Covid Clinical Assessment Service (CCAS), and South East London CCGs (Doctaly platform). The study involves the use of an electronic template that incorporates a list of items (known as RECAP-V0) thought to be associated with disease outcome according to previous qualitative work. Data collected will be linked to patient outcomes in highly secure environments. We will then use multivariate logistic regression analyses for model development and validation. Results: Recruitment of participants started in October 2020. Initially, only the NWL CCGs and RCGP RSC arms were active. As of March 24, 2021, we have recruited a combined sample of 3827 participants in these two arms. CCAS and Doctaly joined the study in February 2021, with CCAS starting the recruitment process on March 15, 2021. The first part of the analysis (RECAP-V1 model development) is planned to start in April 2021 using the first half of the NWL CCGs and RCGP RSC combined data set. Posteriorly, the model will be validated with the rest of the NWL CCGs and RCGP RSC data as well as the CCAS and Doctaly data sets. The study was approved by the Research Ethics Committee on May 27, 2020 (Integrated Research Application System number: 283024, Research Ethics Committee reference number: 20/NW/0266) and badged as National Institute of Health Research Urgent Public Health Study on October 14, 2020. Conclusions: We believe the validated RECAP-V1 early warning score will be a valuable tool for the assessment of severity in patients with suspected COVID-19 in the community, either in face-to-face or remote consultations, and will facilitate the timely escalation of treatment with the potential to improve patient outcomes. Trial Registration: ISRCTN registry ISRCTN13953727; https://www.isrctn.com/ISRCTN13953727 International Registered Report Identifier (IRRID): DERR1-10.2196/29072 UR - https://www.researchprotocols.org/2021/5/e29072 UR - http://dx.doi.org/10.2196/29072 UR - http://www.ncbi.nlm.nih.gov/pubmed/33939619 ID - info:doi/10.2196/29072 ER - TY - JOUR AU - Montazeri, Maryam AU - Multmeier, Jan AU - Novorol, Claire AU - Upadhyay, Shubhanan AU - Wicks, Paul AU - Gilbert, Stephen PY - 2021/5/21 TI - Optimization of Patient Flow in Urgent Care Centers Using a Digital Tool for Recording Patient Symptoms and History: Simulation Study JO - JMIR Form Res SP - e26402 VL - 5 IS - 5 KW - symptom assessment app KW - discrete event simulation KW - health care system KW - patient flow modeling KW - patient flow KW - simulation KW - urgent care KW - waiting times N2 - Background: Crowding can negatively affect patient and staff experience, and consequently the performance of health care facilities. Crowding can potentially be eased through streamlining and the reduction of duplication in patient history-taking through the use of a digital symptom-taking app. Objective: We simulated the introduction of a digital symptom-taking app on patient flow. We hypothesized that waiting times and crowding in an urgent care center (UCC) could be reduced, and that this would be more efficient than simply adding more staff. Methods: A discrete-event approach was used to simulate patient flow in a UCC during a 4-hour time frame. The baseline scenario was a small UCC with 2 triage nurses, 2 doctors, 1 treatment/examination nurse, and 1 discharge administrator in service. We simulated 33 scenarios with different staff numbers or different potential time savings through the app. We explored average queue length, waiting time, idle time, and staff utilization for each scenario. Results: Discrete-event simulation showed that even a few minutes saved through patient app-based self-history recording during triage could result in significantly increased efficiency. A modest estimated time saving per patient of 2.5 minutes decreased the average patient wait time for triage by 26.17%, whereas a time saving of 5 minutes led to a 54.88% reduction in patient wait times. Alternatively, adding an additional triage nurse was less efficient, as the additional staff were only required at the busiest times. Conclusions: Small time savings in the history-taking process have potential to result in substantial reductions in total patient waiting time for triage nurses, with likely effects of reduced patient anxiety, staff anxiety, and improved patient care. Patient self-history recording could be carried out at home or in the waiting room via a check-in kiosk or a portable tablet computer. This formative simulation study has potential to impact service provision and approaches to digitalization at scale. UR - https://formative.jmir.org/2021/5/e26402 UR - http://dx.doi.org/10.2196/26402 UR - http://www.ncbi.nlm.nih.gov/pubmed/34018963 ID - info:doi/10.2196/26402 ER - TY - JOUR AU - Lowe, Cabella AU - Hanuman Sing, Harry AU - Browne, Mitchell AU - Alwashmi, F. Meshari AU - Marsh, William AU - Morrissey, Dylan PY - 2021/5/18 TI - Usability Testing of a Digital Assessment Routing Tool: Protocol for an Iterative Convergent Mixed Methods Study JO - JMIR Res Protoc SP - e27205 VL - 10 IS - 5 KW - mHealth KW - mobile health KW - eHealth KW - digital health KW - digital technology KW - musculoskeletal injury KW - musculoskeletal conditions KW - triage KW - physiotherapy triage KW - usability KW - acceptability N2 - Background: Musculoskeletal conditions account for 16% of global disability, resulting in a negative effect on millions of patients and an increasing burden on health care utilization. Digital technologies that improve health care outcomes and efficiency are considered a priority; however, innovations are often inadequately developed and poorly adopted. Further, they are rarely tested with sufficient rigor in clinical trials?the gold standard for clinical proof of efficacy. We have developed a new musculoskeletal Digital Assessment Routing Tool (DART) that allows users to self-assess and be directed to the right care. DART requires usability testing in preparation for clinical trials. Objective: This study will use the iterative convergent mixed methods design to assess and mitigate all serious usability issues to optimize user experience and adoption. Using this methodology, we will provide justifiable confidence to progress to full-scale randomized controlled trials when DART is integrated into clinical management pathways. This study protocol will provide a blueprint for future usability studies of mobile health solutions. Methods: We will collect qualitative and quantitative data from 20-30 participants aged 18 years and older for 4 months. The exact number of participants recruited will be dependent on the number of iterative cycles required to reach the study end points. Building on previous internal testing and stakeholder involvement, quantitative data collection is defined by the constructs within the ISO 9241-210-2019 standard and the system usability scale, providing a usability score for DART. Guided by the participant responses to quantitative questioning, the researcher will focus the qualitative data collection on specific usability problems. These will then be graded to provide the rationale for further DART system improvements throughout the iterative cycles. Results: This study received approval from the Queen Mary University of London Ethics of Research Committee (QMREC2018/48/048) on June 4, 2020. At manuscript submission, study recruitment was on-going, with data collection to be completed and results published in 2021. Conclusions: This study will provide evidence concerning mobile health DART system usability and acceptance determining system improvements required to support user adoption and minimize suboptimal system usability as a potential confounder within subsequent noninferiority clinical trials. Success should produce a safe effective system with excellent usability, facilitating quicker and easier patient access to appropriate care while reducing the burden on primary and secondary care musculoskeletal services. This deliberately rigorous approach to mobile health innovation could be used as a guide for other developers of similar apps. International Registered Report Identifier (IRRID): DERR1-10.2196/27205 UR - https://www.researchprotocols.org/2021/5/e27205 UR - http://dx.doi.org/10.2196/27205 UR - http://www.ncbi.nlm.nih.gov/pubmed/34003135 ID - info:doi/10.2196/27205 ER - TY - JOUR AU - Acquaviva, Kimberly PY - 2021/4/20 TI - Comparison of Intercom and Megaphone Hashtags Using Four Years of Tweets From the Top 44 Schools of Nursing: Thematic Analysis JO - JMIR Nursing SP - e25114 VL - 4 IS - 2 KW - Twitter KW - hashtag KW - nurses KW - media KW - intercom hashtag KW - megaphone hashtag N2 - Background: When this study began in 2018, I sought to determine the extent to which the top 50 schools of nursing were using hashtags that could attract attention from journalists on Twitter. In December 2020, the timeframe was expanded to encompass 2 more years of data, and an analysis was conducted of the types of hashtags used. Objective: The study attempted to answer the following question: to what extent are top-ranked schools of nursing using hashtags that could attract attention from journalists, policy makers, and the public on Twitter? Methods: In February 2018, 47 of the top 50 schools of nursing had public Twitter accounts. The most recent 3200 tweets were extracted from each account and analyzed. There were 31,762 tweets in the time period covered (September 29, 2016, through February 22, 2018). After 13,429 retweets were excluded, 18,333 tweets remained. In December 2020, 44 of the original 47 schools of nursing still had public Twitter accounts under the same name used in the first phase of the study. Three accounts that were no longer active were removed from the 2016-2018 data set, resulting in 16,939 tweets from 44 schools of nursing. The Twitter data for the 44 schools of nursing were obtained for the time period covered in the second phase of the study (February 23, 2018, through December 13, 2020), and the most recent 3200 tweets were extracted from each of the accounts. On excluding retweets, there were 40,368 tweets in the 2018-2020 data set. The 2016-2018 data set containing 16,939 tweets was merged with the 2018-2020 data set containing 40,368 tweets, resulting in 57,307 tweets in the 2016-2020 data set. Results: Each hashtag used 100 times or more in the 2016-2020 data set was categorized as one of the following seven types: nursing, school, conference or tweet chat, health, illness/disease/condition, population, and something else. These types were then broken down into the following two categories: intercom hashtags and megaphone hashtags. Approximately 83% of the time, schools of nursing used intercom hashtags (inward-facing hashtags focused on in-group discussion within and about the profession). Schools of nursing rarely used outward-facing megaphone hashtags. There was no discernible shift in the way that schools of nursing used hashtags after the publication of The Woodhull Study Revisited. Conclusions: Top schools of nursing use hashtags more like intercoms to communicate with other nurses rather than megaphones to invite attention from journalists, policy makers, and the public. If schools of nursing want the media to showcase their faculty members as experts, they need to increase their use of megaphone hashtags to connect the work of their faculty with topics of interest to the public. UR - https://nursing.jmir.org/2021/2/e25114 UR - http://dx.doi.org/10.2196/25114 UR - http://www.ncbi.nlm.nih.gov/pubmed/34345795 ID - info:doi/10.2196/25114 ER - TY - JOUR AU - Ferris, B. Emily AU - Wyka, Katarzyna AU - Evenson, R. Kelly AU - Dorn, M. Joan AU - Thorpe, Lorna AU - Catellier, Diane AU - Huang, T-K Terry PY - 2021/3/24 TI - Recruitment and Retention Strategies for Community-Based Longitudinal Studies in Diverse Urban Neighborhoods JO - JMIR Form Res SP - e18591 VL - 5 IS - 3 KW - community-based KW - participant engagement KW - natural experiment KW - built environment intervention KW - health disparities KW - study adaptations UR - https://formative.jmir.org/2021/3/e18591 UR - http://dx.doi.org/10.2196/18591 UR - http://www.ncbi.nlm.nih.gov/pubmed/33759799 ID - info:doi/10.2196/18591 ER - TY - JOUR AU - Hanley, Terry AU - Sefi, Aaron AU - Grauberg, Janet AU - Prescott, Julie AU - Etchebarne, Andre PY - 2021/3/22 TI - A Theory of Change for Web-Based Therapy and Support Services for Children and Young People: Collaborative Qualitative Exploration JO - JMIR Pediatr Parent SP - e23193 VL - 4 IS - 1 KW - telepsychology KW - digital mental health KW - online therapy KW - young people KW - Kooth KW - Theory of Change KW - positive virtual ecosystems N2 - Background: Web-based counseling and support has become increasingly commonplace for children and young people (CYP). Currently, there is limited research that focuses on the mechanisms of change within complex telepsychology platforms, a factor that makes designing and implementing outcome measures challenging. Objective: This project aims to articulate a theory of change (ToC) for Kooth, a web-based therapy and support platform for CYP. Methods: A collaborative qualitative research design involving professional staff, academic partners, and young people was used to develop the ToC. The following three major reflective phases were engaged: a scoping workshop involving professional staff and academic partners, a series of explorative projects were completed to inform the development of the ToC, and the draft ToC was reviewed for coherence by key stakeholders (young people, online professionals, and service managers). Results: A collaboratively developed ToC was presented. This was divided into the conditions that lead to individuals wanting to access web-based therapy and support (eg, individuals wanting support there and then or quickly), the mode of service delivery (eg, skilled and experienced professionals able to build empathetic relationships with CYP), and the observed and reported changes that occur as a consequence of using the service (eg, individuals being better able to manage current and future situations). Conclusions: Developing the ToC helps to shed light on how web-based therapy and support services aid the mental health and well-being of CYP. Furthermore, it helps to understand the development of positive virtual ecosystems and can be used to devise evaluative tools for CYP telepsychology providers. UR - https://pediatrics.jmir.org/2021/1/e23193 UR - http://dx.doi.org/10.2196/23193 UR - http://www.ncbi.nlm.nih.gov/pubmed/33749615 ID - info:doi/10.2196/23193 ER - TY - JOUR AU - Weijers, Miriam AU - Bastiaenen, Caroline AU - Feron, Frans AU - Schröder, Kay PY - 2021/2/9 TI - Designing a Personalized Health Dashboard: Interdisciplinary and Participatory Approach JO - JMIR Form Res SP - e24061 VL - 5 IS - 2 KW - visualization design model KW - dashboard KW - evaluation KW - personalized health care KW - International Classification of Functioning, Disability and Health (ICF) KW - patient access to records KW - human?computer interaction KW - health information visualization N2 - Background: Within the Dutch Child Health Care (CHC), an online tool (360° CHILD-profile) is designed to enhance prevention and transformation toward personalized health care. From a personalized preventive perspective, it is of fundamental importance to timely identify children with emerging health problems interrelated to multiple health determinants. While digitalization of children?s health data is now realized, the accessibility of data remains a major challenge for CHC professionals, let alone for parents/youth. Therefore, the idea was initiated from CHC practice to develop a novel approach to make relevant information accessible at a glance. Objective: This paper describes the stepwise development of a dashboard, as an example of using a design model to achieve visualization of a comprehensive overview of theoretically structured health data. Methods: Developmental process is based on the nested design model with involvement of relevant stakeholders in a real-life context. This model considers immediate upstream validation within 4 cascading design levels: Domain Problem and Data Characterization, Operation and Data Type Abstraction, Visual Encoding and Interaction Design, and Algorithm Design. This model also includes impact-oriented downstream validation, which can be initiated after delivering the prototype. Results: A comprehensible 360° CHILD-profile is developed: an online accessible visualization of CHC data based on the theoretical concept of the International Classification of Functioning, Disability and Health. This dashboard provides caregivers and parents/youth with a holistic view on children?s health and ?entry points? for preventive, individualized health plans. Conclusions: Describing this developmental process offers guidance on how to utilize the nested design model within a health care context. UR - https://formative.jmir.org/2021/2/e24061 UR - http://dx.doi.org/10.2196/24061 UR - http://www.ncbi.nlm.nih.gov/pubmed/33560229 ID - info:doi/10.2196/24061 ER - TY - JOUR AU - Hill, Adele AU - Joyner, H. Christopher AU - Keith-Jopp, Chloe AU - Yet, Barbaros AU - Tuncer Sakar, Ceren AU - Marsh, William AU - Morrissey, Dylan PY - 2021/1/15 TI - A Bayesian Network Decision Support Tool for Low Back Pain Using a RAND Appropriateness Procedure: Proposal and Internal Pilot Study JO - JMIR Res Protoc SP - e21804 VL - 10 IS - 1 KW - back pain KW - decision making KW - Bayesian methods KW - consensus N2 - Background: Low back pain (LBP) is an increasingly burdensome condition for patients and health professionals alike, with consistent demonstration of increasing persistent pain and disability. Previous decision support tools for LBP management have focused on a subset of factors owing to time constraints and ease of use for the clinician. With the explosion of interest in machine learning tools and the commitment from Western governments to introduce this technology, there are opportunities to develop intelligent decision support tools. We will do this for LBP using a Bayesian network, which will entail constructing a clinical reasoning model elicited from experts. Objective: This paper proposes a method for conducting a modified RAND appropriateness procedure to elicit the knowledge required to construct a Bayesian network from a group of domain experts in LBP, and reports the lessons learned from the internal pilot of the procedure. Methods: We propose to recruit expert clinicians with a special interest in LBP from across a range of medical specialties, such as orthopedics, rheumatology, and sports medicine. The procedure will consist of four stages. Stage 1 is an online elicitation of variables to be considered by the model, followed by a face-to-face workshop. Stage 2 is an online elicitation of the structure of the model, followed by a face-to-face workshop. Stage 3 consists of an online phase to elicit probabilities to populate the Bayesian network. Stage 4 is a rudimentary validation of the Bayesian network. Results: Ethical approval has been obtained from the Research Ethics Committee at Queen Mary University of London. An internal pilot of the procedure has been run with clinical colleagues from the research team. This showed that an alternating process of three remote activities and two in-person meetings was required to complete the elicitation without overburdening participants. Lessons learned have included the need for a bespoke online elicitation tool to run between face-to-face meetings and for careful operational definition of descriptive terms, even if widely clinically used. Further, tools are required to remotely deliver training about self-identification of various forms of cognitive bias and explain the underlying principles of a Bayesian network. The use of the internal pilot was recognized as being a methodological necessity. Conclusions: We have proposed a method to construct Bayesian networks that are representative of expert clinical reasoning for a musculoskeletal condition in this case. We have tested the method with an internal pilot to refine the process prior to deployment, which indicates the process can be successful. The internal pilot has also revealed the software support requirements for the elicitation process to model clinical reasoning for a range of conditions. International Registered Report Identifier (IRRID): DERR1-10.2196/21804 UR - http://www.researchprotocols.org/2021/1/e21804/ UR - http://dx.doi.org/10.2196/21804 UR - http://www.ncbi.nlm.nih.gov/pubmed/33448937 ID - info:doi/10.2196/21804 ER - TY - JOUR AU - Carlin, Thomas AU - Vuillerme, Nicolas PY - 2021/1/13 TI - Step and Distance Measurement From a Low-Cost Consumer-Based Hip and Wrist Activity Monitor: Protocol for a Validity and Reliability Assessment JO - JMIR Res Protoc SP - e21262 VL - 10 IS - 1 KW - activity monitor KW - pedometer KW - measurement KW - validity KW - reliability KW - walking KW - step count KW - distance N2 - Background: Self-tracking via wearable and mobile technologies is becoming an essential part of personal health management. At this point, however, little information is available to substantiate the validity and reliability of low-cost consumer-based hip and wrist activity monitors, with regard more specifically to the measurements of step counts and distance traveled while walking. Objective: The aim of our study is to assess the validity and reliability of step and distance measurement from a low-cost consumer-based hip and wrist activity monitor specific in various walking conditions that are commonly encountered in daily life. Specifically, this study is designed to evaluate whether and to what extent validity and reliability could depend on the sensor placement on the human body and the walking task being performed. Methods: Thirty healthy participants will be instructed to wear four PBN 2433 (Nakosite) activity monitors simultaneously, with one placed on each hip and each wrist. Participants will attend two experimental sessions separated by 1 week. During each experimental session, two separate studies will be performed. In study 1, participants will be instructed to complete a 2-minute walk test along a 30-meter indoor corridor under 3 walking speeds: very slow, slow, and usual speed. In study 2, participants will be required to complete the following 3 conditions performed at usual walking speed: walking on flat ground, upstairs, and downstairs. Activity monitor measured step count and distance values will be computed along with the actual step count (determined from video recordings) and distance (measured using a measuring tape) to determine validity and reliability for each activity monitor placement and each walking condition. Results: Participant recruitment and data collection began in January 2020. As of June 2020, we enrolled 8 participants. Dissemination of study results in peer-reviewed journals is expected in spring 2021. Conclusions: To the best of our knowledge, this is the first study that examines the validity and reliability of step and distance measurement during walking using the PBN 2433 (Nakosite) activity monitor. Results of this study will provide beneficial information on the effects of activity monitor placement, walking speed, and walking tasks on the validity and reliability of step and distance measurement. We believe such information is of utmost importance to general consumers, clinicians, and researchers. International Registered Report Identifier (IRRID): DERR1-10.2196/21262 UR - http://www.researchprotocols.org/2021/1/e21262/ UR - http://dx.doi.org/10.2196/21262 UR - http://www.ncbi.nlm.nih.gov/pubmed/33439138 ID - info:doi/10.2196/21262 ER - TY - JOUR AU - Sturgill, Ronda AU - Martinasek, Mary AU - Schmidt, Trine AU - Goyal, Raj PY - 2021/1/5 TI - A Novel Artificial Intelligence-Powered Emotional Intelligence and Mindfulness App (Ajivar) for the College Student Population During the COVID-19 Pandemic: Quantitative Questionnaire Study JO - JMIR Form Res SP - e25372 VL - 5 IS - 1 KW - mindfulness KW - COVID-19 KW - college students KW - emotional intelligence N2 - Background: Emotional intelligence (EI) and mindfulness can impact the level of anxiety and depression that an individual experiences. These symptoms have been exacerbated among college students during the COVID-19 pandemic. Ajivar is an app that utilizes artificial intelligence (AI) and machine learning to deliver personalized mindfulness and EI training. Objective: The main objective of this research study was to determine the effectiveness of delivering an EI curriculum and mindfulness techniques using an AI conversation platform, Ajivar, to improve symptoms of anxiety and depression during this pandemic. Methods: A total of 99 subjects, aged 18 to 29 years, were recruited from a second-semester group of freshmen students. All participants completed the online TestWell Wellness Inventory at the start and end of the 14-week semester. The comparison group members (49/99, 49%) were given routine mental wellness instruction. The intervention group members (50/99, 51%) were required to complete Ajivar activities in addition to routine mental wellness instruction during the semester, which coincided with the onset of the COVID-19 pandemic. This group also completed assessments to evaluate for anxiety, using the 7-item Generalized Anxiety Disorder (GAD-7) scale, and depression, using the 9-item Patient Health Questionnaire (PHQ-9). Results: Study participants reported a mean age of 19.9 (SD 1.94) years; 27% (27/99) of the group were male and 60% (59/99) identified as Caucasian. No significant demographic differences existed between the comparison and intervention groups. Subjects in the intervention group interacted with Ajivar for a mean time of 1424 (SD 1168) minutes. There was a significant decrease in anxiety, as measured by the GAD-7: the mean score was 11.47 (SD 1.85) at the start of the study compared to 6.27 (SD 1.44) at the end (P<.001). There was a significant reduction in the symptoms of depression measured by the PHQ-9: the mean score was 10.69 (SD 2.04) at the start of the study compared to 6.69 (SD 2.41) at the end (P=.001). Both the intervention and comparison groups independently had significant improvements in the TestWell Wellness Inventory from pretest to posttest. The subgroups in the social awareness and spirituality inventories showed significant improvement in the intervention group. In a subgroup of participants (11/49, 22%) where the GAD-7 was available during the onset of the COVID-19 pandemic, there was an increase in anxiety from the start of the study (mean score 11.63, SD 2.16) to mid-March (ie, onset of the pandemic) (mean score 13.03, SD 1.48; P=.23), followed by a significant decrease at the end of the study period (mean score 5.9, SD 1.44; P=.001). Conclusions: It is possible to deliver EI and mindfulness training in a scalable way using the Ajivar app during the COVID-19 pandemic, resulting in improvements in anxiety, depression, and EI in the college student population. UR - http://formative.jmir.org/2021/1/e25372/ UR - http://dx.doi.org/10.2196/25372 UR - http://www.ncbi.nlm.nih.gov/pubmed/33320822 ID - info:doi/10.2196/25372 ER - TY - JOUR AU - Do, Quan AU - Marc, David AU - Plotkin, Marat AU - Pickering, Brian AU - Herasevich, Vitaly PY - 2020/12/24 TI - Starter Kit for Geotagging and Geovisualization in Health Care: Resource Paper JO - JMIR Form Res SP - e23379 VL - 4 IS - 12 KW - geographic mapping KW - medicalGIS guidelines KW - information storage and retrieval KW - mapping KW - geotagging KW - data visualization KW - population KW - public health N2 - Background: Geotagging is the process of attaching geospatial tags to various media data types. In health care, the goal of geotagging is to gain a better understanding of health-related questions applied to populations. Although there has been a prevalence of geographic information in public health, in order to effectively use and expand geotagging across health care there is a requirement to understand other factors such as the disposition, standardization, data sources, technologies, and limitations. Objective: The objective of this document is to serve as a resource for new researchers in the field. This report aims to be comprehensive but easy for beginners to understand and adopt in practice. The optimal geocodes, their sources, and a rationale for use are suggested. Geotagging?s issues and limitations are also discussed. Methods: A comprehensive review of technical instructions and articles was conducted to evaluate guidelines for geotagging, and online resources were curated to support the implementation of geotagging practices. Summary tables were developed to describe the available geotagging resources (free and for fee) that can be leveraged by researchers and quality improvement personnel to effectively perform geospatial analyses primarily targeting US health care. Results: This paper demonstrated steps to develop an initial geotagging and geovisualization project with clear structure and instructions. The geotagging resources were summarized. These resources are essential for geotagging health care projects. The discussion section provides better understanding of geotagging?s limitations and suggests suitable way to approach it. Conclusions: We explain how geotagging can be leveraged in health care and offer the necessary initial resources to obtain geocodes, adjustment data, and health-related measures. The resources outlined in this paper can support an individual and/or organization in initiating a geotagging health care project. UR - http://formative.jmir.org/2020/12/e23379/ UR - http://dx.doi.org/10.2196/23379 UR - http://www.ncbi.nlm.nih.gov/pubmed/33361054 ID - info:doi/10.2196/23379 ER - TY - JOUR AU - Sai Prashanthi, Gumpili AU - Deva, Ayush AU - Vadapalli, Ranganath AU - Das, Vipin Anthony PY - 2020/12/17 TI - Automated Categorization of Systemic Disease and Duration From Electronic Medical Record System Data Using Finite-State Machine Modeling: Prospective Validation Study JO - JMIR Form Res SP - e24490 VL - 4 IS - 12 KW - electronic health records KW - data analysis KW - machine learning KW - algorithms KW - ophthalmology N2 - Background: One of the major challenges in the health care sector is that approximately 80% of generated data remains unstructured and unused. Since it is difficult to handle unstructured data from electronic medical record systems, it tends to be neglected for analyses in most hospitals and medical centers. Therefore, there is a need to analyze unstructured big data in health care systems so that we can optimally utilize and unearth all unexploited information from it. Objective: In this study, we aimed to extract a list of diseases and associated keywords along with the corresponding time durations from an indigenously developed electronic medical record system and describe the possibility of analytics from the acquired datasets. Methods: We propose a novel, finite-state machine to sequentially detect and cluster disease names from patients? medical history. We defined 3 states in the finite-state machine and transition matrix, which depend on the identified keyword. In addition, we also defined a state-change action matrix, which is essentially an action associated with each transition. The dataset used in this study was obtained from an indigenously developed electronic medical record system called eyeSmart that was implemented across a large, multitier ophthalmology network in India. The dataset included patients? past medical history and contained records of 10,000 distinct patients. Results: We extracted disease names and associated keywords by using the finite-state machine with an accuracy of 95%, sensitivity of 94.9%, and positive predictive value of 100%. For the extraction of the duration of disease, the machine?s accuracy was 93%, sensitivity was 92.9%, and the positive predictive value was 100%. Conclusions: We demonstrated that the finite-state machine we developed in this study can be used to accurately identify disease names, associated keywords, and time durations from a large cohort of patient records obtained using an electronic medical record system. UR - http://formative.jmir.org/2020/12/e24490/ UR - http://dx.doi.org/10.2196/24490 UR - http://www.ncbi.nlm.nih.gov/pubmed/33331823 ID - info:doi/10.2196/24490 ER - TY - JOUR AU - Walker, M. Eloise AU - Jasper, E. Alice AU - Davis, Lauren AU - Yip, Por Kay AU - Faniyi, A. Aduragbemi AU - Hughes, J. Michael AU - Crisford, A. Helena AU - Spittle, A. Daniella AU - Sapey, Elizabeth AU - Belchamber, R. Kylie B. AU - Scott, Aaron PY - 2020/12/4 TI - Mitigating Health Risks to Reopen a Clinical Research Laboratory During the COVID-19 Pandemic: A Framework JO - JMIR Res Protoc SP - e22570 VL - 9 IS - 12 KW - clinical laboratory KW - risk assessment KW - COVID-19 KW - SARS-CoV-2 KW - framework KW - research KW - risk KW - lab KW - safety N2 - Background: The COVID-19 pandemic has led to many countries implementing lockdown procedures, resulting in the suspension of laboratory research. With lockdown measures now easing in some areas, many laboratories are preparing to reopen. This is particularly challenging for clinical research laboratories due to the dual risk of patient samples carrying the virus that causes COVID-19, SARS-CoV-2, and the risk to patients being exposed to research staff during clinical sampling. To date, no confirmed transmission of the virus has been confirmed within a laboratory setting; however, operating processes and procedures should be adapted to ensure safe working of samples of positive, negative, or unknown COVID-19 status. Objective: In this paper, we propose a framework for reopening a clinical research laboratory and resuming operations with the aim to maximize research capacity while minimizing the risk to research participants and staff. Methods: This framework was developed by consensus among experienced laboratory staff who have prepared to reopen a clinical research laboratory. Results: Multiple aspects need to be considered to reopen a clinical laboratory. We describe our process to stratify projects by risk, including assessment of donor risk and COVID-19 clinical status, the COVID-19 status of the specific sample type, and how to safely process each sample type. We describe methods to prepare the laboratory for safe working including maintaining social distancing through signage, one-way systems and access arrangements for staff and patients, limiting staff numbers on site and encouraging home working for all nonlaboratory tasks including data analysis and writing. Shared equipment usage was made safe by adapting booking systems to allow for the deployment of cleaning protocols. All risk assessments and standard operating procedures were rewritten and approved by local committees, and staff training was initiated to ensure compliance. Conclusions: Laboratories can adopt and adapt this framework to expedite reopening a clinical laboratory during the current COVID-19 pandemic while mitigating the risk to research participants and staff. UR - http://www.researchprotocols.org/2020/12/e22570/ UR - http://dx.doi.org/10.2196/22570 UR - http://www.ncbi.nlm.nih.gov/pubmed/33146625 ID - info:doi/10.2196/22570 ER - TY - JOUR AU - BinDhim, F. Nasser AU - Althumiri, A. Nora AU - Basyouni, H. Mada AU - Sims, T. Omar AU - Alhusseini, Noara AU - Alqahtani, A. Saleh PY - 2020/11/13 TI - Arabic Translation of the Weight Self-Stigma Questionnaire: Instrument Validation Study of Factor Structure and Reliability JO - JMIR Form Res SP - e24169 VL - 4 IS - 11 KW - overweight KW - stigma KW - weight self-stigma KW - Weight Self-Stigma Questionnaire KW - obesity KW - Saudi Arabia KW - questionnaire KW - validation KW - reliability KW - validity N2 - Background: While it is most often associated with its effects on physical health, obesity is also associated with serious self-stigmatization. The lack of a suitable, validated tool to measure weight-related self-stigma in Arabic countries may be partly responsible for the scarcity of literature about this problem. Objective: This study investigated the reliability and validity of an Arabic version of the Weight Self-Stigma Questionnaire (WSSQ). Methods: Data on the Arabic-translated version of the 12-item WSSQ were collected using two cross-sectional electronic questionnaires distributed among Saudi nationals through the Sharik Association for Health Research?s database in June 2020. Internal consistency, test-retest reliability, and exploratory factor analysis of the Arabic WSSQ were assessed and compared with the original English version and other translations. Results: For reliability analysis, 43 participants completed the Arabic WSSQ during two time periods. Internal consistency was ?=.898 for the overall survey, ?=.819 for the fear of enacted stigma subscale (factor 1), and ?=.847 for the self-devaluation subscale (factor 2). The test-retest reliability of the intraclass correlation coef?cient was ?=.982. In the factor structure analysis, 295 participants completed the questionnaire. The Arabic WSSQ loading of the items was consistent with the original WSSQ, except for the loading of item 9, which was stronger in factor 2 than in factor 1. The two factors accounted for the observed variances of 47.7% and 10.6%. Conclusions: The Arabic version of the WSSQ has good internal consistency and reliability, and the factorial structure is similar to that of the original WSSQ. The Arabic WSSQ is adaptable for clinicians seeking to assess weight-related self-stigma in Arabic-speaking people. UR - http://formative.jmir.org/2020/11/e24169/ UR - http://dx.doi.org/10.2196/24169 UR - http://www.ncbi.nlm.nih.gov/pubmed/33185558 ID - info:doi/10.2196/24169 ER - TY - JOUR AU - Marciniak, Anna Marta AU - Shanahan, Lilly AU - Rohde, Judith AU - Schulz, Ava AU - Wackerhagen, Carolin AU - Kobyli?ska, Dorota AU - Tuescher, Oliver AU - Binder, Harald AU - Walter, Henrik AU - Kalisch, Raffael AU - Kleim, Birgit PY - 2020/11/12 TI - Standalone Smartphone Cognitive Behavioral Therapy?Based Ecological Momentary Interventions to Increase Mental Health: Narrative Review JO - JMIR Mhealth Uhealth SP - e19836 VL - 8 IS - 11 KW - mHealth KW - mobile app KW - ecological momentary intervention KW - EMI KW - cognitive behavioral therapy KW - CBT KW - COVID-19 KW - mobile phone KW - smartphone N2 - Background: A growing number of psychological interventions are delivered via smartphones with the aim of increasing the efficacy and effectiveness of these treatments and providing scalable access to interventions for improving mental health. Most of the scientifically tested apps are based on cognitive behavioral therapy (CBT) principles, which are considered the gold standard for the treatment of most mental health problems. Objective: This review investigates standalone smartphone-based ecological momentary interventions (EMIs) built on principles derived from CBT that aim to improve mental health. Methods: We searched the MEDLINE, PsycINFO, EMBASE, and PubMed databases for peer-reviewed studies published between January 1, 2007, and January 15, 2020. We included studies focusing on standalone app-based approaches to improve mental health and their feasibility, efficacy, or effectiveness. Both within- and between-group designs and studies with both healthy and clinical samples were included. Blended interventions, for example, app-based treatments in combination with psychotherapy, were not included. Selected studies were evaluated in terms of their design, that is, choice of the control condition, sample characteristics, EMI content, EMI delivery characteristics, feasibility, efficacy, and effectiveness. The latter was defined in terms of improvement in the primary outcomes used in the studies. Results: A total of 26 studies were selected. The results show that EMIs based on CBT principles can be successfully delivered, significantly increase well-being among users, and reduce mental health symptoms. Standalone EMIs were rated as helpful (mean 70.8%, SD 15.3; n=4 studies) and satisfying for users (mean 72.6%, SD 17.2; n=7 studies). Conclusions: Study quality was heterogeneous, and feasibility was often not reported in the reviewed studies, thus limiting the conclusions that can be drawn from the existing data. Together, the studies show that EMIs may help increase mental health and thus support individuals in their daily lives. Such EMIs provide readily available, scalable, and evidence-based mental health support. These characteristics appear crucial in the context of a global crisis such as the COVID-19 pandemic but may also help reduce personal and economic costs of mental health impairment beyond this situation or in the context of potential future pandemics. UR - https://mhealth.jmir.org/2020/11/e19836 UR - http://dx.doi.org/10.2196/19836 UR - http://www.ncbi.nlm.nih.gov/pubmed/33180027 ID - info:doi/10.2196/19836 ER - TY - JOUR AU - Kalaitzoglou, Evangelia AU - Majaliwa, Edna AU - Zacharin, Margaret AU - de Beaufort, Carine AU - Chanoine, Jean-Pierre AU - van Wijngaard-DeVugt, Conny AU - Sperla, Ervin AU - Boot, M. Annemieke AU - Drop, S. Stenvert L. PY - 2020/11/5 TI - Multilingual Global E-Learning Pediatric Endocrinology and Diabetes Curriculum for Front Line Health Care Providers in Resource-Limited Countries: Development Study JO - JMIR Form Res SP - e18555 VL - 4 IS - 11 KW - pediatric endocrinology KW - diabetes mellitus KW - e-learning KW - online learning KW - continuing education KW - resource-limited country KW - multilingual medical education N2 - Background: Electronic learning (e-learning) is a widely accessible, low-cost option for learning remotely in various settings that allows interaction between an instructor and a learner. Objective: We describe the development of a free and globally accessible multilingual e-learning module that provides education material on topics in pediatric endocrinology and diabetes and that is intended for first-line physicians and health workers but also trainees or medical specialists in resource-limited countries. Methods: As complements to concise chapters, interactive vignettes were constructed, exemplifying clinical issues and pitfalls, with specific attention to the 3 levels of medical health care in resource-limited countries. The module is part of a large e-learning portal, ESPE e-learning, which is based on ILIAS (Integriertes Lern-, Informations- und Arbeitskooperations-System), an open-source web-based learning management system. Following a review by global experts, the content was translated by native French, Spanish, Swahili, and Chinese?speaking colleagues into their respective languages using a commercial web-based translation tool (SDL Trados Studio). Results: Preliminary data suggest that the module is well received, particularly in targeted parts of the world and that active promotion to inform target users is warranted. Conclusions: The e-learning module is a free globally accessible multilingual up-to-date tool for use in resource-limited countries that has been utilized thus far with success. Widespread use will require dissemination of the tool on a global scale. UR - https://formative.jmir.org/2020/11/e18555 UR - http://dx.doi.org/10.2196/18555 UR - http://www.ncbi.nlm.nih.gov/pubmed/33151156 ID - info:doi/10.2196/18555 ER - TY - JOUR AU - McDonnell, Michelle AU - Owen, Edward Jason AU - Bantum, O'Carroll Erin PY - 2020/10/30 TI - Identification of Emotional Expression With Cancer Survivors: Validation of Linguistic Inquiry and Word Count JO - JMIR Form Res SP - e18246 VL - 4 IS - 10 KW - linguistic analysis KW - emotion KW - validation N2 - Background: Given the high volume of text-based communication such as email, Facebook, Twitter, and additional web-based and mobile apps, there are unique opportunities to use text to better understand underlying psychological constructs such as emotion. Emotion recognition in text is critical to commercial enterprises (eg, understanding the valence of customer reviews) and to current and emerging clinical applications (eg, as markers of clinical progress and risk of suicide), and the Linguistic Inquiry and Word Count (LIWC) is a commonly used program. Objective: Given the wide use of this program, the purpose of this study is to update previous validation results with two newer versions of LIWC. Methods: Tests of proportions were conducted using the total number of emotion words identified by human coders for each emotional category as the reference group. In addition to tests of proportions, we calculated F scores to evaluate the accuracy of LIWC 2001, LIWC 2007, and LIWC 2015. Results: Results indicate that LIWC 2001, LIWC 2007, and LIWC 2015 each demonstrate good sensitivity for identifying emotional expression, whereas LIWC 2007 and LIWC 2015 were significantly more sensitive than LIWC 2001 for identifying emotional expression and positive emotion; however, more recent versions of LIWC were also significantly more likely to overidentify emotional content than LIWC 2001. LIWC 2001 demonstrated significantly better precision (F score) for identifying overall emotion, negative emotion, and anxiety compared with LIWC 2007 and LIWC 2015. Conclusions: Taken together, these results suggest that LIWC 2001 most accurately reflects the emotional identification of human coders. UR - https://formative.jmir.org/2020/10/e18246 UR - http://dx.doi.org/10.2196/18246 UR - http://www.ncbi.nlm.nih.gov/pubmed/33124986 ID - info:doi/10.2196/18246 ER - TY - JOUR AU - Gulliver, Amelia AU - Calear, L. Alison AU - Sunderland, Matthew AU - Kay-Lambkin, Frances AU - Farrer, M. Louise AU - Banfield, Michelle AU - Batterham, J. Philip PY - 2020/10/29 TI - Consumer-Guided Development of an Engagement-Facilitation Intervention for Increasing Uptake and Adherence for Self-Guided Web-Based Mental Health Programs: Focus Groups and Online Evaluation Survey JO - JMIR Form Res SP - e22528 VL - 4 IS - 10 KW - mental health KW - internet KW - anxiety KW - depression KW - technology KW - treatment adherence and compliance N2 - Background: Self-guided web-based mental health programs are effective in treating and preventing mental health problems. However, current engagement with these programs in the community is suboptimal, and there is limited evidence indicating how to increase the use of existing evidence-based programs. Objective: This study aims to investigate the views of people with lived experience of depression and anxiety on factors influencing their engagement with self-guided web-based mental health (e?mental health) programs and to use these perspectives to develop an engagement-facilitation intervention (EFI) to increase engagement (defined as both uptake and adherence) with these programs. Methods: A total of 24 community members (female=21; male=3) with lived experience of depression and anxiety or depression or anxiety alone participated in 1 of 4 focus groups discussing the factors influencing their engagement with self-guided e?mental health programs and the appearance, delivery mode, and functionality of content for the proposed EFI. A subsequent evaluation survey of the focus group participants (n=14) was conducted to evaluate the resultant draft EFI. Data were thematically analyzed using both inductive and deductive qualitative methods. Results: Participants suggested that the critical component of an EFI was information that would challenge personal barriers to engagement, including receiving personalized symptom feedback, information regarding the program?s content or effectiveness and data security, and normalization of using e?mental health programs (eg, testimonials). Reminders, rewards, feedback about progress, and coaching were all mentioned as facilitating adherence. Conclusions: EFIs have the potential to improve community uptake of e?mental health programs. They should focus on providing information on the content and effectiveness of e?mental health programs and normalizing their use. Given that the sample comprised predominantly young females, this study may not be generalizable to other population groups. There is a strong value in involving people with a lived experience in the design and development of EFIs to maximize their effectiveness. UR - http://formative.jmir.org/2020/10/e22528/ UR - http://dx.doi.org/10.2196/22528 UR - http://www.ncbi.nlm.nih.gov/pubmed/33118939 ID - info:doi/10.2196/22528 ER - TY - JOUR AU - Draaijer, Melvin AU - Lalla-Edward, Tresha Samanta AU - Venter, Francois Willem Daniel AU - Vos, Alinda PY - 2020/9/30 TI - Phone Calls to Retain Research Participants and Determinants of Reachability in an African Setting: Observational Study JO - JMIR Form Res SP - e19138 VL - 4 IS - 9 KW - retention KW - loss to follow-up KW - phone KW - mobile phones KW - HIV KW - ART KW - South Africa N2 - Background: Long-term retention of research participants in studies is challenging. In research in sub-Saharan Africa, phone calls are the most frequently used method to distantly engage with participants. Objective: We aimed to get insight into the effectiveness of phone calls to retain contact with participants and evaluated determinants of reachability. Methods: A cross-sectional study was performed using the databases of two randomized controlled trials investigating different kinds of antiretroviral therapy in HIV-positive patients. One trial finished in 2018 (study 1), and the other finished in 2015 (study 2). A random sample size of 200 participants per study was obtained. There were up to 3 phone numbers available per participant collected during the studies. Participants received a maximum of 3 phone calls on every available number on different days and at different times. Voicemails were left, and emails sent wherever possible. We documented how many calls were answered, who answered, as well as after how many attempts participants were reached. To further increase our understanding of reachability, we conducted a short questionnaire assessing factors contributing to reachability. The study was approved by the Research Ethics Committee of the University of Witwatersrand, Johannesburg, South Africa (reference number M1811107). Results: In our sample size of n=200 per study, study 1, with a median time of 11 months since the last visit at the research site, had a response rate of 70.5% (141/200) participants while study 2, with a median duration of 55 months since the last visit, had a response rate of 50.0% (100/200; P<.001). In study 1, 61.5% (123/200) of calls were answered directly by the participant while this was 36.0% (72/200) in study 2 (P=.003). The likelihood of reaching a participant decreased with time (odds ratio [OR] 0.73, 95% CI 0.63 to 0.84) for every year since the last face-to-face visit. Having more phone numbers per participant increased reachability (OR 2.32, 95% CI 1.24 to 4.36 for 2 phone numbers and OR 3.03, 95% CI 1.48 to 6.22 for 3 phone numbers compared with 1 number). A total of 141 of 241 reached participants responded to the questionnaire. Of the 93 participants who had changed phone numbers, 5% (50/93) had changed numbers because their phone was stolen. The most preferred method of being contacted was direct calling (128/141) with participants naming this method followed by WhatsApp (69/141). Conclusions: Time since last visit and the number of phone numbers listed were the only determinants of reachability. Longer follow-up time is accompanied with a decrease in reachability by phone while more listed phone numbers increases the likelihood that someone can be reached. Trial Registration: ClinicalTrials.gov NCT02671383; https://clinicaltrials.gov/ct2/show/NCT02671383 and ClinicalTrials.gov NCT02670772; https://clinicaltrials.gov/ct2/show/NCT02670772 UR - http://formative.jmir.org/2020/9/e19138/ UR - http://dx.doi.org/10.2196/19138 UR - http://www.ncbi.nlm.nih.gov/pubmed/32996891 ID - info:doi/10.2196/19138 ER - TY - JOUR AU - Holch, Patricia AU - Marwood, R. Jordan PY - 2020/9/8 TI - EHealth Literacy in UK Teenagers and Young Adults: Exploration of Predictors and Factor Structure of the eHealth Literacy Scale (eHEALS) JO - JMIR Form Res SP - e14450 VL - 4 IS - 9 KW - eHealth literacy KW - irrational health beliefs KW - predictors KW - self-efficacy KW - teenagers and young adults KW - need for cognition KW - exploratory factor analysis N2 - Background: Increasingly, teenagers and young adults (TYAs) seek out health information online; however, it is not clear whether they possess electronic health (eHealth) literacy, defined as ?the ability to select, appraise, and utilize good quality health information from the internet.? A number of factors are included in the Lily model proposed by Norman and Skinner underpinning the development of eHealth literacy. It is important to understand which elements may influence the development of eHealth literacy in young people, as the current generation will continue to ?Google it? when faced with a health problem throughout their lives. Objective: The objectives of this study are to explore potential factors influencing young people?s eHealth literacy and explore the underlying constructs of the eHealth Literacy Scale (eHEALS) in a population of UK university students. Methods: A total of 188 undergraduate psychology students from a large UK University were recruited as an opportunity sample. Of these, 88.8% (167/188) of participants were female with a mean age of 20.13 (SD 2.16) years and the majority were White British (159/188, 84.6%). Employing a cross-sectional design TYAs completed the following measures exploring eHealth literacy (eHEALS): Irrational Health Belief Scale; Newest Vital Sign (NVS), a measure of functional health literacy; Need for Cognition Scale, a preference for effortful cognitive activity; and General Self-Efficacy (GSE) Scale, exploring personal agency and confidence. The eHEALS was also subject to exploratory factor analysis (EFA), for which in addition to the total variance explained, the scree plot, eigenvalues, and factor loadings were assessed to verify the structure. Results: eHEALS and GSE were significantly positively correlated (r=0.28, P<.001) and hierarchical linear modeling revealed GSE as the significant predictor of scores on the eHEALS (F1,186=16.16, P<.001, R2=0.08), accounting for 8.0% of the variance. Other notable relationships were GSE and need for cognition (NFC) were also positively correlated (r=0.33, P<.001), and NFC and irrational health beliefs were significantly negatively correlated (r=?.14, P=.03). Using Spearman correlations, GSE and NVS (rs=0.14, P=.04) and NFC and NVS (rs=0.19, P=.003) were positively correlated. An EFA revealed the scale to be stable and identified a 2-factor structure related to information acquisition and information application. Conclusions: This is the first study in the UK to explore relationships between these key variables and verify the structure of the eHEALS in a TYA population in the UK. The findings that self-efficacy has a major influence firmly consolidate its status as fundamental to the development of eHealth literacy. Future studies will explore the influence of body image and the development of eHealth literacy in more diverse TYA populations. UR - http://formative.jmir.org/2020/9/e14450/ UR - http://dx.doi.org/10.2196/14450 UR - http://www.ncbi.nlm.nih.gov/pubmed/32897230 ID - info:doi/10.2196/14450 ER - TY - JOUR AU - Li, Shiyu AU - Howard, T. Jeffrey AU - Sosa, T. Erica AU - Cordova, Alberto AU - Parra-Medina, Deborah AU - Yin, Zenong PY - 2020/8/31 TI - Calibrating Wrist-Worn Accelerometers for Physical Activity Assessment in Preschoolers: Machine Learning Approaches JO - JMIR Form Res SP - e16727 VL - 4 IS - 8 KW - preschoolers KW - accelerometer KW - physical activity KW - obesity KW - machine learning N2 - Background: Physical activity (PA) level is associated with multiple health benefits during early childhood. However, inconsistency in the methods for quantification of PA levels among preschoolers remains a problem. Objective: This study aimed to develop PA intensity cut points for wrist-worn accelerometers by using machine learning (ML) approaches to assess PA in preschoolers. Methods: Wrist- and hip-derived acceleration data were collected simultaneously from 34 preschoolers on 3 consecutive preschool days. Two supervised ML models, receiver operating characteristic curve (ROC) and ordinal logistic regression (OLR), and one unsupervised ML model, k-means cluster analysis, were applied to establish wrist-worn accelerometer vector magnitude (VM) cut points to classify accelerometer counts into sedentary behavior, light PA (LPA), moderate PA (MPA), and vigorous PA (VPA). Physical activity intensity levels identified by hip-worn accelerometer VM cut points were used as reference to train the supervised ML models. Vector magnitude counts were classified by intensity based on three newly established wrist methods and the hip reference to examine classification accuracy. Daily estimates of PA were compared to the hip-reference criterion. Results: In total, 3600 epochs with matched hip- and wrist-worn accelerometer VM counts were analyzed. All ML approaches performed differently on developing PA intensity cut points for wrist-worn accelerometers. Among the three ML models, k-means cluster analysis derived the following cut points: ?2556 counts per minute (cpm) for sedentary behavior, 2557-7064 cpm for LPA, 7065-14532 cpm for MPA, and ?14533 cpm for VPA; in addition, k-means cluster analysis had the highest classification accuracy, with more than 70% of the total epochs being classified into the correct PA categories, as examined by the hip reference. Additionally, k-means cut points exhibited the most accurate estimates on sedentary behavior, LPA, and VPA as the hip reference. None of the three wrist methods were able to accurately assess MPA. Conclusions: This study demonstrates the potential of ML approaches in establishing cut points for wrist-worn accelerometers to assess PA in preschoolers. However, the findings from this study warrant additional validation studies. UR - https://formative.jmir.org/2020/8/e16727 UR - http://dx.doi.org/10.2196/16727 UR - http://www.ncbi.nlm.nih.gov/pubmed/32667893 ID - info:doi/10.2196/16727 ER - TY - JOUR AU - Ukoha, Chukwuma PY - 2020/8/14 TI - How Health Care Organizations Approach Social Media Measurement: Qualitative Study JO - JMIR Form Res SP - e18518 VL - 4 IS - 8 KW - health care organization KW - social media KW - measurement KW - benchmarking KW - metrics KW - analytics tools N2 - Background: Many health care organizations use social media to support a variety of activities. To ensure continuous improvement in social media performance, health care organizations must measure their social media. Objective: The purpose of this study is to explore how health care organizations approach social media measurement and to elucidate the tools they employ. Methods: In this exploratory qualitative research, Australian health care organizations that use social media, varying in size and locality, were invited to participate in the study. Data were collected through semistructured interviews, and the transcripts were analyzed using thematic analysis. Results: The study identified health care organizations? approaches to social media measurement. While some measured their social media frequently, others used infrequent measurements, and a few did not measure theirs at all. Those that measured their social media used one or a combination of the following yardsticks: personal benchmarking, peer benchmarking, and metric benchmarking. The metrics tracked included one or more of the following: reach, engagement, and conversion rates. The tools employed to measure social media were either inbuilt or add-on analytics tools. Although many participants showed great interest in measuring their social media, they still had some unanswered questions. Conclusions: The lack of a consensus approach to measurement suggests that, unlike other industries, social media measurement in health care settings is at a nascent stage. There is a need to improve knowledge, sophistication, and integration of social media strategy through the application of theoretical and analytical knowledge to help resolve the current challenge of effective social media measurement. This study calls for social media training in health care organizations. Such training must focus on how to use relevant tools and how to measure their use effectively. UR - http://formative.jmir.org/2020/8/e18518/ UR - http://dx.doi.org/10.2196/18518 UR - http://www.ncbi.nlm.nih.gov/pubmed/32795994 ID - info:doi/10.2196/18518 ER - TY - JOUR AU - Llorens-Vernet, Pere AU - Miró, Jordi PY - 2020/7/31 TI - The Mobile App Development and Assessment Guide (MAG): Delphi-Based Validity Study JO - JMIR Mhealth Uhealth SP - e17760 VL - 8 IS - 7 KW - assessment KW - Delphi method KW - MAG KW - mobile apps KW - mobile health KW - validity KW - guide N2 - Background: In recent years, there has been an exponential growth of mobile health (mHealth)?related apps. This has occurred in a somewhat unsupervised manner. Therefore, having a set of criteria that could be used by all stakeholders to guide the development process and the assessment of the quality of the apps is of most importance. Objective: The aim of this paper is to study the validity of the Mobile App Development and Assessment Guide (MAG), a guide recently created to help stakeholders develop and assess mobile health apps. Methods: To conduct a validation process of the MAG, we used the Delphi method to reach a consensus among participating stakeholders. We identified 158 potential participants: 45 patients as potential end users, 41 health care professionals, and 72 developers. We sent participants an online survey and asked them to rate how important they considered each item in the guide to be on a scale from 0 to 10. Two rounds were enough to reach consensus. Results: In the first round, almost one-third (n=42) of those invited participated, and half of those (n=24) also participated in the second round. Most items in the guide were found to be important to a quality mHealth-related app; a total of 48 criteria were established as important. ?Privacy,? ?security,? and ?usability? were the categories that included most of the important criteria. Conclusions: The data supports the validity of the MAG. In addition, the findings identified the criteria that stakeholders consider to be most important. The MAG will help advance the field by providing developers, health care professionals, and end users with a valid guide so that they can develop and identify mHealth-related apps that are of quality. UR - http://mhealth.jmir.org/2020/7/e17760/ UR - http://dx.doi.org/10.2196/17760 UR - http://www.ncbi.nlm.nih.gov/pubmed/32735226 ID - info:doi/10.2196/17760 ER - TY - JOUR AU - Jennings Mayo-Wilson, Larissa AU - Glass, E. Nancy AU - Labrique, Alain AU - Davoust, Melissa AU - Ssewamala, M. Fred AU - Linnemayr, Sebastian AU - Johnson, W. Matthew PY - 2020/7/17 TI - Feasibility of Assessing Economic and Sexual Risk Behaviors Using Text Message Surveys in African-American Young Adults Experiencing Homelessness and Unemployment: Single-Group Study JO - JMIR Form Res SP - e14833 VL - 4 IS - 7 KW - HIV KW - sexual risk behaviors KW - homelessness KW - text messages KW - young adults KW - economic KW - mobile phones N2 - Background: Text messages offer the potential to better evaluate HIV behavioral interventions using repeated longitudinal measures at a lower cost and research burden. However, they have been underused in US minority settings. Objective: This study aims to examine the feasibility of assessing economic and sexual risk behaviors using text message surveys. Methods: We conducted a single-group study with 17 African-American young adults, aged 18-24 years, who were economically disadvantaged and reported prior unprotected sex. Participants received a text message survey once each week for 5 weeks. The survey contained 14 questions with yes-no and numeric responses on sexual risk behaviors (ie, condomless sex, sex while high or drunk, and sex exchange) and economic behaviors (ie, income, employment, and money spent on HIV services or products). Feasibility measures were the number of participants who responded to the survey in a given week, the number of questions to which a participant responded in each survey, and the number of hours spent from sending a survey to participants to receiving their response in a given week. One discussion group was used to obtain feedback. Results: Overall, 65% (n=11/17) of the participants responded to at least one text message survey compared with 35% (n=6/17) of the participants who did not respond. The majority (n=7/11, 64%) of the responders were women. The majority (n=4/6, 67%) of nonresponders were men. An average of 7.6 participants (69%) responded in a given week. Response rates among ever responders ranged from 64% to 82% across the study period. The mean number of questions answered each week was 12.6 (SD 2.7; 90% of all questions), ranging from 72% to 100%. An average of 6.4 participants (84%) answered all 14 text message questions in a given week, ranging from 57% to 100%. Participants responded approximately 8.7 hours (SD 10.3) after receiving the survey. Participants were more likely to answer questions related to employment, condomless sex, and discussions with sex partners. Nonresponse or skip was more often used for questions at the end of the survey relating to sex exchange and money spent on HIV prevention services or products. Strengths of the text message survey were convenience, readability, short completion time, having repeated measures over time, and having incentives. Conclusions: Longitudinal text message surveys may be a valuable tool for assessing HIV-related economic and sexual risk behaviors. Trial Registration: ClinicalTrials.gov NCT03237871; https://clinicaltrials.gov/ct2/show/NCT03237871 UR - https://formative.jmir.org/2020/7/e14833 UR - http://dx.doi.org/10.2196/14833 UR - http://www.ncbi.nlm.nih.gov/pubmed/32706656 ID - info:doi/10.2196/14833 ER - TY - JOUR AU - O'Donovan, Rebecca AU - Sezgin, Emre AU - Bambach, Sven AU - Butter, Eric AU - Lin, Simon PY - 2020/6/16 TI - Detecting Screams From Home Audio Recordings to Identify Tantrums: Exploratory Study Using Transfer Machine Learning JO - JMIR Form Res SP - e18279 VL - 4 IS - 6 KW - machine learning KW - scream detection KW - audio event detection KW - tantrum identification KW - autism KW - behavioral disorder KW - data-driven approach N2 - Background: Qualitative self- or parent-reports used in assessing children?s behavioral disorders are often inconvenient to collect and can be misleading due to missing information, rater biases, and limited validity. A data-driven approach to quantify behavioral disorders could alleviate these concerns. This study proposes a machine learning approach to identify screams in voice recordings that avoids the need to gather large amounts of clinical data for model training. Objective: The goal of this study is to evaluate if a machine learning model trained only on publicly available audio data sets could be used to detect screaming sounds in audio streams captured in an at-home setting. Methods: Two sets of audio samples were prepared to evaluate the model: a subset of the publicly available AudioSet data set and a set of audio data extracted from the TV show Supernanny, which was chosen for its similarity to clinical data. Scream events were manually annotated for the Supernanny data, and existing annotations were refined for the AudioSet data. Audio feature extraction was performed with a convolutional neural network pretrained on AudioSet. A gradient-boosted tree model was trained and cross-validated for scream classification on the AudioSet data and then validated independently on the Supernanny audio. Results: On the held-out AudioSet clips, the model achieved a receiver operating characteristic (ROC)?area under the curve (AUC) of 0.86. The same model applied to three full episodes of Supernanny audio achieved an ROC-AUC of 0.95 and an average precision (positive predictive value) of 42% despite screams only making up 1.3% (n=92/7166 seconds) of the total run time. Conclusions: These results suggest that a scream-detection model trained with publicly available data could be valuable for monitoring clinical recordings and identifying tantrums as opposed to depending on collecting costly privacy-protected clinical data for model training. UR - http://formative.jmir.org/2020/6/e18279/ UR - http://dx.doi.org/10.2196/18279 UR - http://www.ncbi.nlm.nih.gov/pubmed/32459656 ID - info:doi/10.2196/18279 ER - TY - JOUR AU - Kedroske, Jacob AU - Koblick, Sarah AU - Chaar, Dima AU - Mazzoli, Amanda AU - O'Brien, Maureen AU - Yahng, Lilian AU - Vue, Rebecca AU - Chappell, Grant AU - Shin, Youn Ji AU - Hanauer, A. David AU - Choi, Won Sung PY - 2020/1/23 TI - Development of a National Caregiver Health Survey for Hematopoietic Stem Cell Transplant: Qualitative Study of Cognitive Interviews and Verbal Probing JO - JMIR Form Res SP - e17077 VL - 4 IS - 1 KW - hematopoietic stem cell transplantation KW - caregivers KW - mobile applications KW - qualitative research N2 - Background: Roadmap 1.0 is a mobile health app that was previously developed for caregivers of patients who have undergone hematopoietic stem cell transplantation (HSCT). Formative research targeted toward its end users (caregivers) can help inform app design and development, allowing additional components to be incorporated into the app, which can then be tested in a future randomized controlled trial. Objective: This study aimed to create a methodologically rigorous national survey that would help inform the development of Roadmap 2.0. Methods: We conducted a prospective, qualitative research study that took place between November 18, 2018, and February 7, 2019, in a blood and marrow transplant unit within a large academic medical institution in the midwestern part of the United States. Cognitive interviews, including think-aloud and verbal probing techniques, were conducted in 10 adult caregivers (?18 years) of patients who had undergone HSCT. Results: Most participants were female (9/10, 90%), white (9/10, 90%), married (9/10, 90%), employed at least part time (6/10, 60%), caregivers of adult patients (7/10, 70%), and had some college education (9/10, 90%) and an annual household income of $60,000 or higher (6/10, 60%). All but one interview was audio-recorded, with permission. Overall, participants were engaged in the cognitive interview process of the draft survey, which included 7 topics. The interviews highlighted areas wherein survey items could be further refined, such as offering more response choices (eg, ?NA?) or clarifying the type of transplant (eg, autologous or allogeneic) or context of transplant care (eg, pre-HSCT, during HSCT, post-HSCT, inpatient, and outpatient). Apart from these findings, the items in demographics, caregiving experiences, technology, positive activities, and mood were generally interpreted as intended. On the basis of the transcript data and field notes by the interviewer, items within self-efficacy (Caregiver Self-Efficacy Scale) and coping (Brief Coping Orientation to Problems Experienced inventory) questionnaires generated more confusion among interviewer and participants, reflecting difficulties in interpreting the meaning of some survey items. Conclusions: This study incorporated the four cognitive aspects of survey methodology that describe the question-answering process?(1) comprehension, (2) information retrieval, (3) judgment and decision making, and (4) responding?by using the think-aloud and probing techniques in cognitive interviews. We conclude that this methodologically rigorous process informed revisions and improved our final questionnaire design. International Registered Report Identifier (IRRID): RR2-10.2196/resprot.49188 UR - http://formative.jmir.org/2020/1/e17077/ UR - http://dx.doi.org/10.2196/17077 UR - http://www.ncbi.nlm.nih.gov/pubmed/32012037 ID - info:doi/10.2196/17077 ER - TY - JOUR AU - Russomanno, Jennifer AU - Patterson, G. Joanne AU - Jabson Tree, M. Jennifer PY - 2019/12/2 TI - Social Media Recruitment of Marginalized, Hard-to-Reach Populations: Development of Recruitment and Monitoring Guidelines JO - JMIR Public Health Surveill SP - e14886 VL - 5 IS - 4 KW - transgender KW - LGBTQ KW - TGNC KW - marginalized populations KW - cyberbullying KW - engagement KW - compassion fatigue KW - human subjects KW - research protections KW - adverse events N2 - Background: Social media can be a useful strategy for recruiting hard-to-reach, stigmatized populations into research studies; however, it may also introduce risks for participant and research team exposure to negative comments. Currently, there is no published formal social media recruitment and monitoring guidelines that specifically address harm reduction for social media recruitment of marginalized populations. Objective: The purpose of this research study was to investigate the utility, successes, challenges, and positive and negative consequences of using targeted Facebook advertisements as a strategy to recruit transgender and gender nonconforming (TGNC) people into a research study. Methods: TGNC adults living in the Southeast Unites States were recruited via targeted Facebook advertisements over two cycles in April and June 2017. During cycle 1, researchers only used inclusion terms to recruit the target population. During cycle 2, the social media recruitment and monitoring protocol and inclusion and exclusion terms were used. Results: The cycle 1 advertisement reached 8518 people and had 188 reactions, comments, and shares but produced cyberbullying, including discriminatory comments from Facebook members. Cycle 2 reached fewer people (6976) and received 166 reactions, comments, and shares but produced mostly positive comments. Conclusions: Researchers must consider potential harms of using targeted Facebook advertisements to recruit hard-to-reach and stigmatized populations. To minimize harm to participants and research staff, researchers must preemptively implement detailed social media recruitment and monitoring guidelines for monitoring and responding to negative feedback on targeted Facebook advertisements. UR - http://publichealth.jmir.org/2019/4/e14886/ UR - http://dx.doi.org/10.2196/14886 UR - http://www.ncbi.nlm.nih.gov/pubmed/31789598 ID - info:doi/10.2196/14886 ER - TY - JOUR AU - Allison, Rosalie AU - Hayes, Catherine AU - McNulty, M. Cliodna A. AU - Young, Vicki PY - 2019/10/24 TI - A Comprehensive Framework to Evaluate Websites: Literature Review and Development of GoodWeb JO - JMIR Form Res SP - e14372 VL - 3 IS - 4 KW - user experience KW - usability KW - human-computer interaction KW - software testing KW - quality testing KW - scoping study N2 - Background: Attention is turning toward increasing the quality of websites and quality evaluation to attract new users and retain existing users. Objective: This scoping study aimed to review and define existing worldwide methodologies and techniques to evaluate websites and provide a framework of appropriate website attributes that could be applied to any future website evaluations. Methods: We systematically searched electronic databases and gray literature for studies of website evaluation. The results were exported to EndNote software, duplicates were removed, and eligible studies were identified. The results have been presented in narrative form. Results: A total of 69 studies met the inclusion criteria. The extracted data included type of website, aim or purpose of the study, study populations (users and experts), sample size, setting (controlled environment and remotely assessed), website attributes evaluated, process of methodology, and process of analysis. Methods of evaluation varied and included questionnaires, observed website browsing, interviews or focus groups, and Web usage analysis. Evaluations using both users and experts and controlled and remote settings are represented. Website attributes that were examined included usability or ease of use, content, design criteria, functionality, appearance, interactivity, satisfaction, and loyalty. Website evaluation methods should be tailored to the needs of specific websites and individual aims of evaluations. GoodWeb, a website evaluation guide, has been presented with a case scenario. Conclusions: This scoping study supports the open debate of defining the quality of websites, and there are numerous approaches and models to evaluate it. However, as this study provides a framework of the existing literature of website evaluation, it presents a guide of options for evaluating websites, including which attributes to analyze and options for appropriate methods. UR - http://formative.jmir.org/2019/4/e14372/ UR - http://dx.doi.org/10.2196/14372 UR - http://www.ncbi.nlm.nih.gov/pubmed/31651406 ID - info:doi/10.2196/14372 ER - TY - JOUR AU - Engler, Kim AU - Ahmed, Sara AU - Lessard, David AU - Vicente, Serge AU - Lebouché, Bertrand PY - 2019/08/02 TI - Assessing the Content Validity of a New Patient-Reported Measure of Barriers to Antiretroviral Therapy Adherence for Electronic Administration in Routine HIV Care: Proposal for a Web-Based Delphi Study JO - JMIR Res Protoc SP - e12836 VL - 8 IS - 8 KW - HIV KW - antiretroviral therapy, highly active KW - patient-reported outcome measure KW - medication adherence KW - Delphi technique KW - stakeholder participation KW - Canada KW - France N2 - Background: Adherence to lifesaving antiretroviral therapy (ART) for HIV infection remains a challenge for many patients. Routine screening for barriers to ART adherence could help make HIV care more patient-centered and prevent virologic rebound or failure. Our team is currently developing a new HIV-specific patient-reported outcome measure (PROM) of these barriers for use in Canada and France along with a digital app for its electronic administration. In our previous work, we developed the PROM?s multidimensional conceptual framework and generated 100 English items, which have been translated to French. Objective: This study aims to use a Web-based Delphi to help validate and select the content of this new HIV-specific PROM, based on the perspective of anglophone and francophone patients and providers in Canada and France. Here, we present the proposal for this Delphi. Methods: This modified Delphi will involve a diverse panel of patients (n=32) and providers (n=52) recruited especially from the 9 sites of the PROM development study (site locations in Canada: Montreal, Toronto, Vancouver; in France: Paris, Nantes, Clermont-Ferrand, Saint-Martin, Cayenne). Overall, 2 rounds of Web-based questionnaires will be conducted. The threshold for consensus is set at 60% and will determine which items are carried forward to the second round. Per item, 3 aspects will be rated: importance as a barrier to ART adherence, relevance for HIV care, and clarity. In both rounds, space will be available for free text comments. Overall comprehensiveness will be assessed in the second round. Results: This study has undergone a methodological review by experts in patient-oriented research. It has received approval from a research ethics board of the McGill University Health Centre. It is financially supported, in part, by the Canadian Institutes of Health Research?s Strategy for Patient-Oriented Research-Quebec Support Unit (M006). As of May 21, 2019, 15 people living with HIV and 25 providers completed the first round of the Delphi (24 from Canada and 16 from France). Conclusions: To our knowledge, this is the first Delphi to seek consensus on the most relevant and clinically actionable barriers to ART adherence, collecting opinions on an extensive list of barriers. Drawing on a relatively large and diverse panel of HIV patients and providers, it essentially engages key stakeholders in decision making about the PROM?s final content, helping to ensure its utility and adoption. International Registered Report Identifier (IRRID): PRR1-10.2196/12836 UR - https://www.researchprotocols.org/2019/8/e12836/ UR - http://dx.doi.org/10.2196/12836 UR - http://www.ncbi.nlm.nih.gov/pubmed/31376275 ID - info:doi/10.2196/12836 ER - TY - JOUR AU - Vidal-Alaball, Josep AU - Fernandez-Luque, Luis AU - Marin-Gomez, X. Francesc AU - Ahmed, Wasim PY - 2019/05/28 TI - A New Tool for Public Health Opinion to Give Insight Into Telemedicine: Twitter Poll Analysis JO - JMIR Form Res SP - e13870 VL - 3 IS - 2 KW - telemedicine KW - Twitter messaging KW - health care surveys N2 - Background: Telemedicine draws on information technologies in order to enable the delivery of clinical health care from a distance. Twitter is a social networking platform that has 316 million monthly active users with 500 million tweets per day; its potential for real-time monitoring of public health has been well documented. There is a lack of empirical research that has critically examined the potential of Twitter polls for providing insight into public health. One of the benefits of utilizing Twitter polls is that it is possible to gain access to a large audience that can provide instant and real-time feedback. Moreover, Twitter polls are completely anonymized. Objective: The overall aim of this study was to develop and disseminate Twitter polls based on existing surveys to gain real-time feedback on public views and opinions toward telemedicine. Methods: Two Twitter polls were developed utilizing questions from previously used questionnaires to explore acceptance of telemedicine among Twitter users. The polls were placed on the Twitter timeline of one of the authors, which had more than 9300 followers, and the account followers were asked to answer the poll and retweet it to reach a larger audience. Results: In a population where telemedicine was expected to enjoy big support, a significant number of Twitter users responding to the poll felt that telemedicine was not as good as traditional care. Conclusions: Our results show the potential of Twitter polls for gaining insight into public health topics on a range of health issues not just limited to telemedicine. Our study also sheds light on how Twitter polls can be used to validate and test survey questions. UR - http://formative.jmir.org/2019/2/e13870/ UR - http://dx.doi.org/10.2196/13870 UR - http://www.ncbi.nlm.nih.gov/pubmed/31140442 ID - info:doi/10.2196/13870 ER - TY - JOUR AU - Mazor, M. Kathleen AU - King, M. Ann AU - Hoppe, B. Ruth AU - Kochersberger, O. Annie AU - Yan, Jie AU - Reim, D. Jesse PY - 2019/02/14 TI - Video-Based Communication Assessment: Development of an Innovative System for Assessing Clinician-Patient Communication JO - JMIR Med Educ SP - e10400 VL - 5 IS - 1 KW - communication KW - crowdsourcing KW - health care KW - mobile phone KW - patient-centered care KW - video-based communication assessment UR - http://mededu.jmir.org/2019/1/e10400/ UR - http://dx.doi.org/10.2196/10400 UR - http://www.ncbi.nlm.nih.gov/pubmed/30710460 ID - info:doi/10.2196/10400 ER - TY - JOUR AU - Allen, M. Alicia AU - Lundeen, Kim AU - Murphy, E. Sharon AU - Spector, Logan AU - Harlow, L. Bernard PY - 2018/11/05 TI - Web-Delivered Multimedia Training Materials for the Self-Collection of Dried Blood Spots: A Formative Project JO - JMIR Formativ Res SP - e11025 VL - 2 IS - 2 KW - dried blood spot KW - internet KW - feasibility studies N2 - Background: The use of dried blood spots (DBS) in biomedical research has been increasing as an objective measure for variables that are typically plagued by self-report, such as smoking status and medication adherence. The development of training materials for the self-collection of DBS that can be delivered through the Web would allow for broader use of this methodology. Objective: The objective of this study was to evaluate the acceptability and feasibility of the self-collection of DBS using newly developed multimedia training materials that were delivered through the Web. We also aimed to assess the usability of the collected DBS samples. Methods: We recruited participants through Facebook advertising for two distinct studies. The first study evaluated the acceptability of our newly developed DBS training materials, while the second assessed the implementation of this protocol into a larger Web-based study. Results: In the first study, participants (N=115) were aged, on average, 26.1 (SD 6.4) years. Training materials were acceptable (113/115, 98.2%, of participants were willing to collect DBS again) and produced usable samples (110/115, 95.7%, collected DBS were usable). In the second study, response rate was 25.0% (41/164), with responders being significantly younger than nonresponders (20.3 [SD 0.2] vs 22.0 [SD 0.4]; P<.001), and 92% (31/41) of collected DBS samples were usable by the laboratory. Conclusions: Overall, while the protocol is acceptable, feasible, and produced usable samples, additional work is needed to improve response rates. UR - http://formative.jmir.org/2018/2/e11025/ UR - http://dx.doi.org/10.2196/11025 UR - http://www.ncbi.nlm.nih.gov/pubmed/30684406 ID - info:doi/10.2196/11025 ER - TY - JOUR AU - van Velsen, Lex AU - Evers, Mirka AU - Bara, Cristian-Dan AU - Op den Akker, Harm AU - Boerema, Simone AU - Hermens, Hermie PY - 2018/06/15 TI - Understanding the Acceptance of an eHealth Technology in the Early Stages of Development: An End-User Walkthrough Approach and Two Case Studies JO - JMIR Formativ Res SP - e10474 VL - 2 IS - 1 KW - eHealth KW - acceptance KW - design KW - walkthrough KW - agile design N2 - Background: Studies that focus on the acceptance of an electronic health (eHealth) technology generally make use of surveys. However, results of such studies hold little value for a redesign, as they focus only on quantifying end-user appreciation of general factors (eg, perceived usefulness). Objective: We present a method for understanding end-user acceptance of an eHealth technology, early in the development process: The eHealth End-User Walkthrough. Methods: During a walkthrough, a participant is guided by using the technology via a scenario, a persona, and a low-fidelity protoype. A participant is questioned about factors that may affect acceptance during and after the demonstration. We show the value of the method via two case studies. Results: During the case studies, participants commented on whether they intend to use a technology and why they would (not) use its main features. They also provided redesign advice or input for additional functions. Finally, the sessions provide guidance for the generation of business models and implementation plans. Conclusions: The eHealth End-User Walkthrough can aid design teams in understanding the acceptance of their eHealth application in a very early stage of the design process. Consequently, it can prevent a mismatch between technology and end-users? needs, wishes and context. UR - http://formative.jmir.org/2018/1/e10474/ UR - http://dx.doi.org/10.2196/10474 UR - http://www.ncbi.nlm.nih.gov/pubmed/30684434 ID - info:doi/10.2196/10474 ER - TY - JOUR AU - Oremus, Mark AU - Sharafoddini, Anis AU - Morgano, Paolo Gian AU - Jin, Xuejing AU - Xie, Feng PY - 2018/01/23 TI - A Computer-Assisted Personal Interview App in Research Electronic Data Capture for Administering Time Trade-off Surveys (REDCap): Development and Pretest JO - JMIR Formativ Res SP - e3 VL - 2 IS - 1 KW - computer-assisted personal interview KW - health-related quality-of-life KW - REDCap KW - time trade-off N2 - Background: The time trade-off (TTO) task is a method of eliciting health utility scores, which range from 0 (equivalent to death) to 1 (equivalent to perfect health). These scores numerically represent a person?s health-related quality of life. Software apps exist to administer the TTO task; however, most of these apps are poorly documented and unavailable to researchers. Objective: To fill the void, we developed an online app to administer the TTO task for a research study that is examining general public proxy health-related quality of life estimates for persons with Alzheimer?s disease. This manuscript describes the development and pretest of the app. Methods: We used Research Electronic Data Capture (REDCap) to build the TTO app. The app?s modular structure and REDCap?s object-oriented environment facilitated development. After the TTO app was built, we recruited a purposive sample of 11 members of the general public to pretest its functionality and ease of use. Results: Feedback from the pretest group was positive. Minor modifications included clarity enhancements, such as rearranging some paragraph text into bullet points, labeling the app to delineate different question sections, and revising or deleting text. We also added a research question to enable the identification of respondents who know someone with Alzheimer?s disease. Conclusions: We developed an online app to administer the TTO task. Other researchers may access and customize the app for their own research purposes. UR - http://formative.jmir.org/2018/1/e3/ UR - http://dx.doi.org/10.2196/formative.8202 UR - http://www.ncbi.nlm.nih.gov/pubmed/30684429 ID - info:doi/10.2196/formative.8202 ER -