This paper is in the following e-collection/theme issue:

Development and Evaluation of Research Methods, Instruments and Tools (422) Innovations in Mental Health Systems (370) Substance Abuse (379) Patient-Reported Outcome Measures (PROMs) (274) Development of Instruments and Surveys (79) Smoking Cessation (514) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (457) Research Instruments, Questionnaires, and Tools (540)

Published on in Vol 8 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52801, first published .
Feasibility of a 2-Part Substance Use Screener Self-Administered by Patients on Paper: Observational Study

Feasibility of a 2-Part Substance Use Screener Self-Administered by Patients on Paper: Observational Study

Feasibility of a 2-Part Substance Use Screener Self-Administered by Patients on Paper: Observational Study

Authors of this article:

Joanna Kramer1 Author Orcid Image ;   Timothy E Wilens2 Author Orcid Image ;   Vinod Rao2 Author Orcid Image ;   Richard Villa1 Author Orcid Image ;   Amy M Yule1 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Original Paper

    1Department of Psychiatry, Boston Medical Center, Boston, MA, United States

    2Department of Psychiatry, Massachusetts General Hospital, Boston, MA, United States

    Corresponding Author:

    Amy M Yule, MD

    Department of Psychiatry

    Boston Medical Center

    Crosstown Building 408

    801 Massachusetts Avenue

    Boston, MA, 02118

    United States

    Phone: 1 6174141936

    Email: amy.yule@bmc.org


    Background: Measurement-based care in behavioral health uses patient-reported outcome measures (PROMs) to screen for mental health symptoms and substance use and to assess symptom change over time. While PROMs are increasingly being integrated into electronic health record systems and administered electronically, paper-based PROMs continue to be used. It is unclear if it is feasible to administer a PROM on paper when the PROM was initially developed for electronic administration.

    Objective: This study aimed to examine the feasibility of patient self-administration of a 2-part substance use screener—the Tobacco, Alcohol, Prescription medications, and other Substances (TAPS)—on paper. This screener was originally developed for electronic administration. It begins with a limited number of questions and branches to either skip or reflex to additional questions based on an individual’s responses. In this study, the TAPS was adapted for paper use due to barriers to electronic administration within an urgent care behavioral health clinic at an urban health safety net hospital.

    Methods: From August 2021 to March 2022, research staff collected deidentified paper TAPS responses and tracked TAPS completion rates and adherence to questionnaire instructions. A retrospective chart review was subsequently conducted to obtain demographic information for the patients who presented to the clinic between August 2021 and March 2022. Since the initial information collected from TAPS responses was deidentified, demographic information was not linked to the individual TAPS screeners that were tracked by research staff.

    Results: A total of 507 new patients were seen in the clinic with a mean age of 38.7 (SD 16.6) years. In all, 258 (50.9%) patients were male. They were predominantly Black (n=212, 41.8%), White (n=152, 30%), and non-Hispanic or non-Latino (n=403, 79.5%). Most of the patients were publicly insured (n=411, 81.1%). Among these 507 patients, 313 (61.7%) completed the TAPS screener. Of these 313 patients, 76 (24.3%) adhered to the instructions and 237 (75.7%) did not follow the instructions correctly. Of the 237 respondents who did not follow the instructions correctly, 166 (70%) answered more questions and 71 (30%) answered fewer questions than required in TAPS part 2. Among the 237 patients who did not adhere to questionnaire instructions, 44 (18.6%) responded in a way that contradicted their response in part 1 of the screener and ultimately affected their overall TAPS score.

    Conclusions: It was challenging for patients to adhere to questionnaire instructions when completing a substance use screener on paper that was originally developed for electronic use. When selecting PROMs for measurement-based care, it is important to consider the structure of the questionnaire and how the PROM will be administered to determine if additional support for PROM self-administration needs to be implemented.

    JMIR Form Res 2024;8:e52801

    doi:10.2196/52801

    Keywords

    patient reported outcome measures; patient reported outcomes; substance use screening; paper and pencil screening; screening; tobacco; prescription medication; medication; substance use; care; mental health; symptoms 


    Measurement-based care (MBC) is a practice in clinical care that uses patient-reported outcome measures (PROMs) to identify individuals at risk for a disorder, quantify symptoms, and monitor symptoms over time [1]. PROMs can be self-administered and completed by patients on paper or electronically on a device or web-based platform [2]. Although paper-based PROMs are quite accessible, namely, because they do not require modifications to the electronic health record (EHR) to facilitate electronic administration [3], internet access, or technological literacy [2], there has been a shift within health care toward electronic administration of PROMs [3]. The shift toward electronic self-administration has followed the large-scale adoption of EHRs [4] since these systems can identify patients who are due to complete PROMs, streamline administration, improve ease of access to patient-reported outcome (PRO) responses for clinical decision-making, and monitor quality at the clinic and health care system levels [5,6]. Indeed, the integration of PROM into the EHR addresses important implementation barriers to MBC in clinic practice since PRO responses are immediately available to clinicians during the visit, results are easily interpreted within systems that automatically score responses, and results can be monitored within the EHR over time [6].

    As electronic administration of PROMs became more common, new substance use screening questionnaires were specifically developed and validated for electronic administration [7-10]. To minimize the burden associated with completing a PROM, these questionnaires begin with a limited number of questions and skip or reflex to additional questions based on an individual’s initial responses [7-10]. One of these 2-part questionnaires—the Tobacco, Alcohol, Prescription medications, and other Substance (TAPS) screener—was also successfully integrated into the EHR and implemented in the primary care setting with electronic self-administration on tablets [11].

    While electronic administration of PROMs that is automatically integrated with the EHR may ease patient and staff burdens associated with MBC, this system of screening may not be accessible in all health care settings. This type of system requires an information technology team that can help build and maintain the electronic PRO system, which can take a significant amount of time and expense and can also lead to recurring technological challenges once implemented [2,12,13]. In health safety net hospitals or federally qualified health centers, there may be fewer resources to incorporate the necessary information technology changes needed to systematically screen patients using electronic devices integrated with the EHR [14]. With this in mind, the most feasible screening option in these settings may be to continue the self-administration of PROMs on paper.

    However, very little is known about the feasibility of administering a PROM on paper that was specifically developed to be administered electronically. We, therefore, aimed to examine questionnaire completion rates and adherence to questionnaire instructions when new patients completed the 2-part substance use TAPS screener on paper in an outpatient behavioral health clinic within a health safety net hospital. The clinic chose to use the TAPS screener because they were evaluating individuals at high risk for a co-occurring substance use disorder and needed a PROM that broadly assessed for multiple substances within 1 questionnaire to inform the initial treatment plan and referrals.


    Overview

    New patients aged 18 years and older presenting to an urgent care behavioral health clinic within an urban, public safety net hospital between August 2021 and March 2022 were given a paper packet by the front desk staff when they checked in for their appointment. This referral-based clinic provided urgent behavioral health services to adults experiencing a mental health crisis or needing medication to treat a psychiatric disorder or substance use disorder. The paper packet is part of standard care in this clinic and contains a 1-page intake form and 3 PROMs screeners assessing depression (the Patient Health Questionnaire-9) [15], anxiety (the Generalized Anxiety Disorder-7) [16], and substance use (TAPS) [7] to complete prior to their appointment. PROMs were administered on paper because there were no resources in this setting to administer PROMs electronically and immediately provide clinicians with the questionnaire results.

    Research on survey design and presentation of questions on paper was used to derive the paper version of the TAPS from the web-based version [17] (see also Multimedia Appendix 1). The TAPS has 2 parts: part 1 of the TAPS assesses tobacco use, alcohol use, prescription medication misuse, and illicit substance use in the past 12 months. When use of a substance is endorsed, up to 4 additional yes or no questions are asked in part 2 for each substance endorsed to assess for problems associated with use in the past 3 months (up to 27 questions in total). In part 2, a yes response is scored a 1 and a no response is scored a 0. A score for each substance assessed in part 2 is calculated with corresponding categories of no risk (total score 0), problem use (total score 1), or higher risk (total score ≥2). To mimic the electronic questionnaire branching logic that either skips or reflexes to additional questions based on responses in part 1, written instructions were provided that indicated which questions respondents should answer in part 2 based on their response to the questions in part 1.

    The front desk staff instructed patients to complete the packet and to hand it to their clinician when they were called into their appointment. The staff provided these instructions in English and Spanish, and the packets were provided in both languages. The goal of having patients complete the packet prior to their appointment was for their clinician to review the PRO responses and use this information during the appointment to inform the initial treatment plan and referrals. Following the appointment, clinicians would return the packet to a designated shelf in the waiting room for the research staff to collect. The research staff reviewed returned packets and scored them if this had not already been done by a clinician. The research staff maintained a deidentified database that detailed the completion and error rates of those who had completed some, all, or none of the PROMs screeners. A screener was considered incomplete if 1 or more questions were skipped.

    More detailed information was collected regarding completion patterns for the TAPS screener. This included whether individuals accurately completed the TAPS according to the instructions and whether or not their TAPS score was impacted based on how they responded. For example, if someone answered “never” to the use of a substance in part 1, no follow-up part 2 questions needed to be answered for that substance. However, when the TAPS was completed on paper, respondents saw all of the potential part 2 questions. Examples of completion patterns where the instructions were not followed and the score was not impacted (Figure 1), as well as when the score was impacted (Figure 2), are shown.

    ‎
    Figure 1. The questionnaire was not completed according to instructions because the respondent answered more questions than needed. However, their score was not impacted since they answered the TAPS part 2 questions (1a through 1c) in a consistent manner to TAPS part 1 (question 1). TAPS: Tobacco, Alcohol, Prescription medications, and other Substance.
    ‎
    Figure 2. The questionnaire was not completed according to instructions because the respondent answered more questions than needed. Their score was impacted when they answered these additional questions since they answered the TAPS part 2 questions (1a through 1c) in an inconsistent manner compared to TAPS part 1 (question 1). TAPS: Tobacco, Alcohol, Prescription medications, and other Substance.

    Since the deidentified database created to track PROMs completion and adherence to questionnaire instructions did not include patient demographic information, a retrospective EHR chart review was conducted to collect this information. Demographic information for all new patients who were seen in the clinic while the PROMs were tracked was extracted from the medical record and included age, race, ethnicity, sex, and the type of insurance. These demographic variables were classified according to how these items were categorized in the EHR.

    Ethical Considerations

    The Boston University Medical Campus or Boston Medical Center institutional review board approved this research and deemed this study exempt and not requiring informed consent, as all information was deidentified and there was no direct interaction with human participants (H-43045).


    A total of 507 new patients were seen in the clinic between August 2021 and March 2022. These patients had a mean age of 38.7 (SD 16.6) years, and 258 (50.9%) were male. They were predominantly Black (n=212, 41.8%), White (n=152, 30.0%), and non-Hispanic or non-Latino (n=403, 79.5%). Most of the patients were publicly insured (n=411, 81.1%; Table 1).

    Table 1. Demographic characteristics of new patients seen in an urgent care behavioral health clinic between August 2021 and March 2022, when observational data were collected on the completion patterns for the 2-part TAPSa questionnaire when self-administered on paper to patients (N=507).
    Demographic characteristicsValues
    Sex, n (%)

    		Male258 (50.9)

    		Female248 (48.9)

    		Unknown1 (0.2)
    Race, n (%)

    		Asian25 (4.9)

    		Black or African American212 (41.8)

    		Native Hawaiian or Pacific Islander1 (0.2)

    		White152 (30)

    		Multiple11 (2.2)

    		Declined or not available29 (5.7)

    		Other (including Hispanic or Latino)77 (15.2)
    Ethnicity, n (%)

    		Not Hispanic or Latino403 (79.5)

    		Hispanic or Latino96 (18.9)

    		Unknown8 (1.6)
    Age (years), mean (SD)38.7 (14.6)
    Insurance, n (%)

    		Public411 (81.1)

    		Private82 (16.2)

    		Uninsured5 (1)

    		Unknown9 (1.8)

    aTAPS: Tobacco, Alcohol, Prescription medications, and other Substance.

    Out of the 507 new patients, 353 (69.6%) returned their paper packets, and 313 (61.7%) completed some or all of the TAPS screener. Among the 313 patients who completed some or all of the TAPS, 76 (24.3%) completed the full screener accurately according to the instructions on the paper form (Figure 3). Of the 237 individuals who did not complete the TAPS paper form according to the instructions, 166 (70.0%) answered more questions than required, and 71 (30.0%) did not answer required questions.

    ‎
    Figure 3. Observational data on the completion patterns for the 2-part TAPS questionnaire when self-administered on paper. TAPS: Tobacco, Alcohol, Prescription medications, and other Substance.

    Out of the 166 individuals who answered more questions than required in TAPS part 2, most (n=128, 77.1%) did not overanswer in a way that affected their TAPS score (ie, responding “Never” to the part 1 question and then “Never” to the part 2 follow-up questions). However, 38 (22.9%) of those who answered more questions than needed responded to follow-up questions in TAPS part 2 in a way that contradicted their response in part 1 of the screener and ultimately affected their overall TAPS score.

    Of the 71 patients who skipped needed questions on the paper TAPS form, 65 (91.5%) of these respondents did not have their TAPS score impacted by the questions they did respond to. For these individuals, the TAPS was not fully complete. However, for the questions they did answer, they answered more follow-up questions than needed in TAPS part 2, but not in a way that was contradictory to the questions they did respond to in TAPS part 1. There were 6 (8.5%) individuals who did not complete the entire TAPS, but whose score was impacted by the questions they did complete. Each of these individuals, for the questions they did answer, answered more follow-up questions than needed in TAPS part 2 and responded in a way that contradicted their part 1 TAPS response.


    The goal of this study was to examine the feasibility of administering the 2-part TAPS questionnaire, originally created for electronic administration [7], on paper for patient self-administration. Many of the new patients who came to the outpatient behavioral health clinic for urgent evaluation completed the screener (313/507, 61.7%). The majority of these patients (237/313, 75.7%) did not adhere to the questionnaire instructions and answered more or less of the questions required. Furthermore, some of the patients who did not adhere to the questionnaire instructions (44/237, 18.6%) provided responses that contradicted their initial responses which subsequently affected their overall score and the corresponding category of level of risk associated with substance use.

    Our finding that many patients in an outpatient behavioral health clinic completed the substance use PROM is similar to the existing literature on substance use screening in the behavioral health setting. A recent systematic review examined this topic and found that substance use screening rates in adult behavioral health clinics ranged from 48% to 100% [18]. Most of the studies in this review did not report on the method of screening administration, but the 1 study with paper administration reported screening 74.9% of patients [19]. Furthermore, our paper TAPS screening rate of 61.7% (313/507) is similar to the rate of electronic TAPS screening reported in primary care (n=67,042, 72%) [20].

    Our finding that patients did not adhere to questionnaire instructions to either skip or answer additional questions is consistent with the broader literature that has shown errors of commission and omission is common when questionnaires that include skip instructions are self-administered on paper [21]. Similar to our findings, another study that reported on adherence to questionnaire instructions when individuals were asked questions about alcohol use, illicit drug use, and other sensitive health behaviors on paper, also found that it was more common for respondents to answer questions that they did not need to answer and less common for respondents to answer questions in a contradictory way [22].

    Although it is unclear why patients in our sample did not follow the questionnaire instructions when self-administering a 2-part PROM on paper, other research suggests that health literacy may influence accurate completion of self-administered PROMs. Health literacy refers to one’s ability to use and comprehend information in a way that is beneficial to their health [23]. Those with lower health literacy may have a more difficult time understanding and communicating health-related needs [23]. For example, Al-Tayyib et al [22] found that individuals who scored the lowest on health literacy, when compared to those who scored highest, were 8 times more likely to answer questions about alcohol in a contradictory way when the questionnaire was self-administered on paper. Furthermore, Porter et al [24] observed that participants with low health literacy found it more burdensome to answer questions related to their health on paper when compared to participants with high health literacy. Since safety net hospitals typically serve marginalized or underserved populations [25], which generally experience lower health literacy rates than nonmarginalized populations [26], future work should collect information from patients in this setting while they complete the 2-part PROM to elicit feedback on the design and format of the paper questionnaire. This feedback will be important to guide strategies to support paper administration of a 2-part PROM that ensures accurate questionnaire completion and minimizes the burden to patients.

    This study has a number of methodological limitations. Although research staff used a standardized form to track PROMs completion that allowed for free-text responses to describe patterns observed, they did not systematically categorize the types of errors made in TAPS screener completion until clear patterns began to be observed. Additionally, it is unknown why some individuals did not respond to the TAPS screener. Since demographic data for patients who completed the paper TAPS were not documented on the tracker, the study team is also missing potentially meaningful demographic trends in those who did and did not complete the paper TAPS. Further, it is evident from previous literature that health literacy plays a role in how individuals complete self-administered health screeners on paper, and literacy was not assessed in this study. This study was also conducted in 1 unique behavioral health urgent care setting and, therefore, might not be generalizable to other behavioral health settings or primary care.

    Despite these limitations, our data support the feasibility of screening for substance use on paper in the outpatient behavioral health setting. Our data also highlight the challenge of poor adherence to questionnaire instructions when administering a 2-part substance use screener originally developed for electronic administration in a health safety net hospital setting. In the future, it is important to consider this challenge when adapting 2-part electronic screening questionnaires to paper and to implement strategies that minimize patient burden and ensure that accurate information is collected to inform an individual’s treatment plan. For 2-part screeners like the TAPS, it may be that part 2 needs to be completed with trained staff after a patient self-administers part 1.

    Acknowledgments

    The authors appreciate the support of Laura Milan-Melo and Laura Morales (Boston Medical Center) for their help initiating data collection. The National Institutes of Health Helping to End Addiction Long-term Initiative (4UH3DA050252–02) supported this study.

    Data Availability

    The data sets generated and analyzed during this study are available from the corresponding author on reasonable request.

    Conflicts of Interest

    AY has research funding from the National Institutes of Health (NIH), the Doris Duke Charitable Foundation’s COVID-19 Fund to Retain Clinical Scientists collaborative grant program through support from the John Templeton Foundation, and the National Center for Advancing Translational Sciences (NCATS)/NIH through the Boston University Center for Clinical and Translational Science Institute. She also has funding from the Jack Satter Foundation for clinical program development. She is a consultant to the Gavin House and BayCove Human Services (clinical services). TW has received research funding from the NIH through the Helping to End Addiction Long-term (HEAL) Initiative under awards 1UG3DA050252-01 and 4UH3DA050252-02. TW works as a consultant for Ironshore, the Gavin Foundation, Bay Cove Human Services, the US National Football League, the US Minor and Major League for Baseball, and White Rhino/3D Therapy LLC. TW also serves as a coeditor for the journal Elsevier Psychiatric Clinics of North America (ADHD). In addition, he has published the book Straight Talk About Psychiatric Medications for Kids with the Guilford Press and coedited the textbook ADHD in Adults and Children with the Cambridge University Press. JK, VR, and RV have nothing to disclose.

    Multimedia Appendix 1

    Paper version of the two-part Tobacco, Alcohol, Prescription Medication, and Other Substance (TAPS) questionnaire that was used. Observational data were collected on the completion patterns for this questionnaire when self-administered on paper to patients in an outpatient behavioral health clinic within a health safety net hospital.

    PDF File (Adobe PDF File), 787 KB
    1. Scott K, Lewis CC. Using measurement-based care to enhance any treatment. Cogn Behav Pract. 2015;22(1):49-59. [FREE Full text] [CrossRef] [Medline]
    2. Mercieca-Bebber R, King MT, Calvert MJ, Stockler MR, Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Relat Outcome Meas. 2018;9:353-367. [FREE Full text] [CrossRef]
    3. Jensen RE, Rothrock NE, DeWitt EM, Spiegel B, Tucker CA, Crane HM, et al. The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care. 2015;53(2):153-159. [FREE Full text] [CrossRef] [Medline]
    4. Adler-Milstein J, Jha AK. HITECH Act drove large gains in hospital electronic health record adoption. Health Aff. 2017;36(8):1416-1422. [CrossRef] [Medline]
    5. Silveira Bianchim M, Crane E, Jones A, Neukirchinger B, Roberts G, Mclaughlin L, et al. The implementation, use and impact of patient reported outcome measures in value-based healthcare programmes: a scoping review. PLoS One. 2023;18(12):e0290976. [FREE Full text] [CrossRef] [Medline]
    6. Gensheimer SG, Wu AW, Snyder CF, PRO-EHR Users’ Guide Steering Group, PRO-EHR Users’ Guide Working Group. Oh, the places we'll go: patient-reported outcomes and electronic health records. Patient. 2018;11(6):591-598. [FREE Full text] [CrossRef] [Medline]
    7. McNeely J, Wu LT, Subramaniam G, Sharma G, Cathers LA, Svikis D, et al. Performance of the Tobacco, Alcohol, Prescription Medication, and Other Substance Use (TAPS) tool for substance use screening in primary care patients. Ann Intern Med. 2016;165(10):690-699. [FREE Full text] [CrossRef] [Medline]
    8. Levy SJL, Williams JF, Committee on Substance Use and Prevention. Substance use screening, brief intervention, and referral to treatment. Pediatrics. 2016;138(1):e20161211. [FREE Full text] [CrossRef] [Medline]
    9. Levy S, Weiss R, Sherritt L, Ziemnik R, Spalding A, Van Hook S, et al. An electronic screen for triaging adolescent substance use by risk levels. JAMA Pediatr. 2014;168(9):822-828. [FREE Full text] [CrossRef] [Medline]
    10. Kelly SM, Gryczynski J, Mitchell SG, Kirk A, O'Grady KE, Schwartz RP. Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use. Pediatrics. 2014;133(5):819-826. [FREE Full text] [CrossRef] [Medline]
    11. Adam A, Schwartz RP, Wu L, Subramaniam G, Laska E, Sharma G, et al. Electronic self-administered screening for substance use in adult primary care patients: feasibility and acceptability of the tobacco, alcohol, prescription medication, and other substance use (myTAPS) screening tool. Addict Sci Clin Pract. 2019;14(1):39. [FREE Full text] [CrossRef] [Medline]
    12. Horn ME, Reinke EK, Mather RC, O’Donnell JD, George SZ. Electronic health record–integrated approach for collection of patient-reported outcome measures: a retrospective evaluation. BMC Health Serv Res. 2021;21(1):626. [CrossRef]
    13. Hyland CJ, Mou D, Virji AZ, Sokas CM, Bokhour B, Pusic AL, et al. How to make PROMs work: qualitative insights from leaders at United States hospitals with successful PROMs programs. Qual Life Res. 2023;32(8):2259-2269. [FREE Full text] [CrossRef] [Medline]
    14. Lewis CC, Boyd M, Puspitasari A, Navarro E, Howard J, Kassab H, et al. Implementing measurement-based care in behavioral health: a review. JAMA Psychiatry. 2019;76(3):324-335. [FREE Full text] [CrossRef] [Medline]
    15. Beard C, Hsu KJ, Rifkin LS, Busch AB, Björgvinsson T. Validation of the PHQ-9 in a psychiatric sample. J Affect Disord. 2016;193:267-273. [CrossRef] [Medline]
    16. Spitzer RL, Kroenke K, Williams JBW, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-1097. [CrossRef] [Medline]
    17. McColl E, Jacoby A, Thomas L, Soutter J, Bamford C, Steen N, et al. Design and use of questionnaires: a review of best practice applicable to surveys of health service staff and patients. Health Technol Assess. 2001;5(31):1-256. [FREE Full text] [CrossRef] [Medline]
    18. Woodward D, Wilens TE, Glantz M, Rao V, Burke C, Yule AM. A systematic review of substance use screening in outpatient behavioral health settings. Addict Sci Clin Pract. 2023;18(1):18. [FREE Full text] [CrossRef] [Medline]
    19. Karno M, Granholm E, Lin A. Factor structure of the Alcohol Use Disorders Identification Test (AUDIT) in a mental health clinic sample. J Stud Alcohol. 2000;61(5):751-758. [CrossRef] [Medline]
    20. McNeely J, Adam A, Rotrosen J, Wakeman SE, Wilens TE, Kannry J, et al. Comparison of methods for alcohol and drug screening in primary care clinics. JAMA Netw Open. 2021;4(5):e2110721. [FREE Full text] [CrossRef] [Medline]
    21. Redline C, Dillman D, Smiley R, Carley-Baxter L, Jackson A. Making visible the invisible: an experiment with skip instructions on paper questionnaires. United States Census Bureau; Mar 07, 2000. URL: https://www.census.gov/library/working-papers/2000/adrm/sm00-02.html [accessed 2024-03-24]
    22. Al-Tayyib AA, Rogers SM, Gribble JN, Villarroel M, Turner CF. Effect of low medical literacy on health survey measurements. Am J Public Health. Sep 2002;92(9):1478-1480. [CrossRef] [Medline]
    23. Tavousi M, Haeri-Mehrizi A, Rakhshani F, Rafiefar S, Soleymanian A, Sarbandi F, et al. Development and validation of a short and easy-to-use instrument for measuring health literacy: the Health Literacy Instrument for Adults (HELIA). BMC Public Health. 2020;20(1):656. [FREE Full text] [CrossRef] [Medline]
    24. Porter SC, Guo C, Bacic J, Chan E. Health literacy and task environment influence parents' burden for data entry on child-specific health information: randomized controlled trial. J Med Internet Res. 2011;13(1):e13. [FREE Full text] [CrossRef] [Medline]
    25. Sisodia R, Rodriguez J, Sequist T. Digital disparities: lessons learned from a patient reported outcomes program during the COVID-19 pandemic. J Am Med Inform Assoc. 1, 2021;28(10):2265-2268. [FREE Full text] [CrossRef] [Medline]
    26. Schillinger D. Social determinants, health literacy, and disparities: intersections and controversies. Health Lit Res Pract. 2021;5(3):e234-e243. [FREE Full text] [CrossRef] [Medline]

    ‎
    EHR: electronic health record
    MBC: measurement-based care
    PRO: patient-reported outcome
    PROM: patient-reported outcome measure
    TAPS: Tobacco, Alcohol, Prescription medications, and other Substance

    Edited by A Mavragani; submitted 15.09.23; peer-reviewed by T McPherson, D Barger, Shu-Yi Wang; comments to author 22.02.24; revised version received 26.03.24; accepted 08.05.24; published 25.06.24.

    Copyright

    ©Joanna Kramer, Timothy E Wilens, Vinod Rao, Richard Villa, Amy M Yule. Originally published in JMIR Formative Research (https://formative.jmir.org), 25.06.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.