Assessment of Technology Usage and Literacy From a Patient Tech Testing Panel in an Adult Primary Care Practice: Cross-Sectional Study

Introduction

Technology innovations are reshaping health care delivery [1]. To support effective and usable technology innovations in clinical practice, tech testing panels (TTPs) are often included. TTPs are structured, team-based spaces where clinicians and patients can test early versions of digital health interventions, identify potential issues, and offer solutions [2].

There is limited literature on TTP composition. TTPs with patient diversity by sociodemographic characteristics including preferred language and technology literacy could support the design of digital clinical innovations that are person-centered, avoiding a one-size-fits-all approach [3]. As part of an urban, academic primary care practice–based project, a patient TTP was created to support the development of an English/Chinese patient-facing mobile app for patient-reported outcomes. We conducted a cross-sectional survey of the TTP to assess technology usage and literacy, as well as differences by preferred survey language.

Methods

We conducted a cross-sectional survey to assess the technology usage and literacy of a patient TTP within an adult primary care practice in San Francisco, California, with a diverse patient population. This survey was conducted from April to July 2024.

Ethical Considerations

The University of California, San Francisco, Institutional Review Board approved study procedures (study 20‐32130). All survey respondents provided informed consent. All data were anonymized. Respondents received a US $5 gift card.

Creation of the TTP

The TTP is a closed cohort developed as part of a practice-based project focused on developing a patient-facing mobile app in English and Chinese. Patients who had a preferred language of English or Chinese and an email address available in the electronic health record received up to 3 in-language email invitations to join the TTP. The TTP was composed of 471 patients, with about 24% (n=115) preferring Chinese.

Study Procedures

The 24-item survey assessed self-reported sociodemographic characteristics and technology usage and literacy. Technology usage was measured by the 2020 United States Census American Community Survey Questionnaire [4]. We used the full validated Digital Health Care Literacy Scale (DHLS) to assess a broader range of technology skills [5]. All survey questions were translated to traditional Chinese by a bilingual bicultural individual with review and revision by 2 additional bilingual bicultural individuals. Members of the TTP received up to 3 in-language email invitations to complete the online survey.

Statistical Analysis

Descriptive statistics and bivariate analyses with Pearson χ² and independent t test were conducted to observe differences between English-preferring and Chinese-preferring survey respondents. We computed the DHLS score based on the sum of 6 questions.

Results

Of the 471 TTP members, 221 respondents completed the survey, with an overall response rate of 46.9%. The response rate was 53.4% and 27% among English-preferring and Chinese-preferring respondents, respectively. Mean age was 61.6 (SD 14.9) years, with the sample predominantly identifying as women (57.6%) and White (55.3%) (Table 1). About 8% of respondents reported limited English proficiency. About half of respondents reported attaining an advanced degree. About 12% of respondents had Medicaid.

As shown in Table 2, the most reported device used was an Apple iPhone, followed by desktop computer/laptop. Nearly all participants (99.1%) reported access to the internet at home, primarily via cellular data or high-speech internet plans.

Overall, technology literacy was high: 85.5% of participants strongly agreed that they could install apps independently and 83.3% could use them without help. More than half strongly agreed they could solve technical issues independently. When faced with technical problems, 80.5% reported attempting to resolve the issue themselves before seeking help.

In bivariate analysis by preferred survey language, Chinese-preferring respondents reported lower levels of technology literacy compared to English-preferring counterparts. Fewer Chinese-preferring participants strongly agreed they could install apps (64.5% vs 89%, P<.001), use apps independently (64.5% vs 86.3%, P=.012), set up video chats (61.3% vs 85.8%, P<.001), or solve technical issues on their own (38.7% vs 60%, P<.001). Chinese-preferring respondents were also more likely to report frequent need for assistance with digital devices (43.4% vs 10%, P<.001). The DHLS mean score differed by language (Chinese-preferring 17.2, SD 5.8 vs English-preferring 20.8, SD 2.2; P<.001).

Discussion

This overall sample of TTP respondents from a primary care practice reported high technology literacy. However, in stratified analysis by preferred survey language, Chinese-preferring respondents reported lower levels of technology literacy than English counterparts.

Consistent with prior literature, our results suggest that linguistically diverse patients experience technology literacy barriers [6] and reinforce the importance of addressing technology literacy across linguistically diverse groups in the United States to promote accessible health technology design and use [7]. Our study adds new information about TTP composition as TTP composition matters for equitable technology tool development.

This study has a modest sample size, including a small sample size of Chinese-preferring respondents. This study also has limited generalizability due to the study sample being from one primary care practice in one geographic location and due to the study sample being drawn from a TTP composed of self-selected digitally inclined patients with high socioeconomic status and educational attainment who were recruited by email and opted to join the TTP. Nonresponse bias between English and Chinese survey recipients may exist, and the survey was only available online. Our findings are descriptive associations and future studies with larger sample sizes for multivariate analysis are needed. Lastly, self-reported measures may have led to social desirability bias.

Our study of a primary care patient TTP found a difference in technology literacy by preferred language. Recruiting patient TTPs with diversity by language and technology literacy could help support the development of digital health interventions that are more equitable and usable by a broader population.