A Veteran-Centric Web-Based Decision Aid for Lung Cancer Screening: Usability Analysis

Introduction

Background

Shared decision-making (SDM) is a valuable and, in some settings, mandated approach to helping patients make informed and value-aligned decisions in medical care. SDM is especially useful in decisions of equipoise when the benefits of intervention do not clearly outweigh the harms and when the value patients assign to these attributes vary [1,2]. Lung cancer screening (LCS) is an evidence-based intervention that meets these criteria [3,4]. However, despite expert guidelines dating from 2014 that recommend LCS for eligible persons [5], LCS uptake and adherence to follow-up protocols have remained low [6]. A number of SDM tools for LCS have been developed [7-10]; however, the implementation of SDM in practice has been limited [11], and the uptake of LCS remains low [12]. A participatory design approach, including patient end users such as US veterans, may increase the uptake of SDM tools and support value-aligned decisions regarding LCS. A user-centered design of SDM interventions guided by usability metrics is required to advance the integration of web-based SDM tools into clinical practice [13].

LCS with low-dose computed tomography has proven effective in decreasing lung cancer mortality in two large randomized controlled trials: the National Lung Screening Trial in the United States [4] and, more recently, the Dutch–Belgian LCS trial [3]. The trials reported a decrease of 20% and 25% in lung cancer mortality among those screened compared with control populations for the United States and Belgian studies, respectively. However, clinical trials also reported harms, including false positive tests, significant incidental findings, and an excess incidence of lung cancer cases indicative of overdiagnosis. False positive tests require follow-up imaging and, in some cases, invasive diagnostic procedures that can cause harm. False positive rates vary across trials and decrease in the later years of screening. The National Lung Screening Trial reported false positive rates of 26.3 %, 27.2 %, and 15.9% at baseline, year 1, and year 2, respectively. The Dutch–Belgian trial reported false positive rates of 19.8 %, 7.1 %, and 9.0% at baseline, year 1, and year 3, respectively [14].

Decision aids (DAs) are structured tools that include a description of decision options, evidence-based benefits and harms associated with each option, a value clarification exercise, and support for a SDM conversation with one’s health care provider [15]. In systematic reviews, DAs have been found to improve the quality of decision-making with respect to the outcomes of knowledge, preparation for decision-making, and value-aligned decisions [16]. However, the implementation of DAs has been limited. A primary barrier has been their integration into the flow of clinical practice [17-19].

Objectives

Veterans of the United States military are at a higher risk of developing lung cancer than the general population because of older age, higher rates of smoking, and environmental exposures [20]. Veterans are also at higher risk for mental health conditions, including anxiety and posttraumatic stress disorder (PTSD), which increase the burden of LCS, thus affecting the balance of benefits and potential harm from LCS [20]. To address these challenges, we used a user-centered design to develop a veteran-centric LCS decision tool (LCSDecTool) for use in a primary care setting. This tool was designed to be used independently by the patient before the clinic visit (either at home or in the waiting room) with the option of sharing some components with the clinician during the clinic visit. In this study, we report an iterative process of usability testing of a prototype tool and a revised high-fidelity version, with the latter conducted as part of a hybrid type 1 effectiveness-implementation trial. Usability assessment is a key component in the development of web-based decision support tools. Common methods used in usability testing include observation, cognitive interviews, and self-reported feasibility and usability with validated scales [21]. We seek to determine whether veterans find the tool useful to use in the context of a clinical visit and elucidate key aspects of the user experience that had an impact on usability.

Methods

Description of the LCSDecTool

We developed the LCSDecTool using the Promoting Action on Research Implementation in Health Services (PARiHS) implementation framework. The PARiHS framework includes the domains of evidence (scientific evidence supporting intervention efficacy), context (the setting in which the intervention is delivered), and facilitation (training and support needed to deliver the intervention) [22,23]. Systematic reviews have established the efficacy of decision aids in improving the process of medical decision-making, as indicated by an increase in knowledge, perceptions of being informed, accuracy of risk perceptions, and clarity about values [24]. We were guided by criteria from the PARiHS framework in the tool design and usability testing for the prototype and high-fidelity LCSDecTool, both of which were evaluated among eligible veterans receiving primary care in Veteran Affairs (VA) Medical Centers and later in the context of a clinical visit. Both focused on the support needed by veterans to access and navigate the tool. The development and usability assessment were also informed by the Technology Acceptance Model (TAM) [25,26]. The TAM posits that people need to perceive the technology as useful and easy to use to continue using it. This model supported our choice of the quantitative outcome measures of the System Usability Scale (SUS) and the End User Computing Satisfaction (EUCS) measure, which include questions in these domains [27-29].

Stakeholder focus groups (clinicians) and structured interviews (veterans) further informed the content and features of the prototype LCSDecTool [30]. The LCSDecTool was designed to (1) use in advance of or during a primary care clinical visit where LCS may be initiated, (2) ensure that veterans understand the key benefits and potential harms of LCS, (3) help veterans weigh the benefits and potential harms of LCS, (4) provide resources for smoking cessation and mental health treatment, (5) support communication with their provider regarding this decision, and (6) include a clinician portal to streamline the use of the tool with a clinician in the clinical setting. The tool was designed for the primary platform of a tablet but has compatibility with a computer or mobile phone. The stakeholder inputs that informed these goals are summarized in Table 1. The data collection form with illustrations from the prototype version of the LCSDecTool is available in the supplemental materials (Multimedia Appendix 1).

Approach to Usability Testing

We used both quantitative and qualitative methods for usability testing. The SUS and the EUCS scales have been widely used and well-validated to assess interventions in the domains of ease of use and usefulness [28,29,31-36]. The SUS is a 10-item scale developed for the assessment of a broad range of products; scores (ranging from 0=low usability to 100=high usability) correspond to the following adjective descriptors: worst imaginable, awful, poor, okay, good, excellent, and best imaginable [27,37]. The EUCS scale is a 12-item scale that captures the domains of content, accuracy, format, ease of use, and timeliness relevant to a computer application. Scores range from 1=low satisfaction to 5=high satisfaction [28,29]. The Patient Engagement (PE) scale has been used to assess engagement in web-based programs in the medical setting [38,39]. The PE scale assesses engagement in the domains of caring for one’s health, concerns addressed about LCS, understanding LCS guidelines, and understanding information about LCS. Scores on the PE scale range from 1=low engagement to 5=high engagement.

To better interpret the results of the quantitative measures, we used qualitative approaches, including observations of the testing sessions and brief interviews at the completion of the testing sessions. These qualitative approaches have been applied in usability testing to identify barriers to the effective navigation of tools and processing of information [40,41]. The research assistants (RAs) who obtained the qualitative data underwent formal training in qualitative interviewing before the study. This process included 8 hours of training in qualitative methods in the Mixed Methods Research Lab at the University of Pennsylvania for our senior RAs (JP: Bachelors of Science with Major in Biology; JI: Bachelor of Science with a major in Neuroscience). Our junior RA was trained by the senior RAs on the observation of interviews and feedback (JM: Bachelor of Arts with a major in Psychology).

Phase 1 Prototype Usability Testing

Phase 1 Study Participants and Recruitment

Veterans were eligible if they were aged 55 to 80 years, had a 30 pack-year history of cigarette smoking, and continued smoking sometime within the past 15 years. The exclusion criteria were cognitive impairment; a life expectancy of <2 years, as determined by their primary care provider; and having received LCS within the past 18 months. The participating sites were the Michael J Crescenz VA Medical Center in Philadelphia, Pennsylvania, and the West Haven VA Medical Center in West Haven, Connecticut.

We used the Corporate Data Warehouse to identify eligible veterans based on age, an algorithm of indication of tobacco use based on primary care visits and dental visit structured fields in the computer record, and pharmacy records of the use of smoking cessation medications, as well as having an upcoming primary care appointment. We sent recruitment letters with follow-up phone calls to confirm eligibility. Veterans who were interested were then scheduled for a study visit. Recruitment for the phase 1 protocol study occurred between August 2018 and January 2019. A voucher for US $50 was offered to compensate the participants for their time. The recruitment goal for veterans was 16 from each site (total of 32)—a sample size that has been validated in the literature for usability testing [42,43].

Phase 1 Prototype Usability Testing Procedures

Following informed consent and the completion of a baseline survey to assess demographic information, participants were individually seated in a room with an RA. Participants were provided with a tablet device to use the LCSDecTool. The session was audio recorded and transcribed. Following the use of each section of the tool (Table 2), the RA asked the participants to describe their user experience. The RA documented field notes during the session to highlight responses to specific sections. At the completion of the tool, the participants completed the SUS and EUCS scales [27,29].

Phase 1 Prototype Analytic Plan

Quantitative Analysis

Descriptive statistics were used to summarize the following outcomes: (1) time in minutes for completion of the tool, (2) participant characteristics, and (3) scores on the SUS and EUCS scales. Participant characteristics included sex, race, ethnicity, age, and education level.

Qualitative Analysis

Members of the research team (MS, JP, and JM) reviewed the transcripts of the user sessions with veterans and the field notes documented by the RAs from provider interviews. Comments were summarized for each section of the LCSDecTool to indicate feedback from veteran users and clinicians regarding the features and content of the LCSDecTool (Table 2).

Phase 2 High-Fidelity Usability Testing

Phase 2 Study Participants and Recruitment

Usability testing of the high-fidelity version was conducted as part of a pilot type 1 hybrid effectiveness-implementation trial that compared the LCSDecTool with a control web-based intervention providing general education about cancer screening. The participating sites were the Michael J Crescenz VA Medical Center in Philadelphia, Pennsylvania, and the West Haven VA Medical Center in West Haven, Connecticut. Eligibility criteria for veteran participants were identical to the phase 1 prototype usability testing protocol with the following exceptions: (1) the additional exclusion criteria of a prior diagnosis of cancer (except for stable prostate cancer or nonmelanomatous skin cancer) and (2) the additional inclusion criteria that the veterans’ primary care provider consent to participate in the study. Recruitment for the phase 2 high-fidelity usability study occurred between March 2019 and February 2020. Participants received a US $50 voucher for the baseline visit, which included usability testing.

Phase 2 High-Fidelity Usability Testing Procedures

Following informed consent and the completion of a baseline survey, participants were given a tablet that was open to the first page of the LCSDecTool. Participants were instructed to navigate the tool on their own. The RA observed the session, took field notes to document observed difficulties in navigation, and answered the participants’ questions during the usability session. The time from when the participant started to interact with the tool until the summary page was reached (ending the session) was documented by the RA.

Upon completion of the use of the tool, the RA conducted a brief interview with the participant, which included the following questions:

1. What were your general impressions of the LCSDecTool?
2. Did you have any trouble using the tool, and if so, describe (if the RA had noticed any issues in navigating the tool that were not mentioned, they would prompt by adding “I noticed you had trouble with...”)?
3. Do you have suggestions for improvements?
4. On the page that has boxes with knowledge content, do you think that it should be required to click on every box to move forward?
5. Did you find the tool repetitive in any way?

When the user responded affirmatively, follow-up questions were asked so that the patient could further elucidate.

Upon the completion of the usability testing session, participants proceeded to their scheduled primary care appointments. A postclinic survey was completed directly after the primary care appointment. The postclinic visit survey included the SUS, EUCS, and PE scales. The study was approved by the institutional review boards of the participating sites.

Phase 2 High-Fidelity Analytic Plan

Phase 2 Quantitative Analysis

Descriptive statistics were used to summarize participant characteristics, including sex, race, ethnicity, age, annual household income, and comorbidity. Descriptive statistics were used to summarize the responses to the SUS, EUCS, and PE scales.

Phase 2 Qualitative Analysis

We used a thematic analysis to analyze the qualitative data from the following sources: (1) notes taken by the RA while observing usability testing and (2) documented responses to the brief interview that followed the user session. Thematic analysis is an approach for identifying, analyzing, and reporting themes or patterns within a set of data using standard methods for qualitative research [44]. We used an inductive approach to identify themes pertaining to the user experience with the tool. A total of 2 members of the study team (JP and JM) initially reviewed the data and created an initial coding scheme. The data and codes were reviewed and finalized with input from additional members of the study team (MS and DK). Final coding was conducted by 2 independent coders, with differences resolved by consensus.

Ethics Approval and Informed Consent

The study was approved by the institutional review board of the participating sites. Participants provided informed consent to complete the baseline survey in phase 1, and the clinicians of participating veterans signed informed consent forms in phase 2 at the time of enrollment in the study. IRB approval obtained from the Michael J Crescenz VA Medical Center, Philadelphia VA (IRB# 01635, IRB #01721, IRB#01780) and the VA Connecticut Healthcare System (MIRB# 02071, MIRB#02240).

Results

Phase 1 Prototype Usability Testing

Phase 1 Study Participants

A total of 70 recruitment letters were mailed to the Philadelphia VA Medical Center. Of the 70 veterans, 12 (17%) had confirmed eligibility by phone interviews and were enrolled in the study. A total of 101 letters were mailed to the West Haven VA Medical Center. Of these 101 veterans, 6 (5.9%) were confirmed to be eligible and were enrolled in the study. Of the 18 participants, 15 (83%) were male, 8 (44%) were African American, and 15 (83%) had up to a high school–level education. Additional participant characteristics are presented in Table 3.

Phase 1 Prototype Quantitative Results

In the phase 1 prototype usability testing, the mean of the SUS score (potential range: 0-100) was 81.90 (SD 9.80), corresponding to an excellent level of usability. The mean of the EUCS score (potential range 1-5) was 4.30 (SD 0.71; Table 4).

Phase 1 Prototype Qualitative Results

The veterans’ and clinicians’ qualitative feedback for the prototype testing are summarized in Table 2. Key feedback received from veterans included the following: (1) the dialog feature was engaging; (2) the knowledge boxes were an effective way of presenting information and reinforcing content introduced in the dialog feature; (3) the pictograph effectively conveyed a comparison of outcomes among screened versus nonscreened groups; (4) the value elicitation questions using a response scale measuring level of concern regarding potential harms were more intuitive to most than the response scale assessing if the attribute made it more or less likely for the veteran to have LCS; and (5) there was enthusiasm among users about the interactive features, including a text box and summary sheet to ask questions and share questions, values, and preferences with their provider. Clinician feedback included recommendations for simplifying the terminology used and enthusiasm for the pictograph as a visual aid to support provider–patient communication about LCS.

Modifications Made in the High-Fidelity Version

Results of the phase 1 testing informed changes made to the prototype in the development of the high-fidelity version, including the following: (1) use of a color scheme aligned with the US Department of Veterans Affairs branding, (2) replacement of stock graphics with icons and symbols, (3) use of directions and well-placed buttons to improve self-navigation, (4) a more prominent link from the entry page to the clinician portal, (5) simplified text and definitions, (6) single value clarification exercise, and (7) improved graphics and format of summary page to increase the visual impact and clarity. A link to the LCSDecTool is found in Multimedia Appendix 2. The dialog, knowledge box, pictograph, and value elicitation features are illustrated in Figures 1-4.

‎View this figure

‎View this figure

‎View this figure

‎View this figure

Phase 2 High-Fidelity Testing

Phase 2 Study Participants

At the Philadelphia VA Medical Center, 465 recruitment letters were sent. Of these 465 persons, 376 (80.9%) were reached by phone, and 281 (60.4%) were confirmed to be eligible for the study. Of the 281 eligible persons, 80 (28.5%) were enrolled in the study, with 42 (53%) in the experimental arm and participating in the usability analysis. At the West Haven VA Medical Center, 136 letters were sent. Of these 136 individuals, 96 (70.6%) were reached by phone and found to be eligible for the study. Of the 96 eligible persons, 5 (5%) were enrolled in the study, with 1 (20%) in the experimental arm and participating in the usability analysis. Of the 43 participants, 27 (63%) were African American or Black, 39 (91%) were male, and 41 (95%) had up to a high school–level education. Additional demographic details are presented in Table 2.

Phase 2 High-Fidelity Quantitative Results

In the phase 2 high-fidelity usability testing, the mean SUS score was 65.76 (SD 15.23), corresponding to a good level of usability. The mean of the EUCS score (potential range 1-5) was 3.91 (SD 0.95). The mean PE score (potential range 1-5) was 4.12 (SD 0.67; Table 3). The median time to completion in minutes was 13 (IQR 7-30; Table 4).

Phase 2 High-Fidelity Qualitative Results

A total of five themes related to usability emerged from the qualitative analysis of the field note–documented observations and responses to the short interview. These themes were as follows: (1) a low baseline level of awareness and knowledge about LCS increased after using the LCSDecTool; (2) users sought more detailed descriptions about the LCS process; (3) the LCSDecTool was generally easy to use, but specific navigation challenges remained; (4) some noted difficulty understanding medical terms used in the LCSDecTool; and (5) the LCSDecTool evoked veteran struggles with prior attempts at smoking cessation (Textbox 1).

Theme 1 indicates a need for more information about LCS among this population. A participant stated that they “learned a lot and didn’t know much about LCS and lung cancer before using the tool.” Another stated that they “didn’t know that the VA even had a screening test.” This feedback supports our finding of high PE scores in the quantitative testing. Theme 2 reflects the desire for more information about the LCS process, with one of the participants asking, “What actually is a CT scan?” and another questioning, “if it is painful?” This theme indicates what information could be added to the tool to increase the content domain of the EUCS measure. Theme 3 indicates that the tool was generally easy to use; however, navigation was challenging for some of the features. For example, a participant described the tool as “helpful, with a lot of information, easy to use.” However, specific navigation problems were identified. A participant indicated that they “did not know how to scroll through the Dialogue,” and an RA observed and commented that another user was “stuck on Box page until I told him he had to click on the boxes.” Some participants had difficulty using the radio buttons on the value elicitation feature. Theme 4 indicates that some users struggled to understand the medical terminology. A participant stated that “some wording can get you twisted up” and provided feedback to “keep it simple and use plain language.” For example, the meaning of the words “nodule,” “CT Scan,” and “Overdiagnosis” were not understood by some users. This theme is relevant for interpreting the SUS scores in the format domain.

Theme 5 indicates that the tool evokes veterans’ struggles with prior attempts at smoking cessation. One of the specific veteran-centric features included in the tool highlighted the importance of smoking cessation and mental health. Users could click a radio button to request to speak with a primary care provider about smoking cessation or a mental health provider about either smoking cessation or LCS. Some users stated that they were already familiar with these resources or had successfully quit smoking. Others commented on the difficulty of smoking cessation, recalling multiple efforts to do so. A participant commented that he quit smoking for 1 year but recently restarted, stating, “don’t ask me why I started again, I don’t have a reason, I just did it,” and another commented, “I have tried the smoking cessation classes here at the VA but they don’t work.” Other users clicked the boxes conveying an interest in speaking about these topics to a provider.

Discussion

Principal Findings

The LCSDecTool was designed to create an engaging experience, inform veterans about LCS, support a value-aligned decision, and facilitate communication with their provider about LCS. We found that usability among veterans was good when administered in the context of a primary care clinical visit. These results provide evidence that an older group of US veterans can navigate tablet-based DAs. Our study was noteworthy for engaging veterans in the design of a decision support intervention in a meaningful way. In creating a veteran-centric tool, we involved veterans at every step of development and incorporated their feedback in an iterative process of tool development. Veterans were able to help determine the components of the SDM process and provide reactions and comments that could help other veterans. The motivation to provide feedback to help other veterans reflects a military culture of caring for and deriving satisfaction from helping other veterans [45].

Given that veterans in this age group may not have been familiar with technology, usability testing was particularly important to ensure that they could interact with and understand the information in the tool. Our usability findings were strengthened by conducting the assessment in the setting of a primary care visit where the LCSDecTool was designed to be used. Through this evaluation, we identified areas of strength and areas that require further refinement and modification.

The qualitative feedback obtained from our findings revealed 5 underlying themes. Two of these themes (low baseline awareness and knowledge about LCS that increased after use of the tool and the desire for more detailed descriptions of the LCS process) indicate that usability testing increased awareness of LCS. This is a key step in the process of adopting this evidence-based and provides preliminary evidence that the tool will increase awareness of LCS. Two additional themes (the tool was generally easier to use, but navigation challenges remain and difficulties in understanding some medical terminology remained) indicate the importance of veteran feedback in creating a tool that veterans are able to understand and use. These themes will guide further refinements of the tool. A final theme indicates that the tool may enhance discussions with providers about smoking cessation and mental health. This theme reinforces that veterans perceive a relationship between mental health, smoking cessation, and LCS and further supports the decision to address this relationship in the LCSDecTool.

Our study sample was a particularly vulnerable population in terms of sociodemographic factors (low level of education and income) and health status. Among those in the phase 2 high-fidelity cohort, 47% (20 out of 43) reported a diagnosis of PTSD. Rates of PTSD are known to be higher in veterans than in the general US population. Furthermore, among veterans, the rates of PTSD are higher in those receiving VA care than those who do not [46]. Veterans who receive care in the VA are also known to have lower education and income than veterans receiving care outside of the VA. Given these differences in socioeconomic status and health status, usability testing among a sample of veterans receiving VA medical care focuses our study on a more vulnerable population of veterans.

We observed a decrease in the usability measures from the prototype testing cohort to the high-fidelity testing cohort. There are several potential reasons for these differences. First, in the prototype cohort assessment, quantitative usability measures were collected immediately after completing the tool. In contrast, in the high-fidelity cohort, the user experience and quantitative usability assessment were separated by a clinic visit. The goal of this study was to assess the usefulness of the tool when used in the clinic setting. However, the intervening clinic visit may have decreased the salience of user experience. Of note, the purpose of the primary care visit was not limited to the topic of LCS, although addressing age and risk factors for appropriate cancer screening is an expected component of a primary care visit. Second, the study directed RAs to be more engaged with participants in the earlier prototype testing than in the high-fidelity testing. For example, in the prototype testing, the RAs discussed the participants’ experience with each section of the tool before progressing to the next section. The increased level of RA engagement during the prototype user session may have positively affected user experience.

According to the TAM, users need to perceive the technology as useful and easy to use to continue using it. Our qualitative data provide additional insights on participant perceptions of the tool regarding usability and usefulness. The design of our value assessment measures was based on qualitative studies conducted with veterans regarding how they perceived and valued the potential benefits and harms of LCS [30]. In usability testing, the value assessments were completed without difficulty, with some participants commenting that they enjoyed responding to these scales and found them helpful. Our PE survey indicates that users of the LCSDecTool felt engaged in the decision-making process, a primary goal of any SDM intervention, as in the development of other eHealth applications [35,36]. Our results suggest that users perceived that the tool supported them in caring for their health, addressed their concerns, and informed them about LCS.

Comparison With Other Work

In our study, we observed that 100% of the participants completed the use of the LCSDecTool. Completing the tool included moving through all sections, responding to the value assessment questions, and submitting the summary page. In a recent Cochrane review, which was a subgroup analysis of 105 studies involving >31,000 participants, the median effect of a DA on the length of a medical consultation was to lengthen the consultation by 2.6 minutes in comparison with usual care [24]. Although the length of the LCSDecTool did not emerge as a concern among users in our qualitative feedback, the time required to complete the tool could have contributed to lower scores on the SUS and EUCS scales. Our qualitative data suggest that improving navigation and allowing users to choose which sections to review would decrease the time to completion of the LCSDecTool without compromising usability and effectiveness.

Prior studies on cancer screening DAs provide a comparison of the usability scores for the LCSDecTool. Carter-Harris et al [8] developed an LCS DA that included audio and video features and scripts tailored to the user’s smoking status. The tool, named Lung Talk, reported mean SUS scores of 75.7 (SD 7.9), indicating a good level of usability [8]. Coe et al [47] developed a breast cancer screening DA for use among multi-ethnic women. This tool, named Real Risks, reported mean SUS scores of 80.0 (range 50.0-95.0) and 66.3 (range 55.0-75.0) for the English and Spanish versions, respectively [47]. Our study design is unique in reporting usability scores prospectively as the tool moved from the evaluation of a prototype to the evaluation of a high-fidelity version within a clinical setting. In our study, the SUS scores decreased when the tool was integrated into the clinical setting but remained at an acceptable level of usability.

Our LCSDecTool differs from existing LCS DAs in several respects. It was developed using the principles of user-centered design to increase engagement with the tool among a veteran population. This includes the look and feel of the tool, such as using the Department of Veterans Affairs branding, content that acknowledges the mental health conditions that may affect LCS, and smoking cessation referrals specific for veterans receiving care in VA medical centers.

Limitations

Our study has some limitations. First, data for the phase 2 high-fidelity version were obtained in the context of a pilot hybrid type 1 effectiveness-implementation trial. In this research setting, participants were able to ask an RA for help with the tool if needed. This may not be feasible in routine care outside the context of a research study. Second, the LCSDecTool was tested on a limited number of veterans and clinical sites. Our population reports high rates of mental health conditions and other comorbidities and has particularly low levels of education and income. The findings may not be generalizable to a broader population of veterans or the general US population. Despite these limitations, the strengths of our study include the evaluation of the usability of the LCSDecTool among veterans who are diverse in race, meet eligibility criteria for LCS, and are receiving care in a VA primary care setting.

Conclusions

In conclusion, our study found that a web-based LCS decision support tool developed for and tested among US veterans receiving care in a VA medical center demonstrates an acceptable level of usability. We designed the LCSDecTool for use before and during a clinical visit, incorporating content, formats, and functions that can be used across these settings. The decision regarding LCS requires patients and their providers to consider scientific evidence of benefits and harms, as well as patient values, priorities, and beliefs. Given these complexities, it is important to conduct usability testing of this patient-centered LCSDecTool in its target population. Our findings support the use of this eHealth technology in the primary care clinical setting as a way of engaging veterans, informing them about a new cancer control screening test, and preparing them to participate in an SDM discussion with their provider. Our study further indicates that involving veterans in all phases of the development of the tool extends veteran involvement to the design of SDM processes. Future work is needed to fully address the information and decision support tools that will help veterans understand and apply the principles of SDM for LCS in the context of a comprehensive health care program.