The PTSD DA 2.0 website was developed following the steps outlined in Figure 2.

This process aligns with the one outlined by Coulter et al [10] and was consistent with the development and reporting standards of the IPDAS and Standards for Universal Reporting of Patient Decision Aid Evaluation studies [11], the theory of the Ottawa Decision Support Framework [12], the iterative process outlined by Elwyn et al [13] for web-based DA development, and the principles of user-centered design, in which those who will use a product influence the design at multiple points during the development process [14]. Given that we were working from an existing DA, some background work had already been completed, including assessment of decisional needs, creation of written content, and user experience testing. Although no formal field testing was conducted with the DA 1.0, it has been in use for 8 years by patients and clinicians, thus allowing us to gather informal feedback on how it had been used to date and how it could be more useful in the future.

We used an iterative user-centered process in the development of the DA 2.0, involving multiple stakeholders with the goal of designing a DA that was acceptable to both patients and clinicians in routine clinical practice. There were 3 main iterations of the DA 2.0—the first fully functional draft prototype (“alpha”), followed by the next main iteration (the “beta” version), and finally the public version. Several forms of feedback and testing were used to shape changes between each successive version. The main production groups and stakeholders are outlined below.

The project management group consisted of 2 clinical psychologists (masked) and 1 educational specialist (masked), all NCPTSD members with experience in PTSD, clinical work, research, and health communication. The project management group exercised editorial control and was ultimately responsible for the design of the DA. This group decided on content and design via consensus.

A panel was selected to represent areas of expertise most relevant to the DA development. This involved first identifying areas of expertise and lived experience to be included (PTSD clinician; people who had received PTSD treatment, ie, lived experience, along with experts in SDM in mental health, in DA development, in user experience testing for online products, in health care communication and health care disparities, and in PTSD CPGs), then identifying potential experts in each area, before reaching out to invite them. Attention was also paid to the diversity of demographics as well as experiences. This panel was asked for advice, feedback, and review of design decisions and content through the process of DA development, formally meeting 4 times throughout the process (prior to contract kick-off, after initial prototype development, after alpha development, and after beta development), with additional information-seeking along the way.

The project management group contracted a company with expertise in web design and user experience testing to produce the website.

The NCPTSD regularly convenes a panel of veterans to provide feedback on various research and educational products. The DA 1.0 was presented to this team (n=6) prior to the development of the DA 2.0 in order to gain their perspective on needed changes. For instance, they strongly suggested including information for loved ones of people with PTSD, ultimately leading to a dedicated page within the DA with information tailored to loved ones.

Care was taken to seek out feedback from experts as well as stakeholders representing each main user type (patients, providers, and loved ones), as recommended for user-centered design of DAs [15]. Before developing the alpha version of DA 2.0, information was sought about what in DA 1.0 worked well and what could be updated or made more user-friendly. This information was sought through individual interviews with a PTSD-focused VA psychiatrist, a VA caregiver support program social worker, the spouse of a veteran with PTSD, and 2 informal focus group-style meetings with VA PTSD-focused psychologists and social workers (n=27). Clinicians comprised a convenience sample of those available locally and nationwide through their involvement in the VA’s PTSD Mentoring and Implementation Program.

Throughout the development process, the education team at the NCPTSD reviewed versions of the website. The team includes clinicians, researchers, communication specialists, and educational specialists who gave feedback on all aspects of design, understandability, representativeness, and utility.

After developing the beta version of the DA 2.0, 7 VA clinicians specializing in PTSD treatment reviewed the content and function of the website, giving their feedback about each (for more patient and provider feedback, see Field Test Descriptive Results section). Finally, the website was submitted for review to ensure that it adhered to Section 508 of the Rehabilitation Act and Section 255 of the Communications Act, ensuring accessibility to those with disabilities as required by US law.

The evidence for each intervention included in this DA was largely drawn from the VA/DoD CPG for PTSD (2023) systematic literature review and resulting recommendations. This was supplemented as needed by data from the PTSD Repository, a database of all randomized controlled trials of PTSD treatments that was jointly developed by the NCPTSD and the Pacific Northwest Evidence-Based Practice Center following the Agency for Healthcare Research and Quality’s methods guide for effectiveness and comparative effectiveness reviews [16].

In preparation for the creation of the DA 2.0, a needs assessment survey was undertaken, similar to one that informed the content and design of the first DA 1.0 [17]. The sample included 887 adults (378 veterans and 509 nonveterans) who screened positive (total score ≥3) for PTSD on the Primary Care PTSD Screen for Diagnostic and Statistical Manual-5 [18]. The sample included women (n=259), men (n=601), and other or did not answer (n=26), with a mean age of 50.75 (SD 16.09) years, largely non-Hispanic White (548/887, 61.8%), along with non-Hispanic Black (117/887, 13.2%), Hispanic (146/887, 16.5%), non-Hispanic 2 or more races (53/887, 6%), or non-Hispanic identifying as “other” (23/887, 2.6%). This survey was supplemented by feedback as outlined in the Other Stakeholders section.

At the beginning of the project, user personas were constructed using the expertise of the project management team with input from the expert panel. User journey mapping was used to anticipate the needs of potential users of the site, along with their potential emotional states and decisional needs. This was important given that much of the feedback was from veterans and veteran-facing providers, but the site is designed to be useful for both veterans and civilians.

The prior DA 1.0 content was reviewed by the project management team for needed updates, informed by the prior needs assessments. The content was iteratively revised, as the structure of the DA 2.0 was updated. For instance, it was judged that it would not be clear to patients to use the terms “strong recommendation for,” “weak recommendation for,” etc., to describe the varying levels of evidence outlined in the CPG. Rather, this information was consolidated under “effective treatments,” “not enough evidence,” and “not recommended.” The revised content was organized into prototypes, which outline the rough structure of the website along with how users are likely to flow from one page to another. Once prototypes and initial content were finalized, these were reviewed with the expert panel as well.

First, user experience testing was conducted with 9 individuals who screened positive for PTSD (including a mix of veterans and civilians, sexes, race or ethnicities, ages, socioeconomic backgrounds, geographic US locations, and at least 2 people with low health literacy). In one-on-one moderated sessions, participants were guided through an exploration of the draft website and given specific tasks to complete while being asked to “think out loud” about what they were thinking, noticing, or struggling with. Main tasks across the 2 rounds (9 individuals for review of alpha and 9 for review of beta) included overall site navigation, using the PTSD self-screen, and exploring the “Compare your options” and “Your Summary” pages, along with reviewing veteran-specific and treatment-specific pages. The technical production group and the project management team lead separately summarized impressions of what worked well and what did not in each session, and then discussed to come to a consensus about needed improvements. For example, the alpha testing identified a need to make clearer that this was a government website, and the beta testing identified a need for much clearer feedback from the PTSD self-screen, both of which were implemented in later iterations (for more comprehensive changes, please see the Considerations and Decisions section).

Whereas user experience testing allowed a guided exploration of the site with moment-by-moment feedback on specific tasks, field testing was conducted by the project management team to gain the perspective of providers and patients in a real-world setting actually facing the decision at hand. In field testing, VA providers and veterans used the DA as intended in order to test that it was feasible, acceptable, balanced for undecided patients, understood by those with limited reading skills, and delivered with fidelity [11,14]. For this purpose, providers at a VA Medical Center in the Northeastern United States asked patients seeking PTSD treatment if they would like to use and give feedback on the new PTSD DA.

For providers who used the DA with their patients, we asked whether the DA would fit into their workflow, whether it was helpful to their patients, and what worked well or could be improved (open-ended).

Patients were asked to complete a brief assessment both before and after using the draft DA 2.0. Measures included basic demographics (without protected health information), a question about how they had used the DA (eg, by themselves or with providers, on a computer or a phone, and for how many minutes), and room for open-ended feedback about what they liked or would improve. Additionally, the following measures were administered (brief measures were prioritized, given that this was part of standard care and not reimbursed research time for providers or patients).

Prior to using the DA, to differentiate those with high from marginal or low health literacy, patients were asked “how confident are you filling out medical forms by yourself?” [19] with a 5-point Likert-type answer from 1=not at all to 5=extremely. Answers of 1=not at all to 3=somewhat suggest low or marginal health literacy, whereas 4=quite a bit to 5=extremely suggest high health literacy.

At both pre- and post-DA, patients completed a decisional self-efficacy scale to measure their belief that they have the ability to make decisions and to participate in SDM [20]. Patients are asked to “show how confident you feel in doing these things by circling the number” with responses on a 5-point Likert-type response scale from 0=not at all confident to 4=very confident. There are 11 items (sample item: “Figure out the choice that best suits me.”).

At both pre- and post-DA, patients completed a decisional conflict scale that measures a state of uncertainty about a course of action [21]. This short version includes 4 items with a dichotomous yes or no response scale. A sample item is: “Do you feel sure about the best choice for you.” Items are summed (with “yes”=1), and a score of 3 or less indicates decisional conflict. The scale was found to have an α of .86 and was responsive to change following use of a DA and counseling [21].

Following completion of the DA, satisfaction with the DA was assessed by asking: “How likely is it that you would recommend the PTSD Treatment Decision Aid to other people facing the same decision?” [22]. Responses could range from 0=not at all to 10=extremely likely. Generally, 0 to 6 is considered a “detractor,” 7 or 8 “passive” (ie, satisfied but not happy enough to promote the service), and 9 or 10 a “promoter” (loyal and enthusiastic about the service).

We adapted acceptability questions from O’Connor [23] to assess whether the amount of information in the DA was adequate, information was clear and balanced, and whether this tool would be helpful in making a decision or preparing to talk with a clinician. Assessment of feasibility was set a priori at the ability of 5 or more patients to complete the field testing and give us feedback. Fidelity was operationalized as patients reporting using the DA for at least 15 minutes (ideally with a mix of some viewing on own and some with a clinician, along with some on the phone and some on the computer). Fidelity was further operationalized as providers reporting that the DA would fit into their clinic workflow and that their patients reported they were able to use the DA. We were not able to assess which components of the DA patients used.

Based on all sources of input and stakeholder feedback outlined earlier, the project management group considered several significant content and design challenges, with decisions on each outlined below. The lead of the project management team (masked) kept a running log of all feedback received from various stakeholders. Feedback was compared and compiled, and the most crucial points were discussed with the expert panel. The project management team made decisions on which pieces to implement with the ultimate goal of improving the user experience in line with DA design principles.

One decision was which treatments to include in the DA 2.0. This was largely driven by the 2023 CPG alongside a few other considerations. All “recommended” (or first-line) treatments were included in the DA 2.0 (cognitive processing therapy [CPT], prolonged exposure [PE], eye movement desensitization and reprocessing [EMDR], and certain antidepressant medications). We discussed whether and which “suggested” (or second-line) treatments to include, potentially cognitive therapy, present-centered therapy, written exposure therapy, and mindfulness-based stress reduction (MBSR). Additionally, there are numerous other treatments that are either recommended against or considered to have insufficient evidence for inclusion. We considered several factors: (1) the level of availability of treatments (in the United States), (2) the ability for users to have meaningfully different options to select from (eg, having at least 1 nontrauma-focused option), and (3) balancing the desire of our target population for comprehensive information [17] with not overwhelming people with too much information [24]. (4) We also considered that we wanted to help users understand the relatively higher level of evidence for the “recommended” treatments while using language that was not too technical. (5) Finally, IPDAS recommend including information on “doing nothing” as an option. Ultimately, then, we did the following:

Some DAs take a fully guided format in which users must start at the beginning and move linearly through to the end. Others are fully flexible, allowing for exploration. We ultimately decided on a flexible format in which we would guide users toward the treatment comparison chart (the “core” of the DA), while also allowing for users to choose their own path. This was felt to be most appropriate partially based on our user profiles and the knowledge that this DA is broadly available (vs only available via a clinician allowing access to it). Therefore, we anticipated that some people would come to this site after a general internet search for information on PTSD treatment and might want more information about PTSD and its treatment in general before choosing a specific treatment. On the other hand, many people use the DA in the context of their clinician, suggesting that they explore it after they have already been diagnosed with PTSD, sometimes having received significant psychoeducation and treatment prior to using this DA. In those cases, they may want to skip directly to information about specific treatments. The flexible format allows clinicians to recommend it to patients at different stages of treatment seeking.

One challenge is how to best convey the effectiveness of various treatments in a way that is meaningful to patients. Effectiveness is one of the most important pieces of information patients used in making treatment decisions [17]. DAs are expected to include a measure of probable treatment outcome, in a decipherable format, generally a single number compared using the same denominator across all options (eg, comparing 10/100 to 20/100 [26]). This presented a challenge, as there is no clear consensus within the PTSD literature about a standard outcome [27]. Most PTSD treatment trials (and the systematic reviews that aggregate results of these trials) use effect size and tests of statistical significance to determine and communicate treatment effectiveness. These statistics are not easily interpreted by the lay public, nor are they easily converted to a natural frequency (eg, 1 of 10 people). For the DA 1.0, a decision was made to use the loss of a PTSD diagnosis as the metric of change, in part because this has been benchmarked to meaningful change [28], and in part because it was judged to be more interpretable than effect sizes [29]. For the DA 1.0, loss of diagnosis numbers was calculated for each of the treatments included at that time [29], and they were conveyed in the form of icon arrays for ease of interpretation.

This metric, however, was imperfect for a number of reasons. First, loss of diagnosis is not reported in many studies, limiting the data that could be used to calculate these numbers. Second, a number of data points indicated to us that this metric was easily misunderstood by users, especially by veterans. Both veterans and clinicians told us that patients conflated “PTSD” with “trauma,” so they misinterpreted “loss of diagnosis” as meaning that someone would no longer have experienced the trauma or would no longer have symptoms; since this was impossible or unlikely, they interpreted this as an unrealistic and untrustworthy metric. A study found similar results in a qualitative analysis with veterans responding to educational materials with loss of diagnosis statistics [30]. For these reasons, we explored alternatives as we updated for DA 2.0.

We considered an alternative that would use all continuous data from all trials, thereby allowing us to use more of the available data. The challenge, however, was in finding a way to translate this into a dichotomous metric of improvement, which is needed in order to both convey effectiveness simply to the lay public and to be able to compare across treatments. Moreover, it is unclear whether a statistically significant change necessarily leads to meaningful changes in functioning.

Ultimately, we decided to use the same metric (ie, loss of diagnosis) but to label it as “meaningful symptom improvement” instead of “loss of diagnosis.” We reasoned that this would most clearly convey in lay language what was meant by loss of diagnosis. For those who wanted more information, we included a footnote that spells out in more (plain language) detail how we arrived at these numbers, and that they actually involve calculating loss of diagnosis. Data from the PTSD Repository [16] were used to calculate updated effectiveness numbers for each treatment. Specifically, we used randomized controlled trials that were large enough to give reliable estimates (n≥41), that included one of the highlighted treatments (in individual, outpatient format), and that included a clinician assessment of PTSD diagnosis based on a validated interview. For each treatment arm, we calculated the percentage of participants that no longer met diagnostic criteria for PTSD at posttreatment. Data from inactive “control” arms of the same studies were used to estimate loss of diagnosis numbers for “no treatment.” Standard meta-analysis techniques were then used to combine across individual treatment arms, using properly weighted averages, to arrive at an estimated overall success rate for each treatment type reported.

Another criterion for a good DA is that it helps patients identify what they want in a treatment and suggests options that match their preferences. The project management team judged that part of this DA’s goal was to help people identify what they want, ideally with the goal of narrowing down to at most 3 of the 6 possible treatments, to avoid overwhelming patients. The DA 2.0 uses a “decision analysis” [31] approach, in which 4 questions are posed to help people identify their preferences. In total, 2 of the 6 questions from the DA 1.0 were eliminated (group vs individual, since group is not recommended; length of treatment, since that was redundant with medication vs therapy). The research question was modified to help people differentiate between seeing all 6 treatments or only those with the most evidence.

We considered multiple iterations of how those 4 questions would lead to the options presented to users. For instance, it could be that each question is worth one point, and any points that matched a particular treatment were summed to present those with the highest totals. However, using this method, it would be possible for a user to be presented with a treatment that matched many of their preferences but did not match what they said they wanted (eg, indicating not being interested in medications but then having that presented as an option, which may lead to a sense that the algorithm was not working). The questions could also act as a filter (eg, if a user wanted to talk about trauma, the algorithm would filter out those that did not involve a trauma focus), but then, it would be possible to end up with no treatments matching all of a user’s options, which we judged could be discouraging. Ultimately, we decided on a hybrid of those; only a perfect match is highlighted, and if there is not a perfect match, then the wording will make clear that it matches “some” of a user’s preferences. Further, if more than 3 options equally match a user’s preferences, then a flag helps users narrow their options by encouraging them to first consider one of the treatments with the most research support (CPT, PE, and EMDR).

One other consideration was regarding how “neutral” to be in our presentation of information about trauma-focused treatments. A fundamental tenet of DAs is to present information in a fully balanced and objective manner. Yet, there is some debate about whether in some cases it may be most appropriate to deviate from strict neutrality in order to help correct misperceptions [25]. In the case of PTSD treatment, trauma-focused treatments have a stronger evidence base than nontrauma-focused options; yet, a symptom of PTSD is avoidance, which can lead to patients avoiding the most effective treatments. In earlier rounds of feedback, clinicians mentioned that they hoped the DA would encourage patients to consider being willing to talk about (or write about) their trauma. For this reason, after much discussion, we opted to ask patients whether they were open to talking about their trauma, but we also presented basic information about why it might be useful as well as a link to a quote from a veteran about his experience with talking about trauma in PTSD treatment. The full text of the question thus reads: