Decision Aid for Colectomy in Recurrent Diverticulitis: Development and Usability Study

Introduction

Diverticular disease is the eighth most prevalent outpatient gastrointestinal diagnosis with over 2.7 million clinic visits yearly. Diverticulitis accounts for over 216,000 inpatient admissions, with an aggregate cost of US $2.2 billion [1]. While most cases of uncomplicated diverticulitis are treated with antibiotics alone, the incidence of recurrence after an initial episode is as high as 35% [2-7]. A decade ago, elective resection was recommended after 2 episodes of uncomplicated diverticulitis or a single episode in young patients [8]. This practice was based on the idea that recurrence and younger age at onset comprised a more “virulent” syndrome at greater risk for recurrence. These assumptions have recently been challenged and refuted. National guidelines from the American Society of Colon and Rectal Surgeons (ASCRS), in 2020, recommended that “the decision to recommend elective sigmoid colectomy after recovery from uncomplicated acute diverticulitis should be individualized” [9]. However, little guidance is available on how to personalize this decision with each patient.

No evidence indicates the superiority of observation or surgery for recurrent diverticulitis. Surgical complications are well described and include mortality, the need for an ostomy, infection, and other morbidity [10-12]. Conversely, inappropriate observation can lead to continued recurrence with an increased risk of hospitalization, emergency surgery, as well as a decreased quality of life [13,14]. Choosing the option that is most consistent with patient values is critical for optimizing outcomes [15,16]. In addition, there is significant national variation in standardized colon resection ratios for recurrent diverticulitis, with surgeon density and hospital-level factors serving as the main drivers of resection as opposed to patient factors [17]. No research has examined the extent to which shared decision-making is occurring.

Educational decision aids (DAs) and shared decision-making programs have been shown to improve outcomes and reduce decisional conflict in selecting a treatment for diseases, including prostate cancer [18], breast cancer [19], and joint replacement [20]. Recurrent diverticulitis shares several features with joint replacement; both are benign processes where treatment approaches are driven primarily by an improvement in quality of life. Despite this, an extensive literature review of PubMed and Google using the search term “diverticulitis decision aid” identified no current DAs for treatment options for recurrent left-sided diverticulitis. While we hypothesize that such a program would be beneficial for patients with recurrent diverticulitis to improve long-term patient outcomes, function, and satisfaction, there is a foundational gap in the field in that there are no DAs available to support this key aspect of care. The objective of this research is to use rigorous methods, engaging key stakeholders throughout the process, to develop an effective DA to support the selection of either a surgical or monitoring treatment approach in recurrent diverticulitis.

Methods

From December 12 to 15, 2022, we conducted a 4-day design sprint at Vanderbilt University Medical Center in Nashville, Tennessee (Textbox 1). Adhering to the International Patient Decision Aid Standards (IPDAS) [21,22], we used a modified design sprint methodology developed by Google Ventures (Alphabet Inc) to create a prototype DA for considering colectomy in the setting of diverticular disease [23,24]. Design sprint methodology seeks to condense the potentially months-long production cycle of debate, and instead focus a small team on producing a prototype in just a few days. This approach enabled us to rapidly develop a user-centered solution in the form of a prototype that could be tested and revised iteratively, as done elsewhere [25]. The design sprint team comprised 13 participants from diverse backgrounds, including 4 surgeons, 3 behavioral scientists, 2 physician experts in DA development, 1 expert in health information technology, 4 patients who have undergone medical and surgical treatment of diverticulitis, and 1 caregiver of patients with diverticulitis. Core participants of the design sprint team met in person, and all key points of the design sprint were time-constrained to facilitate the timely completion of the project. KB, a masters-level psychologist with extensive qualitative research experience, facilitated the sprint.

Ethical Considerations

Both the design sprint and usability portions of the study were reviewed and approved by the Vanderbilt Institutional Review Board (IRB 220707). Participants involved in the design sprint phase were compensated for their time at a rate of US $20 per hour for patients and US $50 per hour for physicians. Participants involved in the iterative refinement phase were compensated for their time at a rate of US $50 per session for patients and US $200 per session for physicians. This study was not registered as a clinical trial as it only entailed the creation and usability testing of the DA. All data were maintained on a password-encrypted database and the data were analyzed anonymously.

Decision Aid Design

Day 1: Defining the Problem

On day 1, we began by defining the overall goals for our project. Our DA aimed to (1) be a tool for patients to make the best personalized decision possible with the available information; (2) be usable, sustainable, and adaptable; and (3) empower patients and facilitate communication with their surgeons when making a decision regarding colectomy. We then introduced our group’s previous qualitative investigation to assess the key factors that both patients and surgeons evaluate when considering colectomy or observation. The themes identified included limited knowledge about treatment options, difficulty in communication, and uncertainty in important outcomes related to the decision-making process [26,27]. Next, using the input from our core sprint group and stakeholders, we created a “workflow map” outlining the experience of a patient with diverticulitis within the health care system (Multimedia Appendix 1). Drawing upon these themes, we curated a list of potential problems that might arise during each step of the workflow, identifying areas where our DA would be able to improve both patient and clinician decision-making.

We met through videoconference with expert stakeholders (eg, patients, gastroenterologists with a focus on diverticulitis and community surgeons) to assess perceptions of how best to enhance the workflow map and further understand the problems at each node of the patient’s experience within the health care system. Each “problem” was then reframed into an “opportunity” or “question” using the How Might We (HMW) method (Multimedia Appendix 1). The HMW statements were grouped into themes to identify the most useful ideas for building the DA. Major categories identified included (1) facilitating communication, (2) DA characteristics, (3) educational elements, (4) design features, and (5) functionality.

At the end of the conclusion of the first day, we had identified a “sprint target,” a place in the workflow where our group hypothesized the DA would be used. This target was aimed to address the concept: “HMW add efficiency for both the patient and provider?” as the target for the sprint. Thereafter, we frequently revisited our primary goal, reflecting, “Does this design choice meet the need proposed in our goal specifically within the context of our target timeline?”

Day 2: Ideate

On day 2, we reviewed existing DA designs from other areas of health care [18,25,28]. These were chosen by our DA expert as examples of high-quality DAs from a range of clinical decisions. They were presented so that the group could get a wide idea of potential elements to include in the diverticulitis DA. We also interviewed sprint group members who had previously designed and implemented DAs to elicit insight into the development process. Next, this information was contextualized in the setting of our goals, and ideation sessions were held in which each member generated as many ideas as possible in the form of sketches or simple drawings to represent the concept (Multimedia Appendix 2). Thereafter, the team voted to select the most promising solutions, using a democratic voting process. A “super voter” was used to break ties, prevent stagnation, and facilitate the creative process. The super voter is an integral part of the design process. To avoid the tendency to move forward with several good ideas, it is the job of the super voter to decide what exactly will be prototyped. The super voter is excluded from the first round of voting for shortlisting of potential options. Once the group has voted and explained their votes, the super voter selects the strongest concepts to move forward. In our instance, the super voter was the principal investigator.

Day 3: Storyboard

On day 3, design solutions were critiqued and further refined to better meet the needs of the DA in the context of our patient population (Multimedia Appendix 3). Each solution was analyzed in a group setting, where potential pitfalls and improvements were discussed in turn. At the end of the process, the chosen solutions were adapted into a new workflow of our ideal DA. We reviewed this workflow iteratively until we created a storyboard of both a website and a handout for use by patients.

Day 4: Prototype

Finally, on day 4, we created a mock-up of the website and handout. Attention was paid to the content of each webpage, ensuring it would be applicable and understandable to the widest audience possible.

Usability Study Design

The mockup was further developed by the Vanderbilt design department into a working prototype website and handout. This website underwent review and revision by a group of content and design experts, including members from the original sprint team.

We conducted iterative testing and refinement through a mixed methods study using semistructured interviews and surveys from February to May 2023 (Figure 1). Patients were recruited from the Vanderbilt Colorectal Clinic. Inclusion criteria included patients who had previously been seen in consultation for colectomy for recurrent diverticulitis. Exclusion criteria included patients with colo-vesical fistulas, colo-vaginal fistulas, persistent pain, and colonic strictures, as these conditions markedly favor surgical intervention. Semistructured interviews were conducted over videoconferencing using a standardized script developed by a team with experience in qualitative research and patient care (Multimedia Appendix 4). Subjects interacted first with the website and then with the handout. We used video recording to collect responses. We used both scripted questions as well as the think-aloud technique to assess interaction with both elements of the DA. Patients also completed surveys, including the Net Promoter Score (1-item) [29], the System Usability Scale (SUS; 10-item) [30], and the Cultural Sensitivity Questionnaire (10-item) [31].

‎

Analysis Plan

General satisfaction scores were analyzed using top box scoring with the 2 top scores on a 7-point Likert scale considered to be “top box.” The composite SUS score was calculated by reverse scoring even-numbered items so that all items were scored in the same direction. Composite scores were calculated by summing the item responses and multiplying them by 2.5 so that they fell on a scale of 0-100, with 100 representing the greatest usability. Summary statistics were calculated for the composite scores, and frequencies were reported for the overall rating of user-friendliness. The Cultural Sensitivity Questionnaire was calculated by dividing the score for each question by the adjusted question number. A category score greater than 2.5 denotes an acceptable category, and scores of 2.5 or less denote unacceptable categories.

For analysis of the open-ended interview questions, the responses were organized by group (eg, barriers and facilitators) and a thematic analysis of all responses was conducted based on the steps outlined by Braun and Clarke [32]. First, we reviewed all the responses and generated initial themes and categories. These categories were reviewed by 2 of the authors (AF and KB) and confirmed by a third author (AH). Finally, we categorized the themes to provide a description and examples in this report. We quantified the comments in each category to provide a frequency related to participants’ ideas, suggestions, and ideas related to the usability of the DA.

After revisions from this group, the DA underwent iterative usability testing using focused interviews with both patients and surgeons of the target population. In addition to the sprint participants, we met with experts and stakeholders outside the group, including a community surgeon, health care professionals with experience in designing and implementing DAs, and additional patients and caregivers.

Results

At the end of the design sprint, we successfully developed a prototype of a diverticulitis DA in the form of a complimentary interactive website and handout. The DA has 3 discrete sections.

The first section (Figure 2) includes background information designed to increase patient’s knowledge about diverticulitis as a disease entity and to introduce the treatment options of colectomy and observation. This section provides a generic working definition of diverticulitis as a disease process and encourages patients to engage with the website while reflecting on their values regarding their treatment.

‎

The second section (Figure 2) includes detailed information about each treatment option and a section on the risks and benefits of surgery and observation. This section provides patients with a framework for what their life might look like after colectomy or observation, empowering them with information to reflect on before their surgical appointment. After patients gain a general understanding of the disease process and their options, they proceed to the interactive portion of the DA.

In the third section (Figure 2), patients are invited to engage in a values clarification exercise that will help them prepare for a discussion with their surgeon. They are instructed to pick the 5 activities that are most meaningful in their lives and rank them in order from most to least important. Thereafter, patients are provided the opportunity to engage in self-reflection regarding fears, priorities, and important topics they want to discuss with their surgeon concerning their diverticular disease, through several open-ended questions to document their answers before their appointment. Finally, on a Likert scale between “surgery,” “maybe surgery,” “unsure,” “maybe medicine,” and “medicine,” patients are asked to pick what they would choose now for their treatment option after reading the relevant background information and completing the DA. Following completion, patients can print as well as email a report of this activity.

Patients are expected to bring their completed clarification exercise to their clinic appointment to help initiate the conversation with their surgeon. The DA serves as a guide for shared decision-making and equips surgeons with the relevant information to help guide patients in seeking the best individualized treatment for their recurrent uncomplicated diverticulitis.

After initial drafting based on the design sprint prototype, the diverticulitis DA underwent 2 phases of usability testing with patients. First, semistructured interviews were conducted with both patients and surgeons with associated surveys. Table 1 reports patient demographics for both rounds (Table 1). Table 2 reports patient satisfaction with the DA from the initial round (Table 2). Overall, patients reported being highly satisfied with both the website and the handout. Regarding the Net Promoter Scale, 80% reported that they would be “extremely likely to recommend website”. The SUS scores were both >90%. The Cultural Sensitivity Questionnaire mean score was 3.7 (SD 0.16), indicating acceptable sensitivity. Almost universally, surgeons were positive about the tool and in agreement with the content and material. Illustrative quotes from the patient and surgeon interviews are reported in Tables 3 and 4, respectively.

Discussion

We applied a modified version of the design sprint methodology to create an educational decision aid in the form of a deliverable webpage and handout aimed at facilitating communication between patients and surgeons alike. By using this design approach, we rapidly created a prototype that is constructed to be informative, sustainable, adaptable, and widely applicable. Our previous research identified several key decisional needs in the complex decision-making process that we made sure to address in our final product [26,27]. We incorporated feedback from our expert consultants, stakeholders, and patients to ensure the decision aid was informative yet easy to use and relevant. Examples of this include changing the website navigation to make it more intuitive, the addition of pain to the values clarification exercise, and additional content regarding ostomies to both the information portion and the FAQ sections. Assessments of satisfaction and usability indicated high levels of both with adequate cultural sensitivity.

This study has several strengths. This is the first DA designed to facilitate difficult conversations between surgeons and patients with recurrent uncomplicated diverticulitis about treatment options per American Society of Colon and Rectal Surgeons guidelines [9]. Our design process offers a road map for quality DA development that builds on International Patient Decision Aid Standards guidelines, incorporates feedback from stakeholders, and highlights the importance of iterative usability testing. Initial testing indicates high levels of satisfaction and usability. In addition, the DA is online, which makes it relatively easy to update as information changes. This allows for easier integration with health care systems and dissemination to patients before a visit.

There are limitations to this study. The initial design sprint was performed at a single academic medical center, which raises concerns about generalizability. Although a standard 5-day approach is recommended as initially developed by Google Ventures [23,24], this relies on the premise that participants have a strong understanding of the disease process as well as user needs and challenges. Additional time or informative sessions may be needed to lay the necessary groundwork before initiating the sprint design process.

While we present the initial development of this DA here, additional work is required before it may be integrated into clinical practice. The next steps include feasibility testing and preliminary data collection on the implementation of the DA in the form of a pilot trial, followed by a multicenter, randomized controlled trial comparing the decision aid with standard clinical consultation. During these stages, we will use focused interviews of both providers and patients to further improve the tool. Should efficacy be demonstrated, an implementation study will be crucial to ensure appropriate integration into clinical workflow.

Our study illustrates the use of a modified version of a design sprint methodology to create a DA for patients with recurrent diverticulitis considering colectomy aimed at facilitating communication between patient and clinician that was well received by patients. Our experience with this method illustrates the value of the design sprint methodology in the creation of tools to improve the overall care of patients.