This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Upper extremity (UE) vascularized composite allotransplantation (VCA; hand transplantation) is a reconstructive treatment option for patients with UE loss. Approximately 37 UE VCAs have been performed in the United States to date; thus, little is known about long-term psychosocial outcomes and whether the benefits outweigh the risks. To make an informed treatment decision, patients must understand the procedure, risks, and potential benefits of UE VCA. However, few educational resources are publicly available providing unbiased, comprehensive information about UE VCA.
This paper described the development of a neutral, and accessible, educational website supporting informed decision-making about UE VCA as a treatment option for individuals with UE amputations.
Website content development was informed by 9 focus groups conducted with individuals with UE amputations at 3 study sites. After initial website development, we conducted usability testing to identify ways to improve navigability, design, content, comprehension, and cultural sensitivity. Participants were administered the After-Scenario Questionnaire to assess user performance after completing navigational tasks, System Usability Scale to measure the perceived usability of the website, and Net Promoter Score to measure user satisfaction. Quantitative data were analyzed using descriptive statistics. Qualitative data were analyzed using rapid thematic analysis.
A total of 44 individuals with UE amputations participated in focus groups (n=37, 84%) and usability testing (n=14, 32%). Most participants in the focus groups and usability testing were male (24/37, 65% and 11/14, 79%, respectively) and White (27/37, 73% and 9/14, 64%, respectively), had unilateral limb loss (22/37, 59% and 12/14, 86%, respectively), and had mean ages of 48 (SD 9.2) and 50 (SD 12.0) years, respectively. Focus group results are organized into accessibility, website design, website development, website tone and values, sitemap, terminology, images and videos, and tables and graphics. Usability testing revealed that participants had a positive impression of the website. The mean After-Scenario Questionnaire score of 1.3 to 2.3 across task scenarios indicated high satisfaction with website usability, the mean System Usability Scale score of 88.9 indicated user satisfaction with website usability, and the mean Net Promoter Score of 9.6 indicated that users were enthusiastic and would likely refer individuals to the website.
The findings suggest that our educational website,
Vascularized composite allotransplantation (VCA) is a reconstructive option that involves the transplantation of multiple tissues such as skin, muscle, bone, fat, nerves, and lymph nodes as a functional unit from primarily deceased donors [
VCA raises many ethical issues, particularly regarding informed consent [
However, few comprehensive educational resources exist regarding UE VCA as a treatment option. Although some UE VCA programs may provide information about this procedure on their institutional websites, their neutrality may not be apparent as they present either the positive or negative facets of this treatment option. Current VCA resources do not adequately educate the public [
Extensive education is needed to prepare individuals with UE amputations to make informed treatment decisions regarding UE VCA because of the ethical complexity of this procedure. An increasing number of patients use the internet, specifically websites, as a source of information on health topics [
We developed a neutral, patient-centered website,
The research team constituted a collaboration among 3 study sites: Northwestern University (NU), Johns Hopkins University (JHU), and Walter Reed National Military Medical Center (WR). Partners at NU affiliate Shirley Ryan AbilityLab and David Rotter Prosthetics collaborated. Data were collected from September 2021 to September 2022.
Individuals eligible for participation included English-speaking adult (aged 18-65 years) civilians and military service members with an acquired unilateral or bilateral UE amputation, UE VCA
Eligible individuals were recruited through collaborating sites or through community and online support groups for individuals with UE amputations. Each research site mailed or emailed introductory letters to all potentially eligible participants followed by a phone call 1 week later to assess interest in participation. Patients in support groups were recruited through listserves, emails, or web-based postings advertising the study using a flyer; interested individuals contacted the study team directly.
The institutional review boards at NU (STU00209718), JHU (00225728), and WR (WR-EDO-2020-0432, relying on the NU institutional review board) approved this study. The US Army Medical Research and Development Command Human Research Protection Office approved this study at NU (E00798.1a), JHU (E00800.1a, E00799.1a), and WR (E00801.1a). Verbal informed consent was obtained from potential participants before enrollment in the study. Participants were compensated with US $35 for their time.
The website is intended to be used by individuals with hand and upper limb amputations, their families, and their health care providers. This study used a cross-sectional approach for qualitative data collection involving telephone and web-based focus groups. We supplemented data collection with mixed methods research for refinement and usability testing of educational materials. This information enabled the elaboration and clarification of the findings, increasing the validity of the results, and informed subsequent data collection [
Website development was guided by the Health On the Net Foundation code of conduct certification guidelines, which provide credibility that the website follows a code of ethics ensuring that it provides quality information [
The selection of medical content for the website was guided by elements of informed consent, including the risks, benefits, procedures, alternatives, and voluntary nature of UE VCA. In addition, content was driven by a review of the literature; conversations with health care providers working in UE VCA (eg, UE VCA clinicians or surgeons, hand reconstructive surgeons, and occupational therapists); in-depth and semistructured interviews with individuals with UE loss and UE VCA candidates, participants, and recipients about their information needs regarding UE VCA [
The discovery phase entailed determining the mission, goals, target audience, and content of the website. The website is intended to be a patient-centered resource for health information about upper extremity vascularized composite allotransplantation (VCA). The website is Americans with Disabilities Act–compliant, which includes the use of UserWay (UserWay Inc), an accessibility plug-in that increases Web Content Accessibility Guidelines 2.0 compliance. The content and design of the website were derived from published data on upper extremity VCA interviews and focus groups with members of the target audience.
Planning involved developing a website sitemap and a list of topics and subtopics. The sitemap guided the site’s development of content and the navigational system. The website provides information about VCA, including steps in the evaluation process, factors to consider during decision-making, risks of surgery, rehabilitation process, alternative treatment options, and resources.
The design process required determining the appearance of the site. The research team provided the website developer with design examples of health and educational websites that the team preferred. Mock-ups were developed iteratively and jointly by the research team and the website developer based on feedback from focus group participants to ensure that the content had face validity. The content was revised when several participants suggested changes, areas arose as problematic, or the rationale for change was sound. VCA clinicians provided input on the accuracy of the website content.
The development process required creating the actual functional website. The website used quality assurance strategies (ie, the 16-item validated
Testing and delivery involved testing the functionality of the website to ensure that there were no broken links. The website developer ensured that the website was compatible with the Americans with Disabilities Act requirements. The website underwent a final test to ensure that all aspects were functioning correctly before launching it.
Maintenance entails a long-term strategy to update information on the website. The research team collaborated in updating and editing the website during its development. When the study was completed, the site URL was posted on the American Society for Reconstructive Transplantation server.
This screenshot illustrates the steps in the upper extremity vascularized composite allotransplantation evaluation process. OPO: organ procurement organization; UNOS: United Network for Organ Sharing.
The theoretical frameworks that guided website development to enable learning were Social Cognitive Theory by Bandura [
We conducted 9 focus groups (n=3, 33% per site) via teleconference or videoconference. Focus groups were conducted to gather user preferences on website wireframe (ie, blueprint) concepts, content, design, and functionality regarding the user-centered design [
Participants were asked what they liked and disliked, their perceptions of the website’s cultural sensitivity to the community of individuals with UE amputations, and how they would improve website content [
Usability testing was conducted at 2 sites (NU and WR) to assess user experience with
Usability testing was conducted in person and via videoconference. During usability testing, the website was viewed exclusively on laptops provided by the research staff in person or on computers owned by the study participant at each site via videoconferencing. The website was not trialed on a phone or tablet to maintain the consistency and reliability of the results as the website configuration differs slightly depending on whether it is accessed via laptop or mobile phone.
The research staff observed each participant as they freely navigated the website and then completed 5 task scenarios. The task scenarios required participants to find specific information on the website while speaking aloud to convey the thought processes informing their decision to use a certain navigation route. This activity revealed which website sections needed content or design modifications to improve website navigability. The research team tracked website usability metrics (eg, time needed to find sections and satisfaction with navigation and content).
For each task scenario, participants were asked the following: “How easy was it to find what you were looking for?” which assessed information findability on a Likert scale (range 1-5), and “How satisfied are you with the information presented in this section?” which assessed information satisfaction on a Likert scale (range 1-5). Participants completed the After-Scenario Questionnaire (ASQ), a 3-item, 7-point Likert scale anchored by 1=“strongly agree” to 7=“strongly disagree” assessing satisfaction with the usability of the website based on each task scenario [
Upon completing the task scenarios, participants were asked standardized survey questions to determine website usability. Participants completed the System Usability Scale (SUS), a 10-item, 5-point Likert scale that assessed the usability of the website (Cronbach α=.85) [
Descriptive statistics were used on the post–focus group survey items and usability testing items assessing attitudes toward UE VCA, satisfaction with the website, ASQ scores, SUS scores, NPS scores, perceptions of cultural sensitivity, and demographics. We calculated frequencies, means, and SDs. Analyses were performed using SPSS Statistics (version 27; IBM Corp).
A rapid thematic analysis was applied to obtain feedback on the
Of the 138 eligible participants contacted, 37 (26.8% participation rate) enrolled in the focus groups (
Focus group participants’ sociodemographic characteristics (n=37).
Characteristics | Total | NUa (n=12) | JHUb (n=14) | WRc (n=11) | |||||
Age (years), mean (SD; range) | 48.3 (9.2; 32-66) | 52.7 (5.8; 42-60) | 45.6 (10.9; 32-64) | 47 (8.7; 35-66) | |||||
|
|||||||||
|
Male | 24 (65) | 8 (67) | 8 (57) | 8 (73) | ||||
|
Female | 13 (35) | 4 (33) | 6 (43) | 3 (27) | ||||
|
|||||||||
|
Not Hispanic or Latino | 34 (92) | 12 (100) | 13 (93) | 9 (82) | ||||
|
Hispanic or Latino | 2 (5) | 0 (0) | 0 (0) | 2 (18) | ||||
|
|||||||||
|
White | 27 (73) | 10 (83) | 12 (86) | 5 (45) | ||||
|
Black or African American | 6 (16) | 1 (8) | 2 (14) | 3 (27) | ||||
|
Other | 4 (11) | 1 (8) | 0 (0) | 3 (27) | ||||
|
|||||||||
|
Married or domestic partner or civil union | 23 (62) | 7 (58) | 7 (50) | 9 (82) | ||||
|
Separated or divorced | 9 (24) | 4 (33) | 3 (21) | 2 (18) | ||||
|
Never married or single | 5 (14) | 1 (8) | 4 (29) | 0 (0) | ||||
|
|||||||||
|
High school graduate | 6 (16) | 2 (17) | 2 (14) | 2 (18) | ||||
|
Some college | 8 (22) | 2 (17) | 3 (21) | 3 (27) | ||||
|
College graduate | 15 (41) | 5 (42) | 5 (36) | 5 (45) | ||||
|
Postgraduate degree | 8 (22) | 3 (25) | 4 (29) | 1 (9) | ||||
Health literacy (adequate), n (%) | 35 (95) | 11 (92) | 12 (86) | 8 (73) | |||||
|
|||||||||
|
Employed full time | 12 (32) | 6 (50) | 3 (21) | 3 (27) | ||||
|
Employed part time | 2 (5) | 0 (0) | 2 (14) | 0 (0) | ||||
|
Not employed | 2 (5) | 0 (0) | 2 (14) | 0 (0) | ||||
|
Homemaker | 2 (5) | 0 (0) | 1 (7) | 1 (9) | ||||
|
Student | 1 (3) | 0 (0) | 0 (0) | 1 (9) | ||||
|
Disabled | 7 (19) | 3 (25) | 3 (21) | 1 (9) | ||||
|
Retired | 11 (30) | 3 (25) | 3 (21) | 5 (45) | ||||
|
|||||||||
|
<15,000 | 2 (5) | 2 (17) | 1 (7) | 1 (9) | ||||
|
Between 15,000 and 34,999 | 5 (14) | 0 (0) | 3 (21) | 0 (0) | ||||
|
Between 35,000 and 54,999 | 2 (5) | 1 (8) | 1 (7) | 0 (0) | ||||
|
Between 55,000 and 74,999 | 4 (11) | 3 (25) | 1 (7) | 0 (0) | ||||
|
Between 75,000 and 94,999 | 8 (22) | 0 (0) | 3 (21) | 5 (45) | ||||
|
>95,000 | 10 (27) | 5 (42) | 2 (14) | 3 (27) | ||||
|
Prefer not to answer | 6 (16) | 1 (8) | 3 (21) | 2 (18) | ||||
|
|||||||||
|
Private | 13 (35) | 5 (42) | 6 (43) | 2 (18) | ||||
|
Medicaid or Medicare | 19 (51) | 6 (50) | 8 (57) | 5 (45) | ||||
|
Tricare | 10 (27) | 0 (0) | 2 (14) | 8 (73) | ||||
|
Other | 4 (11) | 1 (8) | 1 (7) | 2 (18) | ||||
|
|||||||||
|
Excellent | 5 (14) | 1 (8) | 3 (21) | 1 (9) | ||||
|
Very good | 15 (41) | 6 (50) | 4 (29) | 5 (45) | ||||
|
Good | 12 (32) | 4 (33) | 5 (36) | 3 (27) | ||||
|
Fair | 4 (11) | 1 (8) | 2 (14) | 1 (9) | ||||
|
Poor | N/Ag | N/A | N/A | N/A | ||||
|
|||||||||
|
Yes | 24 (65) | 9 (75) | 7 (50) | 8 (73) | ||||
|
No | 11 (30) | 3 (25) | 5 (36) | 3 (27) | ||||
|
|||||||||
|
Right | 12 (32) | 4 (33) | 2 (14) | 6 (55) | ||||
|
Left | 10 (27) | 3 (25) | 6 (43) | 1 (9) | ||||
|
Both | 13 (35) | 5 (42) | 4 (29) | 4 (36) | ||||
|
Prefer not to answer | 2 (5) | 0 (0) | 2 (14) | 0 (0) | ||||
|
|||||||||
|
Unilateral | 22 (59) | 7 (58) | 8 (57) | 7 (64) | ||||
|
Bilateral | 13 (35) | 5 (42) | 4 (29) | 4 (36) | ||||
|
Prefer not to answer | 2 (5) | 0 (0) | 2 (14) | N/A | ||||
|
|||||||||
|
Below elbow | 22 (59) | 9 (75) | 6 (43) | 7 (64) | ||||
|
Above elbow | 15 (41) | 3 (25) | 8 (57) | 4 (36) | ||||
|
|||||||||
|
Cosmetic | 2 (5) | 0 (0) | 1 (7) | 1 (9) | ||||
|
Mechanic | 15 (41) | 7 (58) | 2 (14) | 6 (55) | ||||
|
Myoelectric | 16 (43) | 3 (25) | 6 (43) | 7 (64) | ||||
|
None | 10 (27) | 3 (25) | 6 (43) | 1 (9) | ||||
|
|||||||||
|
<1 | 1 (3) | 0 (0) | 1 (7) | 0 (0) | ||||
|
1-2 | 6 (16) | 1 (8) | 4 (29) | 1 (9) | ||||
|
3-5 | 6 (16) | 3 (25) | 3 (21) | 0 (0) | ||||
|
6-9 | 5 (14) | 2 (17) | 2 (14) | 1 (9) | ||||
|
10-15 | 10 (27) | 3 (25) | 3 (21) | 4 (36) | ||||
|
16-25 | 3 (8) | 1 (8) | 0 (0) | 2 (18) | ||||
|
>25 | 6 (16) | 2 (17) | 1 (7) | 3 (27) | ||||
|
|||||||||
|
Person with UE amputation | 30 (81) | 11 (92) | 8 (57) | 11 (100) | ||||
|
VCAk candidate or participant | 5 (14) | 1 (8) | 4 (29) | 0 (0) | ||||
|
VCA recipient | 2 (5) | 0 (0) | 2 (14) | 0 (0) |
aNU: Northwestern University.
bJHU: Johns Hopkins University.
cWR: Walter Reed National Military Medical Center.
dPercentages do not add up to 100 as some participants did not respond.
e“Other” included people who identified as American Indian or Alaska Native (1/4, 25%), Asian (1/4, 25%), Native Hawaiian or other Pacific Islander (1/4, 25%), or Malagasy (1/4, 25%).
fPercentages do not add up to 100 as some participants had multiple forms of insurance.
gN/A: not applicable.
hUE: upper extremity.
iPercentages do not add up to 100 as some participants were using multiple prostheses.
jSome participants had multiple surgeries for their amputation or multiple amputations.
kVCA: vascularized composite allotransplantation.
The website content and design entailed addressing the unique challenges that individuals with a UE amputation may encounter when using the website (
Website changes based on focus group feedback.
Design element and subcategory | Original version | Final version | |||
|
|||||
|
Tiles | Links to each subsection of the corresponding section | A matrix of tiles for each subsection provides a larger surface area for users to click on the desired subsection. | ||
|
Search button | Absent | Users could search for specific keywords and be taken directly to relevant sections of the website. | ||
|
Voice-over | Absent | Recorded narrative descriptions of figures and graphs. | ||
|
|||||
|
Gaining attention | Short, interesting factoids about the history, outcomes, and process, among other things, of UEa VCAb were included in a list of questions called “Did You Know?” to gain the attention of users (eg, “Did you know that the first UE VCA was performed in 1999?”). | No change | ||
|
Informing the learner of the objective | Each of the 8 main sections of the website starts with 1-2 sentences explaining the objective of that section. | No change | ||
|
Stimulating recall of previous learning | The “Myths and Facts” subsection under the “Hand/Arm Transplant” section allows for knowledge application by testing the user’s accurate knowledge of a topic. | No change | ||
|
Presenting the stimulus or content | The website content is presented in eight main sections: (1) “Hand and Arm Transplant,” (2) “Process,” (3) “Risks,” (4) “Recovery,” (5) “Options,” (6) “Decision-Making,” (7) “Resources,” and (8) “About Us.” | No change | ||
|
Providing learning guidance | The website uses video testimonials of UE VCA recipients, participants, and candidates as well as UE VCA providers to supplement reading-based learning. | No change | ||
|
Eliciting performance | The “Myths and Facts” section elicits performance by testing users’ knowledge of the accuracy of myth statements. | No change | ||
|
Providing feedback | The “Myths and Facts” section provides feedback by pointing out inaccuracies in the myth statement and providing correct information in the fact statements. | No change | ||
|
Enhancing retention and transfer | The “Question Prompt Sheet” under the “Resources” section provides a list of questions that patients can ask their physicians to learn about hand or arm transplantation. This list of questions enhances the patients’ retention of important UE VCA topics and offers transfer through real-world application in discussion with providers. | No change | ||
|
|||||
|
Target audience | Participants expressed difficultly identifying the target audience of the website when visiting the home page. | The home page includes a statement that the website is designed to help individuals with UE amputations, their families, and health care professionals make informed treatment decisions. | ||
|
|||||
|
Reading level | Participants expressed difficulty in understanding subsection headers such as “voluntariness” and “psychosocial.” | Sitemap headers that were difficult to understand were replaced with lower-reading–grade-level language such as “optional treatment” and “emotional and social.” | ||
|
|||||
|
Language sensitivity | The research team was concerned that the word “amputee” might be offensive to website users as it does not use “people-first” language. | All instances of the word “amputee” on the website were changed to “people with upper limb amputations.” | ||
|
Plain language | Participants urged that the website refrain from using medical jargon. | Website language was changed to use plain language. | ||
|
|||||
|
Diversity | Participants requested that images on the website display UE VCA recipients of diverse sexes, races or ethnicities, and nationalities. | The website included images of recipients from diverse backgrounds, including nationality, age, sex, and race or ethnicity. | ||
|
Cosmetic outcomes | Participants reported wanting to see images of the transplanted limb, including scarring. | Images displaying recipients using their UE VCA to do functional tasks while the scarring of the limbs was visible were included on the website. | ||
|
Functional outcomes | Participants wanted images to showcase UE VCA recipients doing functional tasks that might be difficult to perform living with an amputation. | More action shots were included to show recipients performing activities such as brushing their hair, doing hand therapy, and playing the guitar. | ||
|
|||||
|
Diversity | Participants did not want the website to have the timeline of number of UE VCA surgeries performed in the United States stratified by the transplant recipients’ race and sex. | The graph showing number of UE VCAs performed stratified by recipient race was removed. The graph stratified by sex was retained to assure website users that UE VCA is available to both sexes. | ||
|
Relevance | Participants expressed that graphs showing willingness to authorize deceased donation of one’s own UE or of one’s deceased family member’s UE was irrelevant to them. | The graph of deceased donation willingness rates was removed. | ||
|
Detail | Participants expressed that the table outlining the costs of the UE VCA surgery and medications contained too much information. | Data about the projected cost of taking an immunosuppressive drug regimen were removed from the table. |
aUE: upper extremity.
bVCA: vascularized composite allotransplantation.
The website design and content were guided by the instructional design strategies of the conditions of learning by Gagne et al [
The website gains users’ attention by using thought-provoking questions (ie, “Did you know that the first UE VCA was performed in 1999?”) and attention-grabbing photos or videos on the home page (
This screenshot illustrates a bilateral upper extremity vascularized composite allotransplantation recipient demonstrating his range of motion.
This screenshot illustrates the surgery and recovery section of the Myths and Facts page.
We followed the International Patient Decision Aid Standards version 4.0 (IPDAS v4.0) guidelines for developing the
The IPDAS criteria are pertinent to ensure that patients are well informed about their treatment options. Although the IPDAS emphasize patient participation in shared decision-making, the use of the
I think it’s good that you’re talking about who you spoke to get this information, the purpose so that you’re not trying to say that you’re for or against it. It’s just simply for information and you’ve come from all these different diverse backgrounds to give them the most comprehensive information that you can.
The website was designed to be a patient-centered resource that provides evidence-based information about UE VCA and does not attempt to “convince the reader” or “push” individuals toward pursuing VCA. Furthermore, the website does not focus heavily on the potential benefits associated with the procedure and thoroughly covers the risks associated with VCA. A participant discussed the importance of the website including the risks and negative experiences or testimonials of UE VCA recipients “because that would tell me I’m not being sold something. I’m being given a fair piece of information” (site 1, focus group 3). The website includes testimonials by UE VCA recipients recounting their experiences with the recovery process.
Focus group participants appreciated that the website did not exclusively feature information on UE VCA. For example, the website’s
Several (9/11, 82%) participants at 1 site explicitly stated that they found it challenging to identify the target audience of the website:
It’s a lot of information and as I look through the website it looks like some of it is geared towards amputees and some of the website is geared, it looks like towards medical professionals.
Few (3/11, 27%) participants reported that they thought that the target audience for the website was military service members because of the affiliation with WR. Furthermore, few (3/11, 27%) participants found it difficult to determine the purpose of the website:
You really have to read into it [website] to understand what the website is about.
...it’s a little confusing trying to figure out what it’s about, until you get to the Purpose section. Without reading, you can’t decipher what’s going on.
On the basis of focus group feedback, the website was revised to include a statement at the top of the home page that explicitly identifies the target audience and the 3 organizations that collaborated to develop the website:
This educational website was developed for all people with upper limb amputations based on a collaboration between Northwestern University, Johns Hopkins University, and Walter Reed National Military Medical Center, funded by the US Department of Defense. (
This screenshot presents the top portion of the home page of the Within Reach website. UCLA: University of California, Los Angeles.
Participants had difficulty understanding some section headers initially listed on the sitemap, specifically words such as “voluntariness” and “psychosocial” (
The website developers strived to use terminology that did not stigmatize individuals living with UE amputations (
Several focus groups raised concerns about the medical terminology used on the website:
Keep it simple and keep the doctor’s terminology out of there.
Focus groups requested that we use “plain English” (site 1, focus group 3) and that we “lower the complexity of the words” (site 1, focus group 2). The focus group prompted revisions to the website to incorporate plain language where appropriate, including not using the term
I don’t want a false sense of reality. I like how it shows the raw hands, the scars. You know, versus people expecting it to be basically, like a miracle where you don’t have any scars, you’re not going to—you know what I mean? So, I like the reality of it.
Participants discussed the importance of photographs reflecting “both sides of [the transplant]” experience and not focusing solely on the positive or negative aspects of the experience. Photographs with scarring that are “in your face” may prompt some individuals with UE amputations to reconsider whether a VCA transplant is the right choice for them. Participants reported that the website should display photographs of recipients using their transplanted arm to perform recreational activities and activities of daily living, which have been incorporated into the website. The website features 29 photographs of VCA recipients, prostheses, and prosthesis users.
The website includes video testimonials from clinicians; individuals with UE amputations; and VCA candidates, participants, and recipients. The testimonials include a VCA recipient discussing experiences with rejection and the challenges of being a VCA recipient. The website hosts 108 patient videos including 2 bilateral recipients and 80 health care provider videos (occupational therapists, transplant social workers, hand surgeons, and hand or arm transplant surgeons). See
Focus groups consistently preferred tables that displayed data on the age of VCA recipients, were less enthusiastic about reporting VCA recipients by gender, and disliked tables on recipients by race owing in part to the incomplete race or ethnicity data reported by the Organ Procurement and Transplant Network or United Network for Organ Sharing (
If other people did it at my age like this, and they’re successful, then that could determine my decision on it.
Some focus groups expressed difficulty understanding how data on recipients by gender were relevant and important to people’s decision-making process. Participants suggested that tables on gender and race should not be included on the website. We retained the gender table to ensure that future viewers could recognize that all genders are eligible to receive a UE VCA.
Most focus groups preferred the graphics displaying the location of transplant centers in the United States with established UE VCA transplant programs, including those programs that have not yet performed a transplant (
This screenshot illustrates a resource that enables users to identify hand and arm transplant programs in the United States. KY: Kentucky; NYU: New York University; UCLA: University of California, Los Angeles.
Several focus groups were surprised by a table displaying the number of individuals with UE amputations on the waiting list for UE transplants in the United States and the number of UE recipients in the United States and worldwide. Focus groups assumed that the numbers presented in the tables were incorrect—“this graph can’t be right” (site 1, focus group 3)—and participants expected the number of recipients and those on the waitlist to be higher:
Yeah, I would’ve thought that those numbers would be higher; that there would be more of these procedures performed.
Focus group participants expressed concern that so few people had received a UE transplant, that it is not a common procedure, and concluded that the procedure would be risky:
When I think of it, there’s a lot of unknowns for me as to the potential risks of having it done.
Focus groups noted that the limited number of UE recipients would be worth bearing in mind when considering UE VCA as a treatment option “because not many have done it, I would not be comfortable doing it” (site 2, focus group 3) and “for me to decide if I ever have a transplant, I would want to know if other people have had it” (site 3, focus group 3). Some (7/26, 27%) participants reported that they perceived the low number of individuals on the waiting list in the United States to mean that they would not have to wait long for the surgery.
Focus groups consistently did not prefer graphs displaying information about willingness to authorize deceased donation of one’s own UE or of one’s deceased family member’s UE as that information was not considered “worthwhile” or relevant to decision-making about receiving a UE VCA. Accordingly, tables on deceased donation were not included. Several focus groups found the table on the cost of UE VCA to be “intimidating” and suggested the “need to dummy it down with a breakdown of the cost, make it simple” (site 3, focus group 1). The revised cost table was simplified by removing the detailed immunosuppressive drug regimen.
Of the 24 eligible participants contacted, 14 (58% participation rate) completed usability testing. Those completing usability testing were evenly divided across NU and WR. Usability testing was conducted until saturation was reached. In total, 4% (1/24) of the participants, who were contacted and scheduled for an interview, were not interviewed once saturation was reached.
Participants had a mean age of 50 years, and most were male (11/14, 79%), White (9/14, 64%), and literate (13/14, 93%) and had undergone a unilateral amputation (12/14, 86%;
Usability testing participants’ sociodemographic characteristics (n=14).
Characteristics | Total | NUa (n=7) | WRb (n=7) | ||||
Age (years), mean (SD; range) | 50.0 (12.0; 27-66) | 56.6 (6.7; 45-66) | 42.7 (12.4; 27-65) | ||||
|
|||||||
|
Male | 11 (79) | 6 (86) | 5 (71) | |||
|
Female | 3 (21) | 1 (14) | 2 (29) | |||
|
|||||||
|
Not Hispanic or Latino | 13 (93) | 7 (100) | 6 (86) | |||
|
Hispanic or Latino | 1 (7) | 0 (0) | 1 (14) | |||
|
|||||||
|
White | 9 (64) | 5 (71) | 4 (57) | |||
|
Black or African American | 2 (14) | 1 (14) | 1 (14) | |||
|
Native Hawaiian or other Pacific Islander | 1 (7) | 1 (14) | 0 (0) | |||
|
Multiracial | 2 (14) | 0 (0) | 2 (29) | |||
|
|||||||
|
Married or domestic partner or civil union | 12 (86) | 6 (86) | 6 (86) | |||
|
Separated or divorced | 1 (7) | 0 (0) | 1 (14) | |||
|
Never married or single | 1 (7) | 1 (14) | 0 (0) | |||
|
|||||||
|
Some college | 5 (36) | 2 (29) | 3 (43) | |||
|
College graduate | 5 (36) | 2 (29) | 3 (43) | |||
|
Postgraduate degree | 4 (29) | 3 (43) | 1 (14) | |||
Health literacy (adequate), n (%) | 13 (93) | 7 (100) | 6 (86) | ||||
|
|||||||
|
Employed full time | 7 (50) | 3 (43) | 4 (57) | |||
|
Employed part time | 1 (7) | 1 (14) | 0 (0) | |||
|
Homemaker | 1 (7) | 0 (0) | 1 (14) | |||
|
Disabled | 1 (7) | 1 (14) | 0 (0) | |||
|
Retired | 4 (29) | 2 (29) | 2 (29) | |||
|
|||||||
|
<15,000 | 0 (0) | 0 (0) | 0 (0) | |||
|
Between 15,000 and 34,999 | 1 (7) | 1 (14) | 0 (0) | |||
|
Between 55,000 and 74,999 | 1 (7) | 1 (14) | 0 (0) | |||
|
Between 75,000 and 94,999 | 5 (36) | 1 (14) | 4 (57) | |||
|
>95,000 | 5 (36) | 3 (43) | 2 (29) | |||
|
Prefer not to answer | 2 (14) | 1 (14) | 1 (14) | |||
|
|||||||
|
Private | 4 (29) | 3 (43) | 1 (14) | |||
|
Tricare | 5 (36) | 0 (0) | 5 (71) | |||
|
Medicaid or Medicare | 4 (29) | 3 (43) | 1 (14) | |||
|
None | 1 (7) | 1 (14) | 0 (0) | |||
|
|||||||
|
Excellent | 5 (36) | 3 (43) | 2 (29) | |||
|
Very good | 5 (36) | 1 (14) | 4 (57) | |||
|
Good | 3 (21) | 3 (43) | 0 (0) | |||
|
Fair | 1 (7) | 0 (0) | 1 (14) | |||
|
|||||||
|
Yes | 12 (86) | 7 (100) | 5 (71) | |||
|
No | 2 (14) | 0 (0) | 2 (29) | |||
|
|||||||
|
Right | 7 (50) | 3 (43) | 4 (57) | |||
|
Left | 5 (36) | 2 (29) | 3 (43) | |||
|
Both | 2 (14) | 2 (29) | 0 (0) | |||
|
|||||||
|
Unilateral | 12 (86) | 5 (71) | 7 (100) | |||
|
Bilateral | 2 (14) | 2 (29) | 0 (0) | |||
|
|||||||
|
Below elbow | 7 (50) | 5 (71) | 2 (29) | |||
|
Above elbow | 6 (43) | 1 (14) | 5 (71) | |||
|
Below elbow and above elbow | 1 (7) | 1 (7) | 0 (0) | |||
|
|||||||
|
Mechanic | 8 (57) | 3 (43) | 5 (71) | |||
|
Myoelectric | 6 (43) | 4 (57) | 2 (29) | |||
|
None | 1 (7) | 0 (0) | 1 (14) | |||
|
|||||||
|
<2 | 2 (14) | 1 (14) | 1 (14) | |||
|
2-8 | 2 (14) | 0 (0) | 2 (29) | |||
|
9-15 | 2 (14) | 1 (14) | 1 (14) | |||
|
16-25 | 4 (29) | 2 (29) | 2 (29) | |||
|
>25 | 4 (29) | 3 (43) | 1 (14) |
aNU: Northwestern University.
bWR: Walter Reed National Military Medical Center.
cPercentages do not add up to 100 as some participants used multiple prostheses.
Participants reported an overall positive impression of
...it seems to be really pretty straightforward just when you open the page and like you can find what you want at the top. So, like you have your drop-down, you can click on what you want to click on...Pretty easy to navigate.
A participant with a bilateral amputation said the following regarding their ability to navigate the website:
I found it easy to navigate, especially using a mouse or trackball.
Participants consistently reported that the website was sensitive to people with UE amputation:
I like the feel of this website. I think it addresses the needs of people with limb loss and limb difference, a neutral way.
It seems like there was a pretty good understanding of experiences that folks have as far as how you view yourself and limb loss and stuff like that, so that seems pretty insightful and straightforward. So, I appreciated that.
Participants found the website to be “neutral” or “unbiased” regarding the portrayal of UE transplantation as a treatment option for people with UE amputations. A participant stated the following:
So internally what I was doing was I was looking at the different sections that you had and was there a balance between pros and cons and availabilities and looking at this seems like very, you know, very neutral. And again, my nuances are definitely in the weeds based upon my prior experience history of being an amputee and involvement with the different communities and medical communities and things of that nature.
Most participants’ favorite section of the website was the
The Options page was very interesting and [I] keep coming back to that because then it better informs me as to why my doctor’s notes are decisions and offered up certain options that I was, you know, given, like I was not given the battery-powered [prosthesis] option that was never made available to me.
Overall, participants reported that they found the website culturally sensitive to the target population of people with UE amputations (
I think you got everything; the tutorials are one of the best. I always think that tutorials are one of the best things to use because you can go with not looking at it, but you listen to people.
Cultural sensitivity (n=14).
Question | Strongly agree, n (%) | Agree, n (%) | Neutral, (%) | Disagree, n (%) | Strongly disagree, n (%) |
The words, phrases, and expressions are familiar to the intended audience. | 8 (57) | 5 (36) | 0 (0) | 1 (7) | 0 (0) |
The words, phrases, and expressions are free from stereotypical meaning. | 8 (57) | 5 (36) | 1 (7) | 0 (0) | 0 (0) |
The message is linked to sources credible to the intended audience. | 8 (57) | 2 (14) | 2 (14) | 1 (7) | 1 (7) |
The message addresses stereotypes and myths. | 8 (57) | 4 (29) | 2 (14) | 0 (0) | 0 (0) |
The graphics accurately depict the physical features (eg, hairstyle and clothes) of the intended audience. | 6 (43) | 4 (29) | 3 (21) | 1 (7) | 0 (0) |
Symbols are representative of the intended audience. | 3 (21) | 7 (50) | 4 (29) | 0 (0) | 0 (0) |
The stature and poise of the individual is representative of the gender and social roles of the intended audience. | 4 (29) | 8 (57) | 1 (7) | 1 (7) | 0 (0) |
The educational materials are culturally sensitive to ethnic minority communities. | 3 (21) | 4 (29) | 5 (36) | 2 (14) | 0 (0) |
The website materials seemed neutral and unbiased; that is, the website was neither in favor of nor against hand or upper limb transplantation. | 8 (57) | 4 (29) | 0 (0) | 2 (14) | 0 (0) |
The ASQ mean scores for each task scenario ranged from 1.3 to 2.3 (
Satisfaction scores of usability testing participants (n=14).
|
Total, mean (SD; range) | NUa (n=7), mean (SD; range) | WRb (n=7), mean (SD; range) | |
|
||||
|
Task scenario 1d | 2.0 (1.5; 1.0-7.0) | 1.9 (0.6; 1.0-2.7) | 2.0 (2.2; 1.0-7.0) |
|
Task scenario 2e | 1.3 (0.6; 1.0-3.0) | 1.5 (0.7; 1.0-3.0) | 1.1 (0.4; 1.0-2.0) |
|
Task scenario 3f | 2.0 (1.2; 1.0-4.3) | 2.4 (1.0; 1.0-4.0) | 1.6 (1.2; 1.0-4.3) |
|
Task scenario 4g | 2.3 (1.2; 1.0-4.7) | 2.4 (1.4; 1.0-4.7) | 2.2 (1.1; 1.0-4.0) |
|
Task scenario 5h | 1.4 (0.8; 1.0-4.0) | 1.1 (0.2; 1.0-1.3) | 1.6 (1.1; 1.0-4.0) |
SUSi | 88.9 (10.6; 70-100) | 87.5 (11.1; 70-100) | 90.0 (10.7; 75-100) | |
NPSj | 9.6 (0.6; 8-10) | 9.6 (0.8; 8-10) | 9.7 (0.5; 9-10) | |
Overall website experiencek | 6.1 (0.9; 4.0-7.0) | 6.1 (0.7; 5.0-7.0) | 6.0 (1.2; 4.0-7.0) |
aNU: Northwestern University.
bWR: Walter Reed National Military Medical Center.
cASQ: After-Scenario Questionnaire.
dTask 1: What functions and sensations can you expect to gain after getting a hand or arm transplant?
eTask 2: How can you tell if you are qualified to get a hand or arm transplant?
fTask 3: There are a lot of pros and cons to a hand or arm transplant, and people have to figure out whether this is the right option for them. How do you know if this option is worthwhile for you?
gTask 4: Hand or arm transplant recipients might experience changes in mood after the transplant. What are different ways that recipients emotionally respond to the transplant?
hTask 5: What does hand therapy involve and for how long do recipients need to do it?
iSUS: System Usability Scale.
jNPS: Net Promoter Score.
k“Rate your overall experience with this website” assessed overall website experience on a scale of 1 to 7.
The mean SUS score was 88.9 (SD 10.6; range 70-100;
Participants reported an overall NPS score of +93 (
Participants recommended changing website content, format, and functionality by adding photographs of an individual of African ancestry to the carousel on the home page, adding graphic photos, adding disclaimers to the
I mean really if you’re going to add anything, maybe a voice command type of thing. But not as detailed as like a Siri, but something more focused with keywords or even like in your glossary of terms. I think it would–wouldn’t be as frustrating if somebody didn’t have the dexterity.
In addition, we did not include graphic photos on the website as participants had conflicting views on whether to include graphic photos or images of the UE VCA procedure. Participants recommended a chart comparing the costs of prosthetic options, but we did not make this change as costs are affected by insurance coverage and individual patient circumstances. On the basis of feedback, we changed the format of the video drop-downs at the bottom of each page from a horizontal bar to underlined text with a left-justified header to make the drop-down feature more apparent. In addition, pages with larger paragraphs were broken down into several easier-to-digest bullets.
The
The NPS score of +93 reflects high acceptance and satisfaction with
We recommend integrating
We also recommend that health care professionals rely on
We learned many lessons in developing the
We also realized that recruiting participants with UE loss is exceptionally challenging. The UE loss community is small. In addition, this is a heavily studied population as many initiatives aim to help improve the lives of those with limb loss. Therefore, some health care providers and support group communities were not receptive to disseminating information for study participation among the qualified individuals within their community. Moreover, snowball sampling was not an effective recruitment strategy for this population.
Future research should evaluate whether the use of
The strengths of this study include the multisite design and participant recruitment and representation from geographically diverse US regions, suggesting the generalizability and transferability of the findings. The mixed methods design enabled us to qualitatively and quantitatively assess perceptions of the
This study has several limitations. The study sample comprised primarily White, middle-aged, well-educated men several years after amputation, which may limit the transferability of our findings to individuals with UE amputations of different racial, ethnic, and gender backgrounds. Usability testing was conducted in person and using a videoconferencing platform; remote testing may have made observation of participants’ navigation more challenging in terms of detecting subtle nonverbal communication in body posture and facial expression. Our data suggest that SUS ratings differed slightly between UE VCA candidates, participants, and recipients compared with individuals not interested in pursuing VCA. A larger study sample of UE VCA participants and candidates would afford greater power to assess this. The website content was written above the sixth grade reading level given the subject matter terminology and complexity, which may make it difficult to comprehend [
Focus group moderator guide.
Website materials presented during the focus groups and associated questions about the website.
Screenshot of one of the many videos of a hand and arm transplant surgeon.
Screenshot of an upper extremity vascularized composite allotransplantation candidate explaining why he decided to pursue a hand and arm transplant.
Screenshot illustrating various Did you know? questions used to engage website users. Different questions appear in different website sections.
Screenshot showcasing the experiences of upper extremity vascularized composite allotransplantation recipients by presenting quotes from different recipients in various sections throughout the website.
Screenshot illustrating the topic areas featured under the risks of hand and arm transplantation section.
Health care professionals and an upper extremity vascularized composite allotransplantation recipient discussing the qualities of a good candidate for hand and arm transplantation.
After-Scenario Questionnaire
International Patient Decision Aid Standards version 4.0
Johns Hopkins University
Net Promoter Score
Northwestern University
System Usability Scale
upper extremity
vascularized composite allotransplantation
Walter Reed National Military Medical Center
This study was supported by the Department of Defense through the Congressionally Directed Medical Research Program—Reconstructive Transplant Research Program–initiating principal investigator: EJG (grant W81XWH-19-2-0033); partnering principal investigators: ML (grant W81XWH-19-2-0034), GB (grant W81XWH-19-2-0035), and ST (grant W81XWH1920036). The opinions and assertions expressed herein are those of the authors and do not necessarily reflect the official policy or position of the Uniformed Services University of the Department of Defense, the Henry M. Jackson Foundation for the Advancement of Military Medicine, or the Department of Defense of the US Government.
The data sets generated and analyzed during this study are not publicly available because of the lack of available funding but are available from the corresponding author upon reasonable request.
EJG conceived and designed the study and participated in performing the research, analyzing and interpreting the data, and writing the manuscript. KV and JGS participated in performing the research, analyzing and interpreting the data, and writing the manuscript. BK, MD, MN, ML, and WA participated in data collection and analyzing and interpreting the data. TR and SF participated in data collection. GB, ST, GD, and CC provided access to their patient populations for study recruitment. All authors reviewed and approved the final manuscript.
None declared.