Original Paper
Abstract
Background: The prevalence, clinical burden, and health care costs (>US $100 billion) associated with cerebrovascular disease (CVD) will increase significantly as the US population grows and ages over the next 25 years. Existing 2D imaging modalities have inherent limitations in visualizing complex CVD, which may be mitigated with the use of patient-specific 3D advanced visualization (AV) technologies. There remain gaps in knowledge, however, regarding how and with what impact these technologies are being used in CVD.
Objective: The aim of this study was to characterize the clinical attributes and reported utility associated with the use of 3D AV modeling in CVDs, specifically intracerebral arteriovenous malformations.
Methods: This pilot study employs a combination of retrospective analysis and prospective surveys to describe the utilization and utility of patient-specific AV models at a single high-volume certified comprehensive stroke center.
Results: From July 2017 to February 2023, 25 AV models were created for 4 different clinicians. The average patient age was 37.4 years; 44% (11/25) of the patients were African Americans, 52% (13/25) were on public insurance, and 56% (14/25) were associated with a neurovascular procedure. In this study, 18 clinicians with diverse experience responded to AV model surveys, with a 92.2% (166/180) completion rate. There was an average reported utility of 8.0 on a 0-10 scale, with higher scores reflecting increased utility. Compared to 2D viewing, AV models allowed staff to appreciate novel abnormal anatomy, and therefore, they would have changed their therapeutic approach in 45% (23/51) of the cases.
Conclusions: AV models were used in complex CVDs associated with young, publicly insured individuals requiring resource-intensive interventions. There was strong and diverse clinician engagement with overall report of substantial utility of AV models. Staff clinicians frequently reported novel anatomical and therapeutic insights based on AV models compared to traditional 2D viewing. This study establishes the infrastructure for future larger randomized studies that can be repeated for CVDs or other disease states and incorporate assessments of other AV modalities such as 3D printing and medical extended reality.
doi:10.2196/51939
Keywords
Introduction
Background
Cerebrovascular disease (CVD) is a common cause of stroke, aneurysms, and thromboembolic disease, resulting in significant neurologically based disability and death [,]. Although CVD can be treated with medications, many disease presentations require procedural interventions. It is estimated that by 2030, the annual costs for cerebrovascular accidents alone will exceed US $140 billion []. Improved visualization, greater repetition, and more frequent training experiences are factors known to improve outcomes in CVD treatments requiring complex procedures []. Treatment of patients with difficult or otherwise hostile anatomies carries more risk when operators have less general experience navigating unusual or atypical structures []. The lack of anatomical and procedural familiarity can lead to lengthier procedures, longer anesthetic duration, increased radiation exposure, and poorer outcomes [,]. Access to personalized 3D models designed for patient-specific planning can provide higher fidelity visualization and simulation than traditional 2D viewing, thereby reducing the prevalence, associated costs, and morbidity from complex CVD procedures [].
The rapid adoption of advanced visualization (AV) tools such as 3D rendering, 3D printing, and medical extended reality has created novel opportunities for medical visualization and simulation. Patient-specific 3D digital and physical anatomical models based on clinical radiographic data can now be rendered and fabricated for patient education, clinical training, and procedural planning []. This approach holds great promise as a tool in visualizing complex neurovascular anatomy and simulating complex surgical and endovascular procedures []. The increasing adoption of advanced modeling has prompted organizations such as the Radiological Society of North America to issue guidelines for implementing medical 3D printing as training models for highly complex clinical scenarios []. The Radiological Society of North America guidelines discuss the use of patient-specific 3D models for a variety of specialties, including congenital heart disease, craniomaxillofacial, breast, and musculoskeletal pathologies. There, however, remains limited guidance on the optimal use of 3D modeling in CVD.
CVD interventions are an ideal application of AV modalities due to the idiosyncratic and patient-specific 3D nature of intracerebral vascular pathology. Current evidence suggests that AV technologies such as 3D rendering, medical extended reality, and 3D printing are an appropriate platform for studying and rehearsing plans for treating complex CVD []. Several studies have reported the growing utilization of AV modeling, visualization, and procedural training in CVD []. The current state of research, however, remains limited and warrants more thorough investigation [,]. To address this gap, the field would benefit from the development of a scalable modeling and assessment platform, which could describe and evaluate the use of patient-specific AV anatomical modeling. Such an infrastructure could address the clear, critical, and well-identified need to characterize the use and utility of AV models in CVD.
Objectives
This pilot study was designed and executed to describe the utilization and utility associated with the use of patient-specific anatomical models in the management of intracerebral arteriovenous malformations (iAVMs). This exploratory investigation involves the implementation of a scalable digital fabrication infrastructure to produce patient-specific anatomical AV models, characterize their use, and quantify their potential impact in the management of iAVMs. The aims of this study include describing the clinical, anatomical, and demographic features associated with the use of patient-specific AV models in the management of iAVMs. Additionally, this study aims to quantify the utility of patient-specific AV anatomical models when compared to traditional 2D viewing across different clinical experience levels. We hypothesize there is a negative correlation between experience and reported utility of AV patient-specific anatomical models in CVD. We specifically predict that when compared to standard 2D viewing, clinicians with less experience will report higher utility of AV patient-specific anatomical models than more experienced staff clinicians.
Methods
Digital Fabrication Process
Clinicians request AV modeling based on routine clinical neuroimaging imaging, for example, computed tomography, magnetic resonance, or rotational angiography via an electronic medical record order set created to request patient-specific 3D AV models. Anatomical models are based on the digital imaging and communications in medicine (DICOM) datasets used in clinical imaging. DICOM data are accessed by a staff biomedical engineer via a compliant and secure enterprise picture archiving and communications system. The biomedical engineer segments these DICOM datasets by using computer-aided design software to produce 3D mesh files, specifically standard tessellated language files. Segmentation is the process by which regions of interest are identified through automatic methods (eg, thresholding, edge detection, region growing) and are then refined by manual selection and separation of anatomical structures. This step is completed by the biomedical engineer in conjunction with a radiologist, neurologist, or neurosurgeon to ensure an accurate representation of the region of interest. Once created, these 3D models are rendered and manipulated digitally via an internally developed web-based viewer. These digital models can be further postprocessed for 3D printing to create physical models (
Figure 1).
Epidemiological Data Collection
A CVD AV registry is used to collect data on clinical and utilization metrics associated with AV model use via a compliant research data capture application. Data are collected from the electronic medical record and include patient age at the time of modeling, sex, race, insurance status, imaging modality, comorbidities (hypertension, diabetes, epilepsy, tobacco use), size/scale/location of iAVM, associated current procedural terminology (CPT) codes, presence and type of any associated and relevant procedure within 9 months of model request, the presence and duration of any associated intensive care unit stay, as well as a Spetzler-Martin Grade (SMG). SMG is a clinical tool to assess the procedural risk of iAVMs that incorporates the size, location, and type of venous drainage.
Clinician and Model Survey Data Collection
Twenty-four clinicians were invited to participate in this study from the organization’s neuroscience institute via clinical and educational conferences, 18 of whom responded. Each clinician assessed a set of 10 sequential iAVM cases, which had previously been requested for modeling. Each participant received a secure email, which included a link to the model utility survey as well as the following information to review: (1) a traditional 2D viewer with the axial, sagittal, and coronal source images available in serial via an embedded digital multimedia container format file, specifically an mp4; (2) information to access and view the source imaging via the standard picture archiving and communications system viewing approach; (3) the associated anatomical 3D AV model accessible via an internally developed web-based viewer; and (4) the radiographic report of the imaging from which the model was created.
Drawing from similar instruments used in the literature, a survey is used to assess (1) demographics and level of experience, (2) clinician-reported complexity of each case on 1-10 Likert Scale, and (3) utility of the patient-specific AV models using a 1-10 Likert scale, with higher scores representing increased complexity and utility (-) [,-]. The survey includes questions regarding the potential diagnostic and therapeutic impact of 3D AV models compared to standard 2D representations. The research data capture application was used to digitally distribute and collect survey data in an automated and compliant way.
Analysis Plan
Descriptive statistics (averages, range, frequency) were reported for the epidemiological and utilization data for the AV models as well as the demographic and experience data of the survey respondents. Model survey responses were treated as ordinal. Each respondent’s complexity and utility response was averaged to mitigate the limitations associated with the statistical analysis of the dependent and repeated measurements. The independent variable is experience level, and 2 approaches were used to classify experience groups. One approach categorizes experience by training level: (1) trainee defined as medical student, resident, or fellow; (2) staff as defined by those who have completed their terminal clinical training; and (3) advanced practice provider (APP). An alternative approach categorizes experience based on the reported number of iAVM evaluations: low (0-10), medium (11-100), and high (>101). The dependent variables are reported complexity and utility of the AV model compared to traditional 2D viewing. The nonparametric Kruskal-Wallis H test was used to assess the null hypothesis that there is no difference in reported complexity or utility of an AV model among clinicians in these 3 groups. A significance level or α of .05 was used for statistical testing. A Dunn multiple comparison test was used for post hoc analysis to compare specific experience category combinations. A significance level or α of .05 was used for statistical testing.
Ethics Approval
This study was reviewed and approved by the Ochsner institutional review board (approval 2019.089) and was determined to be exempt and granted a waiver of informed consent, as it posed minimal risk to participants. All data collected for this study were fully deidentified prior to analysis to ensure participant privacy and confidentiality. Study models and data were deidentified and anonymized. Secure storage protocols were implemented in compliance with institutional and federal data protection standards. Access to the data was restricted to authorized study personnel directly involved in data collection and analysis. No compensation was provided to study participants. All procedures were conducted in accordance with appropriate ethical guidelines and applicable regulations.
Results
The aim of this study was to deploy a digital fabrication process that can (1) describe the clinical attributes associated with the use of AV models and (2) assess the utility of AV models across clinical experience levels when compared to standard 2D viewing.
Epidemiology and Utilization
AV models for 26 patients with iAVMs were requested from July 2017 to February 2023. Of the 26 models requested, 25 AV models were created. One request was not modeled because the primary source image was ordered by the clinician but never completed by the patient. Of note, 6 out of the 25 cases were also 3D printed to create physical patient-specific models (). Of the AV models, 60% (15/25) were based on rotational angiography, 32% (8/25) on computed tomography angiography, and 8% (2/25) on magnetic resonance angiography. Patient age ranged from 13 to 71 years (mean age 37.4 years, SD 16.4 years) with various comorbidities and diverse demographics (
Table 1).
| Values, n (%) | ||||
| Smoking (n=25) | ||||
| Nonsmoker | 4 (16) | |||
| Smoker | 21 (84) | |||
| Comorbidities (n=10) | ||||
| Hypertension | 8 (80) | |||
| Diabetes mellitus | 2 (20) | |||
| Hyperlipidemia | 0 (0) | |||
| Seizure/epilepsy | 0 (0) | |||
| Other | 0 (0) | |||
| Total comorbidities (n=21) | ||||
| 1 | 8 (38) | |||
| 2 | 4 (19) | |||
| 3 | 6 (29) | |||
| 4 | 1 (5) | |||
| 5 | 2 (10) | |||
| Other epidemiology metrics (n=25) | ||||
| Gender | ||||
| Female | 8 (32) | |||
| Male | 17 (68) | |||
| Race | ||||
| Caucasian | 14 (56) | |||
| African American | 11 (44) | |||
With regards to pathophysiology, iAVM diameter ranged from 4.5 mm to 62 mm, with an average diameter of 29.2 mm. Intracerebral hemorrhage was associated with 28% (7/25) of the cases. SMG scores were obtained and ranged from 1 to 5 with a median score of 2 (
Figure 2).
The most frequent iAVM location was the frontal lobe (9/25, 36%), and the most frequent venous draining was the superior sagittal sinus (9/25, 36%) (Table 2). Clinically, 56% (14/25) of the models were associated with some form of open or endovascular intervention within 9 months of the AV model request.
With regard to utilization, 9 unique CPT codes were found to be associated with these models. The most frequent CPT code associated with AV models was for selective unilateral catheter access of the common carotid artery (36223) and was billed in 84% (21/25) of the cases. Of the patients who had models created, 64% (16/25) had public insurance, with most patients utilizing Medicaid (14/25, 56%). Intensive care unit stays were associated with 56% (14/25) of the cases with a mean length of 6.9 days and a maximum length of 34 days. The 25 AV models were created for 4 unique clinicians at the following frequencies: interventional neuroradiologist (1/25, 4%), pediatric neurosurgeon (2/25, 8%), interventional neurologist (5/25, 20%), and a vascular neurosurgeon (17/25, 68%).
| Location/category | Values, n (%) | ||
| Intracerebral arteriovenous malformation locations | |||
| Frontal lobe | 9 (36) | ||
| Frontoparietal lobe | 5 (20) | ||
| Occipital lobe | 3 (12) | ||
| Other | 8 (32) | ||
| Venous drainage category | |||
| Superior sagittal sinus | 9 (36) | ||
| Cortical vein | 5 (20) | ||
| Transverse sinus | 3 (12) | ||
| Sigmoid sinus | 2 (8) | ||
| Other | 6 (24) | ||
Descriptive Statistics for Clinicians
Of the 24 clinicians who completed the initial recruitment survey between November 2022 to January 2023, 18 completed at least one model assessment survey. Six clinicians were lost to follow-up, that is, 1 medical student and 5 APPs. The descriptive statistics of the demographics and experience were calculated for the 18 clinicians who responded (Table 3). Clinicians (mean age 36.5 years; median age 34.5 years, range 26-56 years) identified their specialties as neurosurgery (1/18, 6%), neurointerventional radiology (3/18, 17%), and neurology (12/18, 67%). Two respondents (11%) were medical students and did not identify with a specialty. The other respondents identified as residents (4/18, 22%), APPs (5/18, 28%), fellow (1/18, 6%), junior attendings with <5 years of experience from terminal training (3/18, 17%), and senior attendings with >5 years of experience after terminal training (3/18, 17%). Self-reported iAVM evaluations was reported as 0 (2/18, 11%), 1-10 (4/18, 22%), 11-50 (6/18, 33%), 51-100 (3/18, 17%), 101-200 (2/18, 11%), and >201 (1/18, 6%).
| Values, n (%) | ||
| Specialties/level of training | ||
| Neurosurgery | 1 (6) | |
| Neurointerventional radiology | 3 (17) | |
| Neurology | 12 (67) | |
| Medical student | 2 (11) | |
| Level of training | ||
| Trainees | 5 (28) | |
| Staff | 6 (34) | |
| Advanced practice provider | 5 (28) | |
| Number of arteriovenous malformation evaluations | ||
| Low (<10) | 6 (34) | |
| Medium (51-100) | 9 (50) | |
| High (>101) | 3 (17) | |
| Clinician gender distribution | ||
| Male | 10 (56) | |
| Female | 8 (44) | |
Descriptive Statistics of Model Surveys
Each of the 18 clinicians were provided 10 model surveys for a total of 180 possible responses. Approximately 95.6% (172/180) of the surveys were initiated, and 92.2% (166/180) were completed. The possible complexity scores ranged from 1 to 10, with 10 being the most complex. The median of the average complexity reported was 6.95 with a minimum average score of 3.5 and a maximum average score of 8.1 across all respondents. Descriptive statistics, including median and IQR, were calculated for utility, complexity, and improvement attributed to access to AV models for each expertise level (
Figure 3) based on training level and the number of iAVMs evaluated (
Figure 4). Clinicians responded affirmatively in 99.4% (171/172) of the cases that they would like to share AV models with their patients.
There was a total of 6 physicians who had completed their terminal training with a cumulative survey response rate of 85% (51/60). When compared to standard 2D models, these 6 staff clinicians reported that AV models provided some improvement in visualization in 49% (25/51) of the cases and substantial improvement in 41% (21/51) of the cases. Staff level physicians reported that in 49% (25/51) of the cases, they were able to appreciate normal anatomy in the AV model that they could not in standard 2D imaging. Similarly, staff reported that in 45% (23/51) of the cases, they were able to appreciate abnormal anatomy in the AV model that they could not in standard 2D imaging. Staff answered affirmatively in 14% (7/51) of the cases that the AV model would change their diagnosis while reporting affirmatively in 45% (23/51) of the cases that the AV model would change their therapeutic approach (Table 4).
| Values, n (%) | ||
| Change in diagnosis | ||
| No | 39 (76) | |
| Yes | 7 (14) | |
| No response | 5 (10) | |
| Change in therapeutic procedures | ||
| Yes | 23 (45) | |
| No | 21 (41) | |
| No response | 7 (14) | |
| Change in appreciation of normal anatomy in 3D | ||
| Yes | 25 (49) | |
| No | 20 (39) | |
| No response | 6 (12) | |
| Change in appreciation of pathologic defects in 3D | ||
| Yes | 23 (45) | |
| No | 22 (43) | |
| No response | 6 (12) | |
Inferential Analytics of Model Surveys
When categorized by training level, the 3 groups were distributed between APPs (5/18, 28%), trainees (7/18, 39%), and staff (6/18, 33%). The Kruskal-Wallis H nonparametric test was deployed with nonsignificant P values >.05 for the survey questions relevant to model complexity (P=.72) and utility (P=.48). Thus, we fail to reject the null hypothesis. Rather, we conclude that there is no statistically significant difference in reported complexity nor utility between clinician types. When categorized by self-reported experience, the 3 groups were distributed between low (6/18, 33%), intermediate (9/18, 50%), and high (3/18, 17%). The Kruskal-Wallis H nonparametric test was deployed with P=.12 for the survey questions relevant to model complexity. Thus, we fail to reject the null hypothesis for complexity and conclude that there is no statistically significant difference in reported complexity between experience levels. The P value for utility, however, did achieve significance with P=.03. Thus, for utility, we can reject the null hypothesis and conclude that there is a statistically significant difference in the reported utility metric between self-reported experience groups. Post hoc analysis with Dunn multiple comparison test demonstrated that the comparison between the low and high group showed a significant difference in the utility metric. There was not a significant difference in the utility metric between the comparisons of the other groupings: high-intermediate or intermediate-low (
Figure 5).
Discussion
Principal Findings
This study demonstrates the feasibility of creating and deploying a digital fabrication and assessment infrastructure for AV models in complex presentations of CVDs, such as iAVMs. The scalable and secure request, creation, distribution, and evaluation of AV models was confirmed by the implementation of 25 AV models for iAVMs from July 2017 to February 2023 at a single high-volume certified stroke center. This retrospective analysis confirmed the utilization of AV models in CVDs, specifically iAVMs. Multiple clinicians requested AV models that were created from a variety of imaging modalities, most commonly from rotational angiography. Of note, requests for AV models persisted despite the decrease in elective procedures during the COVID-19 pandemic and a nationwide iodinated contrast shortage in 2022, which severely impacted imaging and procedure volumes and is indicative of the utility clinicians derive from AV models.
This analysis demonstrates associations between AV model use in CVD and a diverse set of clinical attributes and utilization metrics. Consistent with the complex and diverse population in Louisiana and the surrounding Gulf Coast region, a substantial number of models were created for individuals younger than 40 years, African Americans, and patients on Medicaid. Many of these patients were not only young but otherwise healthy with few if any other comorbidities, besides tobacco use. The feasibility to collect health care finance and utilization metrics such as CPT codes and intensive care unit lengths of stay associated with the request for AV models in complex CVD presentations was confirmed. The analysis revealed the association of AV models with resource-intensive health resources such as intensive care unit stays, rotational angiography, and neurovascular procedures.
A survey instrument grounded in the literature was successfully created and implemented to assess the impact of AV models in complex CVD at scale. There was strong engagement across a variety of clinical experience levels, particularly APPs, neurological residents, and neurovascular staff, with an overall 92.2% (166/180) survey completion rate. On a set of iAVMs of varying complexity, a diverse array of clinicians reported substantial utility of AV. Staff-level clinicians reported that AV models would have rarely changed their diagnosis compared to traditional 2D viewing. Staff, however, reported that in nearly half of cases, the patient-specific AV models allowed them to appreciate normal and abnormal anatomy that they could not with traditional 2D imaging. Moreover, in 45% (23/51) of the cases, access to the AV model would have changed their therapeutic approach compared to their assessment based on traditional 2D viewing. Staff physicians reported that AV models were much more likely to change their therapeutic approach than their diagnosis. There was no significant difference in reported complexity or utility when clinicians were categorized by training: APP, trainee, staff. There was, however, a statistically significant difference (P=.03) in reported utility when clinicians were categorized by self-reported experience. Interestingly, the difference in reported utility was most pronounced between the high and low experience group, with the most experienced group reporting more utility than those with less experience. This was contrary to our hypothesis and suggests that the ability to understand the idiosyncratic and complex pathologies and extract relevant information from AV models may require a certain level of pre-existing knowledge and clinical expertise. Such a prerequisite appreciation of clinical anatomy may help clinicians derive diagnostic and therapeutic insights from AV models. Alternatively, this observation may be attributable to the psychological phenomena known as the Dunning-Kruger effect in which people with low ability, expertise, or experience regarding a certain type of task or area of knowledge tend to overestimate their ability or knowledge [].
Strengths and Limitations
This study has multiple strengths attributable to a multidisciplinary approach that focused on clinical operational integration as a foundation for clinical research. Evaluation of an AV intervention required coordinating a cross-functional team consisting of biomedical engineers, research staff, clinical administrators, and clinicians. This study was the impetus to expand and ruggedize a scalable digital fabrication and assessment process for AV models. The survey response rates indicate not only enthusiasm for the field of inquiry but the willingness of these busy and increasingly overextended clinicians to engage in clinical research on 3D AV [,]. This points to the importance of continuing to develop and validate AV technologies not just for CVDs but for other complex disease states, including but not limited to spine deformity, genitourinary pathologies, orthopedic trauma, craniomaxillofacial abnormalities, and congenital heart disease.
This study has several limitations. This was a nonrandomized study, which could only establish associations and not causative impacts of AV modeling. The ability to extrapolate these findings is also limited by its small sample size and focus on a single center. Clinician respondents were basing feedback on hypothetical use cases and not the actual implementation of the 10 models in diagnosis or treatment planning. Moreover, the clinicians were all colleagues who may have influenced each other’s responses, which may have introduced a diffusion of treatment effect as described by Urban and van Eeden-Moorefield []. Finally, the results of the statistical testing may be due to the small sample size rather than the absence or presence of differential utility among varying experience levels.
Challenges and Future Directions
The creation of AV models remains resource-intensive in terms of expertise, time, and infrastructure. Complex disease states where AV models are likely to be the most impactful are often low in prevalence requiring longer periods of study and collaboration between multiple sites, engendering a gap in the literature. These barriers have limited the number of conclusive studies validating the clinical and utilization impact of 3D AV models. Approaches to increase the efficiency of AV model creation, for example, machine-learning enhanced segmentation, may mitigate these obstacles to validation and warrant further research and development.
This study expands and utilizes a scalable digital fabrication infrastructure for future larger multisite randomized studies to assess the efficacy of AV modeling. Such future trials could examine the utility of AV models in clinical care delivery by incorporating technical resource requirements such as segmentation time, utilization measures such as operative time and radiation exposure, as well as clinical outcome metrics such as functional status and hospital stay. This study establishes the feasibility of collecting medical billing and utilization metrics associated with 3D AV models that can be used in future cost-effective analyses. This infrastructure could also be applied to understand the use and impact of AV models in other disease states that would benefit from AV techniques. Future studies could incorporate quantitative and qualitative assessments of other AV modalities such as 3D printing or extended reality to investigate the different applications and utility of digital versus physical models in visualization and simulation. Finally, this infrastructure could be utilized to study the impact AV models have on health literacy and patient experience. The use of AV is quickly being adopted in clinical training and care delivery but requires further investigation and validation.
Conclusion
This pilot feasibility study establishes the infrastructure to create and assess patient-specific AV models in a digital and scalable way. This study confirms the use of AV models in complex, resource-intensive disease states such as iAVMs. This study demonstrates enthusiasm for the use of AV across different specialties and experience levels involved in the management of CVD. A heterogenous cohort of clinicians reported substantial utility of AV modeling when compared to standard 2D imaging. Counter to our hypothesis, those clinicians with the most experience found AV models the most useful. Finally, staff level physicians reported at high rates that AV models modified therapeutic approaches and provided anatomical insights not appreciated in standard 2D imaging. Almost universally, clinicians across different specialties and experience levels affirmed that that their patients would benefit from the ability to view their pathology-specific 3D AV models. Collectively, these findings support the idea that compared to traditional 2D viewing, patient-specific AV models are useful to a diverse array of clinicians involved in the treatment of complex disease states, such as iAVMs.
Acknowledgments
We thank our colleagues at Ochsner Health, specifically the academics, philanthropy, and neurosciences departments who supported the development and implementation of this work. The authors would like to thank the faculty and staff of Cedars Sinai Master’s Health Delivery Sciences as well as the 2023 cohort for their guidance, support, and mentorship. Research reported in this publication was supported by the Terence D D’Souza MD Fellowship in Neuroscience and the Ochsner Louisiana State University Collaborative Intramural Research program. The funders had no role in the study design, data collection, analysis, or interpretation of results. Generative artificial intelligence, specifically ChatGPT 4o V 1.2024.346, was used to assist in literature review and improve manuscript content.
Data Availability
The datasets generated during and analyzed during this study are available from the corresponding author on reasonable request.
Authors' Contributions
KS conceptualized, designed, and supported the implementation of this study as well as led the writing of this manuscript. VB supported the data collection, cleaning, and overall implementation of this study. AR supported the biostatistical analysis of the data. CC developed the web-based viewer, performed the segmentation, and supported study implementation. KH designed and deployed the digital request and assessment infrastructure. KS, VB, AR, CC, and KH wrote, edited, and approved the final manuscript.
Conflicts of Interest
KS has past but no current roles as an advisor to digital health companies, none of which have any financial or intellectual property rights to this research. He also engages in medical-legal consulting. During the period when this research was conducted, VB was affiliated with Ochsner Health. VB currently works in the digital health industry. VB has no financial, consulting, patent, or intellectual property interests related to this research, and declares no conflicts of interest that could have influenced this work. AR, CC, and KH have no disclosures to report.
Multimedia Appendix 1
Survey template: clinician, on-screen model survey, and survey email template.
DOCX File , 18 KBMultimedia Appendix 3
Representative web-based 3D viewer included in surveys.
ZIP File (Zip Archive), 9237 KBMultimedia Appendix 4
Photograph of 3D printing of an intracerebral arteriovenous malformation example.
PDF File (Adobe PDF File), 1722 KBReferences
- Portegies M, Koudstaal PJ, Ikram MA. Cerebrovascular disease. Handb Clin Neurol. 2016;138:239-261. [CrossRef] [Medline]
- Ovbiagele B, Goldstein LB, Higashida RT, Howard VJ, Johnston SC, Khavjou OA, et al. Forecasting the future of stroke in the United States. Stroke. Aug 2013;44(8):2361-2375. [CrossRef]
- Colby S, Ortman JM. Projections of the size and composition of the US population: 2014 to 2060. US Census Bureau. 2015. URL: https://www.census.gov/content/dam/Census/library/publications/2015/demo/p25-1143.pdf [accessed 2023-04-16]
- Kim BM, Baek J, Heo JH, Kim DJ, Nam HS, Kim YD. Effect of cumulative case volume on procedural and clinical outcomes in endovascular thrombectomy. Stroke. May 2019;50(5):1178-1183. [CrossRef]
- Perri JL, Nolan BW, Goodney PP, DeMartino RR, Brooke BS, Arya S, et al. Factors affecting operative time and outcome of carotid endarterectomy in the Vascular Quality Initiative. J Vasc Surg. Oct 2017;66(4):1100-1108. [FREE Full text] [CrossRef] [Medline]
- Ghosn Y, Khdhir M, Jabbour Y, Dushfunian D, Kobeissi I, Abbas N, et al. Factors affecting radiation dose, radiation exposure time and procedural time in arterial embolization for active hemorrhage. Emerg Radiol. Oct 2024;31(5):641-652. [CrossRef] [Medline]
- Branson T, Shapiro L, Venter RG. Observation of patients' 3D printed anatomical features and 3D visualization technologies improve spatial awareness for surgical planning and in-theatre performance. Adv Exp Med Biol. 2021;1334:23-37. [CrossRef] [Medline]
- Popescu D, Marinescu R, Laptoiu D, Deac GC, Cotet CE. DICOM 3D viewers, virtual reality or 3D printing - a pilot usability study for assessing the preference of orthopedic surgeons. Proc Inst Mech Eng H. Sep 2021;235(9):1014-1024. [CrossRef] [Medline]
- Liang F, Yang Y, Luo L, Liao B, Zhang G, Ou S, et al. Endovascular treatment of complex middle cerebral artery aneurysms using TuBridge flow diverters. Interv Neuroradiol. Oct 2020;26(5):539-546. [FREE Full text] [CrossRef] [Medline]
- Chepelev L, Wake N, Ryan J, Althobaity W, Gupta A, Arribas E, et al. RSNA Special Interest Group for 3D Printing. Radiological Society of North America (RSNA) 3D printing Special Interest Group (SIG): guidelines for medical 3D printing and appropriateness for clinical scenarios. 3D Print Med. Nov 21, 2018;4(1):11. [FREE Full text] [CrossRef] [Medline]
- Chae MP, Lin F, Spychal RT, Hunter-Smith DJ, Rozen WM. 3D-printed haptic "reverse" models for preoperative planning in soft tissue reconstruction: a case report. Microsurgery. Feb 2015;35(2):148-153. [CrossRef] [Medline]
- Sun Z. A systematic review of 3-D printing in cardiovascular and cerebrovascular diseases. Anatol J Cardiol. 2017:423-435. [CrossRef]
- McGuire LS, Fuentes A, Alaraj A. Three-dimensional modeling in training, simulation, and surgical planning in open vascular and endovascular neurosurgery: a systematic review of the literature. World Neurosurg. Oct 2021;154:53-63. [CrossRef] [Medline]
- Rybicki FJ. 3D Printing in Medicine: an introductory message from the Editor-in-Chief. 3D Print Med. 2015;1(1):1. [FREE Full text] [CrossRef] [Medline]
- Ryan J, Plasencia J, Richardson R, Velez D, Nigro JJ, Pophal S, et al. 3D printing for congenital heart disease: a single site's initial three-yearexperience. 3D Print Med. Nov 08, 2018;4(1):10. [FREE Full text] [CrossRef] [Medline]
- Sebök M, Dufour J-P, Cenzato M, Kaku Y, Tanaka M, Tsukahara T, et al. When is diagnostic subtraction angiography indicated before clipping of unruptured and ruptured intracranial aneurysms? an international survey of current practice. Acta Neurochir Suppl. 2021;132:9-17. [FREE Full text] [CrossRef] [Medline]
- Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc Psychol. Dec 1999;77(6):1121-1134. [CrossRef] [Medline]
- West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. Jun 2018;283(6):516-529. [FREE Full text] [CrossRef] [Medline]
- Rodrigues H, Cobucci R, Oliveira A, Cabral JV, Medeiros L, Gurgel K, et al. Burnout syndrome among medical residents: A systematic review and meta-analysis. PLoS One. 2018;13(11):e0206840. [FREE Full text] [CrossRef] [Medline]
- Urban J, van Eeden-Moorefield BM. Designing and proposing your research project. In: Concise Guides to Conducting Behavioral, Health, and Social Science Research Series. Washington DC. American Psychological Association; 2018:119-127.
Abbreviations
| APP: advanced practice provider |
| AV: advanced visualization |
| CPT: current procedural terminology |
| CVD: cerebrovascular disease |
| DICOM: Digital Imaging and Communications in Medicine |
| iAVM: intracerebral arteriovenous malformation |
| SMG: Spetzler-Martin Grade |
Edited by A Mavragani; submitted 20.08.23; peer-reviewed by M Waleed, M Aksoy; comments to author 06.09.23; revised version received 13.01.25; accepted 22.01.25; published 21.02.25.
Copyright©Korak Sarkar, Vishal Bhimarasetty, Abdul Rahim, Colin Curtis, Kimberly Hughes. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.02.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.