This was a prospective observational study. We used mixed methods to capture clinician ratings and feedback after a 2-week pilot field implementation of the prototype RDHRS teleconsultation system. We included a test call strategy to evaluate call performance in conditions approximating real-world use if clinical use lagged. We followed STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines for reporting observational studies.

The Boston Medical Center/Boston University Medical Campus institutional review board approved this study and granted an exemption under Category 2 of the Human Research Protection Program policies and procedures. Although we used an informed consent process, the study qualified for a Waiver of Documentation of Informed Consent due to the minimal risk of harm to participants and the nature of the study procedures, which typically do not require written consent outside of a research context. Participant data was deidentified and assigned a unique study identifier, with the master code stored separately and accessible only to the study team. Participation was voluntary and unremunerated.

The study was conducted in Massachusetts during the first wave of COVID-19 response in 2020. We gave Massachusetts hospitals temporary access to disaster teleconsultations using a national volunteer pool of critical care experts. Based on participating hospital needs, disaster teleconsultation services were available on demand for 12 hours/day (5 PM-5 AM) from June 1 to June 15, 2020.

We recruited subject matter experts (SMEs) from the Society for Critical Care Medicine as RDHRS teleconsultants via email invitation to a list of 100 member volunteers interested in testing new technologies for remote care delivery provided by the Society. Eligible SMEs had a full medical license in good standing in any US state, territory, or district. Participation was part-time.

All Massachusetts hospitals were invited to participate as field test sites by email invitation sent by the Massachusetts Hospital Association and local health care coalitions on behalf of the RDHRS. Hospitals interested in accessing RDHRS teleconsultation services were introduced to the goals of the pilot in follow-up virtual meetings with the RDHRS team. There was no fee for service use. SME and hospital participation was voluntary.

After testing off-the-shelf video-conferencing platforms, we selected a Health Insurance Portability and Accountability Act (HIPAA)–compliant telehealth platform (Bluestream Health, Inc.) for disaster-relevant capabilities. This commercially available, web-based application had low bandwidth requirements, did not require software or hardware installation, and was device-agnostic and interoperable to support a distributed care delivery model. The platform was configured for relevant workflows (virtual interpreter and tele-emergency services) with remote high-throughput triage capability, virtual team functionality, and rapid scalability, and it was FirstNet-certified for priority network access. Key security features included end-to-end encryption for all data transmissions, secure communication channels to protect patient information during teleconsultations, robust authentication protocols to verify user identities, and restricted access to safeguard sensitive health data.

The primary outcome, acceptability, was measured as TUS≥6 of 7. The secondary outcome, feasibility, was measured via select TUQ items in interaction quality and reliability domains.

Call performance measures were recorded and downloaded from the platform, including the frequency of calls attempted, calls connected, video calls completed successfully, reasons for call failure, and wait times. A connection was successful when a field clinician completed a synchronous video call with the SME. Reasons for call failure included user cancellation (call canceled manually by the requester), orphaned call (call canceled by the server when the SME could not be contacted or the server lost contact with the requester before the SME answered), call routing error (error in call routing from the hospital to the MEOC), expert notification error, device problem, and connection problem.

Questionnaires were administered electronically immediately after the pilot period ended and stored using the REDCap (Research Electronic Data Capture; Vanderbilt University) platform hosted by Boston University [13]. All participants were given 2 weeks with 3 email reminders to complete questionnaires.

After survey completion, brief interviews (30 min) with willing participants were audio-recorded using Zoom Pro by 2 study investigators. Audio recordings were transcribed verbatim by a third-party transcription service.

As a pilot field test, this study was not powered to detect statistically significant outcomes. We determined that a minimum of 30 experts would be required to ensure 24/7 on-demand teleconsultation. Our staffing model assumed that each physician would volunteer for three 4-hour shifts during the 2-week pilot period. Based on preliminary data, we needed approximately 20 participants to estimate acceptability with adequate precision. Assuming each encounter involved 1 field clinician and 1 expert, we anticipated a minimum of 10 teleconsultation encounters during the pilot period.

Of 100 Society of Critical Care Medicine members who volunteered to test new technologies, 66 SMEs were eligible for recruitment based on board certification. Among these, 27/66 (41%) responded to an invitation to volunteer as an RDHRS teleconsultant. Ultimately, 10/66 (15%) eligible SMEs from 6 US states completed the onboarding process and participated in the study.

In total, 4 community hospitals and 3 tertiary care centers in Massachusetts expressed interest in participating in the pilot. Of these, 2 community hospitals and 2 tertiary care centers completed their administrative review and onboarding in time to participate in the pilot. A total of 17 board-certified internal and emergency physicians from these 4 hospitals participated as field clinicians.

Of these, 10 field clinicians and 10 SMEs completed post-pilot questionnaires (75% response rate). Table 1 summarizes participant demographics.

In total, 50 test calls and no clinical calls were logged. While only 3 of the 10 field clinician respondents (30%) reported success on their first attempt, most (70%) successfully completed at least 1 video call during the pilot period. Among the 50 test calls, only 22% (95% CI 10%‐34%) connected successfully (Table 2). The median time to connect to an expert teleconsultant was 1.6 (IQR 3.2) minutes.

Table 3 details the reasons for call failure (n=39) recorded by the Region 1 RDHRS platform. Half (49%) of the failed calls were due to a call routing error in Hospital 1, and 14 (36%) failed for >1 reason. Among 23 test calls routed correctly to the MEOC, 48% (95% CI 27%‐69%) were successful video call connections (Table 2).

Among field clinician respondents (n=10), 50% used smartphone devices, 40% used hospital desktop computers, and 10% used a laptop computer to access RDHRS teleconsultation services. All hospital desktop computer users failed to connect their first call. Lack of camera or microphone capability (30%) and lack of video (40%) or audio (20%) during the call were common reasons for call failure reported by field clinicians. Fewer reported difficulties logging in to the platform (20%), adding a patient (10%), or initiating a video call (20%).

The mean TUS was 5.6 (SD 1.3; Table 4). TUQ item scores were highest in usefulness (mean 6.0; SD 1.1) and ease of use (mean 6.0, SD 1.4) and lowest in reliability (mean 2.4, SD 1.4).

All participants felt comfortable using the RDHRS platform to access disaster teleconsultation services. Most (80%) field clinicians preferred to use personal or institutional smartphones or tablet devices. All SMEs were willing to participate in a national expert registry, including on short notice. All were willing to obtain out-of-state emergency licensure, and 8 in 10 were willing to serve on a volunteer, unpaid basis. Almost all (90%) SMEs were willing to provide teleconsultation services from a personal device.

In total, 6 field clinicians and 6 SMEs completed interviews. Table 5 summarizes key themes with representative quotes. Most physicians perceived the platform as simple and easy to learn, and those who had difficulty learning the system identified discomfort with technology as the primary cause. When video was limited, providers could still communicate via an audio call or real-time chat function. Most field clinicians agreed that simplifying the 2-step process of connecting to SMEs would improve system workflow. Almost all SMEs desired more platform training to support real-time technical assistance for field clinicians during the call.

Overall, the biggest barrier to successful connections was issues related to the lack of audiovisual capabilities on hospital computers, access to reliable and consistent mobile or wireless internet connections in clinical workspaces, and circumventing institutional firewall restrictions. Despite technical limitations, most expressed positive attitudes and future intent to use the platform. Recommendations for future use included clarifying the role and expectations of teleconsultants, streamlining the workflow for field clinicians to minimize risk of call failures, and improving the SME’s knowledge of the system to facilitate real-time communication with field clinicians.

This was a field test of the acceptability and feasibility of implementing a prototype RDHRS disaster teleconsultation system to deliver tele-critical care services to Massachusetts hospitals during the first wave of the COVID-19 pandemic. Clinicians found the prototype platform acceptable, but the workflow requires revision to reduce call failure and improve feasibility and reliability for future use with minimal-to-no training. We expanded access across hospital systems in real-world conditions and deployed SMEs from a national volunteer pool using emergency waivers and state emergency licensure processes, although this delayed implementation and impacted clinical use.

We were able to improve our initial prototype model in important ways [7]. We avoided software or hardware needs by deploying a web-based application. Using typical emergency department workflows and automating triage and expert notification processes that were previously done manually improved call center efficiency by reducing call wait times. However, these metrics are based on small samples under low volume conditions and did not involve patient care. Future evaluations should include broader system performance indicators, such as throughput, consult duration and quality, and user experience, ideally measured in real-world conditions.

Our workflow analysis identified potential sources of call failure at platform, user, and hospital levels that require technology and system solutions to improve call center performance and reliability. Initially, overall call connection success through the MEOC was low because the technology vendor accidentally introduced a call routing error while registering one hospital. Call failures due to expert notification errors, use of hospital computers lacking cameras and microphones, and institutional firewalls were less common. Though many participants could complete a call, the major sources of call failure—orphaned and canceled calls, and call routing errors—occurred early during steps where the consult was requested and routed to the MEOC. In the initial configuration (Figure 1A), the field clinician initiated the video call connection, which required them to remain at the location of the device used to initiate the call until the SME answered. Thus, call routing errors and delayed expert response produced orphaned and canceled calls. Field clinicians also reported confusion with the 2-step process, often mistaking patient registration with consult request submission.

We have since streamlined the workflow to eliminate these errors (Figure 1B). Field clinicians (or support staff) now submit consult requests in a single step and receive confirmation. The consult request is automatically routed to the MEOC, which alerts the SME to a pending consult. The SME then initiates the video call connection by sending a text link and email invitation to the field clinician’s chosen device. The video call connection is established when the field clinician accepts either invitation. This frees the field clinician to remain active in care and may reduce orphaned or canceled calls. These changes improved call connection success in a subsequent large-scale functional exercise conducted by the RDHRS, but these data are preliminary and not yet externally validated. Future third-party assessment is needed to validate system performance.

Each hospital was onboarded individually to institutional cybersecurity standards in this pilot field test. We found this process may hinder scalability and was vulnerable to error. Registering hospitals in advance could mitigate this but requires central coordination and administrative support. Another option is to use a central web portal with an event-specific code that can be distributed via existing emergency notification systems for just-in-time access. This feature could enable an RDHRS to control system activation and deactivation centrally, make services accessible immediately but restricted to affected facilities, and scale system use to demand.

Our findings support prior studies highlighting the importance of access to suitable devices [15,16]. While the platform was device-agnostic, hospital computers without cameras or microphones accounted for many failed calls. Every participant who used a desktop computer failed their first call, and device problems constituted a third of call failures. Using designated institutional mobile devices, tablets, or personal devices could improve call reliability. A 2021 systematic review identified lack of workflow integration, outdated technology infrastructure, and user acceptance among the most commonly cited barriers for telehealth services implemented globally during COVID [15]. Postpandemic institutional technology upgrades may benefit the future implementation of disaster teleconsultation systems [17]. We recommend hospital emergency operation plans include devices with audio-visual capabilities and telecommunication network redundancy to ensure seamless access to disaster teleconsultation services.

Our experience also highlights how technology, workforce, and operational system readiness are essential to harness digital health tools for disaster response. We accelerated prototype system development and tested implementation in March 2020 as the need to expand critical care access emerged. Working with an industry partner, we configured a prototype platform for clinical use within 2 weeks. Despite federal and state emergency laws and waivers [10,11], it took an additional 6 weeks to navigate out-of-state physician licensure, hospital credentialing, and liability coverage for teleconsultants. By the time pilot implementation began in June 2020, COVID-19 cases had declined in Massachusetts, so only simulated calls were performed.

This study mirrors prior research demonstrating how variation in licensing and credentialing procedures can delay service provision, despite emergency declarations [6,18,19]. The United States lacks a national medical license. Instead, state medical boards grant physician licenses to practice medicine in that state. Hospitals and healthcare organizations use credentialing procedures to verify qualifications before permitting physicians to provide care within their institutions. The Interstate Medical Licensure Compact is 1 legal pathway to expedite licensure [20]. However, some states are not members, and full licensure is still required in each state, so the financial and administrative burden for physician volunteers would be prohibitive. Model legislation, like the Emergency Volunteer Health Practitioners Act [21], is an alternate pathway to allow license reciprocity, credentialing waivers, and liability protections during declared emergencies, but only 20 states have enacted such laws. Without adoption of national solutions, RDHRS and similar response entities will need to rely on state emergency waivers and limit scope of practice to advisory only when providing access to disaster teleconsultation services across jurisdictions.

We and others have argued that disaster telehealth uptake requires proactive rather than reactive system development [18,22]. Disasters requiring specialty care are often no-notice events, so disaster teleconsultation systems need simple, reliable, field-tested platforms and preverified, rostered clinicians to support virtual deployment within hours rather than days to weeks. A range of specialists may need to be onboarded quickly because service needs are dictated by the medical consequences of the disaster. Therefore, these systems need to be flexible to support various use cases and easy to use regardless of discipline, training, or exposure. Emulating key features of the workflow and infrastructure of established emergency call centers and providing a clear scope of practice for disaster teleconsultants could encourage adoption.

Participants referenced US Poison Centers [23] as an effective model for delivering on-demand emergency consultations. Field clinicians felt comfortable consulting Poison Center services to access medical experts by telephone. However, once the expert could view the patient via video, providers were unclear whether the interactions constituted the broader practice of medicine (telemedicine) versus offering recommendations (teleconsultation) at a distance.

While teleconsultation is a type of telemedicine encounter, there are important distinctions in provider responsibilities, licensure, liability, and patient privacy [24]. Providers must adhere to standards of in-person practice, but responsibility and liability depend on whether the teleconsultant is directly involved in care or offering advice. Many states provide a “consultation exception” to licensure, but the scope and details of these exceptions vary by state [25]. Clinicians need to verify their malpractice coverage extends to teleconsultation and includes relevant jurisdictions. They should also include the limitations of remote assessment in any medical documentation.

While HIPAA provisions were temporarily relaxed during the COVID-19 public health emergency, using off-the-shelf platforms without encryption introduces the risk of privacy breaches. Disaster telehealth platforms should use robust technical and administrative safeguards aligned with best practices in health information security, including end-to-end data encryption, access control mechanisms, audit trails, and data anonymization techniques [26,27]. Strict adherence to HIPAA and privacy laws and careful attention to cyber security risks are essential to maintain patient and provider trust and legal compliance, as consultations may occur in non-secure environments [28].

Finally, clinicians in our study highlighted how administrative and legislative buy-in were necessary for real-world implementation, a finding supported by pediatric disaster groups attempting statewide implementation of specialty telehealth services [29]. Establishing trust and acceptance of disaster teleconsultation systems by the medical community and the public will require socialization and integration of the RDHRS program within the national strategic health response.

This study has limitations. First, the small, regionally restricted sample may limit generalizability. Still, we captured important perspectives from community and tertiary care providers in real-world conditions to guide future system refinements. Second, lack of opportunity to provide clinical care limited any exploration of how disaster teleconsultation services could impact patient outcomes, so more research is needed. Third, we focused on pilot implementation in Massachusetts, as this was the initial location of clinical need, so our analysis of barriers and facilitators may not apply to states or regions where laws or regulations differ. For example, states that allow license reciprocity or participate in regional compacts could streamline access to disaster teleconsultation services. This could benefit health professional shortage areas and rural communities where disasters are likely to exacerbate underlying disparities in healthcare access. To support future pilots in different state contexts, RDHRS and other regional disaster programs should establish processes to authorize disaster telehealth service delivery in coordination with local and state regulatory bodies and relevant government agencies. Last, hospital and clinician willingness to use regional disaster teleconsultation services for pandemics may differ from no-notice events where care delivery is time-sensitive and specialist supply is even more limited (eg, burn surgeons).

Implementing RDHRS disaster teleconsultation services to connect field clinicians to specialists across state lines and hospital systems was feasible in real-world conditions. We improved system efficiency using a simple, device-agnostic, web-based platform to create a virtual MEOC that automatically routes, triages, and notifies experts of requested consultations. While user attitudes were generally positive, further technology and system refinements are needed to align call performance, reliability, and usability to desired standards. Although emergency waivers enabled the RDHRS to mobilize and deploy a virtual volunteer workforce for short-term disaster response, liability concerns and variable hospital credentialing practices delayed implementation. We encourage government and emergency management agencies to advance policies that expand liability protections for volunteer teleconsultants and incentivize hospitals to use standard procedures to support timely, effective regional disaster telehealth solutions.