This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.
Inadequate assessment of the severity and urgency of medical problems is one of the factors contributing to unnecessary emergency department (ED) visits. The implementation of a software-based instrument for standardized initial assessment—Standardisierte medizinische Ersteinschätzung in Deutschland (SmED) (
This study aims to evaluate the overall perception of SmED by health care professionals using the software, to examine to what extent SmED influences the workload and work routines of health care professionals, and to determine which factors are associated with the use of SmED.
An early qualitative process evaluation on the basis of interviews was carried out alongside the implementation of SmED in 26 outpatient emergency care services within 11 federal states in Germany. Participants were 30 health care professionals who work with SmED either at the joint central contact points of the outpatient emergency care service and the EDs of hospitals (ie, the Joint Counter;
Health care professionals perceived that workload increased initially, due to additional time needed per patient. When using SmED more frequently and over a longer time period, its use became more routine and the time needed per call, per patient, decreased. SmED was perceived to support decision making regarding urgency for medical treatment, but not all types of patients were eligible. Technical problems, lack of integration with other software, and lack of practicability during peak times affected the implementation of SmED.
Initial experiences with SmED were positive, in general, but also highlighted organizational issues that need to be addressed to enhance sustainability.
German Clinical Trials Register DRKS00017014; https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00017014
In recent years, steadily increasing utilization of emergency departments (EDs) has aroused public and political attention, not only in Germany but in many nations [
One category of strategies are programs and policies that aim to reduce the number of unnecessary ED visits. Unnecessary ED visits are defined as using the ED for health issues that do not require immediate medical treatment. Those visits could be classified as
In Germany, the organization of emergency care service is complex [
Since the beginning of 2019, a computer-based software—Standardisierte medizinische Ersteinschätzung in Deutschland (SmED) (
Overview of the three different levels of emergency care in Germany.
This early process evaluation aimed to investigate the following: Reach, Effectiveness and Efficacy, Adoption, Implementation, and Maintenance (RE-AIM), according to the RE-AIM framework [
The overall aim of the DEMAND project is to improve medical care of patients who present with an urgent need for emergency treatment and/or medical advice on the basis of a more efficient use of emergency care resources. The implementation of a software-based instrument for standardized initial assessment (ie, SmED) aims to support health care professionals (eg, nurses, physician assistants, and paramedics) in emergency care and to steer patients with nonurgent medical needs toward the right point of care. The State Associations of Statutory Health Insurance Physicians have implemented SmED since March 2019 at the Joint Counter, as well as the phone number 116117, in 11 of the 16 federal states of Germany. Health care professionals within this setting were expected to change their service delivery by using the software and, therefore, influence individuals in the target population, which included patients who contact the Joint Counter or call 116117 regarding the need for medical treatment and/or advice.
The Institute for Applied Quality Improvement and Research in Health Care (aQua Institute) invited all State Associations of Statutory Health Insurance Physicians from each federal state of Germany to take part in this project. Out of 16 federal states, 11 agreed to take part. No specific reasons for the nonparticipation of the other five federal states were identified. The State Associations of Statutory Health Insurance Physicians informed the research team at the University Hospital Heidelberg about all health care professionals using SmED and were, therefore, potential participants. According to this number, information packages, including an invitation letter, an information leaflet, and an informed consent form for tape recording, were put together and sent to a contact person responsible for distributing. The information sheet included contact details of research team members who were available to participants to discuss the study or address additional concerns or questions. Participants who decided to take part in an interview were requested to contact the researchers directly. An informed consent form for tape recording was signed prior to the start of the interview by the participant and the interviewer. Different strategies, including email reminders, telephone calls, and the annual project coordination team meeting, were used to maximize response rate.
SmED is a computer-based software that requires an internet connection. The software can be used by health care professionals (eg, nurses, practice assistants, and paramedics) for initial assessment as a basis for demand management in outpatient emergency care services. The purpose of this software is to support health care professionals and to steer patients toward the right point of care based on their actual medical needs. SmED can be used in different outpatient emergency care services and applies a number of well-defined questions regarding different medical disorders and issues. SmED uses an algorithm based on the
The aQua Institute organized workshops prior to the implementation of SmED for all State Associations of Statutory Health Insurance Physicians from each participating federal state of Germany. A training concept for potential users and trainers was designed. Additionally, a data protection concept and an implementation plan for each project site was developed. Moreover, quality management and a support management program responsible for implementation sustainability were introduced.
Semistructured telephone interviews were conducted by the two first authors (AB and CR) with 30 health care professionals who used SmED between July and December 2019. The semistructured interview guideline (see
The matrix-based method of framework analysis according to Ritchie and Spencer [
The dataset generated and analyzed during this study will not be made publicly available due to European Data Protection Law but may be available by the corresponding author upon reasonable request.
At the beginning of the study, according to the project coordinator, 391 health care professionals used SmED. All 391 professionals were invited to take part in an interview and 30 decided to take part. Characteristics of the participants are described in
Characteristics of health care professionals.
Characteristic | Value (N=30) | |
|
|
|
|
Female | 23 (77) |
|
Male | 7 (23) |
Age (years), mean (SD) | 43.3 (10.1) | |
|
|
|
|
Baden-Wuerttemberg | 1 (3) |
|
Bavaria | 3 (10) |
|
Berlin | 3 (10) |
|
Bremen | 6 (20) |
|
Hesse | 4 (13) |
|
North-Rhine | 2 (7) |
|
Rhineland-Pfalz | 5 (17) |
|
Schleswig-Holstein | 2 (7) |
|
Thuringia | 1 (3) |
|
Westphalia-Lippe | 3 (10) |
|
|
|
|
State-qualified nurse | 3 (10) |
|
Practice assistant | 17 (57) |
|
Emergency paramedic | 7 (23) |
|
Controller or dispatcher | 1 (5) |
|
Other | 2 (7) |
Work experience (years), mean (SD) | 20.37 (9.62) | |
|
|
|
|
1 month | 1 (3) |
|
Between 1 and 2 months | 1 (3) |
|
Between 2 and 3 months | 4 (13) |
|
Between 3 and 4 months | 4 (13) |
|
Between 4 and 5 months | 1 (3) |
|
Between 5 and 6 months | 5 (17) |
|
Over 6 months | 14 (47) |
|
|
|
|
Initial telephone contact point | 18 (60) |
|
Joint Counter | 10 (33) |
|
Both settings | 2 (7) |
aSmED: Standardisierte medizinische Ersteinschätzung in Deutschland (
Regarding the target population of SmED, the health care professionals explained that SmED is not applicable to patients with hearing impairment (1/30, 3%); elderly patients (2/30, 7%); terminally ill patients (2/30, 7%); patients who are incapable of answering questions due to, for example, neurological diseases (3/30, 10%); or patients suffering from alcohol or drug overdose (1/30, 3%). SmED is also not feasible for patients where someone else (eg, husband or wife) is calling on their behalf (3/30, 10%), patients with psychiatric disorders (5/30, 17%), or patients with language barriers (14/30, 47%). Moreover, SmED is not applicable if qualified health care professionals (eg, nurses working at a nursing home) call on the behalf of a patient (8/30, 27%). Health care professionals working at the initial telephone contact point (ie, phone number 116117) also reported that they do not use SmED for patients who want to order a prescription or for those seeking simple advice (3/30, 10%). Health care professionals working at the Joint Counter explained that, furthermore, SmED is not feasible for patients who are regular visitors to a long-term therapy service (1/30, 3%) or who are in poor medical condition (6/30, 20%). One health care professional stated that patients with chronic diseases or nursing diagnoses (eg, having a urinary catheter), for example, are not included.
80-year-old Turkish person who speaks very little German does not understand the questions and then there is no one to translate, and then SmED, it serves no purpose.
This theme concerns the impact of SmED on steering patients toward the right point of care and the impact of SmED on workload and working methods. A vast majority of health care professionals (21/30, 70%) stated that SmED rates the urgency for medical treatment higher than they themselves would rate the urgency. For instance, if patients report high blood pressure (4/30, 13%), infection of the gastrointestinal system (2/30, 7%), respiratory problems (3/30, 10%), fever and chills (3/30, 10%), severe pain (3/30, 10%), or uncontrolled falls (1/30, 3%), SmED immediately rates them as emergencies. Almost two-thirds of all participants (18/30, 60%) reported that if SmED rates the urgency for medical treatment higher than they expected, they changed the decision based on their professional experience or after consultation with a physician. If the health care professionals changed the category as rated by SmED, they documented and justified it on the final summary. Only 5 participants out of 30 (17%) said that they had to accept how SmED rated patients even though they disagreed with the software.
Handling of patients who are rated as nonurgent varied, depending on the setting. Patients who call the phone number 116117 and are classified as nonurgent are advised to visit an outpatient emergency practice (4/30, 13%), are forwarded to telephone counseling with a physician (9/30, 30%), receive a home visit by the emergency care physician (5/30, 17%), or are given advice (eg, going to the primary care physician within the next few days or treating themselves at home) (6/30, 20%). At some initial telephone contact points, patients are given the opportunity to call back if their medical condition is getting worse (3/30, 10%). Patients who contact the Joint Counter are treated by a physician within the next few hours but have to wait (9/30, 30%) or are referred to the outpatient emergency care sector (1/30, 3%). Due to legal requirements, all patients who contact the Joint Counter have to be treated or seen by a physician; thus, at three Joint Counters, waiting lists based on SmED categories are implemented to organize patients.
This theme concerns whether health care professionals are willing to use SmED. When asked about expectations or disappointments, one-third of the participants said they expected SmED to be an aid for decision making, especially for unexperienced colleagues (11/30, 37%). However, health care professionals stated that their professional experience influenced decisions concerning urgency of medical treatment independently of the software. In addition, their intuition, professional knowledge, and experience influenced how or if they used the software. Health care professionals stated that after working in the field of emergency care service for many years they are able to make safe decisions and rate the urgency for medical treatment of patients without a support system (18/30, 60%).
Well, meanwhile, I personally would say that especially for younger coworkers, it is a chance to ask structured questions. And it does not matter from where the patient is calling, we ask the same questions and eventually it is always the same result...
Health care professionals stated that they assumed that SmED could function as a tool for quality assurance (eg, as a standard tool) (4/30, 13%). According to 7 out of 30 (23%) health care professionals, SmED provides support regarding decision making and helps to structure the assessment and, therefore, to correctly assess patients’ needs for medical treatment. Nevertheless, one-third (11/30, 37%) of the participants reported that the tool is imprecise and unstructured. The order of the questions sometimes does not match a patient’s symptoms. One advantage according to the health care professionals is that they do not forget questions and the system provides an opportunity for a structured assessment (8/30, 27%). In addition, questions asked after implementation have not been asked before (eg, whether the patient has been abroad lately) (1/30, 3%). Nevertheless, 4 health care professionals out of 30 (13%) mentioned that it is possible to skip questions if they are unnecessary or inappropriate, based on their professional judgment.
I think the main advantage is that the conversation is structured...And yes, that we identify people who dramatically report their symptoms even though they aren’t that urgent. Yes, I think that we identify them.
Although health care professionals expected SmED to be more patient oriented (8/30, 27%), some stated that it includes too many questions (3/30, 10%). Particular questions regarding medication or drugs are too complicated to answer for some patients. A small number of participants said SmED is too comprehensive (6/30, 20%). Moreover, almost one-third of interviewees (8/30, 27%) explained that they have the perception that a small group of patients are being rude and are annoyed due to busy lines, long waiting times at the Joint Counter, or being asked too many questions.
...the conversation definitely takes longer and, therefore, the lines are busy. Patients are sometimes annoyed if they have to wait 15 to 20 minutes instead of 10 minutes due to the additional time needed per call. Sometimes I have to ask comprehension questions like “What do you mean by that?” and that’s definitely a disadvantage.
In addition, if a patient is feeling really sick or if the situation is too challenging, answering all SmED questions puts an additional burden on the patient (2/30, 7%). Furthermore, if a patient is calling 116117 and the lines are busy, they may decide to hang up and call the rescue service.
I always think that if someone is really feeling sick, SmED might be a burden. I mean, I do not have feedback regarding that, no one told me that, but I have the feeling that SmED is too much for some patients.
However, when asked about SmED’s impact on patients, almost half of the participants (14/30, 47%) explained that they have the perception that patients, in general, feel like they are in good hands and that they are grateful for a more structured and comprehensive assessment. Thus, according to 6 professionals out of 30 (20%), using SmED increases patient safety.
...and a lot of patients like that we have to talk with them a little longer, they are like “It was very kind of you that you took that much time for me.”
The majority of health care professionals reported that one barrier was that using SmED increases their workload due to the additional time needed per call, per patient (21/30, 70%). However, through more frequent use of SmED, it became more routine and the additional time needed per call, per patient, decreased (9/30, 30%). A facilitator for implementation was motivation through colleagues, trainers, or leaders (3/30, 10%). If these persons are convinced by SmED, they were able to encourage other health care professionals. On the other hand, health care professionals who see only the negative effects of the new software can have a negative impact on successful implementation of SmED.
First of all, long-term employees have to be convinced because they are used to the old system. And yes, during work life, people are usually confronted with new things; however, Germans tend to only see the negative sides of new things at the beginning.
This theme concerns the consistency of the organization and adjustments made during and prior to the delivery of the intervention as well as implementation strategies. All interviewees (30/30, 100%) reported that they participated in a training session prior to implementation. The development and structure of SmED and its beginning in Switzerland were presented. During the training session, participants worked with exemplary patient cases. The success of the training sessions was monitored differently. However, nearly one-third of all health care professionals (8/30, 27%) stated that they used patient cases to learn how to use SmED. Those cases were discussed at the end of the training session. Out of 30 participants, 1 (3%) said that they used SmED once during their working hours with different patient cases to understand how the software works in real-life situations. Another participant (1/30, 3%) explained that they had to use SmED five times during their working hours and documented it. Out of 30 participants, 1 (3%) explained that they watched web-based videos and tested their knowledge afterward. A total of 5 out of 30 (17%) interviewees stated that they were able to ask experienced colleagues further questions at the start of the implementation of SmED to learn how to use the software correctly. The overall training was evaluated as completely positive by 20 out of 30 (67%) interviewees. Out of 30 participants, 1 (3%) brought up that it was too much input for one day. A total of 5 out of 30 (17%) participants mentioned that the theoretical part was too long and 4 (13%) said that more practical examples would have been beneficial.
We had a training session and after that SmED was installed by Medistar, which is the software provider. We then could use the test version to get familiar.
Almost all participants (26/30, 87%) reported that they had not used an initial-assessment software prior to implementation of SmED. Health care professionals explained that they had internal standards and a guideline for conversations with patients but could ask questions individually. A total of 4 health care professionals out of 30 (13%) stated that they had access to another initial-assessment software, which prioritized and categorized patients.
Prior to the start of using SmED, 3 participants out of 30 (10%) from the Joint Counter stated that they introduced a system where patients first pull a number and are then transferred to a waiting area further away from the counter due to data protection law. Out of 30 interviewees, 2 (7%) mentioned that they increased their number of employees.
This theme concerns the extent to which SmED becomes institutionalized or part of the routine organizational practices and policies. Different facilitators and barriers influencing implementation of SmED were identified by the health care professionals. One barrier according to the interviewees (10/30, 33%) is that currently an integration with other software is not possible. Participants explained that a closer connection of SmED with the information system used in daily practice would increase acceptance and feasibility. At this time, health care professionals stated that they had to work with both software programs in parallel. This had a massive impact on workload. Implementing an interface between the two software programs would, therefore, enhance implementation sustainability.
One problem is that there is no link or interface between SmED and the software usually used.
Nevertheless, I think if there is a link between SmED and our software, acceptance by the employees will increase and, with a higher acceptance, I will get more experienced and have a better understanding.
Health care professionals stated that they were disappointed due to challenging technical problems they face during their daily working routines. Hence, SmED is not always practicable (5/30, 17%) due to, for example, a poor internet connection. A total of 2 professionals out of 30 (7%) said that SmED is a computer-based program that is error prone (2/30, 7%) and can be tricked; for example, patients who are frequent callers know how to answer in order to be rated as urgent (1/30, 3%). A total of 2 health care professionals out of 30 (7%) explained that another barrier for successful implementation could be lack of employees in the future. At this time, additional time needed per call, per patient, already increases the workload per employee.
We will definitely need more employees; this is crucial, since the number of incoming calls has increased gradually and also time needed per call rose due to using SmED...
Moreover, using the software during peak times or at times with higher call volumes, such as during national holidays, is not feasible. Almost one-third of participants (9/30, 30%) reported that a large volume of incoming calls during the use of SmED increases pressure on employees and, thus, induces work-related stress. Health care professionals working at the Joint Counter stated that SmED is not practicable if the number of patients waiting is high (6/30, 20%).
At the moment, we use SmED only three days per week: Monday, Tuesday, and Thursday. I guess we are also supposed to use it on weekends and national holidays sometime soon. I think it is not feasible to use it then, due to the high number of patients visiting us.
Another barrier described by 1 participant (3%) is that sharing information with the next point of care is currently not possible. On the other hand, 1 health care professional (3%) mentioned that information can be shared between the Joint Counter and the rescue service. Thus, patients calling 112—the emergency phone number—inappropriately can be steered to the right point of care easily. A total of 4 (13%) participants from the Joint Counter described problems regarding data security and privacy. According to the health care professionals, separate rooms are needed to use SmED to prevent invasion of privacy. At this time, patients usually wait in a queue and can hear what the person in front of them is being asked.
...the problem is data protection! Patients sit here or stand here and then we ask them questions and another person is standing behind them. Even though we say, “Could you keep a proper distance please...”
A total of 2 (3%) health care professionals reported that, although they use SmED, a physician could be sitting next to them asking the patient the same questions and could rate the urgency based on her or his experiences and not based on SmED results. Nearly half of all health care professionals (14/30, 47%) described that, often, physicians do not read the summaries created by SmED. According to 11 participants (37%), summaries were only partly read. Thus, questions may be asked twice. This perhaps could give patients the impression that the professionals do not communicate effectively. Moreover, the health care professionals using SmED feel like their work is unnecessary.
Impact on the patient? I think...in general...patients like being questioned at first...it is not disadvantageous. However, sometimes the physician asks the same questions because they do not read the summary and then the patient thinks we do not communicate...
A total of 1 (3%) interviewee stated that developing a simpler version of SmED that can be used at the Joint Counter while examining patients may positively influence implementation sustainability. Moreover, regular software updates including user feedback (eg, symptoms that are missing) were needed during implementation. More than one-third of all participants (11/30, 37%) described that they could collect suggestions for improvement and share this information with software engineers. If appropriate, those suggestions could be integrated into SmED within the next software update, which could facilitate implementation. A total of 2 (7%) participants stated that this will support implementation sustainability. Additional training will also improve maintenance. Another facilitator for maintenance mentioned by 4 (13%) interviewees is that SmED is a medical product and, therefore, assures legal certainty. The
More than half of the participants (16/30, 53%) explained that the medical responsibility lies with the health care professionals answering the call or admitting the patient to the ED. However, depending on the software ranking, responsibility stays with the professionals or is handed over to the next point of care. The other half (12/30, 40%) reported that the medical responsibility always lies with a physician. A total of 1 (3%) participant stated that after implementing SmED, liability is placed on the medical product, which enhances implementation sustainability. According to 4 (13%) participants, there are neither facilitators nor barriers influencing the implementation, since it will be binding for all State Associations of Statutory Health Insurance Physicians to use SmED for initial assessment from 2020 onward.
Our expectations have been met. We are legally protected, we do not forget to ask certain questions, and it is a support for decision making. It is perfect for us.
This study focused on the perceptions of health care professionals at an early stage of the implementation of SmED. In general, health care professionals evaluated SmED positively. Workload increased initially, due to additional time needed per call, per patient. If SmED had been used more frequently and over a longer time period, its use by health care professionals would have become more routine, which would have a positive impact on time needed per call, per patient. SmED was perceived to support decision making regarding urgency for medical treatment, albeit not all patients were eligible. Technical problems, lack of integration with other software, and lack of practicability during peak times influenced the implementation process negatively. Eliminating given barriers may influence uptake and implementation sustainability.
SmED is not applicable to all patients, neither at the Joint Counter or at the initial telephone contact points. This group seems small, but includes patients with complex needs who may frequently contact out-of-hours care. Particularly, patients with neurological diseases [
The overall aim of SmED is to improve emergency care for patients with urgent conditions, steer patients with nonurgent medical needs toward the right point of care, and, therefore, disburden the EDs. The urgency for medical treatment rated by SmED was perceived to be higher than health care professionals’ assessments. This may have an impact on the effectiveness of SmED to identify patients with nonurgent conditions and to steer them toward the right point of care. This finding is consistent with those by Jansen Van Eijndt et al [
Health care professionals stated that their workload increased due to more time needed per call, per patient, when using SmED. This has a negative impact on the efficacy of the professionals. This finding is supported by the results of Porter et al [
SmED can function as a decision-making process regarding urgency for medical treatment support for unexperienced professionals. Nevertheless, many years of professional experience and knowledge were perceived as a basis for safe decision making. These findings correspond with those by Snooks et al [
Health care professionals expected SmED to be a tool for legal protection and quality assurance. However, at this point in time it is not clear if SmED can provide legal certainty. Moreover, health care professionals are able to skip questions; therefore, the concept of using a software program as a nationwide standard for quality assurance may not be fulfilled. Porter et al [
In general, the introduction of SmED and training prior to implementation was evaluated positively. However, implementation slightly differed among the project sites. Success of implementation was ensured differently, therefore, continuity was not given. This may have an impact on how health care providers evaluated and accepted SmED in general. The main barrier mentioned by the professionals is the lack of integration with the software that was used parallel to SmED. This finding is consistent with those of Porter et al [
According to regulation § 75 SGB V, the outpatient emergency care sector is the first contact point for patients outside of regular consultation hours (eg, during national holidays or on the weekend). However, use of SmED is not feasible during peak times or at times with higher call volumes (eg, national holidays and weekends). Using SmED will be binding for all State Associations of Statutory Health Insurance Physicians at the initial telephone contact points (ie, phone number 116117) [
This study was carried out in only 11 federal states of Germany; including the missing states could have identified new aspects. Although respondents were mostly outspoken about their perceptions, social desirability cannot be precluded. It is possible that comparability may not occur, due to the long time period for conducting the interviews. Moreover, health care professionals started to use SmED at different time points, which may have influenced their perceptions as well. Although different reminders to increase the response rate were used, only a low number of participants were reached. This may be due to not having contacted the health care professionals personally. This analysis was guided by an appropriate methodological strategy to minimize research bias and reduce the risk of losing relevant content. In addition, the analysis was done by the two first authors individually and was compared during several meetings to ensure consistent coding.
This study is probably the first study to investigate perceptions of health care professionals regarding a computer-based instrument for standardized initial assessment in Germany. Despite the limitations, this study shows that using a software-based instrument for standardized initial assessment supports health care professionals’ decision making. Nevertheless, SmED rates patients’ urgencies as higher than do the professionals; it is, therefore, not clear whether SmED steers patients with nonurgent medical needs toward the right point of care, thereby disburdening the EDs. The findings of this study could help to implement SmED in additional federal states of Germany, as well as to implement similar computer-based initial-assessment software systems in other European countries.
Translated interview guide used to conduct the qualitative interviews with health care professionals.
Institute for Applied Quality Improvement and Research in Health Care
computerized clinical decision support
emergency department
Netherlands Triage System
Reach, Effectiveness and Efficacy, Adoption, Implementation, and Maintenance
Swiss Medical Assessment System
Standardisierte medizinische Ersteinschätzung in Deutschland (
We would like to thank all health care professionals who took part in our interviews. This project has been funded by the Federal Joint Committee (
MW and JP conceived the study. CR and JP elaborated the study protocol. MW is principal investigator of the study and provided critical input at every stage of the development of the study protocol. AB, CR, and JP organized data collection. AB and CR conducted and analyzed the interviews. CR and AB wrote the manuscript. All authors provided substantial comments and approved the final version of the manuscript.
None declared.