Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 04.09.20 in Vol 4, No 9 (2020): September

Preprints (earlier versions) of this paper are available at, first published Feb 27, 2020.

This paper is in the following e-collection/theme issue:

    Original Paper

    Implementation of a Standardized Initial Assessment for Demand Management in Outpatient Emergency Care in Germany: Early Qualitative Process Evaluation

    Department of General Practice and Health Services Research, University Hospital Heidelberg, Heidelberg, Germany

    *these authors contributed equally

    Corresponding Author:

    Catharina Roth, BSc, MPH

    Department of General Practice and Health Services Research

    University Hospital Heidelberg

    Im Neuenheimer Feld 130.3



    Phone: 49 62215635736



    Background: Inadequate assessment of the severity and urgency of medical problems is one of the factors contributing to unnecessary emergency department (ED) visits. The implementation of a software-based instrument for standardized initial assessment—Standardisierte medizinische Ersteinschätzung in Deutschland (SmED) (Standardized medical Initial Assessment in Germany in English)—aims to support health care professionals and steer patients toward the right health care provider. This study aimed to explore the implementation process of SmED from a user perspective.

    Objective: This study aims to evaluate the overall perception of SmED by health care professionals using the software, to examine to what extent SmED influences the workload and work routines of health care professionals, and to determine which factors are associated with the use of SmED.

    Methods: An early qualitative process evaluation on the basis of interviews was carried out alongside the implementation of SmED in 26 outpatient emergency care services within 11 federal states in Germany. Participants were 30 health care professionals who work with SmED either at the joint central contact points of the outpatient emergency care service and the EDs of hospitals (ie, the Joint Counter; Gemeinsamer Tresen in German) or at the initial telephone contact points of the outpatient emergency care service (phone number 116117). Matrix-based framework analysis was applied to analyze the interview data.

    Results: Health care professionals perceived that workload increased initially, due to additional time needed per patient. When using SmED more frequently and over a longer time period, its use became more routine and the time needed per call, per patient, decreased. SmED was perceived to support decision making regarding urgency for medical treatment, but not all types of patients were eligible. Technical problems, lack of integration with other software, and lack of practicability during peak times affected the implementation of SmED.

    Conclusions: Initial experiences with SmED were positive, in general, but also highlighted organizational issues that need to be addressed to enhance sustainability.

    Trial Registration: German Clinical Trials Register DRKS00017014;

    JMIR Form Res 2020;4(9):e18456




    In recent years, steadily increasing utilization of emergency departments (EDs) has aroused public and political attention, not only in Germany but in many nations [1-7]. Different studies have examined factors that contribute to increased numbers of emergency admissions [5,8,9]. Factors that may explain high use of emergency resources include an aging population, increased number of chronically ill people, lack of cost awareness, lack of accessibility, unclear organization of outpatient emergency care service, and patients’ personal assessments regarding severity and urgency of medical conditions [9-11]. The high use of EDs negatively affects not only patients (eg, long waiting times, reduced patient satisfaction, and higher mortality) but also health care professionals (eg, high workload, work-related stress, safety, and efficiency issues) working at the ED [2,4,9,12]. A range of strategies have been applied to address this issue by policy makers [6].

    One category of strategies are programs and policies that aim to reduce the number of unnecessary ED visits. Unnecessary ED visits are defined as using the ED for health issues that do not require immediate medical treatment. Those visits could be classified as inappropriate, because ED resources are used for health issues that can be treated elsewhere (eg, in primary medical care or outpatient emergency care) [13]. The estimated prevalence of inappropriate visits at EDs is between 20% and 40% [13,14]. Factors associated with inappropriate use of EDs include patient age, education, and absence of family support [14]. In addition, inadequate assessment of severity and urgency of presented health problems determine inappropriate ED visits [4,14,15].

    In Germany, the organization of emergency care service is complex [9]. The National Association of Statutory Health Insurance Physicians distinguish between three levels of emergency care services: the outpatient emergency care service (Level I), the emergency care service of hospitals (Level II), and the rescue services (Level III) [16]. The outpatient emergency care service is for acute medical conditions that are not potentially life-threatening that are presented outside of regular consultation hours. This service can be subdivided into the initial telephone contact points (ie, phone number 116117) or outpatient emergency practices. Outpatient emergency practices can be regular primary care physician practices or can be located at a hospital. The phone number 116117 is responsible for steering patients with medical conditions toward the right point of care. Patients calling this number have different options depending on their urgency for medical treatment. They can be advised regarding self-treatment at home, receipt of medical treatment by the primary care sector within the next few days depending on regular consultation hours, medical referral to an outpatient emergency practice, home visit by an outpatient physician, medical referral to the ED, or medical referral to the rescue service, based on their urgency for medical treatment. The Joint Counter (Gemeinsamer Tresen in German) is a cooperative arrangement between the outpatient emergency care service (Level I) and the EDs of the hospitals (Level II). Patients who visit the Joint Counter are either admitted to the ED or treated by the outpatient emergency care service (see Figure 1) [16].

    Since the beginning of 2019, a computer-based software—Standardisierte medizinische Ersteinschätzung in Deutschland (SmED) (Standardized medical Initial Assessment in Germany in English)—has been implemented within the Joint Counter and the phone number 116117 to support health care professionals to steer patients toward the right point of care and, therefore, reduce inappropriate ED visits: both settings are highlighted in blue in Figure 1. Previous research showed that many complex interventions (eg, implementing a computer-based software like SmED) that have been shown to be effective failed to be implemented sustainably and widely in health care practice [17,18]. Many types of factors can act as barriers or facilitators of implementation of changes, including individual, organizational, and health system–related factors [19,20]. It is crucial to understand these factors in order to optimize the speed and comprehensiveness of implementation [17]. This study aimed to answer the following research questions: (1) What are health professionals’ overall perceptions of SmED? (2) To what extent did SmED influence workload and work routines of health care professionals? and (3) Which factors were associated with the use of SmED?

    Figure 1. Overview of the three different levels of emergency care in Germany.
    View this figure


    Study Design, Ethics Approval, and Trial Registration

    This early process evaluation aimed to investigate the following: Reach, Effectiveness and Efficacy, Adoption, Implementation, and Maintenance (RE-AIM), according to the RE-AIM framework [21]. A qualitative process evaluation on the basis of interviews with health care professionals was carried out alongside the implementation of SmED. Ethics approval was obtained from the Medical Ethics Committee of the Medical Faculty Heidelberg (S-640/2018) prior to the start of the study in October 2018. The study has been registered at the German Clinical Trial Register prior to the start of the study (registration No. DRKS00017014).

    Study Setting

    The overall aim of the DEMAND project is to improve medical care of patients who present with an urgent need for emergency treatment and/or medical advice on the basis of a more efficient use of emergency care resources. The implementation of a software-based instrument for standardized initial assessment (ie, SmED) aims to support health care professionals (eg, nurses, physician assistants, and paramedics) in emergency care and to steer patients with nonurgent medical needs toward the right point of care. The State Associations of Statutory Health Insurance Physicians have implemented SmED since March 2019 at the Joint Counter, as well as the phone number 116117, in 11 of the 16 federal states of Germany. Health care professionals within this setting were expected to change their service delivery by using the software and, therefore, influence individuals in the target population, which included patients who contact the Joint Counter or call 116117 regarding the need for medical treatment and/or advice.

    Study Population

    The Institute for Applied Quality Improvement and Research in Health Care (aQua Institute) invited all State Associations of Statutory Health Insurance Physicians from each federal state of Germany to take part in this project. Out of 16 federal states, 11 agreed to take part. No specific reasons for the nonparticipation of the other five federal states were identified. The State Associations of Statutory Health Insurance Physicians informed the research team at the University Hospital Heidelberg about all health care professionals using SmED and were, therefore, potential participants. According to this number, information packages, including an invitation letter, an information leaflet, and an informed consent form for tape recording, were put together and sent to a contact person responsible for distributing. The information sheet included contact details of research team members who were available to participants to discuss the study or address additional concerns or questions. Participants who decided to take part in an interview were requested to contact the researchers directly. An informed consent form for tape recording was signed prior to the start of the interview by the participant and the interviewer. Different strategies, including email reminders, telephone calls, and the annual project coordination team meeting, were used to maximize response rate.

    The Intervention

    SmED is a computer-based software that requires an internet connection. The software can be used by health care professionals (eg, nurses, practice assistants, and paramedics) for initial assessment as a basis for demand management in outpatient emergency care services. The purpose of this software is to support health care professionals and to steer patients toward the right point of care based on their actual medical needs. SmED can be used in different outpatient emergency care services and applies a number of well-defined questions regarding different medical disorders and issues. SmED uses an algorithm based on the red flag approach to rate the urgency of patients. There are four different degrees of urgency: (I) emergency, (II) fastest possible medical treatment by a physician, (III) medical treatment by a primary care physician within 24 hours, and (VI) medical treatment within the next couple of days or telephone consulting. Patients are steered toward the right point of care based on this algorithm. In this project, SmED is used at the Joint Counter and during calls to the telephone number 116117. The software can be used immediately and at any time. However, in this project the sites decided individually when and how often they would use SmED. SmED was developed on the basis of an available system from Switzerland—the Swiss Medical Assessment System (SMASS)—and adapted to the German health care system before implementation. Regular software updates based on user feedback were performed to modify SmED during implementation.

    Implementation Activities

    The aQua Institute organized workshops prior to the implementation of SmED for all State Associations of Statutory Health Insurance Physicians from each participating federal state of Germany. A training concept for potential users and trainers was designed. Additionally, a data protection concept and an implementation plan for each project site was developed. Moreover, quality management and a support management program responsible for implementation sustainability were introduced.

    Measures and Data Collection

    Semistructured telephone interviews were conducted by the two first authors (AB and CR) with 30 health care professionals who used SmED between July and December 2019. The semistructured interview guideline (see Multimedia Appendix 1) was developed on the basis of the RE-AIM framework [22]. It covers the following themes: reach, effectiveness and efficacy, adoption and uptake, implementation, and maintenance and sustainability. These themes, as described in the original description of the RE-AIM framework [22], were translated into questions in the context of SmED. All interviews were audio recorded with the consent of all participants and ranged in length from 8 to 39 minutes. The interviews were pseudonymized and transcribed verbatim. Transcripts were not returned to participants for comment or correction. All interview quotes were translated into English by the first author (CR).

    Data Analysis

    The matrix-based method of framework analysis according to Ritchie and Spencer [23] and Gale et al [24] was conducted. This analysis is seen as an appropriate content analysis approach in a study where a conceptual framework is available at the start of the study [24]. As a first step, all interviews were transcribed verbatim (Step 1: Transcription). Additionally, all transcripts were reviewed by the two interviewers (AB and CR) for accuracy. After verbatim transcription of the interviews, the two first authors (AB and CR) became familiar with the whole dataset (Step 2: Familiarization). In step three, the first two interviews were deductively coded independently by the two first authors (AB and CR). Codes and themes of interest were defined based on the interview guide and on the RE-AIM framework. A few themes of interest were identified inductively from the data during the analysis (Step 3: Coding). The results were discussed and a final coding system based on the RE-AIM framework was developed for further analyses (Step 4: Developing a working analytical framework). The remaining interview transcripts were assigned to the existing codes by one researcher (CR or AB) and checked by a second researcher (AB or CR) (Step 5: Applying the analytical framework) [24]. The final step was to analyze whether there were any differences or connections between the codes. Interview data were analyzed using MAXQDA, version 2018.1.0, a computer-assisted qualitative data management software. Additionally, participant characteristics were analyzed descriptively using SPSS Statistics for Windows, version 25.0 (IBM Corp).

    Availability of Data and Material

    The dataset generated and analyzed during this study will not be made publicly available due to European Data Protection Law but may be available by the corresponding author upon reasonable request.



    At the beginning of the study, according to the project coordinator, 391 health care professionals used SmED. All 391 professionals were invited to take part in an interview and 30 decided to take part. Characteristics of the participants are described in Table 1.

    Table 1. Characteristics of health care professionals.
    View this table

    Theme 1: Reach

    Regarding the target population of SmED, the health care professionals explained that SmED is not applicable to patients with hearing impairment (1/30, 3%); elderly patients (2/30, 7%); terminally ill patients (2/30, 7%); patients who are incapable of answering questions due to, for example, neurological diseases (3/30, 10%); or patients suffering from alcohol or drug overdose (1/30, 3%). SmED is also not feasible for patients where someone else (eg, husband or wife) is calling on their behalf (3/30, 10%), patients with psychiatric disorders (5/30, 17%), or patients with language barriers (14/30, 47%). Moreover, SmED is not applicable if qualified health care professionals (eg, nurses working at a nursing home) call on the behalf of a patient (8/30, 27%). Health care professionals working at the initial telephone contact point (ie, phone number 116117) also reported that they do not use SmED for patients who want to order a prescription or for those seeking simple advice (3/30, 10%). Health care professionals working at the Joint Counter explained that, furthermore, SmED is not feasible for patients who are regular visitors to a long-term therapy service (1/30, 3%) or who are in poor medical condition (6/30, 20%). One health care professional stated that patients with chronic diseases or nursing diagnoses (eg, having a urinary catheter), for example, are not included.

    80-year-old Turkish person who speaks very little German does not understand the questions and then there is no one to translate, and then SmED, it serves no purpose.
    [Participant working at the Joint Counter]

    Theme 2: Effectiveness and Efficacy

    This theme concerns the impact of SmED on steering patients toward the right point of care and the impact of SmED on workload and working methods. A vast majority of health care professionals (21/30, 70%) stated that SmED rates the urgency for medical treatment higher than they themselves would rate the urgency. For instance, if patients report high blood pressure (4/30, 13%), infection of the gastrointestinal system (2/30, 7%), respiratory problems (3/30, 10%), fever and chills (3/30, 10%), severe pain (3/30, 10%), or uncontrolled falls (1/30, 3%), SmED immediately rates them as emergencies. Almost two-thirds of all participants (18/30, 60%) reported that if SmED rates the urgency for medical treatment higher than they expected, they changed the decision based on their professional experience or after consultation with a physician. If the health care professionals changed the category as rated by SmED, they documented and justified it on the final summary. Only 5 participants out of 30 (17%) said that they had to accept how SmED rated patients even though they disagreed with the software.

    Handling of patients who are rated as nonurgent varied, depending on the setting. Patients who call the phone number 116117 and are classified as nonurgent are advised to visit an outpatient emergency practice (4/30, 13%), are forwarded to telephone counseling with a physician (9/30, 30%), receive a home visit by the emergency care physician (5/30, 17%), or are given advice (eg, going to the primary care physician within the next few days or treating themselves at home) (6/30, 20%). At some initial telephone contact points, patients are given the opportunity to call back if their medical condition is getting worse (3/30, 10%). Patients who contact the Joint Counter are treated by a physician within the next few hours but have to wait (9/30, 30%) or are referred to the outpatient emergency care sector (1/30, 3%). Due to legal requirements, all patients who contact the Joint Counter have to be treated or seen by a physician; thus, at three Joint Counters, waiting lists based on SmED categories are implemented to organize patients.

    Theme 3: Adoption and Uptake

    This theme concerns whether health care professionals are willing to use SmED. When asked about expectations or disappointments, one-third of the participants said they expected SmED to be an aid for decision making, especially for unexperienced colleagues (11/30, 37%). However, health care professionals stated that their professional experience influenced decisions concerning urgency of medical treatment independently of the software. In addition, their intuition, professional knowledge, and experience influenced how or if they used the software. Health care professionals stated that after working in the field of emergency care service for many years they are able to make safe decisions and rate the urgency for medical treatment of patients without a support system (18/30, 60%).

    Well, meanwhile, I personally would say that especially for younger coworkers, it is a chance to ask structured questions. And it does not matter from where the patient is calling, we ask the same questions and eventually it is always the same result...
    [Participant from telephone contact point (phone number 116117)]

    Health care professionals stated that they assumed that SmED could function as a tool for quality assurance (eg, as a standard tool) (4/30, 13%). According to 7 out of 30 (23%) health care professionals, SmED provides support regarding decision making and helps to structure the assessment and, therefore, to correctly assess patients’ needs for medical treatment. Nevertheless, one-third (11/30, 37%) of the participants reported that the tool is imprecise and unstructured. The order of the questions sometimes does not match a patient’s symptoms. One advantage according to the health care professionals is that they do not forget questions and the system provides an opportunity for a structured assessment (8/30, 27%). In addition, questions asked after implementation have not been asked before (eg, whether the patient has been abroad lately) (1/30, 3%). Nevertheless, 4 health care professionals out of 30 (13%) mentioned that it is possible to skip questions if they are unnecessary or inappropriate, based on their professional judgment.

    I think the main advantage is that the conversation is structured...And yes, that we identify people who dramatically report their symptoms even though they aren’t that urgent. Yes, I think that we identify them.
    [Participant from telephone contact point (phone number 116117)]

    Although health care professionals expected SmED to be more patient oriented (8/30, 27%), some stated that it includes too many questions (3/30, 10%). Particular questions regarding medication or drugs are too complicated to answer for some patients. A small number of participants said SmED is too comprehensive (6/30, 20%). Moreover, almost one-third of interviewees (8/30, 27%) explained that they have the perception that a small group of patients are being rude and are annoyed due to busy lines, long waiting times at the Joint Counter, or being asked too many questions.

    ...the conversation definitely takes longer and, therefore, the lines are busy. Patients are sometimes annoyed if they have to wait 15 to 20 minutes instead of 10 minutes due to the additional time needed per call. Sometimes I have to ask comprehension questions like “What do you mean by that?” and that’s definitely a disadvantage.
    [Participant from telephone contact point (phone number 116117)]

    In addition, if a patient is feeling really sick or if the situation is too challenging, answering all SmED questions puts an additional burden on the patient (2/30, 7%). Furthermore, if a patient is calling 116117 and the lines are busy, they may decide to hang up and call the rescue service.

    I always think that if someone is really feeling sick, SmED might be a burden. I mean, I do not have feedback regarding that, no one told me that, but I have the feeling that SmED is too much for some patients.
    [Participant from telephone contact point (phone number 116117)]

    However, when asked about SmED’s impact on patients, almost half of the participants (14/30, 47%) explained that they have the perception that patients, in general, feel like they are in good hands and that they are grateful for a more structured and comprehensive assessment. Thus, according to 6 professionals out of 30 (20%), using SmED increases patient safety.

    ...and a lot of patients like that we have to talk with them a little longer, they are like “It was very kind of you that you took that much time for me.”
    [Participant from telephone contact point (phone number 116117)]

    The majority of health care professionals reported that one barrier was that using SmED increases their workload due to the additional time needed per call, per patient (21/30, 70%). However, through more frequent use of SmED, it became more routine and the additional time needed per call, per patient, decreased (9/30, 30%). A facilitator for implementation was motivation through colleagues, trainers, or leaders (3/30, 10%). If these persons are convinced by SmED, they were able to encourage other health care professionals. On the other hand, health care professionals who see only the negative effects of the new software can have a negative impact on successful implementation of SmED.

    First of all, long-term employees have to be convinced because they are used to the old system. And yes, during work life, people are usually confronted with new things; however, Germans tend to only see the negative sides of new things at the beginning.
    [Participant from telephone contact point (phone number 116117)]

    Theme 4: Implementation

    This theme concerns the consistency of the organization and adjustments made during and prior to the delivery of the intervention as well as implementation strategies. All interviewees (30/30, 100%) reported that they participated in a training session prior to implementation. The development and structure of SmED and its beginning in Switzerland were presented. During the training session, participants worked with exemplary patient cases. The success of the training sessions was monitored differently. However, nearly one-third of all health care professionals (8/30, 27%) stated that they used patient cases to learn how to use SmED. Those cases were discussed at the end of the training session. Out of 30 participants, 1 (3%) said that they used SmED once during their working hours with different patient cases to understand how the software works in real-life situations. Another participant (1/30, 3%) explained that they had to use SmED five times during their working hours and documented it. Out of 30 participants, 1 (3%) explained that they watched web-based videos and tested their knowledge afterward. A total of 5 out of 30 (17%) interviewees stated that they were able to ask experienced colleagues further questions at the start of the implementation of SmED to learn how to use the software correctly. The overall training was evaluated as completely positive by 20 out of 30 (67%) interviewees. Out of 30 participants, 1 (3%) brought up that it was too much input for one day. A total of 5 out of 30 (17%) participants mentioned that the theoretical part was too long and 4 (13%) said that more practical examples would have been beneficial.

    We had a training session and after that SmED was installed by Medistar, which is the software provider. We then could use the test version to get familiar.
    [Participant working at the Joint Counter]

    Almost all participants (26/30, 87%) reported that they had not used an initial-assessment software prior to implementation of SmED. Health care professionals explained that they had internal standards and a guideline for conversations with patients but could ask questions individually. A total of 4 health care professionals out of 30 (13%) stated that they had access to another initial-assessment software, which prioritized and categorized patients.

    Prior to the start of using SmED, 3 participants out of 30 (10%) from the Joint Counter stated that they introduced a system where patients first pull a number and are then transferred to a waiting area further away from the counter due to data protection law. Out of 30 interviewees, 2 (7%) mentioned that they increased their number of employees.

    Theme 5: Maintenance

    This theme concerns the extent to which SmED becomes institutionalized or part of the routine organizational practices and policies. Different facilitators and barriers influencing implementation of SmED were identified by the health care professionals. One barrier according to the interviewees (10/30, 33%) is that currently an integration with other software is not possible. Participants explained that a closer connection of SmED with the information system used in daily practice would increase acceptance and feasibility. At this time, health care professionals stated that they had to work with both software programs in parallel. This had a massive impact on workload. Implementing an interface between the two software programs would, therefore, enhance implementation sustainability.

    One problem is that there is no link or interface between SmED and the software usually used.
    [Participant from telephone contact point (phone number 116117)]
    Nevertheless, I think if there is a link between SmED and our software, acceptance by the employees will increase and, with a higher acceptance, I will get more experienced and have a better understanding.
    [Participant from telephone contact point (phone number 116117)]

    Health care professionals stated that they were disappointed due to challenging technical problems they face during their daily working routines. Hence, SmED is not always practicable (5/30, 17%) due to, for example, a poor internet connection. A total of 2 professionals out of 30 (7%) said that SmED is a computer-based program that is error prone (2/30, 7%) and can be tricked; for example, patients who are frequent callers know how to answer in order to be rated as urgent (1/30, 3%). A total of 2 health care professionals out of 30 (7%) explained that another barrier for successful implementation could be lack of employees in the future. At this time, additional time needed per call, per patient, already increases the workload per employee.

    We will definitely need more employees; this is crucial, since the number of incoming calls has increased gradually and also time needed per call rose due to using SmED...
    [Participant working at the Joint Counter]

    Moreover, using the software during peak times or at times with higher call volumes, such as during national holidays, is not feasible. Almost one-third of participants (9/30, 30%) reported that a large volume of incoming calls during the use of SmED increases pressure on employees and, thus, induces work-related stress. Health care professionals working at the Joint Counter stated that SmED is not practicable if the number of patients waiting is high (6/30, 20%).

    At the moment, we use SmED only three days per week: Monday, Tuesday, and Thursday. I guess we are also supposed to use it on weekends and national holidays sometime soon. I think it is not feasible to use it then, due to the high number of patients visiting us.
    [Participant working at the Joint Counter]

    Another barrier described by 1 participant (3%) is that sharing information with the next point of care is currently not possible. On the other hand, 1 health care professional (3%) mentioned that information can be shared between the Joint Counter and the rescue service. Thus, patients calling 112—the emergency phone number—inappropriately can be steered to the right point of care easily. A total of 4 (13%) participants from the Joint Counter described problems regarding data security and privacy. According to the health care professionals, separate rooms are needed to use SmED to prevent invasion of privacy. At this time, patients usually wait in a queue and can hear what the person in front of them is being asked.

    ...the problem is data protection! Patients sit here or stand here and then we ask them questions and another person is standing behind them. Even though we say, “Could you keep a proper distance please...”
    [Participant working at the Joint Counter]

    A total of 2 (3%) health care professionals reported that, although they use SmED, a physician could be sitting next to them asking the patient the same questions and could rate the urgency based on her or his experiences and not based on SmED results. Nearly half of all health care professionals (14/30, 47%) described that, often, physicians do not read the summaries created by SmED. According to 11 participants (37%), summaries were only partly read. Thus, questions may be asked twice. This perhaps could give patients the impression that the professionals do not communicate effectively. Moreover, the health care professionals using SmED feel like their work is unnecessary.

    Impact on the patient? I general...patients like being questioned at is not disadvantageous. However, sometimes the physician asks the same questions because they do not read the summary and then the patient thinks we do not communicate...
    [Participant working at the Joint Counter]

    A total of 1 (3%) interviewee stated that developing a simpler version of SmED that can be used at the Joint Counter while examining patients may positively influence implementation sustainability. Moreover, regular software updates including user feedback (eg, symptoms that are missing) were needed during implementation. More than one-third of all participants (11/30, 37%) described that they could collect suggestions for improvement and share this information with software engineers. If appropriate, those suggestions could be integrated into SmED within the next software update, which could facilitate implementation. A total of 2 (7%) participants stated that this will support implementation sustainability. Additional training will also improve maintenance. Another facilitator for maintenance mentioned by 4 (13%) interviewees is that SmED is a medical product and, therefore, assures legal certainty. The red flag system or the priority list (4/30, 13%), which helps to easily identify emergency patients, is another benefit.

    More than half of the participants (16/30, 53%) explained that the medical responsibility lies with the health care professionals answering the call or admitting the patient to the ED. However, depending on the software ranking, responsibility stays with the professionals or is handed over to the next point of care. The other half (12/30, 40%) reported that the medical responsibility always lies with a physician. A total of 1 (3%) participant stated that after implementing SmED, liability is placed on the medical product, which enhances implementation sustainability. According to 4 (13%) participants, there are neither facilitators nor barriers influencing the implementation, since it will be binding for all State Associations of Statutory Health Insurance Physicians to use SmED for initial assessment from 2020 onward.

    Our expectations have been met. We are legally protected, we do not forget to ask certain questions, and it is a support for decision making. It is perfect for us.
    [Participant from telephone contact point (phone number 116117) and the Joint Counter]


    Principal Findings

    This study focused on the perceptions of health care professionals at an early stage of the implementation of SmED. In general, health care professionals evaluated SmED positively. Workload increased initially, due to additional time needed per call, per patient. If SmED had been used more frequently and over a longer time period, its use by health care professionals would have become more routine, which would have a positive impact on time needed per call, per patient. SmED was perceived to support decision making regarding urgency for medical treatment, albeit not all patients were eligible. Technical problems, lack of integration with other software, and lack of practicability during peak times influenced the implementation process negatively. Eliminating given barriers may influence uptake and implementation sustainability.

    Comparison With Previous Work


    SmED is not applicable to all patients, neither at the Joint Counter or at the initial telephone contact points. This group seems small, but includes patients with complex needs who may frequently contact out-of-hours care. Particularly, patients with neurological diseases [25] or psychiatric disorders [26] need to be steered toward the right point of care based on their urgency due to the risk of sudden or unexpected deterioration. The same is true for patients with alcohol intoxication [26]. According to a study by Pajonk et al [27], the number of psychiatric emergencies has increased in recent decades. It, therefore, seems problematic that SmED does not depict those cases.

    Effectiveness and Efficacy

    The overall aim of SmED is to improve emergency care for patients with urgent conditions, steer patients with nonurgent medical needs toward the right point of care, and, therefore, disburden the EDs. The urgency for medical treatment rated by SmED was perceived to be higher than health care professionals’ assessments. This may have an impact on the effectiveness of SmED to identify patients with nonurgent conditions and to steer them toward the right point of care. This finding is consistent with those by Jansen Van Eijndt et al [28]. They had more urgent patients after implementing a computer-based triage system. A more defensive algorithm identifying nonurgent patients may be needed to improve the effectiveness of SmED regarding steering nonurgent patients toward the right point of care. A systematic review by Huibers et al [29] concluded that, on average, patient safety is high if patients with nonurgent medical symptoms contact the out-of-hours service via telephone. However, there is room for enhancing patient safety for patients who present with highly urgent medical symptoms [29]. They found that an average of 10% of telephone triage contacts with patients reporting urgent symptoms resulted in different adverse events, such as medical errors or unplanned ED attendance. In our study, health care professionals stated that patient safety, in general, increased after implementation of SmED. This finding agrees with the results of a study by Van Ierland et al [30] evaluating the validity of the Netherlands Triage System (NTS). They detected an increasing tendency toward more ED referrals among high-urgency-level patients and an increased amount of self-care advice among lower-urgency-level patients for the computer-based telephone triage. The urgency levels ranked by the systems matched with the majority of those using the system; thus, NTS seems to be feasible. However, an independent standard identifying true undertriage and overtriage may be needed [30]. According to Scherer et al [9], the main reason for overcrowding EDs is patients who do not require urgent medical care. Hence, the objective to address this problem may not be achieved, due to lack of effectiveness described above. In addition, at the Joint Counter, patients rated as nonurgent are treated by a physician but have long waiting times, which contribute to the overcrowding. Nevertheless, due to legal regulations (§ 27 SGB V), employees of the Joint Counter are ordered to treat every patient. Patients rated as nonurgent at the initial telephone contact points (ie, phone number 116117) are admitted to the outpatient emergency practice, connected to telephone counseling, or given telephone advice. However, at this point in time it is not clear whether patients accepted this procedure or whether they still sought medical treatment at the ED after calling 116117.

    Health care professionals stated that their workload increased due to more time needed per call, per patient, when using SmED. This has a negative impact on the efficacy of the professionals. This finding is supported by the results of Porter et al [31]. In their study, they evaluated the experiences of paramedics using a computerized clinical decision support (CCDS) system. The interviewed paramedics reported that they need more time at the scene when using the CCDS system. Nevertheless, this finding disagrees with those by Ong et al [32]. In their study, they investigated whether there is a difference in call duration and triage decisions in out-of-hours cooperatives when using or not using an expert system. Professionals interviewed in our study stated that their perceptions regarding additional time needed changed positively over time due to frequent use of SmED, which enhances routine and, in turn, has a positive impact on efficacy.


    SmED can function as a decision-making process regarding urgency for medical treatment support for unexperienced professionals. Nevertheless, many years of professional experience and knowledge were perceived as a basis for safe decision making. These findings correspond with those by Snooks et al [7], who investigated paper-based protocols to assess and triage patients, regarding their need for transportation to the ED. According to the findings by Snooks et al [7], decisions were influenced mainly by experience. Porter et al [31] concluded that paramedics would rather trust their clinical skills than the software [31]. Chang et al [33] described that professional experience and intuition plays an important role for nurses while using a triage system and improves decision making; these findings are consistent with those by Smits et al [34], Van der Linden et al [35], and O’Cathain et al [36]. Smits et al [34] investigated telephone triage in general practices in the Netherlands and concluded that triage accuracy is higher among practice assistants with more years of professional experience. Van der Linden et al [35] investigated whether ED crowding affects triage processes in the Netherlands. They found that more patients were redirected to general practice care when the nurses were more experienced. O’Cathain et al [36] evaluated whether different types of nurses differ regarding their triage decisions. They observed that length of clinical experience and type of software used influenced decision making. Nurses with more experience were more likely to give advice regarding self-treatment at home compared to less experienced nurses [36]. Nurses with more years of experience have more professional knowledge and clinical skills compared to professionals with fewer years of experience; therefore, decision making differs [37]. Although SmED supports decision making among younger professionals, professional experience is perceived as an important factor.

    Health care professionals expected SmED to be a tool for legal protection and quality assurance. However, at this point in time it is not clear if SmED can provide legal certainty. Moreover, health care professionals are able to skip questions; therefore, the concept of using a software program as a nationwide standard for quality assurance may not be fulfilled. Porter et al [31] support this finding. According to their results, several paramedics used CCDS systems retrospectively to save time and not as a decision support. SmED uses an algorithm based on the red flag approach. It is not clear whether only relying on this algorithm is acceptable or if the experience of health care professionals regarding patient safety should be considered. Interestingly, the perception of the structure of SmED varied between imprecise and precise. The participants reported that they can skip questions and that the order of the questions is inappropriate. However, SmED provides a system for a structured initial assessment and functions as a reminder for crucial questions. These findings agree with those by Porter et al [31], who found that health professionals explained that CCDS systems reminded them to do all necessary checks. Patients’ satisfaction increased due to more structured and comprehensive assessments. This finding is supported by Snooks et al [7], where patients’ satisfaction with their care was higher due to more in-depth assessments.

    Implementation and Maintenance

    In general, the introduction of SmED and training prior to implementation was evaluated positively. However, implementation slightly differed among the project sites. Success of implementation was ensured differently, therefore, continuity was not given. This may have an impact on how health care providers evaluated and accepted SmED in general. The main barrier mentioned by the professionals is the lack of integration with the software that was used parallel to SmED. This finding is consistent with those of Porter et al [31], who detected a variety of technical problems (eg, difficulties keeping the PC charged or printing patient records). Health care professionals also assumed that SmED could be a tool for quality assurance but, contrary to expectations, SmED still has some technical limitations, such as a lack of practicability due to a poor internet connection. These results agree with those by Porter et al [31], where paramedics reported having problems with the CCDS system, since it is web based and, thus, not practicable without an internet connection.

    According to regulation § 75 SGB V, the outpatient emergency care sector is the first contact point for patients outside of regular consultation hours (eg, during national holidays or on the weekend). However, use of SmED is not feasible during peak times or at times with higher call volumes (eg, national holidays and weekends). Using SmED will be binding for all State Associations of Statutory Health Insurance Physicians at the initial telephone contact points (ie, phone number 116117) [38]. In addition, advertisements for the phone number 116117 have increased, with more marketing planned in 2020. Thus, the number of patients calling 116117 may rise further. More employees may be needed to ensure that all calls can be answered within an appropriate time period. High incoming call volumes have already increased work-related pressure. Additional time needed per call may exacerbate this issue. Nevertheless, it is not clear whether SmED is the only related factor.

    Limitations and Strengths

    This study was carried out in only 11 federal states of Germany; including the missing states could have identified new aspects. Although respondents were mostly outspoken about their perceptions, social desirability cannot be precluded. It is possible that comparability may not occur, due to the long time period for conducting the interviews. Moreover, health care professionals started to use SmED at different time points, which may have influenced their perceptions as well. Although different reminders to increase the response rate were used, only a low number of participants were reached. This may be due to not having contacted the health care professionals personally. This analysis was guided by an appropriate methodological strategy to minimize research bias and reduce the risk of losing relevant content. In addition, the analysis was done by the two first authors individually and was compared during several meetings to ensure consistent coding.


    This study is probably the first study to investigate perceptions of health care professionals regarding a computer-based instrument for standardized initial assessment in Germany. Despite the limitations, this study shows that using a software-based instrument for standardized initial assessment supports health care professionals’ decision making. Nevertheless, SmED rates patients’ urgencies as higher than do the professionals; it is, therefore, not clear whether SmED steers patients with nonurgent medical needs toward the right point of care, thereby disburdening the EDs. The findings of this study could help to implement SmED in additional federal states of Germany, as well as to implement similar computer-based initial-assessment software systems in other European countries.


    We would like to thank all health care professionals who took part in our interviews. This project has been funded by the Federal Joint Committee (Gemeinsamer Bundesausschuss in German) through the Innovation Committee (01VSD17019).

    Authors' Contributions

    MW and JP conceived the study. CR and JP elaborated the study protocol. MW is principal investigator of the study and provided critical input at every stage of the development of the study protocol. AB, CR, and JP organized data collection. AB and CR conducted and analyzed the interviews. CR and AB wrote the manuscript. All authors provided substantial comments and approved the final version of the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Translated interview guide used to conduct the qualitative interviews with health care professionals.

    DOCX File , 14 KB


    1. Hsia RY, Sabbagh SH, Guo J, Nuckton TJ, Niedzwiecki MJ. Trends in the utilisation of emergency departments in California, 2005-2015: A retrospective analysis. BMJ Open 2018 Jul 23;8(7):e021392 [FREE Full text] [CrossRef] [Medline]
    2. Flores-Mateo G, Violan-Fors C, Carrillo-Santisteve P, Peiró S, Argimon J. Effectiveness of organizational interventions to reduce emergency department utilization: A systematic review. PLoS One 2012;7(5):e35903 [FREE Full text] [CrossRef] [Medline]
    3. Oterino de la Fuente D, Baños Pino JF, Fernández Blanco V, Rodríguez-Alvarez A. Impact in Asturias of primary care emergencies on hospital emergencies for the 1994-2001 period. A time series cointegration analysis [article in Spanish]. Rev Esp Salud Publica 2007;81(2):191-200 [FREE Full text] [CrossRef] [Medline]
    4. Lowthian JA, Cameron PA, Stoelwinder JU, Curtis A, Currell A, Cooke MW, et al. Increasing utilisation of emergency ambulances. Aust Health Rev 2011 Feb;35(1):63-69. [CrossRef] [Medline]
    5. Pasarín MI, Fernández de Sanmamed MJ, Calafell J, Borrell C, Rodríguez D, Campasol S, et al. Reasons for attending emergency departments. People speak out [article in Spanish]. Gac Sanit 2006;20(2):91-99 [FREE Full text] [CrossRef] [Medline]
    6. Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: A review of reviews. Health Policy 2016 Dec;120(12):1337-1349 [FREE Full text] [CrossRef] [Medline]
    7. Snooks HA, Kearsley N, Dale J, Halter M, Redhead J, Foster J. Gaps between policy, protocols and practice: A qualitative study of the views and practice of emergency ambulance staff concerning the care of patients with non-urgent needs. Qual Saf Health Care 2005 Aug;14(4):251-257 [FREE Full text] [CrossRef] [Medline]
    8. Ragin DF, Hwang U, Cydulka RK, Holson D, Haley LL, Richards CF, Emergency Medicine Patients' Access To Healthcare (EMPATH) Study Investigators. Reasons for using the emergency department: Results of the EMPATH Study. Acad Emerg Med 2005 Dec;12(12):1158-1166 [FREE Full text] [CrossRef] [Medline]
    9. Scherer M, Lühmann D, Kazek A, Hansen H, Schäfer I. Patients attending emergency departments. Dtsch Arztebl Int 2017 Sep 29;114(39):645-652 [FREE Full text] [CrossRef] [Medline]
    10. Bezzina AJ, Smith PB, Cromwell D, Eagar K. Primary care patients in the emergency department: Who are they? A review of the definition of the 'primary care patient' in the emergency department. Emerg Med Australas 2005;17(5-6):472-479. [CrossRef] [Medline]
    11. Schmiedhofer MH, Searle J, Slagman A, Möckel M. Exploring patient motives to use emergency departments for non-urgent conditions: A qualitative study [article in German]. Gesundheitswesen 2017 Oct;79(10):835-844. [CrossRef] [Medline]
    12. Hoot NR, Aronsky D. Systematic review of emergency department crowding: Causes, effects, and solutions. Ann Emerg Med 2008 Aug;52(2):126-136 [FREE Full text] [CrossRef] [Medline]
    13. Berchet C. Emergency Care Services: Trends, Drivers and Interventions to Manage the Demand. Paris, France: Organisation for Economic Co-operation and Development (OECD); 2015 Aug 01.   URL: https:/​/www.​​emergency-care-services_5jrts344crns.​pdf?itemId=%2Fcontent%2Fpaper%2F5jrts344crns-en&mimeType=pdf [accessed 2020-08-19]
    14. Carret MLV, Fassa ACG, Domingues MR. Inappropriate use of emergency services: A systematic review of prevalence and associated factors. Cad Saude Publica 2009 Jan;25(1):7-28 [FREE Full text] [CrossRef] [Medline]
    15. Kumle B, Hirschfeld-Warneken A, Darnhofer I, Busch HJ. Telefon-triage und klinische ersteinschätzung in der notfallmedizin zur patientensteuerung [Telephone triage and initial clinical assessment in emergency medicine to manage patient flow]. Notfall Rettungsmed 2019 Aug 08;22(7):568-577. [CrossRef]
    16. Niehues C. Notfallversorgung in Deutschland [book in German]. Stuttgart, Germany: W Kohlhammer; 2012.
    17. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci 2009 Aug 07;4:50 [FREE Full text] [CrossRef] [Medline]
    18. Burnes B. Emergent change and planned change – Competitors or allies? The case of XYZ construction. Int J Oper Prod Manag 2004 Sep;24(9):886-902. [CrossRef]
    19. Grol RPTM, Bosch MC, Hulscher MEJL, Eccles MP, Wensing M. Planning and studying improvement in patient care: The use of theoretical perspectives. Milbank Q 2007;85(1):93-138 [FREE Full text] [CrossRef] [Medline]
    20. Wensing M. Implementation science in healthcare: Introduction and perspective. Z Evid Fortbild Qual Gesundhwes 2015;109(2):97-102. [CrossRef] [Medline]
    21. Glasgow RE, McKay HG, Piette JD, Reynolds KD. The RE-AIM framework for evaluating interventions: What can it tell us about approaches to chronic illness management? Patient Educ Couns 2001 Aug;44(2):119-127. [CrossRef] [Medline]
    22. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [CrossRef] [Medline]
    23. Ritchie J, Spencer L. Qualitative data analysis for applied policy research. In: Huberman AM, Miles MB, editors. The Qualitative Researcher’s Companion. Thousand Oaks, CA: SAGE Publications; 2002:305-329.
    24. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013 Sep 18;13:117 [FREE Full text] [CrossRef] [Medline]
    25. Paulzen M, Gründer G. Psychiatrische notfälle. In: Litmathe J, editor. Neurologische Notfälle: Präklinische und innerklinische Akutversorgung [book in German]. Berlin, Germany: Springer-Verlag; 2016:189-205.
    26. Pajonk F, Messer T. Psychiatrische notfälle. Psychiatrie und Psychotherapie up2date 2009 Jul 1;3(4):257-276. [CrossRef]
    27. Pajonk FG, Gärtner U, Sittinger H, von Knobelsdorff G, Andresen B, Moecke H. Psychiatrische notfälle aus der sicht von rettungsdienstmitarbeitern. Notf Rett Med 2004 May;7(3):161-167. [CrossRef]
    28. Jansen T, de Hoon S, Hek K, Verheij R. Ontwikkelingen op de Huisartsenpost: Veranderingen in Zorgvraag en Gezondheidsproblemen in 2013-2015 [report in Dutch]. Utrecht, the Netherlands: NIVEL; 2017.   URL: [accessed 2020-08-19]
    29. Huibers L, Smits M, Renaud V, Giesen P, Wensing M. Safety of telephone triage in out-of-hours care: A systematic review. Scand J Prim Health Care 2011 Dec;29(4):198-209 [FREE Full text] [CrossRef] [Medline]
    30. van Ierland Y, van Veen M, Huibers L, Giesen P, Moll HA. Validity of telephone and physical triage in emergency care: The Netherlands Triage System. Fam Pract 2011 Jun;28(3):334-341. [CrossRef] [Medline]
    31. Porter A, Dale J, Foster T, Logan P, Wells B, Snooks H. Implementation and use of computerised clinical decision support (CCDS) in emergency pre-hospital care: A qualitative study of paramedic views and experience using Strong Structuration Theory. Implement Sci 2018 Jul 04;13(1):91 [FREE Full text] [CrossRef] [Medline]
    32. Ong RSG, Post J, van Rooij H, de Haan J. Call-duration and triage decisions in out of hours cooperatives with and without the use of an expert system. BMC Fam Pract 2008 Feb 13;9:11 [FREE Full text] [CrossRef] [Medline]
    33. Chang W, Liu H, Goopy S, Chen L, Chen H, Han C. Using the five-level Taiwan Triage and Acuity Scale computerized system: Factors in decision making by emergency department triage nurses. Clin Nurs Res 2017 Oct;26(5):651-666. [CrossRef] [Medline]
    34. Smits M, Hanssen S, Huibers L, Giesen P. Telephone triage in general practices: A written case scenario study in the Netherlands. Scand J Prim Health Care 2016;34(1):28-36 [FREE Full text] [CrossRef] [Medline]
    35. van der Linden MC, Meester BEAM, van der Linden N. Emergency department crowding affects triage processes. Int Emerg Nurs 2016 Nov;29:27-31. [CrossRef] [Medline]
    36. O'Cathain A, Nicholl J, Sampson F, Walters S, McDonnell A, Munro J. Do different types of nurses give different triage decisions in NHS Direct? A mixed methods study. J Health Serv Res Policy 2004 Oct;9(4):226-233. [CrossRef] [Medline]
    37. Noon AJ. The cognitive processes underpinning clinical decision in triage assessment: A theoretical conundrum? Int Emerg Nurs 2014 Jan;22(1):40-46. [CrossRef] [Medline]
    38. Bundestag. Gesetz für schnellere termine und bessere versorgung (Terminservice- und Versorgungsgesetz - TSVG) [article in German]. Bundesgesetzblatt. Bonn, Germany: Bundesanzeiger Verlag; 2019.   URL: https:/​/www.​​xaver/​bgbl/​start.xav?startbk=Bundesanzeiger_BGBl&start=/​/*%5B@attr_id=%27bgbl119s0646.​pdf%27%5D#__bgbl__%2F%2F*%5B%40attr_id%3D%27bgbl119s0646.​pdf%27%5D__1597868722952 [accessed 2020-08-19]


    aQua Institute: Institute for Applied Quality Improvement and Research in Health Care
    CCDS: computerized clinical decision support
    ED: emergency department
    NTS: Netherlands Triage System
    RE-AIM: Reach, Effectiveness and Efficacy, Adoption, Implementation, and Maintenance
    SMASS: Swiss Medical Assessment System
    SmED: Standardisierte medizinische Ersteinschätzung in Deutschland (Standardized medical Initial Assessment in Germany in English)

    Edited by G Eysenbach; submitted 27.02.20; peer-reviewed by R Riessen, C Basch; comments to author 15.04.20; revised version received 16.04.20; accepted 14.05.20; published 04.09.20

    ©Catharina Roth, Amanda Breckner, Jan Paulus, Michel Wensing. Originally published in JMIR Formative Research (, 04.09.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.