This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
Although medical decision-making may be thought of as a task involving health professionals, many decisions, including critical health–related decisions are made by laypersons alone. Specifically, as the first step to most care episodes, it is the patient who determines whether and where to seek health care (triage). Overcautious self-assessments (ie, overtriaging) may lead to overutilization of health care facilities and overcrowded emergency departments, whereas imprudent decisions (ie, undertriaging) constitute a risk to the patient’s health. Recently, patient-facing decision support systems, commonly known as symptom checkers, have been developed to assist laypersons in these decisions.
The purpose of this study is to identify factors influencing laypersons’ ability to self-triage and their risk averseness in self-triage decisions.
We analyzed publicly available data on 91 laypersons appraising 45 short fictitious patient descriptions (case vignettes; N=4095 appraisals). Using signal detection theory and descriptive and inferential statistics, we explored whether the type of medical decision laypersons face, their confidence in their decision, and sociodemographic factors influence their triage accuracy and the type of errors they make. We distinguished between 2 decisions: whether emergency care was required (decision 1) and whether self-care was sufficient (decision 2).
The accuracy of detecting emergencies (decision 1) was higher (mean 82.2%, SD 5.9%) than that of deciding whether any type of medical care is required (decision 2, mean 75.9%, SD 5.25%;
Our study suggests that laypersons are overcautious in deciding whether they require medical care at all, but they miss identifying a considerable portion of emergencies. Our results further indicate that women are more risk averse than men in both types of decisions. Layperson participants made most triage errors when they were certain of their own appraisal. Thus, they might not follow or even seek advice (eg, from symptom checkers) in most instances where advice would be useful.
Increased emergency department (ED) crowding and longer waiting times are associated with higher mortality [
Getting patients to visit the health care facility most suitable for their symptoms can, for example, be achieved by offering phone triage, a telephone hotline patients can call to get a remote urgency assessment of their symptoms. For example, Roivainen et al [
Another solution to disburden health care services lies in empowering patients to self-assess their medical complaints and thereby improving their ability to adequately decide which type of health care facility to visit (ie, self-triage), or where appropriate to stay at home and care for themselves. Since these decisions are made by laypersons instead of medical professionals, they come with various challenges. For example, previous studies indicate that laypersons tend to overtriage [
To address these deficits, decision support systems are designed to aid laypersons in their self-triage decision-making process, for example, kiosks in the ED [
To provide a foundation for tackling these questions, in this study we investigate whether laypersons’ triage accuracy and their risk averseness differ by sociodemographic variables and whether laypersons can potentially gauge whether they require advice in their decision-making, by exploring if laypersons’ confidence in their triage appraisals functions as a reliable predictor of accuracy.
No ethics approval was required for this study. Approval of the original study [
This analysis builds on data collected in a previous study by Schmieding et al [
The layperson sample consisted of 91 US residents without prior professional medical training. They were recruited from Amazon Mechanical Turk (MTurk) in March 2020. Participants were paid US $4 for completing the web-based survey and assessing the 45 case vignettes unaided. As an incentive, a bonus of US $3 was rewarded if they achieved an accuracy above 58%. Compared with the US general population, the layperson sample had a higher level of education (all participants had at least a high school degree) and included a higher proportion of men (55/91, 60.4%) than women (36/91, 39.5%).
A more detailed description of the participants’ characteristics is provided by Schmieding et al [
Schmieding et al [
After every triage assessment, participants were asked how certain they were in their assessment on a 4-point Likert-scale (“Very uncertain,” “Rather uncertain,” “Rather certain,” and “Very certain”). Three sociodemographic variables were surveyed (gender, age, and level of education) and rated on a 5-tiered ordinal scale (“Non-high school graduate,” “High school graduate,” “Some college,” “Bachelor’s degree,” and “Graduate degree”).
We conducted analyses and generated the images using base R 4.0.5 (R Core Team) [
We dichotomized triage levels to explore whether laypersons could reliably distinguish whether emergency care was required or not (decision 1), and whether self-care was sufficient or not (decision 2). Whether and where health care should be sought are the 2 common questions symptom checkers are approached with [
To explore which factors influence laypersons’ risk averseness, triage accuracy, and confidence in their own triage appraisal, we used linear models to quantify relationships between continuous variables (age and risk averseness) and compared proportions between subgroups for ordinal and categorical variables (gender, education, and certainty).
Overtriage errors were defined as appraising the case’s urgency as more urgent than necessary (eg, the participant suggested emergency care when nonemergency care was appropriate) and undertriage errors as judging it as less urgent than required. Risk averseness was defined as the proportion of overtriage errors compared to all errors, whereas a participant’s triage accuracy was calculated as the proportion of correctly solved cases related to all cases.
On average, the participants were able to correctly classify whether a fictitious patient required emergency care or not in about 36 out of 45 cases (mean 82.2%, SD 5.88%; see decision 1 in
Concerning decision 2 (
The difference in accuracy between decision 1 and decision 2 was found to be statistically significant in a post-hoc 2-sided
Accuracy, sensitivity, and specificity for 2 binary triage decisions by participants’ gender.
Participants were generally risk averse [
Age only explained little variance in the decisions made (
Risk averseness by age.
Risk averseness by education.
Risk averseness by gender.
Participants were certain in most of their triage appraisals. They indicated being “very certain” or “rather certain” in about 4 out of 5 (3381/4095, 82.6%) of the triage assessments, see
Triage assessment and accuracy by certainty.
Triage assessment | Degree of certainty | Total | |||
|
“Very uncertain” | “Rather uncertain” | “Rather certain” | “Very certain” |
|
Correct | 24 | 354 | 1145 | 969 | 2492 |
Incorrect | 34 | 302 | 886 | 381 | 1603 |
Total | 58 | 656 | 2031 | 1350 | 4095 |
Accuracy (%) | 41.4 | 53.9 | 56.3 | 71.8 | 60.8 |
Participants’ triage accuracy varied with their degree of certainty, see
When mistaken, participants were on average still commonly certain about their assessment, though less certain than when their appraisal was correct (see
Triage assessment and accuracy by dichotomized certainty levels.
Degree of certainty | Triage assessment | Total | Accuracy (%) | |
|
Correct | Incorrect |
|
|
Uncertain | 378 | 336 | 714 | 52.9 |
Certain | 2114 | 1267 | 3381 | 62.5 |
Total | 3381 | 1603 | 4095 | 60.8 |
Proportion certain (%) | 87 | 79 | N/Aa | N/A |
aN/A: not applicable.
Regarding the 2 types of errors, that is, overtriaging and undertriaging, we observed that participants’ decisional certainty was higher when overtriaging (proportion of overtriage errors where they were either rather or very certain, 783/956, 81.9%) than when undertriaging (484/647, 74.8%). The ratio of overtriage to undertriage errors increased with the level of certainty, from 1.12:1 (18:16) for appraisals where participants were very uncertain to 2:1 (254:127) for appraisals where participants were very certain, see
Triage errors by certainty.
Error type | Degree of certainty, n (%) | Total | |||
“Very uncertain” | “Rather uncertain” | “Rather certain” | “Very certain” |
|
|
Overtriage | 18 (1.9) | 155 (16.2) | 529 (55.3) | 254 (26.6) | 956 |
Undertriage | 16 (2.5) | 147 (22.7) | 357 (55.2) | 127 (19.6) | 647 |
Total | 34 (2.1) | 302 (18.8) | 886 (55.3) | 381 (23.8) | 1603 |
As previously reported [
Concerning decision 1, participants were more prone to undertriage (ie, not identifying emergencies) and made only a few overtriage errors. In decision 2, they mostly made overtriage errors (ie, not recognizing that self-care is sufficient) and only a few undertriage errors. This adds further evidence to previous findings that laypersons tend to overtriage [
Our results indicate that women are more risk averse and overtriage more than men. This finding is in line with a previous study that found women to rate their symptoms as more urgent [
As current symptom checkers used to assist laypersons in their triage decisions are rather risk averse [
Participants’ judgement of their decisional certainty predicted to some extent whether their stand-alone triage assessment was correct: when uncertain, participants were more likely to make incorrect assessments than when being certain. At first glance, this suggests that perceived uncertainty is a good prompt for when to seek decision support. However, whether correct or incorrect, our study participants were certain about most of their judgements. Thus, perceived certainty is not a reliable predictor of triage errors, and based on their level of certainty, it seems likely that laypersons may not be able to correctly determine when they would benefit from a decision aid. Especially as participants’ certainty was greater when overtriaging than when undertriaging, even perfect decision aids might not effectively reduce unnecessary doctor’s visits and disburden health care facilities. This contrasts with the suggestions by Winn et al [
Limitations regarding participants and the evaluation of symptom checker accuracy are reported in detail by Schmieding et al [
Second, although it has been reported that case vignettes are a valid method to assess the health care decision-making of physicians [
Beyond that, there are further limitations specific to the analyses in this study. Our results may have limited external validity: first, the sample was not representative of the US general population; second, triage appraisals might be influenced by the context of the health care system and by recruiting participants online; and third, population groups with no or low (information) technology affinity are not represented. Thus, future studies with more representative panels are required to determine whether our findings hold true for the broader population and whether factors other than gender and perceived certainty influence laypersons’ health care decisions (eg, eHealth literacy [
We applied statistical significance testing sparingly, since this study is a retrospective exploratory analysis and was primarily intended to generate hypotheses that can be tested (experimentally) in future studies to draw inferences.
Methodologically, we considered certainty as a measure of whether participants would consult decision aids and whether they would be open to incorporating the recommendations into their decision-making. This implies that perceived certainty in one’s own appraisal correlates inversely with the openness to follow contradicting advice. However, this relationship is not based on any data. Future studies need to test this assumption.
Our study suggests that laypersons are overcautious in deciding whether they require medical care (decision 2). At the same time, they miss identifying a considerable proportion of emergencies (decision 1). Our results also indicate that women are more risk averse than men in both these decisions. When providing correct advice, decision aids such as symptom checkers could be of benefit to users as they could help reduce the number of missed emergencies and unnecessary visits to low-acuity care facilities. Thus, from a health system’s perspective, decision aids might disburden health care facilities more of low-acuity care than of emergency care. However, layperson participants made most triage errors, and especially overtriage errors, when being certain of their own appraisal. Thus, they might not follow or even seek such advice in most instances where advice would be useful. More studies are needed to better understand laypersons’ ability to self-triage, how this ability could be improved, how decision aids may support laypersons’ medical decision-making, and when laypersons are willing to take advice from decision aids.
emergency department
negative predictive value
positive predictive value
FB reports grants from German Federal Ministry of Education and Research, grants from German Federal Ministry of Health, grants from Berlin Institute of Health, personal fees from Elsevier Publishing, grants from Hans Böckler Foundation, other from Robert Koch Institute, grants from Einstein Foundation, grants from Berlin University Alliance, personal fees from Medtronic, and personal fees from GE Healthcare outside the submitted work.