This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
The digital transformation in health care has been accelerated by the COVID-19 pandemic. Video consultation has become the alternative for hospital consultation. It remains unknown how to select patients suitable for video consultation.
This study aimed to develop a tool based on patient-reported outcomes (PROs) to triage total hip arthroplasty (THA) patients to hospital or video consultation.
A pilot study with expert panels and a retrospective cohort with prospectively collected data from 1228 THA patients was executed. The primary outcome was a PRO triage tool to allocate THA patients to hospital or video consultation 6 weeks postoperatively. Expert panels defined the criteria and selected the patient-reported outcome measure (PROM) questions including thresholds. Data were divided into training and test cohorts. Distribution, floor effect, correlation, responsiveness, PRO patient journey, and homogeneity of the selected questions were investigated in the training cohort. The test cohort was used to provide an unbiased evaluation of the final triage tool.
The expert panels selected moderate or severe pain and using 2 crutches as the triage tool criteria. PROM questions included in the final triage tool were numeric rating scale (NRS) pain during activity, 3-level version of the EuroQol 5 dimensions (EQ-5D-3L) questions 1 and 4, and Oxford Hip Score (OHS) questions 6, 8, and 12. Of the training cohort, 201 (201/703, 28.6%) patients needed a hospital consultation, which was statistically equal to the 150 (150/463, 32.4%) patients in the test cohort who needed a hospital consultation (
A PRO triage tool based on moderate or severe pain and using 2 crutches was developed. Around 70% of THA patients could safely have a video consultation, and 30% needed a hospital consultation 6 weeks postoperatively. This tool is promising for selecting patients for video consultation while using an existing PROM infrastructure.
The digital transformation in health care has been accelerated by the COVID-19 pandemic. Health care institutions are challenged by precautionary measures to contain COVID-19 while continuing to provide health care. Especially, managing the physical flow of patients is challenging. After the pandemic, hospitals will begin to eliminate their waiting lists while maintaining a normal patient flow, and they will be challenged again. As a solution, video consultation has become an alternative to the traditional hospital consultation. The number of hospital consultations has dropped by 30%, and the number of telemedicine visits has increased 5-fold [
Currently, video consultation provides health care institutions and clinicians the opportunity to increase office efficacy and cost-effectiveness in an era of decreasing reimbursements and increasing time constraints [
Orthopedic associations in many countries advise hospitals to collect patient-reported outcomes (PROs) of total hip arthroplasty (THA) using selected patient-reported outcome measures (PROMs) to evaluate health care and improve patient care [
A pilot study with expert panels and a retrospective cohort with prospectively collected data from THA patients was performed. Regarding the cohort, patients were included in this study if they signed the informed consent form preoperatively to allow further scientific analysis using their anonymized data. Therefore, the institutional review board ruled that formal approval was not required for this study. There were no exclusion criteria.
The primary outcome was a PRO triage tool to allocate THA patients to a hospital consultation or a video consultation for their 6-week postoperative consultation. Hospital consultation was defined as needing a physical examination or other examination, such as an X-ray, for which a patient really needed to be in the hospital. If no hospital consultation was needed, patients were allocated to a video consultation. According to the Dutch guidelines, patients should be seen 6 to 12 weeks after a THA [
Measurements were divided into 3 parts: (1) expert panels defined the criteria and selected the PROM questions, including the thresholds; (2) investigation of the clinimetric qualities of selected questions or triage criteria groups in the retrospective cohort with prospectively collected data; and (3) evaluation of the final triage tool.
Two expert panels were created: clinical expert panel and research expert panel. The clinical expert panel consisted of 4 high-volume THA orthopedic surgeons from 2 different health care institutions. The research expert panel consisted of 3 researchers from 2 different health care institutions. As step one, the clinical expert panel defined the clinical triage criteria for the triage tool. These clinical triage criteria were based on the clinical disabilities for which patients needed to have a physical examination or other examination, such as an X-ray. As the second step, based on the clinical triage criteria, the research expert panel selected the appropriate PROM questions, including the thresholds, based on previous studies. As step three, these questions and their thresholds were presented to the clinical expert panel to discuss if these questions and/or thresholds covered the clinical triage criteria. If no threshold was reported in previous studies, the threshold was set using clinical reasoning by the clinical expert panel. These steps resulted in the PROM questions, including the thresholds, that were determined to be clinically relevant for the triage tool.
The retrospective cohort with prospectively collected data consisted of patients who underwent surgery between January 2016 and December 2018 in a medium-sized orthopedic hospital (Kliniek ViaSana, Mill, The Netherlands). Therefore, patients were characterized by an American Society of Anesthesiologists (ASA) score of I-II and BMI ≤35. Four high-volume, experienced orthopedic surgeons performed the primary posterolateral THAs. Length of stay was generally 1 or 2 days.
The data included patient characteristics, PROM response rates, and PROs. Patient characteristics were age on the day of surgery, gender, preoperative BMI, ASA scores, and preoperative Charnley scores collected from the electronic patient records. Response rates were calculated as the number of returned questionnaires that were partially or totally completed divided by the number of THAs minus the number of THAs of patients who were deceased (returned questionnaires / [THAs - THAs of patients who were deceased]) [
Pain at rest and pain during activity were both measured using an NRS question scored from 0 (no pain) to 10 (severe pain). Quality of life was assessed using the EQ-5D-3L questionnaire consisting of 2 parts: EQ visual analogue scale (EQ VAS; 0-100, with 0 as the worst imaginable health state and 100 as the best imaginable health state) and EQ-5D descriptive system existing of 5 questions about 5 dimensions. These 5 dimensions are mobility, self-care, usual activities, pain/discomfort, and anxiety/depression, scored from 1 (no problems) to 3 (extreme problems) [
Regarding the investigation of the clinimetric qualities of selected questions or triage criteria groups by the expert panels, the cohort was divided into 2 groups: training cohort of patients who underwent surgery in 2016 or 2017 and a test cohort of patients who underwent surgery in 2018. To assess which questions were appropriate for the triage tool, the following clinimetric qualities were investigated per selected question in the training cohort: distribution, floor effect, correlation, responsiveness, and PRO patient journey. PRO patient journey was defined as a change in recovery over time. Regarding distribution, if the question did not show any distinction (median and IQR on the same level), the question was found not to be an appropriate question for the triage tool. For floor effect, if more than 15% of the patients scored the worst score [
The final triage tool was applied in the test cohort to provide an unbiased evaluation of the final tool fitted on the training dataset. Results in both cohorts were compared to investigate the hypothesis.
Results are reported as mean (SD), median (IQR), or n (%) based on the test performed. To investigate if there was any difference in patient characteristics, response rates, and preoperative PROs between the training and test cohorts, continuous variables were first checked for a normal distribution. Second, independent
Distribution was investigated with a boxplot distribution, floor effect was determined by calculating the percentage of patients with a minimum score, and correlation was assessed with Spearman correlation analyses. Responsiveness was evaluated by performing Wilcoxon signed rank tests on the change in preoperative and 3-month scores [
Finally, the triage tool was applied for both training and test cohorts. To test the hypothesis, the numbers of hospital and video consultations for both cohorts were compared using Pearson chi-square or Fisher exact tests.
An α of .05 was considered statistically significant. Statistical analyses were performed using SPSS version 25.0 (IBM Corp, Armonk, NY).
“Having moderate or severe pain” and “using 2 crutches” were defined as the triage criteria by the clinical expert panel (step 1). For the criterion of “having moderate or severe pain,” the research expert panel selected the following PROM questions: NRS pain at rest, NRS pain during activities, EQ-5D-3L question 4, and OHS questions 1, 8, 10, and 12. For both NRS pain questions, previous studies reported thresholds of ≤3 for no or mild pain and >3 for moderate to severe pain [
Triage criteria, selected clinically relevant questions, and defined thresholds by the expert panels.
Triage criterion and selected PROMa question | PROM question (score range) | Defined threshold | |||
|
|||||
|
NRSb pain at rest | How much pain from your hip (surgery side) did you experience at rest in the last week? (0-10) | ≥4 | ||
|
NRS pain during activity | How much pain from your hip (surgery side) did you experience during activity in the last week? (0-10) | ≥4 | ||
|
EQ-5D-3Lc question 4 | Pain/discomfort (1-3) | ≥3 (extreme pain) | ||
|
OHSd question 1 | During the past 4 weeks...How would you describe the pain you usually had from your hip? (0-4) | ≤1 (moderate or severe) | ||
|
OHS question 8 | During the past 4 weeks...After a meal (sitting at a table), how painful has it been for you to stand up from a chair because of your hip? (0-4) | ≤2 (moderate, very, unbearable) | ||
|
OHS question 10 | During the past 4 weeks...Have you had any sudden, severe pain — “shooting,” “stabbing,” or “spasms” — from the affected hip? (0-4) | ≤1 (most or every) | ||
|
OHS question 12 | During the past 4 weeks...Have you been troubled by pain from your hip in bed at night? (0-4) | ≤2 (3 or 4, 5 or 6, all) | ||
|
|||||
|
EQ-5D-3L question 1 | Mobility (1-3) | ≥3 (confined to bed) | ||
|
OHS question 6 | During the past 4 weeks...For how long have you been able to walk before pain from your hip becomes severe (with or without a stick)? (0-4) | ≤2 (5-15 minutes, around the house only, not at all) |
aPROM: patient-reported outcome measure.
bNRS: numeric rating scale.
cEQ-5D-3L: 3-level version of the EuroQol 5 dimensions.
dOHS: Oxford Hip Score.
Response rates were statistically significantly equal between both training (n=746) and test (n=482) cohorts preoperatively (745/746, 99.9% versus 482/482, 100%;
Characteristics of training and test cohorts.
Characteristics | Training cohort (n=746) | Test cohort (n=482) | ||||||
|
|
|
|
|||||
|
Preoperative | 745 (99.9) | 482 (100) | .99 | ||||
|
3 months postoperative | 703 (94.2) | 463 (96.1) | .24 | ||||
|
12 months postoperative | 693 (92.9) | 457 (94.8) | .29 | ||||
|
|
|
|
|||||
|
Age (years), median (IQR) | 66.5 (61.0-72.0) | 66.0 (60.0-72.0) | .73 | ||||
|
Gender (male), n (%) | 295 (39.5) | 206 (42.7) | .27 | ||||
|
BMI (kg/m2), median (IQR) | 26.1 (24.1-28.4) | 26.3 (24.1-28.5) | .48 | ||||
|
ASAa score – I, n (%) | 399 (53.5) | 287 (59.5) | .04 | ||||
|
|
|
|
.048 | ||||
|
|
One hip affected by OAb, n (%) | 163 (21.8) | 108 (22.4) | ||||
|
|
Both hips affected by OA, n (%) | 309 (41.4) | 196 (40.7) | ||||
|
|
Contralateral hip OA, n (%) | 163 (21.8) | 82 (17.0) | ||||
|
|
Multiple joints affected by OA, n (%) | 111 (14.9) | 96 (19.9) | ||||
|
|
|
|
|||||
NRSd pain at rest score | 6.0 (4.0-7.0) | 6.0 (3.8-7.0) | .18 | |||||
|
NRS pain during activity score | 8.0 (7.0-9.0) | 8.0 (7.0-8.0) | .13 | ||||
|
HOOS-PSe score | 46.1 (37.7-55.9) | 46.1 (33.9-55.9) | .01 | ||||
|
EQ-5D descriptive systemf | 0.693 (0.298-0.775) | 0.693 (0.569-0.775) | .16 | ||||
|
EQ VASg | 76.0 (63.3-89.8) | 77.0 (60.0-86.3) | .82 | ||||
|
OHSh | 24.0 (19.0-29.0) | 25.0 (19.0-31.0) | .03 |
aASA: American Society of Anesthesiologists.
bOA: osteoarthritis.
cPROs: patient-reported outcomes.
dNRS: numeric rating scale.
eHOOS-PS: Hip disability and Osteoarthritis Outcome Score — Physical Function Shortform.
fEQ-5D descriptive system: EuroQol 5 dimensions descriptive system.
gEQ VAS: EuroQol visual analogue scale.
hOHS: Oxford Hip Score.
Regarding the questions or triage criteria groups selected by the expert panels (
Distribution, floor effect, and correlation per selected patient-reported outcome measure (PROM) question.
PROMa question | Distribution, median (IQR) | Floor effect, n (%) | Correlations | ||
|
|
|
Correlations ( |
Correlated question | |
NRSc pain at rest | 0 (0-1) | 3d (0.4) | 0.659 | NRS pain during activity | <.001 |
NRS pain during activity | 2 (0-3) | 4d (0.6) | 0.659; –0.676 | NRS pain at rest; OHSe question 1 | Both <.001 |
EQ-5D-3Lf question 4 | 1 (1-2) | 13g (1.9) | None | N/Ah | N/A |
OHS question 1 | 3 (3-4) | 6g (0.9) | –0.675 | NRS pain during activity | <.001 |
OHS question 8 | 3 (3-4) | 0g (0.0) | None | N/A | N/A |
OHS question 10 | 4 (4-4) | 2i (0.3) | None | N/A | N/A |
OHS question 12 | 4 (3-4) | 32i (4.6) | None | N/A | N/A |
EQ-5D-3L question 1 | 1 (1-2) | 2i (0.3) | None | N/A | N/A |
OHS question 6 | 4 (3-4) | 3j (0.4) | None | N/A | N/A |
aPROM: patient-reported outcome measure.
bStatistically significant correlations >0.6 or <–0.6 are presented.
cNRS: numeric rating scale.
dn=703.
eOHS: Oxford Hip Score.
fEQ-5D-3L: 3-level version of the EuroQol 5 dimensions.
gn=693.
hN/A: not applicable.
in=694.
jn=690.
Responsiveness for each patient-reported outcome measure (PROM) question.
PROM question | Preoperative, median (IQR) | 3 months postoperative, median (IQR) | |
NRSa pain at rest | 6 (4-7) | 0 (0-1) | <.001 |
NRS pain during activity | 8 (7-9) | 2 (0-3) | <.001 |
EQ-5D-3Lb question 4 | 2 (2-3) | 1 (1-2) | <.001 |
OHSc question 1 | 1 (0-1) | 3 (3-4) | <.001 |
OHS question 8 | 2 (2-3) | 3 (3-4) | <.001 |
OHS question 10 | 2 (1-3) | 4 (4-4) | <.001 |
OHS question 12 | 2 (0-3) | 4 (3-4) | <.001 |
EQ-5D-3L question 1 | 2 (2-2) | 1 (1-2) | <.001 |
OHS question 6 | 2 (2-3) | 4 (3-4) | <.001 |
aNRS: numeric rating scale.
bEQ-5D-3L: 3-level version of the EuroQol 5 dimensions.
cOHS: Oxford Hip Score.
Patient journey per patient-reported outcome measure (PROM) question.
PROM question | Defined threshold | 12-month score of patients with a score below the threshold at 3 months, median (IQR) | 12-month score of patients with a score above the threshold at 3 months, median (IQR) |
NRSa pain at rest | ≥4 | 0 (0-1) | 2 (0-5) |
NRS pain during activity | ≥4 | 0 (0-1) | 2 (0-4) |
EQ-5D-3Lb question 4 | ≥3 | 1 (1-2) | 2 (1.5-2) |
OHSc question 1 | ≤1 | 3 (2-4) | 4 (3-4) |
OHS question 8 | ≤2 | 3 (3-4) | 4 (4-4) |
OHS question 10 | ≤1 | 4 (2.5-3) | 4 (4-4) |
OHS question 12 | ≤2 | 3 (1.5-4) | 4 (4-4) |
EQ-5D-3L question 1 | ≥3 | 1 (1-2) | 1 (1-1)d |
OHS question 6 | ≤2 | 3 (2-4) | 4 (4-4) |
aNRS: numeric rating scale.
bEQ-5D-3L: 3-level version of the EuroQol 5 dimensions.
cOHS: Oxford Hip Score.
dn=1.
Based on the clinimetric qualities of selected questions or triage criteria groups, NRS pain at rest, OHS question 1, and OHS question 10 were removed from the triage tool. The final triage tool consisted of NRS pain during activity; EQ-5D-3L questions 1 and 4; and OHS questions 6, 8, and 12.
The final triage tool resulted in 201 (201/703, 28.6%) patients in the training cohort needing a hospital consultation, which was statistically equal to the 150 (150/463, 32.4%) patients in the test cohort who needed a hospital consultation (
This study aimed to develop a tool based on PROs collected using an existing PROM infrastructure to triage THA patients to hospital or video consultation 6 weeks postoperatively. As the main finding, a triage tool based on PROM questions measuring moderate or severe pain and whether the patient used 2 crutches was developed. The included questions were NRS pain during activity; EQ-5D-3L questions 1 and 4; and OHS questions 6, 8, and 12. Applying the final triage tool in both the training and test cohorts resulted in the same outcome: Around 70% of the patients could safely have a video consultation, and 30% needed to have a hospital consultation 6 weeks postoperatively. Therefore, this PRO triage tool is a promising instrument to select patients for video consultation while using an existing PROM infrastructure. The next step is to further investigate this triage tool in daily practice.
This study showed that 70% of the hospital consultations for THA patients 6 weeks postoperatively could safely be done by video. It was hypothesized that 10% of the THA patients would need a hospital consultation. First, the result that 30% of patients needed a hospital consultation could be explained by the focus of the clinical expert panel. As the experts’ beginning point was seeing all patients during a hospital consultation (100%), by developing the triage tool, they desired to see all patients who potentially needed a physical examination or other examination, such as an X-ray, during a hospital consultation. Furthermore, they desired to prevent obtaining more consultations by needing to schedule a hospital consultation after a video consultation. Both implicitly resulted in more liberal criteria for a hospital consultation leading to more patients triaged to a hospital consultation. Second, it could be that specific questions are missing from the triage tool. It was hypothesized that, after an investigation of the triage tool in daily practice, the criteria for the triage tool could be improved, achieving the right health care for each patient and a further reduction in hospital consultations. It would be interesting to investigate how many additional hospital consultations would be needed if the tool triages to video consultation and which PROs are different for patients with an additional hospital consultation.
It is essential to understand that the PRO triage tool is not a tool on its own, but it is the first step in the selection of patients who need a hospital consultation and those who can have a video consultation instead. PROs and clinical judgment produce complementary data and when combined, provide a more accurate description of the patients’ symptoms [
Previous studies reported that patients rate their video consultations as excellent or very good (92%-95%) [
As a first strength of this study, to the authors’ knowledge, this is the first study in which a tool to triage patients to hospital or video consultation was developed. Second, high response rates preoperatively and even postoperatively (above 90%) were achieved, resulting in a representative cohort to execute this study and to generalize the results to the total THA population. A third strength is the application of the training and test cohorts to provide an unbiased evaluation of the final tool.
As a limitation of this study, the triage tool was not investigated in a prospective cohort, and aspects of reliability, validity, sensitivity, and specificity of the triage tool were not investigated yet. Furthermore, 3-month PROs were used instead of 6-week PROs, as, although based on previous studies, the assumption was made that there is limited clinically relevant difference between PROs at 6 weeks and at 3 months postoperatively [
In conclusion, a THA PRO triage tool based on moderate or severe pain and using 2 crutches was developed. Around 70% of THA patients could safely have a video consultation, and 30% of patients needed a hospital consultation 6 weeks postoperatively. This tool is promising for selecting patients for video consultation while using an existing PROM infrastructure.
American Society of Anesthesiologists
EuroQol visual analogue scale
EuroQol 5 dimensions descriptive system
3-level version of the EuroQol 5 dimensions
Hip disability and Osteoarthritis Outcome Score – Physical Function Shortform
numeric rating scale
Oxford Hip Score
patient-reported outcome measure(s)
patient-reported outcome(s)
total hip arthroplasty
We would like to thank Klaartje van Diepen-Pijnappels and Maud Peters for their consistent data collection and kindness helping all patients who had questions, all patients for completing their PROMs, and the orthopedic surgeons for participating in the clinical expert panel.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sector.
YP was responsible for designing the study, data collection, statistical analysis, and study coordination. All authors were involved in drafting and revising the manuscript and approved the final version of the manuscript.
PP was a paid employee of ZimmerBiomet, and WW is a consultant for ZimmerBiomet, which have nonfinancial associations that may be relevant to the submitted manuscript. BWS receives a grant for providing an educational Stryker course on the Exeter prosthesis and is a board member of the European Hip Society, which has nonfinancial associations that may be relevant to the submitted manuscript. The other authors declare that they have no conflict of interest.