Published on in Vol 4, No 5 (2020): May

Preprints (earlier versions) of this paper are available at, first published .
Reliability, Feasibility, and Patient Acceptance of an Electronic Version of a Multidimensional Health Assessment Questionnaire for Routine Rheumatology Care: Validation and Patient Preference Study

Reliability, Feasibility, and Patient Acceptance of an Electronic Version of a Multidimensional Health Assessment Questionnaire for Routine Rheumatology Care: Validation and Patient Preference Study

Reliability, Feasibility, and Patient Acceptance of an Electronic Version of a Multidimensional Health Assessment Questionnaire for Routine Rheumatology Care: Validation and Patient Preference Study

Original Paper

1Rush University Medical Center, Chicago, IL, United States

2Department of Rheumatology, General University Hospital Gregorio Marañón, Madrid, Spain

3ZiteLab ApS, Copenhagen, Denmark

Corresponding Author:

Theodore Pincus, MD

Rush University Medical Center

1611 West Harrison, Suite 510

Chicago, IL

United States

Phone: 1 3129428268


Background: A multidimensional health assessment questionnaire (MDHAQ) that was developed primarily for routine rheumatology care has advanced clinical research concerning disease burden, disability, and mortality in rheumatic diseases. Routine Assessment of Patient Index Data 3 (RAPID3), an index within the MDHAQ, is the most widely used index to assess rheumatoid arthritis (RA) in clinical care in the United States, and it recognizes clinical status changes in all studied rheumatic diseases. MDHAQ physical function scores are far more significant in the prognosis of premature RA mortality than laboratory or imaging data. However, electronic medical records (EMRs) generally do not include patient questionnaires. An electronic MDHAQ (eMDHAQ), linked by fast healthcare interoperability resources (FIHR) to an EMR, can facilitate clinical and research advances.

Objective: This study analyzed the reliability, feasibility, and patient acceptance of an eMDHAQ.

Methods: Since 2006, all Rush University Medical Center rheumatology patients with all diagnoses have been asked to complete a paper MDHAQ at each routine care encounter. In April 2019, patients were invited to complete an eMDHAQ at the conclusion of the encounter. Analyses were conducted to determine the reliability of eMDHAQ versus paper MDHAQ scores, arithmetically and by intraclass correlation coefficient (ICC). The feasibility of the eMDHAQ was analyzed based on the time for patient completion. The patient preference for the electronic or paper version was analyzed through a patient paper questionnaire.

Results: The 98 study patients were a typical routine rheumatology patient group. Seven paper versus eMDHAQ scores were within 2%, differences neither clinically nor statistically significant. ICCs of 0.86-0.98 also indicated good to excellent reliability. Mean eMDHAQ completion time was a feasible 8.2 minutes. The eMDHAQ was preferred by 72% of patients; preferences were similar according to age and educational level.

Conclusions: The results on a paper MDHAQ versus eMDHAQ were similar. Most patients preferred an eMDHAQ.

JMIR Form Res 2020;4(5):e15815



A multidimensional health assessment questionnaire (MDHAQ) [1-3] has been completed by all patients at all visits to one of the authors (TP) since 1982, based on initial evidence that patient questionnaire scores for physical function are significant in the prognosis of work disability and premature death in rheumatoid arthritis (RA) [4]. Paper MDHAQ data for all patients were entered into databases (Paradox, Access) in the 1990s to provide serial flowsheets of patient scores, laboratory data, and medications, which informed clinical decisions [5]. Clinical research using MDHAQ databases has advanced knowledge concerning the prognosis of mortality [6-8], importance of socioeconomic status [7,9,10], value of methotrexate [11,12], value of low-dose glucocorticoids [13], and recognition of depression [1,14] in RA and other rheumatic diseases.

The MDHAQ is completed by all patients at several rheumatology sites, including New York University Hospital for Joint Diseases (since 2005), a private practice in Ridley Park, PA (since 2006), Rush University Medical Center (since 2006), and Liverpool Hospital in Australia (since 2013) [15]. However, the patients continue to complete a paper MDHAQ despite the use of electronic medical records (EMRs) at each of these sites, as the EMR generally has not included patient questionnaires. Databases of paper MDHAQ data from routine care have been entered into electronic research databases to recognize that disease burden in osteoarthritis is similar to RA [15], and to develop indices such as routine assessment of patient index data 3 (RAPID3) [16-18] and a fibromyalgia assessment screening tool (FAST3). RAPID3 is the most widely used RA index in the United States [19,20], and comparable to disease-specific indices to recognize changes in clinical status in patients with all other rheumatic diseases which have been studied [21-23].

An electronic MDHAQ (eMDHAQ), linked to an EMR through fast healthcare interoperability resources (FHIR), could enhance clinical care with serial flowsheets, enable remote patient completion before or between scheduled visits to report problems, and reduce costs and errors in retrospective data entry of paper versions for clinical research. This report presents analyses of the reliability, feasibility, and patient acceptance of an eMDHAQ.

Ethics and Consent

The Rush University Institutional Review Board waived the requirement for patient consent in the completion of patient questionnaires, as the questionnaire is a component of routine care, analogous to a laboratory test, that provides quantitative quality measures to guide clinical decisions. The database used in this study is part of the Rush University Patient-Reported Outcomes Studies approved by the Rush University Institutional Review Board, with a waiver for patient consent for retrospective data analysis (14090502-IRB02-AM03). The eMDHAQ is regarded as an extension of efforts to implement quality measures.


Since 2006, all patients with all diagnoses seen by all clinical rheumatologists at Rush University Medical Center (14 in 2019) have been asked to complete a paper MDHAQ at each visit to provide quality measures in routine care [24]. Since the introduction of the Epic EMR at Rush in 2011, all completed MDHAQs have been scanned into the EMR as PDFs, incorporated into each patient’s encounter record.

In April 2019, all rheumatologists were asked to request that patients older than 18 years complete an eMDHAQ on an iPad at the conclusion of the encounter, indicating that the patient could decline for any reason. This study was conducted during routine care, as has been the case in all development of the MDHAQ/RAPID3 other than retrospective analyses of clinical trial results to compare RAPID3 to traditional RA indices [24]. Clinicians were not asked to collect formal records of how many patients were asked to volunteer or how many refused. An informal query indicated that most patients declined because of a need to leave the clinic.


The paper MDHAQ was developed over 25 years as a 2-page patient self-report questionnaire, completed in 5-10 minutes, although no formal studies of time for completion have been reported. The MDHAQ includes patient self-report quantitative scores for physical function (FN), 3 visual numeric scales (VNS) for pain, patient global assessment (PATGL), and fatigue, a self-report painful joint count, termed the rheumatoid arthritis disease activity index (RADAI) [25], which is informative in many rheumatic diseases [26], a 60-symptom checklist [27,28], exercise status [29], morning stiffness, and change in status [27]. The MDHAQ queries recent patient medical history information, including possible surgery, hospitalizations, new medications, adverse medication events, changes in medications, and demographic data such as gender, ethnicity, and years of education. A long version of the MDHAQ (termed "4 page version" in paper format) for new patients includes past medical history, illnesses, allergies, family history, social history, comparable to a standard “intake” questionnaire for new patients [24].

MDHAQ scores have been developed into 4 indices. RAPID3 includes 3 scores of 0-10 within the MDHAQ for physical function, pain VNS, and PATGL VNS, which are compiled into a score of 0-30 [16,23,30,31]. FAST3 is a 0-3 cumulative index based on RADAI ≥16 (=1), symptom checklist ≥16 (=1), and pain and/or fatigue VNS ≥6 (=1) [32,33]; a score ≥2/3 for FAST3 agrees more than 80% with the polysymptomatic distress scale, which is the basis for the 2011 revised formal fibromyalgia criteria [32,33]. PSYCH3 (Psychological Index 3) includes queries for sleep quality, anxiety, and depression [1,14]; as a screening tool, it shows good agreement with the Centers for Epidemiologic Studies Depression Scale [1]. MEDI60 is based on the symptom checklist and has been used for remote electronic monitoring of adverse events and patient status without face-to-face patient visits [28].


As noted above, all completed MDHAQs have been scanned as PDFs into the Epic EMR since its introduction at Rush University Medical Center in 2011 as a quality component of the encounter record. A proposed eMDHAQ/RAPID3 presented by Epic in 2015 lacked flowsheets to depict serially patient scores, reports of specific new symptoms on the MDHAQ checklist to facilitate a physician’s review of systems, and other features that had enhanced the efficiency of patient care in the pencil-and-paper data entry versions as early as the 1990s and early 2000s [24]. As features beyond straightforward scores could not be made available, an electronic MDHAQ was developed in 2015-16 with the FHIR interface to be compatible with the Epic EMR, which is termed ClinDat.

The ClinDat software is managed by ZiteLab, a Copenhagen information technology company that has managed a Danish rheumatology registry called DANBIO (initially designated “Danish Biologics Registry”) since 2002. DANBIO currently includes self-report data from more than 50,000 patients; these data have been analyzed in more than 200 published reports [34,35]. The ClinDat software was designed to be compliant with the Health Insurance Portability and Accountability Act (HIPAA), with direct patient entry of MDHAQ responses linked to an EMR through FHIR. Database management includes flowsheets depicting MDHAQ scores, laboratory test results and medications, and possible automated encounter reports, as was previously available with paper entry in the 1990s [5].

Relatively early adoption of ClinDat for routine patient care was anticipated; however, administrative delays have persisted to date. Therefore, in 2016, a decision was made to ask a research assistant or associate to enter selected scanned MDHAQs into ClinDat for specific research protocols. Protected health information, including name, date of birth, and medical record number, has not been entered into ClinDat pending approval by Rush University. ClinDat assigns a unique identifier to each patient; the unique ClinDat number is linked to protected health information in a local Excel spreadsheet on the Rush University server.

Research studies based on retrospective analysis of the ClinDat database were conducted at Rush University from 2016-2019. In one such study, it was found that according to RAPID3 and other MDHAQ scores, the disease burden in patients with osteoarthritis is similar to and often greater than that seen in patients with RA, contrary to traditional paradigms [15,36]. Discordance of global assessments by patients and physicians was found to be significantly associated with female gender, low socioeconomic status, and high pain scores [37]. MDHAQ/FAST3 was found to be in 80% agreement with formal fibromyalgia criteria [33]. The MDHAQ/MEDI60 symptom checklist has been used in remote electronic monitoring to recognize adverse events and their resolution with clinical improvement [28].

The content of the eMDHAQ is identical to that of the paper version; it is presented on 6 screens to minimize scrolling. For the study described in this report, the ClinDat unique identifier of each volunteer (or a new number for a few “new” patients) was entered into the tablet by office staff before the tablet was given to the patient. The patient then completed all eMDHAQ items on the tablet without any further interaction with the staff member before returning the tablet to the staff member.

The patient was then asked to complete a brief paper self-report questionnaire with 3 items. The first 2 items were VNS queries which were identical to those for pain, global assessment, and fatigue on the MDHAQ: A. “How helpful do you feel the questionnaire is to you to help communicate with your doctor?” B. “How helpful do you feel the questionnaire is to your doctor to help communicate with you?” The anchors were “0=not helpful at all” and “10=very helpful.” The third item was a simple query: “Which version do you prefer?” with 3 response options: "computer," "paper," and "doesn’t matter."

The comparison of the eMDHAQ to the paper version is regarded as a quality improvement project and is exempt from patient consent by the Rush University Institutional Review Board. The paper MDHAQ was entered into the ClinDat database at a later time.

Data Analysis

Descriptive statistics were calculated for mean, range, and SD or proportion of patients according to demographic data and diagnosis. The paper and electronic MDHAQ scores were compared for reliability using paired t-tests for continuous variables and the McNemar test for binary variables. Reliability was also examined according to the intraclass correlation coefficient (ICC) with 95% CI; values less than 0.5 indicate poor reliability, 0.5-0.75 indicate moderate reliability, 0.75-0.9 indicate good reliability, and >0.90 indicate excellent reliability [38]. Feasibility was analyzed as the time to complete the eMDHAQ. Patient preference was analyzed according to the proportion of patients who responded that they preferred the eMDHAQ or paper MDHAQ or had no preference; the proportions were calculated for all patients, and according to age (≤65, >65 years) and formal education (<12, 12, or >12 years). The level of statistical significance was set as P<.05. All analyses were conducted using STATA 12.0 for Mac (StataCorp LP).

Among the 98 patients in the study, the mean age was 53.8 years (range 21.0-88.0, SD 16.6); 87 (89%) were female, 47 (48%) were white, 26 (27%) were black, and 25 (25%) were members of other ethnic groups. The patients had various ICD-10 diagnoses, which were assigned by the treating rheumatologist (Table 1). These patients appear to represent a typical cross-sectional group of patients seen in an academic rheumatology setting according to age, gender, and diagnosis.

The electronic and paper MDHAQ mean scores (SD) were almost identical (Table 2).

Differences ranged from –0.4 to 0.7 and were all within 2% of one another; no scores differed significantly, either clinically or statistically. The ICC for the symptom checklist was 0.86, indicating good reliability, and all other ICCs were greater than 0.92, indicating excellent reliability (Table 2).

None of the volunteer patients reported difficulties using the iPad. The mean time to complete the 6-screen eMDHAQ was 8.2 minutes, indicating good feasibility. For the patient self-report paper questionnaire, the mean VNS rating (0-10, where 10=very helpful) of how helpful the MDHAQ was to the patient was 8.8 (SD 1.7); the mean rating of how helpful the MDHAQ was to the physician was 8.7 (SD 1.9). Patient preferences were 72% for the electronic version versus 7% for the paper version, while 21% noted no preference. Preferences did not differ meaningfully according to age or level of education (Table 3).

Table 1. Demographic characteristics and diagnoses of the patients included in the study (N=98).
Age (years), mean (range, SD)53.8 (21.0-88.0, 16.6)
Female gender, n (%)87 (89)
Ethnicity, n (%)

White47 (48)

Black26 (27)

Other25 (25)
Education level (years), mean (range)14.6 (4-20)
Diagnosis, n (%)

Rheumatoid Arthritis18 (18)

Osteoarthritis15 (15)

Systemic lupus erythematosus11 (11)

Osteoporosis6 (6)

Fibromyalgia5 (5)

Spondyloarthropathies3 (3)

Vasculitis3 (3)

Other37 (38)
Table 2. Scores and test-retest reliability of patient-reported measures on the MDHAQ in paper versus electronic format for patients seen in routine care (N=98).
Paper iPad Diff. (95% CI) ICC (95% CI) Paper MDHAQa score, mean (SD)Electronic MDHAQ score, mean (SD)Difference (95% CI)ICCb (95% CI)
Physical function (0-10)1.8 (1.6)1.8 (1.6)0.003 (–0.4 to 0.5)0.97 (0.97 to 0.98)
Pain VNSc (0-10)4.7 (3.1)4.9 (3.2)–0.1 (–1.0 to 0.7)0.95 (0.92 to 0.97)
PATGLd VNS (0-10)4.2 (2.7)4.4 (2.8)–0.2 (–1.0 to –0.6)0.96 (0.95 to 0.98)
Fatigue VNS (0-10)3.3 (3.0)3.5 (3.1)–0.1 (–1.0 to 0.7)0.95 (0.93 to 0.97)
RAPID3e (0-30)10.8 (7.0)11.2 (6.9)–0.4 (–2.3 to 1.6)0.98 (0.97 to 0.99)
60- Symptom checklist (0-60)9.9 (8.8)9.7 (8.6)0.3 (–2.1 to 2.7)0.86 (0.79 to 0.91)
Self-report RADAIf painful joint count (0-48)10.5 (10.1)9.7 (9.4)0.7 (–2.0 to 3.5)0.92 (0.88 to 0.95)

aMDHAQ: multidimensional health assessment questionnaire.

bICC: intraclass correlation coefficient.

cVNS: visual numeric scale.

dPATGL: patient global estimate.

eRAPID3: Routine Assessment of Patient Index Data 3.

fRADAI: Rheumatoid Arthritis Disease Activity Index.

Table 3. Preferred version of the MDHAQ according to patient age and education level, n (%).
Patient demographicPaper MDHAQaeMDHAQNo preference
Age (years)

≤65 (n=69)5 (7)50 (72)15 (21)

>65 (n=28)2 (7)19 (71)6 (22)
Education (years)

<12 (n=9)1 (11)7 (78)1 (11)

12 (n=18)1 (6)13 (72)4 (22)

>12 (n=68)4 (6)48 (70)16 (24)
Total (N=95)6 (6)68 (72)21 (21)

aMDHAQ: multidimensional health assessment questionnaire.

The eMDHAQ performed similarly to the traditional paper MDHAQ. All ICCs were greater than 0.86, indicating good to excellent reliability of the 2 versions; these values are as high as those seen in most comparisons in clinical medicine. For example, an earlier report compared electronic and paper versions of RA core data set measures [38,39], indicating ICCs for a swollen joint count of 0.78 and for a tender joint count of 0.83; ICCs for RA indices which include a formal joint count were 0.85 for Disease Activity Score 28 and 0.89 for the Clinical Disease Activity Index. In an earlier study, the ICC for the self-report physical function was 0.96, that for pain was 0.88, that for patient global assessment was 0.78, and that for RAPID3 was 0.90 [38], compared to 0.97, 0.95, and 0.96, and 0.98, respectively, in the present study (Table 2). The higher ICCs in the present study may be explained in part by the 5-7 day interval between measures in the earlier study compared to 1-2 hours in the present study; also, patients seen at Rush University Medical Center have extensive experience with the MDHAQ. The previous data are presented to document that the ICCs of the measures reported by patients were somewhat higher than those obtained by physicians.

The mean time required to complete the eMDHAQ was 8.2 minutes, which appears to be acceptable. Formal studies have not been reported for the time to complete a paper MDHAQ, although informal observations over more than 30 years suggest that 5-10 minutes are required. In general, <10 minutes appears to be acceptable, although some patients likely will require more time, as is the case with the paper version.

Approximately two-thirds of patients expressed a preference for the electronic version of the MDHAQ. However, no specific information was collected concerning the number of patients who declined to complete an eMDHAQ in the routine care setting. It is reassuring that no meaningful differences in preference or reliability were seen according to age or education level, although it is anticipated that problems with completion of an eMDHAQ will be more likely in older or less educated patients, as has been seen with the paper MDHAQ; thus, differences may emerge with larger numbers of patients. However, several reports indicating poor clinical status associated with low formal education level [7,9,40] required completion of a patient questionnaire by the patients analyzed in the studies. A small fraction of patients preferred the paper version, and it is anticipated that a paper version will be offered to some patients even after the eMDHAQ is incorporated into routine care.

Clinical decisions in rheumatology patients are based more on information from a patient history than in many chronic diseases in which decisions are dominated by a “gold standard” biomarker, such as blood pressure in hypertension or serum glucose in diabetes [41]. A patient self-report questionnaire depicts components of a “subjective” [42] patient medical history as structured, quantitative, standard, protocol-driven, data which meet criteria for the scientific method [43,44]. Physical function reported on a patient questionnaire is far more significant in the prognosis of premature mortality in RA than any laboratory or imaging data [4,6,45] and is as significant as smoking to predict mortality in a nondiseased elderly population [46].

The value of RAPID3 in rheumatology care [19,20,22] is attributable in part to its capacity to depict change in clinical status in all rheumatic diseases studied to date [21,22], while the patient does almost all the work. Nonetheless, availability of additional MDHAQ scales provide a clinician with considerably more information than only RAPID3, for fatigue [47], RADAI self-report painful joint count [26], adverse events of medications and their resolution [28]. Traditional medical history queries on the MDHAQ save time for patients and doctors [31].

Quantitative RAPID3 scores are highest in patients with fibromyalgia [48], which is seen as a comorbid condition in 20–40% of patients with RA, OA, and many rheumatic diagnoses [49-51]. Clinical improvement is far less likely in patients with comorbid fibromyalgia with any diagnosis than in patients who have this diagnosis and no evidence of fibromyalgia. A further MDHAQ index, FAST3 (fibromyalgia assessment screening tool) may be used to screen for fibromyalgia [32,33], and potentially explain unchanged RAPID3 scores in RA patients with comorbid (secondary) fibromyalgia. FAST3 includes the RADAI self-report painful joint count and symptom checklist [32,33], and therefore is not available when only RAPID3 is queried.

The eMDHAQ was developed using FHIR as an internal format to save data in a Zope object database, with capacity to interface with any EMR in the management of individual patients. Further description of such integration is beyond the scope of the present study, which was focused on reliability of the eMDHAQ and its acceptability to patients. Establishment of these features is regarded as an important future stage in development of an eMDHAQ for implementation in routine clinical care.

This study has several important limitations. First, all patients completed the paper MDHAQ as a usual component of routine care before seeing the rheumatologist, and they completed the eMDHAQ at the conclusion of the visit. Ideally, half might have completed the eMDHAQ initially in a formal study, but such a study would have presented increased costs, logistic complexities, and an extra burden to patients in a setting at which all patients complete an MDHAQ routinely. Second, patient recall of previous completion of the MDHAQ approximately 30-90 minutes earlier could have influenced the second recording. Third, neither the number of patients who declined to participate nor the reasons for declining were recorded in a routine care setting; in general, clinicians reported informally that patients who declined participation noted other scheduled activities, but some patients may have wished to avoid the computer version. Fourth, patient preference for the electronic version may be attributed in part to a “Hawthorne effect,” with further attention from a research professional and a new approach. At the same time, many patients had completed many paper MDHAQs previously, and a bias to favor the familiar paper version could have been present. The eMDHAQ appeared acceptable to most patients, although provision for a paper version for some patients is anticipated in clinical implementation of an eMDHAQ for routine care.

The analyses reported here focused on cross-sectional reliability, which is regarded by institutional information technology professionals and developers as a prerequisite for further work toward use of an eMDHAQ in routine care. Several adjustments to the workflow will be needed to implement an eMDHAQ in routine care; these details remain to be addressed in the next phases of development. Further programming and collaboration with EMR vendors for exchange of eMDHAQ data with the EMR, using FHIR available within the software, is anticipated.

An eMDHAQ presents advantages to allow completion at home before scheduled visits, rather than in the waiting area, and remotely between visits to document status when patients report disease flares or adverse medication events. Data entered by patients can be transferred to the hospital EMR using FHIR and can be made available on the physician’s computer screen. The ClinDat software for the eMDHAQ described in this report may partially overcome the problem of incompatibility of different EMRs [52] and the costs of data entry to analyze long-term outcomes of multiple rheumatic diseases in routine care, as it may be possible to facilitate pooling of deidentified data from multiple settings to establish a cost-effective multicenter database to assess clinical status and responses to therapies [34,53,54]. The preliminary results presented in this report suggest further steps toward implementation of an eMDHAQ for routine care.


We thank patients, rheumatologists, nurses, and front desk personnel at Rush University Medical Center, who made this study possible. At the time this study was carried out, IC was affiliated with Rush University Medical Center, but she has since become affiliated with General University Hospital Gregorio Marañón.

Conflicts of Interest

TP is the president of Medical History Services LLC and holds a copyright and trademark on MDHAQ and RAPID3 for which he receives royalties and license fees, all of which are used to support further development of quantitative questionnaire measurements for patients and doctors in clinical rheumatology care.

  1. Pincus T, Swearingen C, Wolfe F. Toward a multidimensional health assessment questionnaire (MDHAQ): Assessment of advanced activities of daily living and psychological status in the patient-friendly health assessment questionnaire format. Arthritis Rheum 1999 Oct;42(10):2220-2230. [CrossRef]
  2. Pincus T, Sokka T, Kautiainen H. Further development of a physical function scale on a MDHAQ [corrected] for standard care of patients with rheumatic diseases. J Rheumatol 2005 Aug;32(8):1432-1439. [Medline]
  3. Pincus T. A multidimensional health assessment questionnaire (MDHAQ) for all patients with rheumatic diseases to complete at all visits in standard clinical care. Bull NYU Hosp Jt Dis 2007;65(2):150-160 [FREE Full text] [Medline]
  4. Pincus T, Callahan LF, Sale WG, Brooks AL, Payne LE, Vaughn WK. Severe functional declines, work disability, and increased mortality in seventy-five rheumatoid arthritis patients studied over nine years. Arthritis & Rheumatism 1984 Aug;27(8):864-872. [CrossRef]
  5. Pincus T, Mandelin AM, Swearingen CJ. Flowsheets That Include MDHAQ Physical Function, Pain, Global, and RAPID3 Scores, Laboratory Tests, and Medications to Monitor Patients with all Rheumatic Diseases: An Electronic Database for an Electronic Medical Record. Rheum Dis Clin North Am 2009 Nov;35(4):829-842. [CrossRef]
  6. Pincus T. Prediction of Long-Term Mortality in Patients with Rheumatoid Arthritis according to Simple Questionnaire and Joint Count Measures. Ann Intern Med 1994 Jan 01;120(1):26. [CrossRef]
  7. Callahan LF, Cordray DS, Wells G, Pincus T. Formal education and five-year mortality in rheumatoid arthritis: Mediation by helplessness scale scores. Arthritis Care Res 1996 Dec;9(6):463-472. [CrossRef]
  8. Callahan LF, Pincus T, Huston JW, Brooks RH, Nance EP, Kaye JJ. Measures of activity and damage in rheumatoid arthritis: Depiction of changes and prediction of mortality over five years. Arthritis Care Res 1997 Dec;10(6):381-394. [CrossRef]
  9. Callahan LF, Pincus T. Formal education level as a significant marker of clinical status in rheumatoid arthritis. Arthritis Rheum 1988 Nov;31(11):1346-1357. [CrossRef]
  10. Callahan LF, Smith WJ, Pincus T. Self-report questionnaires in five rheumatic diseases comparisons of health status constructs and associations with formal education level. Arthritis Care Res 1989 Dec;2(4):122-131. [CrossRef]
  11. Pincus T, Marcum SB, Callahan LF. Longterm drug therapy for rheumatoid arthritis in seven rheumatology private practices: II. Second line drugs and prednisone. J Rheumatol 1992 Dec;19(12):1885-1894. [Medline]
  12. Yazici Y. Long term safety of methotrexate in routine clinical care: discontinuation is unusual and rarely the result of laboratory abnormalities. Ann Rheum Dis 2005 Feb 01;64(2):207-211. [CrossRef]
  13. Pincus T, Sokka T, Castrejón I, Cutolo M. Decline of Mean Initial Prednisone Dosage From 10.3 to 3.6 mg/day to Treat Rheumatoid Arthritis Between 1980 and 2004 in One Clinical Setting, With Long-Term Effectiveness of Dosages Less Than 5 mg/day. Arthritis Care Res 2013 Apr 23;65(5):729-736. [CrossRef]
  14. Pincus T, Hassett AL, Callahan LF. Criterion Contamination of Depression Scales in Patients with Rheumatoid Arthritis: The Need for Interpretation of Patient Questionnaires (as All Clinical Measures) in the Context of All Information About the Patient. Rheum Dis Clin North Am 2009 Nov;35(4):861-864. [CrossRef]
  15. El-Haddad C, Castrejon I, Gibson KA, Yazici Y, Bergman MJ, Pincus T. MDHAQ/RAPID3 scores in patients with osteoarthritis are similar to or higher than in patients with rheumatoid arthritis: a cross-sectional study from current routine rheumatology care at four sites. RMD Open 2017 Jul 18;3(1):e000391. [CrossRef]
  16. Pincus T, Yazici Y, Bergman MJ. RAPID3, an Index to Assess and Monitor Patients with Rheumatoid Arthritis, Without Formal Joint Counts: Similar Results to DAS28 and CDAI in Clinical Trials and Clinical Care. Rheum Dis Clin North Am 2009 Nov;35(4):773-778. [CrossRef]
  17. Pincus T, Bergman MJ, Yazici Y. RAPID3-an index of physical function, pain, and global status as "vital signs" to improve care for people with chronic rheumatic diseases. Bull NYU Hosp Jt Dis 2009;67(2):211-225 [FREE Full text] [Medline]
  18. Pincus T, Swearingen CJ, Bergman MJ, Colglazier CL, Kaell AT, Kunath AM, et al. RAPID3 (Routine Assessment of Patient Index Data) on an MDHAQ (Multidimensional Health Assessment Questionnaire): Agreement with DAS28 (Disease Activity Score) and CDAI (Clinical Disease Activity Index) activity categories, scored in five versus more than ninety seconds. Arthritis Care Res 2010 Feb;62(2):181-189. [CrossRef]
  19. Anderson J, Caplan L, Yazdany J, Robbins ML, Neogi T, Michaud K, et al. Rheumatoid arthritis disease activity measures: American College of Rheumatology recommendations for use in clinical practice. Arthritis Care Res 2012 Apr 02;64(5):640-647. [CrossRef]
  20. Curtis JR, Chen L, Danila MI, Saag KG, Parham KL, Cush JJ. Routine Use of Quantitative Disease Activity Measurements among US Rheumatologists: Implications for Treat-to-target Management Strategies in Rheumatoid Arthritis. J Rheumatol 2017 Nov 15;45(1):40-44. [CrossRef]
  21. Castrejon I. The Use of MDHAQ/RAPID3 in Different Rheumatic Diseases A Review of the Literature. Bull Hosp Jt Dis 2017 Apr;75(2):93-100. [Medline]
  22. Castrejón I, Bergman MJ, Pincus T. MDHAQ/RAPID3 to Recognize Improvement Over 2 Months in Usual Care of Patients With Osteoarthritis, Systemic Lupus Erythematosus, Spondyloarthropathy, and Gout, as Well as Rheumatoid Arthritis. J Clin Rheumatol 2013;19(4):169-174. [CrossRef]
  23. Pincus T. Can RAPID3, an index without formal joint counts or laboratory tests, serve to guide rheumatologists in tight control of rheumatoid arthritis in usual clinical care? Bull NYU Hosp Jt Dis 2009;67(3):254-266. [Medline]
  24. Pincus T, Maclean R, Yazici Y, Harrington JT. Quantitative measurement of patient status in the regular care of patients with rheumatic diseases over 25 years as a continuous quality improvement activity, rather than traditional research. Clin Exp Rheumatol 2007;25(6 Suppl 47):69-81. [Medline]
  25. Stucki G, Liang MH, Stucki S, Brühlmann P, Michel BA. A self-administered rheumatoid arthritis disease activity index (RADAI) for epidemiologic research. Arthritis Rheum 1995 Jun;38(6):795-798. [CrossRef]
  26. Castrejón I, Yazici Y, Pincus T. Patient self-report RADAI (Rheumatoid Arthritis Disease Activity Index) joint counts on an MDHAQ (Multidimensional Health Assessment Questionnaire) in usual care of consecutive patients with rheumatic diseases other than rheumatoid arthritis. Arthritis Care Res 2013 Jan 30;65(2):288-293. [CrossRef]
  27. Pincus T, Swearingen CJ. The HAQ Compared with the MDHAQ: “Keep It Simple, Stupid” (KISS), with Feasibility and Clinical Value as Primary Criteria for Patient Questionnaires in Usual Clinical Care. Rheum Dis Clin North Am 2009 Nov;35(4):787-798. [CrossRef]
  28. Schmukler J, Pincus T. The Rheumatologist. 2019. Remote Use of the Multidimensional Health Assessment Questionnaire (MDHAQ)   URL: https:/​/www.​​article/​remote-use-of-the-multidimensional-health-assessment- questionnaire-mdhaq/​5/​?singlepage=1 [accessed 2020-05-04]
  29. Castrejón I, Yazici Y, Celik S, Pincus T. Pragmatic assessment of exercise in routine care using an MDHAQ: associations with changes in RAPID3 and other clinical variables. Arthritis Res Ther 2016 Sep 6;18(1). [CrossRef]
  30. Pincus T, Furer V, Keystone E, Yazici Y, Bergman MJ, Luijtens K. RAPID3 (Routine Assessment of Patient Index Data 3) severity categories and response criteria: Similar results to DAS28 (Disease Activity Score) and CDAI (Clinical Disease Activity Index) in the RAPID 1 (Rheumatoid Arthritis Prevention of Structural Damage) Clinical Trial of Certolizumab Pegol. Arthritis Care Res 2011 Aug 01;63(8):1142-1149. [CrossRef]
  31. Pincus T, Yazici Y, Castrejón I. Pragmatic and scientific advantages of MDHAQ/ RAPID3 completion by all patients at all visits in routine clinical care. Bull NYU Hosp Jt Dis 2012;70 Suppl 1:30-36 [FREE Full text] [Medline]
  32. Gibson KA, Castrejon I, Descallar J, Pincus T. Fibromyalgia Assessment Screening Tool: Clues to Fibromyalgia on a Multidimensional Health Assessment Questionnaire for Routine Care. J Rheumatol 2019 Sep 01:jrheum.190277. [CrossRef]
  33. Schmukler J, Jamal S, Castrejon I, Block JA, Pincus T. Fibromyalgia Assessment Screening Tools (FAST) Based on Only Multidimensional Health Assessment Questionnaire (MDHAQ) Scores as Clues to Fibromyalgia. ACR Open Rheuma 2019 Aug 22;1(8):516-525. [CrossRef]
  34. Hetland ML. DANBIO--powerful research database and electronic patient record. Rheumatology (Oxford) 2011 Jan 08;50(1):69-77. [CrossRef] [Medline]
  35. Lindström Egholm C, Krogh NS, Pincus T, Dreyer L, Ellingsen T, Glintborg B, et al. Discordance of Global Assessments by Patient and Physician Is Higher in Female than in Male Patients Regardless of the Physician’s Sex: Data on Patients with Rheumatoid Arthritis, Axial Spondyloarthritis, and Psoriatic Arthritis from the DANBIO Registry. J Rheumatol 2015 Aug 01;42(10):1781-1785. [CrossRef]
  36. Chua JR, Jamal S, Riad M, Castrejon I, Malfait A, Block JA, et al. Disease Burden in Osteoarthritis Is Similar to That of Rheumatoid Arthritis at Initial Rheumatology Visit and Significantly Greater Six Months Later. Arthritis Rheumatol 2019 Jul 03;71(8):1276-1284. [CrossRef]
  37. Castrejon I, Shakoor N, Chua JR, Block JA. Discordance of global assessment by patients and physicians is higher in osteoarthritis than in rheumatoid arthritis: a cross-sectional study from routine care. Rheumatol Int 2018 Oct 6;38(11):2137-2145. [CrossRef]
  38. Uhlig T, Kvien TK, Pincus T. Test-retest reliability of disease activity core set measures and indices in rheumatoid arthritis. Ann Rheum Dis 2008 Oct 28;68(6):972-975. [CrossRef]
  39. Katchamart W, Narongroeknawin P, Chanapai W, Thaweeratthakul P. Health-related quality of life in patients with rheumatoid arthritis. BMC Rheumatol 2019 Aug 14;3(1). [CrossRef]
  40. McCollum L, Pincus T. A Biopsychosocial Model to Complement a Biomedical Model: Patient Questionnaire Data and Socioeconomic Status Usually Are More Significant than Laboratory Tests and Imaging Studies in Prognosis of Rheumatoid Arthritis. Rheum Dis Clin North Am 2009 Nov;35(4):699-712. [CrossRef]
  41. Castrejón I, McCollum L, Tanriover MD, Pincus T. Importance of patient history and physical examination in rheumatoid arthritis compared to other chronic diseases: results of a physician survey. Arthritis Care Res (Hoboken) 2012 Aug;64(8):1250-1255 [FREE Full text] [CrossRef] [Medline]
  42. Weed LL. Medical Records That Guide and Teach. N Engl J Med 1968 Mar 21;278(12):652-657. [CrossRef]
  43. Pincus T, Castrejon I. Are Patient Self-Report Questionnaires as "Scientific" as Biomarkers in "Treat-to-target" and Prognosis in Rheumatoid Arthritis? Curr Pharm Des 2014 Nov 21;21(2):241-256. [CrossRef]
  44. Pincus T, Castrejón I. MDHAQ/RAPID3 scores: quantitative patient history data in a standardized "scientific" format for optimal assessment of patient status and quality of care in rheumatic diseases. Bull NYU Hosp Jt Dis 2011;69(3):201-214 [FREE Full text] [Medline]
  45. Sokka T, Abelson B, Pincus T. Mortality in rheumatoid arthritis: 2008 update. Clin Exp Rheumatol 2008;26(5 Suppl 51):S35-S61. [Medline]
  46. Sokka T, Pincus T. Poor physical function, pain and limited exercise: risk factors for premature mortality in the range of smoking or hypertension, identified on a simple patient self-report questionnaire for usual care. BMJ Open 2011 Jun 18;1(1):e000070-e000070. [CrossRef]
  47. Castrejon I, Nikiphorou E, Jain R, Huang A, Block J, Pincus T. SAT0343 How Much Does Fatigue Contribute to the Physician and Patient Global Estimates in Different Rheumatic Diseases? Analysis from Routine Care on a Multidimensional Health Assessment Questionnaire (MDHAQ). Ann Rheum Dis 2015 Jun 09;74(Suppl 2):783.1-78783. [CrossRef]
  48. Pincus T, Askanase AD, Swearingen CJ. A Multi-Dimensional Health Assessment Questionnaire (MDHAQ) and Routine Assessment of Patient Index Data (RAPID3) Scores are Informative in Patients with All Rheumatic Diseases. Rheum Dis Clin North Am 2009 Nov;35(4):819-827. [CrossRef]
  49. Clauw DJ, Katz P. The Overlap Between Fibromyalgia and Inflammatory Rheumatic Disease. J Clin Rheumatol 1995;1(6):335-342. [CrossRef]
  50. Wolfe F, Petri M, Alarcón GS, Goldman J, Chakravarty EF, Katz RS, et al. Fibromyalgia, systemic lupus erythematosus (SLE), and evaluation of SLE activity. J Rheumatol 2009 Jan 01;36(1):82-88 [FREE Full text] [CrossRef] [Medline]
  51. Wolfe F, Häuser W, Hassett AL, Katz RS, Walitt BT. The development of fibromyalgia – I: Examination of rates and predictors in patients with rheumatoid arthritis (RA). Pain 2011;152(2):291-299. [CrossRef]
  52. Wachter R. The Digital Doctor: Hope, Hype, And Harm At The Dawn Of Medicine’s Computer Age. New York: McGraw-Hill Education; 2017.
  53. Pincus T, Keysor J, Sokka T, Krishnan E, Callahan LF. Patient questionnaires and formal education level as prospective predictors of mortality over 10 years in 97% of 1416 patients with rheumatoid arthritis from 15 United States private practices. J Rheumatol 2004 Feb;31(2):229-234. [Medline]
  54. Pincus T. Electronic multidimensional health assessment questionnaire (eMDHAQ): past, present and future of a proposed single data management system for clinical care, research, quality improvement, and monitoring of long-term outcomes. Clin Exp Rheumatol 2016;34(5 Suppl 101):S17-S33. [CrossRef] [Medline]

EMR: electronic medical record
eMDHAQ: electronic multidimensional health assessment questionnaire
FAST3: Fibromyalgia Assessment Screening Tool 3
FHIR: fast healthcare interoperability resources
ICC: intraclass correlation coefficient
MDHAQ: multidimensional health assessment questionnaire
PATGL: patient global assessment
RA: rheumatoid arthritis
RADAI: Rheumatoid Arthritis Disease Activity Index
RAPID3: Routine Assessment of Patient Index Data 3
VNS: visual numeric scale

Edited by G Eysenbach; submitted 08.08.19; peer-reviewed by J Curtis, G Yeowell, M Iversen, S Desai; comments to author 08.11.19; revised version received 20.12.19; accepted 29.03.20; published 27.05.20


©Theodore Pincus, Isabel Castrejon, Mariam Riad, Elena Obreja, Candice Lewis, Niels Steen Krogh. Originally published in JMIR Formative Research (, 27.05.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.