A multidimensional health assessment questionnaire (MDHAQ) [1-3] has been completed by all patients at all visits to one of the authors (TP) since 1982, based on initial evidence that patient questionnaire scores for physical function are significant in the prognosis of work disability and premature death in rheumatoid arthritis (RA) [4]. Paper MDHAQ data for all patients were entered into databases (Paradox, Access) in the 1990s to provide serial flowsheets of patient scores, laboratory data, and medications, which informed clinical decisions [5]. Clinical research using MDHAQ databases has advanced knowledge concerning the prognosis of mortality [6-8], importance of socioeconomic status [7,9,10], value of methotrexate [11,12], value of low-dose glucocorticoids [13], and recognition of depression [1,14] in RA and other rheumatic diseases.

The MDHAQ is completed by all patients at several rheumatology sites, including New York University Hospital for Joint Diseases (since 2005), a private practice in Ridley Park, PA (since 2006), Rush University Medical Center (since 2006), and Liverpool Hospital in Australia (since 2013) [15]. However, the patients continue to complete a paper MDHAQ despite the use of electronic medical records (EMRs) at each of these sites, as the EMR generally has not included patient questionnaires. Databases of paper MDHAQ data from routine care have been entered into electronic research databases to recognize that disease burden in osteoarthritis is similar to RA [15], and to develop indices such as routine assessment of patient index data 3 (RAPID3) [16-18] and a fibromyalgia assessment screening tool (FAST3). RAPID3 is the most widely used RA index in the United States [19,20], and comparable to disease-specific indices to recognize changes in clinical status in patients with all other rheumatic diseases which have been studied [21-23].

An electronic MDHAQ (eMDHAQ), linked to an EMR through fast healthcare interoperability resources (FHIR), could enhance clinical care with serial flowsheets, enable remote patient completion before or between scheduled visits to report problems, and reduce costs and errors in retrospective data entry of paper versions for clinical research. This report presents analyses of the reliability, feasibility, and patient acceptance of an eMDHAQ.

Among the 98 patients in the study, the mean age was 53.8 years (range 21.0-88.0, SD 16.6); 87 (89%) were female, 47 (48%) were white, 26 (27%) were black, and 25 (25%) were members of other ethnic groups. The patients had various ICD-10 diagnoses, which were assigned by the treating rheumatologist (Table 1). These patients appear to represent a typical cross-sectional group of patients seen in an academic rheumatology setting according to age, gender, and diagnosis.

The electronic and paper MDHAQ mean scores (SD) were almost identical (Table 2).

Differences ranged from –0.4 to 0.7 and were all within 2% of one another; no scores differed significantly, either clinically or statistically. The ICC for the symptom checklist was 0.86, indicating good reliability, and all other ICCs were greater than 0.92, indicating excellent reliability (Table 2).

None of the volunteer patients reported difficulties using the iPad. The mean time to complete the 6-screen eMDHAQ was 8.2 minutes, indicating good feasibility. For the patient self-report paper questionnaire, the mean VNS rating (0-10, where 10=very helpful) of how helpful the MDHAQ was to the patient was 8.8 (SD 1.7); the mean rating of how helpful the MDHAQ was to the physician was 8.7 (SD 1.9). Patient preferences were 72% for the electronic version versus 7% for the paper version, while 21% noted no preference. Preferences did not differ meaningfully according to age or level of education (Table 3).

The eMDHAQ performed similarly to the traditional paper MDHAQ. All ICCs were greater than 0.86, indicating good to excellent reliability of the 2 versions; these values are as high as those seen in most comparisons in clinical medicine. For example, an earlier report compared electronic and paper versions of RA core data set measures [38,39], indicating ICCs for a swollen joint count of 0.78 and for a tender joint count of 0.83; ICCs for RA indices which include a formal joint count were 0.85 for Disease Activity Score 28 and 0.89 for the Clinical Disease Activity Index. In an earlier study, the ICC for the self-report physical function was 0.96, that for pain was 0.88, that for patient global assessment was 0.78, and that for RAPID3 was 0.90 [38], compared to 0.97, 0.95, and 0.96, and 0.98, respectively, in the present study (Table 2). The higher ICCs in the present study may be explained in part by the 5-7 day interval between measures in the earlier study compared to 1-2 hours in the present study; also, patients seen at Rush University Medical Center have extensive experience with the MDHAQ. The previous data are presented to document that the ICCs of the measures reported by patients were somewhat higher than those obtained by physicians.

The mean time required to complete the eMDHAQ was 8.2 minutes, which appears to be acceptable. Formal studies have not been reported for the time to complete a paper MDHAQ, although informal observations over more than 30 years suggest that 5-10 minutes are required. In general, <10 minutes appears to be acceptable, although some patients likely will require more time, as is the case with the paper version.

Approximately two-thirds of patients expressed a preference for the electronic version of the MDHAQ. However, no specific information was collected concerning the number of patients who declined to complete an eMDHAQ in the routine care setting. It is reassuring that no meaningful differences in preference or reliability were seen according to age or education level, although it is anticipated that problems with completion of an eMDHAQ will be more likely in older or less educated patients, as has been seen with the paper MDHAQ; thus, differences may emerge with larger numbers of patients. However, several reports indicating poor clinical status associated with low formal education level [7,9,40] required completion of a patient questionnaire by the patients analyzed in the studies. A small fraction of patients preferred the paper version, and it is anticipated that a paper version will be offered to some patients even after the eMDHAQ is incorporated into routine care.

Clinical decisions in rheumatology patients are based more on information from a patient history than in many chronic diseases in which decisions are dominated by a “gold standard” biomarker, such as blood pressure in hypertension or serum glucose in diabetes [41]. A patient self-report questionnaire depicts components of a “subjective” [42] patient medical history as structured, quantitative, standard, protocol-driven, data which meet criteria for the scientific method [43,44]. Physical function reported on a patient questionnaire is far more significant in the prognosis of premature mortality in RA than any laboratory or imaging data [4,6,45] and is as significant as smoking to predict mortality in a nondiseased elderly population [46].

The value of RAPID3 in rheumatology care [19,20,22] is attributable in part to its capacity to depict change in clinical status in all rheumatic diseases studied to date [21,22], while the patient does almost all the work. Nonetheless, availability of additional MDHAQ scales provide a clinician with considerably more information than only RAPID3, for fatigue [47], RADAI self-report painful joint count [26], adverse events of medications and their resolution [28]. Traditional medical history queries on the MDHAQ save time for patients and doctors [31].

Quantitative RAPID3 scores are highest in patients with fibromyalgia [48], which is seen as a comorbid condition in 20–40% of patients with RA, OA, and many rheumatic diagnoses [49-51]. Clinical improvement is far less likely in patients with comorbid fibromyalgia with any diagnosis than in patients who have this diagnosis and no evidence of fibromyalgia. A further MDHAQ index, FAST3 (fibromyalgia assessment screening tool) may be used to screen for fibromyalgia [32,33], and potentially explain unchanged RAPID3 scores in RA patients with comorbid (secondary) fibromyalgia. FAST3 includes the RADAI self-report painful joint count and symptom checklist [32,33], and therefore is not available when only RAPID3 is queried.

The eMDHAQ was developed using FHIR as an internal format to save data in a Zope object database, with capacity to interface with any EMR in the management of individual patients. Further description of such integration is beyond the scope of the present study, which was focused on reliability of the eMDHAQ and its acceptability to patients. Establishment of these features is regarded as an important future stage in development of an eMDHAQ for implementation in routine clinical care.

This study has several important limitations. First, all patients completed the paper MDHAQ as a usual component of routine care before seeing the rheumatologist, and they completed the eMDHAQ at the conclusion of the visit. Ideally, half might have completed the eMDHAQ initially in a formal study, but such a study would have presented increased costs, logistic complexities, and an extra burden to patients in a setting at which all patients complete an MDHAQ routinely. Second, patient recall of previous completion of the MDHAQ approximately 30-90 minutes earlier could have influenced the second recording. Third, neither the number of patients who declined to participate nor the reasons for declining were recorded in a routine care setting; in general, clinicians reported informally that patients who declined participation noted other scheduled activities, but some patients may have wished to avoid the computer version. Fourth, patient preference for the electronic version may be attributed in part to a “Hawthorne effect,” with further attention from a research professional and a new approach. At the same time, many patients had completed many paper MDHAQs previously, and a bias to favor the familiar paper version could have been present. The eMDHAQ appeared acceptable to most patients, although provision for a paper version for some patients is anticipated in clinical implementation of an eMDHAQ for routine care.

The analyses reported here focused on cross-sectional reliability, which is regarded by institutional information technology professionals and developers as a prerequisite for further work toward use of an eMDHAQ in routine care. Several adjustments to the workflow will be needed to implement an eMDHAQ in routine care; these details remain to be addressed in the next phases of development. Further programming and collaboration with EMR vendors for exchange of eMDHAQ data with the EMR, using FHIR available within the software, is anticipated.

An eMDHAQ presents advantages to allow completion at home before scheduled visits, rather than in the waiting area, and remotely between visits to document status when patients report disease flares or adverse medication events. Data entered by patients can be transferred to the hospital EMR using FHIR and can be made available on the physician’s computer screen. The ClinDat software for the eMDHAQ described in this report may partially overcome the problem of incompatibility of different EMRs [52] and the costs of data entry to analyze long-term outcomes of multiple rheumatic diseases in routine care, as it may be possible to facilitate pooling of deidentified data from multiple settings to establish a cost-effective multicenter database to assess clinical status and responses to therapies [34,53,54]. The preliminary results presented in this report suggest further steps toward implementation of an eMDHAQ for routine care.