This study used patient-reported survey data that were recorded so that participants were unidentifiable to the researchers. The institutional review board at Stony Brook University considered this study exempt (IRB2020-00429) from the Code of Federal Regulations Title 45 requirements.

This retrospective cohort study took place at the “Family to Family” medical clinic in the Southeast region of the United States between August 30, 2017, and January 30, 2023, among a sample of 367 participants with a range of chronic diseases. Additionally, for the purpose of PSI to PROMIS T-score calibration, an independent data set consisting of 122,591 assessments from 29,353 customers of Parsley Health was used to establish the PSI T-score conversion table detailed in Multimedia Appendix 1.

The PSI is a 45-item ePROM, similar to a Review of Symptoms, focusing on bodily domains and the most commonly reported symptoms associated with chronic diseases for each domain. The PSI assesses a patient’s perception of symptom burden. The PSI was developed using the Federal Drug Agency’s guidance for PROM development [24]. Items are grouped into 9 systems and ranked on a scale from 0 (asymptomatic) to 10 (extremely symptomatic). A total score is calculated with the following 4 cutoff ranges: “well” (0-24), “symptomatic” (25-43), “very symptomatic” (44-71), and “sick” (≥71). The PSI has shown clinical validity for use in clinical practice [22].

The PROMIS-10 is a single, generalizable, and validated PROM that can be used for various diseases and conditions. It is a shortened version of PROMIS that was developed to minimize respondent burden. This version is a 10-item, patient-reported questionnaire that was created as a general health assessment tool. Nine out of 10 questions on the PROMIS-10 are answered using a 5-point Likert scale, with the tenth question answered using a numeric rating scale. Results can be tracked in three different ways: (1) answers to each of the 10 questions can be evaluated separately, (2) answers can be grouped together to provide a global summary score, or (3) answers can be split into 2 groups to provide a global physical health score and a global mental health score.

We compared the PSI to the PROMIS-10, as it is similar to other general health short-form surveys and is widely adopted due to its ease of use. The PROMIS-10 has been shown to be valid and reliable in clinical settings for patients from the general population [16] and those living with chronic diseases [25,26]. Similar to its more extensive counterpart, the PROMIS-10 has undergone rigorous testing and validation across diverse age groups, including younger and older adults [27,28], and has proven to be reliable across a variety of clinical populations [29-31].

Family to Family is a hybrid (remote and in-person) functional and holistic medicine clinic for adults and children located in the Asheville, North Carolina, metropolitan area. The average patient age was 53.7 years old, and patients predominantly identified as female (73%). While race data were not available, the 2 clinicians at this practice report that their patients are predominantly White.

Patients and their caregivers were prompted to complete both the PSI and the PROMIS-10 through a password-protected electronic medical record web-based portal before each clinical visit. The PSI was added as a PROM to complete along with the PROMIS-10 because the clinicians believed it provided different insight as a Review of Symptoms to capture a more comprehensive view of patients’ symptomatology and progress over time.

Patients were required to complete the PSI and PROMIS before their first clinical visit. If both ePROMs were not completed before a patient’s first clinical visit at Family to Family, the visit was postponed or rescheduled. For all subsequent visits, completing the ePROMs was optional but encouraged. Participants were not compensated for completing the ePROMs. When preparing for the patient’s visit, Family to Family clinicians could view responses to both ePROMs in a patient’s electronic health record and use these responses to guide a clinical encounter. Clinicians were able to ask targeted questions about a patient’s symptoms and identify triggers that might contribute to the symptoms.

The data analyses were conducted using Python (version 3.10; Python Software Foundation) [32].

In our data set, we analyzed a total of 1170 observations from 367 unique patients recorded between August 30, 2017, and January 30, 2023, from Family to Family. On average, participants completed the PSI 3.2 times and the PROMIS 3.4 times during the study period. Adhering to the guidelines for good reporting practices, the CHERRIES (Checklist for Reporting Results of Internet E-Surveys) [35] is included in Multimedia Appendix 2. Detailed patient demographics and general descriptions of the sample are delineated in Table 1. On average, participants reported experiencing 8 distinct symptoms or conditions. Excluding nutrient deficiencies, the most commonly reported diseases and health problems, as classified by International Statistical Classification of Diseases, Tenth Revision (ICD-10) codes, were other fatigue (82/367, 22.3%), anxiety disorder (81/367, 22.1%), hypothyroidism (81/367, 22.1%), and chronic fatigue (65/367, 17.7%), as outlined in Table 2.

The mean value for the PSI T-score is 40.808 (SD 7.00), with a minimum and maximum range of values between 25 and 56, indicating a broad spectrum of reported symptom states within the sample. Since the expected average for a typical population is 50, this lower mean suggests that the sample population exhibits a higher level of symptoms or less optimal health than the general population. The mean value for the PROMIS physical score is 47.952 (SD 8.114), which is slightly below the expected average of 50. This result also implies that the physical health of the sample population is somewhat below average. The minimum and maximum values for the physical T-scores range from 19.9 to 67.7, indicating a broad spectrum of physical health states within the sample. The mean value for the PROMIS mental score is 46.638 (SD 8.551), which is also below the expected average of 50. This suggests that the mental health of the sample population is also somewhat lower compared to the general population. The PROMIS mental ranges from a minimum of 21.2 to a maximum of 67.6, further indicating variation in mental health states within the sample. Additional descriptives for PSI and PROMIS T-scores across time are provided in Table 3.

The distribution characteristics of the PSI and PROMIS T-scores were assessed for normality. The PSI T-scores were found to follow a non-normal distribution (χ²₁₁₆₉=183.324; P<.001). The skewness of 0.577 in the PSI T-scores indicates a distribution with a longer right tail and a concentration of scores on the left, reflecting a higher frequency of lower scores and thus a less healthy population. The negative kurtosis of –0.858 signifies a platykurtic kurtosis and that extreme outliers (very high or low) are less frequent in this data set than they would be in a normally distributed data set. In contrast, the PROMIS physical and mental T-scores were found to follow a normal distribution (P=.10 and P=.46), with a minor skewness of –0.145 and a kurtosis of 0.100 for the physical, while an almost perfect skewness of –0.015 and a slight platykurtic kurtosis of –0.173 for the mental.

Previous studies found the PSI to be a valid tool that can be deployed, completed, and helpful to both patients and clinicians [21,22,36]. This study examined differences in use between the PSI as compared to the PROMIS-10 short form when used in clinical settings with patients with chronic disease. The PSI and PROMIS-10 were chosen for this retrospective study because the clinicians believed it was useful for patients to complete both forms as they provide slightly different insights into how patients perceive their health status. While the differences between the PSI and PROMIS-10 reveal ways that each has its place in clinical practice, they overlap enough to demonstrate a moderate correlation to support validation of the PSI when compared to the PROMIS-10.

The overall mean (SD) statistics for the PSI and PROMIS-10 paint a picture of a sample population that is generally less healthy than the average population, both in terms of physical and mental well-being, as well as symptomatology. The lower PSI score, in particular, stands out, indicating a higher level of symptoms. These findings set the stage for further analysis to understand the underlying factors, relationships, and potential interventions that may be relevant to this specific population.

The univariate generalized linear regression analyses conducted in this study used the Gaussian family with an identity link function to explore the relationship between the PSI T-scores and both mental and physical health as measured by the PROMIS scales. The consistency in the direction and significance of the relationships between the PSI across both mental and physical health domains defined by the PROMIS, as revealed in the univariate analyses, lends credibility to the models and provides a robust foundation for further exploration. The positive associations demonstrate the criterion validity of the PSI, illustrating its potential to predict changes in both mental and physical health as measured by the PROMIS scales. The substantial explanatory power of the models, as evidenced by the pseudo R² values and supported by the deviance statistics and Pearson chi-square values, adds to the robustness of the findings and their potential implications for the PSI as a validated health assessment. This validation underscores the use of the PSI and its capability of offering insights into overall well-being. Future research may benefit from examining these relationships in different populations or contexts, potentially extending the applicability of the PSI.

This study bears several notable limitations. First, our data emanate from a single clinic where a significant majority of Family to Family participants identified as female (73%), with an average age of 53 years. Although race and ethnicity data were not available, the clinic reported that the patient population was predominantly White. Such skewness constrains the ecological validity of our findings.

Our sample size was not large enough to support a robust longitudinal analysis, thereby limiting the depth of insights we could derive. It is imperative for future validation studies to explore the PSI’s use within a more diverse demographic profile and over a more extended time frame. Such studies would not only deepen the understanding of symptom trajectories but also facilitate the evaluation of patient outcomes across a wider demographic landscape.

In terms of the PSI questionnaire itself, the Family to Family participants engaged with an earlier iteration. Based on patient feedback, Parsley Health implemented minor revisions to this version to enhance readability. Consequently, we made retrospective adjustments to ensure alignment with the updated version of the PSI, which encompassed an additional item but was more concise in terms of completion time since responses were no longer categorized as “resolved” or “ongoing.”

Additionally, the nature of this being a retrospective cohort study meant that the PSI and PROMIS-10 items were not presented to participants in a randomized manner, which could have potentially mitigated response biases. We advocate for the implementation of randomization, or A/B testing, in subsequent studies.

Although we know that telehealth tools can be used to deliver effective care to patients with chronic conditions, few—if any—tools exist that are designed as digital-first ePROMS. This predictive criterion study compared the PSI—a digital-first ePROM—to the PROMIS-10, a traditional PROM, in a functional medicine clinic for patients with a range of chronic conditions. This study revealed significant relationships between the PSI and PROMIS physical and mental health scores through comprehensive univariate analyses, thus establishing support for the criterion validity of the PSI. These analyses highlighted the PSI’s potential as an insightful tool for understanding and predicting both mental and physical health dimensions.

Overall, the findings of this study emphasize the importance of the PSI as a versatile clinical instrument. Future research is warranted to further dissect these relationships and enhance our understanding of the PSI’s applicability in various health contexts.