This study was performed using data on hay fever from a previous crowdsourced, cross-sectional, observational study that was conducted using the smartphone app “AllerSearch” between February 1, 2018, and May 1, 2020 (Figure 1A) [3,4,37]. In the previous study, participants were recruited through the Apple App Store, and the study was announced via digital media [38] and on social networking platforms (X, formerly known as Twitter [39] and Facebook [40]).

All procedures involving human participants were performed in accordance with the ethical standards of the institutional or national research committees, in alignment with the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards. The study design was approved by the Independent Ethics Committee of Juntendo University Faculty of Medicine (M17-0061-M02). All participants provided voluntary informed consent electronically [4], with the original consent covering both the primary data collection and any secondary analyses. All study data were anonymized to ensure participant privacy, and no personally identifiable information was included. Participants did not receive any compensation for their involvement in the study.

“AllerSearch,” a self-developed smartphone app, was released for iOS on ResearchKit in February 2018, and for Android in August 2020, under a consignment contract between Juntendo University Graduate School of Medicine and InnoJin Inc, Tokyo, Japan [3,4,8,41,42]. “AllerSearch” is freely available on the Apple App Store and Google Play.

In this study, data from our previous study using “AllerSearch” were used [4]. The data collection process has been described previously [3,4,8]. Briefly, participants who downloaded “AllerSearch” and provided electronic informed consent shared information on demographics, medical history, lifestyle, and response (nonhay fever, hay fever, and unknown) to the question “Do you have hay fever?” Additionally, they reported their hay fever symptoms and preventive behaviors (Multimedia Appendix 1). The participants then completed questionnaires including the JACQLQ, stress level due to hay fever, nasal symptom score, nonnasal symptom score, and work productivity, which pertained to their daily hay fever symptoms (Multimedia Appendices 2 and 3) [14,32]. Responses to the hay fever symptom questionnaires that were inputted into the app for the first time were used in this study. The stress level was scored on an 11-point visual analog scale ranging from 0 (none) to 10 (most stressful).

The app-based JACQLQ was used in this study to measure QoL in patients with allergic conjunctivitis [14,34,43]. The JACQLQ has validity and reliability based on the results of factor analysis and correlation coefficients [43]. Additionally, the app-based JACQLQ has shown validity and reliability based on internal consistency and the scores of Cronbach α coefficients and correlation coefficients [14]. The JACQLQ comprises three domains: I, II, and III. Domain I comprises 9 items related to ocular and nasal symptoms. Domain II comprises 17 items on daily activity (11 items) and psychological well-being (6 items). Each item in Domains I and II (Figure 1B) is scored on a 5-point Likert scale from none (0 points) to severe (4 points). Domain III consists of a face scale (Figure 1C), which is scored on a 5-point scale to depict emotions from 0 (fine) to 4 (crying) [14,34].

Based on the collected data, participants residing in Japan and reporting hay fever were included in this study, whereas individuals without hay fever, with an unknown diagnosis, or with missing questionnaire data related to hay fever symptoms, including the JACQLQ, face scale, and stress level scale, were excluded from this study.

MCIDs were calculated using both anchor- and distribution-based methods [36]; thus, enabling the use of external indicators as anchors to assign patients to clinically relevant categories [29,44]. It is recommended that the estimation of the MCID in the anchor-based method be based on multiple anchors [45]. Therefore, the face scale from the JACQLQ Domain III and the stress level due to hay fever were used as anchors to determine the MCIDs of the JACQLQ Domain II score.

The face scale of the JACQLQ Domain III comprised a 5-scale anchor to categorize the participants into 5 severity categories [29]. A previous study used a 2-point change as a meaningful difference on an 11-point scale [29,46,47]. Based on this method, an 11-point stress level scale was used as the 5-scale anchor, and participants were assigned to one of the five severity groups [47]. To assess the eligibility of anchors for determining the MCID, the Spearman correlation coefficient and the number of participants in each severity category were calculated [29,47]. The anchor was considered eligible if the correlation coefficient between the anchor and the total score of daily activity, psychological well-being, and total JACQLQ Domain II score was ≥0.3 and the number of participants within a severity category included at least 10 entries [29,45,47]. The mean differences between adjacent severity categories of the anchors provided the MCID estimates. The IQR of the anchor-based MCID estimates was used to calculate the MCID ranges [29,47].

Two distribution-based methods based on the standard error of measurement (SEM) or half SD were used to provide supporting data in combination with the anchor-based results as follows [36,45]:

MCID ranges, rather than a single MCID estimate, are recommended for MCID estimation [29]. Therefore, the final MCID ranges were estimated from the MCIDs calculated using the anchor- and distribution-based methods [29]. The final MCID ranges were determined by setting the lower limit as either the 25th percentile, found using the anchor-based method, or the SEM, calculated using the distribution-based method, and the upper limit as the 75th percentile, found using the anchor-based method. To compare the demographics and characteristics of the participants in each severity group, continuous variables were presented as mean and SD, whereas categorical variables were presented as proportions. All statistical analyses were conducted using STATA (version 18.0; StataCorp).

A flowchart depicting the screening and enrollment of participants in this study is shown in Multimedia Appendix 4. A total of 17,597 individuals were identified using a unique identifier. Of these, 15,749 individuals provided electronic consent, whereas 1848 individuals did not respond. In total, 11,442 individuals completed the surveys for basic information on characteristics, lifestyle, and hay fever, as well as the surveys for hay fever symptoms and the JACQLQ Domain II. After excluding 113 individuals with outlier data for height, weight, and age, 45 individuals without geographic data, 2243 individuals with nonhay fever or unknown history, and incomplete face scale or stress level surveys, 7590 individuals were included in this study. The demographics and medical history of the 7590 participants with hay fever are summarized in Tables 1 and 2, respectively. The mean age of the participants was 35.3 (SD 13.9) years, and 57.1% (n=4331) of the participants were women. Approximately 82% (6225/7590) of the data related to hay fever were recorded in February or March of each year (Figure 2), which marks the peak months of cedar and cypress pollen distribution in Japan. Based on the scores of both the face scale and stress level scale, participants in the more severe categories were more likely to be younger or female.

The JACQLQ Domain II scores are presented in Tables 3 and 4. Mean scores of the total daily activity, total psychological well-being, and total JACQLQ Domain II were 8.9 (SD 9.6), 5.9 (SD 6.7), and 14.8 (SD 15.5) points, respectively.

Spearman correlation coefficients between severity anchor categories (face scale score and stress level scale score) and the total scores (daily activity, psychological well-being, and total JACQLQ Domain II score) were ≥0.3, and anchors were considered eligible (face scale category: daily activity 0.555, psychological well-being 0.597, and total JACQLQ Domain II score 0.603; stress level scale category: daily activity 0.527, psychological well-being 0.513, and total JACQLQ Domain II score 0.547). MCID estimates calculated using the anchor-based method are shown in Table 5 (range: daily activity 1.0-6.9; psychological well-being 1.2-5.6; and total JACQLQ Domain II score 2.1-12.6).

Table 5 presents the MCID estimates calculated using the distribution-based method. The MCID estimates based on 0.5 SD of the JACQLQ Domain II scores at baseline were 4.8, 3.4, and 7.8 for daily activity, psychological well-being, and the total JACQLQ Domain II scores, respectively. ICCs were calculated to determine the SEM from the test-retest data of 286 participants (mean age 38.8, SD 14.7 years; 50%, n=143 of the participants were women). ICCs were odds ratio (OR) 0.813 (95% CI 0.769-0.849), OR 0.791 (95% CI 0.743-0.832), and OR 0.841 (95% CI 0.791-0.864) for daily activity, psychological well-being, and the total JACQLQ Domain II scores, respectively (Multimedia Appendix 5). The SEMs calculated using the ICCs were 4.2, 3.1, and 6.4 for daily activity, psychological well-being, and the total JACQLQ Domain II scores, respectively (Table 5).

The SEMs of the JACQLQ Domain II scores were larger than the 25th percentile of the median anchor-based estimates and 0.5 SD, and the SEMs were selected as the lower bound of the MCID ranges. The final suggested MCID ranges for daily activity, psychological well-being, and the total JACQLQ Domain II scores were 4.2-6.0, 3.1-4.7, and 6.4-10.5, respectively (Table 5).

The implementation of an MCID derived from ePRO data is ideal for assessing subjective symptoms and QoL ascertained by electronic means, despite the similarity to its traditional paper-based counterpart [52]. In this study, we determined the MCID for the app-based JACQLQ Domain II score derived using ePRO data from a previous study that targeted the users of a smartphone app for hay fever research. The MCID reported in this study may improve the evaluation of QoL related to allergic conjunctivitis in patients with hay fever, particularly in remote settings.

This study yielded MCID ranges of 4.2-6.0, 3.1-4.7, and 6.4-10.5 for daily activity, psychological well-being, and the total JACQLQ Domain II scores, respectively. A paper-based JRQLQ study performed in Japan on patients with hay fever who visited a University Hospital [12] previously reported MCID values of 6.0-10.5 for the total QoL score collected from the JRQLQ (mean 8.3; per-item: 0.5). With a total JACQLQ Domain II score-related MCID range of 6.4-10.5 (mean: 8.5; per-item: 0.5), the results of this study are comparable to those of the previous study. Of note, our previous efforts to evaluate and challenge the implementation of the electronic JACQLQ using “AllerSearch” [14] demonstrated that the JACQLQ administered via a smartphone app is sufficiently valid and reliable. This posits a similar potential for the use of app-based and paper-based JACQLQ in clinical practice to assess the QoL affected by hay fever–induced allergic conjunctivitis [14].

In addition to using a large-scale clinical dataset collected through smartphone apps, the MCID was derived from two separate calculations: anchor-based and distribution-based methods [27]. Several internal considerations and recommendations were implemented to improve the accuracy of the MCID calculation. First, the inclusion of PRO data collected from a large pool of participants has been recommended [31], for which our app-based methodology was well-suited. Our study successfully analyzed a large dataset of PRO data provided by 7590 participants. Second, deriving an MCID range by tandem use of both anchor- and distribution-based methods has been recommended for a comprehensive evaluation [27,53], both of which were considered in this study. Furthermore, two distinct anchors, the JACQLQ Domain II face scale and stress level scale, were selected to minimize bias and subjectivity that may stem from a single anchor selected by the research team for the anchor-based calculation. For the distribution-based calculation, both SD and SEM were considered for a multifaceted approach and to better reflect the characteristics of PRO data in the final MCID [4,27]. With the multistep strategy implemented to ensure that the final MCID encapsulates various factors, we believe that the MCID determined in this study is sufficiently accurate for use in a mHealth setting to evaluate subjective symptoms and QoL of hay fever and co-occurring allergic conjunctivitis. Using the MCIDs suggested in this study, providers may be able to accurately assess and compare patients’ dynamic ePROs on hay fever-related symptoms and QoL submitted through commonplace smart devices, which should further enable improved evaluation of treatment responses without necessitating frequent visits to medical facilities.

In this study, we aimed to establish new MCID values for app-based JACQLQ Domain II scores by analyzing a large dataset. The originality of this research lies in the use of crowdsourced data collected through a smartphone-specific QoL assessment tool tailored for allergic conjunctivitis [3,4,37]. This approach allowed us to determine the MCID values suitable for app-based QoL assessments, which could not be achieved using traditional paper-based questionnaires. In addition, by using both anchor-based and distribution-based methods, we were able to derive more robust MCID ranges [27,53]. The MCID ranges for daily activity (4.2-6.0), psychological well-being (3.1-4.7), and total JACQLQ Domain II scores (6.4-10.5) closely matched previously reported values from paper-based assessments, demonstrating their clinical relevance [12]. The newly established MCID values for the app-based JACQLQ Domain II score offer a novel approach for assessing the treatment response in patients with allergic conjunctivitis using smartphone apps, potentially allowing for continuous monitoring and assessment of symptoms without the need for frequent in-person visits. The use of app-based interventions also holds promise for apps in telemedicine [54]. The transition from traditional clinical assessments to remote app-based assessments could increase patient accessibility, facilitate early detection and intervention, help prevent disease progression, potentially reduce health care costs, and improve the overall efficiency of allergic conjunctivitis management [55]. In addition, compared to traditional paper-based methods, using smartphones allows data to be stored within the app or in cloud environments, enabling patients to monitor the progression of their symptoms. Providing patients with feedback on their treatment progress can encourage behavioral changes, leading to improved adherence to hay fever prevention measures and medication [56]. This approach may result in more favorable treatment outcomes than traditional face-to-face consultations [57]. Allergic conjunctivitis caused by hay fever is influenced by environmental factors in the patient’s daily living environment and lifestyle habits. By providing feedback to physicians based on real-time monitoring information obtained from patients through a smartphone app, physicians may be able to offer tailored lifestyle counseling and treatment interventions appropriate for each patient. The integration of smartphone-based assessments and established MCID values provides a more efficient and accessible approach for managing allergic conjunctivitis, potentially improving patient outcomes and streamlining clinical care.

This study has some limitations. First, the data of interest in this study were ePROs, and every variable analyzed to assess hay fever status was based solely on subjective ePRO data. Objective data, such as assessments through physical examination and clinical test results, were not considered. It is unclear from the current results whether a correlation between changes in QoL scores above the MCID and objective data exists. Future studies could further evaluate the validity of the MCID determined in this study by comparing it with objective clinical findings, such as the degree of conjunctival congestion, edema, and follicle presence. Second, the data used to determine the MCID were obtained from “AllerSearch” users in Japan. We recommend careful consideration of the user interface and layout of any non-“AllerSearch” app-based JACQLQ, as well as a target population with a high proportion of non-Japanese individuals when applying the reported MCIDs. Third, a significant portion of our ePRO data points, such as those on subjective symptoms and QoL related to hay fever, were collected between February and March. Although the dataset includes ePROs on hay fever data collected throughout the year, it is possible that the MCID reported in this study is weighted toward cedar and cypress pollen–related hay fever symptoms and QoL changes, which represent a major proportion of the Japanese hay fever population. Fourth, the anchor specified in this study was selected based on cross-sectional study data. Therefore, the anchor may not necessarily correlate with longitudinal changes in hay fever symptoms. In addition, the MCID calculated in this study may not be strictly equivalent to the minimally important change. Therefore, it is necessary to evaluate the validity of the MCID calculated in this study in future longitudinal studies. Fifth, this study determined the SEM for the distribution-based method using the ICC. The ICC was derived from comparisons between baseline and next-day measurements to assess the test-retest reliability of the questionnaire while avoiding changes in symptoms caused by external factors such as continued exposure to pollen, changes in medication use, and variations in pollen levels. Our test-retest analysis involves a short interval between measurements, which could lead to the possibility of recall bias. However, because the study participants entered the data voluntarily, they may have reported based on their symptoms at the time rather than their previous answers, potentially reducing the risk of recall bias.

We report MCID ranges of 4.2-6.0, 3.1-4.7, and 6.4-10.5 for daily activity, psychological well-being, and the total JACQLQ Domain II scores, respectively, that have been derived from a large-scale ePRO dataset collected through an in-house smartphone app, “AllerSearch,” and finally calculated using a multifaceted approach. The MCIDs determined in this study potentially have clinical application in the establishment of standard references to enable the remote assessment and monitoring of hay fever and co-occurring allergic conjunctivitis and the implementation of the principles of mHealth in clinical practice. In addition, combining an MCID derived from an entirely ePRO-based methodology with an app-administered JACQLQ could facilitate fully remote monitoring of symptoms and QoL changes in patients with hay fever–related allergic conjunctivitis, and may enable better assessment of day-to-day treatment effects for an extended and improved disease monitoring.