This paper is in the following e-collection/theme issue:

Formative Evaluation of Digital Health Interventions (2226) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1317) mHealth for Data Collection and Research (917) Patient-Reported Outcome Measures (PROMs) (275) Mobile Health (mhealth) (2777)

Published on in Vol 6, No 8 (2022): August

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38475, first published .
Reliability and Validity of Electronic Patient-Reported Outcomes Using the Smartphone App AllerSearch for Hay Fever: Prospective Observational Study

Reliability and Validity of Electronic Patient-Reported Outcomes Using the Smartphone App AllerSearch for Hay Fever: Prospective Observational Study

Reliability and Validity of Electronic Patient-Reported Outcomes Using the Smartphone App AllerSearch for Hay Fever: Prospective Observational Study

Authors of this article:

Yasutsugu Akasaki1, 2 Author Orcid Image ;   Takenori Inomata1, 2, 3 Author Orcid Image ;   Jaemyoung Sung1 Author Orcid Image ;   Yuichi Okumura1, 2 Author Orcid Image ;   Kenta Fujio1, 2 Author Orcid Image ;   Maria Miura1, 2 Author Orcid Image ;   Kunihiko Hirosawa1, 2 Author Orcid Image ;   Masao Iwagami4 Author Orcid Image ;   Masahiro Nakamura5 Author Orcid Image ;   Nobuyuki Ebihara6 Author Orcid Image ;   Masahiro Nakamura7 Author Orcid Image ;   Takuma Ide7 Author Orcid Image ;   Ken Nagino1, 2, 3 Author Orcid Image ;   Akira Murakami1, 2 Author Orcid Image
Article Authors Cited by (7) Tweetations (7) Metrics

    Original Paper

    1Department of Ophthalmology, Juntendo University Graduate School of Medicine, Tokyo, Japan

    2Department of Digital Medicine, Juntendo University Graduate School of Medicine, Tokyo, Japan

    3Department of Hospital Administration, Juntendo University Graduate School of Medicine, Tokyo, Japan

    4Department of Health Services Research, Faculty of Medicine, University of Tsukuba, Ibaraki, Japan

    5Precision Health, Department of Bioengineering, Graduate School of Engineering, The University of Tokyo, Tokyo, Japan

    6Department of Ophthalmology, Urayasu Hospital, Juntendo University, Chiba, Japan

    7Department of Otorhinolaryngology, Head and Neck Surgery, Juntendo University Faculty of Medicine, Tokyo, Japan

    Corresponding Author:

    Takenori Inomata, MBA, MD, PhD

    Department of Ophthalmology

    Juntendo University Graduate School of Medicine

    2-1-1 Hongo, Bunkyo-ku

    Tokyo, 1130033

    Japan

    Phone: 81 338133111

    Email: tinoma@juntendo.ac.jp


    Background: Hay fever is a highly prevalent, heterogenous, and multifactorial disease. Patients may benefit from longitudinal assessments using mobile health (mHealth) principles. We have previously attempted to establish an effective mHealth platform for patients with hay fever through AllerSearch, our in-house smartphone app that assesses electronic patient-reported outcomes through a questionnaire on hay fever and provides evidence-based advice. To be used by the public, an investigation on its reliability and validity is necessary.

    Objective: The aim of this paper is to assess the reliability and validity of subjective symptom data on hay fever collected through our app, AllerSearch.

    Methods: This study used a prospective observational design. The participants were patients aged ≥20 years recruited from a single university hospital between June 2, 2021, and January 26, 2022. We excluded patients who could not use smartphones as well as those with incomplete data records and outlier data. All participants answered the Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire (JACQLQ), first in the paper-and-pencil format and subsequently on AllerSearch on the same day. The JACQLQ comprises the following three domains: Domain I, with 9 items on ocular or nasal symptoms; Domain II, with 17 items on daily activity and psychological well-being; and Domain III, with 3 items on overall condition by face score. The concordance rate of each domain between the 2 platforms was calculated. The internal consistency of Domains I and II of the 2 platforms was assessed using Cronbach alpha coefficients, the concurrent validity of Domains I and II was assessed by calculating Pearson correlation coefficients, and the mean differences between the 2 platforms were assessed using Bland-Altman analysis.

    Results: In total, 22 participants were recruited; the data of 20 (91%) participants were analyzed. The average age was 65.4 (SD 12.8) years, and 80% (16/20) of the participants were women. The concordance rate of Domains I, II, and III between the paper-based and app-based JACQLQ was 0.78, 0.85, and 0.90, respectively. The internal consistency of Domains I and II between the 2 platforms was satisfactory (Cronbach alpha of .964 and .919, respectively). Pearson correlation analysis yielded a significant positive correlation between Domains I and II across the 2 platforms (r=0.920 and r=0.968, respectively). The mean difference in Domains I and II between the 2 platforms was 3.35 units (95% limits of agreement: –6.51 to 13.2).

    Conclusions: Our findings indicate that AllerSearch is a valid and reliable tool for the collection of electronic patient-reported outcomes to assess hay fever, contributing to the advantages of the mHealth platform.

    JMIR Form Res 2022;6(8):e38475

    doi:10.2196/38475

    Keywords

    hay fever; AllerSearch; smartphone app; mobile health; mHealth; patient-reported outcome; reliability; validity; Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire; JACQLQ; questionnaire; allergic conjunctivitis 


    Hay fever is currently believed to be the most common immunologic and allergic disease worldwide, with reports of nearly 30 million cases in the United States and Japan [1-3]. Hay fever symptoms can be chronic and therefore life altering, leading to a decrease in individuals’ quality of life and work productivity [3]. This systemic illness targets multiple organs, most commonly manifesting as allergic rhinitis, conjunctivitis, and dermatitis [4,5]. The disease appears to evolve, changing its presentation with varying onsets, levels of severity, and responses to treatment based on the individual [3,4]. Therefore, a deeper understanding of the underlying pathophysiology and establishing an effective strategy to comprehensively assess changing symptoms become imperative to provide tailored, longitudinal care and to improve patients’ quality of life [4-7].

    Recent findings have increasingly confirmed the advantages of adopting patient-reported outcomes (PROs), which are clinical data grounded in patients’ own subjective experiences that are not readily captured by routine medical evaluations [8,9]. With the recent advancements in mobile health (mHealth), a medical discipline centered around health care and support through advanced mobile devices such as smartphones, the electronic adaptation of PROs (ePROs) has been garnering attention as a novel data accrual option for clinical researchers [4,5,10-12].

    We have previously taken advantage of the novel mHealth platform and conducted studies through our in-house hay fever smartphone app, AllerSearch, released in February 2018 [4,5]. The app successfully gathered comprehensive medical data related to hay fever without interrupting users’ daily lives. By using data collected through AllerSearch, we were able to elucidate various risk factors that could exacerbate the disease, and we stratified the disease into subgroups based on collective symptoms and individual factors [4,5]. In our efforts to implement ePROs via AllerSearch, the app was equipped with features to administer hay fever symptom–related questionnaires, such as the Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire (JACQLQ) [4,5,13,14]. Given the ongoing pandemic and the anticipated postpandemic era, the demand for longitudinal, nonintrusive health care continues to increase, and mHealth appears to hold the key to addressing this need. To realize such nonintrusive care that can also engage the principles of participatory medicine through mHealth, a robust validation of mHealth-accrued clinical data on subjective symptoms and their quantification strategies is required.

    Hence, we evaluated the reliability and validity of the subjective symptom data collected through our mHealth app by conducting a comparative study between paper-based and app-based versions of the JACQLQ to evaluate the applicability of AllerSearch as a novel clinical tool for assessing hay fever.


    AllerSearch Smartphone App

    AllerSearch was initially developed in Japan using Apple Inc’s open-source framework, ResearchKit [4,5], and released on Apple’s App Store on February 1, 2018, under a consignment contract with Juntendo University Graduate School of Medicine and InnoJin, Inc, both based in Tokyo, Japan. The Android version was released on August 26, 2020. The AllerSearch is freely available on the App Store and Google play.

    Design

    This study employed a prospective observational design based on previously published validation studies of medical instruments [15,16].

    Ethical Considerations

    Written informed consent was obtained from all participants prior to the commencement of the study. The study was approved by the Independent Ethics Committee of Juntendo University Graduate School of Medicine (approval number H20-0242-H01, November 6, 2020) and adheres to the tenets of the Declaration of Helsinki.

    Enrollment and Participants

    The participants were patients aged ≥20 years, recruited between June 2, 2021, and January 26, 2022, from the Department of Ophthalmology, Juntendo University Hospital, Tokyo, Japan. We excluded patients who could not use smartphones as well as those with incomplete data records and outlier data.

    All participants answered the paper-based JACQLQ at the outpatient service in the Department of Ophthalmology, Juntendo University Hospital. They subsequently answered the same questionnaire on an iOS version of AllerSearch (app-based JACQLQ) on the same day. AllerSearch had been preinstalled on the mobile phones provided for the purpose of this study. Our previous study contained the description of survey items in AllerSearch [4]. Briefly, participants provided electronic consent and basic information on demographics, medical history, lifestyle, hay fever status, and preventive behavior for hay fever. Subsequently, participants performed daily assessments of their conjunctiva and responded to a questionnaire on hay fever that included the JACQLQ and assessments of nasal symptoms, nonnasal symptoms, daily subjective symptoms, and work productivity.

    JACQLQ

    The JACQLQ is a well-established metric that enables clinicians to comprehensively assess QOL among patients in the Japanese-speaking population who are affected by allergic conjunctival diseases [13]. The JACQLQ comprises the following three domains: Domain I with 9 items on ocular or nasal symptoms; Domain II with 17 items on daily activity and psychological well-being; and Domain III with 3 items on overall condition by face score. The questionnaire requires participants to rate each symptom on a 5-point Likert scale according to its severity, from “None” (0 points) to “Severe” (4 points). The total score (Domains I and II) for the scale and the total score of each domain was calculated as the sum of Domains I and II and the sum of items in each domain, respectively. Of note is that the default settings of the scale bar in AllerSearch, used to represent the 5-point scale, and the face score were both set to the lowest score, but users were able to adjust their ratings to higher scores as they deemed fit.

    Statistical Analysis

    The sample size for the Cronbach alpha test was predetermined based on the formula by Bonett [17]. Using these settings—power=90%, significance level=5%, number of items (k)=26, value of Cronbach alpha at null hypothesis=.0, and expected value of Cronbach alpha=.7—the required sample size was calculated to be 17.08 (rounded up to 18) cases. Accounting for 20% dropouts owing to missing data or withdrawal of consent, the final sample size was 22 cases.

    The median scores for the paper-based and app-based JACQLQ were compared using Wilcoxon matched-pairs signed-rank tests [18,19]. The concordance rate of each item and domain between the 2 platforms was calculated. The internal consistency of the app-based JACQLQ was assessed using Cronbach alpha coefficient, with an alpha score of >.70 considered acceptable [20]. Concurrent validity was assessed by calculating the correlations (Pearson coefficient) and mean differences (Bland-Altman analysis) [15,21].

    Statistical analyses were performed using Stata/MP version 16.1 (Stata Corp) and GraphPad Prism version 9.1.2 (GraphPad Software). Statistical significance was set at P<.05.

    Patient and Public Involvement

    Input on the AllerSearch survey questionnaire was obtained to produce a version that was agreed upon by a committee comprising allergy specialists, ophthalmologists, otolaryngologists, epidemiologists, and the patient and public involvement members [4,5].


    Participant Characteristics

    In total, 22 participants were recruited for this study. Following the exclusion of an individual with incomplete data records and another with outlier data, the data of 20 (91%) participants were analyzed. Table 1 shows the participants’ characteristics. The average age was 65.4 (SD 12.8) years, and 80% (16/20) were female participants. The mean best-corrected visual acuity value for both eyes was –0.06 (SD 0.05). The mean intraocular pressure was 13.9 (SD 2.6) mmHg.

    Table 1. Participants’ characteristics (N=20).
    CharacteristicsValues
    Age (year), mean (SD)65.4 (12.8)
    Gender, n (%)

    		Female16 (80)

    		Male4 (20)
    BCVAa, logMAR (SD)–0.04 (0.07)
    IOPb, mmHg (SD)13.9 (2.6)

    aBCVA: best-corrected visual acuity.

    bIOP: intraocular pressure.

    Scores and Concordance Rate of Paper-Based and App-Based JACQLQ

    The median total score for Domains I and II was 6.5 (range: 1.75-13.25) for the paper-based JACQLQ and 4.5 (range: 1–8) for the app-based JACQLQ (P=.003). Table 2 shows each item’s median score and concordance rate for the paper-based and app-based JACQLQ. The individual total score of Domains I and II was significantly higher in the paper-based JACQLQ compared with the app-based JACQLQ. The concordance rates of each item, subscale, and domain were more than 70%.

    Table 2. JACQLQa item scores and concordance rate between paper-based and app-based JACQLQ.
    JACQLQ itemsPaper-based JACQLQ, median (IQR)App-based JACQLQ, median (IQR)P valueConcordance rate (%)
    Domain I, 0-363.5 (1.75-6)2 (1-4.25).00478

    		Eye symptoms, 0-202 (1-4.25)1.5 (0-3).0182


    		1. Itchy eyes1 (0-1)1 (0-1)>.9995


    		2. Foreign body sensation1 (0-1)0 (0-1).1265


    		3. Red eyes0 (0-1)0 (0-0.25).1380


    		4. Watery eyes0 (0-0)0 (0-0).5090


    		5. Eye discharge0 (0-1)0 (0-0.25).1380

    		Nasal symptoms, 0-161 (0-2.25)0.5 (0-1).0174


    		6. Runny nose0 (0-1)0 (0-0.25).0675


    		7. Sneezing0.5 (0-1)0 (0-0.25).0370


    		8. Stuffy nose0 (0-1)0 (0-0).3875


    		9. Itchy nose0 (0-0.25)0 (0-0).0675
    Domain II, 0-682 (0-5.5)1.5 (0-5.5).0485

    		Daily activity, 0-442 (0-4.25)0.5 (0-2).00285


    		1. Obstacles to studying, working, and housework0 (0-1)0 (0-0).1380


    		2. Poor mental concentration0 (0-1)0 (0-0).5090


    		3. Decreased thinking ability0 (0-0)0 (0-0)>.9985


    		4. Impaired reading newspapers and other materials0 (0-1)0 (0-1).1380


    		5. Poor memory0 (0-1)0 (0-0).5090


    		6. Limitation of outdoor life such as sports and picnics0 (0-0)0 (0-0)>.9985


    		7. Limitation of going out0 (0-0)0 (0-0).3880


    		8. Obstacles to socializing with people0 (0-0)0 (0-0).5090


    		9. Interfering with conversations and telephone calls with others0 (0-0)0 (0-0)>.9990


    		10. Anxiety about people around you0 (0-0)0 (0-0)>.9995


    		11. Sleeping disorder0.5 (0-1)0 (0-1).0370

    		Psychological well-being, 0-240 (0-1.25)0 (0-2)>.9986


    		12. Dullness0 (0-0.25)0 (0-0.25)>.9980


    		13. Fatigue0 (0-0)0 (0-1).7585


    		14. Frustrated0 (0-0)0 (0-0).2585


    		15. Irritable0 (0-0)0 (0-0)>.9995


    		16. Depressed0 (0-0.25)0 (0-0).6380


    		17. Dissatisfaction with life0 (0-0)0 (0-0)>.9990
    Domain III, 0-41 (1-2)1 (1-2)>.9990

    aJACQLQ: Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire.

    Reliability Between App-Based and Paper-Based JACQLQ

    The internal consistency of the total, subscale, and domain scores between the paper-based and app-based JACQLQ is indicated in Table 3. Our results show satisfactory internal consistency for most questionnaire items (Cronbach alpha>.70), except for nasal symptoms in the app-based version (Cronbach alpha=.331).

    Table 3. Reliability between the paper-based and app-based JACQLQa.
    JACQLQNo. of itemsCronbach alpha

    		Paper-basedApp-based
    Domains I and II26.964.919
    Domain I9.897.746

    		Eye symptoms5.856.788

    		Nasal symptoms4.776.331
    DomainII17.953.896

    		Daily activity11.914.915

    		Psychological well-being6.937.731

    aJACQLQ: Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire.

    Correlation Between App-Based and Paper-Based JACQLQ

    Figure 1 shows the correlation between the paper-based and app-based JACQLQ. There were significant positive correlations between the 2 measurements (Domains I and II: r=0.971, P<.001; Domain I: r=0.920, P<.001; and Domain II: r=0.968, P<.001). The x-axis indicates the total score for the paper-based JACQLQ, and the y-axis the total score for the app-based JACQLQ.

    Figure 2 shows the Bland-Altman analysis for the clinical agreement between the paper-based and app-based JACQLQ, revealing a difference (bias) with 95% limits of agreement of 3.35 (–6.51 to 13.2) units for Domains I and II, 1.85 (–3.05 to 6.75) units for Domain I, and 1.50 (–4.28 to 7.28) units for Domain II.

    ‎
    Figure 1. Correlation between the app-based and paper-based Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaires (JACQLQs). The correlation between the paper-based and app-based JACQLQs of Domains I and II, Domain I, and Domain II.
    View this figure
    ‎
    Figure 2. Bland-Altman plot for the paper-based and app-based Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaires (JACQLQs). The x-axis indicates the average of the 2 methods’ scores, and the y-axis indicates the difference between the 2 methods’ scores. The central line indicates the mean difference (bias) between the scores from the 2 methods, whereas the superior and inferior lines depict the intervals, which include the 95% limits of agreement. Differences between the JACQLQ of Domains I and II, Domain I, and Domain II.
    View this figure

    Principal Results

    Hay fever, a highly heterogenous and multifactorial disease, requires personalized assessments to develop effective preventive measures and management strategies. In this study, we examined the reliability and validity of our smartphone app, AllerSearch, regarding collecting data on hay fever symptoms. Our results indicate that the digital administration of the JACQLQ through AllerSearch shows satisfactory reliability and validity metrics; AllerSearch may therefore be an accessible tool for hay fever management. Its accessibility may prove advantageous for screening the undiagnosed population and promoting early, personalized interventions. The COVID-19 pandemic accelerated the breakthrough and subsequent growth of telemedicine and effective self-management. AllerSearch’s ability to assist in the self-management of hay fever, with its extensive reach, aligns well with the aforementioned changing medical paradigm.

    Our results show satisfactory internal consistency for most questionnaire items (Cronbach alpha>.70), except for nasal symptoms in the app-based version (Cronbach alpha=.331). Further, there were significant positive correlations between the 2 measurements (Domains I and II: r=0.971, P<.001; Domain I: r=0.920, P<.001; and Domain II: r=0.968; P<.001). These analyses yielded satisfactory results regarding the reliability and validity metrics of the app-based JACQLQ compared to the paper-based version, suggesting the role of ePROs in the future implementation of mHealth.

    Our results also indicate that the app-based collection of nasal symptoms showed low internal consistency, which may lead to a discrepancy between nasal and nonnasal symptom assessments. However, nonnasal symptoms, as well as overall symptoms, maintained a high internal consistency, and the low internal consistency observed for nasal symptoms may be attributed to the small sample size in this study. Future efforts to increase power should be pursued to verify or improve on the observed low internal consistency for nasal symptoms.

    Traditionally, in-person assessments have not proved very effective in comprehensive evaluations, mostly owing to the low frequency and time constraints of typical outpatient visits [22]. However, as hay fever presents itself as a heterogenous, systemic disease with possible long-term detrimental effects, a holistic evaluation through established questionnaires, such as the JACQLQ, becomes crucial in selecting appropriate treatment regimens. Our findings revealed satisfactory internal consistency and a statistically significant correlation between the paper-based and app-based JACQLQ. It is noteworthy that the Bland-Altman plot analysis on the agreement between the paper-based and app-based JACQLQ resulted in a higher mean (bias) of 3.35 units (95% limits of agreement: –6.51 to 13.2) of the latter compared with the former. We believe that this is owing to a carryover effect stemming from the study design, in which a procedure of the study flow affects another downstream result [23]. We administered the app-based JACQLQ after the paper-based version, and future studies should address the discrepancy through a crossover trial with mixed cohorts on the questionnaire administration order.

    Another explanation for the 3-point mean difference between the 2 platforms could be the length and order of the questionnaire items in the app-based version. Demographic and medical history questions preceded the JACQLQ, which might have led to response fatigue [24], a frequently observed phenomenon with survey-type research methodologies. The app-based JACQLQ, by default, positions the scale bar for responses at 0, which may have predisposed fatigued users to quickly answer the JACQLQ items with a low score [25]. This could partially explain the higher score in the paper-based version, as it does not have a “default” score. For further validation of mHealth-driven ePROs and to minimize response fatigue, trials to reduce the number of questionnaire items and reorganize their sequence may be required. Although response fatigue can be addressed, it is practically inevitable; hence, we suggest that the psychological aspects and the resultant discrepancy based on the temporality of the answered items that may have affected the study’s results should be considered. Future studies should also address the interface-led bias and discrepancies between the digital and paper questionnaires, one of which may call for a distinct cutoff score in the digital version for diagnoses and severity assessments. The 3-point mean difference, which does not appear highly relevant from a clinical perspective at this stage, and the consistent correlation between the 2 platforms suggest that ePROs collected through AllerSearch may be valid and feasible for assessing hay fever symptoms and advising on self-management.

    Limitations

    This study has several limitations. First, there may be a degree of selection bias stemming from the participants’ demographics, including age and gender. This was also a single-center study, making the selection process prone to selection bias. Further, while there has not been a study, to the best of our knowledge, comparing paper-based and app-based questionnaires, this study had a smaller sample size in comparison to previous studies that investigated discrepancies between digital and paper questionnaires [18,24,26]. Therefore, greater sample sizes are needed for generalization. Lastly, this study did not involve any in-person clinical evaluations on allergic conjunctivitis and did not investigate the correlation of clinical findings with the JACQLQ results obtained through AllerSearch. Therefore, any capability of AllerSearch regarding allergic conjunctivitis diagnosis and screening should not be inferred from our results.

    Conclusions

    Our findings indicate that the data collected through the AllerSearch app had good internal consistency, with a Cronbach alpha of >.70 and significant positive correlations between the paper-based and app-based JACQLQ (Domains I and II: r=0.971; Domain I: r=0.920; and Domain II: r=0.968), as an instrument for hay fever symptom management. The mHealth-based PRO enables tailored, longitudinal data-based hay fever management and helps improve patients’ quality of life.

    Acknowledgments

    We thank Drs Fujisawa K, Muto K, Nojiri S, Nagao M, and Okano M, and the members of our patient panel for advising us on how to improve our research project, and Medical Logue, Inc (Tokyo, Japan) for developing and releasing the Android and updated versions of AllerSearch.

    Data Availability

    All data generated or analyzed during this study are included in this published article.

    Authors' Contributions

    Conceptualization was implemented by, T Inomata; methodology by YA and T Inomata; software by YA; validation by YA and T Inomata; formal analysis by YA; investigation by YA, T Inomata, YO, KF, MM, and KH; resources by T Inomata; data curation by YA; writing—original draft preparation by YA, T Inomata, and JS; writing—review and editing by YO, KF, MM, KH, MI, MM, NE, MN, T Ide, KN, and AM; visualization by YA; supervision by AM; project administration by T Inomata; and funding acquisition by T Inomata. All authors have read and agreed to the published version of the manuscript.

    This research was funded by the Japan Agency for Medical Research and Development, grant number JP21ek0410063, the Institute for Environmental and Gender-Specific Medicine, Juntendo University, and the OTC Self-Medication Promotion Foundation. The funding sources had no role in the study design; collection, analysis, and interpretation of data; writing of the report; and in the decision to submit the article for publication.

    Conflicts of Interest

    AllerSearch was created using Apple’s ResearchKit. T Inomata and YO are the owners of InnoJin, Inc, which developed AllerSearch. YA, JS, KF, MM, KH, MI, NM, NE, MN, T Ide, KN, and AM declare no conflicts of interest.

    1. Meltzer EO. The prevalence and medical and economic impact of allergic rhinitis in the United States. J Allergy Clin Immunol 1997 Jun;99(6 Pt 2):S805-S828. [Medline]
    2. Skoner DP. Allergic rhinitis: definition, epidemiology, pathophysiology, detection, and diagnosis. J Allergy Clin Immunol 2001 Jul;108(1 Suppl):S2-S8. [CrossRef] [Medline]
    3. Yamada T, Saito H, Fujieda S. Present state of Japanese cedar pollinosis: the national affliction. J Allergy Clin Immunol 2014 Mar;133(3):632-9.e5. [CrossRef] [Medline]
    4. Inomata T, Nakamura M, Iwagami M, Sung J, Nakamura M, Ebihara N, et al. Individual characteristics and associated factors of hay fever: A large-scale mHealth study using AllerSearch. Allergol Int 2022 Jul;71(3):325-334 [FREE Full text] [CrossRef] [Medline]
    5. Inomata T, Nakamura M, Iwagami M, Sung J, Nakamura M, Ebihara N, et al. Symptom-based stratification for hay fever: A crowdsourced study using the smartphone application AllerSearch. Allergy 2021 Dec;76(12):3820-3824. [CrossRef] [Medline]
    6. Adachi T, Ogawa Y, Fukushi T, Ito K, Koizumi A, Shirabe M, et al. Research impact analysis of international funding agencies in the realm of allergy and immunology. Allergy 2022 May;77(5):1602-1606 [FREE Full text] [CrossRef] [Medline]
    7. Adachi T, Kainuma K, Asano K, Amagai M, Arai H, Ishii KJ, et al. Strategic Outlook toward 2030: Japan's research for allergy and immunology - Secondary publication. Allergol Int 2020 Oct;69(4):561-570 [FREE Full text] [CrossRef] [Medline]
    8. Bottomley A, Jones D, Claassens L. Patient-reported outcomes: assessment and current perspectives of the guidelines of the Food and Drug Administration and the reflection paper of the European Medicines Agency. Eur J Cancer 2009 Feb;45(3):347-353. [CrossRef] [Medline]
    9. Okumura Y, Inomata T, Iwata N, Sung J, Fujimoto K, Fujio K, et al. A Review of Dry Eye Questionnaires: Measuring Patient-Reported Outcomes and Health-Related Quality of Life. Diagnostics (Basel) 2020 Aug 05;10(8):559 [FREE Full text] [CrossRef] [Medline]
    10. Inomata T, Nakamura M, Iwagami M, Shiang T, Yoshimura Y, Fujimoto K, et al. Risk Factors for Severe Dry Eye Disease: Crowdsourced Research Using DryEyeRhythm. Ophthalmology 2019 May;126(5):766-768. [CrossRef] [Medline]
    11. Inomata T, Iwagami M, Nakamura M, Shiang T, Yoshimura Y, Fujimoto K, et al. Characteristics and Risk Factors Associated With Diagnosed and Undiagnosed Symptomatic Dry Eye Using a Smartphone Application. JAMA Ophthalmol 2020 Jan 01;138(1):58-68 [FREE Full text] [CrossRef] [Medline]
    12. Inomata T, Nakamura M, Sung J, Midorikawa-Inomata A, Iwagami M, Fujio K, et al. Smartphone-based digital phenotyping for dry eye toward P4 medicine: a crowdsourced cross-sectional study. NPJ Digit Med 2021 Dec 20;4(1):171. [CrossRef] [Medline]
    13. Fukagawa K, Fujishima H, Fukushima A, Sumi T, Okamoto S, Shoji J, et al. [A quality of life questionnaire for Japanese allergic conjunctival disease]. Nippon Ganka Gakkai Zasshi 2012 May;116(5):494-502. [Medline]
    14. Kirtsreesakul V, Somjareonwattana P, Ruttanaphol S. The correlation between nasal symptom and mucociliary clearance in allergic rhinitis. Laryngoscope 2009 Aug;119(8):1458-1462. [CrossRef] [Medline]
    15. Okumura Y, Inomata T, Midorikawa-Inomata A, Sung J, Fujio K, Akasaki Y, et al. DryEyeRhythm: A reliable and valid smartphone application for the diagnosis assistance of dry eye. Ocul Surf 2022 Jul;25:19-25. [CrossRef] [Medline]
    16. Magner K, Bragg D, Treat R, Lee KJ. Understanding mothers' perception of child's illness: Adapting the CONNECT Instrument for pediatrics. J Clin Nurs 2019 Sep;28(17-18):3279-3287. [CrossRef] [Medline]
    17. Bonett DG. Sample Size Requirements for Testing and Estimating Coefficient Alpha. Journal of Educational and Behavioral Statistics 2016 Nov 23;27(4):335-340. [CrossRef]
    18. Pouwer F, Snoek FJ, van der Ploeg HM, Heine RJ, Brand AN. A comparison of the standard and the computerized versions of the Well-being Questionnaire (WBQ) and the Diabetes Treatment Satisfaction Questionnaire (DTSQ). Qual Life Res 1998 Jan;7(1):33-38. [CrossRef] [Medline]
    19. Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm 2008 Dec 01;65(23):2276-2284. [CrossRef] [Medline]
    20. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika 1951 Sep;16(3):297-334. [CrossRef]
    21. Inomata T, Nakamura M, Iwagami M, Midorikawa-Inomata A, Okumura Y, Fujimoto K, et al. Comparing the Japanese Version of the Ocular Surface Disease Index and Dry Eye-Related Quality-of-Life Score for Dry Eye Symptom Assessment. Diagnostics (Basel) 2020 Apr 07;10(4):203 [FREE Full text] [CrossRef] [Medline]
    22. Inomata T, Sung J, Nakamura M, Iwagami M, Okumura Y, Iwata N, et al. Using Medical Big Data to Develop Personalized Medicine for Dry Eye Disease. Cornea 2020 Nov;39 Suppl 1:S39-S46. [CrossRef] [Medline]
    23. Detmar SB, Muller MJ, Schornagel JH, Wever LDV, Aaronson NK. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA 2002 Dec 18;288(23):3027-3034. [CrossRef] [Medline]
    24. Coles ME, Cook LM, Blake TR. Assessing obsessive compulsive symptoms and cognitions on the internet: evidence for the comparability of paper and Internet administration. Behav Res Ther 2007 Sep;45(9):2232-2240. [CrossRef] [Medline]
    25. O'Reilly-Shah VN. Factors influencing healthcare provider respondent fatigue answering a globally administered in-app survey. PeerJ 2017;5:e3785 [FREE Full text] [CrossRef] [Medline]
    26. Ferrando PJ, Lorenzo-Seva U. IRT-related factor analytic procedures for testing the equivalence of paper-and-pencil and Internet-administered questionnaires. Psychol Methods 2005 Jun;10(2):193-205. [CrossRef] [Medline]

    ‎
    e-PRO: electronic patient-related outcome
    JACQLQ: Japanese Allergic Conjunctival Disease Standard Quality of Life Questionnaire
    mHealth: mobile health
    PRO: patient-reported outcome

    Edited by A Mavragani; submitted 04.04.22; peer-reviewed by H Mehdizadeh, O Pavliuk, I Dronyuk, KZ Yap; comments to author 27.07.22; revised version received 08.08.22; accepted 10.08.22; published 23.08.22

    Copyright

    ©Yasutsugu Akasaki, Takenori Inomata, Jaemyoung Sung, Yuichi Okumura, Kenta Fujio, Maria Miura, Kunihiko Hirosawa, Masao Iwagami, Masahiro Nakamura, Nobuyuki Ebihara, Masahiro Nakamura, Takuma Ide, Ken Nagino, Akira Murakami. Originally published in JMIR Formative Research (https://formative.jmir.org), 23.08.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.