This prospective study was approved by the medical ethics committee of MUMC+ (Ref. No. 2019-1261).

Patients with premenopausal endometriosis aged ≥18 years, diagnosed by physical examination, imaging techniques, or laparoscopy, were recruited at the outpatient gynecology department at MUMC+, a tertiary referral center, between December 2019 and November 2020. Furthermore, patients were recruited by advertisement on the website of the Dutch Endometriosis Foundation. Subjects were eligible for inclusion if they reported one of the endometriosis-related pain symptoms (ie, dysmenorrhea, pelvic pain, or dyspareunia) on average at least 1 day per week during the last 3 months. Pregnant women and patients with any other organic explanation for CPP were not eligible for participation. Participants had to be able to understand written Dutch, and written informed consent was obtained from all participants before participation.

As the identical ESM questionnaire is repeatedly answered several times a day, we expected each patient to complete on average 5 of 10 assessments [13]. Power calculation resulted in a minimum of 25 patients who needed to complete the study protocol to validate this tool [30]. All analyses were performed using SPSS (SPSS Statistics for Windows, Version 25.0, Released 2017; IBM Corp.).

The compliance was calculated as the percentage of ESM questionnaires that were completed by all participants during the study period, per day, and measurement moment. Concurrent validity was assessed by comparing ESM scores with end-of-day pain scores (ie, on day level; no repeated measures within the day) and with end-of-week pain scores (ie, on week level; no repeated measures within the week). To compare ESM scores with end-of-day pain scores, the mean and maximum scores for the ESM were calculated for each of the 7 days, that is, combining all repeated measurements of each subject for the concerning day. Associations between end-of-day scores and ESM scores were tested using linear mixed effects models with the end-of-day score as the dependent and the mean ESM score as the independent variable, a random intercept, corrected for repeated measures using a first-order autoregressive (AR1) correlation structure. The level of agreement between end-of-day and ESM scores was evaluated by calculating intraclass correlation coefficients (ICCs), based on a single-rating consistency 2-way model. The ICCs that were calculated in this study were used to measure agreement between different scores that aimed to measure the same construct, such as end-of-week scores compared with mean ESM scores. To compare ESM scores with end-of-week questionnaire scores, all ESM measures were averaged to 1 score per subject. Differences between the measurement methods were tested using paired sample t tests, and Pearson correlation coefficients (r) were calculated. A value of r greater than 0.7 is considered a strong correlation, between 0.4 and 0.7 a moderate correlation, and anything less than 0.4 is considered a weak or no correlation [31]. For this matter, however, we notice that the paired t test cannot cope with the multilevel structure of the ESM pain scores. Nevertheless, to the best our knowledge, there is no alternative analysis that could cope with this while also comparing the multiple ESM scores with a day/week mean score. Results are displayed for creating insight; no conclusions will be made from significance testing.

The internal consistency of the ESM-PROM was evaluated by dividing the items into 4 symptom domains and calculating the Cronbach coefficient per domain.

In total, 64 women were recruited. Thirty-six patients were excluded as they did not meet inclusion criteria (n=6), declined participation (n=9), did not respond to e-mail or phone calls after providing the patient information leaflet (n=17), or had to quit the study because of technical phone problems (n=4). Of 28 patients who completed the study, 15 were recruited at the outpatient clinic and 13 were recruited by advertisement. Baseline characteristics are listed in Table 2.

Of all ESM morning questionnaires, 87.2% were completed and those took on average 18 seconds to complete. The completion rate of the ESM beep assessments was 52.1%, which corresponds to a mean number of completed assessments of 37 out of 70 per individual over the 7 days. It took participants on average 2 minutes 11 seconds to complete a beep questionnaire. The lowest number of completed measurements per subject was 19; the highest number was 63. Table 3 shows the mean percentage of completed ESM assessments per study day. The response rate was highest on the first 2 study days, with on average 65% of completed assessments. During the study period, the compliance decreased with the lowest number of completed assessments on the last study day (34%). Table 4 shows the mean percentage of completed ESM assessments per measurement moment during the day, with 1 indicating the first assessment of the day (morning) and 10 the last assessment of the day (evening). Response rates fluctuated between 39% and 56% during the day.

In total, 1.8% of questionnaires were started but not completed. The end-of-day pain questionnaire (on paper) was on average completed by 67.6%, and all participants completed the end-of-week questionnaire.

To determine the internal consistency of the ESM-PROM for endometriosis, items were categorized into 4 constructs: abdominal symptoms, general somatic symptoms, positive affect, and negative affect. Cronbach coefficients for each of these domains are shown in Textbox 1. Results demonstrate a good internal consistency for abdominal symptoms, general somatic symptoms, and positive affect, and an excellent internal consistency for negative affect.

This study evaluated the concurrent validity and internal consistency of a previously developed PROM based on the ESM principle for momentary symptom assessment in endometriosis [13]. In this study, we demonstrated that ESM abdominal pain scores in women with endometriosis are strongly correlated with the corresponding end-of-day and end-of-week pain scores. However, end-of-week scores were significantly higher than mean ESM scores but did not differ significantly from the maximum ESM level during the day. This indicates peak reporting of pain scores after a period of recall. Although we notice that the paired t test cannot cope with the multilevel structure of the ESM pain scores and results should be interpreted with caution, the phenomenon of peak reporting was also described earlier in an irritable bowel syndrome population [16].

ESM scores showed strong concurrent validity when compared with symptoms scored by the GSRS-IBS, GAD-7, and PHQ-9, except for a significant moderate correlation for the symptom “feeling down.” This is probably due to the use of different words to express a depressed mood by the ESM (“feeling dispirited, down”) versus PHQ (“feeling down, depressed, hopeless”). Furthermore, the present mental state of a person filling in a retrospective questionnaire might impact the result (psychological bias). The correlation for “feeling down” as assessed by ESM and EHP-30 was strong. Concerning the comparison of the ESM with EHP-30, social and behavioral items were less strongly correlated than the occurrence and severity of physical and psychological symptoms. For example, nonsignificant correlations between the ESM and EHP-30 were found for the experience of being unable to do household chores, difficulty standing, and the need to rest because of symptoms. This could be explained by the fact that social or behavioral factors are less suitable for psychometric evaluation as they are not scored on a fixed scale but either occur or do not. In addition, behavioral and social items might be impacted more strongly by ecological and recall biases. This strengthens the advantages of the ESM tool, as social and behavioral responses to symptoms are important to estimate disease burden. Cronbach coefficients demonstrated a good internal consistency for abdominal symptoms, general somatic symptoms and positive affect, and an excellent internal consistency for negative affect within the ESM-PROM. As the ESM is meant to measure fluctuations within symptoms and experiences, a test-retest reliability is not a good measure of reproducibility of ESM data. Therefore, no test-retest reliability analyses were performed.

A strength of this study is that the evaluated tool was previously designed based on specific recommendations for PROM development [32]. Only 1.8% of questionnaires were started but not completed, indicating that the questionnaire was not too time-consuming. A large number of repeated measures significantly increase the power of the analyses, and the criteria for power calculation, as advised by Schuster et al [30], were met. As the ESM questionnaire is made available by a smartphone app, a potential limitation is that the compliance depends on technical issues, such as signal blocking and network failure by the data company. However, the app does not need an internet connection as it functions offline. Smartphone holders using a specific type of smartphone experienced blocking of ESM messages that could not be addressed, and thus, they could not complete the study (n=4). Another drawback of the ESM is the patient burden, as the repeated assessments during the day are more time-consuming than most conventional assessment methods. This may result in low compliance.

However, in this study, 52% of the total assessments were completed, which is considerably higher than the generally accepted completion rates of 33% in other studies [33,34]. Although 2 of 28 individual patients with endometriosis completed only 19 measurements out of 70 (=27.1% and therefore did not meet the 33% criterion), they were not excluded from the study. Furthermore, a variable number of questions of the ESM assessment due to trigger questions could result in giving negative answers to prevent follow-up questions and therefore win time. As opposed to frequently used static measures, which describe health status at a specific time point, the ESM provides dynamic assessments of symptomatology, allowing for the detection of symptom fluctuation and symptom formation, that is, how symptoms impact symptoms. Given that endometriosis is a heterogeneous disorder often presenting with psychological disturbances or general somatic complaints, which is also reflected by lower levels of quality of life [9], items were included on mental state and frequently reported physical complaints in addition to endometriosis-specific symptoms. Furthermore, environmental and behavioral items were added as they represent and influence disease burden. The assessment of patients’ mental states using the ESM has been thoroughly described in psychiatric and psychosomatic research, also concerning transdiagnostic issues [14,17-24,35,36]. Accordingly, the ESM yields detailed data of the dynamics of endometriosis symptoms in daily life, which could be used to support self-insight, shared decision-making, and prediction of treatment satisfaction in clinical practice. This may also lead to individualized treatment strategies that can improve quality of life and decrease costs [14,17]. The present endometriosis-specific ESM questionnaire was developed in Dutch and was officially translated by following the translation guidelines [37]. Official English language validation is currently planned.

This study shows the validity and reliability of the newly developed electronic instrument for the momentary measurement of symptoms in women with endometriosis, based on the ESM. Compliance for answering the ESM questions was as high as 52%. We demonstrated peak reporting of pain scores after a period of recall since end-of-week pain scores were significantly higher than mean ESM scores but did not differ significantly from the maximum ESM level during the day. This ESM-PROM has the advantage of providing a more detailed view on individual symptom patterns, with the option to analyze interactions between symptoms, the psychological state, and environmental factors, to aid disease monitoring in both clinical practice and research settings. The ESM could offer the possibility of self-insight of patients in their symptomatology, leading to more individualized treatment strategies and shared decision-making in treatment choices, which can considerably improve the quality of life of women suffering from endometriosis.