This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
The experience sampling method (ESM) holds advantages over traditional retrospective questionnaires including a high ecological validity, no recall bias, the ability to assess fluctuation of symptoms, and the ability to analyze the temporal relationship between variables.
This study aimed to evaluate the psychometric properties of an endometriosis-specific ESM tool.
This is a short-term follow-up prospective study, including patients with premenopausal endometriosis aged ≥18 years who reported dysmenorrhea, chronic pelvic pain, or dyspareunia between December 2019 and November 2020. An ESM-based questionnaire was sent out by a smartphone application 10 times a day during 1 week on randomly chosen moments. Additionally, patients completed questionnaires concerning demographics, end-of-day pain scores, and end-of-week symptom scores. The psychometric evaluation included compliance, concurrent validity, and internal consistency.
Twenty-eight patients with endometriosis completed the study. Compliance for answering the ESM questions was as high as 52%. End-of-week pain scores were higher than ESM mean scores and showed peak reporting. ESM scores showed strong concurrent validity when compared with symptoms scored by the Gastrointestinal Symptom Rating Scale–Irritable Bowel Syndrome, 7-item Generalized Anxiety Disorders Scale, 9-question Patient Health Questionnaire, and the majority of questions of the 30-item Endometriosis Health Profile. Cronbach α coefficients demonstrated a good internal consistency for abdominal symptoms, general somatic symptoms, and positive affect, and an excellent internal consistency for negative affect.
This study supports the validity and reliability of a newly developed electronic instrument for the measurement of symptoms in women with endometriosis, based on momentary assessments. This ESM patient-reported outcome measure has the advantage of providing a more detailed view on individual symptom patterns and offers the possibility for patients to have insight in their symptomatology, leading to more individualized treatment strategies that can improve the quality of life of women with endometriosis.
Endometriosis is defined as an estrogen-dependent condition with endometrium-like tissue outside the uterus [
Endometriosis is currently managed by surgical or medical interventions; however, approximately 50% of women with endometriosis have recurrent symptoms over a period of 5 years, irrespective of the treatment approach [
An endometriosis-specific patient-reported outcome measure (PROM) that uses the experience sampling method (ESM) has recently been developed at the Maastricht University Medical Center+ (MUMC+) [
This prospective study was approved by the medical ethics committee of MUMC+ (Ref. No. 2019-1261).
Patients with premenopausal endometriosis aged ≥18 years, diagnosed by physical examination, imaging techniques, or laparoscopy, were recruited at the outpatient gynecology department at MUMC+, a tertiary referral center, between December 2019 and November 2020. Furthermore, patients were recruited by advertisement on the website of the Dutch Endometriosis Foundation. Subjects were eligible for inclusion if they reported one of the endometriosis-related pain symptoms (ie, dysmenorrhea, pelvic pain, or dyspareunia) on average at least 1 day per week during the last 3 months. Pregnant women and patients with any other organic explanation for CPP were not eligible for participation. Participants had to be able to understand written Dutch, and written informed consent was obtained from all participants before participation.
At baseline, patients were asked to fill in an electronic clinical case report form (eCRF) concerning demographic information and medical history in Castor electronic data capture (EDC) [
The ESM questionnaire contains items concerning endometriosis symptoms, general somatic symptoms, psychological symptoms, and contextual/social information as well as questions concerning nutrition and medication. Depending on previous answers on trigger questions, this questionnaire comprises a minimum of 31 and a maximum of 42 items. Additionally, there is a morning questionnaire that includes questions concerning sleep and sexual behavior, which comprises a minimum of 4 and a maximum of 7 questions [
The Brief Pain Inventory (BPI Short Version—Dutch language) was used as an end-of-day pain diary. An average pain score experienced over the last 24 hours was given on an 11-point NRS (0=
Self-reported questionnaires for retrospective symptom assessment, used for validation of the endometriosis-specific ESMa questionnaire.
Self-reported questionnaire | Content | Questions, n |
Pain on NRSb | Abdominal pain, dysmenorrhea, and dyspareunia | 3 |
EHP-30c [ |
Health-related quality of life for patients with endometriosis. Items assess symptoms on 5 domains: pain, control and powerlessness, social support, emotional well-being, and self-image | 30 |
GSRS-IBSd [ |
Gastrointestinal symptoms. Items assess symptoms of pain, bloating, constipation, diarrhea, and satiety | 13 |
PHQ-9e [ |
Screening for subclinical depressive symptoms | 9 |
GAD-7f [ |
Screening for generalized anxiety disorder and assessment of severity | 7 |
aESM: experience sampling method.
bNRS: Numeric Rating Scale.
cEHP-30: 30-item Endometriosis Health Profile.
dGSRS-IBS: Gastrointestinal Symptom Rating Scale–Irritable Bowel Syndrome.
ePHQ-9: 9-question Patient Health Questionnaire.
fGAD-7: 7-item Generalized Anxiety Disorders Scale.
As the identical ESM questionnaire is repeatedly answered several times a day, we expected each patient to complete on average 5 of 10 assessments [
The compliance was calculated as the percentage of ESM questionnaires that were completed by all participants during the study period, per day, and measurement moment. Concurrent validity was assessed by comparing ESM scores with end-of-day pain scores (ie, on day level; no repeated measures within the day) and with end-of-week pain scores (ie, on week level; no repeated measures within the week). To compare ESM scores with end-of-day pain scores, the mean and maximum scores for the ESM were calculated for each of the 7 days, that is, combining all repeated measurements of each subject for the concerning day. Associations between end-of-day scores and ESM scores were tested using linear mixed effects models with the end-of-day score as the dependent and the mean ESM score as the independent variable, a random intercept, corrected for repeated measures using a first-order autoregressive (AR1) correlation structure. The level of agreement between end-of-day and ESM scores was evaluated by calculating intraclass correlation coefficients (ICCs), based on a single-rating consistency 2-way model. The ICCs that were calculated in this study were used to measure agreement between different scores that aimed to measure the same construct, such as end-of-week scores compared with mean ESM scores. To compare ESM scores with end-of-week questionnaire scores, all ESM measures were averaged to 1 score per subject. Differences between the measurement methods were tested using paired sample
The internal consistency of the ESM-PROM was evaluated by dividing the items into 4 symptom domains and calculating the Cronbach coefficient per domain.
In total, 64 women were recruited. Thirty-six patients were excluded as they did not meet inclusion criteria (n=6), declined participation (n=9), did not respond to e-mail or phone calls after providing the patient information leaflet (n=17), or had to quit the study because of technical phone problems (n=4). Of 28 patients who completed the study, 15 were recruited at the outpatient clinic and 13 were recruited by advertisement. Baseline characteristics are listed in
Baseline characteristics (N=28).
Characteristics | Values | |
Recruited by MUMC+,a n (%) | 15 (54) | |
Recruited by advertisement, n (%) | 13 (46) | |
Age, mean (SD) | 35.46 (9.39) | |
BMI, mean (SD) | 25.97 (5.09) | |
|
||
|
High school | 2 (7) |
|
College or university | 26 (93) |
|
||
|
Student | 2 (7) |
|
Unemployed | 7 (25) |
|
Employed | 19 (68) |
|
||
|
Single | 7 (25) |
|
In relationship | 21 (75) |
|
||
|
0 | 13 (46) |
|
≥1 | 15 (54) |
Having a menstrual cycle, n (%) | 17 (61) | |
|
22 (79) | |
|
Oral contraceptives | 5 (18) |
|
Levonorgestrel-releasing intrauterine system (Mirena) | 9 (32) |
|
Progestins | 8 (29) |
|
GnRHb | 1 (4) |
Regular use of pain medication, n (%) | 20 (71) | |
Smoking, n (%) | 6 (21) | |
Alcohol, n (%) | 21 (75) | |
History of abdominal surgery for endometriosis, n (%) | 20 (71) | |
Fertility treatment, n (%) | 7 (25) | |
Current child wish, n (%) | 6 (21) | |
Unplanned childlessness, n (%) | 4 (14) | |
Traumatic life event, n (%) | 3 (11) |
aMUMC+: Maastricht University Medical Center+.
bGnRH: gonadotropin-releasing hormone.
Of all ESM morning questionnaires, 87.2% were completed and those took on average 18 seconds to complete. The completion rate of the ESM beep assessments was 52.1%, which corresponds to a mean number of completed assessments of 37 out of 70 per individual over the 7 days. It took participants on average 2 minutes 11 seconds to complete a beep questionnaire. The lowest number of completed measurements per subject was 19; the highest number was 63.
In total, 1.8% of questionnaires were started but not completed. The end-of-day pain questionnaire (on paper) was on average completed by 67.6%, and all participants completed the end-of-week questionnaire.
Mean % completed ESMa assessments per study day.
Study days | Mean completion, % |
1 | 64.6 |
2 | 65.4 |
3 | 59.6 |
4 | 56.1 |
5 | 45.7 |
6 | 38.9 |
7 | 34.3 |
aESM: experience sampling method.
Mean % of completed ESMa assessments per measurement moment during the day, averaged for all subjects and all 7 days during the study period.
Measurement moment | Mean completion, % |
1 | 39.29 |
2 | 48.98 |
3 | 52.55 |
4 | 55.61 |
5 | 51.02 |
6 | 52.04 |
7 | 56.63 |
8 | 52.55 |
9 | 58.16 |
10 | 54.08 |
aESM: experience sampling method.
In this analysis, 19 patients were included, as 9 patients did not complete or return the end-of-day questionnaire. The average
Mean difference, association, and intraclass correlation coefficients (ICCs) between end-of-day diary scores and experience sampling method (ESM) mean and maximum scores for abdominal pain in 19 patients.a
|
Mean difference | Association | ICCs | |||||
|
Difference | SE | Estimate | SE | ICC | 95% CI | ||
End-of-day vs ESM |
0.94 | 2.08 | 0.53b | 0.08 | 0.80b | 0.71-0.86 | ||
End-of-day vs ESM |
–0.53 | 0.19 | 0.47 b | 0.06 | 0.81b | 0.73–0.87 |
aMean differences tested using the paired sample
b
The comparison and correlation between ESM and end-of-week pain scores scored on the NRS are shown in
End-of-week pain scores compared with experience sampling method (ESM) scores (mean scores on subject level).a
End-of-week vs ESM | End-of-week score, mean (SD) | ESM score, mean (SD) | Mean difference | SE | Pearson correlation coefficient ( |
Abdominal pain | 4.96 (2.36) | 3.21 (2.99); 4.71 (2.95)b | 1.807; 0.371b | 0.394; 0.524b | 0.821; 0.830b,c |
Dysmenorrhea | 5.50 (2.59) | 4.59 (3.18) | 1.765 | 1.641 | 0.813d |
Dyspareunia | 2.69 (2.36) | 2.63 (2.36) | –0.167 | 0.749 | 0.945c |
aDifferences were tested using the paired sample
bMaximum (SD) values.
c
d
To visualize the comparison between the ESM and end-of-day as well as end-of-week scores in more detail, average abdominal pain scores for the ESM per measurement moment and the end-of-day and end-of-week scores are shown in
Abdominal pain scores (on an 11-point Numeric Rating scale) for ESM, end-of-day, and end-of-week scores over the 7-day study period, averaged for all participants. Each day, 10 assessments were available for the ESM; 1 assessment was completed at the end of each day and 1 assessment was completed at the end of the week. ESM: experience sampling method.
The correlation between ESM and end-of-week scores is shown in
Correlation between end-of-week scores and experience sampling method (ESM) scores (concurrent validity).
Symptoms | Pearson correlation coefficient ( |
|
|
||
|
EHP-30: Been unable to go to social events because of the pain? |
0.520b |
|
EHP-30: Been unable to do jobs around the home because of the pain? |
0.222c |
|
EHP-30: Found it difficult to exercise or do the leisure activities you would like to do because of the pain? |
0.505b |
|
EHP-30: Found it difficult to stand because of the pain? |
0.145c |
|
EHP-30: Found it difficult to sit because of the pain? |
0.554b |
|
EHP-30: Found it difficult to walk because of the pain? |
0.446d |
|
EHP-30: Been unable to sleep properly because of the pain |
0.606b |
|
EHP-30: Had to go to bed/lie down because of the pain? |
0.254c |
|
EHP-30: Been unable to do the things you want to do because of the pain? |
0.527b |
|
EHP-30: Generally felt unwell? |
–0.502b |
|
EHP-30: Felt depressed? |
0.729e |
|
EHP-30: Felt weepy/tearful? |
0.748b |
|
EHP-30: Felt bad-tempered or short-tempered? |
0.729b |
|
EHP-30: Couldn't do certain tasks at work because of the pain? |
0.466d |
|
EHP-30: Experienced pain during or after sexual intercourse? |
0.688d |
|
EHP-30: Has avoided intercourse because of the pain? |
0.466d |
|
||
|
GSRS: Have you been bothered by nausea? |
0.803e |
|
GSRS: Has your stomach felt bloated? |
0.841e |
|
GSRS: Have you suffered from a visibly distended abdomen? |
0.859e |
|
GSRS: Have you been bothered by abdominal pain? |
0.691e |
|
||
|
PHQ-9: Feeling down, depressed, or hopeless |
0.380d |
|
PHQ-9: Feeling tired or having little energy |
–0.778e |
|
GAD-7: feeling nervous, anxious, or on edge |
0.726e |
|
GAD-7: Worrying too much about different things |
0.738e |
|
GAD-7: trouble relaxing |
–0.496b |
|
GAD-7: Becoming easily annoyed or irritable |
0.533b |
aEHP-30: 30-item Endometriosis Health Profile. The questionnaire starts with “During the last 4 weeks, how often, because of your endometriosis, have you….” Scale = 5-point Likert (Never – Always).
b
cNot significant.
d
e
fGSRS-IBS: Gastrointestinal Symptom Rating Scale–Irritable Bowel Syndrome. Questions are asked over the past week. The scale used was a 7-point Likert scale (from no discomfort at all to Very severe discomfort).
gPHQ-9: 9-question Patient Health Questionnaire; GAD-7: 7-item Generalized Anxiety Disorder Scale. The questionnaires start with “Over the last 2 weeks, how often have you been bothered by any of the following problems?” The scale is a 4-point Likert scale (from Not at all to Nearly every day).
To determine the internal consistency of the ESM-PROM for endometriosis, items were categorized into 4 constructs: abdominal symptoms, general somatic symptoms, positive affect, and negative affect. Cronbach coefficients for each of these domains are shown in
Abdominal pain, bloating, distended abdomen, urge to defecate, vaginal blood loss
Dizziness, nausea, headache, muscle pain, shortness of breath
Cheerful, relaxed
Dispirited/down, emotional, stressed, worried, irritable
This study evaluated the concurrent validity and internal consistency of a previously developed PROM based on the ESM principle for momentary symptom assessment in endometriosis [
ESM scores showed strong concurrent validity when compared with symptoms scored by the GSRS-IBS, GAD-7, and PHQ-9, except for a significant moderate correlation for the symptom “feeling down.” This is probably due to the use of different words to express a depressed mood by the ESM (“feeling dispirited, down”) versus PHQ (“feeling down, depressed, hopeless”). Furthermore, the present mental state of a person filling in a retrospective questionnaire might impact the result (psychological bias). The correlation for “feeling down” as assessed by ESM and EHP-30 was strong. Concerning the comparison of the ESM with EHP-30, social and behavioral items were less strongly correlated than the occurrence and severity of physical and psychological symptoms. For example, nonsignificant correlations between the ESM and EHP-30 were found for the experience of being unable to do household chores, difficulty standing, and the need to rest because of symptoms. This could be explained by the fact that social or behavioral factors are less suitable for psychometric evaluation as they are not scored on a fixed scale but either occur or do not. In addition, behavioral and social items might be impacted more strongly by ecological and recall biases. This strengthens the advantages of the ESM tool, as social and behavioral responses to symptoms are important to estimate disease burden. Cronbach coefficients demonstrated a good internal consistency for abdominal symptoms, general somatic symptoms and positive affect, and an excellent internal consistency for negative affect within the ESM-PROM. As the ESM is meant to measure fluctuations within symptoms and experiences, a test-retest reliability is not a good measure of reproducibility of ESM data. Therefore, no test-retest reliability analyses were performed.
A strength of this study is that the evaluated tool was previously designed based on specific recommendations for PROM development [
However, in this study, 52% of the total assessments were completed, which is considerably higher than the generally accepted completion rates of 33% in other studies [
This study shows the validity and reliability of the newly developed electronic instrument for the momentary measurement of symptoms in women with endometriosis, based on the ESM. Compliance for answering the ESM questions was as high as 52%. We demonstrated peak reporting of pain scores after a period of recall since end-of-week pain scores were significantly higher than
Table S1. Set of questions (q) for the endometriosis specific ESM-PROM [
first-order autoregressive
chronic pelvic pain
electronic clinical case report form
electronic data capture
30-item Endometriosis Health Profile
experience sampling method
7-item Generalized Anxiety Disorders Scale
Gastrointestinal Symptom Rating Scale–Irritable Bowel Syndrome
intraclass correlation coefficient
Maastricht Electronic Abdominal Symptom REporting
Maastricht University Medical Center+
Numeric Rating Scale
9-question Patient Health Questionnaire
patient-reported outcome measure
We thank the Dutch Endometriosis Foundation (“Endometriose Stichting”) for insights from patients’ perspectives during the development of this new experience sampling method tool. We also thank MEMIC, the Maastricht University Medical Center (MUMC) for data and information management, for their help in developing the endometriosis-specific MEASuRE (Maastricht Electronic Abdominal Symptom REporting) application. This research was financed by crowdfunding organized by the Limburg University Fund/Stichting Wetenschappelijk Onderzoek Limburg (SWOL). The mission of the Limburg University Fund/SWOL is to support scientific research and education at Maastricht University.
None declared.