Background: Knowledge of mental distress and resilience factors over the time span from before to after a stressor is important to be able to leverage the most promising resilience factors and promote mental health at the right time. To shed light on this topic, we designed the RESIST (Resilience Study) study, in which we assessed medical students before, during, and after their yearly exam period. Exam time is generally a period of notable stress among medical students, and it has been suggested that exam time triggers mental distress.
Objective: In this paper, we aim to describe the study protocol and to examine whether the exam period indeed induces higher perceived stress and mental distress. We also aim to explore whether perceived stress and mental distress coevolve in response to exams.
Methods: RESIST is a cohort study in which exam stress functions as a within-subject natural stress manipulation. In this paper, we outline the sample (N=451), procedure, assessed measures (including demographics, perceived stress, mental distress, 13 resilience factors, and adversity), and ethical considerations. Moreover, we conducted a series of latent growth models and bivariate latent change score models to analyze perceived stress and mental distress changes over the 3 time points.
Results: We found that perceived stress and mental distress increased from the time before the exams to the exam period and decreased after the exams to a lower level than before the exams. Our findings further suggest that higher mental distress before exams increased the risk of developing more perceived stress during exams. Higher perceived stress during exams, in turn, increased the risk of experiencing a less successful (or quick) recovery of mental distress after exams.
Conclusions: As expected, the exam period caused a temporary increase in perceived stress and mental distress. Therefore, the RESIST study lends itself well to exploring resilience factors in response to naturally occurring exam stress. Such knowledge will eventually help researchers to find out which resilience factors lend themselves best as prevention targets and which lend themselves best as treatment targets for the mitigation of mental health problems that are triggered or accelerated by natural exam stress. The findings from the RESIST study may therefore inform student support services, mental health services, and resilience theory.
Approximately 1 in 5 young people experience mental distress in the form of anxiety and depression [, ]. According to the World Health Organization (ie, the Regional Office for Europe), the “early identification of such problems—and, when necessary, early intervention or timely management—is critically important...In the absence of appropriate support and intervention, such problems may continue, worsen or lead to mental illness” [ ]. Resilience factors (RFs), such as self-esteem and friendship support, mitigate mental distress in the face of stressful experiences [ ]. The literature contains a considerable amount of knowledge on RFs that mitigate concurrent and subsequent mental distress [ - ]. Yet, studies investigating RFs over the time span from before to after a stressor (ie, stress that causes or triggers mental distress) are scarce [ ]. However, knowledge of mental distress and RFs before and during the stressor is crucial, as this (1) is necessary to determine whether mental distress and RFs are affected by the stressor [ , ], and (2) enables the identification of those RFs that are potentially promising prevention targets. Knowledge on mental distress and RFs during and after the stressor is equally essential, as this (1) enables us to identify whether mental distress and RFs recover after the stressor [ ], and (2) indicates which RFs may be promising treatment targets at times of stress. To this end, we designed the RESIST (Resilience Study) study, in which we assessed perceived stress, mental distress, and RFs in Cambridge University medical students before, during, and after their yearly exam period.
A recent meta-analysis based on 122,356 medical students from 43 countries showed that the prevalence rate for depressive symptoms was 27.2% (range of individual studies: 1.4%-73.5%) . This prevalence rate was higher than that for population-representative peers of a similar age [ ], suggesting that medical students are a high-risk population. In addition to depression, anxiety and general distress levels were also found to be elevated in students pursuing medical degrees, when compared with population-representative samples [ ]. Exam stress has been identified as a potential trigger for mental ill-health in medical students [ - ]. Hence, research suggests that medical students are prone to the development of mental health problems, particularly during times of high and unavoidable exam stress.
The RESIST study is designed to capture (1) a period of moderate stress during the university term several months before exams, (2) a period of high stress during exam time, and (3) a period of what we expected to be low or moderate stress after exams (ie, during the summer vacation for many students). In addition to perceived stress and mental distress, we assessed 8 putative individual-level RFs (eg, self-esteem), 5 putative family-level RFs (eg, parental involvement), and 1 putative community-level RF (eg, friendship support; a complete list of assessed RFs is provided in the Methods section). Importantly, all of these RFs are derived from our preregistered systematic review and are thus empirically supported . In our review, RFs were defined as those factors that moderate and/or mediate, and thereby mitigate, the detrimental relationship between adversity and subsequent mental distress [ ]. Moreover, all assessed RFs are expected to be amenable to intervention, as only those can be successfully targeted by mental health services [ ].
With the RESIST study, we intend to shed light on which RFs lend themselves best as prevention targets (before the stressor) and which lend themselves best as treatment targets (at times of stress) for the mitigation of mental health problems that are triggered or accelerated by a natural stressor. Therefore, the RESIST study may lay the foundations necessary to inform student support services, mental health services, as well as resilience and transdiagnostic mental health theory. Given that our design relies strongly on the assumption that stress and mental distress levels increase during the exam period, we here conduct proof-of-principle analyses to investigate whether this is indeed the case.
RESIST is a cohort study with 3 occasions and a within-subject (natural) stress manipulation (ie, the exam period). Occasion 1 took place in a nonexam period during the university term (February and March 2018). Occasion 2 took place during the end-of-year exam period (approximately April to June 2018, depending on the timing of the exam period). Occasion 3 took place after the exam period, at the end of the term for year 6 students (for whom exams are earlier; approximately end of May to mid-July 2018), and in the summer vacation or autumn for year 1-5 students (approximately mid-August to mid-October 2018;). At all 3 occasions, students were asked to complete a survey containing a series of web-based questionnaires. At occasion 2, students were provided with the questionnaires 3 weeks before their first final exam. The questionnaires had to be completed before attempting the last final exam. In this way, all participants were exposed to the same type of naturally occurring external stressor. We assessed perceived stress, mental distress, RFs, therapeutic treatment, and psychopharmacology usage on all 3 occasions. Past adversity, year of academic education, age, gender, ethnicity, and parental educational level were assessed at occasion 1, and adversity occurring in between the occasions was assessed at occasions 2 and 3 ( ).
We recruited first- to sixth-year Cambridge medical students (from a cohort of approximately 1464 students). The inclusion criterion was that students had to be aged at least 18 years. Participants received monetary reimbursement for partaking (web-based vouchers: £5 [US $6.75] for occasion 1, £7 [US $9.50] for occasion 2, and £5 [US $6.75] for occasion 3). Participants who completed all 3 occasions were additionally enrolled in a prize draw (prize: five £50 [US $67.50] web-based vouchers). The maximum possible sample size we could have included was 800 participants, as we had a limited amount of money that we could spend on participant reimbursement. As a minimum sample size, we aimed for 225 participants. This is because we calculated that for a Gaussian regression-based model (minimum sample size = [((p × (p − 1))/2) × 5] [- ]) with 9 RFs and mental distress (resulting in p=10 variables) we would need at least 225 participants ([((10 × (10 − 1))/2) × 5] = 225), given no longitudinal missingness (which was highly unlikely). Eventually, 451 participants took part on occasion 1 but some dropped out on the other 2 occasions (occasion 2: n=275; occasion 3: n=283; ).
|Occasion 1, n (%)||Occasion 2a, n (%)||Occasion 3a, n (%)|
|Taken part on, at least, 1 occasion||451 (100)||275 (61.0)||283 (62.8)|
|Taken part on, at least, 2 occasions||324 (71.8)||275 (61.0)||283 (62.8)|
|Taken part on occasion 2 but not 3||41 (9.1)||41 (9.1)||N/Ab|
|Taken part on occasion 3 but not 2||49 (10.9)||N/A||49 (10.9)|
|Taken part on all occasions||234 (51.9)||234 (51.9)||234 (51.9)|
aOn occasions 2 and 3, only participants who had already taken part on occasion 1 were invited.
bN/A: not applicable.
The students received a web-based link to the questionnaire (survey software: REDCap [Research Electronic Data Capture]) via email. To prevent double partaking, we sent personalized emails with unique links to the students. We also advertised the study during lectures. Students who had not already participated received reminder emails until the end of the study occasion. For the first occasion, the link expired after 8 weeks. We sent the link for the second occasion approximately 3 weeks before the students’ first final exam and asked the students to confirm that they will submit the survey before their last final exam of the academic year. For year 6 students, the study link for the third occasion was sent out approximately a month before the end of the summer term. For year 1-5 students, the study link for the third occasion was sent 6 weeks before the start of the new academic year (the link for the third occasion expired after 8 weeks for all students).
Before we finalized the web-based survey, we performed a user review with volunteering medical students. On the basis of this pilot study, we evaluated whether the survey was easily understandable and acceptable. We adapted small features, mainly regarding the survey layout (importantly, these data were not part of the study). On occasion 1, a total of 139 items were assessed.
Demographic and Clinical Characteristics
We assessed 8 demographic and clinical variables: academic course, year of academic education, gender, age, ethnicity, parental educational level, psychotherapeutic treatment, and psychopharmacology intake (ie, prescribed drugs).
We assessed the stress level during the last month using a 4-item short form of the validated Perceived Stress Scale (PSS) . The self-report items assess topics such as confidence in handling problems and overcoming difficulties [ ]. The short form of the PSS has been reported to have a Cronbach α of .72 [ ]. In our sample, the PSS had an acceptable reliability (Cronbach α=.75; coefficient Ω=0.75). Moreover, we assessed the global stress severity during the last month on a zero-to-hundred slider.
We assessed general mental health using the 12-item version of the General Health Questionnaire (GHQ-12) . The GHQ-12 provides a broad indication of mental health and well-being across the spectrum from good-to-poor mental health but does not act as a measure of diagnosis for mental illness. The self-report items assess topics such as concentration, sleep, or happiness (measured on a 4-point scale). In a previous study, the GHQ was found to have a Cronbach α of between .78 and .95 [ ]. The GHQ-12 previously had a mean area under the Receiver Operating Characteristic curves of 0.88 [ ]. In our sample, the GHQ-12, in the remainder referred to as mental distress, had a good reliability (Cronbach α=.88; coefficient Ω=0.89).
RFs: Individual Level
We assessed 7 individual-level RFs that were empirically supported in our systematic review : high distress tolerance, low ruminative reflection, low ruminative brooding, high self-esteem, high cognitive reappraisal, low expressive suppression, and low aggression potential. The content and psychometric details are shown in . In , we report Cronbach α values for previous studies, given that this was the reported internal consistency metric. However, for RESIST we report both Cronbach α and coefficient Ω [ ], for completeness.
|RFsa||Content and psychometric information|
|High distress tolerance|
|Low ruminative reflection|
|Low ruminative brooding|
|High cognitive reappraisal|
|Low expressive suppression|
|Low aggression potential|
|High immediate family support|
|High extended family support|
|High family cohesion|
|High positive parenting|
|High parental involvement|
|High friendship support|
aRF: resilience factor.
bDTS: Distress Tolerance Scale.
cRESIST: Resilience Study.
do1: occasion 1.
eo2: occasion 2.
fo3: occasion 3.
gRRS: Ruminative Response Scale.
hRSES: Rosenberg Self-Esteem Scale.
iERQ: Emotion Regulation Questionnaire.
jBAQ: Brief Aggression Questionnaire.
kPSS-Fa: Perceived Social Support from Family Scale.
lKSS: Kinship Social Support Measure.
mSFI-II: Self-Report Family Inventory Version II.
nAPQ: Alabama Parenting Questionnaire.
oPSS-Fr: Perceived Social Support from Friends Scale.
RFs: Family Level
We assessed 5 family-level RFs that were empirically supported in our systematic review : high immediate family support, high extended family support, high family cohesion, high positive parenting, and high parental involvement. The content and psychometric details are shown in . As items of the family-related scales may be hard to answer for participants if they have spent a large amount of their childhood in care homes or frequently changed foster families, we added some specific instructions to those survey parts (Supplement I in ).
RFs: Community Level
We assessed 1 community-level resilience factor, which was empirically supported in our systematic review : high social support, here specifically friendship support. The content and psychometric details are shown in .
Environmental childhood and youth adversity was assessed using an updated version of the 12-item Youth Trauma Scale (YTS) . The 12 self-report items assess topics such as sexual abuse or severe mental or physical illnesses within the family. A complete list of items and assessment details can be found in Supplement II in . The original scale was found to have a low internal consistency (Cronbach α=.63-.67). We adapted the questionnaire [ ] so that the words parent and sibling were supplemented with significant other, as our participants were at an age at which some might have invested in significant interpersonal relationships outside the family, such as long-term romantic partners. Besides the presence versus absence of adversity, the questionnaire also assessed the severity of questions for which the presence of the respective adversity was positively confirmed. Moreover, for positively confirmed questions, we further assessed the frequency or duration of the adversity as well as during which age bins the adversity had taken place. We did not assess the frequency or duration for two of the adversity items: (1) “Were you separated from one of your parents for more than 1 year?” and (2) “Was either of your parents unemployed for more than 1 year when they wanted to be working?” as those have an inherent time requirement of at least one year. To also assess the potential criminality of parents, siblings, or significant others, we added such an item to the original scale (“Did parents, siblings, or significant others engage in criminal activities severe enough to cause significant stress or worry?”). In summary, we assessed the presence versus absence of the 13 adversities, and if present, the severity, frequency, and the age bin in which the respective adversity experience occurred. Owing to the adaptations we made, we will thoroughly evaluate the psychometrics of the amended scale in a separate manuscript [ ].
Furthermore, we assessed the psychological maltreatment and neglect subscales of the Comprehensive Child Maltreatment Scale (CCMS) . The psychological maltreatment subscale consists of 3 items that assess topics such as how frequently the individual was yelled at, ridiculed, or provoked [ ]. The neglect subscale consists of 3 items that assess topics such as whether the individual was provided with sufficient warmth from family members, sufficient nutrition, and protection [ ]. In a previous study, the CCMS psychological maltreatment subscale was found to have a Cronbach α of .78, and the CCMS neglect subscale had a Cronbach α of .84 [ ]. The only adaptation we made to this scale was that we did not assess the questions up to the age of 13 years [ ] but up to the age of 18 years. In our sample, the CCMS psychological maltreatment (Cronbach α=.81; coefficient Ω=0.81) and the CCMS neglect subscale both had a good reliability (Cronbach α=.84; coefficient Ω=0.81).
Measures for Occasion 2
We assessed psychotherapeutic treatment and the use of psychopharmacological drugs for the period between occasions 1 and 2.
We assessed perceived stress (Cronbach α=.74; coefficient Ω=0.75) and global stress severity in the same way as described for occasion 1, while this time specifically focusing on the exam period. Moreover, we quantified the number of exams (completed and not yet completed). The mental distress (Cronbach α=.89; coefficient Ω=0.89) and RF levels (for reliability coefficients, see) were assessed in the same way as on occasion 1.
Adversity was again assessed with the updated version of the YTS ; however, this time we only asked for experiences during the period between occasions 1 and 2 (“This section will ask you about your experiences of potentially traumatic events. Please indicate whether you have experienced those since the last time you filled in this questionnaire.”). Importantly, all the YTS adaptations explained above were applied again. We did not reassess the CCMS subscales [ ], as we used the psychological maltreatment and neglect subscales to measure maltreatment during childhood and the teenage years, while living at home. Therefore, the subscales were not suitable for the time between occasions 1 and 2.
A total of 127 items were assessed.
Measures for Occasion 3
We assessed psychotherapeutic treatment and the use of psychopharmacological drugs for the period between occasions 2 and 3.
We assessed perceived stress (Cronbach α=.74; coefficient Ω=0.75) and global stress severity in the same way as described for occasion 1. Moreover, we asked the students whether they had stressful or significant work during the last 4 weeks. Mental distress (Cronbach α=.90; coefficient Ω=0.91) and RF levels (for reliability coefficients, see) were assessed in the same way as on occasions 1 and 2.
Adversity was again assessed with the updated version of the YTS , which this time asked for experiences during the period between occasions 2 and 3. All YTS adaptations explained for occasions 1 and 2 were applied again. We did not reassess the CCMS subscales [ ] for the same reason as on occasion 2.
As occasion 2 took place during an exam period, we assumed that it could potentially be the case that missingness may not be completely random but dependent on the students’ stress level during occasion 2. Therefore, we asked all participants how stressful the exam period had been.
A total of 129 items were assessed.
Informed Consent and Safety Considerations
Before starting the content part of the web-based survey, participants were asked to read the information sheet, which contained the major study aims, and to complete a consent form. Before completion of the survey, participants were enabled to download their consent form and the information sheet. Moreover, we provided details on how to get help and support, in case the study would bring up difficult feelings or in case a participant would want to report childhood maltreatment or a crime, in a mental health services information sheet, which could be downloaded from the web-based survey. Further details regarding participant safety considerations are provided in Supplement III in.
Ethics Approval and Funding Information
RESIST was approved by the Cambridge Psychology Research Ethics Committee (PRE.2017.096). RESIST was funded by JF’s Medical Research Council Doctoral Training Grant and by POW’s personal research account.
Analytic Methods for the Proof-of-Principle Analyses
Handling Missing Data
To include both participants with incomplete and complete data, we used a full information maximum likelihood (FIML) estimator. The use of this estimator has been shown to function well in longitudinal structural equation models. For example, Kievit et al  report that “FIML usually performs as well or better than alternative methods.” We decided to treat missing data, instead of performing a complete case analysis, to increase statistical power, reduce standard errors, decrease the probability of biased parameter estimates, and improve generalizability. We tested several potential variables as predictors for missing data patterns, including (1) perceived stress, global stress severity (ie, stress slider), mental distress, psychotherapeutic treatment, and psychopharmacological treatment on all 3 occasions; (2) gender, ethnicity, academic year, age, and childhood adversity on occasion 1; and (3) retrospective subjective stress levels for occasion 2 assessed on occasion 3. This was primarily done to enhance the understanding of missingness in the RESIST sample, and because variables that qualify as predictors for missing data patterns can, in conjunction with an FIML estimator, be used as auxiliary variables and thereby potentially enhance the estimation precision.
Latent Growth Models
We conducted a series of latent growth models (LGMs) to explore the mean change trajectory of perceived stress and mental distress over the 3 occasions. We fixed the slope loading of occasion 2 to 1, expecting this occasion to have the highest level of perceived stress (and mental distress), and the slope loading of occasion 3 to 0, expecting this occasion to have a lower level of perceived stress (and mental distress) than occasion 2. Hence, the slope loading of occasion 1 was freely estimated and provides an indication of where the (scaled) mean level lies in comparison to occasion 2 (fixed to 1) and occasion 3 (fixed to 0). We conducted the LGMs with invariant residual variances for the 3 occasions (M1). To test whether our latent growth model is significantly different from a no-change trajectory (ie, no change in overall mean levels), we estimated an additional LGM (M2) with the latent slope mean set to 0, modeling no overall change. In sum, we mainly used the models to identify the change trajectories of perceived stress and mental distress and to test whether these trajectories fit better than a no-change trajectory. For completeness, we refitted the LGMs with freely estimated residual variances (Supplement IV in).
Bivariate Latent Change Score Models
We conducted a series of bivariate latent change score models (BLCSMs; as described by Kievit et al ) to investigate the change of perceived stress and mental distress in conjunction. More specifically, we conducted 3 BLCSMs, 1 for each pairwise combination of the 3 occasions, to allow for direct comparisons without estimating overly complex models. To enable the computation of the models, we used the standard BLCSM estimation (additional details are provided by Kievit et al [ ]). We then investigated the relationship between perceived stress and mental distress on the earlier occasion as well as the relationship between the change scores of perceived stress and mental distress on the later occasion. Moreover, we investigated the autoregressive paths of perceived stress and mental distress with their respective change scores as well as the potentially mutualistic relationship between perceived stress and mental distress, that is, perceived stress predicting change in mental distress and mental distress predicting change in perceived stress.
Data and Analysis-Code Availability
All analyses were conducted in R version 3.5.1 (The R Foundation) , mainly using the packages lavaan [ ] and semTools [ ]. The analysis script can be found on the Open Science Framework [ ] and the anonymized data used for the analyses in this manuscript have been uploaded to the Cambridge Data Repository [ ].
Demographic and Clinical Characteristics
Students were approximately uniformly distributed over all 6 academic years, with percentages ranging from 12.4% (56/451) to 20.6% (93/451) per year. A total of 57.4% (259/451) of the students were female (1.3% [6/451] preferred not to answer) and 58.3% (263/451) were White. Most students were between 18 and 23 years of age and had parents with higher education after secondary school. About 13.5% (61/451) of the students received psychotherapeutic treatment and 10.9% (49/451) received psychopharmaceutic treatment in the 6 months before occasion 1.contains the descriptive statistics for all students who took part on occasion 1. Supplement V in contains the same table, with the inclusion of all students who provided data for at least two occasions (n=324).
|Characteristics||Sample size per answer category, n (%)|
|First year||93 (20.6)|
|Second year||83 (18.4)|
|Third year||66 (14.6)|
|Fourth year||88 (19.5)|
|Fifth year||56 (12.4)|
|Sixth year||65 (14.4)|
|Prefer not to say||6 (1.3)|
|Therapeutic treatment (in the 6 months before occasion 1)|
|Psychopharmaceutic treatment (in the 6 months before occasion 1)|
|Education (further or higher education after secondary school)|
aOne student did not answer this question. Due to missingness, some percentages may not add up.
bThree students did not answer this question. Due to missingness, some percentages may not add up.
cFour students did not answer this question. Due to missingness, some percentages may not add up.
Perceived Stress and Mental Distress Variables
The mean levels suggest that perceived stress and mental distress increased from the time before the exams to the exam period and decreased after the exams to a lower level than before the exams.depicts perceived stress and mental distress levels on the 3 occasions for participants who took part in the respective occasion. The corresponding table with only those participants who provided data for at least two occasions can be found in Supplement V in .
|Measure||Occasion 1||Occasion 2||Occasion 3|
|Score, mean (SD)||Participants, n||Score, mean (SD)||Participants, n||Score, mean (SD)||Participants, n|
|PSSa||10.42 (2.77)||451||11.61 (2.77)||274||9.89 (2.67)||282|
|GHQ-12b||25.40 (5.82)||445||27.39 (6.09)||273||23.31 (5.93)||282|
aPSS: Perceived Stress Scale.
bGHQ-12: General Health Questionnaire, 12-item version.
Gender and ethnicity as well as psychopharmacological medication and global stress severity (ie, stress slider) on occasion 1 were identified as predictors for missing data patterns (see test results in Supplement VI in). Therefore, we decided to include these as auxiliary variables in our analyses. However, as adding the auxiliary variables to our models did not result in positive definite residual matrices, we here describe the models without auxiliary variables. We provide the results with auxiliary variables in Supplement VI in . Importantly, all results remained largely the same when auxiliary variables were added to the models. Moreover, as additional robustness analysis, we reconducted all main analyses, while this time excluding one potentially influential case. Once again, all results remained largely the same (Supplement VII in ).
Latent Growth Models
The LGM showed that, on average, students experience most perceived stress during exams (occasion 2: slope loading fixed to 1) and least perceived stress after the exams (occasion 3: slope loading fixed to 0); before the exams, they experienced more perceived stress than after the exams, but less than during the exams (occasion 1: estimated slope loading=0.29;). The mean level trajectory is shown in (left panel). We further found that the model estimating the change trajectory of perceived stress fits significantly better than the no-change model (χ21=72.4; P<.001; fit indices are presented in ), indicating that perceived stress changed significantly over the 3 occasions. Additional post-hoc analyses confirmed that the mean levels differed significantly between all three occasions.
|Intercept mean||Slope mean||Residual variance, occasion 1||Residual variance, occasion 2||Residual variance, occasion 3||Intercept slope covariance|
aM1: freely estimated trajectory model.
bM2: no-change trajectory model.
cVariance for the latent slope constrained to >0 to render it nonnegative.
|Model||AICa||BICb||CFIc||TLId||RMSEAe||SRMRf||Chi-square (df)||BICwg (%)||AICwh (%)|
aAIC: Akaike information criterion.
bBIC: Bayesian information criterion.
cCFI: confirmatory fit index.
dTLI: Tucker-Lewis fit index.
eRMSEA: root mean square error of approximation.
fSRMR: standardized root mean square residual.
gBICw%: weight percentage for the Bayesian information criterion (compared to the respective other model); the higher the weight, the more in favor is the model.
hAICw%: weight percentage for the Akaike information criterion (compared to the respective other model); the higher the weight, the more in favor is the model.
iM1: freely estimated trajectory model.
jM2: no-change trajectory model.
kVariance for the latent slope constrained to >0 to render it nonnegative.
The LGM showed that, on average, students experience most mental distress during exams (occasion 2, slope loading fixed to 1) and least mental distress after exams (occasion 3, slope loading fixed to 0); before the exams, they experienced more mental distress than after the exams, but less than during the exams (occasion 1, estimated slope loading=0.60;). The mean level trajectory is shown in (right panel). We further found that the model estimating the change trajectory of mental distress fits significantly better than the no-change model (χ21=58.8; P<.001; fit indices are presented in ), indicating that mental distress changed significantly over the 3 occasions. Additional post-hoc analyses confirmed that the mean levels differed significantly between all three occasions.
Bivariate Latent Change Score Models
The BLCSM showed that perceived stress and mental distress on occasion 1 are significantly positively associated, which indicates that students with higher perceived stress report on average higher mental distress (, upper-left panel; for exact coefficients, see Supplement VIII in ). Similarly, changes in perceived stress and mental distress from occasion 1 to 2 are significantly positively associated, which indicates that students with more increase in perceived stress report on average also a greater increase in mental distress. The BLCSM revealed a significant and negative autoregressive effect for perceived stress and its change score, which indicates that higher perceived stress on occasion 1 results, on average, in less increase in perceived stress from occasion 1 to 2. Equally, mental distress had a significant and negative autoregressive effect with its change score, indicating that higher mental distress on occasion 1 results, on average, in less increase in mental distress from occasion 1 to 2. Perceived stress was not significantly associated with the change in mental distress from occasion 1 to 2. However, mental distress was significantly positively associated with the change in perceived stress from occasion 1 to 2, which indicates that higher mental distress on occasion 1 results on average a greater increase in perceived stress from occasion 1 to 2.
The BLCSM showed that changes in perceived stress and mental distress from occasion 1 to 3 are significantly positively associated, which indicates that individuals with a greater decrease in perceived stress report, on average, also a greater decrease in mental distress (, upper-right panel; for exact coefficients, see Supplement VIII in ). Moreover, the BLCSM revealed a significant and negative autoregressive effect for perceived stress and its change score, which indicates that higher perceived stress on occasion 1 results, on average, in a greater decrease in perceived stress from occasion 1 to 3. Equally, mental distress had a significant and negative autoregressive effect with its change score, indicating that higher mental distress on occasion 1 results, on average, in a greater decrease in mental distress from occasion 1 to 3. Mental distress was not significantly associated with change in perceived stress from occasion 1 to 3. Perceived stress was not significantly associated with change in mental distress from occasion 1 to 3; however, the P value was marginal (β=.15; P=.06). If one would opt to interpret the directionality of the effect, it would suggest that, on average, higher perceived stress is marginally associated with less decrease in mental distress from occasion 1 to 3. However, given that the effect was not significant, we suggest to err on the side of caution and shall not interpret it.
The BLCSM showed that perceived stress and mental distress on occasion 2 are significantly positively associated, which indicates that individuals with higher perceived stress report on average higher mental distress (, lower-left panel; for exact coefficients, see Supplement VIII in ). Similarly, changes in perceived stress and mental distress from occasion 2 to 3 are significantly positively associated, which indicates that individuals with a greater decrease in perceived stress report, on average, also a greater decrease in mental distress. Moreover, the BLCSM revealed a significant and negative autoregressive effect for perceived stress and its change score, which indicates that higher perceived stress on occasion 2 results on average in a greater decrease in perceived stress from occasion 2 to 3. Equally, mental distress had a significant and negative autoregressive effect with its change score, indicating that higher mental distress on occasion 2 results on average in a greater decrease in mental distress from occasion 2 to 3. Mental distress was not significantly associated with change in perceived stress from occasion 2 to 3. However, perceived stress was significantly positively associated with change in mental distress from occasion 2 to 3, which indicates that higher perceived stress results on average in less decrease in mental distress from occasion 2 to 3.
Conclusions: Proof-of-Principle Analyses
Both perceived stress and mental distress were lower before the exams (ie, during the regular university term) than during the exam period, but higher before the exams than after the exams. Higher mental distress during term time was, on average, associated with a greater increase in perceived stress from the term time to the exam period, when controlling for perceived stress levels during the term time. Hence, students who already had mental health problems before the exam period were most prone to develop increased levels of stress during the exam period. Higher perceived stress during the exam period was, on average, associated with less recovery of mental distress after the exam time, when controlling for mental distress levels during exams. Thus, students who reported high stress during the exam period were less successful (or quick) in recovering from mental distress. Overall, we found that higher mental health problems before the exams increase the risk of developing more perceived stress during the exams, and higher perceived stress during the exams in turn increases the risk of a less successful (or quick) recovery of mental distress after exams.
Future Research and Outcomes
Plans for Research Questions and Analyses
Future analyses on the RESIST data are primarily set out to shed light on which RFs lend themselves best as prevention targets (before the stressor) and which as treatment targets (at times of stress) for the mitigation of mental health problems that are triggered or accelerated by natural exam stress. Therefore, the RESIST study may lay the foundations necessary to inform student support services, as well as mental health services, as well as resilience and transdiagnostic mental health theory.
Plans for Outcome Dissemination and Data Availability
We aim to publish all articles that are based on RESIST data in peer-reviewed journals, ideally under an open access agreement. Alongside the manuscripts, we aim to release the related and anonymized data on the Cambridge Data Repository.
This manuscript is based on the related ethics proposal coauthored by AVH and POW. The authors want to thank Pascal Schlechter (former MPhil student) for his help with the anonymization of the data and with the participant payments. They also want to thank all participants as well as the students who participated in the user review (ie, the pilot run). Moreover, the authors thank Dr Diana Wood and Dr Robbie Duschinsky, who additionally approved the ethics application as external scientists, and Dr Thelma Quince, who commented on an early sketch of the ethics application. The application was approved on December 20, 2017, by the Cambridge Psychology Research Ethics Committee (approval number: PRE.2017.096).
Data were collected and managed using REDCap  tools hosted at the University of Cambridge, supported by the BRC. REDCap is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources.
JS received support from the National Institute for Health Research, Collaboration for Leadership in Applied Health Research and Care – East of England, at the Cambridgeshire and Peterborough NHS Foundation Trust. RAK was funded by Medical Research Council Programme Grant (grant SUAG/047 G101400) and a Radboudumc Hypatia Fellowship. AVH was funded by the Royal Society Dorothy Hodgkin Fellowship and the Leiden Social Resilience and Securtity Programme. JF was funded by the Medical Research Council Doctoral Training/Sackler Fund and the Pinsent Darwin Fund.
JS and RAK served as statistical advisers. AVH served as a topic-related adviser. POW obtained approval for the study from the Cambridge Medical School, helped recruit the participants, and served as principal investigator. AVH and POW approved the ethics protocol before submission. JF designed the study, wrote the ethics protocol, created the study materials (eg, the web-based survey), recruited the participants, conducted the analyses, and wrote the first draft of this manuscript. All authors commented on several previous versions and approved the final version of this manuscript. AVH and POW are the joint last authors of this manuscript.
Conflicts of Interest
JS disclosed consultation for Ieso Digital Health. None of the authors declared competing interests that affect the manuscript.
Contains 8 supplements (ie, Supplement I: Additional information for family-related questionnaires; Supplement II: Items of the adapted version of the Youth Trauma Scale; Supplement III: Participant safety considerations; Supplement IV: Latent Growth Models with varying residual variances; Supplement V: Demographic and clinical characteristics for the sample with data for at least 2 occasions; Supplement VI: Missingness predictors and analyses results when including the auxiliary variables; Supplement VII: Analyses results when excluding one potentially influential case; Supplement VIII: Model summaries and exact coefficients of the Bivariate Latent Change Score Models).PDF File (Adobe PDF File), 470 KB
- Randall C, Corp A, Self A, Office for National Statistics. Measuring national well-being: life in the UK: 2014. Office for National Statistics. 2014 Mar 18. URL: http://www.ons.gov.uk/ons/dcp171766_352740.pdf [accessed 2021-03-19]
- World Health Organization. Adolescent mental health in the European Region. WHO Regional Office for Europe: Factsheet for World Mental Health Day 2018. Copenhagen, Denmark: WHO Regional Office for Europe; 2018. URL: https://www.euro.who.int/__data/assets/pdf_file/0005/383891/adolescent-mh-fs-eng.pdf?ua=1 [accessed 2021-03-19]
- Fritz J, de Graaff AM, Caisley H, van Harmelen AL, Wilkinson PO. A systematic review of amenable resilience factors that moderate and/or mediate the relationship between childhood adversity and mental health in young people. Front Psychiatry 2018;9:230 [FREE Full text] [CrossRef] [Medline]
- Afifi TO, Macmillan HL. Resilience following child maltreatment: a review of protective factors. Can J Psychiatry 2011;56(5):266-272. [CrossRef] [Medline]
- Marriott C, Hamilton-Giachritsis C, Harrop C. Factors promoting resilience following childhood sexual abuse: a structured, narrative review of the literature. Child Abuse Rev 2013;23(1):17-34. [CrossRef]
- Wright BK, Kelsall HL, Sim MR, Clarke DM, Creamer MC. Support mechanisms and vulnerabilities in relation to PTSD in veterans of the Gulf War, Iraq War, and Afghanistan deployments: a systematic review. J Trauma Stress 2013;26(3):310-318. [CrossRef] [Medline]
- Traub F, Boynton-Jarrett R. Modifiable resilience factors to childhood adversity for clinical pediatric practice. Pediatrics 2017;139(5):- [FREE Full text] [CrossRef] [Medline]
- Braithwaite EC, O'Connor RM, Degli-Esposti M, Luke N, Bowes L. Modifiable predictors of depression following childhood maltreatment: a systematic review and meta-analysis. Transl Psychiatry 2017;7:-. [CrossRef]
- Danese A. Annual Research Review: rethinking childhood trauma-new research directions for measurement, study design and analytical strategies. J Child Psychol Psychiatry 2019;61(3):236-250. [CrossRef] [Medline]
- Kalisch R, Müller MB, Tüscher O. A conceptual framework for the neurobiological study of resilience. Behav Brain Sci 2014;38:e92. [CrossRef]
- Masten AS. Resilience in children threatened by extreme adversity: frameworks for research, practice, and translational synergy. Dev Psychopathol 2011;23(2):493-506. [CrossRef]
- Rotenstein LS, Ramos MA, Torre M, Segal JB, Peluso MJ, Guille C, et al. Prevalence of depression, depressive symptoms, and suicidal ideation among medical students: a systematic review and meta-analysis. J Am Med Assoc 2016;316(21):2214-2236 [FREE Full text] [CrossRef] [Medline]
- Dyrbye LN, Thomas MR, Shanafelt TD. Systematic review of depression, anxiety, and other indicators of psychological distress among U.S. and Canadian medical students. Acad Med 2006;81(4):354-373. [CrossRef] [Medline]
- Dyrbye LN, Thomas MR, Shanafelt TD. Medical student distress: causes, consequences, and proposed solutions. Mayo Clin Proc 2005;80(12):1613-1622. [CrossRef] [Medline]
- Yusoff MS, Rahim AF, Baba AA, Ismail SB, Sidi H, Esa AR. Psychological distress of first year medical students who underwent two different admission processes during a stressful period. Sains Malaysiana 2013;42(3):423-428 [FREE Full text]
- Bollen KA. Structural Equations With Latent Variables. New York: John Wiley; 1989:1-528.
- Bentler PM, Chou CP. Practical issues in structural modeling. Sociol Methods Res 1987;16(1):78-117. [CrossRef]
- Wolf EJ, Harrington KM, Clark SL, Miller MW. Sample size requirements for structural equation models: an evaluation of power, bias, and solution propriety. Educ Psychol Meas 2013;76(6):913-934 [FREE Full text] [CrossRef] [Medline]
- Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983;24(4):385-396. [Medline]
- Hankins M. The factor structure of the twelve item General Health Questionnaire (GHQ-12): the result of negative phrasing? Clin Pract Epidemiol Ment Health 2008;4:10 [FREE Full text] [CrossRef] [Medline]
- Jackson C. The general health questionnaire. Occup Med 2007;57(1):79 [FREE Full text] [CrossRef]
- Goldberg DP, Gater R, Sartorius N, Ustun TB, Piccinelli M, Gureje O, et al. The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychol Med 1997;27(1):191-197. [CrossRef] [Medline]
- Simons JS, Gaher RM. The distress tolerance scale: development and validation of a self-report measure. Motiv Emot 2005;29:83-102. [CrossRef]
- Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: a psychometric analysis. Cognit Ther Res 2003;27:247-259. [CrossRef]
- Rosenberg M. Society and the Adolescent Self-Image. Princeton, NJ: Princeton University Press; 1965.
- Gray-Little B, Williams VS, Hancock TD. An item response theory analysis of the rosenberg self-esteem scale. Pers Soc Psychol Bull 1997;23(5):443-451. [CrossRef]
- Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol 2003;85(2):348-362. [CrossRef] [Medline]
- Gross JJ, John OP. Mapping the domain of expressivity: multimethod evidence for a hierarchical model. J Pers Soc Psychol 1998;74(1):170-191. [CrossRef] [Medline]
- Webster GD, Dewall CN, Pond Jr RS, Deckman T, Jonason PK, Le BM, et al. The brief aggression questionnaire: psychometric and behavioral evidence for an efficient measure of trait aggression. Aggress Behav 2014;40(2):120-139. [CrossRef] [Medline]
- Shahar G, Cohen G, Grogan KE, Barile JP, Henrich CC. Terrorism-related perceived stress, adolescent depression, and social support from friends. Pediatrics 2009;124(2):235-240. [CrossRef] [Medline]
- Procidano ME, Heller K. Measures of perceived social support from friends and from family: three validation studies. Am J Community Psychol 1983;11(1):1-24. [CrossRef] [Medline]
- Taylor RD, Casten R, Flickinger SM. Influence of kinship social support on the parenting experiences and psychosocial adjustment of African-American adolescents. Dev Psychol 1993;29(2):382-388. [CrossRef]
- Boyd RC, Waanders C. Protective factors for depression among African American children of predominantly low-income mothers with depression. J Child Fam Stud 2013;22(1):85-95 [FREE Full text] [CrossRef] [Medline]
- Beaver WR, Hampson RB. Successful Families: Assessment and Intervention. New York: W. W. Norton & Co; 1990.
- Psychometric results. Longscan. URL: [accessed 2017-09-20]
- Shelton KK, Frick PJ, Wootton J. Assessment of parenting practices in families of elementary school-age children. J Clin Child Psychol 1996;25(3):317-329. [CrossRef]
- Essau CA, Sasagawa S, Frick PJ. Psychometric properties of the alabama parenting questionnaire. J Child Fam Stud 2006;15(5):595-614. [CrossRef]
- van Harmelen AL, Kievit RA, Ioannidis K, Neufeld S, Jones PB, Bullmore E, NSPN Consortium, et al. Adolescent friendships predict later resilient functioning across psychosocial domains in a healthy community cohort. Psychol Med 2017;47(13):2312-2322 [FREE Full text] [CrossRef] [Medline]
- McNeish D. Thanks coefficient alpha, we'll take it from here. Psychol Methods 2018;23(3):412-433. [CrossRef] [Medline]
- Schlechter P, Fritz J, Wilkinson PO. The youth and childhood adversity scale: a comparison of presence vs. absence, severity and frequency indicators. OSF Preprints 2019:-. [CrossRef]
- Higgins DJ, McCabe MP. The development of the comprehensive child maltreatment scale. J Fam Stud 2001;7(1):7-28. [CrossRef]
- Kievit RA, Brandmaier AM, Ziegler G, van Harmelen AL, de Mooij SM, Moutoussis M, NSPN Consortium, et al. Developmental cognitive neuroscience using latent change score models: a tutorial and applications. Dev Cogn Neurosci 2018;33:99-117 [FREE Full text] [CrossRef] [Medline]
- R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2018. URL: https://www.R-project.org/ [accessed 2021-03-19]
- Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw 2012;48(2):1-36 [FREE Full text] [CrossRef]
- Jorgensen TD, Pornprasertmanit S, Schoemann AM, Rosseel Y. semTools: useful tools for structural equation modeling. R package version 0.5-1.933 2018:- [FREE Full text]
- Fritz J. Analysis script. OSF Home. 2021 Feb 27. URL: https://osf.io/wa8vc/ [accessed 2021-04-13]
- Fritz J, Stochl J, Kievit R, van Harmelen A, Wilkinson P. Research data supporting RESIST publication "Tracking Stress, Mental Health and Resilience Factors in Medical Students, Before, During and After a Stress-Inducing Exam Period: Protocol and Proof of Principle Analyses for the RESIST Cohort-Study". (Taken from Resilience Study (RESIST): what helps students to adapt to exam stress?). University of Cambridge 2021:-. [CrossRef]
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009 Apr;42(2):377-381 [FREE Full text] [CrossRef] [Medline]
|BLCSM: bivariate latent change score model|
|CCMS: Comprehensive Child Maltreatment Scale|
|FIML: full information maximum likelihood|
|GHQ: General Health Questionnaire|
|LGM: latent growth model|
|PSS: Perceived Stress Scale|
|REDCap: Research Electronic Data Capture|
|RESIST: Resilience Study|
|RF: resilience factor|
|YTS: Youth Trauma Scale|
Edited by G Eysenbach; submitted 12.05.20; peer-reviewed by G Lovell, H Krampe; comments to author 15.06.20; revised version received 31.10.20; accepted 09.11.20; published 08.06.21Copyright
©Jessica Fritz, Jan Stochl, Rogier A Kievit, Anne-Laura van Harmelen, Paul O Wilkinson. Originally published in JMIR Formative Research (https://formative.jmir.org), 08.06.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.