We developed a behavioral intervention called “Baby, Me & NRT” (BMN), which aims to improve pregnant women’s use of, and adherence to, NRT. We collected the data analyzed in this study during three sequential cohort studies in which researchers delivered BMN to pregnant women and sought feedback for intervention optimization. Between cohorts, we made some minor changes to the data collection procedures in response to participant feedback and COVID-19 restrictions.

The methods are fully described in the study protocol [28]. Participants were aged 16 years or over and were less than 25 weeks pregnant, smoked at least one daily cigarette (prepregnancy ≥10 cigarettes), were not currently using an electronic cigarette (e-Cigarette), and agreed to use NRT to try to stop smoking. We aimed to recruit around 40 women across all cohorts. We identified participants as they attended Nottingham University Hospitals Trust antenatal clinics. In the final cohort, due to COVID-19 restrictions on face-to-face recruitment, participants were also identified via Facebook advertisements and remotely from referrals made to the local stop-smoking service. Study dates, recruitment, and follow-up timings per cohort are given in Table S1 of Multimedia Appendix 1. In Cohorts 1 and 2, researchers sought questionnaire data in person or by telephone, whereas in Cohort 3, these data were obtained by online surveys or telephone due to COVID-19 restrictions.

At baseline, we asked women’s age, ethnicity, education, gestation, current and prepregnancy smoking behavior, nicotine dependence [29], previous experience of using NRT, partner smoking status, and concerns and “necessity beliefs” regarding NRT use. We helped participants to install a smartphone app for reporting daily NRT use (“NicUse” app) [30]. At baseline and Day 7 after their quit date (QD), we collected saliva samples for a cotinine assay, using remote methods where necessary.

NRT was offered for 28 days in a manner consistent with the UK standard treatment for smoking cessation in pregnancy [31]. We supplied a daily NRT patch (Nicorette 16-hour 15 mg or 25 mg; NiQuitin 24-hour 14 mg or 21 mg) plus one of the following fast-acting NRTs: Nicorette Cools Lozenges (2 mg or 4 mg), Nicorette Inhalator (15 mg), Nicorette QuickMist mouth spray (1 mg/spray; only in Cohort 3). Researchers advised women to continue using NRT during brief smoking lapses of less than 14 days and, if they preferred, to leave 24-hour patches on overnight. We recorded the type(s) and strengths of NRT accepted by each participant and any subsequent changes in products issued. We did not require participants to return unused NRT or packaging from the NRT they claimed to have used.

All participants gave written informed consent prior to the start of the baseline study appointment. For Cohort 1 and Cohort 2, consent was given face to face. For Cohort 3, due to COVID-19 restrictions, consent was given via an online form. Ethical approval was granted by Nottingham 1 Research Ethics Committee (Integrated Research Application System number 254560; REC reference 9/EM/0193).

In Cohort 1, 50 women were assessed for eligibility and 8 (16%) participated; in Cohort 2, 189 women were assessed and 12 (6%) participated; and in Cohort 3, 199 women were assessed and 20 (10%) participated. Most nonparticipants did not meet the study inclusion criteria, which included willingness to try quitting smoking with NRT plus counseling. After providing baseline data, five women in Cohort 3 took no further part in the study; they were not offered NRT and were not included in analyses. In total, we assessed 438 women for eligibility, 40 consented to join, 35 received counseling and were offered (and accepted) NRT, and 20 provided Day 7 saliva samples. Table 1 provides the participants’ characteristics.

The response rate for providing any NRT adherence data via the app (31/35, 88.6% for both recall periods) was higher than that provided via the retrospective questionnaire (26/35, 74.3% for the Day 7 questionnaire; 24/35, 68.6% for the Day 28 questionnaire; 27/35, 77% for either questionnaire). Among all participants, including those who provided no app data (N=35), the mean percentage of all daily app surveys completed over the Day 7 recall period was 75.1% (SD 35.9) and was 65% (SD 38.6) for the Day 28 recall period. The median number of days on which app surveys were completed among all participants by Day 7 was 7 (IQR 4) and was 24 (IQR 21) by Day 28. Among participants who provided any app data (n=31), the median number of days on which app surveys were completed by Day 7 was 7 (IQR 2) and was 25 (IQR 11) by Day 28. Tables S2 and S3 in Multimedia Appendix 1 show the completeness of app and questionnaire data at the individual participant level for the Day 7 and Day 28 recall periods, respectively. Of 9 participants with no Day 7 questionnaire data, 5 provided app data on a median of 3 days (range 2-5 days). Of 11 participants with no Day 28 questionnaire data, 7 provided app data on a median of 6 days (range 3-19 days).

In pregnant women who were trying to stop smoking, we measured adherence to NRT for 28 days with a daily smartphone app survey and retrospective questionnaires sent at 7 and 28 days following a QD. With an identical maximum value of compensation for participants’ time taken to complete measures (£25 per method), more participants provided some app data than completed either of the questionnaires, and compliance with daily app surveys was encouraging, with respondents completing an average of 25 out of 28 reports. Women reported using NRT on more days and in higher daily doses to questionnaires, and this may have been due to overreporting of NRT use to questionnaires.

As participants were self-selected, pregnant, recruited in a health care setting, and agreed to try using NRT to stop smoking and to receive a bespoke intervention to enhance their NRT use, the generalizability of findings, especially to nonpregnant populations, could be questioned. Participants will probably have been more interested in and motivated to use NRT than other pregnant women, and so may have been more strongly orientated toward recording NRT use as requested. However, one would expect this to increase completion rates for both reporting methods; it is less plausible as an explanation for differences.

The study could be criticized for not randomizing participants to data collection methods. We did not have the resources for a randomized controlled trial solely to determine the best way to collect NRT adherence data. Therefore, we instead interpret differences between within-subject data collected by two methods for a pragmatic study set in a routine clinical setting. The within-subjects design, with questionnaire data collection following app data collection, may mean that study fatigue caused lower questionnaire response rates; however, response rates were similar for Day 7 and Day 28 questionnaires, so this may not have been an issue.

It is possible that different payment schedules affected the relative response rates for questionnaires versus the app. Although the maximum compensation available for data completion was equal and small for each method (£25), questionnaire completion attracted a one-off payment, whereas app completion attracted a small daily amount (£0.50) plus bonuses for complete weeks, with the total to date displayed on the app interface; this could obviously encourage app usage. However, it could be argued that the app required more of participants’ effort for the same overall compensation, and that those failing to report on the app every day might become demotivated. Moreover, payments (shopping vouchers) for both methods were not made until the study ended for a participant. We therefore consider it unlikely that this can explain the greater response rate for the app, although it is likely that responses for both methods would have been lower given no compensation for participants’ time.

The study could also be criticized for not validating reported NRT use other than via exploratory comparison with salivary cotinine concentrations. Participants were not asked to return nonused NRT or empty NRT packaging as evidence of having used NRT. Although such measures have occasionally been used to validate reports of adherence to smoking cessation treatments [20], there is no guarantee that participants would actually have used nonreturned NRT, and there is currently no “gold-standard” means for measuring adherence. As compensation for completion of measures was offered regardless of responses given (eg, NRT used or not), there was no financial motivation for participants to deliberately misreport using NRT.

The principal finding that, compared to app reporting, questionnaire data may overreport adherence is based on the repeated observation of instances in which participants specifically reported not using NRT to the app on one or more days, but later reported to questionnaires that NRT was used on these days. Additionally, some participants appeared to report very high mean daily nicotine doses to the questionnaire but not to the app. It is plausible that app data may have underestimated NRT use, as we assumed that no NRT was used on days when no app report was made; however, we found clear examples where participants reported greater NRT use to questionnaires when app data were present for the same period. It is possible that, relative to the app, social desirability may have affected questionnaire data collection carried out by telephone (ie, in Cohorts 1 and 2), although when questionnaire data were collected by online survey in Cohort 3, the tendency to report having used NRT on all 28 days was still high. Hence, we think it probable that app data reported a maximum of 48 hours after NRT use were more accurate than questionnaire data recalled 7 or 28 days later, and the skew toward questionnaire reports of NRT use on all days in a recall period reflects this effect. Given the study design, there may be alternative explanations; however, as recall of events is probably better closer to their occurrence [27], we think that this is the most plausible explanation.

The exploratory nature of comparisons between average daily nicotine dose and Day 7 cotinine levels should be emphasized. Cotinine has a 19-hour half-life and can remain in saliva for up to 1 week after nicotine exposure; hence, comparing mean daily nicotine doses over the previous 7 days with saliva cotinine concentrations at Day 7 is logical. However, due to accelerated nicotine metabolism in pregnancy, the cotinine half-life of women in this study could be as short as ~9 hours [14], and we cannot be sure how this might have affected the findings. The sample size for this analysis was too small to permit adjusting for potential confounders of cotinine concentrations, such as partner smoking status or nicotine metabolic rate; thus, the findings from these analyses need to be treated with caution. Numbers of participants not smoking “even a puff” between QD and Day 7 were very low, and therefore our correlations based on more than 10 participants relied on adjusting for the numbers of daily cigarettes smoked, which was self-reported and therefore potentially prone to error. In this study, however, we found collection of data on cotinine concentrations to be feasible and, given larger sample sizes, this could be a useful method for validating self-report data on nicotine use in future studies.

Study originality is a strength; few studies have addressed how to best measure adherence to smoking cessation medication, and no such study has been conducted within pregnant women to date [20]. We believe this is the first comparison of methods for NRT adherence measurement, and the first to investigate the relationships between self-report measures of daily NRT dose used and saliva cotinine concentrations. Although in the latter analyses correlation coefficients were imprecise, these are the first and are therefore the best coefficient estimates available. The hard-to-reach nature of the study population (people who smoke in pregnancy) is another study strength as remote data collection methods are ideally suited to such populations, suggesting ecological validity. Additionally, it is plausible that the study results and data collection methods could be generalized/adapted to other medications; medication adherence remains a vital area of health research.

Daily reporting of NRT adherence via an app may have several potential advantages over retrospective reporting at study follow-ups. First, the accuracy of participant recall is likely to be better with shorter intervals since a behavior occurred; this is central to the EMA approach to data collection, which aims to minimize recall bias and maximize ecological validity [27]. Without keeping records, participants may struggle to accurately recall the number of days on which they used NRT given longer time intervals. This seems to have occurred in this study as, for some participants, higher retrospectively reported NRT usage was explicitly contradicted by daily app reports of days when no NRT was used, particularly at Day 28. It is also possible that, with retrospective measures, participants are unclear about the exact period they are supposed to be reporting for, particularly if there is a delay in completion of measures. The precise amounts of fast-acting NRT used are likely to be even more difficult to recall or prone to error. Several large outliers were evident for the Day 7 questionnaire measure of NRT dosage relative to the app, and it is possible that these participants reported a weekly total rather than daily average. Consequently, daily adherence reporting may be preferential for measuring adherence to NRT in trials, which often have treatment periods of 4 weeks or more and where fast-acting NRT might be used, and an app provides an ideal tool for collecting daily measurements.

A second potential advantage is that app data are available to researchers as soon as they are submitted. This means that, with daily app reports, participants lost to follow-up may still provide useful adherence data right up until the point of dropout or withdrawal, whereas periodic or end-of-study measures can only be completed by participants retained in studies. In this study, more than half of participants with no questionnaire data had provided some app data, particularly within the first 7 days. This could permit better estimates of treatment adherence in trials and provide informative data on those who withdraw.

One potential problem with daily adherence reporting is that monitoring one’s own treatment adherence (“self-monitoring”) could make adherence more likely; that is, the act of measurement may affect the behavior [20,27]. However, in a research setting, this can likely be minimized by storing no usage data on the app, so that users cannot review their adherence with treatment, and by configuring apps to provide no other feedback on adherence to users. Enabling self-monitoring of NRT use via the app may be a desirable feature to incorporate in a clinical setting. Another important criterion to consider when judging the utility of a reporting measure, as well as data accuracy and completeness, is participant burden. EMA data such as daily app reports are likely to be more accurate but may also be more burdensome for participants than less frequent questionnaires, and a balance must be made between these needs. Less frequent questionnaires may be adequate in studies if they are found to replicate EMA data closely, and weekly questionnaires are likely to be more valid than monthly questionnaires.

Using a smartphone app to request daily information on NRT use may provide more complete and valid data on adherence to NRT than using periodically administered questionnaires with longer recall periods. Although this analysis was conducted within a small sample of pregnant women, it seems likely that NRT usage reporting patterns will be similar for nonpregnant people, and apps could provide better data than questionnaires for other drug treatments.