Young adults who smoked cigarettes daily were recruited from the Charleston, South Carolina, community from April 2017 through May 2019 through social media advertisements, fliers, word of mouth, friend referrals, etc. Participants had to meet the following inclusion criteria: (1) being 18-25 years old, (2) smoking at least 5 cigarettes daily for at least 3 months, (3) willing to limit or abstain from other tobacco products, and if applicable, limit or abstain from using cannabis during the study, (4) willing to engage in a 48-hour quit attempt, and (5) having interest in quitting smoking (defined as 5 or above on a 10-point scale). While young adults who vaped nicotine (e-cigarettes) primarily were allowed into the study in the final months of enrollment, these participants (n=7) were removed from this analysis given that CO cannot be used to verify nicotine vaping. Exclusion criteria included (1) any serious or unstable psychiatric or medical disorder or any other concern that may impact safety, compliance, or could confound results, (2) current use of smoking cessation medication, or (3) pregnancy or plans to become pregnant in the next month. There were no exclusions for smartphone access, and participants were loaned a study smartphone, if they did not have one. Results from this study focused on emotion differentiation, and using EMA data affect and craving have been published elsewhere [39].

A total of 62 participants were screened (Figure 1), 39 of whom met full criteria and were randomized 1:1 into the in-person, wCO group (n=19) or the rCO group (n=20). A total of 7 participants withdrew consent (n=5) or were lost to follow-up (n=2) during the study (n=32 study completers).

All study procedures were reviewed and approved by the institutional review board (PRO 00060290) at the Medical University of South Carolina. Written informed consent was obtained from all participants by trained research staff prior to any study procedures being completed. Consent could be withdrawn at any time during the study. The study team followed all procedures to protect participant privacy and confidentiality. All data were deidentified with a participant ID, and all protected health information were kept separate from research records in locked cabinets or password-protected databases and were only accessible to approved research staff. Details of study compensation are provided above.

Standard descriptive statistics were used to summarize demographic and smoking characteristics for this study cohort as well as stratified by group (rCO vs wCO) and were compared across randomized groups using nonparametric tests. Within each study group, agreement between self-reported (EMA) and CO-verified smoking was estimated for each day using Cohen κ statistic [48]. Generalized estimating equations were used to assess group difference in agreement at common time points across both groups (days 9, 14, 21, 28, and 35). Using the logit function for binary responses, agreement (no vs yes) was regressed on group membership and study day (time-varying covariate). We selected correlation structures using quasi-information criterion minimization [49]. Models were fit using exchangeable correlation structures, and model-based means were extracted for day level between group estimates. Figure 2 also shows the number of participants in each group who were retained at study visits, and participants who had CO data available for analyses (note that CO was collected as part of study visits for the wCO group but through the app for the rCO group).

To examine group differences in retention, logistic regression with a sandwich variance estimate [50] was used, and risk ratios (RRs) with 95% CIs are presented. Group comparisons were conducted using a Wilcoxon Rank Sum test statistic. Abstinence was compared between groups using logistic regression with a sandwich variance estimate [50]. Analyses were intent-to-treat, and missing abstinence data were imputed (missing=smoking). All analyses were performed using SAS (version 9.4; SAS Institute Inc), and no adjustments were made to the presented P values.

This study was powered to detect a difference in the percent agreement of 24-hour self-reported and CO-verified abstinence between those randomized to the rCO versus the wCO group. Participants in both groups self-reported cigarettes smoked per day, but the rCO group had 58 possible CO samples during this study, while those in the wCO group had only 4 CO samples collected at in-person visits (Figure 2). As such, percent agreement between CO and self-report was assessed within and between groups at common time points. High agreement within the rCO group was expected due to the inclusion of remote biochemical verification (>93%). This study was powered to detect a percent agreement difference between groups of at least 13%, requiring 45 subjects per group (N=90 total) to detect a difference with a type I error rate of 5%.

The inclusion of frequent remote breath CO capture added to EMA self-report [22] compared to EMA without rCO in young adults during a brief cessation study did not yield statistically different rates of agreement in self-reported smoking, compliance with EMA sessions, or retention. Though not statistically significant, rates of agreement, compliance, and retention were numerically higher in the wCO group. Smoking abstinence, confirmed through breath CO and self-reported smoking, was low, yet numerically higher in the wCO group at day 9 and statistically different at day 35, favoring the wCO group.

Agreement in past 24-hour self-reported smoking and breath CO was poor to moderate across both groups. It was hypothesized that rCO capture would lead to a higher agreement, which was not supported. Indeed, agreement was numerically higher (though not statistically significant) in the wCO group. A related finding was that CO-confirmed abstinence and CO-verified 7-day PPA for all participants revealed numerically lower abstinence in the rCO group compared to the wCO group following the quit attempt and statistically lower abstinence rates at the end of treatment (CO-confirmed abstinence, not 7-day PPA). Lower rates of abstinence in the rCO group cannot be explained by treatment differences or readiness to quit between groups, so it is likely that the accurate collection of abstinence through biochemical verification was the driver of group differences. These results suggest that more frequent CO assessment may provide a more accurate picture of abstinence, though it is also possible that the inclusion of rCO led to measurement reactivity, whereby the process of monitoring may have influenced smoking during the quit attempt and throughout monitoring. Research has shown evidence of measurement reactivity during EMA monitoring of smoking cessation [51], and it is possible that the addition of rCO monitoring in this study compounded this effect. Continual observation of CO-confirmed smoking during a cessation study may have even resulted in an abstinence violation effect [52] or the tendency to use following the violation of an abstinence goal, which could have led to increased smoking as a result of monitoring. rCO may have also been perceived as replacing the need to accurately report smoking since CO values indicate the presence of behavior. Further, motivation to continue frequent CO submissions and self-report may have been reduced after a lapse during the monitoring period. Remote biochemical verification of smoking may be a useful advancement to better detect when relapse has occurred to intervene rapidly, though a dynamic system that tailors the frequency of CO collection based on current smoking status (smoking, abstinence, and times of high risk) may function more effectively.

Study retention following the quit attempt was high (ie, 95%), was not statistically different between groups, and was comparable to other studies using an EMA monitoring protocol leading up to a quit attempt [53,54]. Although not statistically significant, wCO group retention was numerically higher (90% vs 75%) than rCO group retention at the end of the study. This was unexpected given that the inclusion of technology to remotely capture CO was hypothesized to improve retention rates, as in-person visits were not regularly needed. This suggests potential value in the established relationship between staff and participants through in-person visits that may not be replaced through remote means. It is also possible that weekly in-person visits required for the wCO group made study participation more salient and increased commitment to completion. In addition, the added burden of frequent rCO collection and video upload through the app (2× per day for 5 weeks) may have adversely impacted retention. CO collection fatigue may have occurred, which is an important consideration for the inclusion of remote monitoring in future trials. Even so, results indicate that in the short term, the addition of remote biochemical verification to confirm self-reported smoking has the potential to provide fine-grained data during quit attempts without adversely affecting retention.

Finally, EMA compliance was consistent between groups for sessions with predictable delivery times (event-based sessions and evening reports) but not for random prompt sessions, which included a CO sample for the rCO group. Random prompt session compliance was significantly lower in the rCO group, which is likely a function of the increased burden of session completion (ie, time to record and upload the video). Sessions may have been prompted when participants did not have the CO device readily accessible, which may have resulted in skipping the session completely; therefore, missing the self-report data as well as CO collection. Random prompt sessions that require equipment to be available may adversely affect compliance. Particularly for CO, which can detect smoking for 12-24 hours, participant-initiated CO samples may be preferable and yield better compliance rates. Despite the difference in compliance between groups, overall compliance across session types (ie, 71%) was comparable to pooled compliance rates (ie, 75%) found in a recent meta-analysis of EMA studies [55].

This study had several limitations. First, missing data were prevalent, particularly in the rCO group, which affected the ability to detect group differences. This may have been the result of burden associated with logging each cigarette smoked per day for 5 weeks and completing random sessions throughout that time and insufficient compensation to promote continued engagement. Further, because this group was required to provide a CO sample twice per day during the study, the data may not be missing at random, but rather be due to the additional burden to frequent sampling. Second, while all participants were interested in quitting smoking (5 on a 10-point Likert scale to be included) and received treatment (NRT, brief counseling, and incentives), treatment was minimal and may not have been sufficient to promote continued abstinence and sustained motivation throughout the trial. Treatment-seeking participants may be more likely to engage in monitoring activities, particularly if cessation content is provided through the same monitoring app. Third, we were underpowered to detect differences in agreement between self-reported smoking and CO measurement. Due to a slow rate of enrollment and the increased prevalence of nicotine vaping during study recruitment, our proposed sample size was not successfully met. Numeric trends favoring the wCO group on many outcomes were found, which indicates that even with the proposed sample size, our hypotheses may not have been supported. Fourth, while study procedures across experimental conditions were kept similar, they varied between remote versus in-person depending on the condition, which may have contributed to group differences. This difference may have even played a larger role in group differences beyond rCO collection. Finally, those participants with serious or unstable psychiatric disorders were excluded from participating, which limits the generalizability of the study findings. Participants with psychiatric disorders who were deemed stable (ie, stable dose of medication and seeing a provider regularly) were assessed and included, though this highlights the need to focus young adult smoking research on those with co-occurring disorders in order accurately represent the population in research studies.

The goal of this study was to evaluate the benefits of technological advancement for young adult smoking cessation research methods. The rCO group did not show benefit over the in-person group (wCO), and in some cases, outcomes were worse for the rCO group. Remote methods did not translate into improved compliance or retention, though they may provide a more accurate and fine-grained characterization of the quit, lapse, and relapse processes. EMA is an important tool in this regard, as assessment in one’s natural environment maximizes ecological validity [56], yet common EMA methods may suffer other threats to validity, such as intentional or unintentional biases in responding. Using objective measures, such as remote biochemical verification in conjunction with EMA, may be an important way forward in dynamically assessing smoking relapse [22,37]. However, others have warned against the unnecessary inclusion of remote biochemical verification in smoking studies as standard practice when not sufficiently warranted [57]. Results from this study help to inform that debate and suggest that technological enhancement may improve data accuracy through objective measurement but may also introduce compliance barriers that could result in higher rates of missing data. The inclusion of technology to inform smoking cessation research and intervention delivery among young adults should consider (1) the research question and necessity of biochemical verification and then (2) how to seamlessly incorporate monitoring into personalized and dynamic systems to avoid the added burden and detrimental effects to compliance and honesty in self-report to not compromise the goals of this study.