Participants were recruited via emails and phone calls between October 2020 and September 2022 from the electronic medical records of the New York University Langone Health (NYULH) system, a private hospital system serving New York, New Jersey, and Connecticut with approximately 6.8 million active patients. Patients were eligible for participation if they (1) had an ambulatory International Statistical Classification of Diseases, Tenth Revision (ICD-10) diagnosis code for COPD, cardiovascular disease, or asthma in the last 12 months; (2) for COPD patients only, a COPD Assessment Test (CAT) score ≥10; (3) were aged 21 to 75 years (the legal age for purchasing ECs in New York is 21); (4) were a current CC user (≥5 packs in a lifetime, smokes ≥4 days/week); (5) smoked at least 5 CCs per day on days they smoke CCs; (6) were motivated to quit smoking (at least a 5 on a 10-point Contemplation Ladder) [12]; and (7) possessed a phone with SMS text messaging capabilities. Potential participants were excluded if they (1), for COPD patients only, had a CAT score ≥30 (representing severe COPD) [13]; (2) reported NRT or EC use in the last 14 days; (3) had a medical condition (eg, unstable angina or heart disease) precluding use of nicotine patch or gum as determined by the study physician or by their treating physician; or (4) were pregnant or breastfeeding. Individuals who were not open to using NRT or ECs were also excluded from the study. Participants using other tobacco products were not excluded from this study.

The NYU Langone Health Institutional Review Board (IRB) approved the study procedures (s20-00839) and written documentation of informed consent was received before starting data collection. The original informed consent allows for secondary analyses without additional consent. All participant data were deidentified prior to analysis. Participants did not receive any incentive for counseling calls but were incentivized to complete study visits and respond to the daily SMS text message smoking diary. Participants received US $10 for completion of a survey at baseline and at 6, 8, and 12 weeks, and US $20 for completion of a survey at 24 weeks. Participants could receive up to US $40 for completing a daily smoking diary via SMS text messages (US $20 if at least 60% of assessments are completed and US $40 if at least 80% are completed). In total, participants had the possibility of receiving a total of US $100 in incentives.

This is a subanalysis of a 2-arm, open-label pilot RCT [14] designed to evaluate the harm-reducing effects of ECs in patients with COPD, coronary artery disease (CAD), peripheral vascular disease (PVD), or asthma. The methods of the parent study are detailed fully elsewhere [14]. Briefly, study participants were recruited from two large urban health centers’ electronic health records (EHRs) and randomized in a 1:1 ratio to either standard smoking cessation care (nicotine replacement therapy [NRT] and counseling) or ECs and counseling [15]. Randomization was stratified by sex and baseline CPD (<20 vs ≥20). Participants were screened via phone or an internet-based screening tool before enrollment.

Participants completed study visits by telephone at baseline and at 1, 2, 3, and 6 months, with the intervention period lasting 3 months. At each visit, participants completed self-report survey assessments and were asked about their issued product supply and potential harm. Sociodemographic characteristics such as age, race, ethnicity, place of birth, gender, marital status, education, work status, living situation, and homelessness were measured via self-report at baseline. In addition, condition-specific assessments (CAT [16], Clinical COPD Questionnaire [CCQ] [17,18], Seattle Angina Questionnaire-7 [SAQ-7] [19], Asthma Control Test [ACT] [20,21]) were also assessed as was motivation and confidence to quit smoking, tobacco use, and craving via the Minnesota Nicotine Withdrawal Scale (MNWS) [22].

Based on EHR diagnoses, participants were characterized into the following groups: COPD, asthma, and CAD or PVD. Participants did not receive any incentive for counseling calls, but were incentivized to complete study visits and respond to the daily SMS text message smoking diary.

In addition to study visit surveys, during the intervention period, participants were asked to record daily CC and EC use along with measures for CC craving and EC and CC satisfaction by SMS text messaging. Since the EMA protocol was embedded within the RCT, it allowed us to directly compare the message-derived outcomes with the survey assessments across intervention groups. The details of branching logic on the texting sequence participants would receive and the phrasing of each question are presented (Figure S1 in Multimedia Appendix 1). Certain responses triggered the follow-up questions about daily use, satisfaction, and craving. If the participant reported no CC or no EC use, only craving questions were triggered. Only participants in the EC arm received questions about EC use. Due to discordance between NRT use instructions (ie, 24 h patch) and the scheduled check-ins, the NRT arm was not asked about NRT use via SMS text message check-ins. This procedure employed time-based sampling with participants receiving SMS text message prompts four times per day, breaking the day into “Early, Lunch, Afternoon, Evening” periods such that approximately 4 hours had passed between check-ins.

Participants received brief instructions sent via an introductory text and response indicators in the body of SMS text message prompts but no formal training; of note, the EMA protocol was explained to participants during the baseline and consent process. Participants were given the opportunity to opt out of receiving SMS text messages before initiation of participation and during the study. Automated SMS text message prompts were sent to participants daily over the 12-week intervention.

The SMS text message data were used to determine daily and weekly summaries for the following measures: response rate, number of CCs, EC uses (EC arm only), craving, satisfaction with the last CC, and satisfaction with the last EC (EC arm only).

To evaluate predictors of EMA adherence, participants were divided into two groups: (1) those with a weekly response rate >60% or more throughout the duration of the study period and (2) those with a weekly response rate <60% at any point in the study period. Descriptive statistics were calculated by group, including sociodemographic characteristics (age, gender, race and ethnicity, marital status, education, and work status) as well as condition-specific assessments, motivation, and confidence to quit smoking and tobacco use for each group. For variables not normally distributed, median and IQR were presented. Characteristics were evaluated using Wilcoxon rank sum test for continuous variables and Fisher exact test for categorical variables.

To assess the comparability between EMA measures of CPD and more traditional measurements gathered in a survey setting, we examined the Pearson correlation coefficient between CPD reported during the final week of EMA prompting (week 12) and CPD reported at the 3-month survey. We also examined the paired difference between EMA- and survey-reported CPD at 3 months using a paired t test. Pearson correlation coefficient was also applied to assess the convergent validity of EMA measurements for craving compared with survey-reported measures (first item of the Minnesota nicotine withdrawal scale: “Do you have a craving for cigarettes or desire to smoke?” with responses coded from 0‐4, corresponding to “Not present” [0], “Slight” [1], “Mild” [2], “Moderate” [3], and “Severe” [4]) [23]. The EMA prompt “on a scale of 0‐9, how much do you want to smoke a cigarette right now?” was used as the comparator for craving with higher numbers corresponding to greater craving.

To assess our primary outcome of change in CPD, we used a negative binomial mixed effects model, adjusting for the intervention arm, time (weeks from baseline), and a random intercept for each individual. We compared the results obtained from surveys by building a similar model, with the time variable expressed as time from baseline to week 12 visit. The negative binomial model has been shown to be more effective for count data than a Poisson model in the presence of overdispersion [24]. CPD reduction rate was evaluated by measuring the change in CPD from the baseline to week 12 from both EMA and survey-based data. To compare reduction rates by arm at week 12, we calculated the risk ratios of reduction rates of those reaching at least 50%, 75%, and 100% reduction as reported via EMA and survey. We used the same negative binomial mixed effects models to assess the satisfaction and craving scores.

For those in the EC arm, we tabulated the number and percentage of individuals each week that demonstrated evidence of switching (CPD=0 and EC use>0) as well as mostly switching (reduction in CPD >75% and EC use>0). We also summarized weekly EMA-based measures including response rates, the number and percentage of individuals with >60% response rate, average craving, average satisfaction with last CC, average satisfaction with last EC, average CPD, and average EC uses.

All analyses were conducted using Stata (version 18; Stata Corp) and R (version 2024.04.2+764; R Foundation for Statistical Computing), and all tests were 2-tailed, with a .05 significance level.

In total, 109 (90%) individuals of the 121 randomized trial participants took part in the EMA protocol. Among the 109, 65 (59.6%) were considered adherent to the EMA protocol with a median weekly response rate that declined slowly from 97.8% in week 1% to 89.4% in week 12. The mean and median response rates over time for all participants are plotted in Figure S2 in Multimedia Appendix 1. Of the 12 individuals with no texting data, 5 withdrew from study activities or opted out, 3 never received the study welcome text, 3 never responded to the study welcome text, and 1 died during enrollment (Figure 1).

Table 1 displays participant characteristics by adherence to EMA protocol. The median age among adherent participants was 61 (IQR 54-65) years and 58 (IQR 52-65) tears among nonadherent individuals. The sample was mostly non-Hispanic white (63/108, 57.8%), and a plurality were married or dating (52/109, 47.7%), had some college or an associate’s degree (44/109, 40.4%), and were retired or on public assistance (47/109, 43.1%). Most participants had COPD (65/109, 59.6%), followed by asthma (36/109, 33%). The median motivation to quit smoking score was 10 (IQR 8‐10), and the mean confidence to quit smoking was 6.5 (SD 2.2). At baseline, the mean number of CPD reported at the survey was 17.3 (SD 9.7), and the mean CPD from week 1 as assessed from EMA was 11.6 (SD 7.5). No significant differences were observed between the adherent and nonadherent groups.

Reported CPD estimates from EMA correlated strongly with the survey setting at 3 months (Pearson r=0.73; 95% CI 0.60-0.82). Of note, there was one EC participant who reported a very high consumption of combustible cigarettes (>40 CPD) via EMA and at their 3-month visit; this individual’s data was included in the current analysis as results did not differ meaningfully when excluding their data. There was no statistically significant paired difference between EMA-reported CPD and survey-reported CPD (paired difference=0.39, 95% CI −1.02 to 1.79). Craving measurements from EMA and surveys also showed a significant positive correlation (Pearson r=0.38, 95% CI 0.17-0.56). In Figure 2, plot A depicts a scatter plot visualizing the association between CPD use via survey and EMA at week 12. The estimated correlation (Pearson r) is 0.73 (95% CI 0.60-0.82). A fitted regression line and 95% CI are included. Data points are stratified by treatment arm. Plot B depicts a histogram showing the distribution of paired CPD differences between EMA and survey at week 12 for both treatment arms. The estimated mean difference is 0.39 (95% CI −1.02 to 1.79). Plot C depicts a scatter plot showing the association between craving scores for CC via survey and EMA at week 12. The estimated correlation is 0.38 (95% CI 0.17-0.56). A fitted regression line and 95% CI are included. Data points are stratified by treatment arm.

Tables 3 and 4 show the comparisons between the reduction rate in CPD (50%, 75%, and 100% reduction) and negative binomial regression results using data reported through EMA and through surveys. Both treatment groups showed a significant decrease in the mean number of CPD at the end of the study period (overall; week 1: mean 14.2, SD 9.9; week 12: mean 4.6, SD 6.7) though there was no evidence that reduction in CPD was different between arms based on reduction rate. Of note, the point estimates for CPD reduction rate were all slightly higher than those computed from survey data.

The average CPD count had a mean of 8.2 (SD 8.03) and a variance of 64.5. With unequal mean and variance, our model must account for overdispersion, which the negative binomial model accomplishes. In both models, we found a significant effect of time on the average number of cigarettes per day. In the model using EMA data, we found a 7% decrease in CPD for each week (IRR for 1-week change 0.929, 95% CI 0.93-0.93), and in the model using survey data, the IRR for time across the 12-week intervention was 0.36 (95% CI 0.29-0.44). In both models, we found no statistically significant effect of randomization group on the rate of CPD; however, interestingly, the point estimates for the IRR for randomization group are in differing directions.

Plot A in Figure 3 shows the reported number of CCs smoked in each week by all participants. The trend lines were fitted using a generalized additive model and are included only for visualization. This visualization aligned with the hypothesis testing of negative binomial mixed effects modeling in that we see a general downward trend in the number of cigarettes smoked for participants in each group over time, with similar trends for each group. To visualize EC use more clearly, we presented EC use in days rather than weeks.

Among those in the EC arm, EC use increased gradually and stabilized after day 20, with daily CC consumption becoming slightly lower than daily EC use. The percentage of individuals who “switched” completely from CCs to ECs in each week ranged from 1.7% (1/58) in week 2 to 12.1% (7/58) in week 12. The percentage of mostly switching increased from 5.2% (3/58) in week 2 to 20.7% (12/58) in week 12.

The results showed that participants in either study arm who reported greater CPD in the 3-month visit survey tended to report greater CPD in the final week of EMA prompting, and vice versa. Specifically, the EMA data indicated that EC use could lead to a reduction in CC use similar to that of NRT. These results suggest that ECs could be a potential alternative smoking cessation tool in the future. Similar to analyses using study-visit data, we found a statistically significant time effect on CPD reduction when analyzing the EMA data. Through the insights provided by EMA, patterns in dual use and switching could also be observed where participants in the EC arm appeared to consume fewer CPD over time and had a steady increase in EC use. Together, these results provide evidence that EMA is likely a suitable method to collect recall-based smoking data from participants with greater frequency and shorter recall periods compared with standard study visits.

In addition, the rigorous EMA protocol implemented in the study, with a high response rate over the 12-week intervention period, demonstrates the feasibility of assessing smoking behaviors in this manner. The relevance and validity of EMA data rely on the comparability with traditional methods of measurement (eg, surveys) and the overall adherence to the EMA protocol among participants. This study provides insight by directly comparing smoking-related measures collected through EMA and surveys. The nonsignificant paired difference found in this study contrasts with other similar studies that found large discrepancies between retrospective and EMA data [25,26]. These results indicate that an EMA protocol can achieve equal or more accurate data collection than traditional methods. Furthermore, levels of EMA adherence have been found to differ based on participant characteristics in some groups [27]. While previous studies have used EMA protocols to assess the prevalence of smoking CCs or ECs [9,26,28], as well as changing smoking patterns over time [29], this study is among the first to use a smoking EMA protocol in a population with chronic conditions, thus, demonstrating the feasibility of an SMS text messaging–based smoking EMA protocol in this population.

The visualization of CPD and EC use over time (see Figure 3) offer examples of the added information provided by EMA, giving researchers more precise data collection points. EMA can also help identify and quantify the switching (from CCs to ECs) patterns of participants closer to when switching actually occurs, which itself is an area of future research. For example, for this trial, using traditional surveys, we would only be able to assess switching at the time of surveys (at 3- and 6-month follow-ups); however, EMA allowed us to determine when and if participants did in fact switch from CC to EC within and immediately following the intervention period. If assessed in near real time, this information could even be leveraged to tailor counseling delivered to participants.

We note several limitations. In total, 109 (90%) of the 121 trial participants agreed and enrolled in SMS text messaging prompts; however, only 65 participants were adherent (EMA response rate greater than 60%) in the final week of prompting. All responses were used in the negative binomial mixed effect model, whether they responded to all prompts in a day or only a subset. This type of missingness could bias the average number of cigarettes smoked downward, particularly as fewer participants responded consistently as the study period continued. While not statistically significant, exploratory analyses seemed to suggest that employed participants may be more likely to drop off in response rates compared with retired or unemployed participants; however, further work is required to better understand effects on response rate and retention. In general, there is a risk of an EMA protocol placing too much burden upon participants leading to fatigue or burnout [30]; in turn, participant fatigue could lead to higher drop-out rates and missingness, which could bias results. Similarly, there was variance in contact intensity between the two study arms, as the NRT participants were not asked about NRT or EC use limiting the total number of questions asked. While no statistically significant difference in EMA protocol adherence was measured, differential contact between the arms could potentially bias perceived burden or engagement with the EMA protocol. Despite the missingness and the potential for participant fatigue, this EMA protocol saw continued adherence and high response rates from a majority of enrolled participants while maintaining a fairly rigorous prompt frequency. Continued adherence indicates relatively low burden of text prompts (vs through other platforms; eg, phone app) and the familiarity most participants have with texting, which could be a more acceptable and feasible method compared with lengthier surveys. Future analyses of EMA data could seek to better understand additional smoking-related measures beyond craving and satisfaction. Due to limitations of measuring experiences using numerical scales [31,32], future EMA protocols may consider free-text responses and apply tools like natural language processing to EMA data to interpret and clean patient responses thereby transforming the data into meaningful information on human behaviors and sentiments. Better implementation of set responses can help ensure researchers are receiving the information desired in workable forms and reduce the data cleaning needed. Another future area of research is determining how well EMA data can detect the real-time switching from CCs to ECs.

SMS text messaging–based EMA represents a feasible and suitable method to collect recall-based smoking-related data. As compared with the survey data, the use of EMA provides unique advantages, namely greater granularity in the smoking timelines and a capability to detect switching patterns between CCs and ECs in near real time. EMA should be considered as a method for data collection when developing future tobacco harm reduction research.