We conducted a 1-time DCE, with a survey consisting of 12 choice tasks where participants were asked to choose between 2 hypothetical mHealth apps, and each app was made up of a combination of different features. Participants were recruited from Qualtrics, a validated source for online recruitment of participants for surveys with reliable data in a short period of time [26]. Qualtrics applies a comprehensive system to recruit from third-party panels, email lists, as well as social media, and participants are targeted for recruitment based on their panel profile information. Quotas for sex and age were provided, and eligibility criteria included ages 21 years or older, who were identified as Black or African American, and who currently used cigarettes or other tobacco products. Current use of tobacco products was defined as either having smoked at least 100 cigarettes in their lifetime and currently smoking in the past 30 days and/or past 30-day use of another tobacco product. Participants had to reside in the United States, complete the survey in English, report interest in tobacco cessation, and have a history of using an mHealth intervention program in the past or willingness to use one in the future. Respondents had to complete a screening survey to confirm eligibility and pass through a CAPTCHA test after receiving the study invitation from Qualtrics to prevent automated bots. Qualtrics also applied verification methods such as IP address checks to confirm the authenticity of participants in addition to a system that detects duplicate responses and evaluates for fraud risk based on specific device and browser attributes. Consent to participate in the study was obtained through the Qualtrics opt-in system. Through this system, participants have to opt in to participate in any research study. An information sheet with a description of the experiment and the rights of participants was included at the beginning of the experiment.

The study was also guided by the CHERRIES (Checklist for Reporting Results of Internet E-Surveys; Checklist 1) [27].

The sample size was estimated based on power calculations using the number of features and levels [28]. In DCEs, statistical power depends on factors such as the sample size, number of choice sets, alternatives within each set, and expected effect size or β coefficients for the DCE model [28]. Based on these parameters, it was determined that a sample size of 900 was needed to achieve over 80% power to detect meaningful differences between features and levels at 95% confidence level.

To ensure the quality of data collected, the team applied predefined criteria related to speeding or incompletion. We defined speeders as participants who completed the survey in less than half the median duration of time from start to completion for the whole sample of participants. Because Qualtrics also has an internal system to automatically remove duplicates, we are only able to report exclusions that were identified after a manual check of the dataset.

We developed a comprehensive list of mHealth app features for tobacco cessation from our formative work and the literature, and then conducted focus groups to improve the list. We decided not to include some features from the comprehensive list in the DCE because those features were either unanimously endorsed as favorable, for example, some form of reward or incentive, or they are basic or core features that need to be included in mHealth apps, for example, the option to anonymize an individual’s identity. The experiment was focused on features with unclear or variable levels of appeal. The final list included the following potential mHealth features: benefits of quitting, strategies for quitting, graphics, personalization, health information, marketing, and connection with others (Table 1).

Following the International Society for Pharmacoeconomic Outcomes Research for Good Research Practice in Health guidelines [29], we designed a conjoint DCE with 12 choice tasks using Sawtooth Software (Sawtooth Software, Inc) [30]. We used design principles of orthogonality and level balance to ensure even distribution of levels such that levels had a similar number of appearances in the experiment and could be selected without dependence on other levels or features [31]. Each choice task was made up of a pair of hypothetical mHealth apps for tobacco cessation containing 1 randomly assigned level from each of the 7 features. We applied the balanced overlap design method in generating the DCE tasks, which allowed for simple overlap of levels within features for each task. This approach reduces excessive and unnecessary processing of information and enables respondents to focus on features that differ, which increases the likelihood of making more meaningful trade-offs as they complete each task [31]. To further reduce cognitive burden and improve consistency of choice, we applied color coding. That is, if both apps shown contained the same level of a feature, that level was grayed out so participants could focus on the differences between the 2 apps [32]. To confirm the quality of the design, Sawtooth’s test design function was applied to check the integrity of the design with simulated data and verify that the relevant effects can be estimated [31]. We conducted cognitive interviews with 6 participants in an iterative manner to ensure clarity of the questions asked in the DCE. After reaching saturation, we carried out a pilot test of the survey with 53 participants recruited from Qualtrics to examine for illogical response patterns. After all concerns were addressed, we fielded the final version of the survey from April to July 2024. Each Qualtrics participant was assigned a unique identifier based on their IP addresses. Participants were compensated in accordance with the set compensation rate at Qualtrics for survey completion.

The first part of the survey began with general questions about demographics, tobacco use history, and health status. Participants were also asked questions about their age, sex, race, and ethnicity since we included those who identified as Black plus another race. We also collected information on health status, state of residence, level of education, and household income.

The second part of the survey contained an explanation of each mHealth app feature, and each feature had 4‐5 levels (Table 1). An example of a DCE using car features was presented. After the example, participants were presented with the following scenario for each choice task: “Imagine that two new mHealth apps for tobacco cessation were recently developed and you want to quit using tobacco products. If these two apps were your only two options, which app would you choose? App A or App B?” Table 2 illustrates an example of one of the 12 DCE tasks that participants completed.

Data for demographics, tobacco use, health status, and other participant characteristics were analyzed using descriptive statistics (SPSS Statistics, version 29.0.1.0; IBM Corp). Differences among age groups were assessed using 1-way ANOVA, with statistical significance set at P<.05.

The response data from the DCE were analyzed with the help of Sawtooth Analytics. We first determined part-worth utilities at the individual level with hierarchical Bayes estimation within Sawtooth Software [31]. Part-worth utilities indicate the preference for levels within each feature. More preferred levels have higher utilities (above 0), and dispreferred levels have the lowest utilities (below 0). With hierarchical Bayes, we were able to apply a Bayesian method to fit a multinomial logit model at the individual level, which allows for heterogeneity while strengthening the overall sample. Our calculations were based on 12 choice tasks per participant, with each task having 2 alternatives.

Next, we determined the importance score for each feature at the individual level by calculating the range of part-worth utilities for levels within the specific feature and dividing by the sum of ranges across all features multiplied by 100. The importance score is the percentage of choice due to each feature or the measure of influence of a feature on the decision-making process to choose a product. The higher the importance score, the higher the influence of that feature on the choice of the product, in this case, the mHealth app. For each feature, the importance score at the individual level was averaged across all participants to obtain the overall importance score. We summarized aggregate preferences with multinomial logit regression with effects coding, with sample-level coefficient and goodness-of-fit statistics provided to enable us to obtain both disaggregate utilities and aggregate estimates.

In addition, we conducted exploratory analysis with paired 2-tailed t tests to compare importance scores across features and to compare part-worth utilities of levels within features in all possible combinations to determine statistical significance.

The hypotheses, study design, and analytical plan were preregistered on AsPredicted [33] (#165678), and the study was reviewed by the institutional review board of the University of North Carolina at Chapel Hill and exempted (Study # 23‐2131). At the beginning of the experiment, we included an information page with the study description and the rights of participants. All participants provided informed consent through the Qualtrics opt-in system before participating in the study. Participants were compensated according to Qualtrics set rate for compensation after completion of surveys.

The analytical sample included 901 respondents. Of the initial 1304 participants, 369 were excluded due to speeding, that is, those who spent less than half median duration of time spent by all participants (239 seconds) to complete the survey, and 34 were excluded due to incomplete responses. The mean age of participants was 41 (SD 14.02) years, and 42% (377/901) were female participants. Most participants (845/901, 94%) were non-Hispanic Black and had annual household income above the federal poverty level (726/901, 81%). In total, 61% (549/901) of participants reported that they had used an mHealth app in the past, and 87% (786/901) had willingness to use an mHealth app in the future. In total, 82% (738/901) of participants reported current cigarette use, 79% (714/901) reported use of other tobacco products besides tobacco cigarettes, and 82% (738/901) reported a history of a prior quit attempt (Table 3).

Of all 7 mHealth app features, graphics was determined to have the highest average importance score at 16.5% (95% CI 16.2-16.9). Marketing and strategies for quitting were next with importance scores of 15.1% (95% CI 14.7-15.4) and 15% (95% CI 14.7-15.4), respectively. Connection with others had an importance score of 14.3% (95% CI 14.0-14.6), while personalization was fifth with an importance score of 13.4% (95% CI 13.1-13.7). The lowest-rated features were benefits of quitting and health information with importance scores of 13.1% (95% CI 12.8-13.5) and 12.6% (95% CI 12.2-12.9), respectively. Paired 2-tailed t test revealed that importance scores were statistically significantly different when compared with one another (P<.001) except for comparison between marketing and strategies for quitting (P=.46) and between personalization and benefits of quitting (P=.19). We found no statistically significant differences in the analysis by age group and sex.

Within each feature, some levels were preferred more than others. The 5 most preferred levels were found within strategies for quitting, marketing, graphics, and connection with others. Within features, strategies for quitting had the highest and third-highest levels of “making a step-by-step quit plan” with a part-worth utility of 21.03 (95% CI 18.36-23.70) and “recommendations to manage relapse or withdrawal” with a part-worth utility of 16.70 (95% CI 14.68-18.73), respectively. Marketing had the second-highest level of “Historically Black Colleges and Universities–endorsed app” with a part-worth utility score of 18.33 (95% CI 15.70-20.95). Graphics had the fourth-highest level of “short video testimonials from people who successfully quit” with a part-worth utility score of 15.55 (95% CI 13.12-17.98), while connection with others had the fifth-highest level of “quit buddy program for support and accountability” with a part-worth utility score of 14.61 (95% CI 12.27-16.95; Table 4).

Least preferred levels were also found. The top 5 dispreferred levels were found within connection with others, strategies for quitting, marketing, and graphics. Within features, connection with others had the least preferred level of “links to support groups on social media” with a part-worth utility of −17.10 (91% CI −19.23 to −14.97), while strategies for quitting had the second and fifth least preferred levels of “interactive games and videos to distract from smoking” with a part-worth utility score of −15.16 (95% CI −17.74 to −12.58) and “recommended tobacco cessation medications to help with quitting” with a part-worth utility of −11.96 (95% CI −14.33 to −9.59). Marketing had the third least preferred level of “endorsement by a celebrity who successfully quit” with a part-worth utility of −12.80 (95% CI −14.97 to −10.62), while graphics had the fourth dispreferred utility of “ photos of racially diverse people who successfully quit” with a part-worth utility of −12.26 (95% CI −14.71 to −9.80; Table 4).

This study identified preferred features of mHealth apps for tobacco cessation that appeal to Black adults who use tobacco products. While all features had some importance, graphics, marketing, strategies for quitting, and connection with others were the most important. We also found that within each feature, some levels were highly preferred, while others were highly dispreferred. To enhance the appeal of mHealth apps to Black adults who use tobacco products, prioritizing the inclusion of highly preferred levels and features in apps may lead to higher use and improved cessation.

Graphics was the most important mHealth feature, particularly video testimonials from others who successfully quit using tobacco. Dynamic graphics such as video testimonials are known to enhance use and interest in mHealth apps for several health conditions [18,34]. One study focused on developing a culturally relevant app for Black individuals on cardiovascular health showed that participants were interested in video narratives from prior participants of a program [34]. Video testimonials can improve the relatability and relevance of the app for the target population [34,35], which can increase engagement and use. Another study to examine features that Black adults prefer in tobacco cessation mHealth apps based on one app found that providing content through videos was appealing [18]. Using videos as a means to provide content has also been associated with increased retention of information [36].

In addition, within the levels of the graphics feature, photos of people from diverse races who successfully quit were significantly dispreferred. The evidence regarding preferences for racially diverse features in mHealth apps is mixed. While our findings support the importance of inclusivity with mHealth app features [37], a concern also exists for targeting, mentioned previously in the literature [18]. The significant dispreference observed may be reflective of the desire or need to protect one’s self from interventions that may be considered racially targeted or that could potentially reinforce negative stereotypes, a concern highlighted in studies of medical mistrust [18,38-40]. Another factor that may have contributed to this finding is the static nature of this type of graphic. Images presented in a dynamic mode such as videos tend to be more appealing, while a lack of appeal and low likelihood of engagement with static information in mHealth apps has been described in the literature [35].

Marketing for the app was also found to be important, especially endorsement by a reliable source such as Historically Black Colleges and Universities (which was the most important level) or a Black celebrity. Relevant health information is valued by app users [37], more so if the information is from a reliable source [34]. Having a trusted group or individual endorse the app increases credibility and trust [41], which in turn can increase the likelihood of use of the app. Accuracy of information and intention toward end users are major concerns that tend to influence the decision-making process of target users [42]. As demonstrated in previous research, these concerns are especially relevant in light of the longstanding challenges associated with medical distrust [43,44]. Many Black adults who use tobacco products may be less likely to trust an mHealth app that lacks culturally relevant features and is promoted through mainstream channels, including non-Black celebrity endorsements.

Strategies for quitting were equally important, and similar to our study, previous research has shown that this feature is very important to other mHealth app users [18,45]. Provision of a practical guide for quitting has been determined to be highly desired in mHealth apps in general [45], which was the most important level of this feature in our study. One study found that Black adults who use tobacco products prefer strategies for quitting in an mHealth app, including ways to deal with cravings and healthy ways to deal with stress other than resorting to tobacco [18]. The next most important feature levels were a step-by-step plan and information on how to deal with challenges such as relapse or withdrawal symptoms. In that study, participants expressed interest in learning about alternatives to quitting abruptly (ie, the “cold turkey” method) and were particularly receptive to combining the app with pharmacotherapy [18].

The limited interest in pharmacotherapy and tobacco cessation resources observed in this study is not unexpected, as Black adults are less likely to be screened for tobacco use and, consequently, less likely to engage in discussions about cessation options [46]. Although dispreferred, the provision of this information within an mHealth app can be a key step to reducing failure rate and increasing the chances of sustained success with cessation for the target population. While interactive games in mHealth apps have been linked to increased engagement and enhanced self-efficacy [47,48], other studies, including our own, have found that such features may be less influential or valued by users [49,50].

We also found that connecting with other app users or health professionals through the app was important to participants, and such connections have been shown to keep users more engaged with apps [34,37,42]. Previous research shows that the addition of a program within the app for users to meaningfully compete among themselves is an attractive component of an app [35,37]. Participants can use competition as a way to challenge or motivate themselves as a way to reach their health goals. Similarly, connection with a health professional through the app can further strengthen the intervention [51], especially if such a connection can mitigate the need to have an in-person encounter with a provider for tobacco cessation concerns. Participants showed little interest in interacting with other users beyond accountability purposes or in connecting through social media groups. This aligns with other studies reporting the limited appeal of social networking features within mHealth apps [42,45]. Although some users may be open to sharing their health information, maintaining privacy, especially from friends on social media, may help reduce feelings of vulnerability often associated with efforts to improve one’s health [42,52].

Consistent with our findings, mHealth apps that offer personalization features, such as goal setting, reminders, and notifications, are well-established as desirable and beneficial to users [34,52]. Features that keep users on track about their smoking-related behavior [18,37], provide a self-management or monitoring system [53], and enhance the ability for users to monitor their own progress have been associated with a higher likelihood of achieving the target health outcome [51,53]. Tracking positive outcomes, such as money saved by not purchasing tobacco products or the number of days between use, has also been shown to be a helpful feature for mHealth app users [18]. In addition, notifications or reminders that help individuals focus on their reasons for quitting can help to keep users motivated and more engaged with the mHealth app. Allowing users to determine the timing and frequency of these reminders can also help to further enhance the appeal and use of the app [42].

Information about the benefits of quitting was the second to the lowest importance score overall, but within that feature, participants preferred information about the financial benefits of quitting. Similar to our study, the benefit of saving money by quitting has been mentioned as appealing content for mHealth apps for tobacco cessation [35,45,54]. It is well established that many individuals who use tobacco products also experience financial hardship [55,56]. Tobacco cessation can positively impact cash flow by reducing the ongoing costs associated with tobacco use. For example, in 2025, the average cost for a pack of cigarettes in the United States is about US $10 [57]. Including information in an mHealth app about the potential monthly or annual savings from quitting tobacco may motivate users to stay engaged in the cessation process.

Within the benefits of quitting, participants also preferred information about the reduced cardiovascular risk associated with cessation but dispreferred information about reduced risk of cancer or the aesthetic benefits of quitting. Information about the association between tobacco use and an adverse health effect like cancer has been well established, and it is possible that participants were already well informed about the cancer risk; hence, they did not think that more information would be beneficial. Other studies also show that information about the harms of smoking in an mHealth app may not be as appealing [45].

Health information was the least important feature overall; however, participants expressed specific interest in learning about the risks of adverse cardiovascular outcomes, particularly as they relate to Black adults who use tobacco products. Studies have shown that presenting relevant health information can generate interest because the information is more relatable [18]. Information about the addition of menthol to tobacco products and its role in addiction was not appealing, as well as information about how the tobacco industry has targeted Black communities. Although information about the tobacco industry targeting can be effective by evoking negative emotions toward the industry [58], the negative response observed in this study may reflect broader concerns about racial targeting and generalized mistrust. Discomfort with targeted interventions among Black adults has been well documented in previous research [43,59,60]. While presenting health information can be beneficial, it should be delivered in a way that is inclusive and does not inadvertently isolate Black adults from broader user groups.

We used a convenience sample, and the majority of participants reported household incomes above the federal poverty level and had attained a college degree or higher. As a result, the findings may not be generalizable to broader populations. Additionally, only a small proportion of respondents were older adults, who may be less comfortable with technology [52]. For analysis, individuals who identified as multiracial (Black and another race) were grouped with Black participants. Although multiracial participants represented only 2% of the study sample, combining them with Black participants may have affected our findings. We also excluded speeders and participants with incomplete or duplicate responses from our analysis. Although the affected participant made up less than 1% of the total sample, this may have disproportionally affected certain subgroups of participants. In addition, although we conducted several paired 2-tailed t tests to check for differences in importance scores and utilities across features and levels, we did not formally control for multiple comparisons; hence, some significant differences found may be due to chance. These analyses were exploratory, and our primary conclusions were based on the hierarchical Bayes estimations. Our findings are limited to the specific features and levels included in the DCE, which do not represent an exhaustive or comprehensive list. While we selected features based on existing literature and our qualitative work, it is possible that other features, equally or even more appealing to users, were not captured. Finally, we cannot determine whether incorporating all preferred features into an mHealth app will lead to sustained and meaningful use, since DCEs only capture stated preferences. Further research is needed to assess whether an app designed with these features ultimately improves tobacco cessation rates.

In summary, the role of mHealth apps for tobacco cessation is promising. This study highlights features of mHealth apps for tobacco cessation that highly appeal to Black adults who use tobacco products. Inclusion of these highly preferred features: graphics, marketing, strategies for quitting, and connection with others in a mHealth app, and more importantly, preferred levels within these features should be considered in the design and development process. The inclusion of preferred features may increase the appeal and use of the app by the target population. Although end-user preferences are critical in mHealth app development, some less preferred features may still offer meaningful benefits for tobacco cessation and should be carefully reconsidered before being excluded entirely. Future research to test the effectiveness of an app with these preferred features is needed.