Smoking continues to be a leading cause of preventable morbidity and mortality. An estimated 11.5% of American adults currently smoke [1], and more than 480,000 die annually due to smoking-related illness attributable to smoking and secondhand smoke [2,3]. Individuals with low education, low income, and those from marginalized minority groups or residing in rural areas are disproportionately affected [4].

Smartphones have extensive reach into even low-income populations and are increasingly integrated into traditional health care [5,6]. Although smartphone-based or other digital interventions for smoking cessation are not new, the vast majority are unvalidated mobile apps that typically require motivated users and rarely engage patients in a sustained and meaningful way [7]. More advanced, responsive, and tailored digital interventions using machine learning and artificial intelligence (AI)—such as chatbots that empower patients and support providers—are necessary to harness the true promise of digital health as a tool for active (and successful) cessation referrals [8-10]. Chatbot research has shown that patients enjoy interacting with chatbots several times per day [11], develop emotional attachments [12], and show high levels of satisfaction [13]—even preferring the chatbot to a human [14,15]. However, like other digital health tools for smoking cessation, the chatbot needs to be adapted to fit individuals’ needs. This includes recognizing users’ diverse technological skills, accommodating busy schedules, addressing privacy concerns about data storage and use, reducing overreliance on chatbots for health care needs, and increasing engagement [16-19]. Despite concerns regarding the accuracy and safety of health-related AI chatbots, studies largely show that they are acceptable and can be beneficial to patients [20]. Even if a digital tool has initial appeal, extended patient engagement is unlikely [16,17]. Therefore, the style of communication any chatbot uses to discuss smoking is a vital consideration.

Motivational interviewing (MI) is “a particular way of talking to people about change and growth to strengthen their own motivation and commitment” [21; pp. 12‐13]. In several reviews and meta-analyses, MI has been observed as an efficacious intervention for substance abuse behaviors, with small to moderate and variable effects [22-27]. Unlike other digital health tools that rely on didactic content, MI is a collaborative communication style that elicits and strengthens personal motivation for change through open-ended dialogue and reflections. MI may be an appropriate communication style to enhance one’s intrinsic interest in a referral to quit smoking. Not a technique or tool, MI is a therapeutic conversational approach to elicit change talk in support of behavior change. Change talk (CT) includes statements that “indicate movement toward a particular change”; in contrast, sustain talk (ST) “moves the speaker away from change in support of the status quo” [28; p. 25]. Current MI theory [21,28] posits that increasing CT and reducing ST increases the likelihood that individuals will change subsequent behavior. Indeed, a greater proportion of CT is significantly associated with a reduction in risk behaviors [29]. Two therapeutic components facilitate this task: a relational component and a technical component [30,31]. The relational component focuses on the clinical relationship that develops between the therapist and the client over the course of the session (eg, therapist empathy). The technical component comprises specific MI-consistent (MICO) therapist skills (eg, reflections, affirmations, and open-ended questions about the benefits of change) associated with increased CT and decreased ST, which in turn should lead to desired health behavior outcomes. Meta-analyses of in-session processes of MI indicate the most support for the technical component (which can be readily translated into a chatbot) in predicting behavior [32], a finding replicated in subsequent meta-analyses [24,29].

Digital tools incorporating MICO skills have been developed for a variety of health behaviors [33,34]. A 2016 review of 41 studies [35] examining technology-delivered adaptation of MI found that the majority (32/41) delivered fully automated interventions with no interaction with counselors, and only 5 used text interactions with counselors using the technology (eg, chat room and computer program) in real time. Since that review, there have been several more advanced, responsive, and tailored digital interventions—such as chatbots that empower patients and support providers—that try to harness the true promise of digital health as a tool for active (and successful) cessation referrals [8,36,37]. Thus, advances in machine learning and AI can improve the delivery of MICO technical skills, thus enhancing the focus and credibility of the MI delivered and enhancing the relational components of MI (empathy, collaboration, and facilitating self-exploration). For example, one study found that a smoking cessation MI chatbot was able to increase self-efficacy by simply programming the MI chatbot to ask open-ended evocative questions; similarly, programming reflections using ChatGPT in addition to evocative questions improved readiness to quit and perceived empathy of the MI chatbot [34]. However, these interactions were often brief, invariably structured, and followed a regular sequence (eg, question, reflection, and question). Therefore, it remains unknown how well a chatbot can have a longer iterative conversation about different topics associated with smoking.

This study details how a technology-assisted motivational interviewing (TAMI) chatbot was developed. The process drew on agile methodology—the dynamic systems development method (DSDM) [38,39]—which is an incremental approach that can incorporate client feedback at different phases of chatbot development. We detail our efforts in 3 phases of the DSDM: the functional model iteration phase, the design and build phase, and the implementation phase. During these phases, we incorporated user-centered design interviews, existing patient and consumer datasets, and user experience interviews to ensure the meaningful involvement of patients with lived experience in the design, development, and deployment of an AI application for smoking cessation. The goal of our work was to develop a chatbot that could evoke and resolve ambivalence about quitting smoking and provide a personalized quit plan to accomplish this goal.

The DSDM consists of 5 phases (see Textbox 1). During the first 2 phases, the feasibility study phase and the business study phase, the research team specified that a credible chatbot will ideally engage the user, maintain a conversational focus on smoking to enhance motivation to quit smoking, and then develop a personalized quit plan. As part of the functional model iteration phase, the concept of a chatbot was then presented to 3 current smokers for evaluation in order to identify which aspects of the chatbot were worth further development.

All procedures received approval from the University of California, San Francisco institutional review board (#19‐28845).

Three participants (2 female and 1 male; 100% White) were recruited from a primary care clinic via posted fliers. They were English-speaking, owned a smartphone, and reported smoking at least 100 cigarettes in their lifetime and smoked at least one cigarette per day for the past 7 days. Previous experience with chatbots was not a requirement for participation. Eligible patients who did not opt out of screening were contacted by the study coordinator, and those who agreed to participate were scheduled for a study onboarding session conducted via Zoom by the study coordinator. At onboarding, the study coordinator reviewed the consent form with each patient, obtaining informed consent via the ask-teach-ask method. Upon obtaining consent, 30‐45 minute interviews were conducted via Zoom by the study coordinator. Participants were reminded that responses were confidential and all feedback (ie, positive and negative) was welcomed to improve the TAMI chatbot. Participants received an electronic US $30 gift card. Participants were told, “Today we are discussing the design of a chatbot that helps patients get ready for smoking cessation referrals. We have done the initial programming and have some design ideas but we want your input before we get too far along.” The participants were then asked (1) how they would feel about interacting with TAMI, (2) the best way for TAMI to ask if someone is ready to quit smoking, and (3) how TAMI could help someone in creating a quit plan.

The 3 interviews were transcribed by Otter.ai voice-to-text software. After transcripts were transcribed and cleaned by listening to audio and comparing the transcribed text, we used a grounded theory inductive approach [40] to understand individuals’ recommendations about TAMI. Two qualitative experts of the study team read the transcribed interviews to generate summary notes to become familiar with the interviews. The summary notes were then used to identify the themes to create a consensus codebook. The identified themes were grouped based on the 3 research questions.

Participants wanted a sense of connection with TAMI in two ways. First, they wanted TAMI to be able to smoothly interact with them by picking up social cues (eg, using and responding to humor and sarcasm) and not have sterile or rudimentary responses. Second, there was a desire to feel understood as a smoker, and to have their personal experience of smoking and motivation to quit evoked and affirmed.

Participants expressed that TAMI should understand the balance between checking in on their motivation to quit smoking versus being too “pushy.” Furthermore, information provided should be sensitive to the content of the check-ins, rather than standard or boilerplate educational materials about the benefits of quitting. Interactions should be friendly, inquisitive, and encouraging, rather than lecturing, advice-giving, or shaming. Ideally, check-ins would occur less frequently (eg, weekly and monthly) when the individual is considering quitting or the quit date is far away.

Then, interactions would ramp up in frequency as the user approaches a quit date and reach their highest frequency during the acute cessation period. Across multiple chat sessions, TAMI should recall previous discussions to avoid participants having to repeat themselves.

Participants wanted a tailored menu of community options and strategies that they could then select and pursue, rather than having a prescriptive set of instructions. Options provided should reflect users’ ideas about optimal ways to quit (eg, nicotine replacement therapy and nonmedication options such as stress-management groups), and interviewees also expressed a desire for a menu of strategies they could use to weather urges to smoke. Information about the immediate and ongoing benefits of quitting (eg, positive effects on the lungs and body, financial savings for a desired item) was also seen as valuable to enhance motivation to quit. Participants wanted to easily access in-depth information about various options, but with only the key points displayed on the front page of the quit plan. Emailing or texting the formal quit plan was acceptable, and all participants agreed that the quit date should be prominently displayed. Participants recommended flexibility in changing the quit plan if needed, as well as having input on when TAMI would contact them.

Taken together, the user-centered design interviews confirmed that MI would be an ideal approach for the TAMI chatbot for several reasons. First, MI has 4 tasks that are sequentially addressed during an MICO conversation about change: engaging the client, focusing on a specific behavior, evoking a person’s own argument in favor of change, and planning behavior change. Second, there are specific MICO skills that the chatbot can use (eg, open-ended questions) to evoke information from users about their smoking, which can then be reflected to foster a user’s sense of being understood. Third, evoked CT and ST can indicate the user’s motivation and interest in changing smoking. Finally, MI is an eliciting approach, rather than a confrontational or prescriptive style of communication.

Once the user was engaged and focused on the session, the challenge was to have TAMI be able to classify the user texts (by topic, CT, and ST) and then respond appropriately using MICO skills that have been demonstrated to evoke further (eg, double-sided reflections ending on user CT) [28].

Once a user completed a session with TAMI and indicated readiness to quit smoking, TAMI guided them through personal preferences for referral and quit-plan elements similar to our prior work [49,50] and suggestions from the user design interviews (eg, a menu of external resources).

The tailored quit plan was typically comprised of several evidence-based cessation interventions that are selected by the user, such as nicotine replacement therapy, behavioral counseling, state telephonic quitlines, and pharmacotherapy (eg, varenicline). TAMI evoked strategies used in the past and reminded users when additional steps need to be taken, such as contacting a prescribing physician or downloading discount coupons to buy nicotine patches. TAMI also encouraged the user to set a quit date and provided tips to increase the likelihood of success, such as removing smoking triggers from the home and informing friends and family about the quit date. The full, tailored quit plan with the quit date prominently displayed as well as relevant links and instructions were downloadable and printable.

Interviews were transcribed, cleaned, and uploaded to Atlas.ti to assist with data management and inputting the codebook. We used a grounded theory inductive approach [40] to understand individuals’ experiences with TAMI. One qualitative expert from the study team read all transcribed interviews to generate summary notes to become familiar with the interviews. The summary notes were then used to identify themes to create a codebook. The identified themes were grouped based on the 3 research questions.

Participants reported liking TAMI for various reasons: TAMI was readily available on their phones, it provided resources to help quit smoking if they were interested in quitting, and TAMI was capable of asking a variety of questions. Response to the check-ins was positive, and one participant felt it humanized the experience:

Others felt that the reminder check-in showed TAMI cared:

Digital interventions provide users with some anonymity. TAMI provided P48 with a private workspace to feel comfortable working through her quitting process and not be judged, versus doing it with other people like friends, family, or a therapist who might pressure her too much:

Some participants felt TAMI being nonhuman made it difficult to express themself emotionally, but the main issue participants reported was that TAMI did not understand or respond appropriately in some conversations:

In addition, personal preferences in who to communicate with and the impersonal nature of TAMI appeared to affect motivation to connect and share personal information about smoking.

Participants who did not express interest in quitting and therefore did not receive a quit plan expressed that they still would have liked to be given options to help them quit. Some participants who did receive a quit plan felt some of the suggestions would take too long.

Participants offered several suggestions to improve TAMI. Other participants thought TAMI could increase engagement by remembering names or previous conversations. Participants adopted strategies to help them quit after setting a quit date. One individual felt that a tracking component would help individuals understand their smoking better; others recommended providing information about cessation options before formally establishing a quit plan.

Using TAMI to connect with others was also mentioned by participants. One suggestion was a peer support group or chat option to see other people’s struggles, not feel so alone, and learn what has been helpful to others when quitting.

One participant stated it would be helpful to speak to a human health coach with more knowledge for approximately 20-minute sessions every week after setting a quit date. Other participants wanted features to make the experience more personal, such as an Avatar that looked like them. Another individual wanted TAMI to develop a deeper connection:

These findings demonstrate the value of using agile methodology to incorporate user-centered feedback and data across multiple development phases, particularly in aligning chatbot tone and content with MI principles. Through the involvement and consultation of patients who were considering quitting smoking, as well as using patient and consumer datasets, the final iteration of TAMI was a fully automated chatbot that uses NLP to “understand” what a patient is saying, followed by the selection of an appropriately matched, prewritten response. Every time a user initiates a conversation with TAMI via a secure website, it assesses readiness to quit smoking, and—depending on the user’s stage of change—TAMI will either begin a “module” of MI to enhance readiness or TAMI guides the user in the creation of a tailored quit plan to quit smoking in the next 30 days. This structured, button-driven module “tours” the user through several evidence-based smoking cessation interventions and asks the user to choose their preferred “quit tool(s).” After the quit tools have been selected, the TAMI Coach asks the user to set a quit date and provides advice regarding quit-day preparations and coping with cravings. A final PDF quit-plan summary is produced that includes instructions on how/when to access the quit tools including community referrals, and access to resources such as low-cost nicotine replacement therapy or local quitlines. By combining proven engagement strategies (eg, targeted incentives, an active chatbot vs passive mobile app) with AI and machine learning, TAMI helps to enhance patient motivation and reduce barriers to utilization by tailoring referrals that are provided at just the right time for all patients, regardless of their initial level of readiness. Our use of the DSDM provided several key findings, and limitations of the current project suggest promising future areas of research.

First and foremost, MI was the appropriate approach to use with this population as user-centered design interviews highlighted a desire to feel understood, affirmed, and not judged. Furthermore, MI was consistent with the request for evocation and support for personal motivation to quit, which in turn could inform the balance between providing relevant advice and suggestions and giving information to inform but not dictate decisions. As such, TAMI was designed to use compelling and high-fidelity MICO skills such as affirmation, and reflections to guide the patient through the 4 tasks of MI—engagement, a focus on smoking, evoking and resolving ambivalence about quitting smoking, and planning for change. Consistent with MI, the incorporation of MICO skills may also enhance the relational components of MI: Participants felt that TAMI could express empathy by providing a variety of responses and unintrusive check-ins and open-ended questions. The themes identified about what was liked about TAMI are consistent with the MI spirit [28]: The participant felt cared for or that TAMI humanized the experience is important when delivering MI. The judgment-free anonymity component of TAMI is consistent with MI principles of conveying empathy and exploring both the cons and benefits of quitting without judgment.

Other clear preferences emerged during our work with patients who were considering quitting smoking. First, the frequency of the check-ins was linked to the patient’s readiness to quit smoking. Participants expressed that they appreciated how TAMI regularly checked in on their progress and, despite being from a chatbot, these check-ins made them feel cared for during their quitting journey. As with all interventions, the timing of the delivery of TAMI is also important, as frequent check-ins if one is not considering quitting smoking were perceived as potentially annoying. In contrast, frequent contact leading up to and following the quit date was seen as helpful to keep the user focused. Second, there was also a desire to be provided with a variety of tips and external resources to facilitate quitting, thus enabling the user to select which would be most helpful. Inductive analysis of exit interviews revealed that anonymity, regular reminders, and a humanized experience facilitated engagement with TAMI. TAMI does not require any provider involvement, although it allows for and encourages exploring professional assistance with quitting smoking when developing a quit plan. Finally, participants may like the anonymity digital interventions provide when trying to quit smoking, particularly the opportunity to privately explore ambivalence about smoking and options for cessation.

Limitations to this study indicate several aspects of TAMI that require improvement. First, while the anonymity component was liked by some participants, one participant expressed difficulty connecting with the chatbot emotionally, which could be a limitation of the relational component of using TAMI. Second, some individuals felt TAMI did not understand them or their responses. The occasional glitches may leave a person not feeling understood. Thus, TAMI needs to be capable of providing complex reflections that convey an accurate understanding of the user’s personal experiences with smoking. Refinement of the reflections is needed to improve the technical component and strengthen the relational component of TAMI. The technical glitches and relational limitations may be barriers to sustained engagement. Third, as with all interventions, the timing of the delivery of TAMI is also important, as some individuals were not ready to quit smoking and reported that was the reason they did not feel the need for TAMI. Finally, there was limited diversity in those who completed the interviews, especially in the user design interview phase. Therefore, it will be vital to recruit a more diverse group of participants in the subsequent development and testing of TAMI.

A key challenge was balancing structured content delivery with the flexibility needed to simulate a natural, responsive dialogue—a limitation that highlights the need for more dynamic conversational AI. TAMI remains a largely scripted and structured intervention with limited ability to adapt to participant input and questions, thus potentially limiting engagement and follow-through with the quit plan. Additional data regarding pilot feasibility-and-acceptability outcomes will be reported in a future paper. Fortunately, since this work was completed, there have been remarkable advances in the development of large language models (eg, ChatGPT 4.0) that incorporate large amounts of data to craft questions and reflections that are equivalent to human-generated utterances in criteria such as specificity, naturalness, appropriateness, and engagement [34,51,52]. For example, Therabot is a chatbot for mental health treatment that was recently deployed among individuals reporting symptoms of depression and anxiety [53] and was capable of several conversations per day over 8 weeks. Previous conversation content was incorporated into Therabot prompts that included empathic responses, questions, or specific relevant information. However, all prompts were monitored by members of the research team who contacted participants to correct responses (13 times over the course of the study) or further assessment for risk (eg, suicidality; 15 times). As a result, the next steps for the further development of TAMI are to effectively graft the structure of TAMI with more advanced large language models to create opportunities for enhanced evocations and strengthening of the participant’s personal argument for quitting smoking without monitoring. This enhancement will be especially important during the evocation component, which was previously limited to scripted reflections and questions pertinent only to the most immediate response by the participant. Specifically, the chatbot may facilitate a more nuanced discussion of the 5R’s (relevance, risks, rewards, roadblocks, and repetition) for use with smokers who are unmotivated to quit [54].

Digital health applications such as TAMI may fill a vital gap in health care by facilitating conversations that evoke a personal argument for change—whether relevant to smoking or other health-related behaviors. The overall structure of focused conversations has been guided by decades of therapy process research, and the clinicians’ utterances that facilitate these conversations are also well established, for example, reflections, affirmations, and open-ended questions [24,29]. While TAMI successfully delivered MI-consistent content using rule-based logic and NLP, future iterations could explore integration with more advanced AI models to improve personalization and conversational fluidity. Although chatbots may never fully replace human interactions and guidance, ever-evolving chatbots may be able to initiate the conversation about change on a broad scale and efficiently facilitate movement through a stepped continuum of care ending with professional clinicians who resolve complex ambivalence toward change.