An uncontrolled, 24-week pilot feasibility and acceptability study that added reSET-O to standard care for patients with OUD initiating buprenorphine in an OTP and serving as the hub in a Hub and Spoke system was conducted. The hub provides both methadone and buprenorphine treatment in an outpatient clinic specializing in medication treatment for OUDs. reSET-O was prescribed for a 12-week period and was then renewed for a second 12 weeks for each participant to reach a total prescription duration of 24 weeks. English-speaking individuals aged ≥18 years who were on buprenorphine treatment for at least 1 week were eligible to participate and enroll in the pilot study.

reSET-O is a commercially available prescription digital therapeutic that delivers interactive, self-paced psychoeducational therapeutic modules regarding substance use and is based on the TES. TES is a web-based program that provides written, auditory, and video modules that instill cognitive behavioral skills based on a CRA. The CRA is a therapeutic system that teaches coping skills for staying off drugs and builds skills and activities that are consistent with a healthy lifestyle. The topics of these modules range from managing triggers of substance use and building healthier coping and social skills to HIV prevention and reducing high-risk sexual behavior (see Figure 1 for examples of topics as displayed on the clinician dashboard). Modest monetary rewards (such as gift cards) or nonmonetary rewards are earned for module completion and when negative urine drug screens (USDs) are achieved (see Figure 2 for an example of the reward programming viewed by the participant). With the commercial release of reSET-O, new additions included contingent rewards that may be delivered in the form of gift cards to retail outlets, with the cost covered by third-party insurances. Furthermore, reSET-O prompts daily check-ins, beginning with in-app reports of substance use and cravings, as indicated by the participant (see Figure 3 for an example of endorsed cravings as displayed on the clinician dashboard). Finally, the Patient Services Center, currently referred to as PearConnect, created by Pear Therapeutics, the makers of reSET-O, is a call center that connects patients to a dedicated advocate for support throughout their treatment and prescribers and health care providers to clinician dashboard resources. If a lower activity of reSET-O is detected, Patient Services Center representatives can communicate with patients on a weekly basis to troubleshoot any technical difficulties. reSET-O includes a web-based provider dashboard that permits treatment providers to examine participation and patient engagement remotely (see Figure 4 for an example of a participant snapshot as displayed on the clinician dashboard), as well as facilitate future patient-provider clinic discussions on lessons learned from the modules completed.

Institutional review board approval was obtained for this pilot study from Penn State College of Medicine, under study number 9931 on November 27, 2018, and procedures were followed in accordance with the ethical standards of the institutional committee on human experimentation and with the Helsinki Declaration of 1975, as revised in 2000.

Eligible individuals were aged ≥18 years; able to read, write, and comprehend English; had a diagnosis of OUD, as determined through routine clinical evaluation; initiated treatment at a community-based OTP serving as the hub in a Hub and Spoke system of care; and were prescribed buprenorphine for OUD. We excluded any individuals engaging in outpatient detoxification or needing a higher level of care, such as inpatient or residential treatment, as well as those not desiring MOUD. Individuals were invited to join the pilot study during their medical appointments and consented in the clinic if interested. All participants provided written informed consent.

After obtaining participant consent, reSET-O was prescribed to participants for 12 weeks at a time, with a total of 2 prescriptions (24 weeks of consecutive treatment). reSET-O asks participants to log onto the app and complete 4 learning modules per week, each focusing on a particular skill. The modules are presented in a fixed order, beginning with modules on skills to avoid drugs, followed by modules on building a healthy lifestyle and HIV risk reduction. Each completed module yields a chance to earn rewards in the form of gift cards to retail outlets. Participants were evaluated via the self-report of drug use with the Timeline Follow-Back (TLFB) method, USDs, and three mood assessments: the Kessler-10 [29,30], the Posttraumatic Stress Disorder (PTSD) Checklist–Civilian [31-33], and the Patient Health Questionnaire-9 [34]. These evaluations occurred at baseline and at 4, 8, 12, and 24 weeks after study entry. Self-reported drug use was collected using TLFB [35,36] and verified using USDs. Participants were scored as not abstinent at a visit if either a self-report or a USD was positive for opioids. Within the reSET-O, craving assessments were performed throughout the duration of the prescription.

Participant feedback data regarding reSET-O was collected on the Intervention Acceptability Feedback Form (IAFF), evaluating seven characteristics—interest, usefulness, new information learned, ease of understanding, relevancy, satisfaction, and likeability—on a Likert scale of 0 to 9 (0=lowest and 9=highest). The IAFF was completed at each visit after baseline (weeks 4, 8, 12, and 24).

Qualitative interviews with participants were conducted by research assistants between weeks 8 and 12 to capture information from participants about their experience using the app, acceptability of the app, and suggestions for improvement. Approximately 60% (9/15) of the participants completed the interviews. Each participant was asked 9 standard questions (Textbox 1); then, the staff member could ask follow-up and probing questions as needed to elicit additional information. The interviews lasted between 30 and 45 minutes, were digitally audio recorded, and transcribed verbatim for analysis purposes.

A total of 3 research staff members reviewed all 9 interviews, extracting transcribed text relevant and responsive to the domains of satisfaction or dissatisfaction and the likeability of the intervention. Responses to the following research question: “How did participants like using reSET-O?” were categorized into unique themes, which were reviewed and discussed by a larger research team to reach a consensus.

Provider data regarding the acceptability and feasibility of reSET-O were also collected. Clinicians working in the treatment programs were asked to create their own reSET-O accounts and review the learning modules. After 3 weeks of reSET-O use, data from 4 clinical providers were collected through the Weiner Intervention Acceptability, Appropriateness, and Feasibility form (WIAAF; Multimedia Appendix 1), evaluating the acceptability, appropriateness, and feasibility of the app intervention. Each category was assessed through 4 prompts, each using a 5-point Likert scale (1=lowest and 5=highest). The highest score for any given category could be 20. The WIAAF, currently in development, has no cutoff scores for interpretation; however, higher scores indicate greater acceptability, appropriateness, or feasibility.

Of the 15 participants, 3 (20%) female patient participants and 12 (80%) male patient participants with OUD were enrolled in the study, with an average age of 36.2 (SD 9.3) years (Table 1). Approximately 73% (11/15) of the participants identified as White, whereas 7% (1/15), 13% (2/15), and 7% (1/15) identified as Black or African American, biracial, or other, respectively, and 13% (2/15) identified as Hispanic (Table 1). Of these 15 participants, 2 (13%) withdrew consent after baseline; 3 (20%) participants dropped out of clinical care before week 8 and were unable to be reached, and another 3 (20%) became incarcerated between weeks 12 and 24 during the study. Thus, of the 15 participants, 8 (53%) withdrew and/or were unable to complete the study, and 7 (47%) were able to complete all 24 weeks of the study.

In this preliminary feasibility pilot study among patients undergoing buprenorphine treatment for OUD, reSET-O appeared generally acceptable and feasible, warranting further study. However, there was still a high rate of dropout, and the extent to which patients used the app varied. Some patients used the learning modules and appreciated the cognitive behavioral skills covered, whereas others did not engage. Opioid use was low among patients who remained in the study, although many of these patients had already achieved abstinence at the study baseline. Depression and anxiety symptoms were within the mild range of severity, with trends toward improvement. In addition, across all 24 weeks, the participant IAFF satisfaction ratings were between 5 and 9, out of a total of 9, for the seven assessed areas (interest, usefulness, new information learned, ease of understanding, satisfaction, relevance, and likeability), suggesting a good level of satisfaction. These scores decreased slightly over time for all assessed categories, which may be a reflection of the decay in enthusiasm over time. There was a high rate of dropout from the study, with 91% (10/11) of the patients who initiated reSET-O completing 12 weeks of reSET-O and 64% (7/11) completing 24 weeks. This is perhaps not surprising, given the general problem of attrition from apps, as well as the high rates of dropout from the treatment characteristics of patients with SUDs. However, this suggests that strategies are needed to improve engagement and retention in treatment with reSET-O.

One such strategy could be more integration between counseling sessions with clinical providers and the therapy modules delivered by the app. reSET-O is largely psychoeducational and didactic in nature. Future studies should invite counselors and medical providers to discuss reSET-O and the topics therein with patients to make the content more relevant and engaging. Clinicians prescribing medication routinely discuss medication adherence, side effects, and whether and how it may be helpful. Similarly, clinicians prescribing reSET-O could ask about what modules the patient has completed, troubleshoot adherence problems, and discuss how the patient is applying the therapy modules delivered by the app in their daily lives. In this way, in-person and mobile treatments can enhance one another to perhaps increase treatment retention. Some patients suggested additional praise or encouragement for reaching certain milestones, such as maintaining a certain number of favorable urines or reaching a specified length of time of sobriety, which could be further facilitated by counselors. A team approach among providers, patients, and reSET-O may better address any problems with adherence, perceived burden, and the relevance of the topics. The integration of remote therapy with in-person treatment might enhance the salience of reSET-O and reduce enthusiasm decay over time.

Second, provider feedback, although encouraging, provided some helpful suggestions on language. On the basis of the WIAAF scores, the feasibility of implementing reSET-O had the highest average score when compared with the other categories. The category with the lowest average score was appropriateness because of the content language. As noted above, some providers felt that the language appeared stigmatizing in some areas or was difficult to understand (eg, reading level). TES was created >15 years ago, and much has changed in the culture and lexicon surrounding SUDs. This presents a challenge in terms of how a digital therapeutic could be adjusted based on the patient population and setting, perhaps based on real-time feedback from local patients and providers, to provide a product that is maximally inviting and relevant. Regarding the scope of services, providers felt that emergency information might be beneficial to the patient. Mobile technology in this arena has been developed through the Addiction–Comprehensive Health Enhancement Support System [28], and perhaps, interapplication communication can broaden the scope to help more patients.

The monetary remuneration for participants was relatively low compared with traditional evidence-based CM programs [20,37]. reSET-O rewards negative USDs and module completions, with the idea that incentives will help patients participate in the therapy modules; thus, internalizing techniques of cognitive behavioral therapy can prevent ongoing drug use and address problems with cravings, mood, and relationships. Traditionally, CM has shown efficacy in rewarding individuals for appointment adherence and negative USDs alone. Moreover, larger rewards were available in prior grant-funded studies [38,39]. That reSET-O funds incentives through third-party reimbursement is an innovation that overcomes the problem of how to fund incentives and makes CM feasible in community-based practice. However, only modest reward values are possible using this approach. reSET-O, and its forerunner TES, use the prize bowl model for low-cost CM developed by Petry [37]. This turns CM into a game where rewards are determined by chance; some are just verbal (eg, good job), and most rewards have low monetary value, approximately US $5, with an occasional larger reward value, which keeps the overall costs of contingencies low across treatment episodes.

Limitations include that this study was an uncontrolled trial with a small sample size and a high attrition rate. This study is most useful for generating ideas for improving feasibility and acceptability; however, without a control group, it is not possible to evaluate effectiveness. Patients retained in the study were mostly abstinent over the course of the study, and mood ratings were in the mild range of severity, which is encouraging. However, it is not possible to evaluate treatment retention or patients’ individual trends in substance use and mood without a larger sample and control condition. The fact that many patients who were abstinent during the study were already abstinent at baseline suggests the importance of covarying for baseline in future trials. The sample comprised predominantly White men, reflective of the local patient population, affiliated with the one particular clinic in which the study was conducted. The providers did not systematically discuss reSET-O with the enrolled patients during clinical visits. Participants might achieve more therapeutic benefits from reSET-O if it is integrated more into in-person counseling.

Prescription digital therapeutics, such as reSET-O, have the potential to bridge a gap in MOUD, which often prevents potential providers from prescribing these life-saving medications, namely by addressing the need to deliver medication in conjunction with behavioral counseling. For providers who practice without live behavioral counseling on site and within financially stressed clinical programs, a mobile app has the potential to expand a clinic’s ability to meet the needs of populations with SUDs. Larger controlled trials are warranted to evaluate whether this intervention improves the adherence to and outcome of MOUD in community-based treatment settings such as the Hub and Spoke model of care in which this pilot study took place. Future work should examine ways that clinicians can integrate the patients’ participation in an app such as reSET-O into the counseling of their patients in an effort to improve adherence to the use of the app and maximize its impact as a clinician extender.