Published on in Vol 5, No 4 (2021): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/21127, first published .
Brief Interventions via Electronic Health Record Messaging for Population-Based Suicide Prevention: Mixed Methods Pilot Study

Brief Interventions via Electronic Health Record Messaging for Population-Based Suicide Prevention: Mixed Methods Pilot Study

Brief Interventions via Electronic Health Record Messaging for Population-Based Suicide Prevention: Mixed Methods Pilot Study

Original Paper

1NowMattersNow.org, Bellevue, WA, United States

2Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States

3Kaiser Permanente Washington Health Research Institute, Seattle, WA, United States

4Health Services Department, University of Washington, Seattle, WA, United States

Corresponding Author:

Ursula Whiteside, PhD

NowMattersNow.org

1645 140th Ave NE

Suite A41053

Bellevue, WA, 98005

United States

Phone: 1 206 679 6349

Email: ursulawhiteside@gmail.com


Background: New opportunities to create and evaluate population-based selective prevention programs for suicidal behavior are emerging in health care settings. Standard depression severity measures recorded in electronic medical records (EMRs) can be used to identify patients at risk for suicide and suicide attempt, and promising interventions for reducing the risk of suicide attempt in at-risk populations can be adapted for web-based delivery in health care.

Objective: This study aims to evaluate a pilot of a psychoeducational program, focused on developing emotion regulation techniques via a web-based dialectical behavior therapy (DBT) skills site, including four DBT skills, and supported by secure message coaching, including elements of caring messages.

Methods: Patients were eligible based on the EMR-documented responses to the Patient Health Questionnaire indicating suicidal thoughts. We measured feasibility via the proportion of invitees who opened program invitations, visited the web-based consent form page, and consented; acceptability via qualitative feedback from participants about the DBT program; and engagement via the proportion of invitees who began DBT skills as well as the number of website visits for DBT skills and the degree of site engagement.

Results: A total of 60 patients were invited to participate. Overall, 93% (56/60) of the patients opened the invitation and 43% (26/60) consented to participate. DBT skills website users visited the home page on an average of 5.3 times (SD 6.0). Procedures resulted in no complaints and some participant feedback emphasizing the usefulness of DBT skills.

Conclusions: This study supports the potential of using responses to patient health questionnaires in EMRs to identify a high-risk population and offer key elements of caring messages and DBT adapted for a low-intensity intervention. A randomized trial evaluating the effectiveness of this program is now underway (ClinicalTrials.gov: NCT02326883).

JMIR Form Res 2021;5(4):e21127

doi:10.2196/21127

Keywords



Background

Significant momentum supports the work of suicide prevention in health care settings as well as the inclusion of people who have personal experience with suicidal thoughts and behaviors, including systems changes initiatives such as Zero Suicide and new national patient care guidelines such as Recommended Standard Care for People With Suicide Risk: Making Health Care Suicide Safe [1-3]. However, there are significant challenges that hamper major advancements in the field. Although research support for clinical and tertiary treatments exists, these treatments are often intended for those who have already made suicide attempts or have been hospitalized for suicide-related reasons and they likely only have a minimal effect on suicide rates of the overall population [4,5]. This may be because of limited availability and postvention focus. Intensive tertiary interventions for suicidal patients, such as dialectical behavior therapy (DBT) and cognitive behavioral therapy (CBT) for suicide prevention, require significant clinical training for clinicians[6,7]. For patients, they are often costly, time intensive, and inaccessible. Empirically supported interventions target individuals who have already engaged in a suicide attempt [8]. Meanwhile, clinical responses to patients who report suicidal ideation generally focus on acute crises and risk management, such as safety planning, emergency room care, and inpatient psychiatric care. Unfortunately, this sometimes leads to coercive interactions because the onus is on the patient to prove that they are no longer suicidal or at risk to leave the health care setting [9].

Selective prevention—otherwise known as secondary or indicated prevention—focused on outreach, support, and skills development would allow for a more collaborative approach. Owing to several conceptual, methodological, logistical, and ethical challenges, selective prevention for suicide remains to be relatively unexplored. Effective selective prevention for suicide requires three things: (1) accurate methods for identifying those at risk before a suicide attempt, (2) effective interventions suitable for large-scale delivery, and (3) acceptable and efficient models for population-based delivery.

The Patient Health Questionnaire (PHQ) [10], a depression questionnaire commonly documented in electronic medical records (EMRs), can be used to identify those greater short-term (3% risk of suicide attempt in the following month) and long-term (5-10 times greater risk for suicide and suicide attempt over the following 2 years) risks [11,12]. These risk levels are specific to PHQ item 9, which asks whether the patient, over the past 2 weeks, has been bothered by “thoughts that you would be better off dead or of hurting yourself in some way.” Responses of “more than half the days” or “nearly every day” are linked to the elevated risk described, and the risk remains to be significant when controlling for the total score of the remaining PHQ items and demographic variables. More recently, PHQ scores in combination with other EMR data have been used to predict suicide attempts and suicide using empirically derived risk scores [13].

A promising low-intensity intervention is caring messages, comprising brief, nondemanding, unsolicited caring messages sent by a health care provider over time. Although caring messages studies are not without limitations, data suggest that they may reduce the incidence of suicide in psychiatric patients [14,15]. It has also been shown to reduce additional suicide attempts [8,16-18]. Caring messages is among the few interventions that have replicated such findings [2,14,19-21]. The Institute of Medicine and the National Action Alliance for Suicide Prevention identify caring messages as a promising and recommended intervention [2,22], and it has been adapted for email [23]. A possible mechanism of action for caring messages is that it may address a lack of social connections [24].

DBT, another intervention, is a high-intensity treatment demonstrated to reduce suicidal behaviors by teaching individuals how to effectively manage intense emotions and tolerate distress [6]. Reductions in suicidal behaviors have been replicated across at least four DBT randomized controlled trials targeting suicidal individuals [19,25-28]. The key elements of DBT can be adapted for brief interventions. In a component analysis of DBT and a subsequent literature review, preliminary analyses suggest that one element—skills training—is effective in reducing suicidal behaviors [29,30]. Skills training has subsequently been adapted for brief, population-based interventions [31-34].

For the delivery of such interventions, the population-level impact requires a model that is affordable and scalable. Web-based interventions have the potential to reach large numbers of high-risk individuals and have been shown to be effective for support inside and outside medical settings [35-38]. Such interventions have demonstrated effects over usual care, in some cases comparable with in-person treatment [39-43]. Intervention websites, both with and without human support, are scalable for population-based uses in ways that in-person interventions are not [44]. Evidence supporting web-based adaptations of high-intensity treatments such as CBT warrants the development and testing of a web-based DBT skills intervention. The advantages of intervention websites include 24-hour accessibility, greater perceived privacy, and flexibility of location.

Meanwhile, EMR systems can be used not only to support the delivery of web-based interventions but also to identify those at risk and to support patient-provider secure messaging used for intervention. EMRs with secure messaging have the potential to significantly increase the efficiency and dissemination of suicide prevention and suicide attempts. Secure messaging has been successfully used to deliver low-intensity interventions [45-47]. Secure messaging can be used to support the delivery of web-based interventions traditionally delivered in person, such as CBT and motivational interviewing.

Objectives

This pilot study aims to assess the feasibility, acceptability, and engagement of a brief suicide prevention intervention in preparation for a full-scale randomized trial. We evaluated this by measuring patient response to an invitation to a web-based, population-based suicide attempt and suicide prevention intervention, intended to be easily adapted into large health care systems, which included elements of DBT and caring messages.


Overview

We evaluated the feasibility, acceptability, and engagement of a brief web-based suicide prevention intervention using qualitative and quantitative methods. The intervention included EMR-mediated secure messaging linked to a web-based DBT skills platform. Participants included outpatients who were receiving care within Kaiser Permanente Washington, a health system serving approximately 700,000 people in Washington State, which routinely administers the PHQ to all patients receiving care for mental health conditions [10,48]. Patients were eligible if they were aged 18 years or older, had ID verification for secure email messaging through the EMR, had sent or read a secure message in the past year, and had completed a PHQ depression severity questionnaire in the previous week and responded “more than half the days” or “nearly every day” to item 9 (“Thoughts that you would be better off dead or of hurting yourself in some way”). Previous research in this setting indicates that this group would have an expected risk of suicide attempt of approximately 4% over the following year [11,12].

Over the course of 3.5 months, we used an automated EMR data query program to pull 6 samples, identify 60 individuals, and conduct outreach via secure messaging accessible on a mobile app or website. We used this sample size, which could be considered large for a pilot, to gain confidence in our estimates for the feasibility of a large clinical trial. We sent secure message invitations to the DBT skills program approximately every other week to between 5 and 20 eligible participants for each sample (starting at the top of the list) to evaluate ideal caseload management. Invitations were sent 1 to 2 weeks after the visit, if an elevated PHQ item 9 was involved. Patient care continued to be the responsibility of the behavioral health and primary care providers in the Kaiser Permanente Washington system who had administered the PHQ at the recent visit. In addition, a suicide risk protocol involving follow-up by study clinicians (author 1 or 3) for those who indicated imminent risk over secure message was in place, although the protocol was not triggered during the study. The study procedures were approved by the organization’s institutional review board, which included a waiver of research consent for patient identification and intervention invitation; however, this restricted the analysis of additional data (eg, descriptive data of eligible participants). The waiver of consent for patient identification and intervention invitation was granted because this pilot and subsequent trial involved no more than minimal risk and because the purpose of the pilot was to evaluate the feasibility and acceptability of procedures planned for the randomized trial of outreach. Evaluating only patients who actively consented to receive intervention would have yielded a result of questionable validity and generalizability.

The invitation messages, and the messages that followed, can be conceptualized as having two elements: a caring message (a brief and unsolicited expression of care) and support for a web-based DBT skills intervention (Textbox 1). We do not know whether the combination of these elements will be effective, which is not measured in this pilot. The footer of each message included the coach name and title and also included the option to opt out of additional messages, noting that messages were part of a research project and would be part of the EMR, and included numbers to call in case of immediate crisis. The secure message invitation included a link to a web-based video consent form (Multimedia Appendix 1). Patients who did not open the invitation received a telephone call. Those who opened it but did not visit the consent received a secure message reminder. Invitation messages were designed based on patient feedback that they would be most drawn to an intervention that made them feel cared for (eg, subject lines expressing concern), that was personalized, that others like them had found helpful, and that included examples with real people [49,50]. Once consent was confirmed, participants arrived at the landing page of the DBT skills site.

The web-based DBT skills site included four DBT skills (mindfulness, opposite action, mindfulness of current emotion, and paced breathing). This content focused on training in specific skills to manage upsetting or painful emotions that can precipitate suicidal thoughts and behavior. Content was based on a brief DBT skills face-to-face intervention developed and pilot-tested by the first author [33] and included a video-based series of didactic and personal story demonstrations of DBT skill use with a team of collaborators with lived experience of suicidal thoughts and behavior (Team Now Matters Now). DBT experts, including the DBT developer Marsha Linehan, also consulted on intervention development. The website landing page included a brief orientation video describing the intervention content and what to expect (Multimedia Appendix 2). Below the video on the landing page was a list of the four DBT skills, with a brief description (eg, “Mindfulness - A way to be present for what is most important in your life”; Textbox 2). Participants could also choose to learn more about Team Now Matters Now or provide feedback about the program. The DBT skills content was housed on a Health Insurance Portability and Accountability Act–compliant modular software platform (DatStat). Each of the four DBT skills followed the same path that included a 30-second overview or teaser, a 5- to 8-minute didactic video describing the skill as it might in a traditional DBT skills group, several 1- to 5-minute demonstration videos with real people talking about how they used the DBT skill in their own life, and a 1-page practice assignment similar to what one might find in a DBT skills group. One could return to the landing page to select another skill at any point in the pathway of a particular skill. In addition to the other benefits of web-based interventions outlined in the introduction, they also provide a consistent patient experience and are baked in fidelity [51].

Secure message invitation.

Dear [First Name]

Sometimes a little extra help can be just what someone needs to get through tough times.

A new web-based program called Now Matters Now was designed to give you that help when you need it. The program uses real people to teach specific coping skills, like mindfulness and paced breathing.

We invite you to try Now Matters Now today. Learn more at <Link to Intervention Website>

We care about you.

My job is to encourage you to use the program over the next three months and to practice the skills you are learning. If you haven’t visited the program in a while, I’ll send you a message to remind you. If you have visited, I’ll check with you about what you find helpful.

I’ll communicate with you through messages, and in order to message you, I will access your medical record.

Sending care,

Textbox 1. Secure message invitation.
Dialectical behavior therapy skills site landing page text.
  • TEAM NOW MATTERS NOW—Introducing the real people who use these skills and helped develop this program
  • MINDFULNESS—A way to be present for what is most important in your life
  • OPPOSITE ACTION—Living a full and healthy life, despite what negative or unhelpful emotions and thoughts are telling you to do
  • MINDFULNESS OF CURRENT EMOTION—Observing, honoring, and moving through strong emotions without being controlled by them
  • PACED BREATHING—A different and scientific approach to the old saying “Just Breathe”
  • GIVE US FEEDBACK—Did you have technical problems with this program? Do you want to give us feedback about the program, good or bad?
Textbox 2. Dialectical behavior therapy skills site landing page text.

The DBT skills intervention was supported by an interventionist or program coach for 3 months. In line with the stage model of behavior therapies, the interventionist for this study was the lead intervention developer (UW) [51-53]. This included caring messages outreach and support for the use of the DBT skills site. For 2 months, the coach delivered this support exclusively via secure messaging, reaching out approximately once per week for participants’ engagement and less frequently for those who had not responded to the most recent secure message. For the third month, the patient could continue to engage in the intervention or send the coach messages and the coach would reply, but the coach would not actively outreach or prompt the patient. The intervention coach did not provide psychotherapy but instead provided caring messages as well as outreach in the form of reinforcement and contingency management surrounding the use of the DBT skills site. This would include following up with patients who had not visited the site in several weeks, suggesting specific skills they might find useful or thanking the patient for visiting the site and include a tip or point relevant to the DBT skill they viewed (Textbox 3 provides an example of this type of message). The research team met weekly with the senior investigator (author GES) to ensure safety monitoring and coach fidelity. The senior investigator also had access to review the secure messages of the coach to monitor secure message fidelity to the described approach.

Coach outreach/and support message example.

Hi [First Name],

I’m writing to check-in from the Now Matters Now program. Do any of the skills look interesting to you to try?

Paced Breathing is a breathing technique that can calm the mind and body naturally. Some people like to use Paced Breathing when they’re feeling nervous or anxious. It can also be a helpful way to clear the mind if you’re having a difficult time falling asleep.

The key to Paced Breathing is to have your exhale be longer than your inhale. It can be a good way to center yourself and makes it easier to then practice other skills.

If you try it, let me know how it goes.

To learn more, please click the link below:

[link to Consent “form” video]

Take care,

Textbox 3. Coach outreach/and support message example.

Feasibility, Acceptability, and Engagement

We evaluated feasibility by measuring the proportion of invited patients who opened the program invitation, the proportion of patients who visited the consent page, and the proportion of patients who consented. We qualitatively evaluated acceptability by using secure message feedback that the coach received from patients invited to participate and summarized relevant themes [54]. We evaluated engagement by measuring the proportion of participants who began using the DBT skills site and the number of site visits. We also measured visits to each of the featured DBT skills in the program among a subset of the sample, and the first 20 participants were invited. We did not collect engagement data beyond the first 20 patients eligible for this pilot study because the original purpose of these analyses was to help confirm that the training website was working as planned (to help evaluate the feasibility of a large randomized trial).


Feasibility

In total, 60 patients were invited to participate (Figure 1). Of these 60 patients, 17 (28%) patients did not open the invitation within 1 week of delivery and received a telephone call (10 were left a voicemail and 7 were spoken to). Up to 2 secure message reminders were sent to 35% (21/60) of patients. Overall, 3% (2/60) of patients refused to participate. Furthermore, 93% (56/60) opened the invitation (indicating that these people did, as far as we know, read and receive the caring message aspect of this contact), 65% (39/60) clicked the link to the consent form, and 43% (26/60) consented to participate.

Figure 1. Study flow.
View this figure

Acceptability

The coach received no complaints from those invited to participate in the program. The feedback the coach received via secure messaging mainly included descriptions of how participants had used the program, specifically their helpful experiences practicing each of the four program skills and the challenges they experienced. One participant described a helpful experience in trying mindfulness skills, which involved bringing attention and awareness to their thoughts:

I did the first lesson last night—I was able to use it today when I started having negative thoughts that wouldn’t stop. I cleared my mind and started looking at the trees, buildings and other things around me and noticing the little details of things. It actually calmed me down and gave me a sense of control.

Another participant described the experience of practicing paced breathing:

I am doing my best to make the Paced Breathing a daily practice. I forget sometimes until I am “over the edge” and am in deep panic/terror, but I can still even then use the breathing to help get me feeling more calm.

Another participant described a powerful experience using an opposite action, which involved choosing to do the opposite of ineffective emotional urges:

Yesterday I began the module called “Opposite Action” and printed it out after I completed it. Doing it was empowering but also very emotional for me.

Finally, a participant described a plan to use mindfulness of current emotion, which involved learning how to bring attention and awareness to the bodily sensations of emotions:

I used [a DBT skill] about 5 times today when I was out walking—it keeps my emotions under control, so I don’t start crying. I’m going to try [the]“mindful emotion” thing tomorrow. I appreciate [my health care organization] offering something like this—I like having concrete tools I can use.

In terms of challenges, participants described difficulties in using their skills effectively while experiencing intense emotions. For example, one participant described:

One inescapable problem I have found is that once the line that gets crossed where “there is no hope” of being able to turn around the anxiety, I have little ability to do anything constructive to stop the terror from escalating ... if I am not too “far gone” the skills do help to lower the physical and mental/emotional distress.

Engagement

In total, 92% (24/26) of the participants that consented had visited the landing page of DBT skills. These DBT skills site users logged in on an average of 5.3 times to the home page (SD 6.0, range 1-26). We examined a subsample of those consenters to look at engagement in the specific DBT modules (eg, opposite action). Among the first 20 participants enrolled, 19 (95%) read the invite. Of these 19 participants, 12 (63%) visited the website and 8 (42%) agreed to participate. Of those who consented, participant engagement in the different skills ranged from 38% (3/8) for paced breathing to 88% (7/8) for opposite action (Table 1).

Table 1. Dialectical behavior therapy skill engagement for subsample (N=8).
Program visitsOpposite actionMindfulnessMindfulness of current emotionPaced breathing
Visited at least once, n (%)7 (88)4 (50)6 (75)3 (38)
Average visit, range (min)23 (1-54)34 (2-65)13 (1-57)18 (3-30)
Visited at least twice, n (%)4 (50)3 (38)1 (13)3 (38)
Visited 3 times or more, n (%)2 (25)3 (38)1 (13)1 (13)

Principal Findings

This pilot study demonstrated the feasibility, acceptability, and engagement of a population-based intervention involving active elements of caring messages and DBT. Specifically, we demonstrated that by using an accurate method for identifying those at risk for suicide and suicide attempt—in this case, PHQ item 9—we can adapt existing interventions to be suitable for delivery on a large scale and that these models are promising for intervention delivery.

The number of patients who received the initial message that involved elements of caring messages was high (56/60, 93%). This provides support for the delivery of caring messages through this type of intervention (delivered over time through secure messaging with the support of a coach or care manager). The rate of patients who actually reached the DBT skills site content (26/60, 43%) indicates a need for strategies to reduce barriers to DBT skills site participation, such as simplifying the invitation and consent process for patients. The steepest drops were between viewing the message and visiting the consent page (17/56, 30% of those remaining were lost), and between visiting the consent page and consenting to the DBT skills intervention (13/39, 33% of those remaining were lost; Figure 1). Although it is reasonable to hypothesize that those dropping off were concerned about loss of autonomy or stigma [55,56], participants eligible for this study had already made a significant suicidal disclosure in endorsing suicidal thoughts at a recent visit. Furthermore, while the consent video explicitly refers to suicidal thoughts, the invitation message did not. Indeed, patients did not know that was part of the reason they were approached until they reached the consent video. Notably, all patients in the program had already received care in the health system. It is certainly possible that some patients declined to participate because they were already satisfied with their care. Patients who gave consent to intervention participation appeared to engage, and some gave positive feedback about the content of the intervention.

Limitations

A limitation of our study was that the data analysis was limited to anonymized information regarding program participation. Health records data to identify potential participants were used under a waiver of informed consent, so we did not retain demographic, diagnostic, or other clinical information, including the total PHQ score. Although patients were identified as eligible based on their high severity of suicidal thoughts and all eligible patients were enrolled in the study, it could be that there were more and less severe patients within this group and that DBT skills intervention participation varied depending on the severity. Furthermore, participation among those invited may have been reduced by the need for a research consent process. We suspect that engagement would have been higher without this step between secure message invitation and the DBT skills site. In addition, we do not yet know whether the combination of elements of caring messages with brief DBT skills support will be effective, given that caring messages are intended to be nondirective. Finally, it is possible that those who received the invitation to the intervention felt singled out in some way, having had a visit recently where they disclosed suicidal thoughts. Indeed, qualitative research indicates that those who receive the PHQ-9 have significant concerns about how the information regarding the ninth item and suicide will be used [55,56]. Instead, the invitation might read, “As part of my job I reach out to many people we think might benefit from this program....” The key would be to not add too much text for patients to read, while remaining mindful of potential patient fears associated with suicide or mental health–related care (eg, loss of autonomy and psychiatric hospitalizations) [56,57]. Furthermore, our sample of engagement data was small (n=20). It would have been helpful to collect engagement data for all patients invited to participate in the intervention.

As noted by one participant, once one’s emotions reach a certain threshold, the DBT skills become more difficult to access. One limitation of this intervention is that the DBT skills content was not framed in terms of skills for day-to-day stressors versus skills for acute crises. Future interventions may benefit from tailoring additional skills to match the spectrum of emotional stress/crises [58].

Conclusions

This project addresses a number of the most important conceptual, methodological, logistical, and ethical challenges that hamper major advancement in the field of suicide prevention. Qualitative and quantitative data from this study were used to inform the design of a full-scale effectiveness trial (ClinicalTrials.gov: NCT02326883). This research, led by four health care systems within the Mental Health Research Network, is underway to determine its effectiveness [59]. In that trial, participants are randomly assigned to continued usual care, an outreach-based care management intervention, or a coach-supported, web-based DBT skills training program with elements of caring messages, based on the intervention developed and evaluated here.

The videos that were developed as part of this project that make up most of the DBT skills content are publicly available to both the researchers and the public as one piece of NowMattersNow.org [60]. Survey participants to this website have described reductions in suicidal thoughts and negative emotions while visiting the website [34]. Although brief web-based interventions cannot replace intensive tertiary interventions (eg, in-person DBT/CBT), they hold promise as a low-intensity, population-based intervention by ensuring that the right content is consistently delivered to individuals at high risk of suicide and suicide attempt at the time it is needed.

Acknowledgments

This study was funded by the National Institute of Mental Health (5R34MH097836 and 1UH2AT007755) and the American Foundation for Suicide Prevention. The authors also thank the real people who helped create this program: Lori, Russ, Jaeden, Julie, and Joey and all the new members of their team. They are the authors’ guided inspiration. The authors thank Drs Marsha Linehan and Kate Comtois for the consultation intervention development.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Consent “form” video.

MP4 File (MP4 Video), 42479 KB

Multimedia Appendix 2

Program landing page video.

MOV File , 16688 KB

  1. Hogan MF. To prevent suicide, address it directly. Psychiatr Serv 2018 Jul 01;69(7):737 [FREE Full text] [CrossRef] [Medline]
  2. Recommended standard care for people with suicide risk: making health care suicide safe. National Action Aliance for Suicide Prevention. 2018.   URL: https:/​/theactionalliance.​org/​sites/​default/​files/​action_alliance_recommended_standard_care_final.​pdf [accessed 2021-03-25]
  3. The way forward: pathways to hope, recovery, and wellness with insights from lived experience. The National Action Alliance for Suicide Prevention. 2014.   URL: https:/​/theactionalliance.​org/​resource/​way-forward-pathways-hope-recovery-and-wellness-insights-lived-experience [accessed 2021-03-25]
  4. Lewis G, Hawton K, Jones P. Strategies for preventing suicide. Br J Psychiatry 2018 Jan 03;171(4):351-354 [FREE Full text] [CrossRef]
  5. Kuckertz JM, Silverman AL, Bullis JR, Björgvinsson T, Beard C. Awareness of and attitudes toward CBT, DBT, and ACT in an acute psychiatric sample. J Clin Psychol 2020 Apr 11;76(4):749-768. [CrossRef] [Medline]
  6. Linehan MM. Cognitive-behavioral treatment of borderline personality disorder. In: Diagnosis and Treatment of Mental Disorders. New York: Guilford Press; 1993:-.
  7. Stanley B, Brown G, Brent DA, Wells K, Poling K, Curry J, et al. Cognitive-behavioral therapy for suicide prevention (CBT-SP): treatment model, feasibility, and acceptability. J Am Acad Child Adolesc Psychiatry 2009 Oct;48(10):1005-1013. [CrossRef]
  8. Turecki G, Brent DA, Gunnell D, O'Connor RC, Oquendo MA, Pirkis J, et al. Suicide and suicide risk. Nat Rev Dis Primers 2019 Oct 24;5(1):74. [CrossRef] [Medline]
  9. Jordan JT, McNiel DE. Perceived coercion during admission into psychiatric hospitalization increases risk of suicide attempts after discharge. Suicide Life Threat Behav 2020 Feb 04;50(1):180-188. [CrossRef] [Medline]
  10. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001 Sep;16(9):606-613 [FREE Full text] [CrossRef] [Medline]
  11. Simon GE, Rutter CM, Peterson D, Oliver M, Whiteside U, Operskalski B, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv 2013 Dec 01;64(12):1195-1202 [FREE Full text] [CrossRef] [Medline]
  12. Simon GE, Coleman KJ, Rossom RC, Beck A, Oliver M, Johnson E, et al. Risk of suicide attempt and suicide death following completion of the patient health questionnaire depression module in community practice. J Clin Psychiatry 2016 Feb 24;77(02):221-227. [CrossRef]
  13. Simon GE, Johnson E, Lawrence JM, Rossom RC, Ahmedani B, Lynch FL, et al. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry 2018 Oct 01;175(10):951-960 [FREE Full text] [CrossRef] [Medline]
  14. Motto JA, Bostrom AG. A randomized controlled trial of postcrisis suicide prevention. Psychiatr Serv 2001 Jun;52(6):828-833. [CrossRef] [Medline]
  15. Motto JA. Suicide prevention for high-risk persons who refuse treatment. Suicide Life Threat Behav 1976;6(4):223-230. [Medline]
  16. Hawton K, Townsend E, Arensman E, Gunnell D, Hazell P, House A, et al. Psychosocial versus pharmacological treatments for deliberate self harm. Cochrane Database Syst Rev 2000(2):-. [CrossRef] [Medline]
  17. Comtois KA, Linehan MM. Psychosocial treatments of suicidal behaviors: a practice-friendly review. J Clin Psychol 2006 Mar;62(2):161-170. [CrossRef] [Medline]
  18. Hassanian-Moghaddam H, Sarjami S, Kolahi AA, Carter GL. Postcards in Persia: randomised controlled trial to reduce suicidal behaviours 12 months after hospital-treated self-poisoning. Br J Psychiatry 2011 Apr 02;198(4):309-316. [CrossRef] [Medline]
  19. Linehan MM, Comtois KA, Murray AM, Brown MZ, Gallop RJ, Heard HL, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry 2006 Jul 01;63(7):757-766. [CrossRef] [Medline]
  20. Lieb K, Zanarini MC, Schmahl C, Linehan MM, Bohus M. Borderline personality disorder. Lancet 2004 Jul;364(9432):453-461. [CrossRef]
  21. Stanley B, Brown GK, Brenner LA, Galfalvy HC, Currier GW, Knox KL, et al. Comparison of the safety planning intervention with follow-up vs usual care of suicidal patients treated in the emergency department. JAMA Psychiatry 2018 Sep 01;75(9):894-900 [FREE Full text] [CrossRef] [Medline]
  22. Institute Of Medicine, Board on Neuroscience and Behavioral Health, Committee on Pathophysiology and Prevention of Adolescent and Adult Suicide. Reducing Suicide: A National Imperative. Washington, DC: National Academies Press; 2002.
  23. Ammerman BA, Gebhardt HM, Lee JM, Tucker RP, Matarazzo BB, Reger MA. Differential preferences for the caring contacts suicide prevention intervention based on patient characteristics. Arch Suicide Res 2020 Aug 01;24(3):301-312. [CrossRef] [Medline]
  24. Landes SJ, Kirchner JE, Areno JP, Reger MA, Abraham TH, Pitcock JA, et al. Adapting and implementing caring contacts in a department of veterans affairs emergency department: a pilot study protocol. Pilot Feasibility Stud 2019 Oct 10;5(1):115 [FREE Full text] [CrossRef] [Medline]
  25. Linehan MM. Dialectical behavior therapy for treatment of borderline personality disorder: implications for the treatment of substance abuse. NIDA Res Monogr 1993;137:201-216. [Medline]
  26. Linehan MM, Armstrong HE, Suarez A, Allmon D, Heard HL. Cognitive-behavioral treatment of chronically parasuicidal borderline patients. Arch Gen Psychiatry 1991 Dec;48(12):1060-1064. [CrossRef] [Medline]
  27. Verheul R, Van Den Bosch LM, Koeter MW, De Ridder MA, Stijnen T, Van Den Brink W. Dialectical behaviour therapy for women with borderline personality disorder: 12-month, randomised clinical trial in The Netherlands. Br J Psychiatry 2003 Feb 02;182(2):135-140. [CrossRef] [Medline]
  28. Koons CR, Robins CJ, Lindsey Tweed J, Lynch TR, Gonzalez AM, Morse JQ, et al. Efficacy of dialectical behavior therapy in women veterans with borderline personality disorder. Behavior Therapy 2001;32(2):371-390. [CrossRef]
  29. Linehan MM, Korslund KE, Harned MS, Gallop RJ, Lungu A, Neacsiu AD, et al. Dialectical behavior therapy for high suicide risk in individuals with borderline personality disorder: a randomized clinical trial and component analysis. JAMA Psychiatry 2015 May;72(5):475-482. [CrossRef] [Medline]
  30. Neacsiu AD, Rizvi SL, Linehan MM. Dialectical behavior therapy skills use as a mediator and outcome of treatment for borderline personality disorder. Behav Res Ther 2010 Sep;48(9):832-839 [FREE Full text] [CrossRef] [Medline]
  31. Waltz J, Dimeff LA, Koerner K, Linehan MM, Taylor L, Miller C. Feasibility of using video to teach a dialectical behavior therapy skill to clients with borderline personality disorder. Cogn Behav Pract 2009 May;16(2):214-222. [CrossRef]
  32. Valentine SE, Smith A, Stewart K. A review of the empirical evidence for DBT skills training as a stand-alone intervention. In: The Handbook of Dialectical Behavior Therapy. Amsterdam: Elsevier; 2020:325-358.
  33. Whiteside U. A brief personalized feedback intervention integrating a motivational interviewing therapeutic style and dialectical behavioral therapy skills for depressed or anxious heavy drinking young adults. Seattle: University of Washington; 2010.   URL: https:/​/www.​nowmattersnow.org/​wp-content/​uploads/​2021/​04/​2010.​Whiteside.​Dissertation.​A_Brief_Personalized_Feedback_Reduced-File-Size.​pdf [accessed 2021-04-05]
  34. Whiteside U, Richards J, Huh D, Hidalgo R, Nordhauser R, Wong AJ, et al. Development and evaluation of a web-based resource for suicidal thoughts: nowmattersnow.org. J Med Internet Res 2019 May 02;21(5):- [FREE Full text] [CrossRef] [Medline]
  35. Lattie EG, Adkins EC, Winquist N, Stiles-Shields C, Wafford QE, Graham AK. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res 2019 Jul 22;21(7):- [FREE Full text] [CrossRef] [Medline]
  36. Chan S, Li L, Torous J, Gratzer D, Yellowlees PM. Review of use of asynchronous technologies incorporated in mental health care. Curr Psychiatry Rep 2018 Aug 28;20(10):85. [CrossRef] [Medline]
  37. Carolan S, Harris PR, Cavanagh K. Improving employee well-being and effectiveness: systematic review and meta-analysis of web-based psychological interventions delivered in the workplace. J Med Internet Res 2017 Jul 26;19(7):e271 [FREE Full text] [CrossRef] [Medline]
  38. Sin J, Galeazzi G, McGregor E, Collom J, Taylor A, Barrett B, et al. Digital interventions for screening and treating common mental disorders or symptoms of common mental illness in adults: systematic review and meta-analysis. J Med Internet Res 2020 Sep 02;22(9):- [FREE Full text] [CrossRef] [Medline]
  39. Andrews G, Davies M, Titov N. Effectiveness randomized controlled trial of face to face versus Internet cognitive behaviour therapy for social phobia. Aust N Z J Psychiatry 2011 Apr;45(4):337-340. [CrossRef] [Medline]
  40. Christensen H. Computerised therapy for psychiatric disorders. Lancet 2007 Jul;370(9582):112-113. [CrossRef]
  41. Kaltenthaler E, Brazier J, De Nigris E, Tumur I, Ferriter M, Beverley C, et al. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Assess 2006 Sep;10(33):iii, xi-ixiv, 1 [FREE Full text] [CrossRef] [Medline]
  42. Neighbors C, Lewis MA, Atkins DC, Jensen MM, Walter T, Fossos N, et al. Efficacy of web-based personalized normative feedback: a two-year randomized controlled trial. J Consult Clin Psychol 2010 Dec;78(6):898-911 [FREE Full text] [CrossRef] [Medline]
  43. Hester RK, Delaney HD, Campbell W. ModerateDrinking.Com and moderation management: outcomes of a randomized clinical trial with non-dependent problem drinkers. J Consult Clin Psychol 2011 Apr;79(2):215-224 [FREE Full text] [CrossRef] [Medline]
  44. Andrews G, Titov N. Is internet treatment for depressive and anxiety disorders ready for prime time? Med J Aust 2010 Jun 07;192(S11):45-47. [CrossRef] [Medline]
  45. Ralston JD, Revere D, Robins LS, Goldberg HI. Patients' experience with a diabetes support programme based on an interactive electronic medical record: qualitative study. Br Med J 2004 May 15;328(7449):1159 [FREE Full text] [CrossRef] [Medline]
  46. Simon GE, Ludman EJ, Goodale LC, Dykstra DM, Stone E, Cutsogeorge D, et al. An online recovery plan program: can peer coaching increase participation? Psychiatr Serv 2011 Jun;62(6):666-669 [FREE Full text] [CrossRef] [Medline]
  47. Simon GE, Ralston JD, Savarino J, Pabiniak C, Wentzel C, Operskalski BH. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med 2011 Jul 8;26(7):698-704 [FREE Full text] [CrossRef] [Medline]
  48. Richards JE, Parrish R, Lee A, Bradley K, Caldiero R. An integrated care approach to identifying and treating the suicidal person in primary care. Psychiatric Times. 2019.   URL: https:/​/www.​psychiatrictimes.com/​special-reports/​integrated-care-approach-identifying-and-treating-suicidal-person-primary-care [accessed 2020-09-21]
  49. Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, et al. Figure correction: designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res 2015 Apr 13;17(4):e69 [FREE Full text] [CrossRef] [Medline]
  50. Whiteside U, Lungu A, Richards J, Simon GE, Clingan S, Siler J, et al. Designing messaging to engage patients in an online suicide prevention intervention: survey results from patients with current suicidal ideation. J Med Internet Res 2014 Feb 07;16(2):e42 [FREE Full text] [CrossRef] [Medline]
  51. Eaton LH, Doorenbos AZ, Schmitz KL, Carpenter KM, McGregor BA. Establishing treatment fidelity in a web-based behavioral intervention study. Nurs Res 2011;60(6):430-435 [FREE Full text] [CrossRef] [Medline]
  52. Carroll KM, Nuro KF. One size cannot fit all: a stage model for psychotherapy manual development. Clin Psychol: Sci and Pract 2002;9(4):396-406 [FREE Full text] [CrossRef]
  53. Rounsaville BJ, Carroll KM, Onken LS. A stage model of behavioral therapies research: getting started and moving on from stage I. Clin Psychol: Science and Pract 2001;8(2):133-142 [FREE Full text] [CrossRef]
  54. Attride-Stirling J. Thematic networks: an analytic tool for qualitative research. Qual Res 2016 Nov 07;1(3):385-405 [FREE Full text] [CrossRef]
  55. Richards JE, Hohl SD, Whiteside U, Ludman EJ, Grossman DC, Simon GE, et al. If You Listen, I Will Talk: the experience of being asked about suicidality during routine primary care. J Gen Intern Med 2019 Oct 25;34(10):2075-2082 [FREE Full text] [CrossRef] [Medline]
  56. Richards JE, Whiteside U, Ludman EJ, Pabiniak C, Kirlin B, Hidalgo R, et al. Understanding why patients may not report suicidal ideation at a health care visit prior to a suicide attempt: a qualitative study. Psychiatr Serv 2019 Jan 01;70(1):40-45 [FREE Full text] [CrossRef] [Medline]
  57. Han J, Batterham PJ, Calear AL, Randall R. Factors influencing professional help-seeking for suicidality. Crisis 2018 May;39(3):175-196 [FREE Full text] [CrossRef] [Medline]
  58. Linehan MM. DBT Skills Training Manual, 2nd edition. New York: Guilford Press; 2015:1-504.
  59. Simon GE, Beck A, Rossom R, Richards J, Kirlin B, King D, et al. Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial. Trials 2016 Sep 15;17(1):452 [FREE Full text] [CrossRef] [Medline]
  60. Whiteside U, Richards J, Ellsworth M. NowMattersNow.   URL: https://www.nowmattersnow.org/ [accessed 2018-12-15]


CBT: cognitive behavioral therapy
DBT: dialectical behavior therapy
EMR: electronic medical record
PHQ: Patient Health Questionnaire


Edited by G Eysenbach; submitted 14.09.20; peer-reviewed by A Graham, C Wilks, N Karnik; comments to author 02.10.20; revised version received 15.11.20; accepted 15.03.21; published 12.04.21

Copyright

©Ursula Whiteside, Julie Richards, Gregory E Simon. Originally published in JMIR Formative Research (http://formative.jmir.org), 12.04.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.