Evaluation of the Effectiveness of Suicide.ca, Quebec’s Digital Suicide Prevention Strategy Platform: Cross-Sectional Descriptive Study

Introduction

Overview

Information and communication technologies (ICTs) are increasingly used for suicide prevention. As demonstrated in a scoping study by Rassy et al [1], ICTs are used in a multitude of universal, selective, and indicated suicide prevention strategies.

As defined by the World Health Organization (WHO) [2], universal prevention strategies are designed to reach a whole population. They aim to optimize health for all and minimize the risk of suicide by removing barriers to care, expanding access to support services, and strengthening protective factors. Universal suicide prevention strategies using ICTs typically aim to promote mental health literacy, health behaviors, and help seeking [1]. They often take the form of psychoeducational websites and web-based awareness campaigns. Selective prevention strategies, in contrast, are aimed at specific groups or communities [2]. They can be developed based on sociodemographic or geographic characteristics or according to the prevalence of risk factors [3]. They may target individuals who do not exhibit suicidal behaviors but who are at risk for suicide [2]. Selective suicide prevention strategies using ICTs focus on identifying suicidal internet users, offering mental health self-management programs, and providing web-based training [1]. These strategies are, for the most part, included in broader programs involving direct contact between distressed individuals and health professionals. Finally, indicated prevention strategies focus on individuals who have suicidal ideation or who have previously attempted suicide [2]. Indicated suicide prevention strategies using ICTs include web-based suicide risk assessment and triage systems, web-based psychotherapy, chat- or text-based crisis interventions, and digital tools to support face-to-face interventions [1].

Despite the vast body of scientific literature discussing the application of ICT in suicide prevention, few studies have addressed how multiple ICT-based interventions can be combined and articulated together in the development of a digital suicide prevention platform [3] or, more broadly, as part of a digital suicide prevention strategy (DSPS). In a context where an increasing number of countries are developing digital health strategies [4], guidelines must be developed to guide organizations that wish to develop a DSPS. To deploy the 3 levels of prevention defined by the WHO (universal, selective, and indicated), it is crucial that these guidelines contain information on the best strategies for reaching the general population, identifying people with risk factors for suicide on the internet, and providing direct support to people struggling with suicidal issues.

Rassy et al [1] discussed ICT-based interventions aimed at a broad audience, including educational websites, web-based awareness campaigns, and social media platforms. They spotlight the It Gets Better campaign launched in 2010, primarily directed at bullied teenagers, especially those identifying as gay, to prevent suicide. The campaign involved videos created by gay adults that conveyed the message that these teens’ lives would improve over time. These videos were hosted on an educational website and YouTube channel and extensively shared on social media, accumulating >50 million views [5]. Despite the limited evaluation of the effects of educational websites, web-based awareness campaigns, and social media initiatives, studies have demonstrated promising outcomes in terms of reducing suicidal ideation and behavior as well as enhancing knowledge, attitudes, and help-seeking behavior [1]. As highlighted by Rassy et al [1], further research is required to determine the specific characteristics of these interventions that are associated with positive outcomes. In addition, there is a need to develop effective methods for identifying beneficial websites and social media posts and to inform users about or guide them toward internet content that may be of help to them.

Using Google searches holds promise in identifying individuals at risk of suicide on the internet and guiding them toward beneficial web-based content, as individuals with suicidal ideations tend to search for suicide-related information on the internet [6-8]. Indeed, multiple studies have established a correlation between suicide-related Google searches and suicide rates across diverse populations [9-11]. The findings from these studies suggest that suicide-related Google searches can be used to identify individuals at risk of suicide on the internet, recognize intervention opportunities, and facilitate real-time monitoring of suicide risk at a population level. However, it is important to acknowledge that only a small number of real-world interventions based on Google searches have been developed to date.

An example of such an intervention is the study conducted by Sueki et al [12], in which they implemented a web-based gatekeeping intervention in Japan. This intervention involved targeted web-based advertisements directed at individuals searching for suicide-related terms on the web, redirecting them to an email-based consultation. The results of the study indicated a significant reduction in users’ suicidal ideation 4 weeks after being identified on the internet and redirected to the email-based intervention. Furthermore, Onie et al [13] conducted 2 studies to assess engagement levels with 2 advertisement campaigns on Google targeting individuals contemplating suicide. The researchers noted significant engagement with both campaigns, as evidenced by high click-through and conversion rates. Similarly, Google displays crisis helpline numbers at the top of the search results when a user performs a suicide-related search. To our knowledge, no empirical data on the actual impact of Google initiative on helplines use worldwide have been published to date. In addition, data on the characteristics of individuals who were identified by Google and directed to help services are also unavailable.

Regarding ICT-based interventions aimed at providing direct support to individuals facing suicidal issues, crisis lines that offer text- and chat-based services are likely to be the primary source of assistance for those seeking help on the internet. Research on the use and effectiveness of crisis text- and chat-based services is limited. Studies so far have shown that they seem effective in reducing users’ distress [14-18]. However, they reported quite different results regarding the suicidality of users. For example, evaluation of the Netherlands’ 113Online chat service [16] and the US Lifeline Crisis Chat [16] suggested that >80% of their users were experiencing suicidal thoughts. In contrast, evaluation of crisis text lines in the United States and Australia [17,18] indicated that the proportion of users experiencing suicidality was <30%.

In addition, studies suggest that conducting suicide risk assessments during chat- and text-based interactions poses challenges, resulting in fewer assessments completed by counselors [14,19]. These gaps in suicide risk assessments can impact the identification of callers at risk of suicide and potentially bias the reported statistics on suicidal callers by these services. Consequently, providing an accurate description of suicidality among users of these services becomes challenging. Therefore, it is imperative for crisis lines offering text- and chat-based interventions to establish strategies that enable a precise depiction of service users and an assessment of their level of suicidality, rather than solely relying on intervention reports completed by counselors.

In a similar vein, Zabelski et al [20] emphasized the need to investigate the accessibility of chat and text services for high-risk populations in their policy focus review of suicide prevention research on crisis lines. They suggested that if the effectiveness of these services is established, additional studies should be conducted to better understand the demographics and risk profiles of service users as well as to develop effective strategies to promote these services among high-risk populations.

This brief overview of ICT-based interventions highlights the potential of ICTs in the field of suicide prevention. However, it also underscores the existence of significant knowledge gaps that can have important implications for the development of DSPSs. Moreover, essential information required by mental health resources seeking to incorporate ICT-based interventions into their clinical practices is frequently absent from scholarly articles. For example, the technical aspects of developing, maintaining, and operating digital interventions have rarely been discussed [21,22]. Furthermore, there is a dearth of comprehensive research that adequately examines the implementation characteristics of ICT-based interventions in real-world settings, encompassing both the obstacles and factors that aid their implementation [23,24]. Ultimately, to support well-informed decision-making regarding the development of DSPSs, it is vital to establish whether ICT-based suicide prevention strategies are effectively reaching various target audiences or whether they are reaching the same individuals through different means. Therefore, research is necessary to explore digital platforms that incorporate a variety of ICT-based interventions, evaluate the presence of diverse user profiles on these platforms, and analyze the specific interventions they use.

Background

In 2017, the Quebec government asked the Association québécoise de prévention du suicide (AQPS) to set up a DSPS. This DSPS was intended to inform the population about suicide and mental health resources, identify people at risk for suicide on the internet, and offer web-based intervention to people who rarely use traditional help services.

The AQPS then launched a broad consultation process with Quebec suicide prevention stakeholders [7]. The objective of this consultation was to paint a picture of the current practices and needs regarding the use of digital technologies in suicide prevention in Quebec. In general, the consultation participants stressed the importance of developing new digital services by building on the programs, services, and intervention practices already in place in Quebec and of using a centralized website that would make it possible to meet several objectives (inform, identify, and intervene).

In this context, the AQPS, in partnership with its technology partner (the Montreal-based technology firm Nventive), developed the suicide.ca digital platform [25]. Suicide.ca is the official digital platform of the Quebec’s DSPS. Through this platform, a multitude of universal, selective, and indicated suicide prevention strategies are deployed.

Objectives

This study illustrates, through the example of suicide.ca, how a single digital platform can be used to reach the general population, identify people with risk factors for suicide on the internet, and provide direct support to people dealing with suicidal issues. Specifically, this study aims to document the use of the suicide.ca platform; describe the profile of users of the suicide.ca platform; and estimate the effectiveness of universal, selective, and indicated prevention strategies deployed via the suicide.ca platform.

Methods

Suicide.ca Platform

This section describes the universal, selective, and indicated suicide prevention strategies implemented through the suicide.ca platform.

Suicide.ca’s Universal Prevention Strategies

The components of the suicide.ca platform that enable the deployment of universal prevention strategies are its search engine optimization (SEO), the psychoeducation pages, the blog pages, and the promotion of the platform on social media.

SEO Strategy

To fulfill its mission, a digital suicide prevention platform must be known to the public and easily found on the internet. This is why positioning itself favorably in search engine results is at the heart of the suicide.ca platform’s promotional strategy. To do so, an SEO strategy was developed. SEO refers to all the strategies implemented to improve the natural referencing of a website on search engines. For example, all the content on the platform was carefully crafted with the explicit objective of prioritizing search engine ranking.

Psychoeducation Pages

When the user arrives at the platform’s home page, they are prompted to choose one of the following sections: I am thinking about suicide, I’m worried about someone with suicidal thoughts, or I’m grieving a loss by suicide. The section for suicidal people is presented as a guide to getting better. The content of the psychoeducation pages can be found on the platform [26].

Blog Pages

The suicide.ca platform includes a blog. The blog facilitates the regular generation of fresh content, thereby aiding the enhancement of the platform’s natural search engine rankings. It also provides a great deal of flexibility in terms of the topics covered. Content is sometimes developed based on frequent searches by suicide.ca users on search engines.

Promotion of the Platform on Social Media

The AQPS promotes the platform via posts on social media. These posts are generally aimed at promoting the intervention service and communicating information found on the platform. Content from the psychoeducation pages and blog pages is regularly used to create new posts.

Suicide.ca’s Selective Prevention Strategies

The components of the suicide.ca platform that enable the deployment of selective prevention strategies are the fact sheets on suicide risk factors, AdWords campaigns, the self-assessment module, the self-management tools, and the mobile app.

Fact Sheets on Risk Factors Associated With Suicide

The suicide.ca platform includes a section with numerous fact sheets on the risk factors associated with suicide. These fact sheets address mental health problems (eg, depression, anxiety, and schizophrenia) and difficult life situations (eg, relationship problems, money problems, and isolation). These fact sheets have a dual function: to help users cope with their situation and to identify the internet users who present risk factors for suicide on the web.

AdWords Campaigns

AdWords is a paid referencing system developed by Google. This allows organizations to promote their services using paid advertisements. These advertisements appear when internet users perform searches with keywords or expressions that are associated with the services offered. The AQPS frequently uses AdWords campaigns to identify internet users who are conducting research related to suicide, depression, or anxiety.

Self-Assessment Module

A self-assessment module consisting of French-validated questionnaires (refer to the details in the Data Collection Methods and Tools section) is available to users on the Taking Stock of Your Mental Health page.

The self-assessment is performed in 2 stages. During the first stage, users are invited to answer questionnaires allowing them to take stock of their suicidal thoughts, to identify the presence of signs of psychological distress, and to evaluate their positive mental health. If the result of this first evaluation reveals that the user is experiencing suicidal ideation, they are invited to contact the platform’s intervention service via a redirection button. If the user is not referred to the intervention service and the results indicate that they may have a mental health disorder (Kessler Psychological Distress Scale [K10] score [27]), additional questionnaires are offered to identify the presence of symptoms related to various mental health disorders.

Self-Management Tools and Mobile App

The platform contains a section called My Tools, which includes a safety plan and various mental health self-management tools. To make the various self-management tools accessible to people who do not have a smartphone, they have been developed in the web-mobile format. Therefore, users can access their information via the web or the platform’s mobile app (eg, like on Facebook or Twitter). Consequently, the user must open an account to access the tools. However, a user account is not required to access the intervention service and the self-assessment module.

Suicide.ca’s Indicated Prevention Strategies

The components of the suicide.ca platform that enable the deployment of indicated prevention strategies are the chat and text intervention service, the triage system, the suicide risk assessment tools, and the follow-up emails.

Intervention Service by Chat and Text Messaging

Suicide.ca is a digital intervention service that operates 24 hours a day, 7 days a week. It is operated by the AQPS and 3 suicide prevention centers. The service can be accessed via the web, the mobile app, or by SMS text messaging.

Triage

The intervention service triage system assigns a priority rating to contact requests. The first questions ask about age, gender, and reason for contact. Individuals at the risk of suicide are then asked, Are you planning to attempt suicide in the coming minutes or hours? If the user answers yes, the triage ends, and they are sent to the queue with a priority rating. If the user answers no or unsure, they are prompted to complete the Suicidal Ideation Attributes Scale—French version (SIDAS-FR [28]). Finally, the user may, if they wish, specify in a few words the reason for contacting the service.

Follow-Up Emails

At the end of the exchange, the counselors can ask users if they would like to receive a follow-up email. These follow-up emails are generally used when the user anticipates a difficult event that could lead to a suicidal crisis (eg, court appearance and meeting with an ex-spouse).

Suicide.ca Prevention Model

Using the classification proposed by Rassy et al [1], Textbox 1 illustrates how the different components of the platform are complementary and how they deploy universal, selective, and indicated prevention strategies.

Study Design

The evaluation of the suicide.ca platform used a cross-sectional design to document the use of the suicide.ca platform; to describe the profile of users; and to estimate the effectiveness of universal, selective, and indicated prevention strategies deployed through the platform.

Population

The population under study in the evaluation of the suicide.ca platform were the internet users who visited the suicide.ca platform.

Recruitment

This study was based on the analysis of secondary data collected by AQPS. Data were collected from internet users from Quebec, Canada, who visited the suicide.ca platform between October 2020 and October 2021.

Data Collection Methods and Tools

Overview

The evaluation of the suicide.ca platform was carefully considered during its development. The questionnaires integrated into the platform’s various components made it possible to measure the platform’s impact. The platform collects various types of data.

Google Analytics

Google Analytics is a statistical tool offered by Google. It allows website administrators to analyze the behavior of their users. The analytics used in this study are the number of times different pages of the platform were visited, the number of visitors (notwithstanding the number of visits), the sources of traffic acquisition on the platform (eg, search engines, social networks, and hyperlinks on other websites), and the landing pages (the first pages visited by users).

Firebase Console

The Firebase Console platform is a mobile app design tool supported by Google. It was used to count the number of times the My Tools app was downloaded and the number of user accounts created.

Self-Assessment Module Questionnaires

The self-assessment module consists of a series of validated questionnaires in French (Table 1). The results of the self-assessments are recorded in the platform’s Customer Relationship Management (CRM).

Triage Questionnaire

The triage questionnaire includes inquiries about age, gender, reason for contact, whether the individual is planning to attempt suicide in the near future, and the SIDAS-FR. The results of the triage questionnaire are recorded in the platform’s CRM.

Intervention Reports

In this study, the data used from the intervention reports were the reasons for contact, the needs and issues discussed with the users, and the users’ suicide risk ratings assigned by the counselors (suicide risk ratings: no foreseeable risk for a suicide attempt in the near future, low foreseeable risk for a suicide attempt in the near future, high foreseeable risk for a suicide attempt in the near future, and imminent danger of suicide attempt or attempt in progress). The categories of needs and issues discussed are not mutually exclusive. The intervention report also contains a section where the counselor is asked to indicate whether they think the user feels better and is more able to cope with their difficulties as a result of the intervention (response choice=yes, unsure, and no). These intervention reports are recorded in the platform’s CRM.

Postintervention Questionnaire

This questionnaire includes 9 items in the form of 4-point Likert scales (1=strongly agree to 4=strongly disagree). The items focus on users’ perceptions of the quality of interventions and their impact on distress, feeling of being able to cope with the situation, and suicidal ideation. The items are illustrated in detail in the Results section.

Analysis Methods

For each objective, several data sources were consulted, and different types of analyses were performed.

Objective 1: To Document the Use of the Suicide.ca Platform

To document platform use, Google Analytics data from the suicide.ca website [26] were analyzed to describe platform traffic and the number of visits to the psychoeducation pages. The Firebase Console data from the My Tools mobile app were analyzed to document the number of downloads of the mobile app and the number of user accounts created. CRM data were analyzed to describe the number of people who completed the self-assessment, the number of people who accessed the triage of the intervention service, and the number of interventions provided.

Objective 2: To Describe the Profile of Users of the Suicide.ca Platform

To describe the profile of platform users, frequency analyses were conducted using data from the self-assessment questionnaires, the triage questionnaire, and the reasons for contact recorded in the intervention reports.

Objective 3: To Describe the Impact of Universal, Selective, and Indicated Prevention Strategies Deployed Through the Suicide.ca Platform

The impact of the platform’s promotional activities on social media was investigated through an analysis of the platform’s traffic peaks. First, a visual inspection of the traffic curve was performed using Google Analytics. It was then decided to define the traffic peaks as days with >600 visits. This threshold corresponded to twice the average traffic and days when the increase in traffic was evident in the Google Analytics graph. AQPS internal documentation was then inspected to determine if these dates corresponded to dates when AQPS conducted promotional activities on social media or traditional media (eg, web-based publications, radio, or television interviews). The data analyzed to assess the impact of AQPS promotional activities cover the period from October 15, 2020, to October 17, 2021.

With respect to the AdWords campaigns and the suicide risk factor fact sheets, 2 analytical strategies were used to estimate their effectiveness. The first strategy was to use Google Analytics to describe the traffic acquisition sources of suicide.ca. In digital marketing, traffic acquisition sources identify how visitors to a website accessed a site (eg, via search engine queries, social media posts, AdWord advertisements [43]). Thus, traffic acquisition sources were used to describe the number and proportion of users who arrived at the site through an AdWords campaign. The second strategy was to use Google Analytics to identify the top landing pages on the platform. A landing page is the first page a user views when they arrive on the platform (eg, risk factor information page, psychoeducation page, and resource information page). In digital marketing, landing pages are used to better understand what users were looking for on the internet before they arrived at a site or what caught their attention [43]. For example, landing pages were used to describe the number and proportion of users who began their site visit by viewing a suicide risk factor fact sheet. The conversion rate for the AdWords campaigns was calculated based on the number of individuals who clicked on the advertisement and initiated the crisis chat triage. The analytical data on AdWords campaigns and fact sheets cover the period from December 31, 2020, to October 31, 2021.

The impact of the intervention service was assessed via an analysis of counselors’ Perceptions of the impact of their interventions (recorded in intervention reports) and the postintervention survey results. Approximately 3473 interventions were provided between the postintervention survey going live in late January 2021 and October 2021. Therefore, we estimated the response rate to the postintervention survey to be approximately 10%. As the SMS text messaging intervention service was in development at the time of data collection, only the data for the chat-based intervention are presented.

Ethical Considerations

This study is based on the secondary analysis of anonymous data collected by AQPS. Consent for primary data collection was obtained through a waiver form in suicide.ca’s self-assessment module and a consent form in the postintervention questionnaire of the intervention service. These forms specify that data collection is for the purpose of continuous service improvement and that any data shared with third parties will be anonymized and subjected to ethics approval by an institutional research ethics committee. The participants were not offered any compensation for their involvement in the study. Ethics approval for the secondary analysis of all data presented in this study was obtained from the institutional review board of the Centre intégré universitaire de santé et de services sociaux de l’Estrie—Centre hospitalier universitaire de Sherbrooke (project 2021-4016).

Results

In this section, the results concerning platform use; the profile of platform users; and the effectiveness of universal, selective, and indicated prevention strategies deployed through the suicide.ca platform are presented.

Use of the Suicide.ca Platform

Overview

A total of 127,495 visits were made to the suicide.ca platform from October 15, 2020, to October 17, 2021. The average number of visits per day was 346.45 (median 313.5, IQR 257-389). The least busy day had 123 visits, and the busiest day had 1774 visits.

Visits to Psychoeducation Pages

The number of visits to the psychoeducation pages of the 3 main sections of suicide.ca were much lower than the total number of visits to the site. Moreover, the number of page views dropped significantly after the title pages of all 3 sections were visited (Table 2).

Number of People Who Completed the Self-Assessment Module

With regard to self-assessment, 2488 users completed the SIDAS-FR. Moreover, 3100 users completed the K10 questionnaire, and 2644 users completed the Mental Health Continuum Short Form questionnaire. Overall, 1471 users completed phase 2 of the self-assessment module.

Number of Downloads of the Mobile App Versus Number of User Accounts Created

The analysis of downloads of the mobile app and the creation of user accounts was carried out for the period from November 23, 2020, to October 17, 2021. During this period, there were 5159 downloads of the mobile app, and 2178 user accounts were created.

Number of Crisis Interventions Provided and Triage Dropout Rate

From October 15, 2020, to October 17, 2021, suicide.ca’s intervention service received 6152 intervention requests. Of these 6152 requests, 1540 (25.02%) did not result in an intervention. The intervention reports provided reasons for not being able to intervene in 1321 of these cases. The most common explanation was the lack of response from the users at the beginning of the contact (840/1321, 63.59%).

Analysis of the intervention service triage data was conducted for the period October 15, 2020, through July 25, 2021 (Figure 1). As the gender question was optional, it was not included in the analysis. Of the 6300 users who responded to the first question, only 2711 (42.86%) completed the triage. The last question, which asked for the user’s first name or alias, resulted in a significant dropout rate. Specifically, only 46.54% (2711/5825) of the users who were presented with the question actually provided a response. As for the questions about suicidality, they resulted in few dropouts. Age, reason for contact, and plan to attempt suicide in the coming minutes or hours were not associated with the probability of abandonment during triage.

‎

Profile of Users of the Suicide.ca Platform

To describe the profile of platform users, frequency analyses were conducted using data from the self-assessment questionnaires, the triage questionnaire, and the reasons for contact recorded in the intervention reports.

Characteristics of Individuals Using the Self-Assessment Module

Table 3 indicates the results of the users who undertaken a self-assessment. With regard to suicidality, the average score of the 2488 users who completed the SIDAS-FR questionnaire was 13.47 (SD 14.04; median 10). Among them, 868 (34.89%) reported no suicidal ideation in the last 30 days, whereas 825 (33.16%) reported severe suicidal ideation. In terms of psychological distress, of the 3100 users who completed the K10 questionnaire, 2721 (87.78%) obtained a score indicating the presence of a mental health disorder. With regard to positive mental health, of the 2644 users who completed the Mental Health Continuum Short Form questionnaire, 1024 (38.73%) exhibited languishing mental health.

In the 1471 users who completed phase 2 of the self-assessment, depressive and anxiety disorders were the most prevalent. Specifically, 62.2% (915/1471) of the users obtained a Patient Health Questionnaire-9 score indicating the presence of depression, 61.73% (908/1471) of the users obtained a Penn State Worry Questionnaire score indicating the presence of generalized anxiety disorder, and 53.2% (451/847) of the users obtained a Spin Phobia Inventory score indicating the presence of social phobia disorder.

Characteristics of Individuals Using the Intervention Service

Table 4 presents information on the age and gender of intervention service users. Information on user age was available for 4000 interventions. Most users were aged between 18 and 40 years (2829/4000, 70.73%). Information on the gender identity of users was available for 3569 interventions. Most users identified as woman (2555/3569, 71.59%). Users who identified as transgender, nonbinary, or listed other were generally young people aged ≤25 years (235/280, 83.9%).

Information on the reason for contact was available for 4000 interventions (Table 5): 84.48% (3379/4000) of the users indicated I am thinking about suicide in the triage questionnaire and 10.1% (404/4000) of the users indicated I am worried about a loved one, and 5.43% (217/4000) indicated I am bereaved by suicide.

With regard to the suicidal risk of the users of the intervention service, 3490 answered the question (Table 6), Are you planning to attempt suicide in the coming minutes or hours? Of the 3490 users, 1800 (51.58%) answered no, 972 (27.79%) answered unsure, and 718 (20.57%) answered yes. A total of 2606 users completed the SIDAS-FR at triage. The mean score was 26.69 (SD 12.79; median 29). Overall, 75.56% (1969/2606) of the users had suicidal ideation of high severity.

Information on the needs expressed by the users was available for 3254 interventions (Table 6). Overall, 75.29% (2450/3254) of the users expressed a need to ventilate. The most frequently discussed issues were suicide (3033/3475, 87.28%), mental health problems (765/3475, 22.01%), emotional problems (657/3475, 18.91%), family problems (455/3475, 13.09%), self-harm (413/3475, 11.88%), loneliness (407/3475, 11.71%), and problems with peers or relatives (361/3475, 10.39%).

Information on the counselors’ suicide risk assessments was available for 1783 interventions (Table 6). The most common assessments were “Low foreseeable risk for a suicide attempt in the near future” (955/1783, 53.56%) and “High foreseeable risk for a suicide attempt in the near future” (602/1783, 33.76%).

Effectiveness of Universal, Selective, and Indicated Prevention Strategies Deployed Through the Suicide.ca Platform

Impact of Promotional Activities on the Platform’s Traffic

Analysis of peak traffic (days with >600 visits) for the period from October 15, 2020, to October 17, 2021, indicated that they were primarily associated with promotion of the platform in traditional media and on social media (Figure 2; Table 7). Traffic tended to remain high for 1 or 2 days following these spikes and then returned to its average trend afterward.

‎

Impact of AdWords Campaigns and Suicide Risk Factors Fact Sheets on the Platform’s Traffic

Table 8 shows the source of traffic and the top landing pages on the platform for the period from December 31, 2020, to October 31, 2021. The source of traffic to the platform was very diverse. AdWords campaigns on suicide risk factors were the second-largest source of traffic acquisition on the platform.

The AdWords campaign targeting anxiety and depression was displayed 72,713 times and received 9576 clicks, resulting in a click rate of 13.7%. This campaign also led to 36 crisis chat interventions, representing a conversion rate of 0.38%. Similarly, the AdWords campaign with suicide-specific keywords was displayed 27,961 times and received 2647 clicks, resulting in a click rate of 9.47%. This campaign resulted in 55 crisis chat interventions, with a conversion rate of 1.93%.

As for the first pages consulted by users (Table 8), 40% (4704/11,760) of the visits from search engine queries, excluding AdWords ads, began with a visit to a suicide risk factor fact sheet. Therefore, suicide risk factor fact sheets were an important source of traffic to the platform.

Effectiveness of the Intervention Service

Table 9 presents the counselors’ perception of the impact of their interventions. Counselors indicated that the users felt better after the intervention in 56.09% (2565/4573) of the interventions. They also reported that the users felt more able to overcome their difficulties in 51.08% (2336/4573) of the interventions.

Table 10 shows the results of the postintervention survey for 259 suicidal users and 91 users worried for a loved one or bereaved by suicide. Overall, 91.5% (237/259) of the suicidal users indicated that the counselor seemed warm and caring, and 89.5% (232/259) of the users indicated that they felt listened to, understood, and respected. In terms of the effectiveness of the interventions, 82.6% (214/259) of the suicidal users indicated that they felt better, calmer, or more relaxed following the intervention; 76.8% (199/259) of the users indicated that they felt more in control of their suicidal thoughts; and 72.5% (188/259) of the users indicated that they felt more able to overcome their difficulties.

Discussion

Principal Findings

The objectives of this study were to document the use of the suicide.ca platform; to describe the profile of its users; and to estimate the impact of the universal, selective, and indicated prevention strategies deployed via the platform.

The results for universal prevention strategies indicated that there was a wide range of sources for acquiring traffic to the platform, that actions to promote the platform on social media were directly associated with increased traffic, and that there was little traffic to psychoeducation pages.

The results for selective prevention strategies indicated that AdWords campaigns and fact sheets on suicide risk factors are important sources of traffic acquisition on the platform, that the platform reaches people with varying degrees of suicidal risk, and that the user account was a barrier to the use of mental health self-management tools.

The results for the indicated prevention strategies indicated that triage questions involving personal information result in a significant number of dropouts, that counselors feel that their interventions have a positive impact on the user in half of the contacts, and that intervention service users are generally satisfied with the service they received.

Universal Prevention Strategies

The sources of traffic acquisition on suicide.ca were very diversified, and only 40% (50,271/125,677) of the users started their visit on the home page. Moreover, the platform’s social media promotion was directly associated with traffic peaks. These results highlight the importance of being proactive in promoting the platform and using a wide variety of promotional strategies (eg, SEO strategy, social media posts, AdWords campaigns, and referral links on other websites). They also highlighted the importance of collaborating with digital experts to adapt traditional digital marketing strategies to suicide prevention.

The number of visits to the psychoeducation pages was far less than the total number of visits to the site. For example, the first page of the psychoeducation section for people with suicidal risk was visited only 5312 times in the first year. This number is far less than the 127,495 visits that were recorded on the site for this period. This result suggests that the vast majority of users do little exploration of the platform after viewing the content that brings them to the site. These results highlight the importance of looking beyond the total number of visits to a digital suicide prevention platform to describe its use and of dissecting its analytics data in detail to fully understand the content that is consulted.

Selective Prevention Strategies

AdWords campaigns and suicide risk factor fact sheets were important sources of traffic acquisition on the platform. For example, 40% (4704/11,760) of the visits from search engines, excluding AdWords advertisements, began with a suicide risk factor fact sheet. Although it is challenging to determine from the analytical data whether the individuals who accessed the fact sheets or clicked on the advertisements were at risk for suicide, it is evident that they conducted a Google search related to suicide or suicide risk factors and landed on the suicide.ca platform. Furthermore, the AdWords campaigns targeting anxiety and depression achieved click rates of 13.7% and 9.47%, respectively, which are significantly higher than the average Google Ads benchmarks of 3.17% observed in e-marketing [44] and the click rates of 4% and 5% reported in the study by Onie et al [13]. However, despite the high click rates, the conversion rates for crisis chat interventions were relatively low, with 0.38% for the anxiety and depression campaign and 1.93% for the suicide-specific campaign. It is challenging to interpret why these clicks did not result in more crisis interventions. Although the average conversion rate in e-marketing is 3.75% [44], there are currently no established benchmarks for conversion rates in suicide prevention. Although our findings should be interpreted with caution, they offer initial support for suicide.ca’s strategy of using AdWords campaigns and fact sheets as tools to identify internet users presenting suicide risk factors through search engine platforms.

Previous research has predominantly focused on how Google can enhance support for suicide prevention, paying limited attention to how suicide prevention actors can leverage Google to identify individuals at risk (eg, [12,13,45,46]). More research is required to discover optimal methods for using Google search to identify internet users at risk of suicide and to guide them toward support resources.

A major finding of this study is that the platform reaches users with different degrees of suicidal risk. The proportion of users with severe suicidal ideation on the SIDAS-FR was 33.16% (825/2488) for the self-assessment module and 75.56% (1969/2606) for the intervention service. Although the self-assessment module users presented less severe suicidal ideations than the intervention service users, 87.77% (2721/3100) of them obtained a K10 score, indicating the probable presence of a mental health problem. Furthermore, 62.2% (915/1471) of the self-assessment module users obtained a Patient Health Questionnaire-9 score indicating the presence of depression, 61.73% (908/1471) obtained a Penn State Worry Questionnaire score indicating the presence of generalized anxiety disorder, and 53.2% (451/847) obtained a Spin Phobia Inventory score indicating the presence of social phobia disorder.

Our best point of comparison is the 113Online chat service in the Netherlands, which also provides users with access to a self-assessment module and crisis interventions via chat. According to the evaluation of their first 3 years of operation [3], 78.36% (4741/6050) of the users who used the self-assessment module scored positively for depression and anxiety on the Depression and Anxiety Stress Scale. The module also included the Beck Scale for Suicidal Ideation, with results indicating that 7.22% (533/7381) of the users were not suicidal, 1.75% (129/7381) were mildly suicidal, 39.29% (2900/7381) were suicidal, and 51.74% (3819/7381) were severely suicidal. When comparing the results of this study with the observations on the 113Online platform, it appears that the suicide.ca self-assessment module is being used by individuals with lower levels of suicidality. These differences may, in part, be explained by the fact that the 2 platforms used different questionnaires to measure suicidality, depression, and anxiety. In addition, it may be explained in part by the identification strategy of the suicide.ca platform, which aims to reach individuals who have risk factors for suicide but may not be actively suicidal.

With regard to the self-management tools, between 22.99% (1186/5159) and 58% (2992/5159) of the users who downloaded the mobile app did not create a user account. It should be noted that a user can download the app to use the intervention service only and that doing so does not require the creation of an account. Technical problems experienced by users when creating their account could also contribute to this discrepancy [47]. Nevertheless, the discrepancy between the number of downloads and the number of accounts created is so large that it seems to indicate that setting up a user account is a practice that should be avoided as it seems to represent the first barrier to use. This finding highlights an important dilemma for digital suicide prevention interventions. Several studies indicate that users of digital interventions have concerns about the security and privacy of their data [48]. However, although the presence of a user account enhances data security and privacy, it is a barrier to use for some users [49]. These findings highlight the importance of identifying technologies that ensure the security of users’ data while preserving their sense of anonymity.

Indicated Prevention Strategies

To our knowledge, this study is the first to report on the impacts of triage on the accessibility of a web-based intervention service. Our results indicate that the last question (What is your first name or alias?) was associated with a significant number of dropouts; only 46.54% (2711/5825) of the users who were asked the question answered it. Studies on crisis intervention through chat and text indicate that users prefer to use these services over phone helplines because of the sense of anonymity provided by these mediums [3,50,51]. It is possible that even asking for personal information or requesting an alias could discourage users, emphasizing the importance of maintaining their sense of anonymity.

With regard to the questions about suicidality, it is surprising to observe the low dropout rates they have caused. The requirement to complete the 5 questions of the SIDAS-FR during the triage resulted in a dropout rate of 4%. Web-based crisis service providers are sometimes concerned about implementing a triage. This study provides an estimate of the dropout rate that triage can lead to.

With regard to the profile of the intervention service users, most users (2019/3569, 56.57%) were women aged between 18 and 40 years. The most frequently discussed issues were mental health, emotional problems, relationship problems, self-harm, and isolation. These findings are consistent with those typically observed in digital intervention services [14-17].

It is also interesting to note that 7.85% (280/3569) of the users were from gender minority groups and 83.9% (235/280) of them were young people aged ≤25 years. This percentage is similar to that observed in the Canada Suicide Prevention Service (7.5%) and Lifeline Crisis Chat (7.5%) [14,15]. The age of these users likely reflects the high level of adversity experienced at this time for many gender minority youth [52]. These findings also highlight the relevance of digital intervention services for these youth.

This study provides valuable insights into the suicidality of users of chat intervention services, as multiple sources of information were used to evaluate the risk of suicide. These include data derived from a standardized suicide questionnaire (SIDAS-FR), risk assessments conducted by counselors, and a specific question regarding the risk of suicide attempts in the near future. Approximately half of the users (1690/3490, 48.42%) answered yes or uncertain to the question, Are you planning to attempt suicide in the coming minutes or hours? Moreover, 75.56% (1969/2606) of the users who answered no or uncertain to the question obtained a SIDAS-FR score indicating severe suicidal ideations. Therefore, the intervention service reaches its target population, that is, people at a high risk for suicide. These findings are consistent with those observed in the 113Online crisis chat evaluation [17], where a content analysis of intervention transcripts revealed that 61.1% (320/524) of the users expressed suicidal intentions, 21.2% (111/524) of the users reported both suicidal intentions and a suicide plan, and 3.8% (20/524) of the users were actively attempting suicide at the time of contact.

In terms of the impact of the intervention service, counselors reported that the user felt better after the intervention in 56.09% (2565/4573) of contacts and reported that the user did not feel better in only 9.81% (449/4573) of contacts. The response rate to the postintervention questionnaire was low. Nonetheless, the scores on the various satisfaction indicators were all very high, and 3 (199/259, 76.8%) out of 4 respondents felt that their suicidal ideation had decreased following the intervention. Again, these results are consistent with those typically observed in digital intervention services [15,16,18,53].

Limitations of the Study

This study had several limitations. First, the results on site traffic, traffic acquisition sources, page views, and app downloads were based on an analysis of the platform’s analytics data. However, analytical data do not allow us to understand the needs and motivations behind users’ browsing behavior on the platform. It is therefore important to complement this type of quantitative data in digital platform evaluations with other observational methods, such as qualitative interviews with users, or by directly observing their browsing and help-seeking behavior on the internet [22].

Similarly, analytical data are often inaccurate. For example, the traffic on the platform was analyzed according to the number of sessions. However, a session is the period during which a user was active on the site. If a user is inactive for ≥30 minutes, Google Analytics will attribute any subsequent activity to a new session. Therefore, the number of sessions is not exactly the same as the number of visits, but it is the closest indicator to it.

With respect to the analysis of app downloads, it was difficult to estimate the number of people who downloaded the app but did not create a user account. Therefore, it was impossible to provide an accurate estimate for this important issue. Moreover, it would be important to understand the needs and motivations that explain why so many people downloaded the app but did not open a user account. We explain this phenomenon by the users’ need for anonymity, but it would be important to validate this hypothesis with empirical evidence.

The results on the needs expressed by the intervention service users and the topics discussed during the contacts were based on the analysis of the intervention reports. The completeness of these reports may vary from one counselor to another, which may explain in part why the numbers on issues discussed were low. Furthermore, data on user-identified needs reflect the perceptions of counselors. Therefore, there is most likely a gap between these perceptions and the actual needs of the users. Therefore, it would be important to conduct qualitative interviews with users to understand their needs and motivations for using the intervention service.

An important limitation of this study was the absence of questions on gender and age in the self-assessment module. Therefore, we were unable to compare users of the self-assessment with users of the intervention service based on these characteristics. This information is crucial to understanding who is and is not being reached by suicide.ca and, consequently, by the Quebec DSPS. This highlights the importance of anticipating how the data collected by the various tools offered on a platform will compare with their users.

Conclusions

The results of this study indicate that it is possible to use the same platform to conduct universal, selective, and indicated suicide prevention. They also highlighted the importance of being strategic in the development of a digital platform to anticipate how different target populations will be identified, how different intervention modalities will be offered to them, and how the data collected by the platform will allow for the measurement of its impact.

Finally, the evaluation of suicide.ca produced several important results and was the source of a considerable amount of learning. We hope that sharing these learnings will help guide organizations interested in developing a digital suicide prevention platform and contribute to the development of guidance for the development of DSPSs (eg, the WHO [54]).