Background: Web-based interventions are at an early stage in non–English-speaking low- and middle-income countries, where they remain scarce. Help for Depression (HDep) is one of the few unguided web-based interventions available in Latin America. The results of a use/usability analysis of the original version served as the basis for generating a more user-friendly second version.
Objective: The aim of this study is to explore participants’ satisfaction and acceptability for the second version of HDep.
Methods: A retrospective cross-sectional design was used. An email invitation to complete a web-based survey was sent to all people who accessed HDep in 2018. The questionnaire included satisfaction and acceptability scales and open-ended questions. Complete questionnaires were retrieved from 191 participants: 35.1% (67/191) from those who visited only the home page (home page users [HPUs]) and 6.47% (124/1916) from those who registered to use the program (program users [PUs]).
Results: In all groups, users experienced high levels of depressive symptoms (189/191, 98.9%; Center for Epidemiological Studies Scale-Depression >16). Moderate levels of satisfaction (HPUs: mean 21.9, SD 6.7; PUs: mean 21.1, SD 5.8; range: 8-32) and acceptability (HPUs: mean 13.8, SD 3.9; PUs: mean 13.9, SD 3.2; range: 5-20) were found in both groups. Logistic regression analyses showed that among HPUs, women were more satisfied with HDep (odds ratio [OR] 3.4, 95% CI 1.1-10.0), whereas among PUs, older respondents (OR 1.04, 95% CI 1.01-1.08), those with paid work (OR 3.1, 95% CI 2.4-7.6), those who had not been in therapy (OR 2.42, 95% CI 1.09-5.98), and those who had not attempted suicide (OR 3.4, 95% CI 1.1-11.1) showed higher satisfaction. None of the sociodemographic/mental health variables distinguished the acceptability ratings among HPUs. Among PUs, those with paid work (OR 2.5, 95% CI 1.1-5.5), those who had not been in therapy (OR 3.1, 95% CI 1.3-7.3), those without disability (OR 2.9, 95% CI 1.3-6.6), and those who had not attempted suicide (OR 2.6, 95% CI 1.0-6.6) showed higher acceptability.
Conclusions: HDep has good levels of satisfaction and acceptability for approximately half of its users, and the information provided by respondents suggested feasible ways to remedy some of the deficiencies. This qualitative–quantitative study from a low- to middle-income, non–English-speaking country adds to existing knowledge regarding acceptance and satisfaction with web-based interventions for depression in resource-limited countries. This information is important for the creation and adaptation of web-based interventions in low- and middle-income countries, where access to treatment is a major concern, and web-based prevention and treatment programs can help deliver evidence-based alternatives. It is necessary to document the pitfalls, strengths, and challenges of such interventions in this context. Understanding how users perceive an intervention might suggest modifications to increase adherence.
The World Health Organization  has ranked depression as the single largest contributor to global disability, accounting for 7.5% of all years lived with disability. More than 80% of this nonfatal disease burden occurs in low- and middle-income countries (LMICs). In Mexico, depressive disorders account for 8.6% of the years lived with disability [ ]. Prevention and treatment interventions can reduce the burden associated with depression and other mental health disorders. However, mental disorders remain untreated in many nations; in Latin America, only 5% of people with affective disorders receive adequate treatment [ ]. In Mexico, only 6.4% of those diagnosed with major depressive disorder receive minimally adequate treatment [ ].
Internet- or web-based interventions represent effective, accessible, and low-cost means to treat and prevent depressive disorders, and they can be broadly disseminated [, ]. Unguided interventions, based purely on self-help with no human support, can reach large numbers of people at low cost, and there is evidence that even those who experience enough symptoms to screen positive for a major depressive episode use preventive interventions [ ]. Meta-analytic studies show that these interventions have benefits but exhibit low adherence rates [ ]. The average rate of adherence in unguided interventions is estimated at 26%, compared with 72% in guided interventions [ ]. Despite the high attrition rates and lesser effectiveness of unguided programs, these low-cost, low-intensity, web-based interventions are suitable from a public health perspective as early intervention in a stepped-care process [ ] and are thus of particular benefit in LMICs [ ].
Web-based interventions for common mental health problems, such as depression, have a long history and have evolved rapidly in high-income countries, but they are at an earlier stage of development and are less common in LMICs [- ]. Web-based interventions to prevent depression are associated with small but positive effects on the symptoms [ ]. Help for Depression (HDep; Ayuda para Depresión) [ ], in Mexico, was the first such web-based intervention in Latin America [ ].
The initial version of HDep (2009-2013) was modeled after a face-to-face psychoeducational intervention in Mexico to prevent depression in women, based on multimodal and cognitive behavioral principles () [ ]. This initial intervention was modeled after that of Muñoz and Ying [ ] in California, designed for the ethnic minority groups, including the Latino population. The content of the program that we developed went through a step-by-step process including focus groups and open-ended questionnaires to verify that language, illustrations, and content were sensitive to the target population [ ].
|1. What is depression?|
|2. Identify negative thoughts|
|3. How to transform negative thoughts|
|4. Childhood experiences|
|5. Adverse events|
|6. Other strategies to improve mood|
A use/usability analysis of the first version of HDep showed that 15.6% of the users were men . Two psychologists with experience working with men reviewed the language and vignettes for cultural sensitivity to male users. The analysis of that version also suggested the need for a shorter and more user-friendly version [ ], which was produced by reducing the number of modules. In the new version, users need to provide minimal information (sex, age, and email) to access home page content. The home page includes (1) a self-assessment scale for depressive symptoms (Center for Epidemiological Studies Scale-Depression [CES-D]), with feedback provided via email; (2) extensive information on depression and places to receive help; (3) a description of the aims and content of the intervention; and (4) a link to register to use the program modules. This arrangement allows for two types of users: home page users (HPUs), who are those who prefer to visit only the home page or landing page and program users (PUs), who decide to register for the program modules after exploring the home page ( ). PUs are asked to provide additional information (marital status, education, psychological or psychiatric treatment for depression, suicide attempts, alcohol and drug use, and medication for mental health problems). HPUs and PUs who answer the CES-D and other health measures are given feedback on their responses; those who may be at risk for clinical depression or mental health disorders are advised to find additional psychological or psychiatric help. HDep is freely accessible, with no inclusion criteria for registration; it is thus available not only for people who meet a clinical diagnosis of depression but also to those with other mental health problems, such as substance abuse [ ].
The lack of adequate mental health services for most of the population of Mexico and other Latin American countries means that very few people receive even minimally adequate treatment for depression . In this context, internet-based interventions play an important role in the prevention and treatment of depression; however, they are still at an early stage of development in these countries.
Since 2014, the revised version of HDep has had around 2956 visits each month from individuals who answered the depression scale (CES-D) and received feedback based on their scores. At the same time, there is a high rate of attrition, which is consistent with data from a scoping review in Latin America . To improve adherence, it is important to analyze users’ experience, perception, and satisfaction with HDep to find ways to encourage people to complete the intervention [ ].
The aim of this study is to assess the acceptability of and satisfaction with the updated version of HDep, considering both types of users separately: HPUs and PUs. This was an exploratory study, with the underlying hypothesis that PUs would show greater acceptance and satisfaction than HPUs, as they were receiving a higher dose of the intervention.
Study Design and Participants
A retrospective cross-sectional design was used in this study. The population corresponds to people who visited HDep: HPUs and PUs from January to December 2018 (N=13,207) and answered the initial depressive symptoms questionnaire (CES-D), which is accessible on the home page. The sample size was determined according to the procedure of Lemeshow et al  for the following parameters: a finite population with a CI of 95% and a margin of error of 10%. This margin of error was chosen because web surveys have response rates approximately 12% points lower than other survey modes [ ]. The minimum number of responses required was 96. Of the 13,207 emailed questionnaires, 191 (191/13,207, 1.45%) were answered and returned. A total of 208 (208/13,207, 1.57%) were undelivered (indicated by automatic replies). There is no information on whether the rest reached the target users ( ). The CI for the response rate was 95%, with a margin of error of 7%.
Data retrieved from HDep databases: email, sex, age, marital status, occupation, education, depressive symptoms (CES-D), disability during the previous month owing to depressive symptoms, psychological or psychiatric treatment for depression, suicide attempt in the last 6 months, alcohol and drug use, and medication .
Depressive symptoms are assessed with the CES-D , which has 20 items with Likert response options (0=rarely or never to 3=most of the time). The cutoff point indicating the presence of depressive symptoms was ≥16. The CES-D has been validated in Mexico [ ].
Satisfaction With HDep
Satisfaction with HDep is measured with the short version of the Client Satisfaction Questionnaire-8 (CSQ-8)  (Spanish version [ ]), which includes 8 questions on a Likert scale of 1 to 4, with higher scores indicating greater satisfaction. A change was made in the wording of the questions to align it with the purpose of the study: HDep was used instead of service. The CSQ-8 is reliable across a variety of ethnic contexts, including Hispanic groups, and the Spanish version is as reliable as the English version (Cronbach α=.90) [ ]. The Cronbach coefficient for the scale in the sample was a Cronbach α of .95.
Acceptability of HDep
As in many studies [, ], an ad hoc questionnaire was developed to evaluate this construct. It consisted of 17 questions, of which only 15 quantitative questions were analyzed (the other two were open-ended questions that received few responses; thus, they were not included: Is there something additional that helped you that is not listed here? and Are there any additional comments you would like to add?). The first five questions were related to general acceptability: Is the home page persuasive? Is the page layout of the modules inviting? Does HDep seem useful for managing depression? Did HDep meet your expectations? and Is the website user friendly? Respondents were asked to answer the questions on a 4-point Likert scale (1=low to 4=high acceptability). The Cronbach coefficient for these 5 questions in the sample was a Cronbach α of .89. In all, 2 questions rated the HDep content and design on a scale of 1 to 10, with a space to provide open-ended elaboration. The final eight questions scored the following elements of HDep on a scale of 1 to 10: module information, sample cases, activities, forums, chats, thought charts, audio, and depressive symptom assessment and feedback. The Cronbach coefficient for the scale in the sample was a Cronbach α of .95. Except for the depressive symptoms assessment and feedback, these questions were analyzed only for users who registered for the program (PUs), as these elements are part of the program modules.
The link to the web survey was delivered via email (using the Mail Chimp and Google Forms platforms), including a cover letter explaining the objectives of the study and why users were being contacted. A follow-up email was sent 1 month later to those who had not answered. As a token of appreciation, those who responded to the survey were sent a list of 10 positive thoughts to reinforce HDep activities and improve mood.
The survey was designed to be user friendly; it was configured to prevent users from leaving questions unanswered.
The percentages of sociodemographic characteristics and the means and SDs of the overall satisfaction scale and for individual items were obtained for HPUs and PUs and compared using chi-square and 2-tailed t tests. The same procedure was followed to evaluate the elements of acceptability (user friendly, scope, usefulness, motivation to register and carry out the activities, and expectations) of HDep overall and for each component. Logistic regression analyses were conducted to assess the characteristics of users who were satisfied with the HDep program among the HPUs and PUs. For these analyses, the acceptability and satisfaction scales were dichotomized: the cutoff points were defined at the 75th percentile for each scale (for the CSQ-8, it was ≥26 and for the scale of acceptability, ≥16). Finally, a qualitative thematic analysis was conducted on the responses to the 2 open-ended questions regarding design and content.
Demographic and Psychological Characteristics
Of the 13,207 emailed invitations, 191 (1.45%) were completed questionnaires; 67 (67/191, 35.1%) were from HPUs and 124 (124/191, 64.9%) from PUs. Most were women (141/191, 73.8%; HPUs: 42/67, 63%; PUs: 99/124, 79.8%; χ21=6.22; P=.01) and aged 20 years (57/191, 29.8%; HPUs: 24/67, 36%; PUs: 33/124, 26.6%) or aged 21-30 years (68/191, 35.6%; HPUs: 24/67, 36%; PUs: 44/124, 35.5%; χ21=6.22; P=.01).
Nearly all users (189/191, 98.9%) experienced high levels of depressive symptoms (CES-D>16), with no significant difference between PUs (124/124, 100%) and HPUs (65/67, 97%; χ21=3.74; P=.12). Questions on mental health problems were answered only by PUs; 70 (70/124, 56.4%) PUs reported disability associated with depressive symptoms, 20 (20/124, 16.1%) PUs reported psychological or psychiatric treatment for depression, 21 (21/124, 16.9%) PUs reported a previous suicide attempt, 27 (27/124, 21.8%) PUs reported excessive alcohol use, 15 (15/124, 12.1%) PUs reported excessive drug use, and 39 (39/124, 31.5%) PUs had received drug treatment ().
|Characteristics||All (N=191), n (%)||Home page user (n=67), n (%)||Program user (n=124), n (%)||Values|
|Chi-square (df)||P value|
|Male||50 (26.2)||25 (37.3)||25 (20.2)|
|Female||141 (72.8)||42 (62.7)||99 (79.8)|
|Age (years)||3.23 (3)||.35|
|20||57 (29.8)||24 (35.8)||33 (26.6)|
|21-30||68 (35.6)||24 (35.8)||44 (35.5)|
|31-40||34 (17.8)||8 (11.9)||26 (21)|
|41||32 (16.7)||11 (16.4)||21 (16.9)|
|With partner||N/A||N/A||37 (29.8)|
|Junior high school or less||N/A||N/A||10 (5.2)|
|High school or more||N/A||N/A||181 (94.8)|
|Depressive symptoms (CES-Db>16)||189 (98.9)||65 (97)||124 (100)||3.74 (1)||.12|
|Disability the previous month owing to depressive symptoms||N/A||N/A||70 (56.5)||N/A||N/A|
|Psychological or psychiatric treatment for depression||N/A||N/A|
|Yes, currently||N/A||N/A||20 (16.1)|
|Not currently but in the past||N/A||N/A||32 (25.5)|
|Suicide attempt in the last 6 months||N/A||N/A||21 (16.9)||N/A||N/A|
|Alcohol use||N/A||N/A||27 (21.8)||N/A||N/A|
|Drug use||N/A||N/A||15 (12.1)||N/A||N/A|
|Medication (for feeling nervous, anxious, or excessively energetic)||N/A||N/A||39 (31.5)||N/A||N/A|
aN/A: not applicable.
bCES-D: Center for Epidemiological Studies Scale-Depression.
Satisfaction With HDep
The mean satisfaction with HDep was 21.9 (SD 6.7; range 8-32) for HPUs and 21.1 (SD 5.8; range 8-32) for PUs. No significant difference was found in the overall level of satisfaction among types of users (t189=0.845; P=.39;) or on individual items. The highest scores were given to If a friend needed help, would you recommend HDep? and If you were to seek help again, would you come back to HDep? The item with the lowest score was To what extent did HDep meet your needs? ( ). No significant differences were found between HPUs and PUs on individual items.
|Item||Home page user (n=67), mean (SD)||Program user (n=124), mean (SD)||Values|
|t test (df)||P value|
|Scale mean||21.90 (6.74)||21.10 (5.83)||0.84 (189)||.39|
|1. How would you rate the quality of help you have received?||2.87 (0.88)||2.73 (0.79)||1.04 (189)||.29|
|2. Did you get the kind of help you wanted?||2.58 (1.00)||2.51 (0.87)||0.53 (189)||.95|
|3. To what extent has the program helped to solve your problems?||2.21 (0.89)||2.07 (0.77)||1.05 (189)||.27|
|4. If a friend were in need of similar help, would you recommend our program to them?||3.04 (0.89)||3.00 (0.79)||0.35 (189)||.72|
|5. How satisfied are you with the amount of help you have received?||2.84 (0.97)||2.64 (0.82)||1.49 (189)||.13|
|6. Has the help you received helped you to deal better with your problems?||2.60 (0.90)||2.60 (0.79)||0.06 (189)||.95|
|7. Overall, how satisfied are you with the service you have received?||2.72 (1.05)||2.65 (0.94)||0.47 (189)||.63|
|8. If you needed to seek help again, would you come back to this program?||3.04 (1.03)||2.90 (0.89)||0.98 (189)||.32|
The mean of general acceptability of HDep was 13.84 (SD 3.97; range 5-20) for HPUs and 13.97 (SD 3.29; range 5-20) for PUs, with no significant difference among types of users (t189=0.24; P=.80;). The item with the highest rating was HDep, a user-friendly program, and the item with the lowest rating was HDep, which met my expectations.
|Item||Home page users (n=67), mean (SD)||Program users (n=124), mean (SD)||Values|
|t test (df)||P value|
|Scale mean||13.84 (3.97)||13.97 (3.29)||0.24 (189)||.80|
|1. Home page specifies what I can find in HDep and persuades me to register.||2.81 (0.89)||2.92 (0.78)||0.91 (189)||.36|
|2. Once I am registered, the layout of the module page motivates me to go into each module.||2.72 (0.91)||2.69 (0.77)||0.24 (189)||.80|
|3. HDep seems like a useful tool to manage depression on my own.||2.70 (0.87)||2.73 (0.88)||0.24 (189)||.80|
|4. Participating in HDep has met my expectations.||2.72 (0.90)||2.58 (0.85)||1.02 (189)||.30|
|5. HDep is a user-friendly program.||2.90 (0.97)||3.05 (0.80)||1.23 (189)||.21|
Profiles of Users’ Satisfaction and Acceptability With HDep
Logistic regression analyses showed that among HPUs, women (odds ratio [OR] 3.44, 95% CI 1.16-10.0) were more satisfied with HDep, whereas among PUs, older participants (OR 1.04, 95% CI 1.01-1.08), those with paid work (OR 3.12, 95% CI 2.40-7.69), those who had not been in therapy (OR 2.42, 95% CI 1.09-5.98), and those who had not attempted suicide (OR 3.44, 95% CI 1.08-11.11) showed higher satisfaction (). There were no significant differences in acceptability ratings among HPUs by sex, age, or the presence or absence of depressive symptoms. Among PUs, those with paid work (OR 2.50, 95% CI 1.16-5.55), those who had not been in therapy (OR 3.17, 95% CI 1.38-7.30), those without disability associated with depression (OR 2.94, 95% CI 1.35-6.66), and those who had not attempted suicide (OR 2.63, 95% CI 1.03-6.66) gave higher acceptability ratings ( ).
|ORb (95% CI)||P value||OR (95% CI)||P value|
|Age||1.01 (0.97-1.05)||.46||1.04 (1.01-1.08)||.01|
|Female||3.44 (1.16-10.0)||.02||1.01 (0.36-2.89)||.98|
|Has partner||N/Ac||N/A||1.51 (0.63-3.70)||.35|
|High school or more||N/A||N/A||3.12 (0.38-2.5)||.29|
|Has not been in therapy||N/A||N/A||2.42 (1.09-5.98)||.04|
|Depressive symptoms||0.98 (0.94-1.03)||.37||0.97 (0.93-1.01)||.28|
|Without disability the previous month owing to depressive symptoms||N/A||N/A||1.85 (0.26-2.35)||.21|
|Without previous depression||N/A||N/A||2.5 (0.08-3.89)||.24|
|Medication (for feeling nervous, anxious, or excessively energetic)||N/A||N/A||2.00 (0.85-4.68)||.11|
|No suicide attempts||N/A||N/A||3.44 (1.08-11.11)||.03|
|Alcohol use||N/A||N/A||1.43 (0.55-3.72)||.45|
|Drug use||N/A||N/A||1.33 (0.19-2.89)||.68|
aCSQ-8: Client Satisfaction Questionnaire-8.
bOR: odds ratio.
cN/A: not applicable.
|Characteristics||Acceptability of HDep ≥16|
|ORa (95% CI)||P value||OR (95% CI)||P value|
|Age||1.01 (0.97-1.05)||.49||1.03 (0.99-1.07)||.63|
|Female||2.43 (0.90-7.14)||.09||3.13 (0.99-9.83)||.60|
|Has partner||N/Ab||N/A||1.92 (0.85-4.34)||.11|
|High school or more||N/A||N/A||2.08 (0.42-2.37)||.36|
|Has not been in therapy||N/A||N/A||3.17 (1.38-7.30)||.01|
|Depressive symptoms||0.97 (0.93-1.02)||.65||0.96 (0.93-1.00)||.26|
|Without disability the previous month owing to depressive symptoms||N/A||N/A||2.94 (1.35-6.66)||.01|
|Without previous depression||N/A||N/A||0.34 (0.74-1.62)||.18|
|Medication (for feeling nervous, anxious, or excessively energetic)||N/A||N/A||1.20 (0.54-2.67)||.65|
|No suicide attempts||N/A||N/A||2.63 (1.03-6.66)||.04|
|Alcohol use||N/A||N/A||1.25 (0.51-3.05)||.62|
|Drug use||N/A||N/A||2.17 (0.57- 8.33)||.26|
aOR: odds ratio.
bN/A: not applicable.
Acceptability of Content, Design, and Tools of the Program
Content was assessed similarly for HPUs and PUs (HPUs: mean 7.1, SD 2.9; PUs: mean 7.0, SD 2.5; range 0-10; t189=0.16; P=.87). The design was also evaluated positively, with means of 7 and 7.5 (HPUs: mean 7.5, SD 2.6; PUs: mean 7.0, SD 2.5), with no significant difference among types of users (t189=1.33; P=.18;).
HDep individual components, evaluated only by PUs, were scored from 5.19 (SD 3.31) to 6.88 (SD 3.14). Forums (mean 5.62, 3.24) and chats (mean 5.19, SD 3.31) had the lowest acceptability. Depressive symptom assessment and feedback, evaluated by both types of users, had the highest acceptability (PUs: mean 6.88, SD 3.14; HPUs: mean 6.73, SD 3.79), with no significant difference between the 2 user types ().
|Item (rate from 0 to 10 how much each of the following has helped you to manage depression)||Home page users (n=67), mean (SD)||Program users (n=124), mean (SD)||Values|
|t test (df)||P value|
|HDep content||7.1 (2.93)||7.00 (2.52)||0.16 (189)||.87|
|HDep design||7.52 (2.60)||7.00 (2.55)||1.33 (189)||.18|
|Module information||N/A||6.52 (3.10)|
|Case samples||N/A||6.52 (3.01)|
|Activities (scales, thought charts, etc)||N/A||6.46 (3.08)|
|Thought exercises||N/A||6.35 (3.33)|
|Depressive symptom assessment and feedback||6.73 (3.79)||6.88 (3.14)||0.27 (189)||.79|
aN/A: not applicable.
Analysis of Open-ended Questions
Responses to open-ended questions fell into one of three categories: (1) Liked it, (2) Had other expectations, or (3) Did not like it. The fourth category included those who did not respond (). Examples of these explanations are presented in .
The three categories were as follows:
- Liked the content—approximately half of the users liked the content: 52% (35/67) of the HPUs and 43.5% (54/124) of the PUs, with no significant difference observed between the 2 groups ( ). HPUs considered the content good, that the help was excellent, and that the use of cases from everyday life encouraged reflection; however, they sometimes considered the web interface cold ( ). PUs who liked the content found it systematic, that it gradually made it easier to face life, that the content was meaningful and friendly, and that it allowed them to measure their personal progress. Some respondents who mentioned positive features also pointed out others that required improvement, including a need for additional resources to strengthen individual commitment and motivation.
- Had other expectations—a significantly greater number of PUs had other expectations (31/124, 25%) than HPUs (8/67, 12%; ). HPUs reported signing up in search of new information and help and someone to tell them everything was okay, which they did not find. PUs also sought not only interaction with a machine but also contact with a human being, an expert, in forums and chat, with email and mobile phone reminders ( ).
- Did not like the content—respondents who did not like HDep content included 12% (8/67) of HPUs and 16.9% (21/124) of PUs ( ). HPUs thought HDep was neither helpful nor harmful, that they did not need a pat on the back, and that a little rough treatment might not hurt. PUs said that the content did not motivate them to continue; in the opinion of one, “there was too much content to read which you can’t when you are depressed” ( ).
|Category||Home page users||Program users|
|Had other expectations|
|Did not like it|
Open responses regarding the HDep design were also organized into 3 categories, plus one for nonrespondents (). Examples of open-ended explanations for the design ratings are presented in .
The three categories were as follows:
- Liked the design—more than half of the users liked the design, with no significant difference between HPUs and PUs (HPUs: 35/67, 52%; PUs: 73/124, 58.9%; ). HPUs found it attractive, user friendly, and colorful; they considered the depression test to be active and they liked how it worked. PUs considered the site neat and well organized, the tools simple, the design motivating, intuitive, and entertaining but perhaps a little too long ( ).
- Had other expectations—fewer participants in each group had other expectations (HPUs: 3/67, 5%; PUs: 4/124, 3.2%). HPUs said they did not receive the expected response or care for their symptoms and had no improvement; PUs believed that the web program, by its nature, reinforced the causes of their depression, not because of problems with its design, but simply because it was a website, meaning that it did not provide contact with a professional, which led to feelings of abandonment ( ).
- Did not like the design—significantly more PUs disliked the design (32/124, 22.8%) than HPUs (9/67, 13%). HPUs noted that HDep was only readable on a computer screen, and it was not encouraging; from their perspective, it should have fewer bright colors and simpler graphics ( ).
|Category||Home page users||Program users|
|Had other expectations|
|Did not like it|
This study explored the satisfaction and acceptability of the updated version of the internet-based preventive intervention, HDep  and found them to be consistent with evaluations in a review of digital interventions in LMICs [ ]. Two types of users were observed: HPUs, who visit only the home page, make use of the information available there, respond to the CES-D questions, and obtain feedback on their scores and PUs, who also register for a 6-module intervention. The results showed that HPUs and PUs with similar initial levels of depressive symptoms (CES-D>16; 97% and 100%, respectively) showed moderate levels of satisfaction (HPUs: mean 21.90, SD 6.7, range 8-32; PUs: mean 21.10, SD 5.8, range 8-32) and acceptability (HPUs: mean 13.84, SD 3.97, range 5-20; PUs: mean 13.97, SD 3.29, range 5-20). Among HPUs, women expressed higher satisfaction than men, and among PUs, those who were older, employed, not in therapy before, and reported no previous suicide attempts showed higher acceptability.
Of the survey respondents, 52% (35/67) of HPUs and 43.5% (54/124) of PUs reported liking the HDep content; 12% (8/67) and 25% (31/124), respectively, had other expectations, and 12% (8/67) and 16.9% (21/124), respectively, did not like it. With respect to the design, 52% (35/67) of HPUs and 58.9% (73/124) of PUs reported liking it; 5% (3/67) and 3.2% (4/124), respectively, had other expectations, and 13% (9/67) and 25.8% (32/124), respectively, did not like it. Thus, there was no evidence that PUs were more satisfied or indicated a higher degree of acceptability than HPUs, as was hypothesized, based on the fact that the latter received a smaller portion of the intervention. It may be that there is self-selection of users to the dose of the intervention they need. For some, the home page information and feedback about their symptoms seem to be sufficient, whereas others seem to need the entire intervention.
The satisfaction level was above the scale mean, suggesting that users were more satisfied than dissatisfied, to the degree that they “would recommend it to someone in need” or “would use it again if depressed.” Less satisfied participants believed that HDep did not meet their needs. This finding may reflect the high levels of depressive symptoms reported by almost all participants, as well as possible psychopathologies in PUs. Such participants were not the target population of the intervention design, but they were the ones seeking help. Muñoz et al  found that this is a common phenomenon, as internet-based interventions intended to be preventive seem to attract individuals who are currently experiencing enough symptoms to screen positive for a major depressive episode; however, only 30% are appropriate for a depression prevention intervention. Other studies have also found a high percentage (up to 90%) of participants in web-based interventions who were highly depressed [ ].
Consistent with this notion, higher satisfaction was observed in PUs with less disability owing to depression, no suicidal ideation, and no experience with psychotherapy. According to Rost et al , the severity of symptoms and possible comorbidities affect users’ perceptions of acceptability and satisfaction. Self-guided interventions have been observed to be more efficacious for users with less psychotherapeutic experience [ ].
Findings from the World Mental Health Survey show that in Mexico, 58.3% of people with a diagnosis of major depressive disorder who felt they needed treatment, only 6.4% received treatment that was minimally adequate . This situation may explain why many people with probable mental health disorders seek help in web-based interventions. One-third of HDep users said they had, at some point in the past, sought either psychological or psychiatric mental health treatment; a similar web-based intervention for depression reported by Christensen et al [ ] also found that 64% of participants had previously sought professional help. Our findings show that users with previous experience in therapy were least satisfied and gave HDep lower acceptability ratings. Participants with this experience could be advised early on about what they could realistically expect from HDep.
The general acceptability ratings of HDep were above the mean, suggesting that users considered it acceptable but with room for improvement. Some of the features they valued were its user-friendliness and usefulness in managing their depression, but it did not fully meet their expectations. Regarding the content, more than half of the respondents described it as good, meaningful, and helpful in making them feel better, friendly, systematic, well-paced, and encouraging reflection. More than half of the respondents liked the design, describing it as easy to understand, well organized, attractive, user friendly, colorful, intuitive, and entertaining.
The test for depressive symptoms and feedback regarding results had the highest acceptability ratings, whereas forums and chats had the lowest ratings (). Some studies have found a high degree of satisfaction with mood-monitoring rating tools in web-based treatment for depression [ ]. Questionnaires and psychometric tests sometimes give participants a feeling of personalized treatment [ ]. The acceptability of HDep and similar interventions may therefore be improved by increasing the use of these tools.
It is noteworthy that forums received low ratings, although it has previously been observed that forums allow for meaningful social exchange of experiences . One possible explanation may lie in the observation that although a large proportion of participants used forums (60.9%), far fewer posted comments (16.3%) [ ]. The lack of participation of a professional in forums and chats is also a possible reason for the low ratings.
Three major areas were found to be lacking in the design and content of HDep. These were described as (1) not being sufficiently persuasive, dynamic, motivating, or appealing; (2) not meeting users’ expectations; and (3) being cold, being just a platform, and lacking professionals to support, advise, and track people’s progress. Other studies have also found that participants in automated programs are concerned about their being too impersonal . Therapeutic persuasiveness, the incorporation of persuasive principles of design, and behavior change have been described as the most robust predictors of adherence [ ]. Not meeting users’ expectations may be particularly important for depressed users, because as Bernard et al [ ] reported, these users become particularly upset when they find unexpected, irrelevant, or inappropriate content. Despite the importance of design in developing successful web-based interventions, Neilsen and Wilson [ ] concluded in a literature review that design elements and human–computer interaction remain poorly understood and that “internet-based e-mental health interventions are routinely implemented without sufficiently describing the relevant human–computer interaction design features applied.” In this respect, HDep would benefit from incorporating a user-centered design approach to improve the layout of its content [ ].
The content deficiency perceived by HDep users was the need for professional help or specialized guidance. This issue has often been reported in the literature on unguided interventions [, ], and it also turned out to be important for Mexican users in our survey. One of the most attractive aspects of web-based interventions for mental health problems is their reduced cost, including personalized professional help, which would be a significant burden. This is very much the case for HDep, which is supported by a public health institution as a translational research project. Features to compensate for the lack of professional support could include automated dialogue components, such as automated SMS text messages or email messages or gaming features [ ].
Overall, HDep showed moderate levels of satisfaction and acceptability and high levels for more than half of its participants, despite a level of depressive symptoms high enough to suggest a major depressive episode. HPUs and PUs rated satisfaction and acceptability similarly, contrary to our hypothesis that the latter, having received a larger portion of the intervention, would be more satisfied. Respondents’ ratings on satisfaction and acceptability may be related to their ability to choose how much of the program they want to do: whether they just want an overview from the home page or want to work on the intervention modules. They considered the content culturally sensitive, reflecting their everyday experiences; they found the design to be friendly, with tools that were well organized, simple, and motivating. The main limitations were the lack of contact with a professional, and in some cases, content that did not motivate and was not encouraging. Users who responded to the survey provided abundant suggestions on feasible ways to alleviate some of the deficiencies. Some of these deficiencies coincide with those observed in other web-based interventions for depression [, , , ].
This study has the strength of being based in a real-world setting, not in a confined research environment, so that it reflects more closely what real participants do. However, this feature also makes it more difficult to implement strict methodological control. HDep is the first web-based self-help intervention for depression in Latin America and has been in operation for 11 years. It is a promising and cost-effective tool that can contribute to reducing the treatment gap for depression in Mexico . It has the potential to provide mental health literacy to a large group of users and can be integrated into a preventive stepped-care approach [ ]. Being a qualitative–quantitative study from a non–English-speaking LMIC, this study adds to existing research on acceptance and satisfaction with cognitive behavioral therapy–based programs for depression in high-income countries. There appear to be no studies directly comparing users’ performance in web-based interventions to assess differences among countries with different income levels. These could include cultural differences: English-speaking users of web-based interventions tend to emphasize the benefits of introspection and self-awareness, which is congruent with the dominant individualized focus of Euro-American cultural orientation compared to a more collectivist and relational Latin American cultural orientation [ ].
Although there are many effective internet-based interventions for depression, there are far fewer with open access for general use . Web-based interventions have great potential in Mexico, where it is estimated that 80.6 million people use the internet [ ], 58% of whom are interested in health content [ ]. The findings of this study are important for the creation and adaptation of web-based interventions in an LMIC, such as Mexico, where access to treatment is a major concern [ ] and web-based prevention and treatment programs can help to deliver evidence-based alternatives. HDep is promising, but it is necessary to further document the pitfalls, strengths, and challenges of this type of intervention in this particular context.
A major limitation of this study was its response rate of 1.45% (191/13,207). This response implied a CI of 95% and a margin of error of 7%. Low response rates are an important concern that affects the validity of web surveys . In a meta-analytic study, Daikeler et al [ ] found that web surveys yielded a response rate that was 12% points lower than other survey modes and concluded that the difference is not the result of a particular study design. They also found evidence that low response rates do not necessarily indicate a large nonresponse error.
Participants who did not respond may have been less satisfied and less positive than those who did . The levels of depressive symptoms and other mental health problems of the people surveyed may explain the low response rate; people with emotional stability are more likely to complete a survey [ ]. It is also likely that many of the people contacted were no longer using the intervention, as the survey was sent to those registered in the previous year. There was just 1 reminder email; additional reminders may have increased the response rates [ ]. Some of those contacted may have visited HDep just once. It may also be that the incentive (receiving a list of positive thoughts to practice) was not found attractive [ ]. There was also no confirmation that users received the survey; email questionnaires are often treated as spam or blocked [ , ].
To estimate other possible sources of bias, the characteristics of the study sample were compared with those of a previous study on HDep , which obtained data directly from registered users. No differences >10% were found between the 2 samples in terms of sex, depressive symptoms, disability, suicide attempts, previous psychological or psychiatric treatment, alcohol or drug use, or drug treatment. The proportion of users aged >30 years was >12.5% in the current sample. The proportion of HPUs and PUs in this study was similar to that in a previous study [ ]: HPUs made up 35.1% (67/191) of respondents in this study and 38.32% (10,760/28,078) in the previous study; PUs made up 64.9% (124/191) and 61.69% (17,318/28,078), respectively. These figures suggest that the low response rate did not skew the sample with respect to these variables, but there could still be other biases, such as a self-selection bias of eagerness to respond.
We wish to thank Lucero Sáenz of the Department of Psychiatry of the Autonomous University of Nuevo León, who participated in the design of the study protocol, while an intern at the National Institute of Psychiatry and Jorge Nuñez, from Insurgentes University, also an intern who participated in emailing and received the completed questionnaires.
MAL and MT conceptualized the study; MAL, MT, and PP worked on the methodology; PP and LN worked on the statistical and qualitative analyses; MAL wrote the original draft; MT reviewed and edited the manuscript. All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
- Depression and other common mental disorders : global health estimates. World Health Organization. 2017. URL: https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf [accessed 2021-01-16]
- The burden of mental disorders in the region of the Americas, 2018. Pan American Health Organization. 2018. URL: https://iris.paho.org/bitstream/handle/10665.2/49578/9789275120286_eng.pdf [accessed 2021-03-04]
- Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry 2017 Dec;210(2):119-124 [FREE Full text] [CrossRef] [Medline]
- Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther 2009 Dec;38(4):196-205. [CrossRef] [Medline]
- Muñoz RF, Leykin Y, Barrera AZ, Dunn LB, Gutierrez R, Curland RA, et al. Who comes to a self-help depression prevention website? Characteristics of Spanish- and English-speaking visitors. Internet Interv 2021 Mar;23:100359 [FREE Full text] [CrossRef] [Medline]
- Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an 'individual patient data' meta-analysis. Psychol Med 2015 Oct;45(13):2717-2726. [CrossRef] [Medline]
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
- Andersson G. Using the internet to provide cognitive behaviour therapy. Behav Res Ther 2009 Mar;47(3):175-180. [CrossRef] [Medline]
- Quiñonez-Freire C, Vara MD, Herrero R, Mira A, García-Palacios A, Botella C, et al. Cultural adaptation of the Smil ing is Fun program for the treatment of depression in the Ecuadorian public health care system: a study protocol for a randomized controlled trial. Internet Interv 2021 Mar;23:100352 [FREE Full text] [CrossRef] [Medline]
- Arjadi R, Nauta MH, Chowdhary N, Bockting CL. A systematic review of online interventions for mental health in low and middle income countries: a neglected field. Glob Ment Health (Camb) 2015;2:e12 [FREE Full text] [CrossRef] [Medline]
- Martínez P, Rojas G, Martínez V, Lara MA, Pérez JC. Internet-based interventions for the prevention and treatment of depression in people living in developing countries: a systematic review. J Affect Disord 2018 Jul;234:193-200. [CrossRef] [Medline]
- Salamanca-Sanabria A, Richards D, Timulak L, Connell S, Perilla MM, Parra-Villa Y, et al. A culturally adapted cognitive behavioral internet-delivered intervention for depressive symptoms: randomized control. JMIR Ment Health 2020 Jan 31;7(1):e13392 [FREE Full text] [CrossRef] [Medline]
- Deady M, Choi I, Calvo RA, Glozier N, Christensen H, Harvey SB. eHealth interventions for the prevention of depression and anxiety in the general population: a systematic review and meta-analysis. BMC Psychiatry 2017 Aug 29;17(1):310 [FREE Full text] [CrossRef] [Medline]
- Ayuda para depresión. Instituto Nacional de Psiquiatría. 2008. URL: https://ayudaparadepresion.org.mx [accessed 2021-10-18]
- Lara M, Tiburcio M, Aguilar-Abrego A, Sánchez-Solís A. A four-year experience with a web-based self-help intervention for depressive symptoms in Mexico. Pan American Health Organization. 2014. URL: https://iris.paho.org/handle/10665.2/7999 [accessed 2022-01-14]
- Lara MA, Navarro C, Rubí NA, Mondragón L. Outcome results of two levels of intervention in low-income women with depressive symptoms. Am J Orthopsychiatry 2003 Jan;73(1):35-43. [CrossRef] [Medline]
- Muñoz R, Ying Y. The Prevention of Depression: Research and Practiceled Trial. Charles Village, Baltimore: Johns Hopkins University Press; 1993.
- Lara MA, Acevedo M, López E. La investigación aplicada a la educación para la salud: el caso de la depresión en mujeres. Psicología y Salud 1998;12:55-66.
- Jiménez-Molina A, Franco P, Martínez V, Martínez P, Rojas G, Araya R. Internet-based interventions for the prevention and treatment of mental disorders in Latin America: a scoping review. Front Psychiatry 2019;10:664 [FREE Full text] [CrossRef] [Medline]
- Lemeshow S, Hosmer Jr DW, Klar J, Lwanga SK. Adequacy of Sample Size in Health Studies. Hoboken, New Jersey, United States: John Wiley & Sons; 1990.
- Daikeler J, Bošnjak M, Lozar MK. Web versus other survey modes: an updated and extended meta-analysis comparing response rates. J Surv Stat Methodol 2020;8(3):513-539. [CrossRef]
- Radloff LS. The CES-D Scale: a self-report depression scale for research in the general population. Appl Psychol Measur 1977 Jun 01;1(3):385-401. [CrossRef]
- Tiburcio M, Natera G. Adaptación al contexto ñahñú del Cuestionario de Enfrentamientos (CQ), la Escala de Síntomas (SRT) y la Escala de Depresión del Centro de Estudios Epidemiológicos (CES-D). Salud Mental 2007;30(3):48-58 [FREE Full text]
- Larsen DL, Attkisson C, Hargreaves WA, Nguyen TD. Assessment of client/patient satisfaction: development of a general scale. Eval Program Plan 1979 Jan;2(3):197-207. [CrossRef]
- Roberts RE, Attkisson C. Assessing client satisfaction among Hispanics. Eval Program Plan 1983 Jan;6(3-4):401-413. [CrossRef]
- Attkisson C, Zwick R. The client satisfaction questionnaire. Eval Program Plan 1982 Jan;5(3):233-237. [CrossRef]
- Rost T, Stein J, Löbner M, Kersting A, Luck-Sikorski C, Riedel-Heller SG. User acceptance of computerized cognitive behavioral therapy for depression: systematic review. J Med Internet Res 2017 Sep 13;19(9):e309 [FREE Full text] [CrossRef] [Medline]
- Martínez V, Espinosa-Duque D, Jiménez-Molina A, Rojas G, Vöhringer PA, Fernández-Arcila M, et al. Feasibility and acceptability of "Cuida tu Animo" (Take Care of Your Mood): an internet-based program for prevention and early intervention of adolescent depression in Chile and Colombia. Int J Environ Res Public Health 2021 Sep 13;18(18):9628 [FREE Full text] [CrossRef] [Medline]
- Carter H, Araya R, Anjur K, Deng D, Naslund JA. The emergence of digital mental health in low-income and middle-income countries: a review of recent advances and implications for the treatment and prevention of mental disorders. J Psychiatr Res 2021 Jan;133:223-246 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Korten A. Web-based cognitive behavior therapy: analysis of site usage and changes in depression and anxiety scores. J Med Internet Res 2002 Feb;4(1):e3 [FREE Full text] [CrossRef] [Medline]
- Bücker L, Schnakenberg P, Karyotaki E, Moritz S, Westermann S. Diminishing effects after recurrent use of self-guided internet-based interventions in depression: randomized controlled trial. J Med Internet Res 2019 Oct 02;21(10):e14240 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Jorm AF. Delivering interventions for depression by using the internet: randomised controlled trial. Br Med J 2004 Jan 31;328(7434):265 [FREE Full text] [CrossRef] [Medline]
- Richards D, Murphy T, Viganó N, Timulak L, Doherty G, Sharry J, et al. Acceptability, satisfaction and perceived efficacy of "Space from Depression" an internet-delivered treatment for depression. Internet Interv 2016 Sep;5:12-22 [FREE Full text] [CrossRef] [Medline]
- Pretorius C, McCashin D, Kavanagh N, Coyle D. Searching for mental health: a mixed-methods study of young people's online help-seeking. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020 Presented at: CHI Conference on Human Factors in Computing Systems; April 25 - 30, 2020; Honolulu HI USA p. 1-13. [CrossRef]
- Crisp DA, Griffiths KM. Reducing depression through an online intervention: benefits from a user perspective. JMIR Ment Health 2016 Jan 08;3(1):e4 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Yom-Tov E. Predicting user adherence to behavioral eHealth interventions in the real world: examining which aspects of intervention design matter most. Transl Behav Med 2018 Sep 08;8(5):793-798. [CrossRef] [Medline]
- Bernard R, Sabariego C, Cieza A. Difficulties encountered by people with depression and anxiety on the web: qualitative study and web-based expert survey. J Med Internet Res 2019 Oct 31;21(10):e12514 [FREE Full text] [CrossRef] [Medline]
- Neilsen AS, Wilson RL. Combining e-mental health intervention development with human computer interaction (HCI) design to enhance technology-facilitated recovery for people with depression and/or anxiety conditions: an integrative literature review. Int J Ment Health Nurs 2019 Feb 22;28(1):22-39. [CrossRef] [Medline]
- Juárez-Ramírez R. User-centered design and adaptive systems: toward improving usability and accessibility. Univ Access Inf Soc 2016 Jun 22;16(2):361-363. [CrossRef]
- Kostera T. Digital applications in psychotherapy: finding an efficient blend between the technology and the human ? An interview with Dr. Adrian Aguilera. The Digital Patient. 2017. URL: https://blog.der-digitale-patient.de/en/digital-applications-psychotherapy-interview-aguilera/ [accessed 2022-02-15]
- Rogers MA, Lemmen K, Kramer R, Mann J, Chopra V. Internet-delivered health interventions that work: systematic review of meta-analyses and evaluation of website availability. J Med Internet Res 2017 Mar 24;19(3):e90 [FREE Full text] [CrossRef] [Medline]
- Comunicado de prensa Núm 103/20 ENDUTIH 2019. El Instituto Nacional de Estadística y Geografía (INEGI). 2019. URL: https://www.inegi.org.mx/contenidos/saladeprensa/boletines/2020/OtrTemEcon/ENDUTIH_2019.pdf [accessed 2020-10-28]
- Bureau M, Kantar M. Estudio de consumo de medios y dispositivos 2018. Corte Salud. 2018. URL: https://www.iabmexico.com/estudios/salud-consumo-medios-2018/ [accessed 2020-10-28]
- Fan W, Yan Z. Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav 2010 Mar;26(2):132-139. [CrossRef]
|CES-D: Center for Epidemiological Studies Scale-Depression|
|CSQ-8: Client Satisfaction Questionnaire-8|
|HDep: Help for Depression (in Spanish, ADep: Ayuda Para Depresión)|
|HPU: home page user|
|LMIC: low- and middle-income country|
|OR: odds ratio|
|PU: program user|
Edited by A Mavragani; submitted 14.04.21; peer-reviewed by A Salamanca-Sanabria, CY Lin; comments to author 17.08.21; revised version received 25.10.21; accepted 03.12.21; published 04.04.22Copyright
©Ma. Asunción Lara, Pamela Patiño, Marcela Tiburcio, Laura Navarrete. Originally published in JMIR Formative Research (https://formative.jmir.org), 04.04.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.