The Focus on New Fathers pilot [19] delivered brief plain language texts to soon-to-be and new fathers that contain tips, prompts, information, and links to web support—SMS4dads. NSW Health promoted the availability of SMS4dads via social and paid media. Participants (fathers) from 4 local health districts in the state of New South Wales, Australia, were able to enroll in SMS4dads at any time from 16 weeks of gestation until 24 weeks after birth and receive messages (approximately 3 per week) until 48 weeks after birth. The messages, which were limited to 160 characters and a reading age of 7 years (Flesch-Kincaid level <9), provided tips that were timed to match the development of the fetus or baby and were designed to encourage father-infant engagement, father-partner support, and self-care. Approximately 30% of texts included a hyperlink to not-for-profit mental health or parenting websites with further information. At enrollment and at 3 subsequent points in the schedule of messages, participants were invited to complete the Kessler Psychological Distress Scale (K10) [20]. A purpose-designed Mood Checker interactive text was sent at 3 weekly intervals. Mood Checker messages were keyed to common challenges faced by parents during pregnancy and after birth, such as infant crying or reestablishing intimacy. Participants who indicated that they were distressed (high score on the K10 or response to interactive Mood Checker text that they were distressed and “wanted help now”) were linked to online professional mental health support. Although the program resulted in over 3000 fathers enrolling, an important measure of the success of the program was the ability of SMS4dads to retain participants in the service to keep receiving the texts.

Participants could leave the program at any point by texting back “STOP” or similar words to any message. When such a reply was received, SMS4dads staff manually activated the request using a password-protected webpage to remove the mobile number from all SMS4dads active elements. An automated message was then sent to the participants advising them that messages had stopped and asking them to provide a reason for their dropout request.

Data collected for the attrition analysis included demographics, exit survey data, data related to 2 program features, and data on inactive participants.

The study received approval from the Human Research Ethics Committee of the University of Newcastle, New South Wales (approval number H-2016-0055). Participants provided informed consent during registration to the program and received an information statement explaining the potential use of their data for research activities, allowing secondary analysis without additional consent. To protect the privacy of participants, some data were deleted during extraction, such as the participants’ name, address, and contact information. To the researchers, the participants were anonymous or unidentified. No compensation was received by participants.

Frequencies for all reasons for dropping out are presented. Chi-square tests compared differences among the following demographic variables between participants who completed the exit survey (ie, provided a reason for dropping out) and those who did not complete the exit survey: rurality (rural or urban), Aboriginal or Torres Strait Islander status, education level (bachelor degree or above, advanced diploma or diploma, certificate III/IV, year 12, and year 11 or below), perinatal stage at sign-up (prenatal or postnatal), smoking status (smoker or nonsmoker), AUDIT-C (not at risk or at risk), and K10 distress score (low, 10-15; moderate, 16-21; high, 22-29; and very high, 30-50).

Table 1 shows descriptive statistics of those who exited before completion. Participants who dropped out were enrolled for an average of 17.4 (SD 17.1) weeks and received an average of 66 (SD 64) texts. In terms of baby age, participants sent dropout requests from 25 weeks before the expected due date through to 53 weeks after birth (mean 9; SD 14 weeks).

Table 2 shows the crude and adjusted HRs for each baseline characteristic of interest. The sample considered by the models included all participants. The adjusted model used 3229 observations. Compared with their counterparts, participants who were smokers, recorded risky alcohol consumption, had a lower education level, or signed up prenatally had 30% to 47% higher hazard of dropout from the program. Additionally, compared with participants expecting their first child, participants who were expecting another child had 82% higher hazard of dropout from the program (HR 1.82, 95% CI 1.48-2.23; P≤.001). Participant age, Aboriginal or Torres Strait Islander status, rurality, and psychological distress score (K10 category) were not associated with dropout.

Of all participants who were sent an exit survey, approximately one-third replied (202/605, 33.4%) with a reason for dropping out. The reasons for the dropout request are given in Table 5.

Chi-square tests indicated that there were no significant differences in baseline demographic characteristics between participants who responded to the exit survey and those who did not, including rurality, Aboriginal or Torres Strait Islander status, education level, perinatal stage at sign-up, smoking status, AUDIT-C risk, and K10 distress score.

Of the 2612 participants currently enrolled who had not opted out, 216 had not clicked on any links or the Mood Checker link for at least 10 weeks in May 2021 and were sent a reminder text message (Figure 1). One-third of participants (71/216, 32.9%) provided a response indicating that they wanted either to continue in the program (53/71, 74.6%) or wanted to drop out (18/71, 25.4%). The remaining participants were sent a second reminder message, to which one-quarter (36/145, 24.8%) responded (22/36, 61.1% opted to continue and 14/36, 38.9% opted to drop out), leaving 109 participants to be telephoned. A total of 45 participants who were called answered or responded to the voicemail message (“checking if you want to continue”), with most choosing to continue in the program (38/45, 84.4%) and others opting to drop out (7/45, 15.6%), leaving 64 participants who made no response during this process and did not take action to drop out in response to the phone message reminder. Among those who responded, some participants offered explanations for inactivity, such as “a busy period at work” or “I never click on links because of scams.” Several stressed that they valued the messages even though they did not click links.

In sum, of the participants who had not clicked on any links or Mood Checker for at least 10 weeks, 52.3% (113/216) confirmed that they wanted to remain in the program, 28.7% (62/216) remained in the program but did not indicate their intention, and 18% (39/216) chose to drop out. Applying this ratio of inactive users (216/2612) to the total participants (n=3261) suggests that 270 participants would be inactive users. Of these, 49 (18%) could be expected to drop out, indicating a total estimated dropout rate of 20.1%.

Assessing the attrition of participants is an important feature of program evaluation and improvement. In this analysis, 4 approaches were adopted to identify possible key features of attrition, which may, in turn, provide guidance for participant retention in text-based parenting or mental health interventions. When demographic factors were analyzed, several factors were identified that do not appear to influence retention: age, living in a rural area, being Aboriginal or a Torres Strait Islander, or experiencing high levels of psychological distress. However, men enrolling postnatally, men with previous children, smokers, men without tertiary education, and men consuming alcohol at risky levels were found to be more likely to exit before completion.

The increased attrition for experienced fathers is understandable, as they may have previously encountered the information on infant development that is provided in SMS4dads. However, for those enrolling before the birth, the higher numbers exiting before the final texts at 48 weeks may reflect participant fatigue; possibly, the participants did not find the messages stimulating after many months. Because smoking and alcohol behaviors are not mentioned in the text messages, the increased attrition is unlikely to be linked to specific content of the program but may reflect characteristics of substance users such as impulsivity [25] or be linked to lifestyle factors [26]. Those with lower educational attainment, who are also more likely to smoke and be at risk in their alcohol consumption, may also be negatively influenced by lifestyle factors. A digital intervention targeting men from low-income areas found that the rate of attrition was higher in men from “high-risk” neighborhoods than from “resilient” neighborhoods [27].

Both smoking and alcohol consumption could be the subject of texts for fathers, as these behaviors are recognized as potential risks to the health and well-being of pregnant women and young infants [28,29]. However, even though programs specifically targeting fathers’ smoking and alcohol use have been reported [30,31], the branding of SMS4dads and message content is designed as “tips and information” for new fathers, not as a resource for “fathers who have a problem,” and so any additional messages for fathers on these topics may require an “opt in” process. The education level of participants may be better accommodated by reducing the literacy demands of the texts, and the postnatal texts may be reviewed to maintain interest among those participants who enrolled prenatally.

Two aspects of the SMS4dads program were investigated as possible triggers for participants to leave: being asked to complete the K10 scale and receiving particular messages. The analysis found that there was no significant increase in participants’ likelihood of exiting within 1 or 2 weeks of receiving the invitation to complete the K10. A frequency count of messages that occurred immediately before participants requested “STOP” revealed that some messages had higher rates of subsequent dropout, but no discernible pattern was evident apart from the common feature of a reminder to “opt out.” The message with the highest linked dropout number occurred at 9 weeks after birth. As discussed by Saleem et al [12], an approach to engagement that aims for “effective engagement” may be more appropriate to digital interventions than simply aiming to maximize engagement. The period after the birth, when particularly stressful challenges such as infant crying have peaked (after 6 weeks [32]), may be a period when fathers feel as if they are managing their family situation well enough that they do not require the text messages. In this case, an analysis of dropouts by baby age revealed no significant points of increased risk of dropout. However, a prerequisite to defining an optimum number of weeks for fathers to continue enrollment may be a clearer understanding of the needs of various subgroups of fathers in Australia.

Of those giving reasons for asking to be removed from the SMS4dads program, the low number of “not helpful” responses may indicate general agreement with the program content and delivery. However, for improving retention, having more detailed reasons for dissatisfaction would provide more guidance. Similarly, with only one-third of those exiting early providing any feedback response, and the most frequent reason provided being “other reasons,” the brief survey used in this study has not provided guidance for increasing retention or improving the content of the program. A more detailed exit survey, however, may not provide a useful measure of fathers’ engagement. Fathers and mothers expecting their first baby were recruited to the Baby Steps Wellbeing program, which consists of 9 web-based modules involving setting goals, solving problems, developing a plan, and taking action. Although only 21% of fathers used more than 1 module, compared with 54% of mothers, and only 13% of fathers set a goal, compared with 44% of mothers, scores for overall satisfaction, relevance, usefulness, and ease of finding what they wanted were not significantly different for mother and father participants [33]. Interviews have also been widely used to gain an understanding of disengagement [9]. However, interpreting the data obtained in the absence of guidelines or recognized standards in definitions of engagement has become difficult [9].

To investigate “inactive” participants, an arbitrary parameter of 10 weeks without clicking was chosen. Although other studies have used 30 days or 14 days without accessing the program to define nonuse, the extensive time frame for participation in SMS4dads, potentially 18 months if enrolled at 16 weeks, suggested a longer time measure. The process undertaken found that a subgroup of fathers, when contacted, asserted that they enjoyed the texts but had no interest in viewing the linked websites and resources. With this population in mind, a text-only version might be developed that would obviate the need for internet connection and use but that would require a different method of assessing active engagement in the program.

Overall, although demographic factors linked to exiting early were found and dropouts could be identified among inactive participants, the analysis of program features provided limited guidance for improving retention. Those designing “light touch” text-based health and parenting interventions may do well to plan their attrition analysis and include features that can give useful direction to improve retention.

The high level of participant retention in SMS4dads (80%) is in contrast to reported attrition rates in mental health and health promotion digital programs in general [2,34]. It is also markedly higher than programs promoted for parents that hoped to recruit and retain fathers in the intervention. An analysis of 28 Triple P research studies found that, in some studies, 100% of fathers dropped out [17]. The Baby Wellbeing program, which restricted recruitment to cohabiting couples, also failed to engage fathers and found only mothers benefited [33]. The ParentWorks program was heavily promoted to fathers with father-specific advertising but was offered as a generic parenting program to mothers and fathers. A total of 90% of enrolled fathers quit the program before completing the final module [4]. A feature of these programs is that, like almost all therapeutic and health promotion programs for parents, the content was trialed on mothers, and fathers were added as a target group only after efficacy was established with mothers [35]. By contrast, the content and language in the SMS4dads program is specifically tailored to fathers’ role during the perinatal period.

The SMS4dads program was designed as “light touch” so as to maximize program attractiveness to fathers. The study design therefore sought the minimum demographic data in the registration process and avoided the collection of data on the fathers’ experiences during the pregnancy, birth, and postnatal period. Characteristics of new fathers, such as unplanned pregnancy [36], experiencing a traumatic birth [37], or having a partner with mental distress [38], may have affected their commitment to their new role and, by extension, to the SMS4dads program. These factors were not assessed. The exit survey multiple-choice question on the “reason” for opting out, also designed with participant burden in mind, provided minimal guidance for improving retention.

Analyzing program features can provide guidance for improving program adherence by identifying which features of the participant population and of the program are linked to dropout rates. Approaches to improving measurement of attrition have included scales assessing engagement [11,39] and acceptability [40], and several systematic reviews have provided overviews [1,5,34]. However, the variety of components included in defining and describing attrition have continued to expand, and the difficulty in comparing studies that assess differing parameters continues. In our analysis, demographic features that may be expected to increase attrition, such as rurality, elevated levels of distress, or age, were not related to dropout, nor were the 2 program features of psychological test requests and message content. Smoking and alcohol use were linked to dropout, and provision of texts targeting these groups may reduce disengagement. Also, reducing literacy demand and increasing stimulation level of postnatal texts may reduce dropout among low-education-level participants or those who enrolled prenatally. Seeking feedback from inactive participants allowed an estimate of dropouts among this group and thereby an overall retention rate of 80%. The SMS4dads program may provide guidance for other interventions seeking to engage fathers. Compared with programs offering self-paced online support for fathers using content designed to be equally appealing to fathers and mothers, the overall retention level of 80% is high. Future studies may usefully investigate if tailoring the content of “parenting” interventions to specifically address fathers’ role may reduce disengagement and attrition among male parents.