Two physiotherapists recruited the patients with T2D for the MySteps trial from the primary care units of the Siun Sote area and through a local newspaper. Inclusion criteria for participation were age 20-69 years, BMI of <40 kg/m², not meeting the current PA recommendation for health, and the ability to conduct a 6-minute walk test [42]. Exclusion criteria for participation were any problem that limited the ability to walk without an aid, any health problem that hindered participation in PA, and inability or unwillingness to use a smartphone or wear an accelerometer.

A face-to-face session for the baseline measurements was arranged for each participant individually. The measurements included 7-day 24-hour accelerometer use, sleep diary, 6-minute walk test, and blood samples, which are not reported here. As part of the baseline measurements, the participants completed a questionnaire, which, among other things, included questions on intensity-specific PA and perceived behavior change needs for increasing daily walking and other nonexercise PA.

After the baseline measurements, the participants were randomized into three arms: (1) self-monitoring with counseling (SMC), (2) self-monitoring only, and (3) usual care. SMC was considered the primary intervention arm of the study and is the focus of the implementation evaluation of this paper.

SMC was built around an mHealth system, which included an accelerometer (MoveSense, Suunto Ltd) and a smartphone app (ExSed 2, UKK Terveyspalvelut Oy) for 26 weeks. The accelerometer was connected to the app via Bluetooth. The app provided cloud-computed personal data and feedback on accelerometer-measured PA, sedentary behavior, and time in bed (a proxy for sleep; Figure 1). It also served as a platform for setting and monitoring personal goals.

Previous studies on patients with T2D have shown more considerable impacts on physical behavior [11] and glycated hemoglobin [28] if self-monitoring was complemented with other intervention features. In the MySteps trial, self-monitoring with mHealth was supplemented with a PA leaflet devised for patients with T2D, 3 face-to-face counseling sessions, 4 telephone counseling contacts, and a YouTube video on how to increase daily walking. The features were designed by using information from former studies about the motivators and barriers of PA in a Finnish sample of adults at high risk of or diagnosed with T2D [43,44]. The timing, content, behavior change techniques (BCTs), and delivery modes of SMC are introduced in Multimedia Appendix 1. The BCTs were drawn from the Coventry, Aberdeen & London—Refined taxonomy refined for PA and healthy eating [45] and BCT taxonomy v1 [46].

The physiotherapists participated in a 4-hour training about the study protocol, measurements, and intervention contents before the first recruitment. The participant-specific counseling cards guided the physiotherapists to the protocol and contents of each intervention arm. The cards were also used in documenting the realization and duration of the sessions and contacts. At the first face-to-face session, the physiotherapists instructed the participants of the SMC and self-monitoring only group to use the accelerometer and smartphone app and to replace the accelerometer battery if necessary.

Proctor et al [34] have introduced 3 interrelated types of outcomes of implementation research: implementation outcomes, service outcomes, and client outcomes. This paper focuses on implementation outcomes, which, according to Proctor et al [34], include acceptability, adoption, appropriateness, costs, feasibility, fidelity, penetration, and sustainability. The same outcomes have been determined for evaluating the implementation of behavioral intervention technology [49]. However, the terminology and definitions related to the implementation outcomes are inconsistent, and the operationalization and measurement of the same outcome vary across the studies [34]. Especially, the constructs of acceptability, appropriateness, and feasibility have been used interchangeably [34].

The constructs selected for acceptability in the planning phase of this study (usefulness, ease of use, credibility, and satisfaction) considerably overlapped with the constructs of adoption, appropriateness, and feasibility introduced by Proctor et al [34] and Hermes et al [49]. Moreover, the rest of the introduced outcomes (costs, penetration, and sustainability) are considered more suitable for organization-level evaluation [34,49] and interventions aiming at larger-scale clinical uptake and maintenance. As a result, the fidelity and acceptability remained the most applicable implementation outcomes of the intervention.

Fidelity is defined as the extent to which the intervention was delivered as intended by the program developers [49,50]. The recommendations on fidelity evaluation in health behavior studies cover study design, provider training, treatment delivery, treatment receipt, and enactment of treatment skills [51]. In studies involving behavioral intervention technologies, fidelity is traditionally measured at the provider level [49]. Examples of typical fidelity measurements are protocol adherence, dose or amount of program delivered, and quality of program delivery [34].

Fidelity can be measured with observations, checklists, or self-reports [34]. A systematic review by Lewis et al [52] did not find any measures of fidelity in behavioral health interventions. A recent systematic review by Mettert et al [53] detected a rapid increase in measures across the implementation outcomes and identified 18 fidelity measures. However, the information on the psychometric properties of the instruments was poorly reported, which prevented us from making conclusions about their suitability.

In the MySteps trial, the fidelity of SMC was evaluated during the 6-month intervention (Figure 2) with a measure designed for the study purposes. The physiotherapists kept records of completed face-to-face sessions, telephone contacts, and their contents with participant-specific counseling cards. Compliance with using the accelerometer and smartphone app was obtained from the cloud data. A more specific description of fidelity evaluation is presented in Table 1.

Low acceptability may lead to poor fidelity and weak effectiveness [50]. Acceptability has thus been recognized as a key factor for the effectiveness of interventions [54]. Consequently, acceptability has been included in some national guidelines on conducting process evaluation in complex interventions [55]. However, the definition and operationalization of acceptability have been incoherent in guidelines and interventions, which has hampered its assessment and led to a need for developing a more theoretically grounded framework [54]. Consequently, acceptability has been defined as “a multi-faceted construct, represented by seven component constructs: effective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs, and self-efficacy” [54].

Acceptability is the most evaluated implementation outcome in health care [52,56]. A systematic review [52] discovered 50 acceptability instruments, and another systematic review [53] found 32 measures used in behavioral health care settings. However, both reviews concluded that the psychometric strength of the instruments is poorly reported, and most instruments show low psychometric quality. Moreover, those with some psychometric evidence were lengthy, which raised concerns about their clinical feasibility [52].

In the MySteps trial, the acceptability of SMC was assessed retrospectively from the participants’ viewpoint as part of 6-month end-point measurements (Figure 2). The purpose was to examine the user experiences on the contents and delivery of SMC. At the time of the study design, no ready-made and valid instrument was found for the present purpose. Thus, previous literature was used adaptively in determining the constructs of acceptability [57,58]. The constructs selected were usefulness, ease of use, credibility, and satisfaction. In each construct, the participants were asked to assess the primary features of SMC, that is, accelerometer (not applicable in credibility), feedback views of the smartphone app, face-to-face counseling sessions, and telephone counseling contacts. The response alternatives were presented on a Likert scale from 1=not at all to 5=extremely. After each construct, the participants could tell in an open space what would have improved their experience.

Altogether, 39 participants of the MySteps trial were randomized into SMC after the baseline measurements. As each participant was to have 3 face-to-face counseling sessions and 4 telephone counseling contacts, the maximum number of face-to-face sessions and telephone contacts in SMC was 117 and 156, respectively.

Based on the participant-specific counseling cards, the number of realized face-to-face counseling sessions was 112 of 117 (96%). The second session was missing from 2 participants, and the third session was missing from 3 participants. The total time used for the 3 face-to-face sessions varied from 80 to 230 minutes, being on average 152 (SD 34) minutes. The mean time spent for 1 session was 51 (SD 11) minutes, ranging from 27 to 77 minutes.

The total number of realized telephone counseling contacts was 145 of 156 (93%). In total, 2 participants missed the first contact, and 3 participants missed the second, third, and fourth contacts. The total time used for the telephone counseling sessions varied from 72 to 175 minutes, being on average 109 (SD 30) minutes. The mean time spent for 1 contact was 27 (SD 8) minutes, ranging from 18 to 44 minutes.

At the beginning of the intervention, 3 (8%) participants failed to pair their accelerometer with the smartphone app (Figure 3). In the rest of the participants, the use of the devices declined moderately and steadily until intervention week 21 and dropped notably thereafter. The number of weekly use days was on average 219 (80% from the maximum of 273) during weeks 1‐8, 183 (67%) during weeks 9‐16, 136 (50%) during weeks 17‐24, and 46 (17%) during weeks 25‐26. As a result, the mean weekly accelerometer use was 54%.

In those succeeding in pairing the devices (n=36), the mean number of users was 33 (92%) during weeks 1‐8, 28 (78%) during weeks 9‐16, 22 (61%) during weeks 17‐24, and 8 (22%) during weeks 25‐26. Correspondingly, the mean number of high-engagement users was 31 (86%), 25 (69%), 20 (56%), and 7 (19%), and the mean number of nonusers was 3 (8%), 8 (22%), 14 (39%), and 28 (78%), respectively. As shown in Figure 3, the weekly number of moderate- and low-engagement participants remained quite stable throughout the intervention.

According to the GLMM analysis, the face-to-face counseling sessions and telephone contacts in weeks 4, 8, 12, and 16 seemed to affect the participants’ use of the smartphone app at least 3 days per week (moderate engagement) since the odds for lower engagement increased as the time elapsed from the contact. After the contacts, the odds ratios for using the smartphone app less than 3 days per week in the 3 following weeks increased and were 1.36 (95% CI 0.896-2.063; P=.15), 1.79 (95% CI 1.124-2.859; P=.01), and 2.49 (95% CI 1.511-4.111; P<.001).

A total of 34 participants in SMC (87% of the 39 participants randomized into SMC) provided information on the acceptability of SMC at the end of the MySteps trial at 6-month measurements. The background characteristics of the respondents in SMC at the end of the study did not differ from the baseline background characteristics of all participants: the majority were women (27/34, 79%), aged 55 years or older (28/34, 82%), and overweight or obese (30/34, 88%). Most participants were working (24/34, 71%) and perceived their health better or equal to their age-mates (27/34, 79%). However, at the end of the intervention, when the questionnaire on acceptability was addressed, the participants of SMC had more self-reported MVPA than all participants at baseline (mean 240 vs 139 minutes).

The construct-specific distributions (%) and means of the acceptability responses are shown in Table 4. Overall, it seems that the features of SMC were mostly acceptable to the participants. The lowest mean scores for acceptability were seen in the credibility of the feedback views and in the satisfaction related to the accelerometer.

The counseling procedure of the main arm of the MySteps trial was implemented as intended, regardless of the COVID-19–induced challenges to patients with T2D. One reason may be that the procedure mostly included telephone contacts. Only a few sessions or contacts were missed, which suggests that the number of sessions and contacts was feasible for the participants. The mean lengths of the individual sessions and contacts also seemed reasonable in clinical practice, although there was a large variation. The large variation probably illustrates the demand for adjusting the length of the sessions and contacts according to individual needs, which may not always be executable in real-life health care settings. The findings on fidelity and length of the sessions are in line with the most comparable study to the present one, where 83% of the patients (63% with T2D and 37% with chronic obstructive pulmonary disease) received the intended intervention of 4 sessions and where most of the sessions exceeded the intended 20 minutes [65].

In the use of accelerometer and smartphone app, the number of weekly use days and number of weekly users declined during the intervention from 80% to 17% and from 92% to 22%, respectively. Similar descending trends have been found in other mHealth interventions among patients with diabetes, which show approximately 40% user rates at the end of the study [66,67]. One exception is the study by Verwey et al [65], where 88% of the patients wore the accelerometer and smartphone app until the end of the 25-week intervention period. The progressively lowering use has been recognized also in interventions involving adults without chronic diseases, although their attrition rates have been more moderate [21]. In any case, the engagement plays an important role in mHealth interventions since it has been shown that it associates with intervention effectiveness [68].

In this study, one explanation for the deep decline in use in weeks 23‐26 may be that the accelerometer battery faded, and the participants, despite the instructions, did not replace the battery since the intervention was already coming to its end. If the weeks 23 through 26 were eliminated, the average user rate for the last weeks would be 64%. In any case, the attrition rate in mHealth studies seems generally quite high and is worth considering in future studies [68].

Interestingly, while the proportion of participants with a high engagement declined from 86% to 19%, the proportion of participants with moderate and low engagement remained practically the same, and the proportion of nonusers increased from 8% to 78%. This indicates that at least in the present intervention, the participants rather stopped using the devices than just reduced their use. If this discovery applies more widely, its impact on successful implementation needs to be considered in future mHealth interventions.

The relevance of individual contacts in facilitating the use of smartphone apps is seldom examined. In this study, the findings of the GLMM analysis emphasized their importance. The discovery is somewhat contradictory to a recent systematic review, which indicates that, in general, feedback on self-monitoring might not prevent attrition from self-monitoring interventions [69]. The more comparable studies, nonetheless, show that consultations are much appreciated among patients with T2D and not substitutable with an app [39].

The fidelity evaluation would have gained from more extensive cloud-based information on the use of the accelerometer and smartphone app. Unfortunately, this information could not have been extracted from the cloud data. Earlier studies have identified similar drawbacks and highlighted the importance of close collaboration with the technology team to develop tools for collecting fidelity information from raw digital data [70].

Overall, previous research shows that the delivery of PA interventions in patients with T2D is seldom evaluated at a practical level, and usually, the process data are limitedly and inconsistently reported [36]. This is regardless of the 2004 recommendation by the National Institute of Health Behavior Change Consortium to facilitate fidelity evaluation, especially in health behavior change studies [51].

With the increasing number of web-based and mHealth interventions aiming at health behavior change, the need for improving and standardizing evaluation reports has also been identified [71]. As a result, the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) guideline on reporting eHealth and mHealth interventions has been published [71]. The guideline extends various checklist items of the CONSORT (Consolidated Standards of Reporting Trials) statement for randomized controlled studies and highly recommends, for example, reporting metrics of use (checklist item 17a-i) and technological problems (checklist item 19) as intervention outcomes (Checklist 1) [71]. Nevertheless, the systematic review by Blackman et al [72] shows that external validity is poorly reported in mHealth interventions promoting PA and provides specific recommendations for more comprehensive reporting including engagement.

The findings on the acceptability of SMC are encouraging, given that the majority of participants were women, aged 55 years or older, overweight, and physically not so active. On the other hand, the present results compare to previous studies, which show that adults with T2D appreciate self-monitoring and health professional facilitation as features of digital health intervention [39,40] and that age is not necessarily a barrier to using smartphone apps [66].

However, there were indications that some improvements would be needed to strengthen the credibility of the smartphone views and satisfaction related to the accelerometer. In the open spaces of the questionnaire, the respondents brought up similar concerns indicated in the few previous studies, including the inability of the accelerometer to account for all PA such as cycling and gymnasium activities [73] and technical problems related to a wireless connection [65,74].

The findings are not comparable with previous studies because acceptability has not been evaluated in research using mobile-based technology to promote health behavior change in people with T2D [38]. In many studies, the goal of the intervention has been to improve glycemic control through more comprehensive self-management apps [29,39]. These studies show variable satisfaction ratings and multiple usability problems such as multistep tasks, limited functionality and interaction, and difficult system navigation [29]. Usability problems, in turn, have been suggested to cause the experience of failure and additional burden to the participants, which can subsequently lead to so-called digital distress in some users [39].

A larger sample size would have brought more variance to the responses and may have shown more clearly the developmental demands related to the features of SMC. Moreover, the inclusion of the health providers’ views would have deepened the understanding of acceptability, as it has been found that the technical problems, for example, are easy for the participants to manifest during counseling [73]. Providers’ perspectives may have also brought out valuable information, especially about the applicability of the digital features of the intervention to everyday practice [37]. Previous studies have identified multiple influences on the use of digital tools in primary care [75]. In addition, using a measure with strong psychometric qualities would have improved the validity of acceptability findings. However, until now, only a few such measures have been identified in health care [53,56]. The unvalidated measures and specific features of the intervention may limit the generalization of the findings on acceptability.