This study addressed the first 2 stages of the Integrate, Design, Assess, and Share (IDEAS) framework in designing digital health interventions to promote changes in health behaviors [43]. The IDEAS framework allowed the study team to systematically assess the study needs and incorporate evidence-based strategies from the literature and our previous work in designing a user-friendly, multipurpose app delivering the 50K4Life intervention to public school employees [41,44] using the Pathverse platform [45]. Pathverse is a no-code mobile app builder that can customize app features and functions to deliver and monitor intervention content from the researcher portal with full autonomy. Specifically, Pathverse offers the features and functions to deliver program content (eg, program announcement, access to health lessons, videos, and resources), embed behavioral change tools (eg, self-monitoring and goal setting), gather input from participants (eg, quizzes and surveys), communicate with participants (push notifications and community forum), and collect use data of features and functions for monitoring and evaluation. The app is available in English and Spanish for iOS in the Apple App Store and for Android in the Google Play Store.

In stage 1, Integrate, the multidisciplinary research team collaborated with the 50K4Life study community advisory board comprised of representatives of school administration, instruction and curriculum and school wellness program personnel, and stakeholders from communities to discover and define the background target users and create a profile, that is, public school employees of diverse socioeconomic background, and to specify the targeted behavioral outcomes of the 50K4Life intervention that will be accommodated in the app. The research team shared the delivery strategies and expected outcomes of the 50K4Life intervention and available features and functionalities of the Pathverse platform with community advisory board members, and received initial input on the layout of the 50K4Life app to meet the study’s needs in a workplace setting. In addition, the research team worked closely with Pathverse engineers to understand the app’s capabilities, delivering the intervention in alignment with the socioecological model [46], the behavioral change theory guiding the design of the 50K4Life intervention, and to assure appropriate targeting of individual, interpersonal, and organizational and environmental factors influencing walking behavior at workplace and home [44].

In stage 2, Design, the study team translated the information gathered in stage 1 to develop a prototype app, including features and functionalities to deliver various aspects of the walking challenge (Table 1). The prototype app was evaluated in a beta test with a small group of public school employees who were asked to perform a series of tasks that provided data on access, use, and satisfaction of features and functions in the app. The participants also provided feedback on the user experience and recommendations to improve user experience and optimize app features and functions. Finally, the study team incorporated the results from the beta test and built a minimum viable app for a pilot on its usability and feasibility in stage 3 of IDEAS, Assess. Results of stage 3, the pilot study, were not reported in this manuscript.

Participants were recruited from 2 public school districts in El Paso County, Texas. El Paso residents are disproportionately burdened by health conditions associated with inadequate physical activity engagement [47]. Currently, 1 in 3 El Paso residents did not engage in any physical activity in the past month [48]. This population experiences earlier onset and poorer control of cardiometabolic conditions such as diabetes, hypertension, and fatty liver disease and lives more years with disabilities requiring assistance from family members or skilled nursing care [47]. Regular engagement in physical activity, including walking, can delay and possibly prevent the sequelae of disease, disability, and poor quality of life. Nevertheless, innovative physical activity programs widely used in other regions of the United States have not been implemented in this region despite their evidence-based effectiveness.

During the app-based walking challenge, participants were asked to walk as many steps as they could each week for 2 weeks. Participants were able to monitor their personal, teammates’ and overall team’s step totals through the app’s leaderboard. The team with the most steps after 2 weeks was deemed the winner of the challenge. The winning team received the 50K4Life program and school district merchandise.

Participants were recruited from the administration departments of 2 school districts. The administrative staff were tied to school instruction or services (eg, physical education, cafeteria, and custodial) and at some point in their career had worked on a school campus and therefore had insight into the feasibility and usability for school-based participants in the full trial. Inclusion criteria were employment in the administrative offices, aged ≥18 years, not pregnant, and owner of a smartphone. Individuals were excluded if they had a mobility issue that prevented them from fully participating in a 2-week-long walking challenge. Participants were identified by a designated school liaison, and contact information was given to the study staff. Study staff then contacted potential participants via email, inviting them to participate in the study. Once a potential participant was identified, an in-person appointment was arranged where informed consent and study orientation were conducted. A total of 12 participants were enrolled in the study.

The beta test procedure included the following: (1) participants were tasked with activities related to using the 50K4Life app, such as finding information, tracking steps, completing educational modules, and other activities related to the study (Table 1); (2) participants took part in a 2-week-long walking challenge and tracked their progress through the 50K4Life app; (3) participants were sent a link to a survey in Research Electronic Data Capture (REDCap; Vanderbilt University) at the end of the 2 week walking challenge; and (4) after completing the walking challenge, participants took part in a debriefing session.

The study protocol was reviewed and approved by the institutional review board at the University of Texas at El Paso (protocol number 2089761). Written informed consent was obtained from all participants in either English or Spanish, depending on each participant’s preferred language. Participants were provided with a copy of the consent form, which included detailed information about the study, data collection, and their rights as participants. Data collection training for all study staff emphasized the importance of protecting participants’ privacy and maintaining confidentiality of data. Data collection, management, and quality assurance for this study were facilitated using REDCap electronic data capture tools hosted at the University of Texas at El Paso. REDCap is a Health Insurance Portability and Accountability Act—a secure data management system that ensures effective data collection, storage, retrieval, and quality control. As this was a walking challenge where participants were able to view each other’s steps and identities, an effort was made to provide options for participants to take part without using names or other identifiers. First, participants were given the option to create aliases when they registered with the 50K4Life app. The alias appeared on the leaderboard instead of their name or email address. The alias option was also offered during the debriefing sessions to protect confidentiality. Additionally, unique identification numbers were assigned to each participant and used to link all data. Once transcripts from debriefing sessions were cleaned and verified, aliases were removed, and initials were used instead. Participants were sent a US $25 gift card upon completing the debriefing session.

The primary outcomes of the study were app functionality, usability, and acceptability. Data were collected through surveys, interviews during debriefing sessions, and use data of the 50K4Life app.

Participant survey responses were tabulated to check for validity and missingness. Means and SDs for continuous variables, and frequency and percentage for categorical variables were calculated.

Transcriptions from the debriefing sessions were then proofread for accuracy, formatting, and clarity. The study staff identified and extracted the main discussion points from each debriefing session. Themes were organized inductively into likes, dislikes, and suggestions for improvement. Likes were 50K4Life app aspects that participants found appealing, helpful, or motivating. Dislikes were areas of the app that caused frustration or dissatisfaction among users. Finally, suggestions for improvement were ideas and recommendations for enhancing the app’s functionality, user interface, or overall experience.

The beta test was conducted with 12 participants with diverse administrative roles and experiences, as well as an understanding of employees in public schools. A total of 3 participants withdrew from the walking challenge before completing it due to scheduling conflicts (n=2) and technical difficulties (n=1). A total of 7 participants used a study-issued Fitbit tracker, while the rest used either a personally owned Apple Watch or a Samsung Galaxy Watch. All participants completed the 2-week walking challenge and posttest assessment. Table 2 shows the participant demographics.

Participants had mixed responses regarding the acceptability of features and functions in their experience with the 50K4Life app (Table 3). Acceptability was high for overall design and options in the app, access to the app, the app’s effect on battery life, the smoothness and operability of the app’s functions, the layout of features and functions in the app, and the app’s visual appearance (color scheme and fonts). The participants reported negative experiences navigating the app, viewing or accessing information display (step count updates), syncing the app with tracker and/or phone (requiring third-party apps to sync data), using certain features and functions to complete tasks, and enjoying the app’s features (lacking features engaging and motivating participants).

Table 4 shows the participant’s satisfaction with the 50K4Life app. With scores >3 on 11 questions, participants were satisfied with the app’s features and functions. The lowest scores were associated with navigation, viewing step counts, and app layout.

Table 5 shows the usability data from participants’ access to functions and content in the 50K4Life app. The participants were instructed to use the features and functions to receive study-related information (eg, instructions for walking challenge and notifications from the study), access educational content or resources (eg, health education lessons), or perform a function (eg, setting walking goals and completing a survey) during the walking challenge. Except for using the survey function, the usability rate was ≥75%, indicating participants met the expected level of engagement (Table 1). The Pathverse platform cannot track if participants viewed the notifications sent by the 50K4Life app.

Overall, the participants reported that the app was easy to navigate and access, and the layout was organized and simple. The participants found that the leaderboard created a competitive environment that motivated and engaged them in the walking challenge. In addition, participants liked the step tracking feature, which used synced step counts from activity trackers, and found it effective at keeping them focused on the walking challenge. By contrast, participants reported difficulties and concerns that negatively affected their experience with the app during the beta test. In Table 6, we organized the issues and challenges reported by the participants into 5 problem areas and summarized the suggestions from participants to address these issues in the 50K4Life app. A quote from a participant is also included for each theme. These themes and suggestions provided additional guidance for refining the 50K4Life app.

Figure 1 displays the daily step counts of individual participants who synchronized their steps with the 50K4Life app over the 2-week duration of the walking challenge. It should be noted that most participants had daily step counts <7000 steps/day.

User experience and preference can influence participants’ engagement of app-delivered physical activity promotion programs [13,21]. The purpose of this beta test in the 50K4Life study was to assess the usability and acceptability of the 50K4Life app designed to deliver an evidence-based walking challenge to public school employees [44,49]. The 2 main research questions guiding our work were whether the app had the features and functions to deliver the intervention as planned, and how feasible and acceptable the app was to the participants. As anticipated, participants appreciated certain aspects of the app, while identifying other aspects they wanted improved, despite efforts to follow evidence-based mHealth promotion recommendations [49,50]. Participants were generally satisfied with the app and felt it was usable. However, participants found the app unengaging and unattractive and had issues with syncing and setup. Overall, the findings are consistent with published studies testing the usability and acceptability of mHealth apps in community-based health promotion programs [41,51].

Overall, participants demonstrated high engagement with the 50K4Life app features, though their satisfaction with some of them was mixed. In this study, participants appreciated the app’s simplicity and ease of use. Reminders were viewed favorably. While being able to sync with multiple devices was viewed positively, participants voiced difficulties in synchronization of step counts and a need for additional support to use the app. Most synchronization difficulties reported by participants stemmed from third-party integrations (Fitbit, Google Fit, and Apple Health), where correct user setup (permissions, account linking, and device syncing) is required. Misconfigurations on the participants’ side are a well-documented source of error in mHealth studies [52]. At the time of the beta test, our monitoring was limited to client-side reports, and we did not systematically capture application programming interface success or failure rates on the server. This limited our ability to provide independent verification of the Pathverse platform’s backend reliability for the study period. In the future, we plan to implement server-side monitoring to track application programming interface request success rates, error codes, and latency. This will allow us in future studies to differentiate more clearly between user-side misconfigurations, third-party service constraints, and Pathverse platform reliability. In addition, we will provide a user tutorial on the features and functions to familiarize the participants with the features and functions in the app as suggested by the participants [52].

The delivery of multiple components of the beta test was generally found to be feasible. Participants engaged in the walking challenge, tracking and syncing their own steps and those of their teammates. This enabled them to monitor their teammates and other school districts’ teams on the leaderboard. However, participants’ engagement in the interactive educational sessions was low due to a short window. Notification reminders of upcoming lessons would have increased the open rates of the lessons [1].

The beta test and evaluation generated useful feedback for improving the app for subsequent implementations. This beta test is part of a larger study designed to increase walking among school employees. Recruiting participants from local school district administrative offices who had a first-hand understanding of school employees’ daily life reflects the community-based approach to the overall 50K4Life study. Moreover, input from the community advisory board and participants in the beta test helped shape the app’s content and tailor the design to the target community’s needs and interests.

Despite having extensive preparation meetings leading up to the beta test, it was evident that unforeseen technicalities hindered the study’s execution as planned [1]. Consequently, during the beta test, we encountered several challenges. During implementation, we found that while Pathverse provided a user-friendly platform for the 50K4Life app, it did not allow communications with the participants regarding their engagement in the beta test activities. For instance, we required the ability to send automated messages to participants based on their progress in the walking challenge, that is, did they meet their daily goal. This problem can be resolved by creating an algorithm-based feature to track daily step counts in the app and provide updates on goal achievement in a timely manner [53]. Participant feedback will allow us to work with the Pathverse development team to develop new features and functions that will improve user experience and engagement in the intervention.

User experience testing is intended to be an iterative process, sometimes with multiple rounds of testing and refinement [53]. A major strength of the app is that the content is based on findings of ongoing walking challenges in the El Paso community, providing a theory-driven motivational approach with tailoring for the setting [37]. Additionally, the beta test followed the IDEAS framework to garner acceptability and usability from a sample similar to the intended 50K4Life population, informing the design and testing of the app to meet future users’ needs [43]. The limitations of the study included a short period that prevented an in-depth evaluation of the participant experience and the app’s features and functions. A second limitation is that there were no beta test participants who selected the Spanish assessments. It is possible that participants who prefer Spanish may experience the app features differently, and that this input was not captured as part of the beta test. Issues in synchronization with the 50K4Life app was another limitation that might have influenced the participant’s engagement in the pilot study. Finally, some of the issues identified were beyond the scope of the beta test. However, we will take the participants’ suggestions in the refinement of the app and evaluate their impact in future implementations of the app [54]. Finally, we did not conduct a cost or time analysis at this phase of the study. Having that information would improve our understanding of the implementation feasibility of the 50K4Life app.

The next logical step in the development of the 50K4Life app is to address the constructive feedback provided by beta test participants and then evaluate the app again in the pilot study. This is part of the planned pilot study where additional participant comments will be used to further refine the user experience. In this future study, we will investigate cultural or linguistic barriers to accessing the app for Spanish-speaking staff. Additionally, we will be conducting a cost-effectiveness evaluation to determine operational costs that would have implications for real-world adoption and scalability. This beta study represents the first step in implementation, and participant feedback will inform future revisions to the app preceding the planned full-scale trial [55].

The use of the IDEAS framework as a community-based participatory approach in the design of the 50K4Life app contributed to the enhancement of the functionality and acceptability in target communities with high chronic disease burden and increased the potential of success in “real-world” use. Through the 50K4Life app beta test, we gained valuable knowledge on barriers that need to be addressed so that we may optimize our use in the full-scale trial. Lessons learned from the beta test can inform future dissemination efforts of mHealth interventions in different settings for health promotion.