Original Paper
Abstract
Background: The gap between research and its practical application in community settings limits its impact on public health. Closing this gap has the potential to improve the well-being of underserved groups, such as children with disabilities. Mobile health has the potential to improve access to community resources and support for underserved populations, thereby encouraging improved health behaviors.
Objective: In this feasibility pilot study, we describe the development of the mobile app Jooay. Jooay was developed in partnership with stakeholders to facilitate access to leisure and physical activity community programs for children and youth with disabilities. We also reflect on the lessons learned throughout the implementation process that are relevant for improving the health behaviors of children with disabilities.
Methods: We used a participatory action research approach to develop the app. We also administered a survey to current Jooay users and analyzed various app usage indicators to explore use patterns, user feedback, and preferences. Finally, we critically appraised the implementation process through a best practices for implementation research framework.
Results: We developed a product that responds to users’ identified need to find information and follows accessibility and user-centered design standards. The analysis of usage data revealed that access to the Jooay app is concentrated in urban areas. Perceptions, attitudes, and information needs varied according to the type of user. The use of the mobile app changed over time, and usage decreased after the app was downloaded, indicating a need for the sustained engagement of app users. Users found value in the ability to identify activities that they would not otherwise know about. However, app use alone was not sufficient to improve participation. Although the app was developed based on users’ active input in multiple iterations, we encountered challenges with survey recruitment and attrition, suggesting the need for more seamless and engaging means for collecting data within this population.
Conclusions: Interactions between users and the app can sustain user engagement and behavior change. We will improve the app’s next iterations by using the information gained from this study to conduct a larger study to assess the relationship among social and material deprivation, urban design, and access to inclusive and adaptive leisure programs. This study will inform the improvement of app listings to improve the use of Jooay by different user groups and promote health through mobile apps for marginalized groups.
doi:10.2196/23877
Keywords
Introduction
Background
Implementation research involves studying research uptake and its effect on the outcomes of multiple stakeholders [
, ]. In medicine and public health, a significant gap remains between research knowledge and action, limiting the direct impact of research on health [ ]. Closing this gap requires the consideration of multiple contextual variables, consideration of technology use, and other strategies to improve health behaviors [ ].Innovative strategies for facilitating changes in health behavior are particularly important for underserved communities and populations. People with disabilities comprise approximately 15% of the global population and are at risk of poor health outcomes [
]. Children and youth with disabilities face challenges in accessing health services and health-promoting activities [ ]. They rarely meet the recommended standards for physical activity and have lower physical activity levels than their peers [ - ]. They also have a higher prevalence of noncommunicable diseases, such as obesity, diabetes mellitus, and coronary artery disease [ ], and are disproportionately affected by environmental, socioeconomic, and interpersonal barriers to healthy lifestyles [ ]. Although parents or other caregivers (hereafter caregivers) value their children’s participation in physical activities, they face multiple participation challenges, such as inadequate access to adapted programs and inclusive settings [ - ].Health promotion initiatives can improve outcomes for children living with a diverse range of social, emotional, and behavioral disabilities [
- ]. Contextual factors that can serve as participation barriers or facilitators for children with disabilities include information about activities, the cost of activities, the accessibility of facilities, and the presence of trained staff and support [ , - ]. The alignment of needs of people with disabilities with effective health promotion initiatives can foster better health outcomes, provide a sense of empowerment, reduce health disparities, and improve overall individual quality of life and community well-being [ ].Mobile health (mHealth) is gaining primacy for the creation of targeted, accessible, and context-appropriate health promotion solutions. mHealth tools include various devices, software, and solutions that use mobile phones to improve health [
]. Potential benefits include time savings, convenience, and improved access to underserved populations [ - ]. mHealth tools have also improved health behaviors among young people and in chronic disease management [ , , ].Although there have been several pilot studies on mHealth interventions, knowledge gaps remain regarding the appropriate development and use of mHealth to promote health equitably [
]. Preliminary studies suggest that users of mHealth are younger, more educated, have better health, and belong to a higher income group than nonusers [ ]. Despite its promise in improving health, mHealth may actually exacerbate health disparities if underserved populations do not have access to digital tools and the ability to develop literacy in using them.In addition, research on mHealth has not yet sufficiently demonstrated efficacy, effectiveness, user engagement, effective scale-up, and competitive value [
, ]. Many mHealth efforts have been inadequately designed for implementation and evaluation [ ]. Evaluations that do exist do not prioritize data disaggregation, limiting considerations of equity and impact for marginalized groups [ , ].Objective
In this study, we explore the feasibility of mHealth to improve access to information on community-based inclusive leisure activities for children with disabilities. Our specific objectives are to (1) describe the development process of the mHealth solution and (2) identify use patterns and user preferences. The secondary objective is to establish the feasibility of using a mobile app to test behavior change and to pilot test data collection through app analytics and users.
Understanding the development, implementation, and uptake of mHealth can facilitate the design and use of future technologies for health promotion in high-risk groups. The knowledge gained will also inform the scale-up of the app and, more broadly, the field of implementation research.
Methods
Overview
In this pilot feasibility study [
], we asked the following question: can a mobile app be developed in collaboration with stakeholders and used to promote health behavior changes in children with disabilities, and if so, how? (feasibility component). We also conducted a pilot study to assess the extent to which the app supports health behavior change (pilot component). In this paper, we describe the app development process and the results of a small-scale survey with a subset of app users.We adopted a hybrid implementation research design [
], whereby the intervention or solution is developed and tested concurrently instead of using the traditional approach, in which development is conducted before the intervention is tested with the population comes first and then is tested out in the population followed by testing. The value of hybrid designs resides in the possibility of cocreating knowledge while simultaneously incorporating and testing intervention improvements. Multiple iterations required in a technology development project make the hybrid design an optimal approach to effectively test and implement user-responsive mHealth-based interventions.Ethical approval for this study was obtained from the McGill University Institutional Ethics Review Board as well as the Centre for Interdisciplinary Research in Rehabilitation.
Theoretical Frameworks
Several theoretical frameworks informed our approach and study objectives.
illustrates the integration of these frameworks with the project objectives and procedures.The overall approach to app development and evaluation was based on the participatory action research (PAR) framework [
]. PAR aims to share power between researchers and those being researched. Our approach was thus reflective and iterative and involved multiple stakeholders throughout the research process, from the elaboration of the initial research question to app development and evaluation.The overarching objective and the development process (specific objective 1) were informed by the best practices for implementation research prescribed by Peters et al [
]. Specifically, we used the following steps: (1) identify the audience and how they will use the research; (2) clearly describe the program, practice, or policy being implemented; (3) examine the implementation strategy thoroughly; (4) describe the real-world context and sample population clearly; and (5) appropriately consider outcome variables of the implementation. The last 2 steps consider specific context variables and appropriately consider changes in contextual factors over time, unintended consequences, and system complexity, which will be considered in future studies.To expand on step 5 (appropriately considering outcome variables of the implementation), we adopted the Health Behavior Framework [
]. This framework includes a detailed consideration of outcome variables related to the expected implementation (ie, use of mHealth technology, the Jooay app). The elements of the framework informed the key aspects to consider while identifying users’ patterns and preferences (specific objective 2). Determination of the feasibility of use requires a comprehensive understanding of the variables involved in the health behavior change proposed by this framework.mHealth Tool Development
Overview
Our pilot intervention tool is Jooay, a free mobile app that aggregates information on leisure activities for children and youth with disabilities aged 6-21 years in Canada. The Jooay app was launched in the spring of 2015 on iOS and other web platforms. The version used for this pilot study listed approximately 1000 activities distributed in 5 of the 10 Canadian provinces.
Stakeholder Forums
First, we organized 4 stakeholder forums across the Canadian provinces. Participants were a purposeful sample of youth with disabilities, caregivers of children with disabilities, health and education providers, policy makers, and community organization leaders. They were invited by pediatric rehabilitation center collaborators and city leisure departments in Montreal, Toronto, Calgary, and Vancouver.
Each forum was a 1-day event with the following objectives: (1) to present the current research evidence on determinants of leisure participation for children and youth with disabilities and (2) to identify strategies to improve access to leisure opportunities for children and youth with disabilities across Canada.
Using a business canvas model [
], participants were grouped according to the stakeholder group to which they belonged (health care providers, caregivers, youth with disabilities, policy makers, and grassroots organizations) and were asked to consider barriers to access to leisure participation. Subsequently, they were grouped into mixed groups representing different stakeholders to discuss implementation solutions to promote the participation of children with disabilities.Each participant was invited to write their selected top solution on a card. Instructions were that the top solution depicted in the card should be actionable, be feasible, and have an impact on the participation levels of children with disabilities. Participants then engaged in a prioritization exercise in which they exchanged cards and, in pairs, compared and ranked 2 ideas. The solution that ranked higher at the end of 5 rounds of paired ranking was considered the one that, according to the forum participants, would yield the highest impact on participation in leisure for children with disabilities. The solution ranked as the most feasible and promising solution to overcome systems barriers and promote participation in leisure across the 4 forums was “[a]n electronic list of inclusive and adapted activities.” Participants in each forum volunteered to form a working group to support the development of the idea.
Advisory Group
The Jooay app was developed in 2 phases: (1) prototype development and (2) test version development.
A convenience sample of 5 users (2 caregivers, 1 physical educator, and 2 occupational therapists [OTs]) and several research team members (including community organization leaders and physical and recreational therapists) provided input into the user interface development, user experience development, and final test versions during both phases.
A larger group of stakeholders comprising clinicians (OTs and physical therapists), physical educators, caregivers of children with disabilities, and representatives of community organizations constituted the user partners who provided input during the development phase of the app. These stakeholders provided insights into the best features and types of information to be listed and were involved in the conception of the app, the testing of multiple versions, and the development of the research protocols to address the feasibility and pilot testing of the app. The stakeholders were chosen based on the following considerations: (1) willingness to participate; (2) direct experience working or advocating for children with disabilities; and (3) representation across diverse areas of involvement that included pediatric rehabilitation centers, school boards, community organizations, newsletters of organizations, and networks related to adapted leisure and addressing caregivers of children with disabilities.
A dynamic protocol for testing and responses was established using a collaboration platform and the research team made connections between the developers and user testers.
Prototyping and Test Versions Development
User Experience
The mobile app was initially developed as a prototype during a hackathon (weekend event grouping developers, user experience designers, programmers, and project managers). The first test version was developed for iOS, Android, and the web. It was made available to users free of charge through regular channels (App Store, Google Play, and website).
Development
A partnership with an external mobile app developer was necessary to secure further development, and multiple funding sources were required to attain industry standards. Suggestions that arose from user partners and technology developers resulted in the creation of 3 native platforms for iOS, Android, and the web.
Core technology developments included mandatory implementation of full accessibility protocols for mobile and web platforms (including voice-over, voice control, color contrast, and easy access—a feature available in the iOS and Android accessibility protocols where buttons and number of clicks to action are reduced). An accessibility consultant was hired to test and assess these features during development.
Content
The participants provided important information on preferences for the type of information displayed for each activity listed. The selection of included domains had an impact on development cost; therefore, there was an assessment of the most relevant information components to be retained in the app.
The final domains reflected stakeholders’ preferences for information and included activity description, types of equipment required, type of disability (eg, physical, intellectual, or those classified as all are welcome), cost, and time frame. The same decision algorithm was used to create filters within the map search as well as to collect basic demographic information from users when they registered to use the app.
Stakeholders also suggested a list of other resources to be listed on Jooay, such as reference links to other types of supports toward leisure participation, such as respite care and support groups for youth and caregivers, research related to leisure, and the Jooay web-based Facebook community. Our user partners supported the development of the survey questions by reviewing the initial questions and providing feedback on how to phrase the questions, question content, and structure (eg, options for participants to suggest other fields or useful resources that should be added to the app). They also supported survey distribution to a larger sample of community organizations and rehabilitation centers, in addition to the list of app users, and acted as champions to disseminate the survey and the app as a product to other users.
Activities in the app were initially populated on the basis of pre-existing lists of adapted and inclusive leisure activities from pediatric rehabilitation centers in the targeted provinces and schools serving children with disabilities in 6 provincial capitals across Canada. Using the key terms identified on the websites of these organizations, the research team searched for additional activities and continued populating the app database.
Data Collection and Sample
This pilot project used 2 sources of data: (1) analytics of the Jooay app users and (2) electronic surveys sent to registered participants.
Participants
Through a preliminary analysis of app analytics data from approximately 600 users, we determined that approximately half were health or education service providers and the other half were caregivers of children with disabilities. No further sociodemographic information was required from the users upon registration. Furthermore, 2 different surveys were subsequently developed to target service providers and caregivers. This pilot study was conducted with all app users who had registered with the app using their email. Registration was not a mandatory requirement to gain access to information on Jooay.
A total of 273 of approximately 600 user emails were initially available. Additional users who downloaded the app and registered during the 6-month data collection period also received an invitation to participate. A brief explanation of the study and survey links were also posted through social media channels, specifically through parent support groups and the Jooay page on Facebook, asking app users who might not have registered emails to complete the survey. Study knowledge brokers in pediatric rehabilitation centers, school boards, community organizations, newsletters of organizations, and networks related to adapted leisure and addressing caregivers of children with disabilities also shared information about the app and survey. Additional participants in the regions where the activities were published on Jooay at the time of the study, namely, the 6 provinces—Alberta, British Columbia, Quebec, Nova Scotia, Saskatchewan, and Ontario—were targeted in addition to existing users of the app.
Participants were required to own a smartphone, have access to the internet, and understand English or French. Users aged <13 years were excluded from the survey, given that 13 was the established minimum age to own social media accounts, such as Facebook and Twitter, and the legal age required to download Jooay from the App Store. Assent was required for participants aged <18 years.
Procedures
Potential participants received an email with a brief explanation of the study and a link to a web-based survey on REDCap (Research Electronic Data Capture; developed by Vanderbilt University) hosted in a research database. Participants were prompted to provide consent before the initiation of the survey. A total of 5 invitations were sent by email within a 2-week interval.
Confidentiality and anonymity of survey responses were ensured by deidentifying survey responses. Participant emails were only used to prompt their participation in the study and were not associated with their answers. A 2-step password-protected REDCap account accessible only to the survey team was used to ensure data privacy. The survey questions analyzed as part of this pilot study are provided in
.Survey questions analyzed as part of this study.
Questions about the app and app use
1. Rank the relevance of the current existing features of the app (are these sections important in helping you find an activity to pursue?)
2. Was information in the following sections helpful in finding activities?
- cost
- type of activity
- description of activity
- type of disability
- location
- age range
- schedule
- season
- reviews or ratings
- other links and related information
Questions with "agree," "neutral," or "disagree" responses
1. The app is easy to use
2. The app has a comprehensive list of existing activities
3. The information on the app is accurate
How did you learn about Jooay?
1. Social media
2. Health or education professional
3. Other sources (which one?)
Open-ended questions
1. What feature you would like to see in the app that is currently not available?
2. What are main issues of the app?
3. Do you intend to uninstall the app? (yes or no)
If yes, explain why (open ended)
Questions about health behavior
1. The person for whom the app information is being used (child, client, or student) has engaged in regular leisure activities before using the app? (yes or no)
2. My child, client, or student participation in leisure has increased because of information
found in the app? (not at all, a little-moderately, or a lot)
If participation improved, explain why (open ended)
3. The activities in the app fits my, my child’s, or my client’s needs (yes—moderately—no)
Sociodemographic questions
1. Your age (respondent)
2. Age of the person with disabilities for whom you’re using the app info
3. Province of residence
Analysis
After the REDCap surveys were distributed [
], responses were exported into SPSS (IBM Inc), and responses from the English and French surveys were merged. Descriptive analyses were conducted for participant characteristics, app usage patterns, and preferences for app features and information. We also explored the associations between the sociodemographic characteristics of caregivers (age and gender), with perceptions of app features, app use patterns, and other characteristics.The question of power calculation is often a challenge in assessing the impact of mHealth tools on health outcomes [
], given the challenges of generating a significant sample size and reducing attrition. This pilot study also aimed to set parameters for sample size calculations for future research using this mHealth tool.Results
The results of the development process and feasibility testing are presented in the context of the 2 guiding theoretical frameworks: the Implementation Framework and the Health Behavior Framework. We identify which domains of these frameworks are addressed in each of the following sections.
Development Process of the mHealth Solution
Implementation Framework: Identifying the audience and how they will use the research
Health Behavior Framework: Health policy environment, health or community care system, community capacity and engagement, and social advocacy
More than 200 stakeholders representing diverse groups participated in the development process of the mHealth solution. An average of 50 participants in each of the 4 stakeholder forums identified that a mobile listing of leisure activities was the most desirable and feasible solution to promote participation in leisure activities for children with disabilities.
describes the main steps of development, the strategies adopted, and the main outputs in each step. shows screenshots of the final app that was developed.
Steps of development | Strategy | Output |
Needs assessment |
|
|
Design and prototyping |
|
|
User interface |
|
|
User experience |
|
|
Test versions |
|
|
Pilot version |
|
|
Stakeholders in the forums represented a range of health and policy environments, communities, and health care systems. They presented the key factors for the use of the app in the health and community environments (eg, information that should be added to make the mHealth solution relevant). Stakeholders in the forums and in the advisory group also contributed the listings that they currently had in the municipal, local listings of activities to the database and engaged in social advocacy.
User Patterns and Preferences
Implementation Framework: Examining the implementation strategy thoroughly, describing the sample population clearly
Health Behavior Framework individual variables: knowledge, health beliefs, social norms and supports, cultural factors and beliefs, barriers and supports, and structural factors
A total of 239 participants received the survey. The response rate was 38.9% (93/239).
includes participant characteristics. Twenty-four participants responded to the question about age of the person for whom they were seeking activities. From those 62% (15/24) of participants indicated that they were seeking information for children aged 4-12 years.Sample characteristics | Participant, n (%) | ||
Caregivers of children and youth with disabilities | 38 (41) | ||
Health care or education service providers | 20 (22) | ||
Youth with disabilities | 4 (4) | ||
Other stakeholder groups | 31 (67) | ||
Living in urban area | |||
Caregivers | 31 (82) | ||
Service providers | 18 (92) | ||
Gender (female) | |||
Caregivers | 34 (89) | ||
Service providers | 15 (76) |
Participants were asked to rank the relevance of app features. Participants reported that the most useful information was age range and location, followed by the activity type and description of activity. Most participants found that information in all sections of the app was helpful; sections that had less information, such as reviews and ratings (which is expected because the app is new and not many reviews had been done yet), were perceived as less helpful (
). The links provided in the settings sections included research summaries and were among the least useful sections, along with activity schedule, probably due to the frequent change in schedules, making the information not accurate.Participants were asked to indicate (open-ended questions) the features they would like to see added to the app. A total of 34 participants responded to this question, and their responses were categorized as follows: development of a web-based community or forum (13/34, 38%), a means to track their participation in leisure and physical activities (8/34, 23%), positive prompts for action (7/34, 20%), and a points or rewards system to incentivize participation (6/34, 18%).
When asked about their concerns with the app, the main negative feedback was related to the insufficiency of listings (9/26, 35%), followed by the lack of activities in the regions where participants worked or lived (4/26, 15%). Half of the respondents agreed that the app was easy to use and accessible. However, 37% (10/26) disagreed that it was comprehensive, and 62.5% (16/26) indicated that they were neutral about whether the information was accurate. Although 85% (17/20) of the service providers using the app noted that they did not intend to uninstall the app, 72% (27/38) of the caregivers said they might intend to.
We asked participants to identify how they heard about the Jooay app as a means of increasing our understanding of how information spreads in this population. Overall, 22% (20/93) of respondents indicated social media as their source of information, followed by those who indicated that they heard about it from their physical therapist, OT, or recreational therapist (18/93, 20%); 32% (30/93) indicated other sources, such as word of mouth or through advertising in hospitals.
Respondents had the opportunity to respond to open-ended questions on whether the app contributed to increased child leisure participation, and if so, how. One of the ways in which participants indicated that the app contributed to increased participation in leisure was by raising their awareness of community resources and programs. Service providers (clinicians and educators) who responded to the survey noted that they had used the app to provide information to families, caregivers, and clients about leisure opportunities, but they could not indicate whether the app had contributed to an actual increase in leisure participation. Caregivers indicated that although the app increased their awareness of activities that they were not aware of, it was still challenging to find activities that were suitable for their children. They indicated that the app needed to include more activities to enhance opportunities for participation.
Associations
We originally intended to explore the associations of sociodemographic characteristics, such as age, gender, type of disability, and place of residence, with perceptions of the app’s features and app use patterns. However, a significant amount of missing data (61%-91% of data are missing for some variables) precluded us from doing so. However, the preliminary exploratory analysis identified significant associations between the age of the caregiver and whether the person with a disability had engaged in physical activity before using the Jooay app (n=16; Fisher-Freeman-Halton Exact test: P=.01). Most of the caregivers indicated that their child was actively engaging in physical activities before accessing the Jooay app (28/38, 75%).
There was also a significant relationship among the users who indicated that the app had the information they were looking for and the province where they were located (n=20; Fisher-Freeman-Halton Exact test: P=.05). Most of the responses (12/20, 60%) were from Quebec, and of those responses, 9 (75%) respondents found that the app had a moderate fit with their needs. The lack of diversity in the users’ responses precluded a valid test of the association between user type (caregiver vs professional) and whether users found the activities in the app were a good fit; however, half of the respondents in both categories (n=10) indicated that the activities they found in the app had moderate to no fit with what they were looking for.
There was insufficient data from multiple user types to test for an association between user type and whether participation of their child increased as a result of the use of the Jooay app. Most of the respondents to this category of questions were caregivers (19/20, 94%), and 84% (17/20) of them indicated that their child’s participation increased after app use only slightly or moderately.
There was also an association between user type (caregiver vs service provider) and how frequently they used the app (n=24; Fisher-Freeman-Halton Exact test: P=.005). Most of the respondents to this category of questions were community organizations (7/24, 29%) and health care professionals (9/24, 37%); 45% (11/24) of respondents reported using the app multiple times without finding an appropriate activity for their client or child. The other 55% (13/24) of respondents had used the app less frequently and did not indicate whether they found an activity they were looking for or were only browsing through activities or exploring the app.
Discussion
Principal Findings
Using a PAR approach, we codeveloped and pilot-tested an mHealth tool with stakeholders to improve access to information and participation in inclusive leisure activities for Canadian children and youth with disabilities. Our aims were to understand feasibility aspects related to the use of a stakeholder-driven mHealth solution and understand user preferences and patterns to inform efforts to improve participation in leisure for children and youth with disabilities.
We used the Health Behavior Framework [
] to explore the individual and social characteristics of users and their relationship with the intention to use the app, measured as downloading the app, as well as the relationship with actual behavior change, reflected in increased participation. This enabled us to explore the potential impact of the app through users’ perceptions of usefulness, preferences for specific content, and suggestions for modifications. We also began to explore the sociodemographic characteristics of users and how they relate with use of mHealth technology and the expected health behavior outcomes.We applied the implementation framework by Peters et al [
] to understand the process-based factors that shaped the development and use of the app and opportunities to improve them. Below, we discuss the lessons learned, challenges faced, and implications for larger-scale efforts to use mHealth technology to promote health for children and youth with disabilities.Process Development Challenges and Opportunities
Participatory research is valuable and has potential key outcomes in health and implementation science [
]. Important challenges to consider in co-designing technology with multiple stakeholders include the ethical challenges of developing study protocols that are constantly changing and require multiple ethics review board amendments, the need to respond to divergent opinions in all steps of a project (eg, questionnaires and administration forms) and product development (eg, user interface vs user experiments design phases), and the extended length of time necessary for an authentic co-design process. Most of these issues have been identified in previous reports on participatory research; however, additional challenges learned in this study include communication barriers between end user stakeholders and the technology development team because of divergent language, culture, and operational modes.Ethics
We maintained a close discussion about the nature of the project with the institutional ethics review board and agreed on the elements that did not require ethics approval (ie, the stakeholder forums and app development stages) and which elements did (the surveys sent to app users and the information taken from app analytics). For elements requiring ethics approval, we agreed to an open protocol with the core elements of the project being initially approved through the regular, extended ethics review board procedures, and future iterations (ie, length of activity description, recruitment materials, or wording changes requested by our user collaborators) to undergo an expedited review, allowing for a reduced turnover time.
Stakeholder Engagement and Co-Design
It is important to advance implementation science on mHealth; it requires careful consideration of the interaction between technology development, participatory research and stakeholders involvement. We must develop protocols and standard operational procedures detailing aspects such as legal agreements between industry and research partners and business development plans that include design and maintenance discussions, establish a clear communication platform and verify with end users that they are able to access it. Flexibility in accepting other forms of communication that may be preferred by stakeholders have to be considered, and detailed note taking, with designated communication contacts on the team, is ideal.
Technology Development, User Preferences, and Health Behavior Change
The vast majority of the app users found that the information contained in the app was relevant, albeit not comprehensive (ie, not enough activities listed in their region of residence), to really affect the desired health behavior change of increased participation for their child. This may have been the reason why 72% of caregivers planned to uninstall the app. Service providers indicated that they did not intend to uninstall the app; thus, they likely perceived greater utility. Several studies have shown that lack of information is one of the key barriers to participation and perception of good health services for families of children and youth with disabilities [
- ]. Creating a mobile app that has accurate, up-to-date information and responds to the users’ needs and preferences is a challenge but is also essential to health behavior change.The lack of comprehensive information about activities in the app can be attributed to 2 main reasons: (1) the scarcity of activities offered in the community for this population and (2) the limited capacity to generate a comprehensive list of existing activities manually. The first issue is being addressed in a separate study (E Mogo, K Shikako, and A Majnemer; unpublished data; June 2021) where we conducted an in-depth analysis of the sociodemographic characteristics of regions and the availability of inclusive leisure activities, as listed in the app, with the objective of informing policy and program creation.
The second challenge relates to technology development. Creating comprehensive listings of activities that are constantly changing is a key challenge that can be addressed through technology but requires extensive sustainability planning. Sustainability and business models of mobile technology may be typical in design and industry but are foreign in health research when primary funds for development and co-design are obtained through research grants of limited duration. A sustainability model for mHealth needs to be further developed and tested to ensure the efficient use of research and user resources [
- ].Population Characteristics and mHealth Characteristics
Most survey respondents were caregivers of children with disabilities living in urban areas. Most activities listed in the app are located in urban areas. In the stakeholder forums, participants indicated that it is paramount to consider mHealth solutions that target populations that face multiple layers of marginalization, such as those who live in rural areas, indigenous children and families, and those who may not have access to mobile technology. Caregivers and service providers in rural areas have been identified as populations lacking access to services and other resources [
, ]. Therefore, the utility of Jooay as an mHealth tool in these regions is limited. We intend to apply these data to inform policy on gaps in service provision.This pilot study sheds light on the challenges of including proxies as the main users of interventions. The target population using the app are caregivers and service providers, but the actual expected behavior change (participation in leisure) is targeted at the children under their care. Information on the participation patterns of families came mainly from caregivers providing answers to the survey. We identified an association between previous participation levels and the caregivers’ age and between their familiarity with apps and the technology associated with the actual frequency of use of the Jooay app. It is known that caregiver behavior regarding leisure has an influence on the child’s level of participation [
]. The primary respondents of the survey were female (83/93, 89%), indicating that health promotion efforts could target female caregivers to affect the health behaviors of children with disabilities.Another important characteristic of this particular mHealth solution was accessibility for persons with disabilities. Although the app users are not necessarily children and youth with disabilities (only 8/93, 9% of our sample were persons with disabilities themselves) but rather the caregivers, it is important to consider that a mobile app for persons with disabilities should comply with accessibility standards. The challenges of following accessibility standards were perceived by our accessibility consultant and were outlined in previous research [
]. It was clear from the multiple iterations of testing that industry accessibility standards are not fully accessible for different individual needs, a factor that will be considered in future app development and iterations.mHealth and Health Behavior Outcomes
The survey results suggested that participants were not sure of the impact of the app on their children or clients’ participation levels but that they were certain that their knowledge about existing community activities had increased because of the app use. Ideally, mHealth should include artificial intelligence to directly track participation and objectively quantify the increase in participation as the desired outcome [
]. Although missing data prevented measuring desired behavior changes, a side effect of app use noted by some participants was the building community. Although the use of web-based communities by caregivers of patients with chronic health conditions is a relatively new phenomenon, several benefits and challenges have been identified [ ]. Perceived benefits include connecting with others with similar lived experiences and challenges and increasing awareness about a medical condition or, in the present case, about existing activities and resources. Participants in this study indicated that they may be using the app not just as a resource to change health behaviors but also as a resource to connect to others and increase their awareness about possible activities, even if they are not available in the region where they live. Such indirect positive outcomes are worth investigating further. Public health implementation efforts should consider the power of connecting people and the possibilities brought about by mHealth technologies on this front.For effective mHealth implementation, movement beyond pilot studies is needed to better understand the characteristics, preferences, and real-time use patterns of users. Partnership development with community organizations, cities, and other layers of governments is also needed to identify solutions to link resource databases and create machine learning algorithms that maintain relevant information for the public.
Limitations and Future Directions
This pilot feasibility study faced limitations that supported several important considerations for future studies. First, this study was conducted in a real-world setting. We had no control over the location, type of activity offered, and the match between these activities and the participants’ preferences or needs. Previous research has indicated that preference for certain types of activities is associated with engagement in these activities [
]. This pilot study shows that, in fact, preferences are not easily matched to the availability of resources in a community, and this is a barrier to participation. This study sheds light on the importance of adapting individualized mHealth interventions to public health impacts.The implementation strategy for this project was built in partnership with stakeholders. Implementation strategies included word of mouth and the use of local and web-based champions to disseminate information about the app and invite people to download it and use it. This project informed the important aspects of the implementation strategy on a larger scale. Recent studies have assessed the implementation of different data collection strategies through mobile apps and have reported mixed results. One study found that improving compliance with medication through digital data entry was feasible and reliable in a population of adults with HIV/AIDS [
]. Alternatively, another study that compared 3 electronic data collection methods for patients with a urinary tract infection—mobile app, electronic survey, or text message [ ]—found no differences in response rates. They concluded that participants often stopped data completion after their first interaction with technology, leading to missing data. They also raise the issue of the variability of user demographics as a factor influencing response rates and preferences. We found similar challenges in completion rates. Our survey had a low response rate of approximately 39.83% and missing data, which limited the ability to make generalizations. Our stakeholder advisory group confirmed that individuals are highly interested in using the app but will respond only to very short surveys. We noticed through backend data that people often stopped completing the survey at the point at which they had to scroll through and sign the consent form before completing the survey, which also suggests the need to review ethics procedures when using mHealth technologies to conduct research.Improvements in the recruitment strategy that will be implemented in the next phase of this study are the use of push notifications directly through the app, the shortening of the consent form to the minimal requirements, and the shortening of the survey. We will also ask our parent-partners, clinicians, and other coinvestigators to design a message that will be sent through push notifications. This message should be more welcoming of the participants’ responses. Future implementation efforts include making registration mandatory to use the app, increasing the relevance of information by implementing machine learning to update information, and increasing opportunities for interactions (ie, through push notifications and gamification). Future research must also address potential ethical constraints by appropriately adjusting their study design to elicit participation by using a mobile app as the only source for data collection.
The limited capacity of using app analytics across platforms and automated database updates imposed limitations to the data collected. Artificial intelligence to directly track participation, to objectively quantify increases in participation as the desired outcome, and improved analytics protocols are necessary to evaluate the effects on health behavior [
]. Therefore, monetary commitments associated with these strategies must be considered. Indeed, a thorough cost-effectiveness analysis should be an integral part of the scale-up implementation efforts of any technology [ ].Large-scale implementation research efforts using mHealth will need to consider better ways to engage the ecosystem of stakeholders, from users to rehabilitation centers, and community-based leisure centers to scale up and test more complex interventions. Efforts to elicit information from this population may require automated modes for data collection and partnerships between municipalities and organizations to link databases of activities and the app.
Conclusions
Implementation research holds the potential to drive real increments in public health by translating research findings to real-world testing. At-risk and underserved populations, such as people with disabilities, require increased efforts toward change, as they face multiple contextual barriers often leading to poor health outcomes [
].This study piloted the development and use of Jooay, a mobile app listing inclusive leisure activities. We also sought to understand user demographics and characteristics and the corresponding variables that would be of value to support its scale-up and effectiveness testing. Our intended use of this information was to better meet the needs of children and young people with disabilities and their support systems while also informing the literature to guide similar efforts.
mHealth is promising as a viable and feasible tool to execute implementation efforts, especially for this subpopulation. mHealth tools should integrate health promotion strategies for children with disabilities considering how to overcome poverty [
], be enjoyable [ ], improve access to care [ ], sensitize their health care providers, be person-centered, and provide the needed support for them to engage in healthy lifestyles [ , , , , ]. These tools will also have to be supported by more information to support their efficacy, effectiveness, cost utility, and engagement. Scale-up studies are necessary to move mHealth development science beyond pilot studies [ , ]. Finally, information on the demographics and characteristics of mHealth users and the impact of mHealth on behavioral predictors and health behaviors is needed [ ]. This can happen if only such mHealth efforts are designed for implementation and evaluation [ ].The next phase of this project will also inform programs and policy changes that can support a sustained model of inclusive leisure activities, mHealth integration into macrosystems of information sharing, and equitable distribution of health-promoting opportunities for children with disabilities and their families.
Acknowledgments
The Jooay Research Team is composed by multiple stakeholders who actively contribute to app and research development, including Mehrnoosh Movahed (Research Associate and Project Coordinator), Nick Katalifos (parent-partner), Joanne Charron (parent-partner), and Rachel Martens, (parent-partner and stakeholder engagement strategist); Melanie Bergthorson and Stephanie Glegg (clinical partners); Jess Silver (patient-partner); Michelle McClure and Dave Sora (community partners); and Tiiu Poldma and Janet McCabe (academic collaborators). The authors would like to thank Jose Correa from McGill University for statistical consultation, Richard Cooper from McGill University for proofreading the manuscript, and Luca Martial and Icoquih Badillo for research assistance. Infrastructure support was provided by the Center for Interdisciplinary Research in Rehabilitation of the Greater Montreal hosting the Participation and Knowledge Translation in Childhood Disabilities lab, which is where the project was developed. This project was funded by the Government of Canada through a Community Accessibility Grant of the Employment and Social Development Canada, the Canadian Institutes of Health Research through the Strategic Patient-Oriented Research program CHILD-BRIGHT (Child Health Initiatives Limiting Disability - Brain Research Improving Growth and Health Trajectories) Network in partnership with the Operation Enfant Soleil and the Montreal Children’s Hospital Foundation, and partially by the Kids Brain Health Network (Network of Centres of Excellence).
Conflicts of Interest
None declared.
References
- Bhattacharyya O, Reeves S, Zwarenstein M. What is implementation research? Res Soc Work Pract 2009 May 27;19(5):491-502. [CrossRef]
- PAR-13-056: Dissemination and implementation research in health. Department of Health and Human Services. URL: https://grants.nih.gov/grants/guide/pa-files/PAR-13-056.html [accessed 2020-05-28]
- Kuper H, Smythe T, Duttine A. Reflections on health promotion and disability in low and middle-income countries: case study of parent-support programmes for children with congenital zika syndrome. Int J Environ Res Public Health 2018 Mar 14;15(3):514 [FREE Full text] [CrossRef] [Medline]
- Davis K, Carter S, Myers E, Rocca N. Health promotion for young people with profound and multiple learning disabilities. Nurs Child Young People 2018 Mar 07;30(1):28-34. [CrossRef] [Medline]
- Sit CH, McKenzie TL, Cerin E, Chow BC, Huang WY, Yu J. Physical activity and sedentary time among children with disabilities at school. Med Sci Sports Exerc 2017 Feb;49(2):292-297. [CrossRef] [Medline]
- Oviedo GR, Travier N, Guerra-Balic M. Sedentary and physical activity patterns in adults with intellectual disability. Int J Environ Res Public Health 2017 Sep 07;14(9):1027 [FREE Full text] [CrossRef] [Medline]
- Hsieh K, Hilgenkamp TI, Murthy S, Heller T, Rimmer JH. Low levels of physical activity and sedentary behavior in adults with intellectual disabilities. Int J Environ Res Public Health 2017 Dec 04;14(12):1503 [FREE Full text] [CrossRef] [Medline]
- Mascarinas A, Blauwet C. Policy and advocacy initiatives to promote the benefits of sports participation for individuals with disability. In: De Luigi AJ, editor. Adaptive Sports Medicine. Switzerland: Springer; 2017:371-384.
- Columna L, Dillon SR, Norris ML, Dolphin M, McCabe L. Parents’ perceptions of physical activity experiences for their families and children with visual impairments. Br J Vis Impair 2017 May 08;35(2):88-102. [CrossRef]
- Anaby DR, Law MC, Majnemer A, Feldman D. Opening doors to participation of youth with physical disabilities: an intervention study. Can J Occup Ther 2016 May;83(2):83-90. [CrossRef] [Medline]
- Shikako-Thomas K, Lach L, Majnemer A, Nimigon J, Cameron K, Shevell M. Quality of life from the perspective of adolescents with cerebral palsy: "I just think I'm a normal kid, I just happen to have a disability". Qual Life Res 2009 Oct;18(7):825-832. [CrossRef] [Medline]
- Ash T, Bowling A, Davison K, Garcia J. Physical activity interventions for children with social, emotional, and behavioral disabilities-a systematic review. J Dev Behav Pediatr 2017;38(6):431-445. [CrossRef] [Medline]
- Baksjøberget PE, Nyquist A, Moser T, Jahnsen R. Having fun and staying active! Children with disabilities and participation in physical activity: a follow-up study. Phys Occup Ther Pediatr 2017 Dec;37(4):347-358. [CrossRef] [Medline]
- Wickman K, Nordlund M, Holm C. The relationship between physical activity and self-efficacy in children with disabilities. Sport Soc 2016 Sep 15;21(1):50-63. [CrossRef]
- Roll AE. Health promotion for people with intellectual disabilities - a concept analysis. Scand J Caring Sci 2018 Mar;32(1):422-429. [CrossRef] [Medline]
- Anaby D, Mercerat C, Tremblay S. Enhancing youth participation using the PREP intervention: parents' perspectives. Int J Environ Res Public Health 2017 Sep 02;14(9):1005 [FREE Full text] [CrossRef] [Medline]
- Catalano D, Holloway L, Mpofu E. Mental health interventions for parent carers of children with autistic spectrum disorder: practice guidelines from a Critical Interpretive Synthesis (CIS) systematic review. Int J Environ Res Public Health 2018 Dec 14;15(2):341 [FREE Full text] [CrossRef] [Medline]
- Bassett-Gunter RL, Ruscitti RJ, Latimer-Cheung AE, Fraser-Thomas JL. Targeted physical activity messages for parents of children with disabilities: a qualitative investigation of parents' informational needs and preferences. Res Dev Disabil 2017 May;64:37-46. [CrossRef] [Medline]
- Jin J, Yun J, Agiovlasitis S. Impact of enjoyment on physical activity and health among children with disabilities in schools. Disabil Health J 2018 Jan;11(1):14-19. [CrossRef] [Medline]
- Park Y. Emerging new era of mobile health technologies. Healthc Inform Res 2016 Oct;22(4):253-254 [FREE Full text] [CrossRef] [Medline]
- Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: where is the evidence? PLoS Med 2013;10(2):e1001382 [FREE Full text] [CrossRef] [Medline]
- Miller DP, Weaver KE, Case LD, Babcock D, Lawler D, Denizard-Thompson N, et al. Usability of a novel mobile health iPad app by vulnerable populations. JMIR Mhealth Uhealth 2017 Apr 11;5(4):e43 [FREE Full text] [CrossRef] [Medline]
- Wingo BC, Rimmer JH. Emerging trends in health promotion for people with disabilities. Int J Environ Res Public Health 2018 Apr 13;15(4):742 [FREE Full text] [CrossRef] [Medline]
- Fedele DA, Cushing CC, Fritz A, Amaro CM, Ortega A. Mobile health interventions for improving health outcomes in youth: a meta-analysis. JAMA Pediatr 2017 May 01;171(5):461-469. [CrossRef] [Medline]
- Lee J, Choi M, Lee SA, Jiang N. Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC Med Inform Decis Mak 2018 Feb 20;18(1):12 [FREE Full text] [CrossRef] [Medline]
- Carroll JK, Moorhead A, Bond R, LeBlanc WG, Petrella RJ, Fiscella K. Who uses mobile phone health apps and does use matter? A secondary data analytics approach. J Med Internet Res 2017 Apr 19;19(4):e125 [FREE Full text] [CrossRef] [Medline]
- McKay FH, Cheng C, Wright A, Shill J, Stephens H, Uccellini M. Evaluating mobile phone applications for health behaviour change: a systematic review. J Telemed Telecare 2018 Jan;24(1):22-30. [CrossRef] [Medline]
- Eldridge SM, Lancaster GA, Campbell MJ, Thabane L, Hopewell S, Coleman CL, et al. Defining feasibility and pilot studies in preparation for randomised controlled trials: development of a conceptual framework. PLoS One 2016;11(3):e0150205 [FREE Full text] [CrossRef] [Medline]
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care 2012 Mar;50(3):217-226 [FREE Full text] [CrossRef] [Medline]
- Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health 2006 Oct;60(10):854-857 [FREE Full text] [CrossRef] [Medline]
- Peters DH, Adam T, Alonge O, Agyepong IA, Tran N. Republished research: implementation research: what it is and how to do it: implementation research is a growing but not well understood field of health research that can contribute to more effective public health and clinical policies and programmes. This article provides a broad definition of implementation research and outlines key principles for how to do it. Br J Sports Med 2014 Apr;48(8):731-736. [CrossRef] [Medline]
- Bastani R, Glenn BA, Taylor VM, Chen MS, Nguyen TT, Stewart SL, et al. Integrating theory into community interventions to reduce liver cancer disparities: the Health Behavior Framework. Prev Med 2010;50(1-2):63-67 [FREE Full text] [CrossRef] [Medline]
- Greenwald T. Business model canvas: a simple tool for designing innovative business models. Forbes. 2012. URL: https://www.forbes.com/sites/tedgreenwald/2012/01/31/business-model-canvas-a-simple-tool-for-designing-innovative-business-models/ [accessed 2020-05-28]
- REDCap. URL: https://www.project-redcap.org/ [accessed 2020-05-28]
- Free C, Phillips G, Galli L, Watson L, Felix L, Edwards P, et al. The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med 2013;10(1):e1001362 [FREE Full text] [CrossRef] [Medline]
- Thabrew H, Fleming T, Hetrick S, Merry S. Co-design of eHealth interventions with children and young people. Front Psychiatry 2018;9:481 [FREE Full text] [CrossRef] [Medline]
- Zeidan J, Joseph L, Camden C, Shevell M, Oskoui M, Lamotte P, et al. Look around me: environmental and socio-economic factors related to community participation for children with cerebral palsy in Québec. Phys Occup Ther Pediatr 2021 Jan 25:1-18. [CrossRef] [Medline]
- Anaby D, Hand C, Bradley L, DiRezze B, Forhan M, DiGiacomo A, et al. The effect of the environment on participation of children and youth with disabilities: a scoping review. Disabil Rehabil 2013 Sep;35(19):1589-1598. [CrossRef] [Medline]
- Steinhardt F, Ullenhag A, Jahnsen R, Dolva A. Perceived facilitators and barriers for participation in leisure activities in children with disabilities: perspectives of children, parents and professionals. Scand J Occup Ther 2019 Dec 18:1-15. [CrossRef] [Medline]
- Coskun-Setirek A, Tanrikulu Z. Technological sustainability of mobile learning. Online J Sci Technol. 2017. URL: https://www.tojsat.net/journals/tojsat/articles/v07i03/v07i03-14.pdf [accessed 2021-06-09]
- Ammenwerth E, Iller C, Mahler C. IT-adoption and the interaction of task, technology and individuals: a fit framework and a case study. BMC Med Inform Decis Mak 2006;6:3 [FREE Full text] [CrossRef] [Medline]
- Liu Y, Yang Y, Liu Y, Tzeng G. Improving sustainable mobile health care promotion: a novel hybrid MCDM method. Sustainability 2019 Jan 31;11(3):752. [CrossRef]
- Service delivery in rural areas. Organisation for Economic Co-operation and Development (OECD). URL: https://www.oecd.org/cfe/regional-policy/service-delivery-in-rural-areas.htm [accessed 2020-05-28]
- Martinez J, Pampalon R, Hamel D, Raymond G. Does living in rural communities rather than cities really make a difference in people’s health and wellness?. Montreal, Quebec: Institut National De Santé Publique Du Québec; 2004.
- Trost SG, Sallis JF, Pate RR, Freedson PS, Taylor WC, Dowda M. Evaluating a model of parental influence on youth physical activity. Am J Prev Med 2003 Nov;25(4):277-282. [CrossRef] [Medline]
- Yu D, Parmanto B, Dicianno B. An mhealth app for users with dexterity impairments: accessibility study. JMIR Mhealth Uhealth 2019 Jan 08;7(1):e202 [FREE Full text] [CrossRef] [Medline]
- Bonin F. Could AI help people change their behaviour? IBM Research Blog. 2018. URL: https://www.ibm.com/blogs/research/2018/12/ai-help-change-behaviour [accessed 2020-05-28]
- Ravoire S, Lang M, Perrin E, Participants of Giens XXXII Round Table No. 6. Advantages and limitations of online communities of patients for research on health products. Therapie 2017 Feb;72(1):135-143. [CrossRef] [Medline]
- Shikako-Thomas K, Dahan-Oliel N, Shevell M, Law M, Birnbaum R, Rosenbaum P, et al. Play and be happy? Leisure participation and quality of life in school-aged children with cerebral palsy. Int J Pediatr 2012;2012:387280 [FREE Full text] [CrossRef] [Medline]
- Przybyla SM, Eliseo-Arras RK, Krawiec G, Gower E, Dermen K. Feasibility and acceptability of a smartphone app for daily reports of substance use and antiretroviral therapy adherence among HIV-infected adults. AIDS Res Treat 2016;2016:1-12 [FREE Full text] [CrossRef] [Medline]
- Duane S, Tandan M, Murphy AW, Vellinga A. Using mobile phones to collect patient data: lessons learned from the SIMPLe study. JMIR Res Protoc 2017 May 25;6(4):e61 [FREE Full text] [CrossRef] [Medline]
- de la Torre-Díez I, López-Coronado M, Vaca C, Aguado JS, de Castro C. Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: a systematic review. Telemed J E Health 2015 Feb;21(2):81-85 [FREE Full text] [CrossRef] [Medline]
Abbreviations
mHealth: mobile health |
OT: occupational therapist |
PAR: participatory action research |
REDCap: Research Electronic Data Capture |
Edited by G Eysenbach; submitted 26.08.20; peer-reviewed by C Cheng, K Arbour-Nicitopoulos; comments to author 06.10.20; revised version received 29.01.21; accepted 19.05.21; published 16.09.21
Copyright©Keiko Shikako, Ebele R I Mogo, Valerie Grand-Maison, Robert Simpson, Lesley Pritchard-Wiart, Annette Majnemer, Jooay App Research Group. Originally published in JMIR Formative Research (https://formative.jmir.org), 16.09.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.