Background: The Australian Dietary Guidelines (ADG) translate the best available evidence in nutrition into food choice recommendations. However, adherence to the ADG is poor in Australia. Given that following a healthy diet can be a potentially cost-effective strategy for lowering the risk of chronic diseases, there is an urgent need to develop novel technologies for individuals to improve their adherence to the ADG.
Objective: This study describes the development process and design of a prototype mobile app for personalized dietary advice based on the ADG for adults in Australia, with the aim of exploring the usability of the prototype. The goal of the prototype was to provide personalized, evidence-based support for self-managing food choices in real time.
Methods: The guidelines of the design science paradigm were applied to guide the design, development, and evaluation of a progressive web app using Amazon Web Services Elastic Compute Cloud services via iterations. The food layer of the Nutrition Care Process, the strategies of cognitive behavioral theory, and the ADG were translated into prototype features guided by the Persuasive Systems Design model. A gain-framed approach was adopted to promote positive behavior changes. A cross-modal image-to-recipe retrieval model under an Apache 2.0 license was deployed for dietary assessment. A survey using the Mobile Application Rating Scale and semistructured in-depth interviews were conducted to explore the usability of the prototype through convenience sampling (N=15).
Results: The prominent features of the prototype included the use of image-based dietary assessment, food choice tracking with immediate feedback leveraging gamification principles, personal goal setting for food choices, and the provision of recipe ideas and information on the ADG. The overall prototype quality score was “acceptable,” with a median of 3.46 (IQR 2.78-3.81) out of 5 points. The median score of the perceived impact of the prototype on healthy eating based on the ADG was 3.83 (IQR 2.75-4.08) out of 5 points. In-depth interviews identified the use of gamification for tracking food choices and innovation in the image-based dietary assessment as the main drivers of the positive user experience of using the prototype.
Conclusions: A novel evidence-based prototype mobile app was successfully developed by leveraging a cross-disciplinary collaboration. A detailed description of the development process and design of the prototype enhances its transparency and provides detailed insights into its creation. This study provides a valuable example of the development of a novel, evidence-based app for personalized dietary advice on food choices using recent advancements in computer vision. A revised version of this prototype is currently under development.
In 2017, approximately 11 million deaths and 255 million disability-adjusted life years were attributable to dietary risk factors worldwide . Poor food intake contributes to 5 of the top 6 chronic disease risk factors (eg, obesity and high blood pressure) in Australia [ ]. This poor intake is accountable for over 40% of the heart disease and 34% of the endocrine disease burden in Australia [ ]. Healthy eating behaviors, such as increased fruit and vegetable consumption, are associated with a reduced risk of cardiovascular diseases and certain cancers [ ].
National dietary guidelines such as the Australian Dietary Guidelines (ADG) are intended to influence the dietary behaviors of the general population to maintain health and inform food, nutrition, and health policies and programs . The ADG are statements to translate the best available evidence for dietary patterns, foods, and nutrients for health into food choice recommendations [ ]. However, adherence to the ADG is poor in Australia. For example, <10% of Australian adults met the recommendations for daily vegetable consumption in 2020-2021, and this low rate has remained consistent over time [ , ]. There are many barriers to adherence to the ADG. The current food supply is complex, with many foods available to the public. Cognitive skills, such as planning and problem-solving, are required to incorporate the dietary guidelines into the everyday practicalities associated with navigating the food system [ ]. Moreover, particular circumstances are also common in today’s lifestyles, for example, being rushed, having too little sleep, and experiencing psychosocial stress [ ]. These circumstances can increasingly challenge people to make healthy food choices [ ]. It implies that a supportive tool is required to provide real-time personalized, practical advice for food choices, facilitating adherence to the ADG.
Mobile health (mHealth) refers to the use of mobile devices such as smartphones and tablets in health and health service delivery . Mobile apps are increasingly being used to deliver and support nutrition interventions in community settings [ ]. With just over 6 billion smartphone users worldwide [ ], the increasing popularity of smartphones may offer new opportunities to implement innovative, scalable tools to improve the reach and effectiveness of nutrition interventions and make positive behavior changes [ - ]. The use of mobile apps for nutrition interventions has many potential advantages because of their ubiquity, cost-effectiveness, less-invasive nature, ability to provide immediate feedback, and tracking capabilities. Good adherence and usability of apps for nutrition interventions were established in controlled environments (eg, testing a mobile app); however, adherence to the intervention in real-world settings was low [ , ].
Adherence to digital interventions appears to rely heavily on the interactive design of digital tools [, ]. Interactive apps focus on human-app interaction to support users in communicating and interacting with digital tools on a daily basis [ ]. The concept of user experience has been centered on interaction design [ ]. The emotional elements of interaction design tend to elicit a desirable user experience (eg, enjoyment) and avoid undesirable user experiences (eg, frustration) [ ]. For example, users tend to continue using an app if it provides an enjoyable user experience, whereas frustration with an app often leads to users stopping its use. These emotional aspects of interaction can also be applied in designing an app to persuade people for behavior changes [ , ].
Persuasive technology is defined as an interactive information system designed to change people’s attitudes and behaviors with a focus on the implementation side related to the features of information systems . For example, a mobile app for persuasive technology can be used as a tool to persuade users for behavior changes by making target behavior easier to perform [ ]. Such an app provides a medium for gaining experience in planned interventions for behavior changes [ ]. This experience assists in persuading users to motivate and rehearse a target behavior. The Persuasive Systems Design (PSD) model offers a framework for developing and evaluating persuasive systems at the interface level [ ]. The PSD model categorizes the design principles of a persuasive system into primary task support (carrying out the user’s primary task), dialogue support (providing feedback and implementing computer-human dialogue support in a manner that helps users keep moving toward their goal or target behavior), social support (motivating users by leveraging social influence), and credibility support (being more credible and thus more persuasive) [ ]. The aim of applying the design principles of the PSD model at the interface level is to guide the planned persuasive effects when using the tool, which facilitates communication of the nature, purpose, procedures, potential risks, and benefits of using the tool [ , ]. Furthermore, gamification refers to the use of game-like elements, such as rewards and competition, in nongame contexts to motivate individuals to engage in target behavior [ ]. For example, a gamified nutrition intervention provides rewards for reaching the goals for increasing vegetable intake [ - ]. Thus, gamification can be viewed as a subfield of persuasive technology [ , , ]. It often involves the use of technology to design and implement game-like features, with the goal of persuading and influencing users.
To evoke a desirable user experience when using an app, a user-centered design principle tends to be applied to create solutions specific to the intended users . However, previous evidence has demonstrated that the apps did not meet the needs of users, although users were involved in the early design process [ ]. It has been suggested that there is a gap in translating users’ specific needs for apps into technological features [ - ]. A certain level of digital literacy, including the competencies and skills required to navigate mobile devices, is required to use and interpret digital health tools [ ]. For example, users may need to understand the language, hardware, and software to effectively navigate the technology and communicate their needs. Therefore, engaging users in the design process can have advantages for managing expectations and fostering a sense of ownership over an app. However, determining the appropriate timing and manner in which users are involved in the development process may require careful planning [ ].
The use of behavior change theory in nutrition interventions is pivotal to the effectiveness of mHealth interventions. Commercial apps are typically service-oriented and contain an appealing user interface, but evidence reveals that they tend to have limited basis in evidence and behavior change theory [, ]. The underlying philosophy of validated behavior change theories within the field of nutrition and dietetics offers the assumptions for behavior changes [ ]. These assumptions can assist in understanding external and internal issues and the dynamics that facilitate behavior changes [ ]. Specific behavior change strategies for nutrition interventions can be selected based on these insights [ ]. For example, cognitive behavioral theory (CBT) [ ] is one of the oldest and most tested behavior change theories used in nutrition and dietetics [ ]. The literature suggests that CBT-based interventions are effective in changing targeted dietary habits, such as increasing the intake of fruits and vegetables [ ]. The assumption underlying CBT is that behavior can be learned through or facilitated by cognitive and behavioral strategies [ ]. Applying CBT, people tend to be taught a variety of strategies to make them aware of internal and external cues and their response [ ]. Examples of strategies endorsed by CBT include self-monitoring, goal setting, problem-solving, and rewards [ ]. Thus, the underlying philosophy and behavior change strategies of CBT offer a rationale for the cognitive aspects of the interaction design for an app.
Moreover, there has been growing awareness that a one-size-fits-all approach to dietary advice has limited effectiveness, and there is a need to adopt a personalized approach. In nutrition science, personalized nutrition has gained popularity in recent years. Personalized nutrition can be based on biological evidence of differential responses to food or nutrients (eg, genotypic or phenotypic characteristics) . It can also be based on an analysis of current behavior, preferences, barriers, and goals related to dietary intake [ ]. This personalization approach attempts to deliver a nutrition intervention at an individual level [ ]. However, evidence suggests that the biological characteristics of individuals tend to act only as catalysts to encourage changes in eating behavior [ ]. Eating foods is repeated multiple times during a day as meals (eg, breakfast, lunch, dinner, or snacks), which are distinguished from one another by specific food choices [ - ]. It indicates that there is a need to focus on developing supportive approaches to effectively motivate and enable users to make positive dietary changes based on an individual’s dietary intake. Despite this, the review of highly ranked 80 dietary tracking apps from the Google Play Store, Apple App Store, and Microsoft Store in 2022 demonstrated that the absence of personalized dietary advice that considers users’ dietary intake is a notable gap in the features of these apps [ ].
The limitations of the available apps for nutrition interventions indicate that obtaining high-quality dietary intake data is central to the design of an app for personalized dietary advice. In the food layer of the Nutrition Care Process (NCP), Nutrition Assessment is the first step to systematically collect timely and relevant dietary intake data needed to identify nutrition-related problems and causes (Nutrition Diagnosis) . However, diet is complex because of the different foods that are consumed in varying portions, fluctuating quantities over time, and differing combinations [ , - ]. There is a trade-off between user burdens and the level of details required for collecting dietary intake data because tracking of dietary intake usually requires substantial involvement from users [ , ]. Furthermore, reliable food composition data are required to assess dietary intake, which is the key determinant of the quality of apps [ ].
The widespread use of smartphones with integrated high-quality cameras has driven the inclusion of technology-assisted dietary assessments in nutrition apps. The use of these methods reduces user burden and maximizes the effectiveness of tracking dietary intake over time . The review of highly ranked 80 dietary tracking apps from the app stores in 2022 demonstrated that 51% of apps have a barcode scanner for food recognition [ ]. However, barcode scanning is not suitable for all types of food (eg, mixed dishes and unpacked foods) and cannot provide the actual portion of consumed food. Despite the significant advancements in using computer vision to automatically assess dietary intake from food photos [ ], their application beyond academic settings is still quite limited. For instance, 3 out of the reviewed 80 commercial apps in 2022 use this technique, with only 1 app, Foodvisor, featuring automatic recognition and nutritional information calculation from food photos [ ].
In recent years, image recognition (eg, food items and portion size estimation) has undergone a paradigm shift toward using deep learning for its strong capability in feature learning of images to achieve greater performance [, ]. However, food recognition from food images is still unable to achieve optimal performance, particularly for mixed foods such as foods containing more than 1 ingredient [ , ]. Food recognition from food images requires fine-grained ingredient recognition. Directly recognizing ingredients is challenging, because ingredients from prepared food are mixed during food preparation. Furthermore, cutting and cooking food may change the surface color, shape, and texture of food. Using image-to-recipe retrieval, computer vision techniques analyze the visual characteristics of food images and establish a connection between the rich contextual details found in recipes with the visual content presented in food images [ ]. A larger data set with full recipe information extracted from the internet was used in a study of image-to-recipe retrieval to identify multiple foods and ingredients in a food image with 93% accuracy [ ].
This study describes the development process and design of a prototype mobile app for personalized dietary advice based on the ADG for adults in Australia, with the aim of exploring the usability of the prototype. The goal of the prototype was to provide personalized evidence-based support for self-managing food choices in real time.
The design science paradigm incorporates behavioral aspects into the design of information systems to solve real-world problems . This approach is well suited for designing, developing, and evaluating mobile apps because it provides a conceptual framework, along with flexible guidelines for understanding, executing, and evaluating an information system [ ]. Thus, the guidelines of the design science paradigm were applied to guide the design, development, and evaluation of the prototype in 3 phases, including the determination of the requirements (Guideline 2: Problem relevance and Guideline 5: Research rigor), development of the prototype (Guideline 6: Design as a search process and Guideline 1: Design as an artifact), and evaluation of the prototype (Guideline 3: Design evaluation). A co-design approach was applied by a team of researchers in computer science and information technology as well as an Accredited Practising Dietitian (APD), with the latter taking the lead.
Determination of Prototype Requirements
The lead dietitian (VG) developed the initial requirements of the prototype based on her previous work on food choices and the flow of generating dietary intake data using detailed dietary intake data, gold standard biomarkers, mixed research methods, and a data-mining tool [, - , ]. The requirements of the prototype were shaped by the integration of the food layer of the NCP [ ], the strategies of CBT [ ], and the ADG [ ], along with insights from the reviews of the characteristics of apps for nutrition interventions [ , ]. A gain-framed approach was adopted to guide the design of the prototype, with the aim of promoting positive behavior changes (eg, increase in awareness and motivation for healthy eating, as well as improve self-management skills and nutrition knowledge) and avoiding negative outcomes (eg, obsessive energy counting and feelings of guilt or disappointment) [ ]. The PSD model [ ] was used to guide the interaction design at the interface level. The final requirements of the app were defined by the research team.
The wireframes of each screen were developed. The wireframes focused on the screen layout and were organized to reflect the food layer of the NCP. The wireframes were then mapped to mock-ups showing the actual visual designs for each screen. The primary researchers (VG and PW) were consulted at each iteration for feedback. All the requirements of the prototype were required to be met.
Development of the Prototype
Deployment of an Open-Source Machine Learning Model for Image-to-Recipe Retrieval
An Apache-2.0 license is an open-source license, allowing users to freely use, modify, and distribute the products without restrictions . Thus, a recently published cross-modal image-to-recipe retrieval model under an Apache 2.0 license [ ] was deployed in this study. The deployed machine learning model is an end-to-end model based on hierarchical transformer-based encoders for images (eg, food photos) and text (eg, titles of recipes and ingredients). This model is composed of an image encoder, a recipe encoder, and the training objectives of supervised loss for paired data and self-supervised recipe loss. The results of the experiments demonstrated that the training objectives leveraging semantic relationships within recipes achieved 93% accuracy in the image-to-recipe retrieval task on the Recipe1M data set [ , ].
The primary requirement for deploying the machine learning model was to retrieve food items in the food photos. The psychometric properties (eg, validity and reliability) of image-to-recipe retrieval were not assessed in this study, as such evaluations were beyond the scope of this research. The code for the machine learning model was downloaded from a publicly available source . Originally, the code was to generate the metrics for evaluating the performance of the model on the benchmark. We modified the code for the image-to-recipe retrieval model using PyTorch [ ] on the Gadi Supercomputer at the National Computational Infrastructure of Australia. The output of the image-to-recipe retrieval model was a list of potential ingredients in a food photo.
The subset of Recipe1M+ with nutrition information  was used to perform the image-to-recipe retrieval in this study. The ingredients of 50,637 recipes in the subset were matched with the United States Department of Agriculture National Nutrient Database for Standard Reference release 27 [ ]. The healthiness of the recipes was evaluated and presented using the traffic lights system established by the Food Standards Agency based on the content of saturated fat, sodium, sugar, and total fat per 100 g [ ]. To assign Australian food composition data, the ingredients in the subset of Recipe1M+ with nutrition information were manually matched to a single ingredient or to a recipe listed in the Australian Food, Supplement and Nutrient Database (AUSNUT) 2011-13 database [ ] based on the published method [ ]. The Food and Agriculture Organization International Network of Food Data Systems Guidelines for Food Matching have been applied [ ]. The matching guidelines provided criteria and a confidence code for each ingredient based on how well it matched with another food item. After removing the duplicates, a total of 352 food items in the subset of Recipe1M+ were matched with the AUSNUT 2011-13 database. More than three-quarters (270/352, 76.6%) of the food items were matched with exact food items. Approximately 7.4% (26/352)of the food items were matched with alternative food items, but 15.9% (56/352) of the food items were matched with similar foods.
Evaluation of the Prototype
Ethics approval was granted by the University of Wollongong Human Research Ethics Committee (2022/327).
Participants were recruited through convenience sampling from the Illawarra community, a major coastal region 70 km south of Sydney, Australia. Social media from the University of Wollongong were used to advertise the study. The inclusion criteria were adults aged 18 to 65 years who own a smartphone and were willing to use the prototype and trial the prototype thoroughly. The exclusion criteria included inability to communicate in English, impaired ability to participate in the study, and having difficulties or major impediments to participating in the study’s components. APDs were also included in this study. All participants provided written informed consent to participate in the study. Each participant was given a deidentified participant code after completing the study.
A semistructured, cross-sectional evaluation led by an experienced facilitator (VG) was conducted to explore participants’ opinions on the prototype in a face-to-face meeting or via the web-based meeting platform Zoom (Zoom Video Communications Inc, 2022) as chosen by the participants. The Mobile Application Rating Scale (MARS) offers a multidimensional measure of the quality of health-related apps . This questionnaire can also be used as a checklist for the design and development of health-related apps [ ]. The MARS has been applied to evaluate health-related apps with end users in many studies [ - ]. Thus, the MARS was used as a list of closed-ended questions to explore the participants’ opinions on prototype quality and the perceived impact of the prototype. The instructions for the questionnaire were followed to conduct this study using the participants’ smartphones. The prototype was accessed through a web browser. Subsequent semistructured in-depth interviews were conducted to deliberate and discuss the participants’ opinions on the design and perceived impact of the prototype. The interviews were audio recorded.
A numerical summary of the responses to the MARS was created to provide a descriptive summary. Given that this study aimed to evaluate a prototype, the questions of accuracy of app description in the app store, goals, and evidence based on the informational content were excluded from the analysis. The semistructured interviews were transcribed verbatim. A process of coding and thematic analysis were performed to identify dominant themes . The transcripts from adults and dietitians were pooled for joint coding. This method enables an in-depth understanding of user experience [ ]. Exemplar quotes for the themes were identified and reported. All themes were managed and reviewed using the qualitative analysis software NVivo (version 12; QSR International Pty Ltd).
A summary of the final requirements is presented in. The primary requirement of Nutrition Assessment for the food layer of the NCP in the prototype was to transform the way that dietary data were collected, shifting from an active method of manually adding food items to a more passive approach of automatically retrieving food items from a food photo. The aim of this requirement was to reduce user burden of logging dietary intake. Meal patterns appear to be closely linked to eating habits [ ]. Thus, food choice events were labeled as meals (eg, breakfast, lunch, dinner, or snacks). For Nutrition Diagnosis in the food layer of the NCP, a computational system was developed to facilitate continuous assessment of dietary intake and enable real-time feedback on food choices. This system used user input (dietary intake data) and logical rules (satisfaction rules) to determine compliance with personal food choice goals. For example, the system assessed whether a user had met their goal for a specific food group based on their reported dietary intake. This information was then used to provide real-time feedback on food choices. Both the internal-focused strategies (goal setting, self-monitoring, and rewards) and the external-focused strategies (environmental support and reinforcement) of CBT [ ] were integrated into the design of the Nutrition Intervention. Given that the use of persuasion strategies and techniques is determined by the goals of information systems [ ], the design principles of primary task support, dialogue support, and system credibility in the PSD model [ ] were applied in this study. For example, the app was used as a tool to persuade users to change their eating behavior by making it easier to track their food choices. The prototype encouraged human-app interaction to help users keep moving toward their food choice goals by using gamification principles.
Food layer of the Nutrition Care Process and requirements for the prototype.
Nutrition Assessment or Reassessment
- Log daily foods and beverages organized by meals
- Take and upload food photos to retrieve food items in the food photos
- Log dietary intake manually
- Store and display the recently used food items to offer adaptive diet history records
- Automatically calculate daily intake values of food choices (food choices included 5 food groups [vegetables, fruits, grains, meat and alternatives, and milk and alternatives] and discretionary foods listed in the Australian Dietary Guidelines) based on dietary intake input
- Automatically compare daily dietary intake to personal goals for food choices
- Provide personalized information on the Australian Dietary Guidelines recommended number of daily servings for food choices based on age and gender
- Set personal goals for numbers of daily servings for food choices
- Provide immediate feedback to visualize the progress of daily dietary intake against personal goals of food choices
- Provide rewards when personal goals met
- Display information on the Australian Dietary Guidelines
- Search and display recipes
Nutrition Monitoring or Evaluation
- Provide 24/7 access
- Set up an account to allow user tracking of food choices on a personal basis
- Store and display daily intake over time to allow users to view their food choice history
- Provide challenges to affirm the user’s commitment to use the prototype
Development of the Prototype
Structure of the Prototype
The structure of the prototype is illustrated in. The prototype was developed as a web application. The prototype became a PWA when a manifest file and service worker were used. Users can create an app icon on the home screen of their smartphone. Although the prototype can be downloaded and immediately run offline as an app, it required computing resources for running the image-to-recipe retrieval. The primary requirement of a manifest file was to create an app icon on the home screen, serving as a shortcut of the prototype. The prototype was further enhanced using the cache facilitated by the service worker. For example, the site displayed the content in a weak or offline network.
Design of the Prototype
Screenshots of the screens are presented in. The details of the information architecture are shown in . On the home screen, to provide environmental support for food choices, users can search and select recipes to obtain recipe suggestions ( B). The suggestion included information on the healthiness of the recipe, an ingredient list, and cooking instructions ( D). The subset of Recipe1M+ with nutrition information [ ] was used to provide recipe suggestions. The ADG, Guideline 1, Guideline 2, and Guideline 3 [ ] were displayed as “Tips” on the home screen to reinforce the recommendations of the ADG ( B). The Australian Guide to Healthy Eating poster [ ] was used to present relevant information related to the guidelines. The source of the recommendations was displayed on the screen. A 7-day challenge was created to ask users to actively log in the challenges ( C). Trophies lighted up for daily challenges as rewards. A 4-week challenge was included to display a historical record of the completed challenges. An archery target figure appeared for logging into the challenges for 1 week (7 days).
To log dietary intake, food intake was recorded as user-defined meals including breakfast, lunch, dinner, and snacks (E). Users input their food items by taking a real-time food photo, uploading a food photo stored on the smartphone, and selecting a food item from a drop-down food list based on the food names from the AUSNUT 2011-13 database [ ] ( F and 2G). The users were allowed to use these methods simultaneously. If the image-to-recipe retrieval function was used (eg, taking or uploading a food photo), users edited the retrieved food items from the food photos ( H). Then, users input the quantities of their food consumption. The prototype stored and displayed the recently used food items to offer adaptive diet history records ( I), which made food logging quick and simple.
The analysis of dietary intake was automatically performed. The Australian Health Survey–ADG food composition database  was applied to aggregate dietary intake data for 5 food groups (vegetables, fruits, grains, milk and milk alternatives, and meat and meat alternatives). Intakes of discretionary foods were identified using the Discretionary Food List database [ ] and AUSNUT 2011-13 database [ ]. One serving of discretionary foods is defined as containing 600 kJ or 10 g of alcohol, respectively [ ]. The prototype stored and displayed the daily dietary intake over 7 days to allow users to view and self-monitor the history of their food choices ( L).
In goal setting, the prototype provided information on the ADG recommended number of daily servings for 5 food groups and discretionary foods based on the user’s age and gender (K) [ , ]. The source of the recommendations was displayed on the screen. Users set their personal goals for 5 food groups and discretionary foods. The progress bar charts were used to provide immediate feedback to visualize the progress of dietary intake against the personal goals of each food group ( J). The icons of the 5 food groups and discretionary foods were initially displayed as whited-out icons ( J). If the logged dietary intake met personal goals, then the icons were changed to colored icons. The prototype also displayed daily and weekly overall performance in achieving personal goals. The parameters were as follows: (1) if dietary intake met the daily or 7-day average of 2 goals, then it displayed the “need to improve”; (2) if the dietary intake met the daily or 7-day average of 4 goals, then it displayed “good”; and (3) if the dietary intake met the daily or 7-day average of all 6 goals, then it displayed “great.” Examples of the design related to the applied design principles of the PSD model [ ] and the general gamification principles in health care [ ] are presented in and , respectively.
|Design principle and definition ||Example|
|Primary task support|
|System should reduce effort that users perform their target behavior.|
|System should guide users in the attitude change process by providing means for action that brings them closer to the target behavior.|
|System should offer personalized content and services for its users.|
|System should provide means for users to track their performance or status|
|System should provide means for rehearsing a target behavior|
|System should provide virtual rewards for users to give credit for performing the target behavior|
|System should have a look and feel that appeals to its users.|
|System credibility support|
|System should provide information that is truthful, fair, and unbiased.|
|System should provide information showing knowledge, experience, and competence.|
|System should have competent look and feel.|
|System should provide means to verify the accuracy of site content via outside sources.|
|Gamification principle||Description ||Example|
|Adaptive game mechanics||System should adapt to a user’s performance.|
|Appointment dynamics||System should ask users to finish a task in a certain amount of time.|
|Badges or medals||System should reward users with badges or medals for designed tasks.|
|Immediate feedback||System should provide immediate feedback after completing designed tasks.|
|Progress status||System should allow users to check their progress.|
Evaluation of the Prototype
A total of 12 adults and 3 APDs completed the evaluations (November 2022 to December 2022). The adults consisted of 4 male individuals and 8 female individuals aged 20 to 40 years, with a median age of 25.5 (IQR 21-33.5) years. There were 7 adults who completed tertiary education. The dietitians were female, aged 46 to 49 years, and had worked as dietitians for a mean of 20 years. The length of the evaluations ranged from 23 to 44 minutes, with a median of 35 (IQR 30-40) minutes.
The summary of the MARS scores for the prototype is presented in. The overall quality score was “acceptable,” with a median of 3.46 (IQR 2.78-3.81) out of 5 points. There were 3 participants who rated the prototype with 4 out of 5 stars ( A). For functionality, out of 15 participants, approximately half of the participants rated the navigation (n=8, 53%) and gestural design (n=7, 47%) of the prototype as good or excellent. However, the performance of the prototype showed potential for improvement, as 7 (47%) participants rated it poor. A total of 8 (53%) participants rated the layout of the prototype as good or excellent. Moreover, 60% (n=9) of participants rated the informational content of the prototype as good or excellent. A discrepancy was observed between the adults’ and dietitians’ opinions on credibility of the prototype. The median score for the perceived impact on healthy eating based on the ADG was 3.83 (IQR 2.75-4.08) out of 5 points. The feedback regarding the perceived impact of the prototype with respect to healthy eating based on the ADG is presented in B. Both adults and dietitians advised that the prototype had the potential to impact awareness and knowledge of healthy eating based on the ADG.
|Adults (n=12), median (IQR)||Dietitians (n=3), mean (SD)||Total (N=15), median (IQR)|
|Entertainment||3 (3-3)||3.67 (0.58)||3 (3-3.5)|
|Interest||3 (3-4)||3.33 (1.15)||3 (3-4)|
|Customization||3 (3-3.25)||2.67 (0.58)||3 (3-3)|
|Interactivity||3 (3-4)||3.00 (1.00)||3 (3-4)|
|Target group||3.5 (3-4)||3.00 (1.00)||3 (3-4)|
|Mean score||3.1 (2.95-3.65)||3.13 (0.83)||3.2 (2.9-3.7)|
|Performance||2.5 (2-4)||2.67 (0.58)||3 (2-3.5)|
|Ease of use||3 (3-3.25)||2.67 (0.58)||3 (3-3)|
|Navigation||4 (2.75-4.25)||3.33 (0.58)||4 (3-4)|
|Gestural design||3.5 (3-4)||3.33 (0.58)||3 (3-4)|
|Mean score||3.5 (2.69-3.81)||3.00 (0.50)||3.5 (2.63-3.63)|
|Layout||4 (3-4)||3.00 (0)||4 (3-4)|
|Graphics||3 (3-4)||3.00 (1.00)||3 (3-4)|
|Visual appeal||3 (2.75-4)||3.33 (0.58)||3 (3-4)|
|Mean score||3.33 (2.92-3.83)||3.11 (0.51)||3.33 (2.83-3.67)|
|Quality of information (content)||4 (3.75-4.25)||3.33 (2.08)||4 (3.5-4.5)|
|Quantity of information (extent)||4 (3-5)||3.00 (2.00)||4 (3-5)|
|Visual information||4 (3-4)||3.67 (0.58)||4 (3-4)|
|Credibility||4 (3.5-4)||2.33 (1.53)||4 (2-4)|
|Mean score||4 (3.13-4.31)||3.08 (1.51)||4 (3-4.38)|
|Prototype quality mean score||3.48 (2.8-3.82)||3.08 (0.83)||3.46 (2.78-3.81)|
|Overall star rating||3 (2.75-3)||3.00 (1.00)||3 (2.5-3)|
|Perceived impact of the prototype|
|Awareness||4 (2.75-4)||4.00 (1.73)||4 (2.5-4)|
|Knowledge||4 (2.75-4)||4.00 (1.73)||4 (2.5-4.5)|
|Attitudes||4 (2.75-4)||2.67 (1.53)||4 (2.5-4)|
|Intention to change||4 (2.75-4)||3.00 (1.00)||4 (2.5-4)|
|Help seeking||4 (2.75-4)||3.00 (1.00)||4 (2.5-4)|
|Behavior change||4 (2.75-4)||3.33 (1.15)||4 (2.5-4)|
|Mean score||3.75 (3.13-4.04)||3.33 (1.30)||3.83 (2.75-4.08)|
In-depth interviews lasted between 4.50 and 22 minutes, with a median of 7.75 (IQR 7.15-10.75) minutes. The participants elaborated on their experience of using the prototype, with a focus on prototype design.
User Experience of Using the Prototype
The results demonstrated that the prototype design evoked both positive and negative user experiences. For example, the participants found using the prototype fun and enjoyable:
It was overall really fun to use and trial.
I really did enjoy it.
The main driver of the positive user experience was interaction design, which provided immediate feedback on the progress of daily dietary intake against personal goals. This interaction design integrated the aesthetic elements into the gamification, creating an entertaining, interactive, and visually appealing experience of tracking food choices:
It has like the colored bars that tell you how far along you are reaching that the serve goals. So, it’s kind of like a visual. You can say, oh, I’m this close to getting it. I really liked that.
The participants found that the provision of recipe ideas was helpful. Suggested improvements included a need for a shopping list, images of the recipes, and a demonstration video; an option to upload, review, and rate recipes; and suggestions for quick 10-minute recipes:
I really liked when you did go into a recipe, it said, had like a green yellow system, like a traffic lights system. And then, yeah, it was just really simple recipe that seemed which was really helpful.
The analysis revealed that confusion when using the prototype was the major contributor to the negative user experience, which hindered the participants’ ability to learn how to use the prototype. The participants suggested including embedded instructions to provide guidance on how to use the prototype. Furthermore, the findings suggest that the current navigation design caused this confusion. The poor navigation design affected the performance of the prototype:
It did take a while for me to sort of work out what to do. So maybe if there were some more instructions first up, like if sometimes you go into an app and it sort of you get these navigation comments, go here, go here, go here ... may be a prompt to say this is where you go to.
I ticked some functions work that lagging or contains major technical problems [in the questionnaire]. I don’t think that’s true. They’re not lagging and it doesn’t contain major technical problems. It just takes a little bit of time for me to figure it out. So that is slowing my performance down. It makes it more time consuming to enter.
The feedback on the gestural design of the prototype showed that the participants had high expectations for the functions embedded in the touch screen of a smartphone. For example, the participants scrolled and stretched the images for information:
I couldn’t really see the pictures [on the homepage], like to zoom in to see what it was saying.
Furthermore, the participants were excited about the innovation of image-based dietary assessment in the prototype and valued its potential to reduce user burden for logging dietary intake. However, the participants were concerned about the validity and reliability of the image-based dietary assessment and the slow loading time of the food photos:
I think that’s, that’s [image-based dietary assessment] new. I haven’t seen that before. I think my concern is like kind of, detect ethnic meals, like say if someone would have like a traditional Asian dish, everything mixed together. I’m just wondering, can that detect as compared to like a western dish where everything’s nicely portioned out.
I couldn't upload the photo because too big ... If the one [image-based dietary assessment] is working, yeah, I would like it very much. Because I hate entering it [dietary intake] manually.
Perceived Impact of the Prototype on Healthy Eating
Almost all the participants reported that the prototype was useful for increasing their knowledge and awareness of healthy eating based on the ADG:
Definitely tracking your food and understanding the [Australian Dietary] guidelines.
I think the biggest one would be education about what the dietary guideline is and understanding what food groups are and giving them a reality check of how many of each group was that need to be consuming a day.
The perceived impacts on healthy eating based on the ADG were reported to depend on the use of a gain-framed approach, with a focus on food choices based on the ADG and tracking food choices with immediate feedback:
It shows you how much each of the core food groups that you’ve eaten each day. I think that will be a really useful tool for people. Really simple. And, yeah, that feedback would be really good ... I think you know, there are other apps that are available, but are probably a bit more complicated, and they concentrate on kilojoules ... taking the emphasis back to whole foods and with the push now for, you know, less ultra-processed foods. Yeah, I think there's a really good distinction there between.
However, the dietitians suggested that using the current version of the prototype was unlikely to change the attitude and eating behavior toward healthy eating based on the ADG. Suggested improvements included the provision of personalized tips on food choices and the use of reminders:
For example, if I just saw that I’d had 3.5 serves of meat. And in order to get under that, do I need to cut my meat intake in half or do I need to cut it by a third? So that kind of, in order to make behavior change happen, it’s not clear what I need to do. That’s probably the missing step. And maybe a tip that comes up to help them with that might be helpful. Or even just a motivational message, “great work, you’ve had your five vegetables today, here’s some other examples of servings of veggies,” for example.
For example, if a client, you know, I guess when they’re entering the target goals for food groups, for example, three serves of vegetables a day is their target, and then there is a day where they haven’t met it. And they have the app, and pop up for visual prompts for example, you only consumed, as a reminder, I guess, you know, you’ve only had one of your three target goals. One serve of the vegetables a day, and something like that. Something very simple, but it just reminds them that, oh, I haven’t had enough veggies today.
To our knowledge, this study was the first to describe the detailed development process and design of a PWA to provide real-time, personalized, and evidence-based support for self-managing food choices using persuasive technology, computer vision, and cloud computing. Combining theoretical frameworks with cross-disciplinary collaboration in the fields of nutrition and dietetics, computer science, and information technology have resulted in a novel, evidence-based prototype for personalized dietary advice. The elaborations on the user experience of using the prototype have provided an in-depth understanding of the roles of the design and its potential impact on healthy eating based on the ADG. A revised version of this prototype is currently under development.
The qualitative results suggest that the prototype can be used as a supportive tool to improve the awareness and knowledge of the ADG. The ADG were developed through systematic literature reviews, which follow transparent processes to identify, evaluate, and synthesize relevant available research on specific questions in diet-disease relationships . In the ADG, food modeling was also applied to model the nutrient reference targets based on diet scenarios (eg, foundation diets, total diets, rice-based and pasta-style diets, and plant-based diets) using a linear programming and constraint optimization technique [ ]. The quantitative recommendations bring together all food consumption patterns from national surveys, the recommended dietary intake values, and evidence related to the potential impact of food consumption on health [ ]. Food consumption patterns from national surveys were used in the modeling to account for food supply and food preference. The outcomes of food modeling provided the foundation diets, including 5 food groups, further constructed to form total diets with discretionary foods. However, the food environment and food systems have changed rapidly in recent years, such as increased availability, accessibility, and affordability of energy-dense foods, and intense marketing of such foods has become common [ , ]. Thus, the strength of the ADG lies in their reliance on food modeling based on national food consumption survey data and the body of scientific evidence from published studies on health and disease prevention. This is partly a theoretical strength. In practice, there appears to be insufficient knowledge and consensus among adults in Australia on how to achieve a healthy diet [ , ]. Thus, a supportive tool is required to translate the ADG into daily practice. Incorporating the ADG into the prototype provides users with evidence-based dietary recommendations on food choices. This has improved the quality of the prototype in this study and offered a supportive tool that can work within the food system to provide personalized practical advice for food choices in real time for adherence to the ADG.
In this study, a prototype was successfully developed and integrated the food layer of the NCP, CBT strategies, and the ADG into the actual prototype features guided by the PSD model and a gain-framed approach. The goal of using apps for nutrition intervention tends to automate and advance current evidence-based practice by leveraging the rapid development and maturation of digital tools [, ]. The apps for nutrition interventions tend to provide ongoing support for behavior changes on a daily or on-demand basis by offering the features such as dietary tracking, goal setting, immediate feedback, and environmental support [ ]. The results of the evaluations highlighted that the prototype simplified the process of goal setting and self-monitoring of personal food choices. The capabilities of tracking food choices in the prototype were further enhanced by integrating a state-of-the-art machine learning model for retrieving recipes from food photos. Although dietary intake was still self-reported, its objectivity may be improved owing to the presence of food photos [ ]. Providing real-time feedback on food choices using apps can help users adopt healthy eating by enabling them to immediately observe the link between dietary intake and compliance with their personal goals. This approach appears to encourage healthy eating without coming across as preachy [ ].
Moreover, the qualitative results suggest that the prototype provides a supportive environment for gaining an entertaining and interactive experience in healthy eating based on the ADG by leveraging gamification principles. Gamification in health care has become increasingly popular in recent years . Gamified interventions tend to illustrate target behavior, simplify the intervention to manageable tasks, provide immediate feedback, and reinforce good performance [ ]. Thus, gamification makes complex tasks or goals more engaging and enjoyable and has the potential to empower individuals to make positive behavior changes [ ]. By incorporating the aesthetic design elements into the gamification elements of the prototype, users are more likely to be engaged and motivated. The emotional aspects of interaction design appear to assist in creating a stronger connection between users and the app and encourage users to achieve their goals. Adopting this approach assists in improving adherence to using an app over time.
Despite this, the qualitative findings of the evaluation revealed that high-level personalization is required to change attitudes and eating behavior toward healthy eating based on the ADG. In this study, users set their personal goals for food choices and track their progress of personal goals based on their dietary intake at meals. Thus, the dietary advice received by a user is personalized based on their dietary intake. However, the dietitians have suggested that high-level personalization (eg, specific food choice and its portion size) is required for eating behavior changes. With advances in machine learning, high-level personalized dietary advice can be provided by using collaborative filtering–based methods, content-based methods, context-awareness methods, and hybrid methods . For example, the collaborative filtering–based method has been applied to model the interactions between a user and food items for recommendation [ ]. The context-awareness approach has been used to explore the characteristics of dietary patterns by leveraging information on social media for food recommendations [ ]. Future studies are required to understand users’ needs for high-level personalized dietary advice leveraging digital tools.
Furthermore, the results suggest that navigation was a crucial part of the overall experience of using the prototype. Navigation plays a critical role in learnability, discoverability, and user engagement when using a digital tool . It provides a way for users to access the different features and functionality of the app and helps them find the information they need [ ]. In this study, a discrepancy was observed between participants’ opinions on navigation in the MARS scores and in-depth interviews. This may be due to the limitations of the methods used to evaluate the prototype. Participants may not accurately reflect their true experiences in the survey. The present findings of the in-depth interviews tend to provide a deeper understanding of the user experience.
In this study, although users were not involved in developing the prototype, the prototype was developed using a user-centered design approach. By adopting the user-centered design principle, users become the focus of the design process, and their perspectives are reflected in each step of development [, ]. The requirements of this prototype were developed based on the lead dietitian’s previous work [ , - , ], which informed the design of the features of this prototype. This work has offered novel insights into users’ needs for dietary advice on food choices, which is beyond the traditional methods of gathering information through surveys and interviews with users at the initial stage of development. The previous work has added new dimensions to understand users’ needs for dietary advice on food choices. Involving users in the later stages of prototype development also allowed the team to focus on technology feasibility and rapid prototyping, resulting in a more efficient development process. The qualitative results of the evaluation suggest that the prototype was made available to the participants for evaluation, enabling them to directly engage with it and gain practical experience. This facilitated an in-depth understanding of the user experience and improved communication of users’ needs, thus enhancing the prototype in future iterations. Therefore, involving users in the later stages of prototype development assists in creating a digital health solution that balances users’ needs, requirements, and technical capabilities. The prototype can be a useful tool for both users and health care professionals as it can be readily embedded in the relevant health care system.
This study provides a practical example of integrating a recent, advanced machine learning model for image-to-recipe retrieval into an app for dietary assessment. Computer vision is a rapidly evolving field in recent years. Developing a new state-of-the-art machine learning model for image-to-recipe retrieval is a challenging task, which requires a substantial amount of time, expertise, and resources. In this study, a recently developed machine learning model under an Apache 2.0 license was integrated into the prototype. It not only saves time and cost but also allows for rapid technology feasibility assessment. Furthermore, Graphics Processing Unit (GPU) acceleration is required to improve the performance of image-to-recipe retrieval using the current state-of-the-art model. A GPU is a specialized processing unit that is optimized for processing data in parallel, which is effective for handling large amounts of data simultaneously . Advances in computer vision and natural language processing have enabled the development of more sophisticated models for image-to-recipe retrieval, which have shown promising results in improving the accuracy of retrieval and generate higher-quality results [ ]. However, the implementation of more complex machine learning models can be computationally demanding. Both image processing and natural language processing are required to run the model for image-to-recipe retrieval in this study. Moreover, current smartphones are typically equipped with high-quality cameras to capture high-resolution images. Thus, GPUs may be helpful in speeding up the process to improve the overall performance of image-to-recipe retrieval to assess dietary intake from food photos.
The detailed description of the development process and design of the prototype in this study offer valuable insights into the rapid development of a novel evidence-based mobile app for personalized dietary advice on food choices. Existing studies on developing mobile apps for nutrition interventions tend to lack information on the design of information systems [, ]. Thus, developing an information system solution still requires intuition, experience, and trial-and-error methods [ ]. The development of a mobile app as an experiment in design science research offers comprehensive insights into the nature of the problem and possible solutions for a broad audience [ ]. The prototype developed in this study was formalized as a representation of information system problems, solutions, and solution processes by following the guidelines of the design science paradigm ( ). This formalization helped clarify the knowledge generated by this study.
|Guideline 2: Problem relevance||To create technological solutions that address critical and applicable business issues|
|Guideline 5: Research rigor||To apply rigorous methods in both the construction and evaluation of the design artifact|
|Guideline 6: Design as a search process||To use available means to achieve desired outcome while satisfying laws in the problem environment|
|Guideline 1: Design as an artifact||To generate a viable artifact in the form of a construct, a model, a method, or an instantiation|
|Guideline 3: Design evaluation||To establish the utility, quality, and efficacy of a design artifact by applying well-executed evaluation methods|
|Guideline 4: Research contributions||To offer clear and verifiable contributions in the areas of the design artifact, design foundations, and design methodologies|
|Guideline 7: Communication of research||To effectively communicate the research to both technology-oriented and management-oriented audience|
aADG: Australian Dietary Guidelines.
bmHealth: mobile health.
Participants in the evaluation were recruited via convenience sampling. The findings are not generalizable to the larger population but rather provide insights into the specific context of the study. The sample size of the evaluation was small. Although data saturation was not pursued in this study, previous research has demonstrated that this sample size is effective in identifying the majority of the usability issues [, ]. In this study, a major strength of the evaluation was the triangulation of methods using a combination of quantitative and qualitative methods and analyses. The reported usability problems in this study also overlapped substantially across participants. Qualitative research involves collecting in-depth, open-ended data from a small sample of participants to gain a deeper understanding of their experiences, attitudes, and perceptions [ ]. Thus, conducting in-depth interviews with end users to evaluate a prototype offered a deeper understanding of how users interact with the prototype, including how they use it, what features they find useful, and what challenges they encounter. This provides insights into how to improve the usability and functionality of the prototype to better meet the needs of its users. In addition, qualitative research helps to identify any barriers or challenges that users may face when using the prototype, such as technical issues and navigation difficulties. Understanding these barriers helps to make improvements to the design and functionality of the prototype to address these concerns and improve user experience. Moreover, investigator subjectivity may have influenced the design of the prototype. Although the use of theories tends to be sufficient to guide the design of the prototype, future iterations can benefit from more complex development processes. Evaluating the psychometric properties of image-to-recipe retrieval may be required to gain insights into the validity and reliability of the image-based dietary assessment used in the prototype.
This study describes the detailed development process and design of the prototype to provide personalized, evidence-based support for self-managing food choices from the ADG in real time. Through cross-disciplinary collaboration, a recently developed machine learning model for the image-based dietary assessment was integrated into the prototype. The food layers of the NCP, CBT strategies, and the ADG were then translated into the actual prototype features guided by the PSD model and a gain-framed approach. Participants reported that their experience of using the prototype was enjoyable and engaging. Both the users and dietitians appreciated the novel features of the prototype, such as using image-based dietary assessment and gamification principles for tracking food intake. The detailed description enhances the transparency of the prototype and provides detailed insights into its creation. This study provides valuable insights into developing a novel, evidence-based app for personalized dietary advice on food choices using the recent advancements in computer vision.
This study was supported by Illawarra Health and Medical Research Institute Young Investigator Award, Multiple Sclerosis Australia Incubator Grant (grant 21-1-110), National Computational Infrastructure Adapter Allocation Scheme 2022 Q3, and the University of Wollongong and National Computational Infrastructure Partner Share Scheme. VG was supported by the Multiple Sclerosis Australia Postdoctoral Fellowship (grant 20-223). The funders had no role in the design of the study; in the development, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.
Data cannot be shared publicly because the study analyzed qualitative data, and the participants did not consent to have their full transcripts made publicly available. Data requests may be sent to the principal investigator of the project, VG, or the University of Wollongong and Illawarra Shoalhaven Local Health District Human Research Ethics Committee.
VG conceptualized and designed the study, contributed to prototype development and machine learning algorithm deployment, conducted prototype evaluation, wrote and edited the manuscript, performed project supervision and administration, and acquired resources and funding. CZ, HW, RZ, DZ, and SZ developed the prototype and deployed the machine learning algorithm. WY contributed to prototype development. BPV was responsible for machine learning algorithm deployment. LW designed the study, provided feedback on the manuscript, and acquired resources and funding. KTW designed the study, contributed to prototype evaluation, and provided feedback on the manuscript. PW contributed to methodology, prototype development, machine learning algorithm deployment, provided feedback on the manuscript, performed project supervision, and acquired resources and funding. All the authors agree with the final version of the manuscript.
Conflicts of Interest
- GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the global burden of disease study 2017. Lancet 2019 May 11;393(10184):1958-1972 [https://linkinghub.elsevier.com/retrieve/pii/S0140-6736(19)30041-8] [CrossRef] [Medline]
- National Strategic Framework for Chronic Conditions, reporting framework: indicator results, catalogue number PHE 299. Australian Institute of Health and Welfare. 2022. URL: https://www.aihw.gov.au/getmedia/891c2a30-800b-4117-919f-fde48ef9f2f9/aihw-phe-299.pdf.aspx?inline=true [accessed 2023-02-10]
- Australian Dietary Guidelines. National Health and Medical Research Council. 2013. URL: https://www.nhmrc.gov.au/adg [accessed 2022-12-14]
- Diet. Australian Institute of Health and Welfare. 2022. URL: https://www.aihw.gov.au/reports/australias-health/diet [accessed 2023-02-10]
- Consumption of food groups from the Australian Dietary Guidelines - A comparison of 1995 to 2011-12. Australian Bureau of Statistics. 2017. URL: https://www.abs.gov.au/ausstats/abs@.nsf/Lookup/by%20Subject/4364.0.55.012~2011-12~Main%20Features~Consumption%20of%20Food%20Groups%20from%20the%20Australian%20Dietary%20Guidelines%20-%20a%20comparison%20of%201995%20to%202011-12~10003#:~:text=Average%20consumption%20of%20foods%20from,coming%20from%20greater%20poultry%20consumption [accessed 2020-06-22]
- Poelman MP, Dijkstra SC, Sponselee H, Kamphuis CB, Battjes-Fries MC, Gillebaart M, et al. Towards the measurement of food literacy with respect to healthy eating: the development and validation of the self perceived food literacy scale among an adult sample in the Netherlands. Int J Behav Nutr Phys Act 2018 Jun 18;15(1):54 [https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-018-0687-z] [CrossRef] [Medline]
- Jabs J, Devine CM. Time scarcity and food choices: an overview. Appetite 2006 Sep;47(2):196-204 [CrossRef] [Medline]
- mHealth: new horizons for health through mobile technologies: second global survey on eHealth. World Health Organization. 2011. URL: https://apps.who.int/iris/bitstream/handle/10665/44607/9789241564250_eng.pdf?sequence=1&isAllowed=y [accessed 2023-02-10]
- Tonkin E, Brimblecombe J, Wycherley TP. Characteristics of smartphone applications for nutrition improvement in community settings: a scoping review. Adv Nutr 2017 Mar 15;8(2):308-322 [https://linkinghub.elsevier.com/retrieve/pii/S2161-8313(22)00719-0] [CrossRef] [Medline]
- The Mobile Economy 2022. GSM Association. 2022. URL: https://www.gsma.com/mobileeconomy/wp-content/uploads/2022/02/280222-The-Mobile-Economy-2022.pdf [accessed 2022-07-20]
- Scarry A, Rice J, O'Connor EM, Tierney AC. Usage of mobile applications or mobile health technology to improve diet quality in adults. Nutrients 2022 Jun 12;14(12):2437 [https://www.mdpi.com/resolver?pii=nu14122437] [CrossRef] [Medline]
- McCarroll R, Eyles H, Ni Mhurchu C. Effectiveness of mobile health (mHealth) interventions for promoting healthy eating in adults: a systematic review. Prev Med 2017 Dec;105:156-168 [CrossRef] [Medline]
- Fakih El Khoury C, Karavetian M, Halfens RJ, Crutzen R, Khoja L, Schols JM. The effects of dietary mobile apps on nutritional outcomes in adults with chronic diseases: a systematic review and meta-analysis. J Acad Nutr Diet 2019 Apr;119(4):626-651 [CrossRef] [Medline]
- Villinger K, Wahl DR, Boeing H, Schupp HT, Renner B. The effectiveness of app-based mobile interventions on nutrition behaviours and nutrition-related health outcomes: a systematic review and meta-analysis. Obes Rev 2019 Oct;20(10):1465-1484 [https://europepmc.org/abstract/MED/31353783] [CrossRef] [Medline]
- Iribarren SJ, Akande TO, Kamp KJ, Barry D, Kader YG, Suelzer E. Effectiveness of mobile apps to promote health and manage disease: systematic review and meta-analysis of randomized controlled trials. JMIR Mhealth Uhealth 2021 Jan 11;9(1):e21563 [https://mhealth.jmir.org/2021/1/e21563/] [CrossRef] [Medline]
- Akinosun AS, Polson R, Diaz-Skeete Y, De Kock JH, Carragher L, Leslie S, et al. Digital technology interventions for risk factor modification in patients with cardiovascular disease: systematic review and meta-analysis. JMIR Mhealth Uhealth 2021 Mar 03;9(3):e21061 [https://mhealth.jmir.org/2021/3/e21061/] [CrossRef] [Medline]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [https://www.jmir.org/2012/6/e152/] [CrossRef] [Medline]
- Asbjørnsen RA, Smedsrød ML, Solberg Nes L, Wentzel J, Varsi C, Hjelmesæth J, et al. Persuasive system design principles and behavior change techniques to stimulate motivation and adherence in electronic health interventions to support weight loss maintenance: scoping review. J Med Internet Res 2019 Jun 21;21(6):e14265 [https://www.jmir.org/2019/6/e14265/] [CrossRef] [Medline]
- Sharp H, Rogers Y, Preece J. Interaction Design: Beyond Human-Computer Interaction. 5th edition. Indianapolis, IN: Wiley; 2019.
- Hassenzahl M, Tractinsky N. User experience - a research agenda. Behav Inf Technol 2006 Mar;25(2):91-97 [https://www.tandfonline.com/doi/abs/10.1080/01449290500330331] [CrossRef]
- Fogg BJ. Persuasive Technology: Using Computers to Change What We Think and Do. Cambridge, MA: Morgan Kaufmann; 2003.
- Oinas-Kukkonen H, Harjumaa M. Persuasive systems design: key issues, process model, and system features. Commun Assoc Inf Syst 2009;24(28):485-500 [https://aisel.aisnet.org/cgi/viewcontent.cgi?article=3424&context=cais] [CrossRef]
- Miller AS, Cafazzo JA, Seto E. A game plan: gamification design principles in mHealth applications for chronic disease management. Health Informatics J 2016 Jun;22(2):184-193 [https://journals.sagepub.com/doi/10.1177/1460458214537511?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub 0pubmed] [CrossRef] [Medline]
- Appleton KM, Passmore D, Burn I, Pidgeon H, Nation P, Boobyer C, et al. An interactive mobile phone app (SMART 5-A-DAY) for increasing knowledge of and adherence to fruit and vegetable recommendations: development and pilot randomized controlled trial. JMIR Mhealth Uhealth 2019 Nov 20;7(11):e14380 [https://mhealth.jmir.org/2019/11/e14380/] [CrossRef] [Medline]
- Mummah SA, King AC, Gardner CD, Sutton S. Iterative development of Vegethon: a theory-based mobile app intervention to increase vegetable consumption. Int J Behav Nutr Phys Act 2016 Aug 08;13:90 [https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-016-0400-z] [CrossRef] [Medline]
- Hendrie GA, James-Martin G, Williams G, Brindal E, Whyte B, Crook A. The development of VegEze: smartphone app to increase vegetable consumption in Australian adults. JMIR Form Res 2019 Mar 27;3(1):e10731 [https://formative.jmir.org/2019/1/e10731/] [CrossRef] [Medline]
- Deterding S, Sicart M, Nacke LE, O'Hara KP, Dixon DH. Gamification. Using game-design elements in non-gaming contexts. In: Proceedings of the 2011 Extended Abstracts on Human Factors in Computing Systems. 2011 Presented at: CHI EA '11; May 7-12, 2011; Vancouver, Canada p. 2425-2428 URL: https://dl.acm.org/doi/10.1145/1979742.1979575 [CrossRef]
- McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012;Suppl:49-56 [CrossRef] [Medline]
- Hone T, Palladino R, Filippidis FT. Association of searching for health-related information online with self-rated health in the European Union. Eur J Public Health 2016 Oct;26(5):748-753 [http://hdl.handle.net/10044/1/30113] [CrossRef] [Medline]
- Bevens W, Weiland T, Gray K, Jelinek G, Neate S, Simpson-Yap S. Attrition within digital health interventions for people with multiple sclerosis: systematic review and meta-analysis. J Med Internet Res 2022 Feb 09;24(2):e27735 [https://www.jmir.org/2022/2/e27735/] [CrossRef] [Medline]
- Torous J, Nicholas J, Larsen ME, Firth J, Christensen H. Clinical review of user engagement with mental health smartphone apps: evidence, theory and improvements. Evid Based Ment Health 2018 Aug;21(3):116-119 [CrossRef] [Medline]
- Marcu G, Ondersma SJ, Spiller AN, Broderick BM, Kadri R, Buis LR. Barriers and considerations in the design and implementation of digital behavioral interventions: qualitative analysis. J Med Internet Res 2022 Mar 30;24(3):e34301 [https://www.jmir.org/2022/3/e34301/] [CrossRef] [Medline]
- Jaeger PT, Bertot JC, Thompson KM, Katz SM, DeCoster EJ. The intersection of public policy and public access: digital divides, digital literacy, digital inclusion, and public libraries. Public Libr Q 2012 Jan;31(1):1-20 [https://www.tandfonline.com/doi/abs/10.1080/01616846.2012.654728] [CrossRef]
- Arigo D, Jake-Schoffman DE, Wolin K, Beckjord E, Hekler EB, Pagoto SL. The history and future of digital health in the field of behavioral medicine. J Behav Med 2019 Feb;42(1):67-83 [https://europepmc.org/abstract/MED/30825090] [CrossRef] [Medline]
- Khazen W, Jeanne JF, Demaretz L, Schäfer F, Fagherazzi G. Rethinking the use of mobile apps for dietary assessment in medical research. J Med Internet Res 2020 Jun 18;22(6):e15619 [https://www.jmir.org/2020/6/e15619/] [CrossRef] [Medline]
- Spahn JM, Reeves RS, Keim KS, Laquatra I, Kellogg M, Jortberg B, et al. State of the evidence regarding behavior change theories and strategies in nutrition counseling to facilitate health and food behavior change. J Am Diet Assoc 2010 Jun;110(6):879-891 [CrossRef] [Medline]
- Skinner BF. The Behavior of Organisms. New York, NY: Appleton-Century-Crofts; 1938.
- Ordovas JM, Ferguson LR, Tai ES, Mathers JC. Personalised nutrition and health. BMJ 2018 Jun 13;361:bmj.k2173 [https://europepmc.org/abstract/MED/29898881] [CrossRef] [Medline]
- Guan VX, Probst YC, Neale EP, Batterham MJ, Tapsell LC. Identifying usual food choices at meals in overweight and obese study volunteers: implications for dietary advice. Br J Nutr 2018 Aug;120(4):472-480 [CrossRef] [Medline]
- Guan VX, Neale EP, Tapsell LC, Probst YC. Identifying usual food choice combinations with walnuts: analysis of a 2005-2015 clinical trial cohort of overweight and obese adults. Front Nutr 2020 Sep 23;7:149 [https://europepmc.org/abstract/MED/33072791] [CrossRef] [Medline]
- Guan VX, Neale EP, Probst YC. Identifying usual food choices with avocados in a clinical trial cohort of overweight and obese adults in Australia. PLoS One 2023 Jan 26;18(1):e0279567 [https://dx.plos.org/10.1371/journal.pone.0279567] [CrossRef] [Medline]
- Samad S, Ahmed F, Naher S, Kabir MA, Das A, Amin S, et al. Smartphone apps for tracking food consumption and recommendations: evaluating artificial intelligence-based functionalities, features and quality of current apps. Intell Syst Appl 2022 Sep;15:200103 [https://www.sciencedirect.com/science/article/pii/S2667305322000412] [CrossRef]
- Swan WI, Vivanti A, Hakel-Smith NA, Hotson B, Orrevall Y, Trostler N, et al. Nutrition care process and model update: toward realizing people-centered care and outcomes management. J Acad Nutr Diet 2017 Dec;117(12):2003-2014 [CrossRef] [Medline]
- Guan VX, Probst YC, Neale EP, Tapsell LC. Evaluation of the dietary intake data coding process in a clinical setting: implications for research practice. PLoS One 2019 Aug 12;14(8):e0221047 [https://dx.plos.org/10.1371/journal.pone.0221047] [CrossRef] [Medline]
- Guan VX, Probst YC, Neale EP, Tapsell LC. Predictors for misreporting sodium and potassium intakes by overweight and obese participants in a food-based clinical trial: implications for practice. Eur J Clin Nutr 2020 Jan;74(1):203-207 [CrossRef] [Medline]
- Guan V, Probst Y, Neale E, Martin A, Tapsell L. A systematic method to evaluate the dietary intake data coding process used in the research setting. J Food Compost Anal 2017 Dec;64:27-32 [https://www.sciencedirect.com/science/article/abs/pii/S0889157517301643] [CrossRef]
- Eicher-Miller HA, Prapkree L, Palacios C. Expanding the capabilities of nutrition research and health promotion through mobile-based applications. Adv Nutr 2021 Jun 01;12(3):1032-1041 [https://linkinghub.elsevier.com/retrieve/pii/S2161-8313(22)00128-4] [CrossRef] [Medline]
- Min W, Jiang S, Liu L, Rui Y, Jain RC. A survey on food computing. ACM Comput Surv 2019 Sep 13;52(5):1-36 [https://dl.acm.org/doi/10.1145/3329168] [CrossRef]
- Lo FP, Sun Y, Qiu J, Lo B. Image-based food classification and volume estimation for dietary assessment: a review. IEEE J Biomed Health Inform 2020 Jul;24(7):1926-1939 [CrossRef] [Medline]
- Salvador A, Gundogdu E, Bazzani L, Donoser M. Revamping cross-modal recipe retrieval with hierarchical transformers and self-supervised learning. In: Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021 Presented at: CVPR '21; June 20-25, 2021; Nashville, TN p. 15470-15479 URL: https://ieeexplore.ieee.org/document/9577849 [CrossRef]
- Hevner AR, March ST, Park J, Ram S. Design science in information systems research. MIS Q 2004 Mar;28(1):75-105 [https://www.jstor.org/stable/25148625] [CrossRef]
- Guan VX. Dietary intake data quality in the clinical research setting: Implications of quality improvement. Doctor of Philosophy Thesis: University of Wollongong. 2019. URL: https://ro.uow.edu.au/theses1/488/ [accessed 2023-02-10]
- Gallagher KM, Updegraff JA. Health message framing effects on attitudes, intentions, and behavior: a meta-analytic review. Ann Behav Med 2012 Feb;43(1):101-116 [CrossRef] [Medline]
- Lerner J. The scope of open source licensing. J Law Econ Organ 2005 Apr 01;21(1):20-56 [https://academic.oup.com/jleo/article-abstract/21/1/20/883680?redirectedFrom=fulltext] [CrossRef]
- Marin J, Biswas A, Ofli F, Hynes N, Salvador A, Aytar Y, et al. Recipe1M+: a dataset for learning cross-modal embeddings for cooking recipes and food images. IEEE Trans Pattern Anal Mach Intell 2021 Jan 01;43(1):187-203 [CrossRef] [Medline]
- Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019 Presented at: NIPS '19; December 8-14, 2019; Vancouver, Canada p. 8026-8037 URL: https://dl.acm.org/doi/10.5555/3454287.3455008 [CrossRef]
- AUSNUT 2011-13 - Australian Food, Supplement and Nutrient Database for Estimation of Population Nutrient Intakes. Food Standards Australia New Zealand. 2014. URL: https://www.foodstandards.gov.au/science/monitoringnutrients/ausnut/ausnutdatafiles/pages/foodnutrient.aspx [accessed 2015-05-25]
- Probst Y, Guan V, Neale E. Development of a choline database to estimate Australian population intakes. Nutrients 2019 Apr 23;11(4):913 [https://www.mdpi.com/resolver?pii=nu11040913] [CrossRef] [Medline]
- Charrondiere UR, Rittenschober D, Nowak V, Stadlmayr B, Wijesinha-Bettoni R, Haytowitz D. INFOODS Guidelines for Food Matching. Food and Agriculture Organization. 2012. URL: https://www.fao.org/fileadmin/templates/food_composition/documents/upload/INFOODSGuidelinesforFoodMatching_version_1_2.pdf [accessed 2023-02-10]
- Gu D, Yang X, Deng S, Liang C, Wang X, Wu J, et al. Tracking knowledge evolution in cloud health care research: knowledge map and common word analysis. J Med Internet Res 2020 Feb 25;22(2):e15142 [https://www.jmir.org/2020/2/e15142/] [CrossRef] [Medline]
- Biørn-Hansen A, Majchrzak T, Grønli TM. Progressive web apps: the possible web-native unifier for mobile development. In: Proceedings of the 13th International Conference on Web Information Systems and Technologies. 2017 Presented at: WEBIST '17; April 25-27, 2017; Porto, Portugal p. 344-351 URL: https://www.scitepress.org/papers/2017/63537/63537.pdf [CrossRef]
- Rêgo F, Portela F, Santos MF. Towards PWA in healthcare. Procedia Comput Sci 2019;160:678-683 [https://www.sciencedirect.com/science/article/pii/S1877050919317284] [CrossRef]
- Grinberg M. Flask Web Development: Developing Web Applications with Python. Sebastopol, CA: O′Reilly Media; 2018.
- Agrawal H, Mathialagan CS, Goyal Y, Chavali N, Banik P, Mohapatra A, et al. CloudCV: large-scale distributed computer vision as a cloud service. In: Hua G, Hua XS, editors. Mobile Cloud Visual Media Computing: From Interaction to Service. Cham, Switzerland: Springer; 2015:265-290
- Veit A, Kovacs B, Bell S, McAuley JJ, Bala K, Belongie S. Learning visual clothing style with heterogeneous dyadic co-occurrences. In: Proceedings of the 2015 IEEE International Conference on Computer Vision. 2015 Presented at: ICCV '15; December 7-13, 2015; Santiago, Chile p. 4642-4650 URL: https://ieeexplore.ieee.org/document/7410884 [CrossRef]
- Sheela T, Sowmyaa VR, Karthika A, Shivani B. Cloud based e-commerce application for organic fertilizers, pesticides and other products and crop disease identification using computer vision. In: Proceedings of the 2022 International Conference on Computer Communication and Informatics. 2022 Presented at: ICCCI '22; January 25-27, 2022; Coimbatore, India p. 1-3 URL: https://ieeexplore.ieee.org/document/9740833 [CrossRef]
- Madhyastha TM, Koh N, Day TK, Hernández-Fernández M, Kelley A, Peterson DJ, et al. Running neuroimaging applications on Amazon web services: how, when, and at what cost? Front Neuroinform 2017 Nov 03;11:63 [https://europepmc.org/abstract/MED/29163119] [CrossRef] [Medline]
- Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the mobile application rating scale (uMARS). JMIR Mhealth Uhealth 2016 Jun 10;4(2):e72 [https://mhealth.jmir.org/2016/2/e72/] [CrossRef] [Medline]
- White BK, Martin A, White JA, Burns SK, Maycock BR, Giglia RC, et al. Theory-based design and development of a socially connected, gamified mobile app for men about breastfeeding (Milk Man). JMIR Mhealth Uhealth 2016 Jun 27;4(2):e81 [https://mhealth.jmir.org/2016/2/e81/] [CrossRef] [Medline]
- Paldán K, Steinmetz M, Simanovski J, Rammos C, Ullrich G, Jánosi RA, et al. Supervised exercise therapy using mobile health technology in patients with peripheral arterial disease: pilot randomized controlled trial. JMIR Mhealth Uhealth 2021 Aug 16;9(8):e24214 [https://mhealth.jmir.org/2021/8/e24214/] [CrossRef] [Medline]
- Woods LS, Duff J, Roehrer E, Walker K, Cummings E. Patients' experiences of using a consumer mHealth app for self-management of heart failure: mixed-methods study. JMIR Hum Factors 2019 May 02;6(2):e13009 [https://humanfactors.jmir.org/2019/2/e13009/] [CrossRef] [Medline]
- Win H, Russell S, Wertheim BC, Maizes V, Crocker R, Brooks AJ, et al. Mobile app intervention on reducing the myeloproliferative neoplasm symptom burden: pilot feasibility and acceptability study. JMIR Form Res 2022 Mar 31;6(3):e33581 [https://formative.jmir.org/2022/3/e33581/] [CrossRef] [Medline]
- Saldaña J. The Coding Manual for Qualitative Researchers. Thousand Oaks, CA: SAGE Publications; 2015.
- Australian Health Survey: Users' Guide, 2011-13. Australian Bureau of Statistics. 2014. URL: https://www.abs.gov.au/ausstats/abs@.nsf/Lookup/4363.0.55.001Chapter65062011-13 [accessed 2019-10-15]
- Assessing the 2011-13 AHS against the Australian dietary guidelines. Food Standards Australia and New Zealand. 2017. URL: https://www.foodstandards.gov.au/science/monitoringnutrients/australianhealthsurveyandaustraliandietaryguidelines/Pages/default.aspx [accessed 2023-07-19]
- Australian Guide to Healthy Eating - poster Canberra: NHMRC. National health and Medical Research Council. 2013. URL: https://www.eatforhealth.gov.au/sites/default/files/2022-11/n55i_australian_guide_to_healthy_eating_0.pdf [accessed 2023-02-10]
- Australian Guide to Healthy Eating. National Health and Medical Research Council. 2013. URL: https://www.eatforhealth.gov.au/guidelines/australian-guide-healthy-eating [accessed 2023-02-10]
- Thorpe MG, Milte CM, Crawford D, McNaughton SA. A revised Australian dietary guideline index and its association with key sociodemographic factors, health behaviors and body mass index in peri-retirement aged adults. Nutrients 2016 Mar 11;8(3):160 [https://www.mdpi.com/resolver?pii=nu8030160] [CrossRef] [Medline]
- Garett R, Young SD. Health care gamification: a study of game mechanics and elements. Technol Knowl Learn 2018 Jan 29;24(3):341-353 [https://link.springer.com/article/10.1007/s10758-018-9353-4] [CrossRef]
- A modelling system to inform the revision of the Australian Guide to Healthy Eating. National Health and Medical Research Council. 2011. URL: https://www.eatforhealth.gov.au/sites/default/files/files/the_guidelines/n55c_dietary_guidelines_food_modelling.pdf [accessed 2023-02-10]
- Swinburn BA, Sacks G, Hall KD, McPherson K, Finegood DT, Moodie ML, et al. The global obesity pandemic: shaped by global drivers and local environments. Lancet 2011 Aug 27;378(9793):804-814 [CrossRef] [Medline]
- Riesenberg D, Backholer K, Zorbas C, Sacks G, Paix A, Marshall J, et al. Price promotions by food category and product healthiness in an Australian supermarket chain, 2017-2018. Am J Public Health 2019 Oct;109(10):1434-1439 [CrossRef] [Medline]
- Ting DS, Carin L, Dzau V, Wong TY. Digital technology and COVID-19. Nat Med 2020 Apr;26(4):459-461 [https://europepmc.org/abstract/MED/32284618] [CrossRef] [Medline]
- Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, et al. Digital technologies in the public-health response to COVID-19. Nat Med 2020 Aug;26(8):1183-1192 [CrossRef] [Medline]
- Krath J, Schürmann L, von Korflesch HF. Revealing the theoretical basis of gamification: a systematic review and analysis of theory in research on gamification, serious games and game-based learning. Comput Human Behav 2021 Dec;125:106963 [CrossRef]
- Robson K, Plangger K, Kietzmann JH, McCarthy I, Pitt L. Game on: engaging customers and employees through gamification. Bus Horiz 2016 Jan;59(1):29-36 [https://www.sciencedirect.com/science/article/abs/pii/S0007681315000968] [CrossRef]
- Kalbach J, Gustafson A. Designing Web Navigation: Optimizing the User Experience. Sebastopol, CA: O'Reilly Media; 2007.
- Based web design & usability guidelines. U.S. Department of Health and Human Services. 2006. URL: https://www.usability.gov/sites/default/files/documents/guidelines_book.pdf [accessed 2023-07-19]
- Maguire M. Methods to support human-centred design. Int J Hum Comput Stud 2001 Oct;55(4):587-634 [https://www.sciencedirect.com/science/article/abs/pii/S1071581901905038] [CrossRef]
- Sanders L. An evolving map of design practice and design research. interactions 2008 Nov;15(6):13-17 [https://www.dubberly.com/wp-content/uploads/2008/11/ddo_article_evolvingmap.pdf] [CrossRef]
- Owens JD, Houston M, Luebke D, Green S, Stone JE, Phillips JC. GPU computing. Proc IEEE 2008 May;96(5):879-899 [https://ieeexplore.ieee.org/document/4490127] [CrossRef]
- Bastien JC. Usability testing: a review of some methodological and technical aspects of the method. Int J Med Inform 2010 Apr;79(4):e18-e23 [CrossRef] [Medline]
- Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inform 2019 Jun;126:95-104 [CrossRef] [Medline]
- Hennink M, Hutter I, Bailey A. Qualitative Research Methods. Thousand Oaks, CA: SAGE Publications; 2010.
|ADG: Australian Dietary Guidelines|
|APD: Accredited Practising Dietitian|
|AUSNUT: Australian Food, Supplement and Nutrient Database|
|AWS: Amazon Web Services|
|CBT: cognitive behavioral theory|
|EC2: Elastic Compute Cloud|
|GPU: Graphics Processing Unit|
|MARS: Mobile Application Rating Scale|
|mHealth: mobile health|
|NCP: Nutrition Care Process|
|PSD: Persuasive Systems Design|
|PWA: progressive web app|
Edited by A Mavragani; submitted 27.02.23; peer-reviewed by I Donadello, C Tziraki, L Loeb; comments to author 11.04.23; revised version received 23.04.23; accepted 10.05.23; published 07.08.23Copyright
©Vivienne Guan, Chenghuai Zhou, Hengyi Wan, Rengui Zhou, Dongfa Zhang, Sihan Zhang, Wangli Yang, Bhanu Prakash Voutharoja, Lei Wang, Khin Than Win, Peng Wang. Originally published in JMIR Formative Research (https://formative.jmir.org), 07.08.2023.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.