Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 04.11.19 in Vol 3, No 4 (2019): Oct-Dec

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/12966, first published Nov 28, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    A Mobile Phone App for the Provision of Personalized Food-Based Information in an Eating-Out Situation: Development and Initial Evaluation

    1Research Centre for Behaviour Change, Department of Psychology, Faculty of Science and Technology, Bournemouth University, Poole, United Kingdom

    2Faculty of Management, Bournemouth University, Poole, United Kingdom

    3Faculty of Science and Technology, Bournemouth University, Poole, United Kingdom

    4University of Macedonia, Thessaloniki, Greece

    5Food and Hospitality Research Center, Institut Paul Bocuse, Ecully, France

    6Department of Food Science, University of Copenhagen, Copenhagen, Denmark

    7Ronge and Partner, Baden-bei-Wien, Austria

    Corresponding Author:

    Katherine Marie Appleton, PhD

    Research Centre for Behaviour Change, Department of Psychology

    Faculty of Science and Technology

    Bournemouth University

    Poole House

    Fern Barrow

    Poole, BH12 5BB

    United Kingdom

    Phone: 44 1202 965985

    Email: k.appleton@bournemouth.ac.uk


    ABSTRACT

    Background: Increasing pressure from governments, public health bodies, and consumers is driving a need for increased food-based information provision in eating-out situations. Meals eaten outside the home are known to be less healthy than meals eaten at home, and consumers can complain of poor information on the health impact and allergen content of meals eaten out.

    Objective: This paper aimed to describe the development and early assessment of a mobile phone app that allows the provision of accurate personalized food-based information while considering individual characteristics (allergies, diet type, and preferences) to enable informed consumer choice when eating out.

    Methods: An app was designed and developed to address these requirements using an agile approach. The developed app was then evaluated at 8 public engagement events using the System Usability Scale (SUS) questionnaire and qualitative feedback.

    Results: Consideration of the literature and consultation with consumers revealed a need for information provision for consumers in the eating-out situation, including the ability to limit the information provided to that which was personally relevant or interesting. The app was designed to provide information to consumers on the dishes available in a workplace canteen and to allow consumers the freedom to personalize the app and choose the information that they received. Evaluation using the SUS questionnaire revealed positive responses to the app from a range of potential users, and qualitative comments demonstrated broad interest in its use.

    Conclusions: This paper details the successful development and early assessment of a novel mobile phone app designed to provide food-based information in an eating-out situation in a personalized manner.

    JMIR Form Res 2019;3(4):e12966

    doi:10.2196/12966

    KEYWORDS



    Introduction

    Eating Out

    Eating out (defined as eating food that is prepared by others and consumed out of the home in food establishments such as restaurants, cafes, canteens, and fast food outlets) is a growing trend [1-3]. Food consumed out of the home can represent a significant contribution to daily energy intake [4,5] and is often higher in calories, fat, and sugar, lower in fiber, and served in larger portions than food consumed in the home [4,5]. It has been suggested that both the increasing practice of eating out and the increasing energy provision in this situation contribute to the obesogenic environment [1,2,6-8], and positive associations between the increased prevalence of eating out and high body weight in recent years have been made [6,8]. Obesity and its related conditions constitute a major public health concern, and strategies for prevention are required [9]. Furthermore, most adults underestimate their calorie intake when eating out and are therefore unlikely to mitigate their own risk [1,2].

    Food-Based Information

    With an aim of reducing overeating, a regulation within Europe (EU No 1169/2011) makes it mandatory for all prepacked foods to display energy value and amounts of fat, saturates, carbohydrates, protein, sugars, and salt per 100 g or 100 ml food item [10]. Presently, only the United States has passed legislation requiring the provision of this type of labeling for out-of-home food provision [11], although voluntary labeling, particularly of energy content, is undertaken in various countries across the world [12]. Importantly, however, consumers are also asking for increased food-based information in the eating-out scenario [13-17], and the provision of information has been found to facilitate the adoption of healthier nutrition practices in this scenario [1-3,15,16,18,19]. Bates et al [1] and Burton et al [2] report lower repurchase intentions for unhealthy dishes following exposure to objective calorie and nutrient information. VanEpps et al [19] report fewer calories ordered following the provision of nutrition information, and Hammond et al [18] report fewer calories ordered and consumed following a nutrition-labeling intervention. Not all studies have reported benefits [20], but effects are typically small, and a lack of effects has been largely attributed to poor study size or methodology [20].

    Food-based information provision in an eating-out situation, however, can result in menus or information boards that are described as cluttered and untidy, the consequences of which are that consumers feel overwhelmed and report that they would not use the information provided [13,16,17,21]. Consumers can report an unwillingness to search for information or high costs of this practice compared with benefits [1,22,23]. Food operators also acknowledge constraints to providing ingredient information on menus [15,16,21], including a reluctance to overload the menu with visual clutter, as well as a lack of knowledge on the part of the operator and a high perceived cost to the business, particularly where menus and dish specifications may change frequently [15,16,21].

    Furthermore, different consumers can prefer or typically act on different types of information [2,16,17,22,24]. Yepes [24], for example, found menu calorie labeling was most valued by health conscious and older consumers and was effective in reducing calorie consumption in these individuals, whereas Burton et al [2] demonstrate impacts of calorie information on low-health consciousness consumers and impacts of calorie information and a color coding system for both low- and high-health consciousness consumers. Ellison et al [3] also found increased impacts of calorie information and a color-coding system compared with calorie-based information alone. The lack of effects of food information in some studies has been attributed to the type of information provided combined with the abilities of the consumer [20,25]. Different types of information may also be differentially effective in different settings. Bleich et al [20] and Dumanovsky et al [25] both report increased impacts of calorie information in fast food restaurants compared with sit-down type establishments. Food-based decisions are often made using simple heuristic processing methods [25], and decisions that are required under time pressure and high cognitive load can further be reliant on simple or automatic processing strategies, such as a calorie limit. Decisions that take place in a more relaxed setting, however, such as at home, can be more consciously taken and so more informed, for example, through the consideration of complex ingredient information [19,25].

    Food-Based Information Provision

    For decades, printed mediums have been the most common platform used for information provision in an eating-out scenario, but digital or electronic menus can deliver rich knowledge in an efficient manner [16,26] and may offer a neat solution to the existing constraints highlighted earlier [16,22,26]. Several studies have highlighted the potential that technology may hold in providing information [16,22], and a small number of mobile phone apps have been developed specifically for providing food-based information [27,28]. SmartAPPetite encourages people to eat local and healthy food [29]. Tapingo provides university students with canteen-based information and allows consumers to order food [30], and the Smartmenu system discussed by Pieskä et al [31] allows consumers to browse a menu, check additional information such as nutrient profile, and order items. The potential value of mobile phone apps for providing information is also increased by the rapidly growing numbers of mobile phones. Penetration rates of 68.4% in North America and 64.7% in Western Europe have been reported, with estimations of use by 32% of the global population [32], ranging recently in the top 50 markets between 11.2% in Ethiopia and 82.2% in the United Kingdom [33].

    Increased information provision via digital platforms may also enable transparency and evidence of greater integrity for the food service operator [16,17,26,34]. Consumers with specific dietary needs are often limited in their choices not just by their personal constraints but also by a lack of information available from serving staff or a lack of trust in the information provided [22,26,35]. Lack of control, insufficient knowledge, and a lack of trust in any information provided can be key concerns when eating out [22,23,26,34,36]. Trust is an important component of health-based decision making [34-36], and catering operators that are open and transparent demonstrate commitment and trustworthiness to consumers [17,34]. Furthermore, even if the actual content is not always used, consumers can be reassured by the presence of such information [37,38]. Thus, food operators will also potentially benefit from increased information provision [16,21,26].

    To enable healthy decision making in an eating-out situation, communication with consumers is clearly required, but any such communication should be carefully considered to ensure that it is well understood, suitable for each consumer, and suited to specific dishes and food operators [1-3,16,17,22,24,28]. Many existing apps use generic recipe data and so provide only generic information [27,28,39]. This generic information typically provides only estimations of nutrient content and is rarely sufficiently detailed to protect consumers with allergies or other very specific dietary requirements. For increased specificity, information should ideally be provided for every specific recipe from every specific manufacturer or food operator. For many consumers, more detailed and specific information is preferable, and more detailed information could increase trust and return business for food providers. For allergens, very specific information is required.

    A Mobile Phone–Based Technological Solution

    This study aimed to develop a mobile phone app to provide consumers with food-based information in an eating-out situation. The app was intended to provide information as required for current European legislation and as recommended by a number of public health bodies, also with the consideration of consumer desires. The study was undertaken for the workplace canteen situation. There is a growing acceptance that the food provided in a workplace canteen setting can have a significant impact on health [40], as this is a captive environment where the contribution of the meal served could constitute an important element of the overall diet owing to the frequency of use. It is estimated that most employees eat one or more meals per day while they are at work [41]. Promotion of healthy diets in the workplace will also have benefits not only for the individual but also for employers and society [40]. The workplace canteen also typically offers limited dishes that can be more easily, accurately prespecified than may be the case for a chef-led restaurant [21]. The study was undertaken in Europe with a focus on European consumers, but evaluations also extended the study beyond Europe.


    Methods

    Overview

    The study was undertaken in 3 stages. First, the requirements for the app were elicited through consideration of current legislative and scientific literature and consultation with potential end users, and then prioritized with potential stakeholders using the Must have, Should have, Could have, Won’t have (MoSCoW) method [42]. Second, a mobile phone app was designed and developed using an agile approach. Finally, the developed app was evaluated at 8 public engagement events using the System Usability Scale (SUS) questionnaire [43,44] and qualitative feedback. Full ethical approval was granted from Bournemouth University Research Ethics Committee before commencement. The research complied with Directive 95/46/EC of the European Parliament on the Protection of Individuals and with Directive 2002/58/EC of the European Parliament concerning the processing of personal data and the protection of privacy.

    Stage 1: Defining the App Requirements

    Eliciting Government Requirements and Recommendations

    Current legislation and recommendations for Europe were obtained from the European Union and from relevant public health agencies in Europe.

    Eliciting Consumer Requirements

    Consumer requirements were gained from a search of the scientific literature and from consultations with consumers. The search of the scientific literature was undertaken using known articles and snowballing from these by also looking at cited and citing articles. Consumer consultations were undertaken using one mixed methods study comprising a qualitative and then quantitative component [45,46], and one qualitative study [47]. A formal review of existing apps was not undertaken as part of the development work. No comparable apps with the necessary specificity and flexibility of dish information existed as far as we were aware, and few apps are developed following (reported) formal consultation at an early stage with end users; thus, a review of existing apps was unlikely to be informative.

    The consumer consultation work has been published in detail elsewhere [45-47]. Briefly, the mixed methods study [45,46] used 8 focus groups of canteen users (N=40), 2 groups in each of the 4 European countries (Denmark, France, Greece, and the United Kingdom), to elicit the criteria used for making food-based decisions in a canteen scenario and known formats of food-based information provision. The decision-related criteria were then used in a best-worst scaling questionnaire to ascertain the relative importance of these criteria for making food-based decisions in 452 employees (Denmark [N=100], France [N=100], Greece [N=100], the United Kingdom [N=152]), who had access to a canteen at their place of work. The full sample was largely composed of females (61.1%, 276/452), aged 20 to 29 years (51.3%, 232/452), who had completed some form of higher tertiary education (74.1%, 335/452), and who now worked full time (60.4%, 273/452) in occupations classified as technicians and associate professionals (74.1%, 335/452). The known formats of and preferences for food information were also considered in a second best-worst scaling questionnaire administered to the same individuals to ascertain the most preferred format for the provision of food information. The study was undertaken in several European countries to enhance the generalizability of the work. Focus groups were used in a small sample to ascertain relevant criteria of canteen use and consumer preferences for food information provision in depth, given a lack of existing data in this area. The questionnaire was then used to determine the relative importance of these criteria and information formats in a much larger sample. The best-worst scaling method requires respondents to choose their most preferred and least preferred option of several sets, allowing relative assessments of the criteria of interest without the use of absolute judgments that can differ between contexts and cultures.

    The additional qualitative study [47] used 4 focus groups of canteen users from the United Kingdom (N=28) to confirm the information desired for food-based decisions in a canteen scenario and ascertain attitudes and opinions toward the use of information communication technology (ICT) in this context. Focus groups were again used in a small sample to ascertain attitudes and opinions in depth, given a lack of existing data in this area.

    Prioritizing Requirements

    Elicited requirements for the app were then prioritized using MoSCoW principles following full consideration by the research team and potential stakeholders to ensure wide use of the app and increased transferability. The MoSCoW method [42] is a technique used in software development to prioritize the importance of the delivery of all identified requirements. Requirements are categorized as Must have, Should have, Could have, and Won’t have, based on importance, and then prioritized during the development process in this order. Requirements identified as must have are considered central to project success; those identified as should have are considered important, but not necessary; those identified as could have are considered desirable but not necessary; and those identified as won’t have are considered least important [42]. The research team included academic researchers in eating behaviors, hospitality, and food service (KMA, JB, SP, FJAPC, and HH), academic researchers and developers in computing and app development (GL, NJ, and IM), caterers, food operators, and personnel from the catering industry (RC and MR) and researchers working within the food industry (LS and AG). Additional potential stakeholders from the food industry (caterers, food operators, and personnel from the food and hospitality industries) were also consulted. Each of the elicited requirements were discussed and considered for inclusion in the app.

    Stage 2: Designing and Developing the App

    The app was intended for use in a workplace canteen using a predetermined food menu offering a number of dishes and side dishes per day and was envisaged as a consumer-facing user interface attached to a back-end database. All aspects of the app from the user’s perspective were carefully considered from visual esthetics of the user interface (eg, logo, color scheme, and picture placement) to the workflow of user tasks. A user journey map was first created to visualize the timeline of interactions with the potential app from the landing page. Wireframes of each app screen were then produced using Balsamiq (Balsamiq Studios). These wireframes focused on app screen layout and content structure and were organized to reflect the user journey map. These wireframes were then mapped to mockups showing the actual visual designs for each screen. An interactive prototype was created using InVision (InVision Technologies, Inc), and from this an Android app was developed using native Android Studio. Particular consideration was also required for the setup of the back-end database, as databases rely on precise definitions of entities stored, whereas natural languages do not have this level of precision and contain synonyms and terminology imprecision. For instance, the terms dish, food item, and meal needed clarification. The specification of ingredients, for example, milk, within ingredients, for example, cheese, within dishes, also added complexity.

    The app was developed for Android mobile phones following Google’s Material Design Guidelines and industrial best practices, with reference to the adapted Technology Acceptance Model (TAM) [48-50]. The adapted TAM proposes that technology usage is positively predicted by perceived usefulness (“the degree to which a person believes that using a particular system would enhance his or her job performance”) [48], perceived ease of use (“the degree to which a person believes that using a particular system would be free of effort”) [48], perceived enjoyment (“the extent to which the activity of using the [technology] is perceived to be enjoyable in its own right, apart from any performance consequences that may be anticipated”) [49], and perceived visual attractiveness (“the degree to which a person believes that the [technology] is aesthetically pleasing to the eye”) [50].

    Throughout app development, several consultation meetings were held with all stakeholders to make sure the app would meet all demands and support further needs. A prototype app was developed to include all requirements identified as must have and should have using the MoSCoW framework. This prototype was then assessed and refined by the research team to create a second version.

    Stage 3: Initial Evaluation of the App

    Following development in 2016, version 2 of the prototype app was demonstrated to potential stakeholders and end users at several public engagement events in 2017, where assessments of the app were made using a questionnaire.

    Public Engagement Events

    Data were collected from adult participants at 8 global public research engagement events. These public engagement events were largely intended to promote research for the general public and included displays from a range of disciplines—that is, they were not held specifically to demonstrate or promote our app. The app was demonstrated, and data were collected at 4 public engagement events held in the United Kingdom, 1 event held in France, 1 event held in Denmark, 1 event held in Malaysia, and 1 event held in China. These events were chosen to represent the European consumers in the countries involved in the study, but also expanded consideration of the app to different markets. Demonstration at these events was intended to gain feedback and insights from a wide range of potential users of the app. These events are typically attended by members of the public who are educated and have an interest in science, education, and/or progress and growth. All attendees who were interested were permitted to try the app and provide data. There were no inclusion/exclusion criteria, to increase generalizability, but the information on the app was provided in English. Mock data on 4 mock dishes from 1 mock canteen were included in the app for demonstration purposes. Respondents were asked to view and manipulate the app for as long as they wished and then complete the evaluation questionnaire in paper form.

    Evaluation

    Assessments of the app were made using the SUS questionnaire [43,44]. The SUS consists of a 10-item questionnaire based on 5-point Likert scales (5=strongly agree to 1=strongly disagree) to assess usability. In total, 5 questions are positively phrased: “I think that I would like to use this system frequently,” “I thought the system was easy to use,” “I found the various functions in this system were well integrated,” “I would imagine that most people would learn to use this system very quickly,” and “I felt very confident using the system.” A total of 5 questions are negatively phrased: “I found the system unnecessarily complex,” “I think that I would need the support of a technical person to be able to use this system,” “I thought there was too much inconsistency in this system,” “I found the system very cumbersome to use,” and “I needed to learn a lot of things before I could get going with this system.” An additional question was also added to the end of the questionnaire “I believe the FoodSmart App will be useful to customers in a canteen setting to help them to get informed about the dishes offered.” There was also the opportunity for respondents to leave open-ended comments if desired. The SUS is recommended to assess usability, because it is technology-independent, short (and therefore easy to complete and analyze), and can provide a single score per person [51]. It was chosen for this study because it has been extensively used on a variety of products and systems, notably for measuring the functionality of apps. It is a well-validated instrument and supported by a large pool of comparison data.

    Analysis

    Questionnaire responses were entered into SPSS version 24. All questionnaires providing complete SUS data were used for analysis, although not all questionnaires included complete demographic data. The results of the tests for normality were acceptable, and the results of a confirmatory principal component analysis (Varimax rotation) on the responses from the 10 SUS questions were comparable with those conducted by other researchers [51,52]. Analysis of the raw item data gave 2 factors that explained 53.8% of the variance, consisting of (1) the 5 positively phrased items (explaining 41.4% of the variance) and (2) the 5 negatively phrased items (explaining a further 12.4% of the variance). Overall SUS scores based on the 10 SUS questions were calculated as described by Brooke [43]. Values of positively phrased items were reduced to a 0 to 4 scale by subtracting 1. Values of the negatively phrased items were subtracted from 5 to reverse their sense and to similarly reduce them to a 0 to 4 scale. The sums of the 10 items together were multiplied by 2.5 to provide individual SUS scores out of 100. SUS scores were analyzed using descriptive statistics and t tests. For significance testing, probability was considered at P<.05. The additional question given at the end of the questionnaire was considered as a separate individual question and analyzed using descriptive statistics only. Open-ended feedback comments were analyzed through open coding techniques and grouped into themes to provide an added richness of understanding.


    Results

    Stage 1: Defining the App Requirements

    Government Requirements and Recommendations

    European legislation requires the provision of information for all dishes served in an eating-out situation on 14 common allergens (celery, cereals, crustaceans, eggs, fish, lupin, milk, molluscs, mustard, nuts, peanuts, sesame, soybeans, and sulfur; EU regulation No 1169/2011). Recommendations at the European level also suggest the inclusion of information on calorie content and amounts of fat, saturates, carbohydrates, protein, sugars, and salt of a food per 100 g or 100 ml or per weight of portion served [10].

    Consumer Requirements

    The study by Price et al [45,46] revealed 8 criteria for food selection in a canteen setting: Animal Welfare (how an animal is coping with the conditions in which it lives, where an animal is in a good state of welfare if it is healthy, comfortable, well nourished, safe, able to express innate behavior, and is not suffering from unpleasant states); Environmental Impact (the effect the food production has on the environment); Fair Trade (Fair Trade aims to help producers to get a fair price for their products so as to reduce poverty, provide the ethical treatment of workers and farmers, and promote environmentally friendly and sustainable practices); Naturalness (the extent to which fresh ingredients are used, or that there is less use of processed foods containing additives and preservatives); Nutrition (the nutritional composition of the food); Organic (food produced in a way that respects natural life cycles, minimizes the human impact on the environment, and operates as naturally as possible); Provenance (where the food was grown/produced); and Value for Money (the ratio between the perceived quality of the dish and the price paid for it). Consumers recognized a variety of formats currently used to provide food-based information to consumers and expressed a desire for information that was easy to understand and easy to use, and preferences for only the information that was relevant to themselves, such as for their religious beliefs and their dietary preferences, eg, vegetarianism. Results from the questionnaire study on food-based decision making suggested that in all 4 European countries, the criteria of Nutrition, Naturalness, and Value for Money were those most valued by consumers, followed by criteria based on Animal Welfare, Organic foods, and Provenance, followed by those of Environmental Impact and Fair Trade [45]. Results from the questionnaire study on food-based information provision suggested that in all 4 European countries, the formats most preferred by consumers were traffic light labeling, information boxes, and quality assurance markings. Latent class cluster models also identified 5 clusters of consumers in relation to information use, described as Heuristic Processors (individuals who preferred easy-to-find-and-use information); Brand Orientated (individuals who were persuaded by brand authority); Systematic Processors (individuals who prefer more detailed information); Independent Processors (individuals who use a mix of heuristic and systematic processing); and Tech-Savvy (individuals who indicated a high preference for technology and interactive displays) [46]. Some differences between countries were also found [45,46].

    The study by Bray et al [47] confirmed the importance for consumers of information on nutritional content (Nutrition), ingredients and allergens (Naturalness), and confirmed desires for information to be presented in a clear and simple manner, desires for different information by different consumers, and a desire to personalize the information that each person receives. Options to remove or override any personal preferences were also preferred, as opposed to limits to free choice. In relation to ICT, consumers currently used ICT for food-based reasons related to: Marketing, for example, discounts or loyalty schemes; Increased convenience, for example, Web-based booking and viewing menus in advance; and Accessing additional information, for example, identifying ingredients and customer reviews. Avenues for future ICT usage focused on the following: the provision of digital menus and increased information per dish; the consumer gaining control over and confidence in what they were eating; the value of personalized (relevant) information provision; and the value of additional information/free choice despite personalization. The idea of using a mobile phone app to provide personalized information was generally supported.

    The literature in the field was also found to highlight similar opinions and concerns regarding food-based decision making [13,21,38,53] and supports similar conclusions regarding the provision of nutritional information in a retail setting [15,16,37] and in an eating-out setting [3,19,38,53]. Similar opinions toward technological solutions were also found [15,16,53].

    Requirement Priorities

    The priorities for the app based on MoSCoW principles [42] are presented in Table 1.

    Table 1. MoSCoW (Must have, Should have, Could have, Won’t have) requirements for the app.
    View this table

    Stage 2: Designing and Developing the App

    The app was developed as a consumer-facing user interface attached to a back-end canteen-based database. The back-end database held all required information per dish (ingredients, allergens, and nutritional composition), as supplied by caterers and food manufacturers. The app was designed such that caterers and food manufacturers would be given free and unlimited access to the database to upload information for as many dishes as they wished based on their own dish specifications and could update this information as often as they wished. Information in the database is stored per canteen.

    The user interface was designed to allow consumers to view all information provided by caterers and food manufacturers, and to manipulate the information displayed if desired, through the selection of settings on the user interface that allowed consumers to input personal details and preferences. The information for each dish could be revealed by accessing a menu or by scanning a QR (quick response) code placed on a menu or dish label, thus allowing very quick access to all information if desired.

    Particular consideration, throughout development, was given to the storage and security of personal data from users and dish and ingredient data from manufacturers; thus, all personal information is stored on the user’s device, and all industry data are owned and managed by the operator.

    Version 1 of the prototype included features to meet all requirements identified as must have and most of the features identified as should have using the MoSCoW method [42]. Full details of the user interface are given in Table 2. In addition to the features identified, version 1 of the app also included functionality for users to label dishes that they liked, but this functionality has not yet been linked to databases to allow feedback to caterers. Version 2 of the prototype app included all features identified as must have and should have, and so included all features of version 1, plus 3 additional features.

    Table 2. Details of the prototype app per user interface screen.
    View this table

    First, the information provided per dish was no longer presented on a single screen but split over 3 screens (overview/description; nutritional information; and ingredients/allergens) to facilitate users to access only desired information and avoid information overload. Second, the app included a calorie calculator. This facility allowed consumers to select the dishes they intended to consume, and a value for total calorie content was automatically calculated and provided for the meal as a whole. This facility recognizes that individuals do not typically only consume single dishes. Finally, the app included a notification system allowing caterers to send messages to users, for example, on special offers and promotions, recommendations, or advice. This feature has also not yet been fully activated.

    Details of the app, per screen, are given in Table 2. Images of the app screens are also displayed in Figures 1-3. The app is currently available for download from the Google Play Store. A video demonstration of both versions of the prototype app can be viewed in the dissemination section of the project website [54].

    Figure 1. Screenshot of the FoodSMART app: screen 1.
    View this figure
    Figure 2. Screenshots of the FoodSMART app: screen 4.
    View this figure
    Figure 3. Screenshots of the FoodSMART app: screen 5.
    View this figure

    Stage 3: Initial Evaluation of the App

    A total of 233 respondents evaluated the prototype app and provided usable questionnaire data: 79 respondents from the UK events, 54 respondents from the French event, 18 respondents from the Danish event, 34 respondents from the Malaysian event, and 48 respondents from the Chinese event. Of these, 81 (34.7%, 81/233) were male, 143 (61.3%, 143/233) were female; 115 (49.3%, 115/233) were aged 20 to 29 years, 39 (16.7%, 39/233) were aged 30 to 39 years, 35 (15.0%, 35/233) were aged 40 to 49 years and 32 (13.7%, 32/233) were aged 50 years and older. Demographic details were incomplete from some respondents.

    The mean overall SUS score was 68.3 (SD 15.4). Responses to individual questions are given in Table 3. Mean responses for all questions were positive relative to the midpoint of the scale and so represented a positive perception of the app (smallest t229=26.40; P<.01).

    Total SUS scores by gender, age, and event location are given in Table 4. Scores were comparable for males and females and no linear trend with age was found (largest t196=0.81; P=.42). Differences were found between event locations, where higher total SUS scores were found in the United Kingdom and Denmark, compared with France and also with Malaysia and China (F4,201=11.96; P<.01).

    The following common themes emerged from the open-ended feedback:

    • Good for informing healthy choices: Consumers liked the provision of various health and nutritional information in a manner that they found useful, for example, “Very useful for health conscious people, especially those who are busy, usually hard for them to buy healthy foods they wanted.”
    • Particularly useful for those on restricted diets: Some participants highlighted a particular interest for those consumers who are health conscious or observing a restricted diet, for example, “Extremely useful and important app that would also be important to people with dietary restrictions.”
    • Easy to use: Consumers offered support for the provision of information in a simple manner, and the use of the QR code, menus, and screen swipes to aid full use.
    • Support for the personalization aspect and specific functions: The concept of using a mobile phone app to access personalized information on the nutrients, ingredients, and allergens was viewed positively by many participants. The calculator function was well received, and respondents felt it was easy to use, for example, “Very interesting and promising tool for personal use as well as for service providers—the calculator function is good.”
    • Concerns around data accuracy: A small number of respondents expressed concerns around the possible accuracy of information provided through the app, not realizing the direct link between the caterer or food manufacturer and the information provided.
    • A wish for improved functionality: Some consumers also suggested a desire for additional features, such as links with social media, or an option to provide feedback to suppliers about missing or questionable information.
    Table 3. Mean and standard deviation of responses to all individual questions in the SUS questionnaire plus the additional question, for all consumers (N=233).
    View this table
    Table 4. Mean, SD, minimum, and maximum total System Usability Scale scores by gender, age, and event location for all consumers (N=233).
    View this table

    Discussion

    This study aimed to develop and evaluate a mobile phone app to provide consumers with food-based information in a workplace canteen setting in a manner that also allowed them to limit and/or personalize the information they received if desired. Initial stages of the study ascertained the information that European consumers (legally) should and would like to receive about the food on offer and established that consumers would like to personalize this information and receive this information in a certain manner. A mobile phone app was subsequently developed to address these identified needs. Early assessments of the developed app suggest support for the app and use, if available.

    Consumers were able to provide clear suggestions for information provision, which largely matched those provided by current legislation and previous research. App development resulted in a fully functioning prototype app, and the prototype was then evaluated positively. The SUS feedback gained highlights the strong usability and clarity of the developed app. Our app received a mean SUS score of 68.3. According to Bangor et al [51], scores in the range of 68 to 71 are typical of customer premise equipment (eg, phones and modems), graphical user interfaces, and interactive voice response phone systems. According to Kortum and Bangor [55], a score of 68.3 is about the value for Global Positioning System systems and slightly lower than that for digital video recorders. Furthermore, qualitative comments reaffirmed the potential ease of use and emphasized the value of information that was provided in a simple manner. Comparable SUS scores were also found between males and females and regardless of age, suggesting comparable appeal, but the app was found to be more positively evaluated in the United Kingdom and Denmark, compared with France and then compared with Malaysia and China. These location-based differences may warrant further examination and highlight a need for cultural considerations in app development and testing.

    The study has confirmed that many consumers seek additional information when eating out and are keen to consider the constituents and nature of the food that they eat [13,15,16,53]. This study also confirms that the nature of the desired information can vary between consumers [1,2,13,16] and that consumers like the idea of personalized information [13]. The use of mobile phone technology facilitated development and was considered appropriate by consumers. The use of mobile phone technology to display detailed information on menus has been demonstrated previously to be successful [16,38], with particular focus on increased capabilities to navigate quickly and the provision of added value information that is integrated fundamentally and placed prominently [16].

    Other studies have also indicated that consumers have greater trust in information provided through a technological solution rather than that gained from serving staff [16,17], and although a small number of respondents expressed concerns around the possible accuracy of information provided through our app, it may be that some additional information is required to inform consumers of the close links between the app and food providers, and the abilities for immediate and frequent updates where necessary. Some studies also report a need for more traditional forms of information provision to supplement technological information provision [53], but increased familiarity with technological solutions may reduce these concerns.

    Components of the app were also designed to benefit caterers and food providers. The app is quick and easy to populate and does not take technological skill or experience. Difficulties were encountered during the development stage, however, owing to competing requirements and considerations from different stakeholders. Competition over requirements was experienced among stakeholders dependent on their intended end-use of the app. Difficulties also arose as a result of the differential use and consideration of similar terms and similar concerns between stakeholders, and the relative importance given to issues such as consumer privacy and data storage.

    Further testing of our app is required. The app has been demonstrated as likely to be used and potentially useful, but further research is required in real canteen settings, both in terms of consumer acceptability and in terms of value to the consumer. Work is required specifically to demonstrate the value of the app for improving meal choice in a canteen scenario, for example, through the greater selection of more healthy meals, and for improving consumption, for example, through the consumption of less daily calories. Various research demonstrates that increased information may not necessarily be used or does not benefit all consumers [15,25,53], and some research even demonstrates less healthy consumption following the provision of nutritional information [56]. Considerable research demonstrates a strong distinction between the intention to perform a behavior and actually undertaking that behavior [57]. Behavioral tests of the benefit of the app are required. Preferably, these tests would be conducted in the form of randomized controlled trials to reduce potential bias, using behavioral outcomes based on food selection and consumption, and where calculations of benefits for health compared with relative costs would also be possible. We also accept that not all canteen users would use a mobile phone app, thus testing needs to be conducted in an appropriate volunteer sample.

    Evaluation of the app from a service provider’s perspective may also be of value. Consideration of additional criteria or additional functionality for food service operators such as feedback mechanisms could be of use. Food service providers are increasingly using digital means for informing and understanding consumers [15,16,58]. Some of the could have criteria that were not addressed during our development could be implemented. Facilities for consumer feedback, for example, via stores of purchases and purchase history may be beneficial.

    Facilities for providing recommendations based on purchase storage and history may also be desirable for consumers. Using purchasing data from 39 burger restaurants, it has previously been found that software recommendations can change the mix of items purchased. The share of adult main course items requesting no sauce increased by 6.8%, the share of children’s meals with apples instead of chips rose by 7.0% and the share of a breakfast sandwich without sausage increased by 3.8% [59]. Although these changes indicate only modest order refinement, the results suggest that targeted, adaptive food-based information could have behavioral potential. Facilities for social comparison may also be beneficial. Many apps related to social activities include a share option to allow others to view choices or allow comparisons between users or with an established norm. Notably, our app also currently does little to foster motivation for changes from the consumer. Engagement with the app requires consumers to be motivated to gain food-based information and then to act on this, and additional functionality, for example, feedback options for others to comment on food choices, through likes, may encourage and facilitate this motivation and so facilitate engagement with and action based on the app. Increased information provision may also be desirable to include an ability to flag any ingredient as an allergen. Allergies exist to more than the specified 14 commonly occurring allergens, and consumers have previously expressed a desire for all ingredients to be considered as potential allergens [26].

    Benefits for health may also be enhanced by the inclusion of additional features not requested by the consumer. Considerable interest is currently focusing on the use of nudges to encourage consumers to select and consume more healthy dishes, or those that are more sustainably sourced [60-62]. The order in which items are displayed on the app may nudge individuals to select dishes toward the top of the list compared with those further down [62]. Tests of strategies such as these would be of interest.

    Additionally, different versions of the app may be desirable, for example, through the use of different formats, different controls, or different setups. Tsai and Cheng [63], for example, identified 4 consumer clusters based on willingness to use technology (technology explorers, technology recipients, technology optimists, and technology laggards), and the app may be differentially viewed by these groups. Others also report the different use and preference of technology in different consumers [16,59].

    Consideration of the non-European market in testing and development would also be of value. Food provision and the interests and needs of consumers can differ widely between countries and cultures, and further consideration of these differences is required. Although our app was developed in Europe and may be transferable to other Western cultures, it may not transfer to other more developing cultures where food provision is less regulated, for example, where the majority of takeaway food is sold by individual street vendors.

    In conclusion, this study demonstrates the development and early positive evaluation of a prototype mobile phone app for the provision of food-based information in a canteen scenario in a manner that can be personalized. The study also confirms an interest by consumers in food-based information provision in eating-out scenarios, demonstrates a wish by consumers for abilities to personalize and limit the information provided, and demonstrates the value of a mobile phone app as a potential solution to current needs. Further research allowing further refinement of the app and demonstrating a health benefit from use of the app is required.

    Acknowledgments

    This study was funded by the European Commission, under grant agreement #643999: FoodSMART. The funder had no role in the conduct of the research, in the decision to publish, or in reviewing or approving the manuscript. The authors would like to thank Aris Koutsiamanis and Aimilia Tasidou, University of Thessaloniki, Greece, and Toby Osman, Bournemouth University, United Kingdom, for help with app development.

    Conflicts of Interest

    None declared.

    References

    1. Bates K, Burton S, Huggins K, Howlett E. Battling the bulge: menu board calorie legislation and its potential impact on meal repurchase intentions. J Consum Market 2011 Mar 22;28(2):104-113. [CrossRef]
    2. Burton S, Creyer EH, Kees J, Huggins K. Attacking the obesity epidemic: the potential health benefits of providing nutrition information in restaurants. Am J Public Health 2006 Sep;96(9):1669-1675. [CrossRef] [Medline]
    3. Ellison B, Lusk JL, Davis D. Looking at the label and beyond: the effects of calorie labels, health consciousness, and demographics on caloric intake in restaurants. Int J Behav Nutr Phys Act 2013 Feb 8;10:21 [FREE Full text] [CrossRef] [Medline]
    4. Orfanos P, Naska A, Trichopoulou A, Grioni S, Boer JM, van Bakel MM, et al. Eating out of home: energy, macro- and micronutrient intakes in 10 European countries. The European prospective investigation into cancer and nutrition. Eur J Clin Nutr 2009 Nov;63(Suppl 4):S239-S262. [CrossRef] [Medline]
    5. Orfanos P, Naska A, Trichopoulos D, Slimani N, Ferrari P, van Bakel M, et al. Eating out of home and its correlates in 10 European countries. The European prospective investigation into cancer and nutrition (EPIC) study. Public Health Nutr 2007 Dec;10(12):1515-1525. [CrossRef] [Medline]
    6. Bhutani S, Schoeller DA, Walsh MC, McWilliams C. Frequency of eating out at both fast-food and sit-down restaurants was associated with high body mass index in non-large metropolitan communities in Midwest. Am J Health Promot 2018 Jan;32(1):75-83 [FREE Full text] [CrossRef] [Medline]
    7. Edwards J, Engstrom K, Hartwell H. Overweight, obesity and the food service industry. Food Serv Technol 2005 Jun;5(2-4):85-94. [CrossRef]
    8. Ma Y, Bertone ER, Stanek EJ, Reed GW, Hebert JR, Cohen NL, et al. Association between eating patterns and obesity in a free-living US adult population. Am J Epidemiol 2003 Jul 1;158(1):85-92. [CrossRef] [Medline]
    9. World Obesity Day. 2017. The First Unified World Obesity Day Will Take Place on 4th March 2020   URL: http://www.obesityday.worldobesity.org/ [accessed 2018-11-27] [WebCite Cache]
    10. European Commission. 2015. Labelling and Nutrition   URL: https://ec.europa.eu/food/safety/labelling_nutrition_en [accessed 2018-11-27] [WebCite Cache]
    11. US Food and Drug Administration. 2015. Overview of FDA Labeling Requirements for Restaurants, Similar Retail Food Establishments and Vending Machines   URL: https:/​/www.​fda.gov/​food/​food-labeling-nutrition/​overview-fda-labeling-requirements-restaurants-similar-retail-food-establishments-and-vending [accessed 2018-11-27] [WebCite Cache]
    12. Fernandes AC, Oliveira RC, Proença RP, Curioni CC, Rodrigues VM, Fiates GM. Influence of menu labeling on food choices in real-life settings: a systematic review. Nutr Rev 2016 Aug;74(8):534-548. [CrossRef] [Medline]
    13. Banterle A, Cavaliere A, Ricci EC. Food labelled information: an empirical analysis of consumer preferences. Int J Food Syst Dyn 2012;3:156-170. [CrossRef]
    14. Case DO, Given LM. Looking for Information: A Survey of Research on Information Seeking, Needs and Behavior. Fourth Edition. Bingley, United Kingdom: Emerald Group Publishing; 2016.
    15. Din N, Zahari MS, Othman C, Abas R. Restaurant operator's receptiveness towards providing nutritional information on menu. Procedia Soc Behav Sci 2012;50:699-709. [CrossRef]
    16. Lessel P, Böhmer M, Kröner A, Krüger A. User Requirements and Design Guidelines for Digital Restaurant Menus. In: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. 2012 Presented at: NordiCHI'12; October 14-17, 2012; Copenhagen, Denmark p. 524-533. [CrossRef]
    17. Price S, Hartwell H, Hemingway A, Chapleo C. Workplace foodservice; perception of quality and trust. Appetite 2016 Feb 1;97:169-175. [CrossRef] [Medline]
    18. Hammond D, Lillico HG, Vanderlee L, White CM, Reid JL. The impact of nutrition labeling on menus: a naturalistic cohort study. Am J Health Behav 2015 Jul;39(4):540-548. [CrossRef] [Medline]
    19. VanEpps E, Downs J, Loewenstein G. Calorie label formats: using numeric and traffic light calorie labels to reduce lunch calories. J Public Policy Mark 2016;35:26-36 [FREE Full text]
    20. Bleich SN, Economos CD, Spiker ML, Vercammen KA, VanEpps EM, Block JP, et al. A systematic review of calorie labeling and modified calorie labeling interventions: impact on consumer and restaurant behavior. Obesity (Silver Spring) 2017 Dec;25(12):2018-2044 [FREE Full text] [CrossRef] [Medline]
    21. Price S, Bray J, Brown L. Enabling healthy food choices in the workplace: the canteen operators’ perspective. Int J Workplace Health Manag 2017 Aug 7;10(4):318-331. [CrossRef]
    22. Lowe B, Fraser I, Souza-Monteiro DM. A change for the better? Digital health technologies and changing food consumption behaviors. Psychol Mark 2015 Apr 20;32(5):585-600. [CrossRef]
    23. Nocella G, Romano D, Stefani G. Consumers’ attitudes, trust and willingness to pay for food information. Int J Consum Stud 2014 Jan 23;38(2):153-165. [CrossRef]
    24. Fotouhinia-Yepes M. Menu calorie labelling in a fine dining restaurant: will it make a difference? J Qual Assur Hospit Tourism 2013 Jul;14(3):281-293. [CrossRef]
    25. Dumanovsky T, Huang CY, Nonas CA, Matte TD, Bassett MT, Silver LD. Changes in energy content of lunchtime purchases from fast food restaurants after introduction of calorie labelling: cross sectional customer surveys. Br Med J 2011 Jul 26;343:d4464 [FREE Full text] [CrossRef] [Medline]
    26. Ogawa M, Tanaka A, Noda K, Kawai A, Amoroso DL. Research on food allergy information using smart mobile media devices to enhance communication at restaurants. Int E-Bus Res 2012;8(3):1-17. [CrossRef]
    27. European Commission. 2016. Irst European Directory of Health Apps Recommended by Patients and Consumers   URL: http://www.patient-view.com/uploads/6/5/7/9/6579846/ [accessed 2018-11-27] [WebCite Cache]
    28. Flaherty SJ, McCarthy M, Collins A, McAuliffe F. Can existing mobile apps support healthier food purchasing behaviour? Content analysis of nutrition content, behaviour change theory and user quality integration. Public Health Nutr 2018 Feb;21(2):288-298. [CrossRef] [Medline]
    29. SmartAPPetite: Buy Local, Eat Smart, Get Healthy. 2018.   URL: http://www.smartappetite.ca [accessed 2018-11-27] [WebCite Cache]
    30. Barfield J. Uatrav. 2014. Food Service App Allows Students to Order Meals With Smartphones, Avoid Long Lines   URL: https://home.tapingo.com/ [accessed 2018-11-27] [WebCite Cache]
    31. Pieskä S, Liuska M, Jauhiainen J, Auno A, Oy D. Intelligent Restaurant System SmartMenu. In: Proceedings of the 4th International Conference on Cognitive InfoCommunications. 2013 Presented at: CogInfoCom'13; December 2-5, 2013; Budapest, Hungary p. 2-5. [CrossRef]
    32. Statista. 2018. Smartphone User Penetration as Percentage of Total Global Population From 2014 to 2021   URL: https://www.statista.com/statistics/203734/global-smartphone-penetration-per-capita-since-2005/ [accessed 2018-11-27] [WebCite Cache]
    33. Newzoo. 2019. Top 20 Countries/Markets by Smartphone Users and Penetration   URL: https://newzoo.com/insights/rankings/top-50-countries-by-smartphone-penetration-and-users/ [accessed 2019-06-03]
    34. Tonkin E, Wilson AM, Coveney J, Webb T, Meyer SB. Trust in and through labelling – a systematic review and critique. Br Food J 2015 Jan 5;117(1):318-338. [CrossRef]
    35. Meyer SB, Coveney J, Henderson J, Ward PR, Taylor AW. Reconnecting Australian consumers and producers: identifying problems of distrust. Food Policy 2012 Dec;37(6):634-640. [CrossRef]
    36. Coveney J. Food and trust in Australia: building a picture. Public Health Nutr 2008 Mar;11(3):237-245. [CrossRef] [Medline]
    37. Hodgkins C, Barnett J, Wasowicz-Kirylo G, Stysko-Kunkowska M, Gulcan Y, Kustepeli Y, et al. Understanding how consumers categorise nutritional labels: a consumer derived typology for front-of-pack nutrition labelling. Appetite 2012 Dec;59(3):806-817. [CrossRef] [Medline]
    38. Yepes M. Mobile tablet menus attractiveness and impact of nutrition labeling formats on millennials? Food choices. ‎Cornell Hosp Q 2014 Aug 28;56(1):58-67. [CrossRef]
    39. Teixeira V, Voci SM, Mendes-Netto RS, da Silva DG. The relative validity of a food record using the smartphone application MyFitnessPal. Nutr Diet 2018 Apr;75(2):219-225. [CrossRef] [Medline]
    40. World Health Organisation. 2004. The Global Strategy on Diet, Physical Activity and Health (DPAS)   URL: https://www.who.int/nmh/wha/59/dpas/en/ [accessed 2018-11-27] [WebCite Cache]
    41. Lassen AD, Thorsen AV, Sommer HM, Fagt S, Trolle E, Biltoft-Jensen A, et al. Improving the diet of employees at blue-collar worksites: results from the 'Food at Work' intervention study. Public Health Nutr 2011 Jun;14(6):965-974. [CrossRef] [Medline]
    42. Clegg D, Barker R. CASE Method Fast-track: A RAD Approach. Boston, Massachusetts, United States: Addison-Wesley; 1994.
    43. Brooke J. SUS: a quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation In Industry. London: Taylor & Francis; 1996.
    44. Brooke J. SUS: a retrospective. J Usability Stud 2013;8:29-40 [FREE Full text]
    45. Price S, Viglia G, Hartwell H, Hemingway A, Chapleo C, Appleton K, et al. What are we eating? Consumer information requirement within a workplace canteen. Food Qual Prefer 2016 Oct;53:39-46 [FREE Full text] [CrossRef]
    46. Bray J, Hartwell H, Price S, Viglia G, Kapuściński G, Appleton K, et al. Food information presentation: consumer preferences when eating out. Br Food J 2019 Jun 19;121:1744-1762. [CrossRef]
    47. Bray J, Hartwell H, Perez-Cueto F, Mavridis I. FoodSMART: Think Smart, Eat Smarter. In: Food Product Marketing Conference. 2018 Presented at: FPM'18; June 13-14, 2018; Bournemouth, UK.
    48. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989 Sep;13(3):319-340. [CrossRef]
    49. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manag Sci 1989 Aug;35(8):982-1003. [CrossRef]
    50. van der Heijden H. User acceptance of hedonic information systems. MIS Q 2004;28(4):695-704. [CrossRef]
    51. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Human Computer Interact 2008 Jul 30;24(6):574-594. [CrossRef]
    52. Lewis JR, Sauro J. Revisiting the factor structure of the system usability scale. J Usability Stud 2017;12:183-192 [FREE Full text]
    53. Cohen DA, Babey SH. Contextual influences on eating behaviours: heuristic processing and dietary choices. Obes Rev 2012 Sep;13(9):766-779 [FREE Full text] [CrossRef] [Medline]
    54. Bournemouth University. 2019. FoodSMART   URL: https://microsites.bournemouth.ac.uk/foodsmart/ [accessed 2019-09-20]
    55. Kortum PT, Bangor A. Usability ratings for everyday products measured with the system usability scale. Int J Human Computer Interact 2013 Jan;29(2):67-76. [CrossRef]
    56. Appleton KM, Pidgeon HJ. 5-a-day fruit and vegetable food product labels: reduced fruit and vegetable consumption following an exaggerated compared to a modest label. BMC Public Health 2018 May 15;18(1):624 [FREE Full text] [CrossRef] [Medline]
    57. de Ridder D. Health psychology. In: Smelser NJ, Baltes PB, editors. International Encyclopedia of the Social & Behavioral Sciences. Second Edition. London: Pergamon; 2015:674-678.
    58. Ruiz-Molina M, Gil-Saura I, Berenguer-Contrí G. Information and communication technology as a differentiation tool in restaurants. J Foodserv Bus Res 2014 Nov 17;17(5):410-428. [CrossRef]
    59. Bedard K, Kuhn P. Micro-marketing healthier choices: effects of personalized ordering suggestions on restaurant purchases. J Health Econ 2015 Jan;39:106-122 [FREE Full text] [CrossRef] [Medline]
    60. Friis R, Skov LR, Olsen A, Appleton KM, Saulais L, Dinnella C, et al. Comparison of three nudge interventions (priming, default option, and perceived variety) to promote vegetable consumption in a self-service buffet setting. PLoS One 2017;12(5):e0176028 [FREE Full text] [CrossRef] [Medline]
    61. Saulais L, Massey C, Perez-Cueto FJ, Appleton KM, Dinnella C, Monteleone E, et al. When are 'Dish of the Day' nudges most effective to increase vegetable selection? Food Policy 2019 May;85:15-27. [CrossRef]
    62. Wilson AL, Buckley E, Buckley JD, Bogomolova S. Nudging healthier food and beverage choices through salience and priming. Evidence from a systematic review. ‎Food Qual Prefer 2016 Jul;51:47-64. [CrossRef]
    63. Tsai MC, Cheng CC. Consumers' intention to use Mobile-Menu and destination clusters. Int J Mob Commun 2013;11(1):36-55. [CrossRef]


    Abbreviations

    ICT: information communication technology
    MoSCoW: Must have, Should have, Could have, Won’t have
    QR: quick response
    SUS: System Usability Scale


    Edited by G Eysenbach; submitted 28.11.18; peer-reviewed by J Shirk, G Hendrie, J Nunez-Cordoba, W Han; comments to author 01.05.19; revised version received 23.06.19; accepted 19.08.19; published 04.11.19

    ©Katherine Marie Appleton, Jeff Bray, Sarah Price, Gernot Liebchen, Nan Jiang, Ioannis Mavridis, Laure Saulais, Agnès Giboreau, Federico J A Perez-Cueto, Rebecca Coolen, Manfred Ronge, Heather Hartwell. Originally published in JMIR Formative Research (http://formative.jmir.org), 04.11.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on http://formative.jmir.org, as well as this copyright and license information must be included.