A Pilot Randomized Controlled Trial of a Digital Intervention Aimed at Improving Food Purchasing Behavior: The Front-of-Pack Food Labels Impact on Consumer Choice Study

Introduction

Background

Poor diet is a major risk factor for noncommunicable diseases (NCDs) in the United Kingdom, responsible for more than 10% of all morbidity and mortality [1]. Food purchasing precedes and affects food consumption, which makes food purchasing environments a prime setting for intervention studies aimed at improving diet and nutrition. In the United Kingdom, most food shopping is conducted in supermarkets [2], and supermarket purchases have been shown to correlate well with food and nutrient consumption [3].

Front of pack (FOP) nutrition labeling on food packaging has been used in the United Kingdom since the mid-2000s [4]. In October 2012, traffic light labeling of nutrients was recommended by the UK Government for FOP labeling [5], and it is currently being used by many UK manufacturers and retailers. Traffic light labeling involves a color-coded assessment (green for low, amber for medium, and red for high) of the level of total fat, saturated fat, sugar, and salt (see Figure 1 [6]).

Supermarket loyalty card data are a potential source of “big data” that could allow for remote, objective monitoring of food purchasing behavior, enabling interventions aimed at improving the healthiness of food purchases that could be delivered at scale to a large population as they require no additional burden beyond continued use of loyalty cards during routine food purchasing. However, loyalty card data are owned by the supermarket industry, and little is known about the feasibility of using such data for the development of public health interventions that incorporate tailored feedback on previous purchases. This is because loyalty card data are commercially sensitive and consumers have privacy concerns over the handling of loyalty card data [7]; as such, it is typically difficult to access. Therefore, the use of such data is rare in the evaluation of public health interventions [8-11].

This study reports on a pilot 2-arm equal allocation parallel randomized controlled trial (RCT) of a digital intervention that incorporated a number of behavior change techniques. The intervention consists of a password-protected website where users can access tailored feedback on previous purchases of ready meals and pizzas, set goals for behavior change, and model and practice the recommended healthy shopping behavior. A theoretical approach based on selection of the most relevant behavior change mechanisms was adopted rather than utilization of an entire theoretical framework, as it has been suggested that this is an optimal approach in a food context [12]. The intervention was aimed at increasing the use of traffic light labeling to encourage healthier purchase decisions when purchasing ready meals and pizzas, thereby leading to purchases lower in total fat, saturated fat, sugar, and salt. Design of the intervention was based on reviews of behavior change literature, and previous research has shown that remotely delivered interventions that provide tailored feedback on previous behavior can improve dietary behavior [13].

The pilot trial was part of the Front-of-pack food Labels: Impact on Consumer Choice (FLICC) project. The protocol for the pilot trial has been previously published and includes a detailed description of development of the intervention design and content [14], and the pilot trial was registered at the ISRCTN registry (ISRCTN19316955).

‎View this figure

Objectives

The objectives of the trial were to assess the feasibility of a full RCT by measuring recruitment, retention, and data completion rates of participants, producing estimates for the potential effect size of the intervention on healthiness of purchased own-brand ready meals and pizzas—the primary outcome measure, and producing estimates for the potential effect size of the intervention on all food purchases, purchases of fruit and vegetables, and psychosocial variables associated with label use—secondary outcome measures.

Our hypotheses were that the intervention would increase the healthiness of purchased ready meals and pizzas, while not affecting the total amount of ready meals and pizzas purchased, nor affecting purchasing behavior in other food categories. We hypothesized that the intervention would operate by impacting on mechanisms affecting beliefs and behavioral intention formation as well as those associated with planning and goal setting and the adoption and maintenance of the behavior of interest, namely, traffic light labeling use during purchases of ready meals and pizzas. Due to constraints in the availability of data and the fact that not all branded products contain traffic light labels, we limited our analyses to supermarket’s own-brand products only. We hypothesized that the majority of ready meal and pizza purchases would be own-brand products, and so, restricting analyses to these product lines would have limited effect on results.

Methods

Data collection for the FLICC pilot trial took place over 58 weeks from November 11, 2014, to December 23, 2015. The data collected comprised food purchase data obtained from the supermarket loyalty card database and self-completion participant questionnaire data. The trial was split into 4 distinct time periods: 26 weeks of baseline historical shopping data (T-1), 4 weeks of recruitment (T0), 6 weeks of intervention (T1), and 12 weeks of follow-up without intervention (T2). A further 10 weeks between the end of T0 and the start of T1 were used to request, receive, and process the shopping history data for use in the intervention. The trial stages and types of data collected at each point are shown in Figure 2.

The primary focus of this trial was purchases of own-brand ready meals and pizzas. These food categories were chosen because they are highly likely to carry traffic light labeling in the participating supermarket; there is considerable nutritional variation in these food categories, allowing participants scope for buying healthier products; and ready meals and pizzas represent a large and growing proportion of food sales in the United Kingdom [14,15].

‎View this figure

Ethical Approval

Ethics was granted by the University of Oxford Central University Research Ethics Committee (SSD/CUREC1/14-008) and the University of Surrey Ethics Committee (EC/2014/153/FAHS).

Recruitment

To be eligible for the FLICC pilot trial, individuals needed to be UK residents, have had a loyalty card with the participating supermarket for at least 6 months at recruitment, be older than 18 years, do most of their shopping at stores larger than 8000 square feet (this criterion was to ensure that participants would have access to a large supply of own-brand ready meals and pizzas), be the primary food shopper for their household, not be planning to leave the United Kingdom for longer than 3 weeks during the study period, and have purchased at least 10 ready meals and pizzas in the previous 6 months (self-reported).

On the basis of a power calculation shown in the protocol [14], we aimed to recruit 1300 participants by email from a loyalty card database of the participating supermarket, using a passive approach—a recruitment method that requires a potential participant to make the first contact with the study following an invitation [16]. The supermarket chain emailed 50,000 cardholders selected at random in batches of 3000 from the whole population of cardholders, after filtering to exclude individuals who would not meet the first 4 inclusion criteria. The recruitment email included information on the trial and a link to a FLICC registration website, where participants were screened for the remaining 3 eligibility criteria, provided extra information for allocation through block randomization, and gave informed consent for participation in the trial.

Allocation

Block randomization was used to allocate individuals to the intervention or control arm, stratifying by sex and whether or not participants had dependent children. Participants were told the study was about the influence of traffic light labels on purchasing decisions but not informed whether they were in the intervention or control arm. A total of 2 researchers (RAH and PS) implemented the randomization and had access to the list of control and intervention participants during the study.

Intervention

A full description of the intervention is provided in the published trial protocol [14]; however, a summary of the key components is given in Table 1. A theoretical approach based on the selection of the most relevant behavior change mechanisms was adopted [12,17], and the intervention contains both passive components (information delivery via the Web application) and interactive components (participant is encouraged to engage with the Web application). The passive and interactive elements of the intervention are highlighted in Table 1. Overall, the intervention was designed to help people make intracategory decisions (eg, to compare pizza A and pizza B) and by focusing only on the use of the traffic light element (ie, colors) of the nutrition label.

Participants in each arm were sent an email containing a URL to a password-protected Web application, which remained open for 6 weeks (T1). The control group received a subset of the digital intervention: information on the importance of healthy eating, a description of traffic light labeling, and the message “Green is better than amber but amber is better than red!” Screenshots of the intervention are available from the Centre on Population Approaches for NCD Prevention website [18] or by request to the corresponding author.

Data Collection

Data on food purchases by the participants were collected by the participating supermarket’s loyalty card system. Data on all foods and drinks purchased (while using the loyalty card) by the participants in any store across the United Kingdom during the T-1 period were transferred by the participating supermarket to the research team after recruitment had closed. A second transfer of equivalent data covering the periods T1 and T2 was conducted after the study had finished. Where participants withdrew from the study, food purchase data were transferred only up to the withdrawal date, and their questionnaire data were not included for secondary outcome analyses.

The participating supermarket also provided data on the nutritional quality of all own-brand food products currently on sale at 2 time points (before recruitment and after the study had completed), which were used to derive traffic light labels and to calculate outcome measures. Nutrition data for own-brand products found in the shopping history data that were not present in the supermarket nutrition dataset were extracted from the Brandbank database [19].

Demographic (age, sex, ethnicity, educational status, and household size) and socioeconomic (income and job classification) data were collected in the first of 3 Web-based questionnaires delivered at recruitment (T0). Psychosocial variables were collected at T0 and at the end of T1 and T2. In addition, Web analytics were collected to provide data on participants’ engagement with the intervention. A single reminder was sent out for completion of the second and third questionnaires, which were incentivized by a £10 online gift voucher.

Measures

This pilot trial collected data on recruitment and retention rates and estimates of effect size for the primary and secondary outcome variables. The primary outcome measure for this trial was the healthiness of own-brand ready meals and pizzas that had traffic light labeling measured at both T1 and T2, where “healthiness” of each item was calculated from a combination of the information provided on the traffic light label. The score ranges from 0 (for 4 red lights) to 1 (for 4 green lights). Foods are awarded 0.15 points for each amber light and 0.25 points for each green light. The weighting for the different colors was derived from a choice experiment [20] where participants were asked to decide between the healthiness of 2 foods based purely on the traffic light label information, thereby revealing the different prominence awarded to each color in the decision-making process.

In a sample of 406 ready meals and pizzas from the participating supermarket collected in November 2013, the healthiness score was reasonably normally distributed with a mean of 0.63 and SD of 0.21. Foods with scores of 0 and 1 were identified in the sample, demonstrating that the full range of the score was used.

The secondary outcome measures were as follows:

1. The number of ready meals and pizzas purchased in T1/T2
2. Expenditure (measured in £) on ready meals and pizzas purchased in T1/T2 
3. The total amount (measured in grams) of fat, saturated fat, sugar, and salt in ready meals and pizzas purchased in T1/T2
4. Expenditure (measured in £) on all foods purchased in T1/T2 
5. Expenditure (measured in £) on fruit and vegetables purchased in T1/T2
6. Psychosocial variables including “Stage model of health awareness” [21,22], “Perceived intake,” “Perceived need to change,” “Expectation and Intention” [23], and “Potential barriers to labelling use” [24-26] measured in T1/T2.

These psychosocial variables were selected to measure the effectiveness of the mechanisms we employed in the intervention design (Table 2) in terms of their potential to shift participants to using traffic light labeling (stage model of health awareness) by impacting on belief formation (perceived intake and perceived need to change), intention formation, and expectation. Potential barriers to labeling use were also measured.

Statistical Analysis

All analyses were conducted in accordance with a predetermined statistical analysis plan, available from the Centre on Population Approaches for NCD Prevention website [18] or by request to the corresponding author. For normally distributed outcomes, analysis of covariance was used to assess differences between intervention and control arms at periods T1 and T2, adjusted for sex, whether or not the participants had dependent children, and measures collected in T-1 using the following model proposed by Vickers and Altman [27], illustrated by the primary outcome analysis at T1, where beta are regression coefficients, ε is an error term, and “group” refers to allocation to intervention or control:

Healthiness _T1 = β ₀ + β ₁*Sex + β ₂*Dependents + β ₃*Healthiness _T-1 + β ₄*Group + ε

Where not normally distributed, differences were assessed using Mann-Whitney U tests not adjusted for sex, dependent children, or measures collected at T-1. Predetermined subgroup analyses of the primary outcome measure were conducted, stratified by socioeconomic status based on job classification (National Statistics Socio-economic Classification (NS-SEC) 1 or 2 vs 3 to 5) using a standard UK definition [28].

Within this study, missing outcome data (MOD) occurred for a number of reasons. For both the sales data and questionnaire data, MOD were generated by participants withdrawing post randomization. For the questionnaire data, MOD were generated by failure to complete some or all of the questions within a questionnaire. The primary outcome variable (average healthiness of ready meals and pizzas purchased in T-1, T1, and T2) included MOD if the participant did not purchase any own-brand ready meals or pizzas using their loyalty card in any of the 3 study phases. A systematic review of methods used to cope with MOD in intention-to-treat analyses demonstrated that there is no consensus toward a preferred approach, with arguments for restricting to complete case analysis and for imputation of missing data [29]. For the FLICC study, we dealt with MOD in the sales data by employing imputation techniques (multiple imputation for the primary outcome variable and single imputation for the secondary outcome variables). The imputation method used regression analysis with sex and dependent children as predictor variables as these were used in the block allocation, so there was no chance of missing data. The imputation datasets were all observations within T-1 for MOD at T-1 and equivalent for periods T1 and T2. We assessed the nature of the MOD from the primary outcome variable by conducting chi-square tests of the presence of missing data at any of the 3 study periods with sex and dependent children. For the questionnaire data, where retention rates were expected to be lower, we conducted analysis on a complete case basis (defined as being participants that provided electronic sales data and questionnaire data at all time points) as predetermined by our statistical analysis plan.

Results

Results to Predetermined Analyses

Of the 50,000 loyalty cardholders who received invitation emails, 869 clicked the link to the FLICC recruitment website. Of these, 496 were eligible and completed the consent process. These figures are illustrated in the flowchart in Multimedia Appendix 1. Our passive recruitment method recruited approximately 1% (496/50000) of the participant pool.

Of the recruited participants, 3 withdrew without reason after randomization but before the T1 period commenced—complete purchase data were collected for all other participants. The completion rates for the recruitment (T0), second (T1), and third (T2) questionnaires were 79% (394/496), 54% (270/496), and 63% (313/496), respectively. A chi-square test showed no evidence of difference in provision of complete data by allocation group. Of the 496 recruited participants, 208 (42%) provided complete data (ie, completed all 3 questionnaires and allowed transferal of purchasing data for the complete study period). The majority of the participants were older than 46 years, white, female, with no dependent children, and were in high socioeconomic groups (NS-SEC 1 or 2; Table 3). More than three-quarters of the sample reported a high interest in health using a predefined measure [30].

During the intervention period, 438 ready meals and pizzas were available to purchase through the participating supermarket. A total of 317 (72.4%) of the products were supermarket own-brand and 121 (27.6%) were branded products. Of 10,416 ready meal and pizza purchases used in the analyses, 8263 (79.3%) were supermarket own-brand and 2153 (20.7%) were branded products.

In all 3 data collection periods, there were high levels of MOD for both control and intervention groups for the primary outcome (average healthiness of traffic light and ready meals), indicating zero recorded purchases of own brand ready meals and pizzas (Table 4). Assessments of the association between the presence of MOD and the 2-block randomization variables (sex and dependent children) showed no evidence of association (P>.2 in all cases), suggesting that the MOD were missing completely at random. The results for the primary outcome showed no difference between intervention and control groups during periods T1 (P=.32) and T2 (P=.59). Predetermined subgroup analyses stratified by socioeconomic status also did not find any differences between intervention and control groups (Multimedia Appendix 1). Exploratory analyses did not find a significant interaction between allocation group and socioeconomic status.

The difference between the control and intervention arms for the food purchasing secondary outcome measures are shown in Table 5 and reveal no evidence that the intervention changed purchasing behavior. In terms of the psychosocial secondary outcome measures (Table 6), these similarly demonstrate no difference between intervention and control groups for the effect of the intervention.

In the context of the lack of an observed effect in the primary outcome for this intervention, it is interesting to note that just over half the participants in the intervention arm (54%) logged onto the FLICC website (as measured by Web analytics) during the study period, and therefore, many participants did not receive or engage with the intervention material at all. Similar levels of engagement were observed in the control arms (48%). In terms of some of the key elements of the intervention content reported in Table 7, in all cases, less than half the intervention arm participants engaged with these.

Post Hoc Results

A complete case analysis of the primary outcome variable, where participants are only included if they purchased at least one ready meal or pizza in both the baseline and either intervention period (n=213) or washout period (n=266), showed a significant increase in the healthiness of food purchases in the intervention group of 0.04 (P=.03)—roughly equivalent to switching 1 red light for 1 amber light for every 3 ready meals or pizzas purchased. There was no difference between intervention and control groups at washout in the complete case analysis.

Discussion

Principal Findings

The FLICC pilot trial was an example of a partnership between academia and the supermarket industry to allow for a randomized trial of a behavior change intervention that utilized supermarket loyalty card data for(1) recruitment, (2) provision of tailored feedback on previous purchases, and (3) objective and remote measurement of participant food purchases. The pilot study showed that remote delivery of dietary studies across wide populations allows for speedy recruitment (albeit recruiting only 1% of the participant pool), that measurement of outcomes using loyalty cards can lead to very high retention rates, but that engagement with remotely delivered digital behavior change interventions can be low. The trial did not provide evidence to suggest this specific intervention would be effective at changing purchasing behavior, but the process of conducting the trial has revealed much information about using supermarket loyalty card data for both the delivery of public health interventions and for trials of their effectiveness—issues that are growing in relevance [31-33].

Supermarket loyalty card data provide benefits to public health research; however, the use of such data is rare in the evaluation of public health interventions [8-11]. This study revealed both benefits and limitations attached to the use of loyalty card data. The benefits include high retention rates and the potential to recruit quickly from a wide population. Limitations include generic problems with drawing conclusions about consumption from purchase data (discussed below) and specific issues associated with data sharing between academia and the supermarket industry. Loyalty card data are commercially sensitive, and supermarkets have a duty to ensure that individual-level data are handled with an appropriate level of security as consumers have privacy concerns over the handling of loyalty card data [7]. Recruiting from a loyalty card database means we had little control over recruitment methods, and a low participation rate may not have resulted in a representative sample. These issues are discussed here to highlight practical issues that researchers will need to consider if pursuing similar arrangements in the future. In the case of the FLICC study, the agreement of terms and conditions for transferal of data and the subsequent signing of contracts took over 12 months, and this was far longer than anticipated by either party. This caused significant amendments to the planned timetable for the study. For practical reasons, the participating supermarket was only able to provide us with 2 data extracts over the course of the project, which precluded us from providing participants with multiple updates on their shopping history as the trial progressed. For such collaborative studies to achieve the most success in the future, we advise that at an early stage, all collaborative partners develop an understanding of what can and cannot be delivered by the collaboration. We are grateful that the participating supermarket remained flexible to our needs throughout the research process, without which this study would not have been possible.

The main strengths of our study are the strong internal validity associated with the RCT design, the remote nature of the delivery of the intervention and collection of data (allowing for scalability of an intervention if shown to be effective), and the use of the supermarket loyalty card dataset for recruitment, which allowed quick and efficient recruitment of a large number of participants. The pilot study was able to answer questions about the practicality and feasibility of conducting trials in a supermarket setting in partnership with a supermarket chain, highlighting both the advantages and disadvantages of such a partnership while providing evidence on essential study features that could not have been known in advance. Of the examples, 1 includes the average amount of food purchasing that was recorded by study participants. In the FLICC study, the average amount of money spent on food captured by the loyalty cards was less than £20 per week, whereas the average amount of money spent on food and nonalcoholic drinks in the United Kingdom is £56.80 per household per week [2], which suggests that purchases in our study were a subset of all food purchases made by the participants. This has implications for whether our outcome measures truly reflect consumption behavior. Supermarket loyalty cards can only measure food purchases when a loyalty card is used, which may not be at every occasion that participants visit the supermarket. They also restrict the range of a study to include purchases in 1 supermarket chain only, which may not reflect shopping behavior in many people who regularly frequent more than 1 supermarket chain. They may also be shared by friends and family, so may not collect data from the individual recruited to the study [34]. Although only own-brand ready meal and pizza purchases were included for our analyses and some other branded products may have traffic light labels, we have shown that 79.3% of all purchases were of own-brand products, and thus, we have captured the vast majority of relevant purchasing behavior.

Another limitation was the amount of MOD for the primary outcome variable, which was a result of participants’ loyalty card data showing zero purchases of own-brand ready meals or pizzas in T1 and/or T2 (despite participants’ self-reporting at recruitment that they were frequent purchasers of these products). Ideally, we would have filtered the recruitment email so that only individuals who have shown frequent purchases of own-brand ready meal and pizzas in their loyalty card data were contacted to take part in the study—unfortunately this was not possible, as the recruitment email was delivered by a market research company aligned with the participating supermarket who had access to geographic and demographic data on loyalty card holders but not on previous purchases. Our screening question at recruitment was “Thinking about the last 6 months, on average have you purchased either Ready Meals or Pizzas at least twice per month? (It doesn’t matter if the Ready Meals or Pizzas were for you or for other members of your household).” However, 22% of the recruited participants did not have any records of own-brand ready meal or pizza purchases on their loyalty card data from the previous 6 months (T-1). This was a far higher percentage than we anticipated and increased to 52% during the intervention period (T1). Our predetermined statistical analysis plan stated that we would conduct analyses with imputation for the primary outcome variable but because of the amount of MOD, the imputation effectively overwhelms the analysis.

We measured engagement with our digital intervention using Web analytic tools. Similar methods are now regularly used to measure the engagement of participants with Web-based interventions [35]. We found that overall engagement with the intervention was low but at a similar level (50%) found in a systematic review of use of Web-based behavior change interventions [36]. Other studies have found that engagement with digital interventions can be boosted by telephone-based coaching [37] or professional support [38]; however, this would affect the potential reach of such an intervention delivered at scale. Provision of frequent updates of the intervention, which was not possible in our study because of the data-sharing relationship with the participating supermarket, has also been shown to increase engagement with digital behavior change interventions [38,39].

Other trials have investigated the impact of remotely delivered tailored feedback on dietary behavior and found more encouraging results. Alexander et al [40] conducted an RCT of 2540 participants, where arms 2 and 3 received a website with tailored feedback on previous diet compared with a control of a nontailored dietary advice website, and found that the intervention increased consumption of fruit and vegetables reported in a food frequency questionnaire by 2.7 and 2.8 portions per day, respectively, compared with 2.3 portions per day for the control group (P=.177 for arm 2, P=.050 for arm 3). Huang et al [41] found that a website that provided tailored advice resulted in a reduction in saturated fat of 0.66% of food energy in food purchases from an online supermarket compared with a control of nontailored advice (P<.001), in a randomized trial of 497 participants. Tapper et al [42] found an increase in fruit and vegetable consumption reported in a food frequency questionnaire in intervention compared with control group (P=.08) in an RCT of a website providing tailored dietary feedback with 100 participants. The interventions studied in these trials were more intensive and required a larger time commitment from participants than the intervention in the FLICC study. A recent randomized trial of a phone app that provides feedback on potential food purchases (suggesting switches for lower salt products) found a significant reduction of salt in food purchases of 0.3 g/MJ (95% CI 0.03-0.58) in a study of 66 adults with diagnosed cardiovascular disease [43], suggesting feedback on nutritional content of foods combined with proposed alternatives could be an effective mechanism for improving diets.

Conclusions

Although the FLICC study did not find evidence of an impact of the intervention on food purchasing behavior, the unique methods used in this pilot trial are informative for future studies that plan to use supermarket loyalty card data in collaboration with supermarket partners. The experience of the trial showcases the possibilities and challenges associated with the use of loyalty card data in public health research.