Estimating the effect of calorie menu labeling on calories purchased in a large restaurant franchise in the southern United States: quasi-experimental study.

Introduction

Nutrition labeling aims to help consumers make healthier dietary choices in retail settings by communicating information about products’ nutrient content or nutritional quality through clearly visible text or images.1 Nutrition labeling has been implemented in various ways across the world, including calorie labeling in the United States.1 2 In May 2018, the US Food and Drug Administration began requiring compliance with the menu labeling provision of the 2010 Affordable Care Act.3 This rule requires large chain food establishments (≥20 locations nationwide) to label their menus with kilocalorie (calorie) information for all items and to post a statement about recommended total daily calorie intake.4 This policy was adopted to increase awareness of the often underestimated calorie content of prepared foods offered at chain food establishments,5 6 with a secondary benefit of encouraging these establishments to offer lower calorie items.7 8 The ultimate goal of the policy is to reduce calorie intake from prepared foods in retail environments for long term reductions in obesity and related chronic diseases.9 10 11 12 This mirrors the overall goal of other labeling initiatives outside of the US, which is to either reduce calorie intake or improve dietary quality.1 13

Evidence of the effect of calorie labeling on calories purchased in fast food settings is mixed and incomplete.7 Most prior studies in restaurants used customer surveys and were powered to detect only large changes in calories purchased.14 15 16 17 18 Small reductions (observed in one study using sales data from Starbucks locations in New York City)19 might be more realistic and could still have important population level effects on the prevalence of obesity and nutrition related diseases, as well as healthcare costs associated with these diseases.20 21 Moreover, most studies have been conducted in large coastal urban cities that required labeling in chain restaurants before the federal mandate.7 The lack of calorie labeling evaluations in the southern US is a particularly important knowledge gap because the prevalence of obesity in this region (33.6% in 2018) is among the highest in the country.22 Additionally, it is important to evaluate calorie labeling outside of non-urban environments, where there might be fewer dining alternatives for consumers.23

We therefore conducted a quasi-experimental study to estimate the effects of calorie labeling on calories purchased per transaction in a large national fast food franchise in the southern US, covering a diverse region with both urban and non-urban restaurants. In anticipation of the federal mandate, the company labeled all menus with calories in April 2017 (despite the delay in the federal compliance date to May 2018).24 The franchise provided sales data from April 2015 to April 2018, comprising nearly 50 million transactions. This large dataset allowed for a well powered investigation of the immediate and longer term associations of calorie labeling on calorie purchases at popular fast food restaurants.

Methods

The fast food franchise comprised three separate chains based in Louisiana, Texas, and Mississippi. The franchise did not give permission to disclose the names of the restaurant chains under a data use agreement. For each restaurant, the franchise provided weekly data on the total number of transactions and total units purchased of each specific menu offering from April 2015 to April 2018. The franchise labeled its menus across all locations during the week of 6 April 2017, and the menus remained labeled until the end of the study. This provided us with two years of pre-labeling and one year of post-labeling sales data.

Over the study period (156 weeks) the franchise owned 139 restaurants; the number operating in a given week ranged from 79 to 134 because of restaurant openings and closures. Multiplying the number of restaurants and weeks of data available from each resulted in 15 568 total restaurant weeks of data. We excluded 176 restaurant weeks (1%) because of missing sales or transaction data. For our primary analysis we excluded 35 restaurants without sales data in both the pre-labeling and the post-labeling periods (599 restaurant weeks; 4%). We additionally excluded 40 restaurant weeks (0.3%) from Texas locations during the week of Hurricane Harvey because of few transactions and another 17 restaurant weeks (0.1%) that were outliers for items purchased or calories per transaction. These outliers were possibly errors or might have represented an unusual event for the restaurant (eg, closures due to weather events or construction). Our final sample size was 14 736 restaurant weeks, 95% of the total available data (fig 1).

The primary outcome was calories per transaction. We identified each menu item using a unique transaction code and accompanying item description. Then we determined the total calories of each item by matching it with its corresponding entry in Menustat, a database created by the New York City Department of Health and Mental Hygiene that contains nutrition information for menu offerings from top revenue generating restaurant chains in the US.25 Menustat provided data in January of each year, allowing us to make contemporaneous matches and account for restaurant reformulations and new offerings annually. Using this approach we matched 96% of total purchased items. If an item was not listed in Menustat for any year, we used a previous year in Menustat (2.3% of items matched) or nutrition information from the restaurant website if available (1.6% of purchased items). We deleted the remaining unmatched purchased items (<0.2% of purchased items). Of the matched items we deemed 97% to be high quality (ie, the item’s description exactly matched the Menustat/restaurant website item description). For each restaurant in a given week we multiplied the total number of each item purchased by the item’s calorie content, summed over all items, and divided the sum by the total number of transactions in the restaurant in that week (ie, the total number of customer receipts) to calculate calories per transaction. We additionally classified all items into one of five mutually exclusive categories: entrees, sides (including desserts), sugar sweetened beverages (>10 calories/serving), low calorie beverages (≤10 calories/serving), and condiments.

As described in our preregistered analysis plan (www.aspredicted.org/4xx8v.pdf), which we completed after data cleaning and item matching but before any analyses, the franchise changed methods of recording combo meals (combinations of individual items offered together) in early 2017, requiring us to “unbundle” combos in the pre-implementation period (7% of total data). This necessitated assumptions about what was purchased in combos during this period (see supplementary methods).

We created indicator variables for season and the weeks of Thanksgiving and Christmas for each year because, by visual inspection, calories per transaction increased around these holidays. To determine the racial composition and median household income of the census tract of each restaurant we linked census tract level data from the most recent five year American Community Survey (2013-17) to geocoded restaurant addresses.26

We estimated the effect of calorie labeling on calories per transaction using an interrupted time series with segmented regression.27 This approach determines whether an intervention (calorie labeling) is associated with a post-implementation level and trend change in the outcome (calories per transaction). We implemented interrupted time series using linear mixed models with robust standard errors, random intercepts and slopes, and an unstructured covariance matrix. This model accounted for autocorrelation between measures of the same restaurants over time and simultaneously accounted for clustering of purchases within individual restaurants. Thus we allowed restaurants to vary in terms of baseline calories per transaction, the level change in calories per transaction, and the pre-implementation and post-implementation trends in calories per transaction.

Interrupted time series assumes that the pre-intervention trend would have persisted had the intervention not occurred (ie, the counterfactual). Therefore, before conducting the interrupted time series in the full data, we compared several models using only pre-intervention data to determine which best fit the pre-intervention trend. We fit a simple model that included only a continuous term for week (time trend) and models that included state, season, and holidays. The model adjusting only for season and holidays had the lowest Akaike Information Criterion and was therefore used as the primary model for the analysis.

For our main analysis we conducted the interrupted time series using all weeks of data except for the week of labeling implementation and the two weeks before and after to account for the anticipatory change and the potential slight variation in implementation rollout. The two primary measures of interest were the level change in calories per transaction immediately after implementation and the post-implementation change in the trend of calories per transaction independent of the baseline trend. To estimate these effects, we fit a model with a continuous term for week, representing the baseline trend in calories per transaction; an indicator variable for whether the week was before or after labeling implementation, representing the level change; a continuous term for weeks after implementation, representing the post-implementation change in trend; and indicators for season and holidays to adjust for seasonal variation in calories per transaction, as guided by our model selection strategy using the pre-intervention data. The supplementary methods show the full model.

We conducted several sensitivity analyses to test the robustness of our main findings. First, we included only restaurants with complete data for the 156 weeks (63 restaurants). Second, we included all 139 restaurants, regardless of whether they had sales in both pre-implementation and post-implementation periods. Third, we included only one year of pre-implementation data, in case this represented the true baseline trend. Fourth, we removed the 3% of items purchased that did not have high quality matches to Menustat data. Fifth, we analyzed the data using generalized estimating equations instead of mixed models. Sixth, for the combo meals that we unbundled, instead of assuming all customers chose the default option for each combo component, we assumed they chose the option in the same proportion as customers who bought that component a la carte during the same week. Lastly, after conducting the prespecified analyses, we learned that a side offered for 11 weeks (January to April 2018) was one of the most popular promotions in the franchise’s history. To examine whether this promotion might have affected calorie purchases (and thus our results), we conducted a post hoc sensitivity analysis that only included data from sales made before this promotion—this analysis included nine months of post-implementation data instead of the full year.

In secondary analyses we examined associations between labeling and calories per transaction separately among entrees, sides, and sugar sweetened beverages (we did not examine condiments because the mean was <5 calories/transaction), using the same model as for our primary analyses. For entrees and sides, we additionally adjusted for the 11 week period when the popular limited time side was offered because as the number of sides increased during this period, entree sales declined, suggesting that customers substituted entrees for sides during this period. We also conducted stratified analyses by the census tract racial composition (above or below median: 69.4% white) and median income (above or below median: $50 329 (£40 962; €46 000)/household) of restaurants. In sensitivity analyses, we used zip codes rather than census tracts because restaurants might serve people living in larger geographic regions,28 especially for those making purchases while commuting between work and home.

To determine whether reformulation could have contributed to calorie changes, we explored whether the calorie content of menu items changed from 2017 (pre-implementation) to 2018 (post-implementation) for the top 50 items purchased (74% of sales) in 2017-18 across the franchise. We also examined the distribution of calories among menu options in each category for each year of the study to explore whether consumers had enough choices to select lower calorie items. Lastly, we did an exploratory analysis to assess whether any changes in calories per transaction after labeling were driven by consumers purchasing overall lower calorie items or consumers purchasing fewer items per transaction. To do this we conducted interrupted time series analyses that were similar to our main analysis but used calories per item or items per transaction, respectively, as the dependent variable in our model.

We used SAS version 9.4 (Cary, NC) for all analyses and calculated two sided 95% confidence intervals for statistical tests. Any deviations from our analysis plan are explained in the supplementary methods.

This research was done without patient involvement. No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for recruitment, design, or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.

Results

Our main analysis included 59 restaurants (57%) in Louisiana, 41 (39%) in Texas, and four (4%) in Mississippi, constituting an analytic sample of 14 352 restaurant weeks. Restaurant census tracts showed varied racial (median per cent white: 69.4% (interquartile range 52.4-82.7%)) and socioeconomic characteristics (median income: $50 329 ($35 800-$66 120)). Over the study period, 242 726 953 items were purchased across 49 062 440 transactions; most items were entrees (72.8%) followed by sugar sweetened beverages (14.6%) (table 1). Because several restaurants newly opened toward the end of the pre-implementation period, slightly more transactions occurred in the last year (about 18 million) than in the first two years (about 15.5 million per year). As a result, these restaurants contributed more sales data to the last year of the study than to the first two years. Over the study period the mean calories/transaction was 1490 (SD 149).

Transactions, item purchases, and mean calories purchased in all participating restaurants overall and by calorie labeling implementation period

Slightly more transactions occurred in the last year (18 055 559) than in the first two years (average of 15 503 441 per year) because several restaurants were newly opened toward the end of the pre-implementation period. As a result, these restaurants contributed more sales data to the last year of the study than to the first two years.

Before labeling implementation, there was an estimated baseline increasing trend of 0.53 calories/transaction/week (95% confidence interval 0.36 to 0.70). After implementation, a level decrease was observed of 60 calories/transaction (95% confidence interval 48 to 72), representing an approximate 4% decrease. This was followed by a post-implementation trend increase of 0.71 calories/transaction/week (95% confidence interval 0.51 to 0.92) above and beyond the baseline trend (table 2; fig 2; supplementary fig 1). This increasing post-implementation trend meant that by the end of the study (ie, one year after labeling), the estimated reduction in calories per transaction was only 23 calories lower than the counterfactual. These results were similar in sensitivity analyses (supplementary table 1) except when only one year of pre-implementation data were used, in which case the level change was attenuated (level change −10 calories/transaction, 95% confidence interval −24 to 4), but the post-implementation trend change was stronger (trend change 1.40 calories/transaction/week, 95% confidence interval 1.12 to 1.68). We believe that our main analysis with two years of pre-implementation data is able to capture the baseline trend and adjust for seasonality better because it provided data during two of each season (instead of just one) and may therefore be more robust.

Model based estimates (β (95% CI))* of mean baseline level, baseline trend, and level and trend change in calories purchased per transaction after implementation of calorie labeling overall and by category

Adjusted for season and holidays (spring (reference), summer, fall, holidays (week of Thanksgiving to week of New Year’s), winter).

Model based estimate of number of calories at baseline (ie, week 1).

Interpreted as mean weekly change in number of calories purchased before implementation of calorie labeling (eg, for all purchases, the mean weekly increase was 0.53 calories per transaction before implementation).

Interpreted as mean change in number of calories purchased immediately after implementation of calorie labeling (eg, for all purchases, the mean decrease was 60 calories per transaction after implementation).

Interpreted as mean weekly change in number of calories purchased after implementation of calorie labeling independent of baseline trend (eg, for all purchases, the mean weekly increase was 0.71 calories per transaction after implementation in addition to the baseline trend of 0.53 calories/transaction/week).

Linear terms were included for an 11 week period when a high selling limited time side was offered.

Level and trend changes in mean purchased calories per transaction after implementation of calorie labeling. Light orange dots=mean calories per transaction for individual restaurants in a given week; dark orange dots=weekly mean calories per transaction over all restaurants. Purple line=predicted calories per transaction from linear mixed model, which is also adjusted for season and holiday periods (seasonal and holiday effects are shown in supplementary fig 1 panel B). Excluded from the model are transactions made the week of labeling implementation as well as the two weeks before and after, which are represented by vertical dashed lines

The association between calorie labeling and calories per transaction differed between item categories (table 2; supplementary fig 2). After implementation, a level decrease was observed of 11 calories/transaction (95% confidence interval 3 to 20) for entrees (about 1%) and 40 calories/transaction (32 to 48) for sides (about 24%), but no level change in calories per transaction for sugar sweetened beverages. A slight post-implementation trend decrease was observed for entrees (trend change −0.15 calories/transaction/week, 95% confidence interval −0.31 to 0.02) and sugar sweetened beverages (−0.29, −0.34 to −0.25), but a post-implementation trend increase was observed for sides (0.48, 0.35 to 0.61).

Post-implementation level and trend changes in calories per transaction were similar when stratifying by racial composition of restaurant census tracts (table 3). Level changes were also similar when stratifying by median household income of census tracts, but the post-implementation trend change appeared slightly stronger in lower income census tracts (level change in calories/transaction/week 0.94, 95% confidence interval 0.67 to 1.21) than in higher income census tracts (0.50, 0.19 to 0.81). To further explore this possible disparity, a post hoc analysis was done in which we estimated the effect of labeling in four groups split by quartiles of median income of census tracts (instead of dichotomizing above and below the median) and we observed even stronger disparities (supplementary table 2). Results were similar when stratifying by zip code level characteristics of the restaurants rather than census tract level characteristics (supplementary table 3).

Model based estimates (β (95% CI))* of mean baseline level, baseline trend, and level and trend change in calories purchased per transaction after implementation of calorie labeling by characteristics of restaurant census tracts

Adjusted for season and holidays (spring (reference), summer, fall, holidays (week of Thanksgiving to week of New Year’s), winter). All subgroups comprised 52 restaurant locations.

The top 50 menu offerings purchased in 2017-18 had a median of 350 calories (interquartile range 440-760) pre-implementation and a median of 340 calories (440-760) post-implementation. These offerings included one item that was removed and two that were newly added in 2018. Of the 47 items that remained on the menu in both years, 31 (66%) did not change in calorie content post-implementation, 10 (21%) increased by 20 calories or fewer, five (11%) decreased by 20 calories or fewer, and one (2%) decreased by 50 calories. In general, the distribution of calories among menu options in each category were similar over time (supplementary table 4). Each category contained a wide range of calorie options, suggesting a high potential for consumers to change their calorie purchases.

In exploratory analyses we found that calorie labeling was associated with a minor level increase of 4.1 calories/item (95% confidence interval 2.2 to 5.9; about 1% increase) but a level decrease of 0.28 items/transaction (0.20 to 0.35; about 5% decrease). This finding shows that the 4% level decrease in calories per transaction in our main analysis might have been largely driven by consumers purchasing fewer items rather than purchasing lower calorie items (supplementary table 5). A post-implementation trend increase was also observed in numbers of items per transaction (trend change 0.005 items/transaction, 95% confidence interval 0.004 to 0.006), which could explain the attenuation of the decrease in calories per transaction over the course of the study.

Discussion

In this large, quasi-experimental study of fast food restaurants in the southern US, calorie labeling was associated with a decrease of 60 calories per transaction (4%) after implementation, followed by a small weekly increase over the next year, independent of the baseline trend. The estimated effects of labeling differed slightly by median restaurant census tract income and category of item purchased, but changes were small. The increasing pre-implementation trend we observed shows increasing or larger calorie purchases of fast food over time that was independent of labeling, which has been documented in prior studies.29 30

Given that about one third of daily calorie intake in the US comes from away-from-home purchases30 and one third of US adults consume fast food daily,31 the initial 60 calorie per transaction decrease might have led to slight improvements in population level diet quality. However, there was a weekly increase in calories per transaction independent of the baseline trend, such that by the end of the study the mean calories per transaction decreased by only 23 compared with the counterfactual (ie, the association diminished 62% over one year). Additional data are needed to determine whether this trend will eventually meet or even exceed the counterfactual, or whether it will plateau. Under the optimistic scenario that the trend plateaus, and if we assume that the average American consumes fast food about once every three days,30 31 this would result in an approximate decrease of 8 calories/day, which roughly translates to a loss of one pound (0.45 kg) over three years in adults32 and a smaller effect over a shorter period in children and adolescents.33 Although this suggests minimal individual level impact, microsimulation studies indicate that on a population level even such a small decrease in calories could result in tens of thousands of fewer cases of obesity and substantial savings in annual healthcare costs.21 Based on our results, however, the benefits of calorie labeling would be even smaller than past projections because the reduction in calories per transaction we observed likely reflects purchases for multiple people.

The post-implementation trend increase we observed could have occurred if customers initially responded to the novelty of calorie labels but stopped noticing them over time. Our exploratory analyses found that the changes in calories per transaction after labeling were driven primarily by an immediate small decrease in the number of items purchased per transaction, followed by a gradual increase over time. Meaningful changes did not seem to occur in the calorie content of purchased items after labeling; if anything, there was a slight increase. Consumers might have limited ability to purchase lower calorie items if restaurants offer few low calorie options, but the restaurants in this study offered items with a wide range of calorie content over the study period. We found little reformulation of top selling items, which may have reduced the anticipated effect of labeling. It is possible that these restaurants will reformulate existing menu offerings and add more low calorie offerings in the future, as there could be a lag between labeling implementation and reformulation. Chains also might have waited to reformulate their menus until after labeling became official nationwide policy in May 2018, which was one month after our study ended. Few studies have examined whether food retailers directly respond to labeling by reformulating items to be lower calorie or by introducing new lower calorie items, though one study found that the calorie content of menu items at 37 restaurant chains declined on average by 41 calories after labeling.8

Restaurants in lower income census tracts showed a greater post-implementation trend increase in calories per transaction than restaurants in higher income census tracts. We observed this when examining calorie labeling in two subgroups (high and low median household income) and found stronger disparities in post hoc analyses of four groups split by quartiles of median household income. These results suggest that any decrease in calories per transaction might attenuate faster among people with lower incomes. Because we had aggregated purchase, not individual, level data, however, these results should be viewed cautiously.

Calorie labeling was associated with an initial 1% decrease in calories per transaction from entrees and a 24% decrease in calories per transaction from sides; the post-implementation trend increased for sides but decreased marginally for entrees. One possible explanation is that immediately after labeling, customers were more likely to reduce calorie purchases from sides (which included desserts) rather than from entrees, but then resumed their pre-implementation purchasing habits as they became accustomed to the labels. Evidence supporting this hypothesis is mixed. One cross sectional study after calorie labeling reported that users of calorie labels (versus non-users) were more likely to select healthier side dishes, but no difference was found for entrees.34 In contrast, a prior experimental study found that labeling led to decreases in calorie purchases from entrees but no changes in calorie purchases from supplemental items, including sides.35 The decreasing post-implementation trend we observed for sugar sweetened beverages was small but could be meaningful over time if it persisted beyond one year after implementation.

Although most quasi-experimental studies conducted in fast food settings have not detected associations between calorie labeling and calorie purchases,14 15 16 17 18 these studies were generally not powered to detect small differences. One important exception was a study19 that analyzed more than 100 million transactions and found a 15 calorie/transaction (6%) decrease post-implementation in Starbucks restaurants in New York City from 2008 to 2009, compared with Starbucks restaurants in Boston and Philadelphia. That study also found that decreases in calories per transaction were primarily due to consumers being less likely to purchase a food item, rather than substitutions for lower calorie items, as we found in our exploratory analyses. However, the Starbucks study did not report a post-implementation trend change specifically, making it difficult to compare directly with our study. Compared with previous studies, our study used more recent data over a longer period and sampled restaurants in the southern US. Despite these differences, our study is consistent with previous studies reporting only small decreases in calories per transaction after implementation of calorie labeling in fast food restaurants. Studies suggest that associations of calorie labeling in sit-down settings might be stronger,36 37 38 39 40 though few studies have been conducted in these types of restaurants.7

Strengths of this study include its large sample size; inclusion of all sales and thus all customers of the franchise over a three year period, reducing concerns related to selection and generalizability; use of objective sales data rather than relying on participant recall; and diversity of restaurants’ locations. Additionally, our findings were robust to various analytic decisions, though the estimated effect of labeling was attenuated (and the post-implementation trend stronger) when we used one year of pre-implementation data. We believe that our primary analysis using two years of pre-implementation data are preferred because it included more data and likely adjusted better for seasonality.

Our study also has limitations. First, because we only had weekly aggregated sales data, we could not adjust for or conduct analyses by individual participant characteristics. Although interrupted time series is generally robust to confounding by individual level characteristics,41 not knowing how many people were included in each transaction limited us to population level inferences only. Moreover, because we did not know the number of people represented by each transaction (and therefore could not calculate calories purchased per person), we were unable to determine whether meals met or exceeded recommended guidelines for calorie intake. Second, we did not have a control group. We therefore had to estimate the counterfactual calories per transaction using only the pre-intervention data, which is more vulnerable to time varying confounding. Third, we did not have data on most meal modifications (eg, adding condiments, removing ingredients), beverage refills, or consumption. If calorie labeling compels people to make healthy modifications or eat less of their meal instead of changing items purchased, our results would underestimate the effect of labeling. Fourth, the indicators we chose for our subgroup analyses (median household income and per cent non-white residents) are commonly used area level demographic measures but might not fully capture the socioeconomic status of restaurant neighborhoods.42 Lastly, if labeling caused some people to stop purchasing food from this franchise, our results would underestimate the effect of labeling on calories purchased in these restaurants. The effect this would have on overall diet quality is unclear because these people could have consumed higher or lower calorie meals than what they would have purchased at the franchise had labeling not been implemented. Total transactions were stable after labeling, potentially minimizing this possibility, but we could not formally assess this with our data.

In this sample of 104 fast food restaurants in the southern US, calorie labeling was associated with a small decrease in mean calories per transaction after implementation, but this was followed by a gradual weekly increase that partially attenuated this association over the next year. These results imply that calorie labeling alone may not be enough to make sustainable reductions in calorie intake in fast food restaurants. Several other nutrition labeling approaches are in use outside of the US. Evidence from these, such as pictorial front-of-package labeling,2 13 is also mixed, although “traffic light” labeling might be more effective.13

Before drawing conclusions on the overall effectiveness of calorie labeling as a nutrition policy, future research should be done to estimate the effects of labeling over a longer period, especially once restaurants have had sufficient time to reformulate their menus. Additionally, labeling could increase purchases of healthier products without changing energy intake.13 Few studies have evaluated overall diet quality of purchases in response to labeling. Lastly, little research has been done on the effects of calorie labeling in large full service restaurant and supermarket chains, where calorie labels are required for prepared foods.7

Calorie labeling has been required in chain restaurants in the US since May 2018, but evidence for this policy’s effects on calorie purchases is mixed and incomplete

Most previous studies were small and assessed calorie purchases using participant surveys, raising questions related to selection and recall

Calorie labeling has not been properly evaluated in non-urban settings or in the southern US, despite obesity rates in this region being among the highest in the country

Calorie labeling in a large restaurant franchise in the southern US was associated with a modest decrease in calories purchased, but this association diminished over time