We conducted a retrospective ecological analysis using anonymized, aggregated data from 5 BLS implemented by Meta on Facebook and Instagram. BLS are randomized ad-exposure experiments embedded within the social media platform, where users are automatically assigned to test (exposed) or control (unexposed) conditions. Because no individual-level longitudinal data, identifiers, or person-level outcomes are produced, this study does not constitute a retrospective cohort or case-control design. All analyses were performed using the aggregate outputs supplied by Meta.

All BLS results are aggregated and anonymized. BLS results for Meta are disaggregated by age and sex. Each audience member is likely to be presented with only 1 question in their newsfeed. This implies that all creative content should align with the BLS questions, as there is no control over which creative or question they will encounter. Facebook considers any Brand Lift result that shows a 2 percentage point or higher increase with at least a 90% probability of a Brand Lift to be statistically significant. Below are several questions to consider if a Brand Lift result greater than 2 percentage points is observed.

For the implementation of the BLS programs, an agreement was signed for the “Scaled Program Support” Program Participation between Facebook Inc and Asociación Mexicana de Vacunología.

The results provided by Facebook do not contain personal information about the users; they are only statistics about the BLS as presented in Tables 1-3, which were used to obtain the statistical results. The data were obtained by the BLS tools of Facebook Inc, in accordance with its privacy policies [13]. Table 1 summarizes problem statements, behavior change and communication goals, target audiences, survey questions, advertising objectives, campaign duration, and investment.

The Asociación Mexicana de Vacunología A.C. accepted and approved the use of the results of the BLS performed for the realization of this study.

The Asociación Mexicana de Vacunología (@Vacunologia) participated in Facebook’s 2021 and 2022 Scaled Workshop Social and Behavior Change Communication (SBCC) Program. As part of this program, the Asociación Mexicana de Vacunología received Facebook Ad Credits and support to conduct a public health campaign on Meta. The campaign characteristics are detailed in Table 1.

We analyzed information from 5 BLS conducted by the Asociación Mexicana de Vacunología on Facebook and Instagram (@Vacunologia). BLS are experimental tools developed by Meta to evaluate the impact of advertising campaigns independently of other marketing activities. They work by randomly dividing a representative sample of eligible users into 2 groups: a test group, which is exposed to the ads, and a control group, which is not. Both groups are then asked standardized poll questions, and the difference in responses indicates the incremental effect, or “lift,” attributable to the campaign [14].

Randomization and audience segmentation are performed automatically by Meta’s platform, using predefined targeting criteria such as age, sex, location, or interests. This design ensures internal validity and allows causal inference about the effect of ad exposure [15]. Lift is reported as the percentage point difference between the test and control groups, accompanied by a confidence estimate. Outcomes typically include awareness, recall, or perceptions related to the advertised content.

The study population consisted of adult Facebook and Instagram users in Mexico who were eligible to be exposed to the campaigns according to predefined targeting criteria (eg, all adults nationwide, or women aged 18‐44 years in major cities). Randomization was performed automatically by Meta’s BLS platform at the user ID level, assigning individuals to either the test group (with the opportunity to see the ads) or the control group (withheld from exposure). This design ensured balance between groups and minimized systematic differences at baseline. The only demographic variables available for analysis were age group and sex, which were provided in disaggregated form in the BLS reports. Because the data were anonymized and aggregated, no additional covariates were accessible.

Statistical analyses included 1-way ANOVA to compare mean scores across campaigns, unpaired t tests for sex differences, and Fisher exact tests to examine associations between campaign characteristics and the probability of achieving a Brand Lift (≥80% or ≥90% confidence). Bayesian confidence estimates from Meta’s BLS are not equivalent to frequentist P values. Frequentist tests (ANOVA, t tests, and Fisher exact tests) were used only for comparisons derived from the summary metrics provided in Tables 2-7.

Bayesian confidence estimates from Meta’s BLS reflect the probability of observing the same incremental lift in a repeated randomized sample, and therefore should not be interpreted as frequentist measures of statistical significance.

This study was based exclusively on secondary, anonymized, and aggregate data provided by Meta’s BLS platform and did not involve any direct interaction or intervention with human participants. In accordance with national and international ethical guidelines, including the Declaration of Helsinki and the Council for International Organizations of Medical Sciences (CIOMS) International Ethical Guidelines, this type of research does not require formal review or approval by an institutional ethics committee. The Asociación Mexicana de Vacunología, as the coordinating institution, authorized the use of these data for academic and research purposes.

Because no individual-level data were accessed and no direct interaction with participants occurred, informed consent was not required. The original data collection by Meta was conducted under its own user agreements and privacy policies, which include provisions for the use of anonymized, aggregate data for research purposes. All data analyzed were anonymized and aggregated prior to researcher access. No personally identifiable information was available to the research team. Data were handled in accordance with Meta’s privacy policies and the Asociación Mexicana de Vacunología’s internal data protection standards.

No participants were recruited directly for this study, and therefore, no compensation was provided. No images or supplementary materials containing identifiable individuals are included in this paper.

This study adheres to the APA Journal Article Reporting Standards for Quantitative Research (JARS-Quant), which provide structured guidelines for reporting research design, participant characteristics, measures, and analytic procedures. We followed the recommendations outlined in the APA JARS-Quant Table 1 to organize the “Methods” section and ensure transparent and comprehensive reporting of quantitative analyses [18].

To our knowledge, this is the first study to use experimental BLS data to evaluate vaccine communication in Mexico, providing a scalable model for assessing digital health interventions in other LMICs. This study evaluated the effectiveness of 5 large-scale social media advertising campaigns on COVID-19 vaccination in Mexico, using Meta’s BLS to assess standard ad recall and perceptions of vaccine importance, safety, efficacy, and side effects. The findings demonstrate that digital campaigns can generate measurable shifts in awareness and attitudes, although the magnitude and consistency of these effects varied across campaigns and subgroups. These results partially confirm our hypothesis that targeted campaigns, particularly those directed at younger populations and women, would generate stronger positive perceptual shifts.

Standard ad recall increased significantly in 4 of the 5 campaigns, with the women-focused campaign (aged 18‐44 years in major cities) achieving the largest effect (+11.6 percentage points compared to control). Perceived importance of vaccination improved in 3 of the 4 campaigns where it was measured, with incremental lifts ranging from +1.4 to +3.2 points. Perceptions of safety showed modest but positive gains in 2 campaigns (up to +2.5 points), while efficacy perceptions improved only marginally (≤1.1 points). Concerns about side effects did not change significantly. These results indicate that recall and importance were the most responsive outcomes, whereas safety and efficacy perceptions were more resistant to change.

Targeted campaigns produced stronger effects than general population campaigns. In particular, the women-focused campaign not only achieved the highest recall but also showed significant improvements in safety perceptions. In contrast, Campaign 5, which addressed doubts about booster effectiveness, did not produce significant changes in recall or perceptions. This suggests that audience segmentation and message framing are critical determinants of campaign success [3,4].

Younger audiences (18‐34 years) consistently exhibited stronger positive shifts in recall and importance compared to older groups. For example, in Campaign 3, recall increased by +5.0 points among men aged 18‐24 years, while no significant change was observed in older age groups. No consistent differences were observed by sex, except in the women-targeted campaign, where effects were concentrated among the intended audience. These findings align with prior evidence that younger, digitally native populations are more responsive to online health interventions [2,3].

Our results are consistent with international evidence showing that digital advertising can shift health-related attitudes [3,4]. The stronger effects observed among younger audiences and in targeted campaigns suggest that segmentation strategies enhance the effectiveness of digital health communication [2]. However, the limited changes in perceptions of efficacy and side effects highlight the challenges of addressing entrenched concerns through short-format digital ads alone. These dimensions may require complementary strategies, such as longer-form educational content or integration with offline communication [9].

This study has several limitations. First, outcomes were limited to recall and perceptions; no direct behavioral measures (eg, vaccination uptake) were available. Thus, while perceptual shifts are important precursors to behavior, they cannot be equated with actual vaccination. Although the BLS do not capture behavioral end points such as vaccination uptake, these attitudinal outcomes are therefore considered health-related because they influence individuals’ likelihood of seeking vaccination and engaging with preventive health services.

Second, although randomization at the user ID level ensured balance between test and control groups, the data available to researchers were anonymized and aggregated, with only age group and sex provided in disaggregated form. As a result, we could not control for additional individual-level covariates (eg, prior exposure to vaccine information and baseline attitudes), which may have introduced residual confounding in subgroup analyses. Third, small sample sizes in some subgroups reflect the BLS survey design, where each respondent answers only 1 question, and should be interpreted with caution. Finally, the findings represent the digital population and may not generalize to individuals without internet or social media access. Our study includes a lack of individual participant data for conducting a cost-benefit evaluation and insufficient information on the number of vaccines administered in Mexico during the campaigns to assess their impact on the vaccination program. It is important to note that not all segments of the Mexican population use digital platforms. Therefore, the findings of this study primarily represent the digital population and may not be generalizable to those who are not connected to or active on social media platforms.

Despite these limitations, this study provides novel evidence on the role of social media in vaccine communication in a middle-income country. The results indicate that digital campaigns can effectively increase standard ad recall and improve perceptions of vaccine importance and safety, particularly when targeted to specific audiences. These findings reinforce that audience segmentation—especially toward younger adults and urban women—enhances the effectiveness of digital health messaging compared with broad, non-targeted approaches.

This study offers an innovative application of Meta’s experimental BLS data to evaluate large-scale, real-world vaccination campaigns in Latin America—a setting where such experimental evidence remains scarce. By demonstrating that different demographic groups respond differently to campaign messages, the study provides practical insights for designing more efficient communication strategies.

These findings have several implications for public health practice. First, digital vaccination campaigns should prioritize precise audience segmentation to maximize impact while optimizing resources. Second, because BLS provide rapid, experiment-based feedback, they can be incorporated into real-time monitoring systems that allow authorities to adjust message framing, creative content, or targeting while campaigns are still active. Third, combining digital strategies with community-based interventions may help reinforce attitudinal changes and extend impact beyond online spaces.

For policymakers, this study illustrates how experimental digital metrics can complement traditional epidemiological surveillance. Future digital vaccination efforts in Mexico and other LMICs should integrate attitudinal indicators (eg, confidence and perceived safety) with real-world vaccination uptake from national immunization systems. Linking shifts in vaccine acceptance with the number of doses administered during campaign periods would provide a more robust measure of true impact and generate evidence to guide investment in digital public health interventions.

Overall, this study contributes to the field by offering a replicable framework for measuring attitudinal effects of social media advertising using experimental causal methods seldom applied in LMIC settings. In practical terms, the findings demonstrate how digital platforms can complement traditional communication strategies by improving awareness and perceptions in key demographic groups, guiding authorities in resource allocation, message tailoring, and integration of digital analytics into broader vaccination efforts.

This study provides new evidence on the effectiveness of large-scale social media campaigns to influence vaccine-related perceptions in a Latin American middle-income country. Across 5 Meta BLS, digital advertising significantly increased standard ad recall and improved perceptions of vaccine importance and safety, particularly among younger adults and women in urban areas.

Our findings demonstrate that audience segmentation and message targeting enhance the impact of digital communication, suggesting that health authorities and organizations can use social media strategically to strengthen vaccine confidence. Although the study did not measure vaccination behavior directly, perceptual shifts represent important precursors to uptake and highlight the potential of digital platforms to complement traditional public health strategies.

This work contributes to the field by applying experimental BLS for LMICs data to evaluate real-world public health communication, where evidence on digital health interventions remains limited. These results can inform the design of future digital campaigns, guide resource allocation, and support the integration of digital analytics into broader immunization efforts. Future research should link digital engagement metrics with epidemiological data to assess downstream behavioral outcomes.

There is no doubt that there is still much work to be done, but it is of great importance for health specialists to continue collaborating and engaging audiences in digital media. By creating experts in the dissemination of medical information and generating behavioral changes, we can improve the health of the population. However, to identify the best strategies, it is essential to generate evidence to use as a basis for creating, designing, and evaluating successful campaigns that lead to real behavioral changes in health worldwide.