The dissemination of disordered information on social media has long been a critical area of study, significantly impacting public interest [1]. With rapid advancements in technologies such as deepfakes [2] and generative artificial intelligence (AI) [3,4], individuals now have various tools, sources, and channels at their disposal to create misinformation or even false information. Although technological developments offer convenience [5], they also inevitably complicate the fact-checking and governance of misinformation on social media. In particular, in the era of generative AI, such technologies can become crucial for producing misinformation, either actively or passively [6], including news production [7] and health care provision [6]. Research observed that, even before the advent of ChatGPT (based on Generative Pre-trained Transformer [GPT]-3.5), humans struggled to distinguish between AI-generated content (AIGC) and human-generated content (HGC) during the GPT-2 era. This is particularly concerning for medical information that requires rigorous verification, as it can have a significant impact on health care and the general public [8]. Although generative AI can assist with writing summaries, the credibility and accuracy of AIGC are not guaranteed to be 100% [9,10]. This ongoing issue underscores the challenge of effectively governing misinformation as AI capabilities continue to advance [7].

AI is iterating at an unimaginable pace, leading to an even more challenging distinction between AIGC and HGC for the average internet user in the future. What is more important, as AIGC increasingly becomes part of social media content, driven by online traffic and profit motives, the presence of misinformation will be inevitable. Therefore, it is essential for platforms to implement governance measures to help users differentiate between AIGC and HGC. Previous research [11] has proposed nudging and boosting as 2 types of interventions, with nudging involving the integration of cognitive cues into interface design through proactive notifications. Current popular measures like accuracy prompts [12] and warning labels [13] fall under the category of nudging interventions. Research has shown that nudging can effectively reduce the spread of misinformation [13]. However, the effectiveness of nudging interventions can be influenced by factors such as political affiliation [7] and geographical area (urban, suburban, rural) [14]. Further, the effects of nudging interventions might also lead to divergent attitudes and unintended consequences [15], such as the implied truth effect, where the absence of interventions could imply the veracity of information [16], illustrating the complexity of implementing such measures. Clearly, the factors affecting nudging and the additional impacts caused by nudging are extensive and multifaceted.

Facing the rise in generative AI, in 2023, the Chinese social media platform Douyin (Chinese version of TikTok) issued a platform operation policy requiring content creators to prominently label AIGC [17], with similar initiatives seen on other platforms like Zhihu and Little Redbook (Xiaohongshu). These AIGC labels, akin to accuracy prompts [12], serve as a nudging intervention. Despite the widespread adoption of these labels, research on the factors that influence their effectiveness and the consequences they yield remains sparse. Previous studies have uniquely combined electroencephalogram technology with behavioral experiments to explore how AIGC labels impact users’ perceptions of automated news, including their effect on content credibility and whether the news type interacts with these labels, alongside cognitive-physiological impacts [18], indicating that AIGC labels prompt users to engage in deeper information processing, consequently lowering the perceived credibility of the content. This suggests that, although AIGC labels are intended to enhance transparency and reliability, they may paradoxically lead users to view labeled content with increased skepticism, highlighting a complex dynamic between labels’ presence and users’ perceptions [18].

AIGC labels do not exist in isolation; they may be influenced by various contextual factors. It is necessary to integrate AIGC labels with the surrounding context, such as combining them with images, videos, advertisements, and other information. AIGC labels may interact with these factors as described in the following paragraphs [19].

First, regarding information type and AIGC labels, the governance of misinformation or inaccurate information has been a crucial focus of academic research. The regulation of misinformation dissemination when AI is involved has become a new issue. Previous studies have found an interaction effect between accuracy prompts and information type, known as the “implied truth” effect, where people may mistakenly perceive unlabeled misinformation as more credible [16]. This finding has also been supported in research on labels related with stance and credibility, which may inadvertently promote the spread of misinformation [20]. Thus, there can be an interaction effect between inaccurate information and AIGC labels, as well as between accurate information and AIGC labels. In summary, there could be a potential interaction effect between information type (accurate and inaccurate information) and AIGC labels.

Second, in terms of content category and AIGC labels, the effectiveness of AIGC labels may vary depending on the content category, demonstrated by the presence of cases in which AIGC labels are less effective for certain types of content [13]. Previous research on labels has typically examined them in isolation [21], but some researchers have begun studying the interaction between labels and information types from a contextual perspective [22]. Though there is substantial research on nudging interventions in specific content areas, such as political content [13], climate change [23], and pandemic-related information [24], little research has explored the relationship between nudging interventions and content category. Based on the profitability of the content, online content can be discretely divided into for-profit and not-for-profit categories, which is a common classification method [25]. This aligns with the current landscape of social media, which is filled with for-profit content, such as advertisements, and not-for-profit content, such as news. From a content category perspective, the aforementioned studies mainly focus on not-for-profit content. However, for-profit content may also influence the effectiveness of AIGC labels. Additionally, different topics within the same content category may produce unexpected effects [26].

In addition, the impacts of AIGC labels have extended beyond awareness levels. According to recent studies, such nudging interventions not only influence message credibility [18,23] or perceived accuracy, which is the capability to differentiate between accurate and inaccurate information to some extent [26], but also affect sharing intention [27]. This phenomenon can be attributed to a psychological inoculation effect induced by AIGC labels [28], which acts as a preemptive defense against misinformation, giving users a latent resistance before they encounter false information. Therefore, by preemptively introducing users to the concept of AIGC, these labels do impact message credibility, sharing intention, and even the capability to identify misinformation.

AIGC has visibly altered the content production and dissemination ecosystem on social media. In practical applications, AIGC labels have become cognitive cues on some social media platforms to help users distinguish between AIGC and HGC, functioning through nudging interventions. However, theoretical and empirical research related to AIGC labels is significantly lacking, and extensive research around AIGC labeling is required. Therefore, this study aimed to investigate AIGC labels to guide practical application. This research focused on addressing 2 main issues: (1) the predictive factors that influence the effectiveness of AIGC labels as a nudging intervention and (2) the impacts that these AIGC labels have on social media users. This study used a web-based experimental method. The independent variables were the AIGC labels (presence vs absence), information type (accurate vs inaccurate), and content category (for-profit vs not-for-profit). The dependent variables were perceived accuracy, message credibility, and sharing intention.

This study aimed to address the following research questions:

This study examined 3 independent variables, with 1 between-subjects variable, namely the AIGC labels (presence vs absence), and 2 within-subjects variables, namely the information type (accurate vs inaccurate) and content category (for-profit vs not-for-profit). Based on this, a 2×2×2 mixed experimental design with 2 groups—an experimental group with AIGC labels and a control group without AIGC labels—was used. This study was not a clinical trial to recruit participants but a web-based experiment to measure the effect of internet governance measures, so clinical trial registration was not required. Participants in both the experimental and control groups were exposed to 4 combinations of information type and content category, resulting in a total of 8 experimental conditions. Hence, this study used a 2×2×2 mixed experimental design, as outlined in Table 1.

This study recruited participants through the data platform Credamo [29], which boasts a representative sample size of over 3 million people from China and has been used by multiple universities and research institutions [30-32], with publications in top-tier journals across various fields. There were no specific demographic quota requirements for the participants in this study. On the Credamo data platform, 957 participants were recruited, with 476 in the group with AIGC labels and 481 in the group without AIGC labels. The data for this study were collected in April 2024. The participants agreed to participate and received a small compensation for their participation.

To ensure the data quality of the online experiment, this study recruited 7 participants to take part in an offline behavioral experiment before the formal experiment. The participants were instructed to complete the task with both attention and speed, resulting in a minimum time threshold of 151.8 seconds to complete the experiment (Table 2). Thus, participants in the formal online experiment who completed the survey in less than 151.8 seconds were excluded from the analysis, as this was deemed insufficient to fully engage with the experimental content. A total of 157 invalid responses were removed. The final valid sample consisted of 800 participants, with 400 in the group with labels and 400 in the group without labels. All participants were Chinese (254 women and 546 men), and the majority had at least an undergraduate-level education (736 individuals).

At the conclusion of the experiment, participants were asked to provide demographic information including their gender (male, female), age (<18, 18-25, 26-29, 30 years), exact date of birth (by selecting a specific date), level of education (high school or below, undergraduate, master graduate, doctorate or above), and primary city of residence (by selecting both the provincial and municipal area).

Descriptive analyses were first conducted to examine demographic characteristics and outcome data. The influence of 3 variables—AIGC labels (presence vs absence), information type (accurate vs inaccurate), and content category (for-profit vs not-for-profit)—on perceived accuracy, message credibility, and information sharing intention was then assessed using repeated measures ANOVA in SPSS version 29 (IBM Corp). When significant interaction effects were identified, simple effect analyses were performed to explore specific differences among the conditions.

An a priori power analysis was conducted using G*Power software [47]. Based on an assumed medium effect size (f= 0.25), at least 36 participants were needed to achieve adequate power, with a minimum of 18 participants per experimental group, using a significance threshold of α=.05 and aiming for a power of 95%. Statistical analysis was performed using SPSS version 29 [48]. To accommodate the unpredictability associated with mobile experiments, the sample size was increased to 800 participants.

On April 22, 2024, this study obtained ethical approval from the ethics committee of the School of Journalism and Communication, Beijing Normal University (approval number BNUJ&C20240422002). The research commenced after obtaining informed consent from all participants, who were informed that their data would be used anonymously and that they could withdraw from the study at any time without penalty. Throughout the study, all data were collected and reported anonymously to ensure participant confidentiality, and no identifiable personal data were included. Upon completing their participation, individuals received a small remuneration of ¥1 (US $0.14) through the Credamo platform.

By April 2024, this study recruited 957 participants. Of these, 157 participants (16.4% of the initial sample) were excluded due to their extremely short completion times, suggesting that they did not thoroughly engage with the experimental materials. Consequently, the final valid sample comprised 800 participants, randomly divided between 2 groups: 400 in the group with AIGC labels and 400 in the group without AIGC labels. The demographic data are shown in Table 5.

This study aimed to assess the impact of a new internet governance initiative, AIGC labels, and explore factors that might influence their effectiveness. The findings revealed that AIGC labels do not significantly affect perceived accuracy, message credibility, or sharing intention of users. AIGC labels function as a nudging intervention, helping users distinguish between AIGC and HGC. However, AIGC labels serve an informative role [11], intending to alter cognition rather than actively guide users’ behaviors, which may explain why they have limited influence on users’ perceptions. Importantly, they do not worsen users’ views of platform content or the platform itself, indicating that AIGC labels are a viable strategy for fact-checking and internet governance.

On a physiological level, cognitive neuroscience explains the underlying mechanism of the informative function of AIGC labels, which increase users’ attention to the content they read and deepen cognitive processing, leading to more cautious judgments [18]. This has been corroborated by other studies, which found that labels prompting users to recognize content from minority or anonymous sources encourage more careful consideration of such information [50]. This study highlighted the potential of AIGC labels to foster more cautious content evaluation, contributing to the adaptive development of a nudging [11] approach in internet governance in the generative AI era.

In fact, AIGC labels also play a crucial role in alignment. Current research on AI alignment focuses primarily on value and moral alignment [51-53], but alignment is a complex system-level issue that should not be limited to data and code between humans and AI. Focusing solely on techniques to learn from feedback and handle distribution shifts [54] is a relatively narrow approach. More importantly, we need to address aspects that have been previously overlooked—namely, exploring alignment from the perspective of human-computer interaction and interface design [55]. AIGC labels, by reshaping the interface of the virtual ecosystem, add nudging value, helping users align AIGC with HGC. Addressing this issue requires collaboration between governments, social media platforms, and content creators and producers to label AIGC. This can restructure the online environment without compromising users’ choices, using subtle symbolic cues to prompt users [11] and influence their cognition and behavior. Overall, AIGC labels can enrich the theoretical framework of AI alignment, suggesting new directions for enhancing the integration of AI technology into social platforms.

In addition, AIGC labels do not function in isolation, instead interacting with both the information type and content category to create a comprehensive effect. This study examined how AIGC labels influence perceived accuracy, message credibility, and sharing intention across both accurate and inaccurate information and for both for-profit and not-for-profit content. Notably, our findings indicated that both the information type and content category significantly affect these perceptions, and in particular, accurate information and not-for-profit content were perceived more favorably than their counterparts. Although there is extensive research [56,57] highlighting a general distrust and negative attitude toward for-profit content [58], our results revealed a significant interaction between information type and content category.

This study focused on the specific impact of AIGC labels under various informational conditions. We found that, for accurate information, whether for-profit or not-for-profit content, the presence of an AIGC label slightly reduced perceived accuracy, message credibility, and sharing intention, though these effects were not statistically significant. This suggests that, although AIGC labels do influence users’ perceptions, their impact is relatively modest and manageable. For inaccurate information or misinformation, when the content is for profit, AIGC labels slightly enhanced perceived accuracy while slightly decreasing message credibility and sharing intention. However, these effects were also not significant. Notably, the degree of decrease was less than that observed for accurate information, likely because people inherently perceive inaccurate information as less credible and less shareable than accurate information. Additionally, when the content is not for profit, AIGC labels tend to increase sharing intention to some extent and slightly improve perceived accuracy and message credibility, though these effects remain statistically insignificant. It is important to investigate what specifically enhances perceived accuracy, message credibility, and sharing intention about misinformation under the influence of AIGC labels.

Nudging through labels may also lead to unexpected psychological effects. Another nudging intervention, warning labels, has revealed the implied truth effect [16], meaning that people tend to perceive content without warning labels as more credible. In contrast, AIGC labels have produced a truth effect for misinformation, where people generally perceive misinformation with AIGC labels as more accurate and credible and are more willing to share it (without knowing the information is inaccurate). One possible explanation is that AIGC labels enhance users’ attention and increase the complexity of cognitive processing [18]. Another reason could be due to cognitive elaboration effects [59], as people are reluctant to engage in deeper thinking and reasoning [60]. Previous research has found that inaccurate information is 70% more likely to be shared than accurate information [56]. Humans are not robots, and social media technology amplifies the spread of inaccurate information [56]. This study revealed that generative AI technologies may also intensify this issue. However, there is no need for excessive concern; according to our findings, AIGC labels did not significantly influence perceived accuracy, message credibility, or sharing intention, as these impacts were within manageable limits.

The findings of this study provide important insights for misinformation governance and content moderation. Although AIGC labels did not significantly impact perceived accuracy, message credibility, or sharing intention for most content, they did slightly increase these metrics for misinformation, though not to a statistically significant degree. This highlights the potential value of AIGC labels in the governance of AI-driven misinformation. Rather than rejecting AIGC labels, platforms should consider incorporating them to enhance oversight. AIGC labels not only guide users but also offer regulatory benefits by circumventing issues related to watermarks [61], thereby raising awareness of AIGC and promoting more cautious engagement, particularly in areas prone to misinformation.

Additionally, the limited impact of AIGC labels on users’ perceptions and behaviors suggests the need for complementary strategies. Platforms may need to integrate AIGC labels with fact-checking systems or educational initiatives to more effectively curb the spread of inaccurate content.

This study offers several key implications for platforms, content creators, and regulators. First, although AIGC labels have a modest effect on perceived accuracy, message credibility, and sharing intention, platforms should prioritize improving label design to ensure clear identification of AIGC. Enhancing transparency in this way can build trust, especially when distinguishing between AIGC and HGC.

Second, the slight increase in perceived accuracy for profit-driven misinformation labeled as AIGC suggests a need for stronger content moderation. Platforms should adopt more advanced systems to prevent AIGC labels from unintentionally legitimizing false content. A combination of sophisticated algorithms and human oversight could mitigate these risks.

Third, for not-for-profit misinformation, AIGC labels may slightly raise sharing intention and perceived accuracy. Although these effects are not statistically significant, platforms should be cautious when applying AIGC labels to not-for-profit misinformation, as this could inadvertently amplify its dissemination. Providing additional context or adjusting label visibility might help users better understand the nature of the content.

Last, educating users on the purpose and function of AIGC labels is essential. By improving users’ understanding of the labels, platforms can reduce misinterpretation and help them make more informed judgments about content credibility and accuracy, fostering a more resilient information ecosystem.

One potential limitation of this study is the lack of formal attention and manipulation checks. Although participants were informed about the AIGC labels, we did not explicitly assess their attention or directly confirm that the labels had no effect on the outcomes. Although we believe this did not significantly impact the results, it could be addressed in future studies to further validate the findings.

Additionally, this study was conducted as a web-based experiment, using experimental materials that did not replicate and simulate the real interface of a typical social media platform, which limited the external validity of the findings. In addition, the study focused on health-related content, both for-profit and not-for-profit, excluding other significant topics such as politics and technology. Consequently, it did not investigate potential differences across various content categories. Previous research indicates that cross-topic content variations can influence results [26]. Therefore, the strategies to use AIGC labels proposed by this study require further empirical validation through future experiments.

This study explored the impact of AIGC labels on perceived accuracy, message credibility, and sharing intention. Although it highlighted how AIGC labels affect these factors, the underlying psychological mechanisms remained ambiguous. The study inferred actual sharing behavior from self-reported willingness to share, following the approach used by Mosleh et al [62]. Previous research indicates that the ability to distinguish between accurate and inaccurate information minimally influences sharing intention [46]. However, prompting users to consider the accuracy of information before deciding to share can enhance both their willingness to share and their capacity to identify accurate information [46]. Further findings suggest that the perceived trustworthiness of sources, habitual belief tendencies, and expertise significantly affect perceived accuracy and message credibility, which in turn influences sharing intention [63]. In essence, there is a positive correlation between message credibility and sharing intention. Nonetheless, enhancing message credibility alone does not ensure that users will share the information; the interplay between personal relevance and message credibility exerts a stronger influence on sharing behavior [63]. In conclusion, the psychological mechanisms underlying perceived accuracy, message credibility, and sharing intention are complex and not yet fully understood, necessitating further investigation in future studies.

The experimental data demonstrate that AIGC labels serve as a practical intervention and novel approach, extending the domain of AI alignment beyond value and the moral level at the interaction interface. AIGC labels facilitate the ability of individuals to distinguish between AIGC and HGC, without significantly impacting their perceived accuracy, message credibility, or sharing intention. The AIGC labels and the content they mark together form a cohesive whole, and the effectiveness of labels is also influenced by the inherent nature of the content. It is particularly noteworthy that, when inaccurate information is not-for-profit, AIGC labels can potentially though insignificantly enhance perceived accuracy, message credibility, and sharing intention, which pose a certain disruption to the online ecosystem, necessitating further clarification of the application scope of AIGC labels in the future. Beyond explicit AIGC labels, integrating embedded evasion watermarks can help platforms discern between AIGC and HGC, thus addressing technical issues within its evolvement processes.