Quality and Reliability of Transarterial Chemoembolization Videos on TikTok and Bilibili: Cross-Sectional Content Analysis Study

Introduction

Hepatocellular carcinoma (HCC) ranks as the fourth most common cause of cancer-related deaths globally and poses a significant challenge to health care systems worldwide [1,2]. According to the Barcelona Clinic Liver Cancer staging system, transarterial chemoembolization (TACE) is the first-line treatment for intermediate and advanced HCC, including unresectable, multinodular HCC without extrahepatic spread [3]. TACE has been widely accepted and recommended across various clinical settings in Asian countries, demonstrating strong acceptability and favorable treatment outcomes [4]. However, the complexity of the procedure, potential adverse effects, and need for repeated interventions often leave patients with limited understanding and high emotional distress [5-8]. Therefore, enhancing public awareness and patient education about TACE is critical.

With the increasing prevalence of internet technology, the importance and ubiquity of social media use have risen, with an estimated global user base of 4.9 billion [8]. In China, TikTok and Bilibili are the top two video-based social media platforms, boasting hundreds of millions of users. According to the latest data from QuestMobile, TikTok’s monthly active users surpassed the 1 billion mark as of September 2024 [9]. This figure highlights the widespread acceptance and influence of short videos, emphasizing the public’s strong preference for fast and intuitive methods of information access. Short videos can be easily created and shared on these platforms, enabling users to interact through likes, comments, and bookmarks, which fosters broader participation across social groups, regardless of gender or age. Health care professionals and organizations have also leveraged these platforms to disseminate health-related information, promote healthy habits, and encourage disease prevention. These platforms present an unparalleled opportunity to improve public health literacy, enhance self-efficacy, and promote treatment adherence [10].

A recent systematic review of health information systems underscores accuracy and completeness as foundational data quality elements critical for informed decision-making [11]. While videos related to TACE are available on TikTok and Bilibili, providing accessible information to patients, the quality of these videos remains uncertain. Prior studies suggest that misinformation is common on social media and that users often lack the digital health literacy needed to evaluate content critically [12,13]. Therefore, understanding the sources of information, the characteristics of social media channels, and the content and framing of health information is crucial for navigating the dynamic landscape of TACE-related content on these platforms.

Content analysis achieves an objective and replicable examination of communication materials by encoding the content into predefined categories. This method enables quantitative and qualitative interpretation of media content, making it particularly suitable for evaluating health information on web-based platforms. Moreover, the Global Quality Scores (GQS) and DISCREN systems are widely used to assess the content, quality, and reliability of short videos related to various diseases on social media platforms [12,14-16]. Therefore, this study aims to investigate the top 100 videos related to TACE on TikTok and Bilibili, evaluating their content, quality, and reliability. Additionally, the correlation between video quality and factors such as video source, content, length, likes, comments, shares, and saves was analyzed and discussed. This analysis can help foster the development of social media platforms, offering the general public more comprehensive and appropriate ways to access TACE-related information. Ultimately, our goal is to promote the correct treatment of HCC.

Methods

Ethical Considerations

Shandong Provincial Hospital affiliated to Shandong First Medical University did not conduct ethical approval. The information in this study does not involve clinical data, human specimens, or experimental animals. All data were obtained from publicly available videos on TikTok and Bilibili. We complied with informed consent guidelines and have adhered to Chinese law and regulations regarding the protection of personal information, privacy, and human rights.

Study Design

Search Strategy

Videos were obtained from two platforms, Bilibili [17] and TikTok (Chinese version [18] on November 18, 2024. The search was conducted using the keyword “TACE” (Figure 1). The search was conducted through a mobile phone to ensure consistency and to avoid algorithm adjustments specific to desktop browsers. To minimize bias from personalized recommendation algorithms, we registered and logged into new accounts on each platform. Furthermore, we have disabled personalized recommendations in the platform settings.

The following filters were applied: (1) language restriction: Chinese only; (2) upload date: no restriction; (3) sorting method: “Comprehensive ranking,” which prioritizes videos with video completion rate, high engagement (likes, comments, and shares), and recent uploads. The exclusion criteria were as follows: (1) duplicate videos, (2) advertisement content, and (3) irrelevant content.

‎

Sampling Procedure

We retrieved the top 100 videos based on overall rankings. This cutoff was chosen for three reasons: (1) it reflects common user experience, and focusing on the mainstream content recommended by the platform algorithm, and thus reflecting the current situation of health information dissemination in a more representative manner; (2) it balances depth and feasibility of manual coding across platforms; and (3) it is consistent with prior social media health content analyses, improving comparability [13,19,20]. There were no duplicate videos between the Bilibili and TikTok datasets.

Data Extraction

Two trained reviewers independently recorded. Data collected included the title, number of likes, number of comments, number of saves, number of shares, video duration, upload date, number of days since posting, uploader, video content, and video source. Disagreements were resolved by discussion or a third reviewer.

Video Quality Assessments

Two widely adopted standardized scales were used. The GQS was used to assess the quality of information in the videos. The GQS consists of five criteria, with scores ranging from 1 to 5. The higher the score, the better the quality [21]. The DISCERN questionnaire was used to evaluate their reliability. In addition, we used the full version of the DISCERN tool [22]. The questionnaire consists of 16 questions, scored on a 5-point scale (totaling 80 points), and is divided into three sections (the reliability of the video, the quality of treatment options, and the overall score). The total score and average value of 16 items are used to distinguish between high-quality and low-quality health information. The specific items can be found in Multimedia Appendix 1. Two independent reviewers with a background in medicine received training and independently conducted the review and evaluation. In the event of any discrepancies or disagreements between the two reviewers, a third expert was included in the discussion to determine the final score. The intraclass correlation coefficient values evaluated by the assessors were 0.901 and 0.887, indicating better reliability.

Classification of Videos

Three groups were classified according to the source of the video, and five groups according to the content. The video sources were categorized as follows: (1) professional individuals, (2) nonprofessional individuals, and (3) professional institutions. The video content was classified into the following categories: (1) treatment description, (2) image analysis, (3) treatment suggestion, (4) precautions after intervention, and (5) experience and feeling. Specific classification criteria are shown in Multimedia Appendix 2.

Statistical Analysis

Data analysis was performed using SPSS (IBM Corp) software. Normally distributed continuous data were expressed as ${\displaystyle \left(\overline{\chi }\pm \left[s\right]\right)}$, while skewed data were presented as median (IQR). Categorical data were presented as frequency and percentage. The normality of GQS and DISCERN scores was assessed using the Shapiro-Wilk test. For normally distributed continuous data, comparisons between groups were conducted using the independent samples 2-tailed t test or ANOVA. For nonnormally distributed continuous data, comparisons between groups were made using the Mann-Whitney U test or Kruskal-Wallis H test. Spearman correlation analysis was performed to assess the relationships between variables. Variables with P<.05 were selected for inclusion in the multivariate analysis. Multiple linear regression was applied to identify the factors influencing video quality. GraphPad Prism (version 9.0.2; GraphPad Software, LLC) software was used for plotting the graphs.

Results

Overview of the Video Screening Process

To obtain the top 100 eligible videos from Bilibili and TikTok, we removed 16 noncompliant videos from Bilibili and 25 from TikTok. In the end, a thorough evaluation was carried out on a total of 200 eligible videos, comprising content from both platforms. Each video was unique in content and uploader across the two platforms (Figure 1).

Landscape of the TACE Videos on TikTok and Bilibili

TikTok and Bilibili are public platforms in China, with a wide variety of uploaded content and types. In this study, we categorized the videos based on the uploader’s level of expertise. The results showed that videos from professionals dominated, accounting for 85.5%. The remaining sources were as follows: professional institutions (8.5%) and nonprofessional individuals (6%; Figure 2A). Overall, both TikTok and Bilibili have a large number of videos uploaded by professionals (Figure 2B). Notably, in terms of video content, “Treatment description” was the most prevalent category (36.5%), followed by “Treatment suggestions” (33%), while “Experience and feeling” had the lowest proportion (3.5%; (Figure 2C). A separate comparison of content types on TikTok and Bilibili showed similar results (Figure 2D).

‎

General Information and Index of Web-Based Videos

The number of views, likes, comments, shares, and other metrics on TikTok and Bilibili serve as indicators of the attention these videos receive and are suitable tools for assessing their influence. First, we compared these metrics between TikTok and Bilibili. In contrast, related videos on TikTok received more likes, comments, shares, and saves compared to those on Bilibili (P<.001). However, videos on Bilibili had longer durations and were uploaded earlier (P<.001; Table 1).

Next, we further compared the differences in the number of likes, comments, shares, saves, video duration, and upload time of web-based videos based on their source and content type (Tables 2 and 3). In Bilibili, although nonprofessional individuals uploaded a smaller number of videos, their videos had a high median number of saves and longest durations, suggesting a potential preference among viewers for longer, personal, or experience-based content. Notably, videos from professional institutions had significantly higher median save counts and share counts, suggesting these videos may be more valued as reference material. On TikTok, the number of comments from professionals is relatively high. To our surprise, “Experience and feeling” videos showed the highest quartile value in terms of likes, though differences did not reach statistical significance. In addition, when comparing across content types, videos classified as “Treatment description” received the most saves and shares in Bilibili and TikTok. This indicates that people have a high level of attention to information related to treatment.

Quality Analysis of Web-Based Videos

To objectively assess the quality of web-based videos, we conducted a thorough evaluation using the internationally recognized GQS and the DISCREN scoring system. For GQS scores, the majority of videos received a score of 2, making up 47% of the total, followed by scores of 3 (34.5%), 1 (11%), 4 (6.5%), and 5 (1%).

Regarding DISCREN scores, the range of 17‐32 represented the largest proportion at 53%, while scores of 33‐48 accounted for 44.5%, and 49‐64 for 2.5%. The quality scores of videos from both platforms were distributed primarily in the low to medium range, as shown in Figure 3 These results suggest that the quality of TACE-related videos is unsatisfactory and needs further improvement.

‎

Quality Comparison Across Platforms and Formats

Considering the unique characteristics of Bilibili and TikTok, we performed a comparative analysis of video quality between the two platforms. Regarding TikTok videos, the average GQS score was 2.31 (SD 0.81), and the average DISCERN score was 1.86 (SD 0.40), indicating that TikTok videos generally have moderate quality and low reliability. For Bilibili videos, the average GQS score was 2.48 (SD 0.80), and the average DISCERN score was 2.00 (SD 0.44), suggesting that Bilibili videos also have moderate quality and reliability. Furthermore, there was a significant difference in DISCERN scores between Bilibili and TikTok, with Bilibili videos being more reliable (P=.02). However, no significant difference was found in the GQS scores between the two platforms (P=.14).

To investigate whether the type of uploader affects video quality, we further compared the GQS and DISCERN scores of videos uploaded by different types of creators. The results showed no statistically significant differences in GQS scores or DISCERN ratings between different uploaders (P=.06; P=.48). However, there were differences in GQS scores and DISCERN scores between different content (P=.04; P=.04). Nevertheless, pairwise comparisons revealed no differences between the two groups (Table 4).

Correlation Analysis

Spearman correlation analysis was conducted to explore the relationships between various video variables. Positive correlations were found among the following variables: likes and comments (r=0.864; P<.001), likes and shares (r=0.631; P<.001), likes and saves (r=0.921; P<.001), saves and comments (r=0.734; P<.001), comments and shares (r=0.594; P<.001), and saves and shares (r=0.750; P<.001). Additionally, saves, shares, duration, and days since publication were positively correlated with both GQS and DISCERN. Furthermore, a strong correlation between GQS and DISCERN was observed, confirming their significant consistency and reinforcing the robustness of our scoring results (Table 5).

An Investigation Into Determinants of Video Quality

By incorporating meaningful factors from univariate and correlation analyses into multivariate analysis, the results showed that the number of saves and the number of days since upload were closely related to GQS. Specifically, videos with a higher number of saves and longer time since upload tended to have higher GQS quality. Additionally, the number of saves, video duration, and number of days since upload were closely related to DISCERN. Specifically, videos with more saves, longer duration, and earlier upload times demonstrated better reliability (Table 6).

Discussion

Overview

TACE treatment usually requires multiple surgeries for patients. Therefore, the patient’s knowledge, attitude, and treatment compliance are crucial for the success of the treatment. In this regard, educational videos on social media provide a convenient means of accessing relevant information, helping to overcome the traditional barriers to obtaining medical support and resources. Several studies, including those on attention deficit [23], laryngeal cancer [24], thyroid cancer [25], and gallstones [20], provide evidence that short videos facilitate easy access to and positive dissemination of disease-related information. However, the overall quality of these videos is unsatisfactory, raising concerns about misinformation on social media. Inaccurate or misleading information spreads rapidly on these platforms, potentially having a negative impact on health-related attitudes and behaviors. To evaluate the quality and identify the factors influencing TACE-related videos, we performed a comprehensive content analysis of these videos.

Principal Findings

This study is the first to comprehensively evaluate TACE-related videos across major video platforms. We reviewed the top 100 short videos on TACE from two popular Chinese platforms, TikTok and Bilibili. In this cross-sectional study, the general characteristics of these videos were analyzed, the DISCERN and GQS tools were used to assess their quality and reliability, and the factors influencing video quality were investigated. Our findings reveal that the overall quality of short videos on both platforms was lower than anticipated. Furthermore, we identified that factors such as the number of saves and the number of days since upload were associated with video quality. Additionally, the number of saves, video duration, and days since upload were linked to video reliability.

Video Characteristics

We found that TikTok is more popular than Bilibili. To be precise, the number of likes, comments, shares, and favorites is all high. However, videos on Bilibili tend to have longer durations and earlier upload times. Research indicated that shorter videos were generally more engaging and spread more quickly, which helped explain why TikTok videos received more likes despite having shorter durations [26]. Additionally, we analyzed the types of video creators and found that most TACE-related videos on both platforms were uploaded by professionals. On the other hand, we analyzed the different types of TACE-related content on both platforms. Treatment descriptions were the most prevalent. This underscores the strong demand for treatment-related knowledge among patients with HCC. We observed “experience and feeling” elicited higher levels of interaction, including shares and saves, compared to more technical formats like “image analysis.” This implies that emotionally engaging or explanatory content may be more effective in attracting user attention and encouraging dissemination. However, videos describing treatment experiences and personal feelings were the least prevalent. This highlights a gap in TACE-related content on TikTok and Bilibili. Therefore, we encourage health care professionals to expand and diversify video content to include aspects such as prevention, treatment, and precautions, ensuring that the public receives comprehensive and reliable medical information.

Quality of the Short Videos on TACE

The quality and reliability of TACE-related short videos on both platforms were found to be low, with overall performance primarily falling within the moderate to poor range. Research has shown that inaccurate and misleading videos on social media often receive more views from users compared to accurate content, which is similar to our research findings [27,28]. While watching the videos, we noticed that in order to draw attention, some nonprofessionals pretended to be professionals and promoted unproven and unreliable treatment methods. What surprised us was that these videos received a high number of likes and saves. Due to the complexity of the surgery and the psychological burden associated with liver cancer treatment, inaccurate or misleading video content regarding TACE may have serious consequences [29,30]. The patient may thus develop unrealistic expectations regarding such matters, underestimate potential complications, or improperly delay seeking clinical advice. Furthermore, incorrect information may erode people’s trust in health care professionals, especially when professional sources are perceived as less appealing than nonprofessional ones [30]. This highlights the urgent need for a balanced health communication strategy that combines accuracy, clarity, and engagement.

Improving health content quality requires a multilevel strategy. First, the platform can modify its algorithm to enhance the visibility of content provided by verified medical professionals and certification organizations, especially in searches related to disease terms. Second, social media companies can establish professional health content review teams, and these teams should possess basic medical knowledge and the ability to judge health literacy. Third, the platforms lack scientific oversight from experts in relevant fields and are unable to conduct peer reviews of the videos being published and shared. It is regrettable that implementing peer reviews may not be feasible. They can assist these platforms in developing effective tagging systems to flag potentially inaccurate or harmful health content. Finally, it is recommended that health care professionals provide evidence-based information when posting videos, including links or citations to credible sources.

We further analyzed the factors related to the quality and reliability of the video content. Interestingly, we found that video duration was closely related to content reliability. This result is likely related to platform limitations. Short video platforms have time constraints, making it difficult to provide the necessary background or depth to convey a complete picture or concept within such a short period. The lack of sufficient information somewhat reduces the quality of the content. Moreover, viewers are often unable to distinguish between high-quality and low-quality videos [13,20]. The audience for these short videos is not composed of professional health care workers, and they often find specialized knowledge unengaging, which leads to content being simpler and more superficial. Additionally, although there were no significant differences in video quality between the two platforms, there were notable differences in reliability, with Bilibili’s content being rated as more reliable. This result may also be related to video duration. Bilibili allows for more flexibility in video length, enabling clearer presentation of treatment-related information, while TikTok typically has shorter video durations. Therefore, given the complexity and specialization of medical content, the length of medical-related videos should be extended accordingly to ensure that sufficient information is provided. Furthermore, health care professionals should seek diverse forms of video content to increase its richness and engagement. Interestingly, we found that the longer the video has been published, the better the content quality and reliability. This serves as a warning: the quality of content tended to decrease. Given the widespread focus on entertainment and viral trends, the importance of video content should not be overlooked.

Strengths and Limitations

This study has several strengths. First, it is the first study to analyze the quality of short videos related to TACE on social media platforms, offering guidance for the public in accessing health information and providing valuable insights for content creators and platforms. Second, our research focused on China’s two largest short video platforms, overcoming the limitations of studying a single platform, which enhances the reliability of our results. Finally, we used widely recognized tools, the GQS and DISCERN, to assess the quality and reliability of the videos, ensuring a robust evaluation of our findings.

However, this study has some limitations. First, the videos included in our analysis were limited to those from China, and global platforms like YouTube, which dominate the web-based video space, were excluded due to access restrictions. Second, we focused on the top 100 videos from each platform. This algorithm does not equate to relevance or quality. It may systematically overlook content with lower visibility but potentially higher accuracy. Future research should consider broader sampling strategies to capture less visible but potentially high-quality content. Third, due to the limited sample size within certain categories, we did not perform subgroup or interaction analyses, which might have missed some subtle differences. We identified it as a priority for future, larger-scale analyses.

Conclusions

The overall quality and reliability of the top 100 videos related to TACE videos on TikTok and Bilibili were relatively low. Higher numbers of saves and longer time since upload were positively associated with better video quality and reliability. These findings highlight the need for caution when using social media as a health information source. The quality of the video should be further improved. Platforms should adjust their algorithms to prioritize content from verified medical professionals. Establishing review teams with basic medical literacy can help flag misleading information. Health care professionals are also encouraged to share evidence-based information and cite credible sources when publishing videos.