Tumor Immunotherapy–Related Information on Internet-Based Videos Commonly Used by the Chinese Population: Content Quality Analysis

Introduction

Background

Tumor immunotherapy is an innovative treatment today. After the implementation of China's new medical insurance rates in 2022, the monthly treatment cost of immunotherapy has entered the “thousand era,” which greatly improves the accessibility of drugs. However, tumor immunotherapy has obvious uncertainty and complexity [1]. Accurate transmission of immunotherapy information to the population is important to the survival and quality of life of patients with cancer [2]. The study found that patients were open to video education and found it helpful and worth watching [3].

The world’s population is increasingly referring to health-related internet-based information as it represents an easily accessible educational tool [4,5]. The Chinese population, overseas Chinese individuals, and people who master Chinese worldwide prefer web-based video applets or websites, such as videos on TikTok, Tencent, iQIYI, or BiliBili [6]. These sites, similar to YouTube, are popular for their rich content, convenient log-in methods, quick sharing, and 24-hour multiplatform seamless application experiences. Recently, the originality, interactivity, and sociable nature of TikTok and BiliBili have provided the younger generation a better user experience and sense of engagement while seeking health information [7]. The penetration and usage of TikTok and BiliBili are also on the rise among some older age groups [8]. However, the medical content available on the internet is controversial and has not been properly examined. Di Bello et al [9] reported that YouTube videos have contributed to the spread of misinformation by underestimating the role of information on immunotherapy for urological tumors in a multimodality approach and missing the findings of published clinical trials. Not only were audiences not availing of accurate therapy, but also they were opting for therapies that may be harmful, which could lead to other complications [10,11].

Objectives

This study aims to report an evaluation of the quality, reliability, and content of videos related to tumor immunotherapy on the internet among the Chinese population. Our findings could serve as a guide for health care providers and awareness campaigns.

Methods

Ethical Considerations

Ethics approval was not required as this descriptive study was conducted by examining publicly accessible videos on the internet. Also, no human participants or animals were included in this study. The study data are anonymous. This study was registered in the Chinese Clinical Trial Registry (ChiCTR2400081071).

Search Strategy and Data Collection

Using the keyword “肿瘤免疫治疗” (“tumor immunotherapy” in Chinese), we searched TikTok, Tencent, iQIYI, and BiliBili on March 5, 2022, which yielded 1820, 395, 400, and 1000 results for each search, respectively. The videos were sorted in accordance with the video's default “the most viewed” sorting parameter, and the first 50 videos per website were evaluated.

Inclusion and Exclusion Criteria

A total of 200 videos were considered from all the searches. Duplicate videos, paid videos, and videos not related to tumor immunotherapy were excluded. After the screening, we obtained 118 videos for further data extraction and analysis (Figure 1).

‎

Variables Extracted

Basic information obtained included the URL, video duration, likes, forwarding number, subscription, comments, and upload date. Profiles of the uploaders were recorded and classified under 5 categories: lectures, interviews, health-related animations, academic institutions or universities, and news agencies. The extracted data were recorded in Excel (Microsoft Corp).

Scoring System

The videos were evaluated independently by 2 authors (CN and Y-BF). The raters were blinded to each other's ratings (they could not consult each other). We reviewed the screened 118 of 3615 videos on TikTok, Tencent, iQIYI, and BiliBili on “tumor immunotherapy,” using the Patient Education Materials Assessment Tool (PEMAT)—a validated instrument for obtaining consumer health information (Multimedia Appendix 1) [12]. Moreover, we adopted 6 questions from Goobie et al [13] to evaluate the videos’ content. These 6 questions ask to what degree a video addresses the definition of a disease, its signs and symptoms, risk factors, evaluation, management, and outcomes. Each aspect was scored on a 3-item scale: 0=not addressed, 1=partially addressed, and 2=sufficiently addressed.

The DISCERN quality criteria [14] and the JAMA (Journal of the American Medical Association) Benchmark Criteria [15] were used for assessing the quality and reliability of the health information. The modified version of the original DISCERN questionnaire was used to assess the reliability and quality of the health information. It consists of five questions, each with a “yes/no” answer (yes=1 point; no=0 points; maximum score=5): (1) Is the video clear and complete? (2) Are reliable sources of information used? (3) Is the information presented balanced and unbiased? (4) Are additional sources of information listed for reference? (5) Are uncertain areas mentioned? The JAMA assessment is used to evaluate web-based videos and resources on the basis of 4 criteria: authorship, attribution, disclosure, and currency (1 point each).

• Authorship (1 point): the video should include authors, contributors, and contact information.
• Attribution (1 point): the references and sources should be listed properly.
• Disclosure (1 point): conflicts of interest, financing, sponsorship, advertising, support, and video ownership should be disclosed.
• Currency (1 point): the dates on which the videos were published and updated should be indicated.

After the scores are calculated, a score of 4 indicates that the source is of high quality.

We assessed the presence of misinformation using an analogous 5-point Likert scale [16,17]. Videos were independently coded by 2 authors with random coding checks to verify intercoder reliability. Each video was rated separately, and its mean score was calculated.

Statistical Analysis

The mean, median, IQR, and SD were used as descriptive statistics for continuous variables. To identify differences among the variables extracted, the Mann-Whitney U test was performed. The intraclass correlation coefficient was determined to ensure interrater reliability. A P value of less than .05 was considered significant. Statistical analysis was performed by using the GraphPad Prism 8 (GraphPad Software, Inc).

Results

The 118 videos about tumor immunotherapy mostly uploaded by channels dedicated to lectures, health-related animations, and interviews (Table 1; median length 5 minutes; 93, 79% uploaded in and after 2018). The median forwarding number and number of likes was 12 and 15, respectively. However, the median understandability and actionability of the videos were 71% and 71%, respectively. Overall, the quality of information was moderate to poor in 54% of videos (overall DISCERN scores of 1-3 out of 5) and 64% of videos (overall JAMA scores of 1-2 out of 4).

Only 12 videos contained misinformation (score >1 out of 5). Videos with a doctor (published by channels dedicated to lectures and interviews) not only were significantly less likely to contain misinformation but also had better quality and a greater forwarding number. Videos on Tencent and BiliBili had lesser misinformation than TikTok and iQIYI. Regarding DISCERN criteria and JAMA Benchmark Criteria, the quality of information on TikTok and iQIYI was higher than that on BiliBili and Tencent.

Moreover, our results show that more than half of the videos contain little or no content on the risk factors and management of tumor immunotherapy. Overall, over half of the videos had some or more information on the definition, symptoms, evaluation, and outcomes of tumor immunotherapy (Table 2). The overall scores for all internet videos are presented in Figure 2.

‎

Discussion

We screened 118 videos on “tumor immunotherapy” from TikTok, Tencent, iQIYI, and BiliBili commonly used by the Chinese population. Chinese websites or applets uploaded videos related to tumor immunotherapy for the first time in 2011, and the number of videos has significantly increased since 2018. The median duration of the videos was 5 minutes, which is acceptable to the public.

Numerous studies have evaluated videos on YouTube only and not on other networks [18-20]. Our study evaluated information about tumor immunotherapy on the most popular Chinese websites or applets, using validated instruments to evaluate the quality of information. Videos on BiliBili and TikTok had a significantly greater forwarding number and likes than those on iQIYI and Tencent; a possible reason is that there is no advertisement played before videos on BiliBili and TikTok.

Health care providers should recommend trustworthy sources of information to patients and should actively participate in social media for dissemination of evidence-based medicine. There is a great need for accurate tumor immunotherapy–related content that is also understandable and actionable. Suggestions for content creators include discussing both the benefits and risks of management alternatives, refraining from the use of medical terminology, and presenting the viewer with clear action items. Meanwhile, patients should be wary of internet-based videos. Misinformation, albeit well-intentioned, may be disseminated when a poorly informed patient advises others. Patients should talk to their physicians not only about immunotherapy but also their need for more information.

In conclusion, although the quality of tumor immunotherapy–related information on internet-based videos commonly used by Chinese people is moderate, it has less misinformation and better content. Caution must be exercised when using these videos as a source of tumor immunotherapy–related information.