Abstract
Objective
To inform patients and healthcare providers by investigating the content, reliability and quality of YouTube videos accessed using the keyword leukemia.
Methods
The research was conducted on YouTube using the keyword “leukemia” and the top 200 videos were listed, sorted by relevance. Video features (duration, views, comments, likes-dislikes and the time since the upload date) and source of upload were recorded. The quality, reliability and accuracy of the information were evaluated by two independent specialist physicians using the Journal of American Medical Association (JAMA) score, modified DISCERN tool and the global quality score (GQS). A total number of 153 videos were analyzed.
Results
The majority of the videos (38.6%) were uploaded by physician/universities/professional organisations. General information was the most frequent video content (33.3%). According to the modified DISCERN score, 50.9% of the videos included in the study were low, 17.6% were medium and 31.3% were high quality. There was a statistically significant difference between the groups in terms of the average duration of the low, medium and high-quality videos. JAMA, GQS and the modified DISCERN scores were higher in videos uploaded by physician/universities/professional organisations than in videos uploaded by other sources.
Conclusion
The majority of leukemia-related videos on YouTube are of low and medium quality and run the risk of misinformation. Health professionals should be careful about this issue and warn their patients who choose the way to get information from YouTube.
Introduction
YouTube, the most frequently visited website after Google, reaches a billion hours of watch time per day, with 500 hours of video uploaded every minute(1). Videos on various medical topics are also available on YouTube. The person suffering from a disease turns to YouTube to obtain information about the disease and to research existing and new treatment methods. Patients may be exposed to commercial content that may lead to low-quality, biased and/or dangerous consequences(2). On the other hand, accurate and reliable information can reduce the patient’s anxiety level and contribute positively to the treatment process.
Leukemia is a hematological disorder characterized by malignant proliferation of bone marrow stem cells. Depending on the duration of symptoms and the characteristics of the malignant clone, acute or chronic lymphoid or myeloid leukemia may develop. Uncontrolled proliferation of leukemic blasts affects normal hematopoiesis, causing anemia, leukopenia/leukocytosis and thrombocytopenia. For this reason, patients are admitted to the hospital due to fatigue, bleeding, and infection. The diagnosis is typically made through blood tests or bone marrow biopsy(3). The treatment process varies depending on the type of leukemia, the characteristics of the malignant clone, and patient-related factors. While chemotherapy and cellular therapies are generally applied in acute leukemias, patients with chronic lymphocytic leukemia can be monitored without treatment(3).
To our knowledge, our study is the first to investigate the content, reliability, and quality of leukemia videos on YouTube. The aim of this study is to inform patients and healthcare providers through an investigation of the content, reliability, and quality of YouTube videos accessed using the keyword leukemia.
Materials and Methods
YouTube Search
The study was designed as a cross-sectional study. The research was conducted on YouTube using the keyword “leukemia” on 15 November 2024 without a filter. The top 200 videos were listed and sorted by relevance (default option on YouTube), assuming that users typically reviewed the first five pages in their search. All videos were aimed at adult patients. Since the search results on YouTube can change from day to day, these videos have been saved to a playlist for ease of review. Once the videos were recorded in the playlist, they were reviewed and analyzed by two independent specialist physicians with over a decade of experience treating leukemia patients in their daily practice. Both physicians were blinded to each other’s evaluation. The reliability of the assessment among the observers was determined by calculating the kappa coefficient. Non-English videos, videos without sound, dubbed videos, and non-relevant videos were excluded from the study, and a total of 153 videos were included. Since the study did not involve animal or human participants, and because similar studies in the literature also do not require it, ethics committee approval was not required as the videos were accessible to everyone(4).
Video Features
The duration, number of views, number of comments, number of likes-dislikes, and the time since the upload date, and source of each video included in the study were recorded. To analyze the popularity of videos, the like rate [like/(like + dislike) × 100] and the video power index [like rate × view rate (view per day/100)] were used. The videos were divided into five groups according to the upload source: physicians/universities/professional organisations, independent health-related websites, patient experience, commercial (profit organisations/advertisements) and TV shows/talk shows. According to their content, the videos were grouped as general information, symptoms and diagnosis, treatment, lifestyle and others. Other videos included patient experiences.
Video Quality and Reliability Analysis
The quality and reliability analysis of the included videos was performed using the modified DISCERN; global quality score (GQS); and the Journal of the American Medical Association (JAMA) scoring questionnaires. According to these forms, the videos included in the study were divided into three groups of low, moderate, and high quality. Quality videos are videos with useful information for patients. When encountering medium-quality videos, patients should consult additional resources for appropriate information. Low-quality videos should not be used by patients to gain information.
The original DISCERN inquiry form was first created by Charnock et al.(5) to assess the quality of health information provided to patients. The original DISCERN inquiry form is a scoring system consisting of 16 questions, of which each scores between 1 and 5 points. In our study, a modified DISCERN form consisting of 5 questions was used, which provides more reliable evaluation of visual health information. There are 5 yes/no questions in this form, with each yes answer scored as 1 point, making a maximum of 5 points. In this scoring system, videos with a score below 3 points are considered low quality, videos with 3 points are considered medium quality, and videos with more than 3 points are considered high quality.
GQS is a 5-question scale developed by Bernard et al.(6) that measures the quality, accessibility, and educational value of videos. On this scale, videos are grouped as low quality at 1-2 points, medium quality at 3 points, and high quality at 4-5 points.
The JAMA scoring system is a tool intended to assess the quality of health-related websites. It consists of four criteria (authorship, citation, validity, description) with 1 possible point each, for a total possible score of 4 points. A score of four indicates the highest quality.
Statistical Analysis
The relationship between two categorical variables was investigated using the chi-square test. The Kruskal-Wallis test was used if the differences among the groups did not meet the parametric test assumption, and a one-way ANOVA test was used for pairwise analysis. The kappa coefficient was used to test the inter-rater agreement. The Spearman test was performed for correlation analysis. The data analysis was performed using the Statistical Package for the Social Sciences (SPSS Inc, Chicago, IL, USA) version 22.0, and a p-value of <0.05 was considered significant.
Results
Of the 200 videos listed, 28 were excluded from the study because they were off-topic, 9 were dubbed, 7 were in non-English languages, and 3 had no sound. A total of 153 videos were analyzed. The characteristic features of the videos have been summarized in Table 1. The majority of the videos (38.6%) were uploaded by physicians/universities/professional organisations. General information was the most frequent video content (33.3%). The mean JAMA, modified DISCERN, and GQS scores were 1.9±1.2, 2.9±1.3, and 2.9±1.3, respectively. When the cut-off value was selected as ≥3 according to the JAMA score, 66 videos (43.1%) fulfilled the quality criteria. The Cohen kappa score was 0.849, 0.946, and 0.898 for the JAMA, modified DISCERN, and GQS score, respectively. According to the modified DISCERN score, 50.9% of the videos included in the study were low, 17.6% were medium, and 31.3% were high quality. The mean duration of all videos was 13.2±20.2 minutes, and the mean duration of high-quality videos was 17.6±25.1 minutes. There was a statistically significant difference between the groups in terms of the average duration of the low, medium, and high-quality videos (p<0.001) (Table 2). There was no statistically significant difference between the groups in terms of other video features (Table 2). JAMA, GQS, and the modified DISCERN scores were higher in videos uploaded by physicians, universities, or professional organisations than in videos uploaded by other sources. This difference was statistically significant (p<0.001) (Table 3). 17.9% of the low-quality videos were uploaded by physicians/universities/professional organisations and the majority (82%) were uploaded by other sources. Correlation analyses of quality scales and video features have been displayed in Table 4. The GQS score and modified DISCERN were positively correlated with the video duration (rho = 0.675, p<0.001 and rho = 0.328, p<0.001, respectively). There was no significant correlation between the JAMA score and the video duration (rho = 0.67, p=0.407). The modified DISCERN, GQS, and JAMA scores correlated significantly with each other. A positive significant correlation was observed between the video likes and video duration, number of views and number of comments
(Table 4).
Discussion
Unlike ancient times, the ways of accessing information differ as a result of the developing technology in the present day. As a result of the widespread use of the Internet, access to health-related information has become easier. Digital platforms such as YouTube, Twitter, Facebook, where information spreads quickly and uncontrollably, are frequently used by patients and physicians. In a survey, it was revealed that 41% of the patients were affected by social media in the selection of doctors and institutions(7). A study of 400 patients in Saudi Arabia, reported that more than 40% of patients dropped out of their treatment due to social media referrals(8).
Unfortunately, the spread of misinformation is an important social problem. Therefore, it is vital to examine the content, quality, and reliability of videos on YouTube. It may not always be possible for non-medical professionals to synthesize medical information from their own perspective or conduct an internet search with the right keywords to test the accuracy of the medical information they obtain from the internet.
Loeb et al.(9) have evaluated the quality of prostate cancer videos on YouTube. In their study examining 150 videos, they reported the overall information quality as medium. Videos published by government agencies and medical institutions were found to contain higher quality content, but these videos had lower monthly views. Nonetheless, 77% of these videos contained potentially false information and 27% were found to be commercially biased.
In the literature, the sources of videos on YouTube have been classified in various ways. In the study by Wong et al.(10), where they evaluated botox videos on YouTube, they found that the majority of videos (43%) had been uploaded by healthcare professionals. Another study evaluating fibromyalgia videos reported that the most common source of was doctors with 28%, while TV programs had the lowest rate at 6%(11). Similar to this study, the most common source of loading in our study was doctors with 38.6%, and TV programs had the lowest rate, which was 2.6%.
When videos are considered in terms of content, in the study of Gokcen and Gumussuyu(12) where they evaluated disc herniation videos, they found that non-surgical treatments (40%) were the most common and this was followed by general information (30%). While 55% of publications on fibromyalgia were related to the emergence and causes of the disease, 23% were reported to include treatment methods(11). On the other hand, in the study by Loeb et al.(13), which evaluated videos on bladder cancer, the videos which were related to treatment were watched by 41%, and those related to symptoms/diagnoses were watched by 24%. In the videos about robotic surgery and pyeloplasty in the pediatric age group, the surgical methods applied were covered at a rate of 70%. Indications and general information were the second most frequently covered topic at 14%(14). In our study, the definition of leukemia ranked first with 33.3%, followed by symptoms and diagnosis with 23.5%. We think that the nature of the medical branch, the variety of subjects, and the original evaluation played a role in the emergence of content and source differences between the studies.
The popularity of YouTube in accessing information has increased due to the importance of visual and auditory factors. When people have an ailment, many turn to YouTube to learn more about their illness. Therefore, it is important to evaluate the content of public health and disease-related posts on social media, especially on YouTube. The fact that individuals and organizations that publish videos resort to dramatic elements to increase the number of views can lead to misinforming patients and damage to the patient-doctor relationship. Furthermore, studies in the literature have shown that quality information on the internet reduces patients’ anxiety about their illness(15). In accordance with the literature, GQS, modified DISCERN and the JAMA scores were used to evaluate the quality of YouTube videos in our study. 50.9% of the videos included in the study were low, 17.6% were medium, and 31.3% were high quality; the majority of the high-quality videos (70.3%) had been uploaded by the physicians. Loeb et al.(13) reported that 67% of bladder cancer videos on YouTube were of poor and medium quality. In the YouTube studies of Adorisio et al.(14), it was found that the DISCERN, JAMA, and the GQS scores of the visual broadcasts uploaded by physicians and academic organizations were higher than those of other source videos. In the study of Selvi et al.(16), where they evaluated testicular examination videos, they found that 23.1% of the videos had been uploaded by physicians, universities, or professional organisations and reported that the majority of these videos (42.3%) contained useful information. Unlike these studies, Culha et al.(17), where they evaluated pelvic floor muscle exercise training videos, found no statistical difference between the publishing source and the quality of videos.
In our study, there was also a statistically significant difference between the groups in terms of the average duration of low, medium, and high-quality videos. Similar to our study, Duran and Kizilkan(15) in their evaluation of testicular cancer videos, reported that high-quality videos had longer durations. In another study of nocturnal enuresis videos, it was observed that the duration of high-quality videos was long(4).
Study Limitations
We are aware of the limitations of our study. The first limitation of our study is that only a certain keyword was searched, and a limited number of videos were evaluated. Different results can be obtained with a larger number of videos and different keywords. Similar studies considering the subgroups of leukemia separately can be carried out. In addition, YouTube is a dynamic platform, and hundreds of videos can be added and deleted every day. Therefore, YouTube searches may vary according to geographic location and time. Another limitation is that only English videos were examined in our study. Different results can be obtained with languages other than English. Additionally, the lack of information about the socioeconomic and educational levels of the audience is considered a limitation. In this and similar studies, the relatively subjective evaluation of visual broadcasts brings with it the risk of observer bias. In our study, we addressed observer bias by analyzing the data from two independent reviewers, using tools that have been validated for reliability.
Conclusion
Although YouTube is a social media platform that conveys information through visual content, the majority of leukemia-related videos on YouTube are of low and medium quality and run the risk of misinformation. Health professionals should be careful about this issue, and warn their patients who choose to get information from YouTube. They should also recommend reliable additional sources of information for patients and actively engage in social media for the spread of evidence-based medicine.
