The Potential and Pitfalls of AI and Cancer Information

When was the last time you used Google to diagnose yourself? While most of us trust our doctors more than the internet, it often takes months to get a primary care appointment, and even longer to see a specialist. To get quicker answers for our health concerns, many of us search online before consulting a physician.

It’s a running joke that online resources can lead anxious Googlers to assume that their headaches, fatigue, or other minor symptoms are caused by cancer. The internet can overdiagnose individuals and allow for the spread of erroneous cancer information. Access to inaccurate cancer information can be harmful, discouraging individuals from seeking recommended screenings and encouraging the use of alternative and ineffective cancer treatments.

With artificial intelligence (AI) being integrated into many online platforms, a simple Google search will now produce an AI-generated response. If trained and implemented well, AI chatbots may be able to share quality health information from trusted sources, which could help people know when and how to get screened for cancer and share treatment options.

What do AI chatbots need to do to effectively communicate about cancer? First, they need to share quality information. If developers train their AI chatbots using trusted sources, like the American Cancer Society and the National Cancer Institute, they are likely to share helpful information with patients. If developers are not selective about how they train their chatbots, they may also integrate information from less trustworthy sources and share bogus information as if it were fact.

Second, AI chatbots must answer questions in ways that are easy to read and understand. Most trusted health websites gear their content towards a sixth-grade reading level. Experts in health communication recommend this tactic to circumnavigate patients’ potential lack of literacy. If an AI chatbot cannot communicate at a sixth-grade reading level, it may not be able to answer cancer-related questions in ways that the general public can understand.

To be effective health communicators, all AI chatbots, free and paywalled, will need to share information only from quality sources and to simplify their language.

Health communication experts have been studying the effectiveness of AI as a cancer information source. They are concerned that answer quality may depend on if the chatbots are free to access or require payment.

 David Musheyev and colleagues evaluated the quality of ChatGPT’s answers to cancer-related questions. The researchers first asked both the free and paywalled versions of ChatGPT to explain the symptoms or treatment methods of multiple types of cancer, including colorectal, breast, and lung cancers. They then asked both versions of ChatGPT to explain the symptoms of the cancer at a sixth-grade reading level. Using previously developed readability and quality scales, the researchers compared answers between the free and paywalled versions before and after prompting the chatbot.

Overall, the paid chatbot provided responses that were easier to read and understand. Low-income individuals without the means to upgrade their chatbots may struggle to understand the AI-generated responses, the investigators suggest, furthering pre-existing gaps in knowledge. While asking the chatbot to answer at a sixth-grade reading level did improve readability in the free version without changing the quality of the answers, the authors concluded that the contents were still not well suited for a sixth-grade reading level.

The quality of information on treatment, including risks and benefits of treatment types, was relatively low for both free and paid versions. Further, neither version of ChatGPT cited its sources, which would allow deeper investigation into a topic by readers.

AI chatbots have the potential to effectively spread knowledge of cancer symptoms, treatments, and prevention methods to individuals who might not be able to visit a health professional or who might want to prepare for an upcoming visit. To be effective health communicators, all AI chatbots, free and paywalled, will need to share information only from quality sources and to simplify their language.