【公开学术报告-第155期】Toward Trustworthy Health Information

Toward Trustworthy Health Information: Automated Quality Assessment of Health News Using Fine-Tuned LLMs

【迈向可信健康信息：基于微调大语言模型的健康新闻质量自动化评估】

Speaker: Lan Sang (Ph.D. Candidate at University of Colorado Boulder)

Date & Time:  Tue. 30, December 2025, from 10:00 to 11:30 AM (Beijing Time)

Location: Tongji Building A2101

ABSTRACT

The quality of online health information significantly influences public trust and healthcare decisions. Despite initiatives such as expert-driven reviews, large-scale manual assessments remain costly and unsustainable. This study introduces an automated, scalable method for assessing the quality of health information using fine-tuned large language models (LLMs). Leveraging an expert-annotated dataset from HealthNewsReview.org, I benchmarked multiple state-of-the-art LLMs, including open-weight models from Meta’s Llama and Alibaba’s Qwen series. Our best-performing fine-tuned model demonstrated strong performance in accurately classifying information quality based on complex, expert-defined criteria. Building on this technical foundation, I propose a behavioral study to examine whether AI-generated labels and justifications improve user decision-making and introduce HealthCheck, a browser-based tool that integrates these capabilities into everyday browsing. Collectively, these components establish a framework that not only automates assessments traditionally conducted by experts but also delivers them through an accessible tool, supporting informed health decision-making and advancing IS research on information quality and intelligent systems.

Keywords: Health Information Quality, Large Language Models (LLMs), Information Quality Assessment, IS Healthcare