Can AI Distinguish Truth from Fiction?

Can AI Distinguish Truth from Fiction?

Evaluating LLMs' Ability to Assess News Source Credibility

This research evaluates how effectively leading Large Language Models (LLMs) can identify credible information sources—a critical skill as AI increasingly curates online content.

Key Findings:

  • Nine LLMs from OpenAI, Google, and Meta were tested on their ability to distinguish high-credibility from low-credibility sources
  • Results reveal LLMs' capabilities and limitations in identifying accurate vs. misleading information
  • Political bias was measured in how these models evaluate news sources

Security Implications: As LLMs become integrated into search engines and information discovery tools, their ability to accurately assess source credibility directly impacts information security and the spread of misinformation. Biased or inaccurate source evaluation could amplify harmful content at scale.

Accuracy and Political Bias of News Source Credibility Ratings by Large Language Models

2 | 141