From Distorted Facts to Uncontrolled Agents: Can Large Language Models Reliably Detect Multidimensional Hallucinations?

The researchers introduce HaluCheck, a large-scale bilingual benchmark containing more than 42,000 samples, covering factual hallucinations in text and faithfulness hallucinations in agent trajectories, and providing multi-level fine-grained annotations for detection, localization, and correction. Experiments on 14 large language models show that the models have basic sample-level detection capability, but perform poorly on fine-grained localization, correction, and agent hallucination detection; thinking mode, web search, and supervised fine-tuning each bring partial improvements, and none fully solves the problem. The benchmark provides a foundation for evaluating and diagnosing the reliability of large language models across text and agent scenarios.