HaluEval-Wild: evaluating hallucinations of language models in real-world scenarios

Researchers introduce the HaluEval-Wild benchmark, which collects questions from ShareGPT, a dataset of real user interactions with LLMs, to evaluate the hallucination rates of multiple large language models in real-world scenarios; the authors call it the first benchmark designed specifically for real-world hallucination evaluation. These questions were filtered adversarially with Alpaca to retain challenging samples and categorized into five types to support fine-grained analysis of models' hallucination types; reference answers were synthesized by GPT-4 combined with retrieval-augmented generation (RAG). The paper states that existing hallucination benchmarks for traditional NLP tasks such as knowledge-intensive question answering and summarization struggle to capture dynamic real-world interactions. The benchmark is released at https://github.com/HaluEval-Wild/HaluEval-Wild。