PQR: A framework for generating diverse, realistic user queries that can induce QA Agent failures

The PQR framework lets a query rewriting module and a prompt refinement module interact iteratively to generate queries that closely match real user intent and can also trigger QA Agent failures on specific objectives such as helpfulness and safety. In unhelpful-answer detection for an e-commerce QA Agent, PQR finds 23%-78% more unhelpful answers than previous methods, and the generated queries are also more diverse and more realistic.