Hallucination is a system property
Models do not hallucinate in isolation. RAG systems hallucinate when retrieval fails silently and the model fills the gap.
Instrument retrieval first
Every response should carry the retrieved chunks. If you cannot show your work, you cannot debug your failures.
Refuse gracefully
"I do not have enough information to answer" is a valid response. Reward it in your eval set. Users trust systems that admit ignorance.
Rerank, always
Embedding search alone is not good enough for anything customer-facing. A cross-encoder rerank on the top 20 hits routinely doubles answer quality.
Log everything
Query, retrieved chunks, response, user feedback. Six months of logs is the foundation of every serious improvement cycle.