Refusing instead of guessing: the grounding gate in my multimodal chatbot

The worst failure of a document chatbot is a fluent answer that the documents do not support. In my multimodal chatbot, retrieval does not go straight to generation. Several cheap checks decide whether there is enough evidence first.

A three-zone dense gate

• pass: the best dense similarity is high enough to trust retrieval (RAG_DENSE_PASS_SIMILARITY, 0.62 in the example config).
• soft: borderline (above RAG_MIN_DENSE_SIMILARITY, 0.45). Retrieval continues and the grader decides.
• weak: retrieval still runs, but insufficient context can end in a refusal without a web search.

Grader reason codes, not a yes/no

The grader returns sufficient, partial, insufficient_context, needs_live_data or irrelevant. That makes the next step explicit: generate with citations, go to the web for live data, or refuse gracefully.

Don't trust an empty web search either

Tavily results are treated as empty when they are too weak to trust, for example very short snippets, or an unknown date combined with a generic title. In that case the assistant refuses instead of inventing an answer.

Don't cache bad answers

The semantic cache skips answers that are short, look like a fallback, or appear truncated, so a refusal or a cut-off response is never replayed to the next user.

Trade-off

Thresholds trade hallucinations for refusals. The right values depend on the corpus, so the next step is a labelled set of answerable and unanswerable Turkish questions to tune them against. Benchmark coming soon.