# Response to Claude's RAG retrieval review

All eight findings in `review-rag-retrieval-2026-08-03.md` were accepted and
addressed. The previous `10/10` headline is withdrawn.

## Fixes by finding

1. Removed score-tie abstention. Added a separate human-clinical scope guard;
   the veterinary case now returns `out_of_scope_veterinary`, while the adult
   wording remains answerable.
2. Deleted `QUANTITATIVE_TERMS` and `_structured_boost`. Ranking now uses a
   corpus-derived BM25 score plus actual-order character n-gram overlap.
3. Evaluation reports Recall@1, Recall@3, and Recall@5 separately.
4. Regenerated both `out/all` and `out/100` with the new provenance schema.
   Prose is no longer restricted to drugs that have a reconstructed table.
   `out/all` now has 15,727 documents across all 684 drug IDs.
5. `run_eval` no longer supplies `drug_id` to retrieval. A catalog resolver
   resolves exact names and aliases and handles the `famciclovia` typo. Queries
   with multiple distinct drug entities abstain as ambiguous rather than
   silently choosing one.
6. Manual cases are now `manual_routing_diagnostic`; only expert cases appear
   under `expert_release_gate`. There are currently zero expert cases.
7. `run_eval` now uses the shipped `EvidencePolicy()` defaults.
8. Character n-grams are generated from normalized text in original order,
   not sorted unique terms.

## Measured result after fixes

The manual diagnostic was run against `out/all`, not the six-drug hard-10
artifact:

- documents: 15,727
- unique drug IDs: 684
- manual cases: 10 (9 positive, 1 negative)
- Recall@1: 0.8889
- Recall@3: 0.8889
- Recall@5: 0.8889
- negative abstain rate: 1.0
- expert cases: 0; all expert metrics remain `null`

The one positive miss is intentionally safe: the Oresol composition question
mentions both `oresol` and the separate monograph entity `natri clorid`. The
resolver returns `drug_resolution_ambiguous` instead of selecting the wrong
drug. A later multi-entity planner must resolve subject versus ingredient.

The source-derived full-scope TF-IDF run remains diagnostic only:

- 2,436 generated queries
- hybrid Recall@1: 0.9413
- hybrid Recall@5: 0.9955
- MRR: 0.9667

## Verification run

- `python -m pytest -q` from `ingestion`: 204 passed.
- `python -m pytest tests -q` from `apps/ai-service`: 10 passed.
- `python -m ruff check rag tests`: passed.
- `load_documents(out/100/...)`: 15,593 documents loaded.
- `load_parents(out/100/...)`: 126 parents loaded.

No cloud call was made and no AWS cost was incurred.