Files
duocthu/coordination/response-rag-retrieval-round2-2026-08-03.md
T

2.6 KiB

Codex response to retrieval review round 2

All round-2 findings were accepted. This response distinguishes policy enforcement from natural-language classification; the latter is not claimed to exist yet.

Changes

  • Removed HumanClinicalScopeGuard and its animal keyword list. Routing now requires structured SubjectScope and QueryIntent inputs. Non-human and recommendation requests are refused; unknown values fail closed. The API or classifier that supplies these fields remains future work.
  • Removed Recall@5 because shipped retrieval returns at most three evidence items. Reports contain Recall@1 and Recall@3 only.
  • Added resolved_drug_id and drug_resolution_status to results. Evaluation now reports drug-resolution accuracy and status counts.
  • Replaced the live-path assert with an explicit invalid-state abstention.
  • Added a deterministic entity builder and generated ingestion/data/verified/drug_entities.json: 684 entities, all 344 explicit X - xem Y relations mapped, 492 trade-name sections consumed, zero unresolved/orphan index aliases, and 10,164 source-derived alias strings.
  • Parenthesised headings are split into valid aliases. paracetamol, acetaminophen, and aspirin now reach their canonical monographs.
  • Added regression coverage for every canonical substring collision currently measured in the 684-entity artifact (13 pairs).
  • Added an evidence-based disambiguation loop for subject-versus-component queries. It selects a subject only when its evidence contains all other mentioned entities and the reverse relation is not also supported. The ORS composition case resolves; symmetric multi-drug cases remain ambiguous.

Measured diagnostic

Against scratch/rag-table-pilot/out/all (whole-corpus prose plus the complete identified structured-block inventory; not every source page contains a structured block):

  • 10 manual cases: 9 positive, 1 policy-enforcement negative
  • Recall@1: 1.0
  • Recall@3: 1.0
  • drug-resolution accuracy: 1.0 (9/9 in-scope human cases)
  • negative policy enforcement: 1.0 (1/1)
  • expert release gate: 0 cases, metrics null

These ten cases are a diagnostic, not clinical-production evidence.

Commands reproduced

python -m pytest -q                           # ingestion: 206 passed
python -m pytest tests -q                     # ai-service: 14 passed
python -m ruff check rag tests                # passed
python -m ingestion.entities.catalog ...      # 684 / 344 / 492 / 0 unresolved
python -m rag.run_eval ...                    # R@1 1.0, R@3 1.0, resolver 1.0

No cloud call was made and no AWS cost was incurred.