57 lines
2.6 KiB
Markdown
57 lines
2.6 KiB
Markdown
# Codex response to retrieval review round 2
|
|
|
|
All round-2 findings were accepted. This response distinguishes policy
|
|
enforcement from natural-language classification; the latter is not claimed
|
|
to exist yet.
|
|
|
|
## Changes
|
|
|
|
- Removed `HumanClinicalScopeGuard` and its animal keyword list. Routing now
|
|
requires structured `SubjectScope` and `QueryIntent` inputs. Non-human and
|
|
recommendation requests are refused; unknown values fail closed. The API or
|
|
classifier that supplies these fields remains future work.
|
|
- Removed Recall@5 because shipped retrieval returns at most three evidence
|
|
items. Reports contain Recall@1 and Recall@3 only.
|
|
- Added `resolved_drug_id` and `drug_resolution_status` to results. Evaluation
|
|
now reports drug-resolution accuracy and status counts.
|
|
- Replaced the live-path `assert` with an explicit invalid-state abstention.
|
|
- Added a deterministic entity builder and generated
|
|
`ingestion/data/verified/drug_entities.json`: 684 entities, all 344 explicit
|
|
`X - xem Y` relations mapped, 492 trade-name sections consumed, zero
|
|
unresolved/orphan index aliases, and 10,164 source-derived alias strings.
|
|
- Parenthesised headings are split into valid aliases. `paracetamol`,
|
|
`acetaminophen`, and `aspirin` now reach their canonical monographs.
|
|
- Added regression coverage for every canonical substring collision currently
|
|
measured in the 684-entity artifact (13 pairs).
|
|
- Added an evidence-based disambiguation loop for subject-versus-component
|
|
queries. It selects a subject only when its evidence contains all other
|
|
mentioned entities and the reverse relation is not also supported. The ORS
|
|
composition case resolves; symmetric multi-drug cases remain ambiguous.
|
|
|
|
## Measured diagnostic
|
|
|
|
Against `scratch/rag-table-pilot/out/all` (whole-corpus prose plus the complete
|
|
identified structured-block inventory; not every source page contains a
|
|
structured block):
|
|
|
|
- 10 manual cases: 9 positive, 1 policy-enforcement negative
|
|
- Recall@1: 1.0
|
|
- Recall@3: 1.0
|
|
- drug-resolution accuracy: 1.0 (9/9 in-scope human cases)
|
|
- negative policy enforcement: 1.0 (1/1)
|
|
- expert release gate: 0 cases, metrics `null`
|
|
|
|
These ten cases are a diagnostic, not clinical-production evidence.
|
|
|
|
## Commands reproduced
|
|
|
|
```text
|
|
python -m pytest -q # ingestion: 206 passed
|
|
python -m pytest tests -q # ai-service: 14 passed
|
|
python -m ruff check rag tests # passed
|
|
python -m ingestion.entities.catalog ... # 684 / 344 / 492 / 0 unresolved
|
|
python -m rag.run_eval ... # R@1 1.0, R@3 1.0, resolver 1.0
|
|
```
|
|
|
|
No cloud call was made and no AWS cost was incurred.
|