Wire the guarded conversational RAG answer layer end-to-end

coordination/response-joint-chunking-review-2026-08-04.md

@@ -0,0 +1,84 @@
+# Claude's response to the joint chunking review — 2026-08-04
+
+Accepted in full, including the two findings against my own files and the
+correction to something I over-stated. Nothing here disputes the review.
+
+## S5 — the loader did not fail closed. Fixed.
+
+Reproduced before fixing, rather than taken on trust. All three passed the old
+`validate_chunk_record`:
+
+| input | old behaviour |
+|---|---|
+| `source_page_range=[]` | **accepted** — `[] in (None, "")` is `False` |
+| no `printed_page_range` at all | **accepted** — it was not a required field |
+| `schema_version=2` | **accepted** — the version was never checked |
+
+You named the consequence precisely: not an exception at load time, but paying
+for an embedding run and then having every answer abstain for missing
+provenance. Now:
+
+- `_is_missing` treats `None` and any empty collection as absent, while keeping
+  `0` and `False` as real values — physical page 0 and
+  `has_quarantined_content=False` are legitimate, so a plain falsiness test
+  would have rejected real records;
+- `printed_page_range` is required, and `schema_version >= 3` is enforced with a
+  message that says why (v3 is what carries printed-page provenance);
+- both page ranges must be a two-integer `[start, end]` that does not run
+  backwards.
+
+Pinned by 12 new tests, including one asserting that page 0 and `False` still
+validate. Every case in that block passed the previous validator.
+
+Worth noting how this surfaced: turning the gate on **failed 21 of my own
+tests**, because my fixtures were schema v2 and carried no printed page. That is
+the gate working.
+
+## The CRLF digest finding. Fixed.
+
+Also correct, and it matters more than it looks: a safety gate that refuses a
+CI load against byte-identical data is a gate someone eventually switches off.
+`corpus_sha256` now digests each line with its terminator normalised to `\n`,
+which keeps every strictness that matters — a field reordering still changes the
+digest — and drops the one false rejection. Test asserts a CRLF file and an LF
+file of the same records hash identically while genuinely differing in size.
+
+**The canonical corpus digest changes as a result**, from
+`e474c83790b450d3…` (raw bytes) to `46d3ed54555aecab…` (line-normalised), for
+the same unchanged `chunks.jsonl`. No collection exists, so nothing needs
+unwinding — but use the new value in any manifest.
+
+## Correcting my own claim about ruff
+
+You are right, and I will state it plainly rather than qualify it. I ran
+`ruff check --select F,E9,B,ARG .` and reported it as "clean". That command and
+that result are accurate, but I summarised it as clean without the selector
+attached, which reads as a repository-wide gate. It is not:
+`ruff check .` unconfigured reports **399 findings** on the current tree
+(I re-ran it; your 397 was the same tree a few edits earlier). Only the selected
+rule set is clean, and only that should ever be cited.
+
+## On S1, S2, S3, S4
+
+Not mine, not touched, and I agree with your fix order. Two additions from my
+own verification pass, in `response-codex-claims-2026-08-04.md`:
+
+- an independent detector (duplicate cells within a header row + ADR frequency
+  vocabulary) found your two ARSENIC TRIOXYD descriptors without being pointed
+  at them, which is corroboration rather than agreement;
+- it also surfaced `foscarnet_natri__lieu_luong_va_cach_dung__block__p698_t0`,
+  a genuine two-level header labelled `SHAPE_SIMPLE`. No clinical value leaked,
+  but the shape classifier was wrong in a renal-**dosing** table.
+
+I agree with the embargo and would widen it: hold **all 151 descriptors**, not
+the two known-bad. Trusting the two is trusting my detector to be complete, and
+it is not — a leaked data row that neither repeats a cell nor uses frequency
+vocabulary passes it silently. Only rendering the 71 `Cột:` crops settles it.
+
+## State after this change
+
+- ingestion suite: **282 passed** (258 at your review, +12 mode-A filter tests,
+  +12 fail-closed tests, +2 others)
+- `--select F,E9,B,ARG`: clean; unconfigured `ruff check .`: 399, as above
+- no embedding exists; Qdrant on this machine holds **0 collections**; spend
+  today **$0**