Wire the guarded conversational RAG answer layer end-to-end

docs/adr/0006-quarantined-block-references-in-chunks.md

@@ -2,7 +2,7 @@
 
 ## Status
 
-Proposed, with implementation to follow immediately. Resolves the item ADR
+Accepted and implemented in schema v4. Resolves the item ADR
 0005 explicitly deferred ("Table/formula content blocks … the precise
 `ContentBlock`/table-row/formula-unit shape is deferred to a follow-up
 revision of this ADR once the survey reports real numbers"). The survey has
@@ -22,12 +22,10 @@ the current whole-corpus output:
 
 | quantity | value |
 |---|---|
-| lifted blocks | 167, all quarantined |
-| monographs affected | 96 of 683 (**14.1%**) |
-| sections affected | 108 |
-| **blocks in `lieu_luong_va_cach_dung`** | **127 (76%)** |
-| next largest section | `duoc_ly_va_co_che_tac_dung`, 16 |
-| shapes | simple_table 136, multi_level_or_merged_header 16, formula_2d 14, cross_page_continuation 1 |
+| lifted blocks represented by descriptor chunks | 151, all quarantined |
+| sections affected | 103 |
+| **blocks in `lieu_luong_va_cach_dung`** | **125** |
+| unverified header rows admitted to embedding text | **0** |
 
 So three quarters of everything removed from prose was removed from the
 dosing section, in a drug formulary, for an audience of doctors and
@@ -57,9 +55,11 @@ class ChunkAttachment:
     shape: str           # simple_table | multi_level_or_merged_header |
                          # cross_page_continuation | formula_2d
     physical_page: int
+    printed_page: int
     bbox: List[float]
     quarantined: bool
-    header_row: List[str] = ()   # simple_table only; see caveat below
+    header_row: List[str] = ()   # always empty until separately verified
+    source_crop: str | None = None
 
 @dataclass(frozen=True)
 class Chunk:
@@ -82,28 +82,20 @@ A block also gets its own chunk so it is retrievable at all:
 chunk_id  = "{drug_id}:{section_key}:block:{block_id}"
 chunk_kind = "block_descriptor"
 text       = "AMPICILIN VÀ SULBACTAM — Liều lượng và cách dùng — bảng,
-              trang in 204. Cột: Độ thanh thải creatinin | Nửa đời |
-              Liều ampicilin/sulbactam."
+              trang in 204."
 ```
 
-The text is assembled from the drug name, the section display name, the kind,
-the printed page and — for `simple_table` only — the header row. **No cell
-value ever appears.** A header row is a row of labels; linearising it cannot
-invent a numeric relationship, which is precisely what linearising a body row
-does. For every other shape the header is omitted, because
-`multi_level_or_merged_header` is the shape whose header extraction is least
-trustworthy.
-
-Caveat recorded in the schema itself: `header_row` comes from
-`pdfplumber.find_tables()`'s first row and has **not** been verified by eye
-(the 180 real tables' individual shapes are rule-derived; only the 20
-"not a table" verdicts were visually confirmed). It is retrieval bait, never
-an answer.
+The text is assembled only from verified metadata: drug name, section display
+name, block kind and printed page. **No cell value or inferred header appears.**
+The earlier proposal to use `pdfplumber.find_tables()`'s first row was rejected
+after corpus audit: a guessed first row can be a body row or can merge numeric
+relationships. Until a separate human-verified header dataset exists,
+`header_row` is embargoed for every shape and serialized as empty.
 
 ### 3. The answer layer's obligations (binding on `ai-service`)
 
-These are stated here because they are the reason the schema exists; they are
-not implemented by `ingestion/`.
+These obligations are implemented across `ingestion/` and `ai-service` and are
+enforced by tests/readiness gates.
 
 1. A retrieved chunk with `has_quarantined_content: true` **must** cause the
    answer to state that a table or formula exists at the cited page, and to
@@ -158,14 +150,18 @@ Added to `cli chunk-ready` and to the chunk stage's own tests:
    contains a quarantined block's text
 5. `descriptor_chunk_count == block_count`
 6. `descriptor_chunk_without_attachment = 0`
+7. `attachment_header_row_present = 0`
+8. `descriptor_with_unverified_columns = 0`
+9. `descriptor_range_not_attachment_page = 0`
+10. `attachment_without_printed_page = 0`
 
 ## Consequences
 
 - Prose chunks shrink slightly in trustworthiness terms but grow in honesty:
   the ones missing a table now say so.
-- The index gains 167 descriptor chunks (≈1.4% of the expected chunk count),
+- The current candidate index gains 151 descriptor chunks,
   each cheap and none carrying unsafe text.
-- `ai-service` cannot be built to answer a dosing question from prose alone
-  for the 108 affected sections without violating a stated contract.
+- `ai-service` cannot answer a dosing question from prose alone for the 103
+  affected sections without violating a stated contract.
 - The 14 `formula_2d` attachments make the two Cockcroft-Gault formulas
   answerable as crops today, which they are not now.