Wire the guarded conversational RAG answer layer end-to-end

This commit is contained in:
2026-08-05 14:33:13 +07:00
parent 834d9e51b0
commit ef08b4929e
127 changed files with 37921 additions and 169 deletions
+78 -5
View File
@@ -792,6 +792,81 @@ mistaken for complete formula coverage.
book is **16/23 = 69.6%**, and its recall is unknown. A geometric heuristic
finds candidates; it never proves absence.
### 26. Exact section vocabulary can occur as wrapped prose or inside tables; context must precede label matching
**What it looks like:** several unrelated defects shared one cause. A wrapped
body sentence can put `chống chỉ định.` alone on the next visual line
(NADROPARIN, physical page 1016); a dosing-table cell can literally be named
`Chỉ định` (WARFARIN p1485 and IOBITRIDOL p826); and a verified fraction band
widened to capture its numerator can geometrically overlap prose in the other
column (NETILMICIN p1042). Exact vocabulary matching alone classified these as
structure or quarantined content.
**Why it matters:** the output remains grammatical while moving or deleting a
clinically decisive phrase, assigning a dosing table to indications, or hiding
a cross-reference. Aggregate “all spans assigned” and section-level provenance
gates all passed before these defects were found.
**Handling:** classify out-of-scope spans and known table regions before title/
section matching; treat a non-bold exact label as prose when it is the adjacent
line of an unterminated span in the same PDF block; require a formula region's
column to agree with the source span's column; and validate source-span IDs on
every individual part. Confirmed aliases (`Tên chung quốc tế và mã ATC`, `Dạng
bào chế và hàm lượng`, and the tetanus-toxoid dosing heading) are recorded in
the open vocabulary.
**Whole-corpus result:** 684 monographs (was 683), maximum monograph range 7
pages (was the false 164-page ZOLPIDEM range), 11,974 sections, 151 quarantined
blocks, 15,066 chunks, 0 unassigned spans, and every readiness gate passing.
**Generalizes:** vocabulary is evidence, not sufficient context. Apply known
geometric scope (page, table, column, visual-line continuity) before interpreting
a label-shaped string as document structure.
### 27. One physical table can be non-contiguous in PDF block order
**What it looks like:** a table is contiguous on the rendered page, but the PDF
content stream interleaves a visually later section heading between its cells.
This split CAPECITABIN p308 and IMATINIB p795 into multiple blocks with the same
region ID and conflicting section owners. CAPECITABIN p309 adds a second case:
two explicitly captioned dose-adjustment tables are printed after the ordinary
`Tên thương mại` field without repeating the dosage heading.
**Why it matters:** sorting or classifying one extracted span at a time makes a
single physical object acquire several meanings. The flattened text remains
plausible, so ordinary text and coverage gates do not expose the defect.
**Handling:** collect all spans belonging to a verified region before semantic
classification and emit the region atomically at its first occurrence. A narrow
caption rule maps only `Bảng N. Điều chỉnh liều ...` appendices to
`lieu_luong_va_cach_dung`; generic occurrences of the word “liều” are not used.
A readiness gate now requires unique physical-region IDs.
**Verification:** all **151/151 unique regions** were rendered and read against
the PDF. The regenerated corpus has 151 blocks, 151 unique IDs, and zero
duplicate-ID gate failures; CAPECITABIN p309 tables are both owned by dosage.
**Generalizes:** physical-region identity must outrank text-stream adjacency for
tables, formulas, figures, and other layout objects.
### 28. A bar-less formula needs an asymmetric band, but geometry cannot prove its operator
**What it looks like:** ADENOSIN p147 prints a wrapped numerator followed by
`Nồng độ adenosin (3 mg/ml).` with no horizontal fraction rule. The generic
symmetric formula band captured the numerator only, making a plausible but
incomplete source crop.
**Why it matters:** the missing denominator changes the calculation. Visual
review of all reconstructed sandbox crops found the defect even though ordinary
readiness and block-count gates passed.
**Handling:** verified bar-less regions use a 31pt lower margin from the
synthetic anchor. On this page the denominator ends about 29pt below the anchor;
the following `Ví dụ:` begins immediately after the new boundary. A regression
requires the denominator boundary and excludes that prose. The reconstructed
record still sets `requires_human_operator_confirmation`: layout supplies no
bar from which multiplication versus division can be proven.
**Generalizes:** expand a verified crop to preserve all visible operands, but
never invent a mathematical operator that the source geometry does not encode.
## Not yet investigated (flagged for future work, not silently ignored)
- **Footnote-style superscript reference markers** (seen as `a, b, c, d` in
@@ -799,11 +874,9 @@ finds candidates; it never proves absence.
correctly associated with its marker/row during extraction.
- **How many bar-less formulas exist** (item 25) — one confirmed, total
unmeasured; no geometric signal can bound it.
- **Merging the 51 transcribed outlined runs back into monograph text**
(item 24) — transcribed and stored, but the corpus still contains
`Độ n định`.
- **2D grid table reconstruction** (item 7) — no implementation yet for
recovering row/column-correct values from a nomogram-style table.
- **Production 2D grid reconstruction** (item 7) — the 100-page sandbox now
reconstructs grids and logical cross-page tables, but merged-cell semantics
and whole-book recall are not yet production gates.
- **Exact shortest monograph name+page** — a quick unmerged crude scan (no
multi-line title merge) gave a different longest-monograph ranking than
the already-documented authoritative one (item 12e: "AMOXICILIN VÀ KALI