Wire the guarded conversational RAG answer layer end-to-end
This commit is contained in:
@@ -792,6 +792,81 @@ mistaken for complete formula coverage.
|
||||
book is **16/23 = 69.6%**, and its recall is unknown. A geometric heuristic
|
||||
finds candidates; it never proves absence.
|
||||
|
||||
### 26. Exact section vocabulary can occur as wrapped prose or inside tables; context must precede label matching
|
||||
**What it looks like:** several unrelated defects shared one cause. A wrapped
|
||||
body sentence can put `chống chỉ định.` alone on the next visual line
|
||||
(NADROPARIN, physical page 1016); a dosing-table cell can literally be named
|
||||
`Chỉ định` (WARFARIN p1485 and IOBITRIDOL p826); and a verified fraction band
|
||||
widened to capture its numerator can geometrically overlap prose in the other
|
||||
column (NETILMICIN p1042). Exact vocabulary matching alone classified these as
|
||||
structure or quarantined content.
|
||||
|
||||
**Why it matters:** the output remains grammatical while moving or deleting a
|
||||
clinically decisive phrase, assigning a dosing table to indications, or hiding
|
||||
a cross-reference. Aggregate “all spans assigned” and section-level provenance
|
||||
gates all passed before these defects were found.
|
||||
|
||||
**Handling:** classify out-of-scope spans and known table regions before title/
|
||||
section matching; treat a non-bold exact label as prose when it is the adjacent
|
||||
line of an unterminated span in the same PDF block; require a formula region's
|
||||
column to agree with the source span's column; and validate source-span IDs on
|
||||
every individual part. Confirmed aliases (`Tên chung quốc tế và mã ATC`, `Dạng
|
||||
bào chế và hàm lượng`, and the tetanus-toxoid dosing heading) are recorded in
|
||||
the open vocabulary.
|
||||
|
||||
**Whole-corpus result:** 684 monographs (was 683), maximum monograph range 7
|
||||
pages (was the false 164-page ZOLPIDEM range), 11,974 sections, 151 quarantined
|
||||
blocks, 15,066 chunks, 0 unassigned spans, and every readiness gate passing.
|
||||
|
||||
**Generalizes:** vocabulary is evidence, not sufficient context. Apply known
|
||||
geometric scope (page, table, column, visual-line continuity) before interpreting
|
||||
a label-shaped string as document structure.
|
||||
|
||||
### 27. One physical table can be non-contiguous in PDF block order
|
||||
**What it looks like:** a table is contiguous on the rendered page, but the PDF
|
||||
content stream interleaves a visually later section heading between its cells.
|
||||
This split CAPECITABIN p308 and IMATINIB p795 into multiple blocks with the same
|
||||
region ID and conflicting section owners. CAPECITABIN p309 adds a second case:
|
||||
two explicitly captioned dose-adjustment tables are printed after the ordinary
|
||||
`Tên thương mại` field without repeating the dosage heading.
|
||||
|
||||
**Why it matters:** sorting or classifying one extracted span at a time makes a
|
||||
single physical object acquire several meanings. The flattened text remains
|
||||
plausible, so ordinary text and coverage gates do not expose the defect.
|
||||
|
||||
**Handling:** collect all spans belonging to a verified region before semantic
|
||||
classification and emit the region atomically at its first occurrence. A narrow
|
||||
caption rule maps only `Bảng N. Điều chỉnh liều ...` appendices to
|
||||
`lieu_luong_va_cach_dung`; generic occurrences of the word “liều” are not used.
|
||||
A readiness gate now requires unique physical-region IDs.
|
||||
|
||||
**Verification:** all **151/151 unique regions** were rendered and read against
|
||||
the PDF. The regenerated corpus has 151 blocks, 151 unique IDs, and zero
|
||||
duplicate-ID gate failures; CAPECITABIN p309 tables are both owned by dosage.
|
||||
|
||||
**Generalizes:** physical-region identity must outrank text-stream adjacency for
|
||||
tables, formulas, figures, and other layout objects.
|
||||
|
||||
### 28. A bar-less formula needs an asymmetric band, but geometry cannot prove its operator
|
||||
**What it looks like:** ADENOSIN p147 prints a wrapped numerator followed by
|
||||
`Nồng độ adenosin (3 mg/ml).` with no horizontal fraction rule. The generic
|
||||
symmetric formula band captured the numerator only, making a plausible but
|
||||
incomplete source crop.
|
||||
|
||||
**Why it matters:** the missing denominator changes the calculation. Visual
|
||||
review of all reconstructed sandbox crops found the defect even though ordinary
|
||||
readiness and block-count gates passed.
|
||||
|
||||
**Handling:** verified bar-less regions use a 31pt lower margin from the
|
||||
synthetic anchor. On this page the denominator ends about 29pt below the anchor;
|
||||
the following `Ví dụ:` begins immediately after the new boundary. A regression
|
||||
requires the denominator boundary and excludes that prose. The reconstructed
|
||||
record still sets `requires_human_operator_confirmation`: layout supplies no
|
||||
bar from which multiplication versus division can be proven.
|
||||
|
||||
**Generalizes:** expand a verified crop to preserve all visible operands, but
|
||||
never invent a mathematical operator that the source geometry does not encode.
|
||||
|
||||
## Not yet investigated (flagged for future work, not silently ignored)
|
||||
|
||||
- **Footnote-style superscript reference markers** (seen as `a, b, c, d` in
|
||||
@@ -799,11 +874,9 @@ finds candidates; it never proves absence.
|
||||
correctly associated with its marker/row during extraction.
|
||||
- **How many bar-less formulas exist** (item 25) — one confirmed, total
|
||||
unmeasured; no geometric signal can bound it.
|
||||
- **Merging the 51 transcribed outlined runs back into monograph text**
|
||||
(item 24) — transcribed and stored, but the corpus still contains
|
||||
`Độ n định`.
|
||||
- **2D grid table reconstruction** (item 7) — no implementation yet for
|
||||
recovering row/column-correct values from a nomogram-style table.
|
||||
- **Production 2D grid reconstruction** (item 7) — the 100-page sandbox now
|
||||
reconstructs grids and logical cross-page tables, but merged-cell semantics
|
||||
and whole-book recall are not yet production gates.
|
||||
- **Exact shortest monograph name+page** — a quick unmerged crude scan (no
|
||||
multi-line title merge) gave a different longest-monograph ranking than
|
||||
the already-documented authoritative one (item 12e: "AMOXICILIN VÀ KALI
|
||||
|
||||
Reference in New Issue
Block a user