Add read-only production runtime audit
This commit is contained in:
@@ -0,0 +1,45 @@
|
||||
# ADR 0001: Use Qdrant as the vector database
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The RAG pipeline needs a vector store for drug-monograph chunks. The main
|
||||
alternative considered was **pgvector** (a Postgres extension), which would
|
||||
let us reuse the Postgres instance already needed for users/chat history —
|
||||
one fewer moving part to operate.
|
||||
|
||||
The corpus is not free-flowing prose: it's a structured per-drug reference
|
||||
with rich per-chunk metadata (drug name, section type, page range). The
|
||||
common retrieval pattern this domain calls for is "vector similarity search,
|
||||
filtered by metadata" — e.g. "search only within chỉ định sections" or
|
||||
"filter to a specific drug the user named" combined with the semantic query.
|
||||
|
||||
## Decision
|
||||
|
||||
Use **Qdrant** as a dedicated vector database, separate from Postgres.
|
||||
|
||||
## Rationale
|
||||
|
||||
- Qdrant gives first-class combined payload-filtering + ANN search in a
|
||||
single query, which is exactly the retrieval pattern this structured
|
||||
corpus needs — pgvector supports filtering too, but it's a less natural
|
||||
fit layered on top of a general-purpose relational engine.
|
||||
- Vector search becomes its own independent scaling axis, separate from the
|
||||
transactional Postgres workload (users/chat) — re-indexing or re-ingesting
|
||||
the formulary doesn't contend with transactional traffic.
|
||||
- Mature standalone Docker image for local dev, a well-supported Python
|
||||
client, and a Helm chart for the production Kubernetes deployment target.
|
||||
- Corpus size (tens of thousands of chunks) is trivial for Qdrant's HNSW
|
||||
indexing.
|
||||
|
||||
## Consequences
|
||||
|
||||
- One additional service to operate/deploy/monitor compared to pgvector
|
||||
(which would ride on the existing Postgres).
|
||||
- Revisit if operational overhead becomes a real burden at our actual scale,
|
||||
or if we want tighter transactional consistency between chat data and
|
||||
retrieval — pgvector remains a viable fallback documented here for that
|
||||
case.
|
||||
@@ -0,0 +1,68 @@
|
||||
# ADR 0002: Use the team's existing ArgoCD instance for deployment (GitOps)
|
||||
|
||||
## Status
|
||||
|
||||
**Accepted — still the target, not yet implemented.** Not superseded by the
|
||||
current production setup.
|
||||
|
||||
Since 2026-08-10 the project has a *different*, interim deployment: a single
|
||||
EC2 box running `infra/docker/docker-compose.prod.yml` behind Caddy, deployed
|
||||
by `.github/workflows/deploy.yml` over SSH. That was built to get a working
|
||||
demo online, not to replace this decision. Migrating to the team's Kubernetes
|
||||
+ ArgoCD remains planned work, and the expensive prerequisite — containerising
|
||||
both apps — is already done, so the Dockerfiles and compose services port over.
|
||||
|
||||
Two things must still happen and neither has been started:
|
||||
|
||||
1. **Repository move to the team's self-hosted Gitea** (company domain), which
|
||||
is where the GitOps repo is meant to live. The project stays on private
|
||||
GitHub until that move is deliberately made. Note the hard boundary already
|
||||
in force: the team's existing `git.vinmec.tech/ai-team/gitops` repository is
|
||||
**reference-only** — never push this project into it.
|
||||
2. **Filling in the scaffolds this ADR assumes exist.** `infra/helm/medical-chatbot/templates/`
|
||||
and `infra/k8s/**` are empty (`.gitkeep` only), the chart is version `0.0.0`,
|
||||
and every `infra/argocd/applications/*/app.yaml` still carries unresolved
|
||||
TODOs for project, repo URL and destination cluster.
|
||||
|
||||
## Context
|
||||
|
||||
Phase 6 of the build roadmap needs a way to actually deploy the Helm chart to
|
||||
Kubernetes across dev/staging/prod. The original scaffold (`infra/ci/github-actions/deploy-cd.yml`)
|
||||
assumed a push-based CI deploy step (CI runs `helm upgrade`/`kubectl apply`
|
||||
directly against the cluster). The team already runs an ArgoCD instance used
|
||||
by other projects.
|
||||
|
||||
## Decision
|
||||
|
||||
Deploy via **GitOps through the team's existing ArgoCD instance** instead of
|
||||
building a custom push-based CD pipeline. ArgoCD Applications
|
||||
(`infra/argocd/applications/{dev,staging,prod}/app.yaml`) point at
|
||||
`infra/helm/medical-chatbot` in this repo; ArgoCD watches the repo and
|
||||
reconciles the cluster to match.
|
||||
|
||||
## Rationale
|
||||
|
||||
- Reuses infrastructure the team already operates and trusts, instead of
|
||||
standing up a parallel deploy mechanism.
|
||||
- GitOps gives an auditable history of every deploy (it's just git commits
|
||||
changing values files/image tags) and a built-in rollback path (revert the
|
||||
commit).
|
||||
- Removes the need for CI to hold cluster credentials — CI's job shrinks to
|
||||
"build, test, push image, bump tag," which is a smaller security surface
|
||||
than "CI can directly mutate the production cluster."
|
||||
- Prod uses a non-automated `syncPolicy` (manual approval in ArgoCD) while
|
||||
dev/staging auto-sync, matching normal caution around production changes.
|
||||
|
||||
## Consequences
|
||||
|
||||
- CI workflows (`infra/ci/github-actions/*.yml`) build/test/push images and
|
||||
bump the relevant `values-<env>.yaml` image tag + push that commit; they do
|
||||
**not** call `kubectl`/`helm` against any cluster directly.
|
||||
- Actual deploy execution and health/sync status live in the team's ArgoCD
|
||||
UI/CLI, outside this repo — runbooks in `docs/runbooks/` should document how
|
||||
to check sync status and roll back once the team's ArgoCD instance details
|
||||
(cluster/server, project, repo URL) are confirmed (see TODOs in
|
||||
`infra/argocd/README.md`).
|
||||
- If the team's ArgoCD instance becomes unavailable or this project needs to
|
||||
fully own its own deploy tooling later, the push-based `deploy-cd.yml`
|
||||
approach remains a documented fallback.
|
||||
@@ -0,0 +1,208 @@
|
||||
# ADR 0003: PDF parsing strategy for the drug formulary — validated empirically
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (validated against the real 1668-page source PDF, not assumptions)
|
||||
|
||||
## Context
|
||||
|
||||
The original scaffold's ingestion design (see `docs/architecture.md` history)
|
||||
assumed generic best practices for structured-PDF parsing: prefer the PDF's
|
||||
bookmark/outline (`doc.get_toc()`) for section boundaries, fall back to
|
||||
font-size heuristics. Before writing real ingestion code, this assumption was
|
||||
tested against the actual `duoc-thu-quoc-gia-viet-nam-2018.pdf` (1668 pages),
|
||||
because a 1668-page book has enough real-world irregularity that guessing
|
||||
from a handful of sample pages is not sufficient grounds to trust a parsing
|
||||
strategy — every claim below was checked against the whole document or a
|
||||
independently-sourced ground truth, not a small sample.
|
||||
|
||||
## What was actually tested
|
||||
|
||||
1. **`doc.get_toc()`**: returns **0 entries**. No usable bookmark/outline.
|
||||
2. **Tagged-PDF structure tree** (`/StructTreeRoot`): exists, but is shallow
|
||||
— ~29 generic `/H1`/`/P` elements, evidently covering only a small
|
||||
fraction of the document. Not usable as a structural signal at scale.
|
||||
Confirmed dead end.
|
||||
3. **Cross-tool text-extraction comparison** on the same real pages
|
||||
(a known drug monograph, "Abacavir"):
|
||||
- **PyMuPDF (`fitz`)**: correct reading order, matches the visual source.
|
||||
- **pdfplumber** (`extract_text()`): **incorrect** — scrambles paragraph
|
||||
order on this layout and surfaces a stray marked-content artifact
|
||||
(`"PB <Header tên thuốc>"`) as if it were visible text. Decision:
|
||||
pdfplumber is kept **only** for its `extract_tables()` API (a genuinely
|
||||
different, table-specific algorithm), never for general body text.
|
||||
- **opendataloader-pdf** (Java-based, benchmarks #1 in public leaderboards
|
||||
for reading order/tables): correct reading order, and its own computed
|
||||
font metadata (per-span `font`/`font size` in its JSON output)
|
||||
**independently agreed** with PyMuPDF's raw span data — two unrelated
|
||||
tools agreeing on the same font facts is real cross-validation, not
|
||||
opinion. However, its higher-level paragraph/heading classifier is
|
||||
**inconsistent**: identical bold section-heading text (e.g. "Dược lý và
|
||||
cơ chế tác dụng", "Liều lượng và cách dùng") is sometimes promoted to a
|
||||
markdown `##` heading and sometimes silently merged into the following
|
||||
body paragraph, for no discernible content-based reason. Conclusion: its
|
||||
Markdown/heading output is not reliable enough to be the sole
|
||||
structural signal, but it's a useful independent check and its
|
||||
header/footer-stripping was notably better than raw PyMuPDF text.
|
||||
- **docling**: attempted, blocked by a `numpy`/`pyarrow` ABI conflict in
|
||||
the environment (numpy 2.x vs a pyarrow build expecting numpy 1.x,
|
||||
pulled in transitively via `torch`/`transformers`). Tested inside an
|
||||
isolated venv rather than fixed globally, to avoid destabilizing other
|
||||
tools on the machine. See progress log for current status.
|
||||
4. **The definitive structural signal — bold font spans**: at the raw
|
||||
PyMuPDF span level, every section heading and every monograph title is
|
||||
rendered in a **bold** font (`"...-BoldMT"`), while body text is not.
|
||||
Italic spans exist too (foreign/Latin species names inline) but are
|
||||
never confused with headings since they're not bold and appear mid
|
||||
sentence. This was cross-confirmed by opendataloader's independently
|
||||
computed font metadata for the same spans (see above) — not a
|
||||
single-tool guess.
|
||||
- **Font size is NOT a reliable discriminator on its own**: a monograph
|
||||
title was observed at both 10.0pt ("ABACAVIR") and 9.5pt ("ACARBOSE")
|
||||
for equally genuine, equally top-level monograph headings. An earlier
|
||||
draft of the detector required `size >= 9.8` based on the first
|
||||
example seen and it silently dropped ~15% of real monographs as a
|
||||
result — a concrete instance of exactly the "don't generalize from one
|
||||
example" risk this investigation was meant to guard against. The fix:
|
||||
drop the size floor; use **bold + all-caps + short line length** for
|
||||
monograph titles, and **bold** alone (cross-checked against the known
|
||||
section-name vocabulary) for section headings.
|
||||
5. **Ground truth for validation**: the book has **two** indexes:
|
||||
- The front-matter "Danh mục các chuyên luận thuốc" (pages 12-31,
|
||||
0-indexed): an alphabetical name list with **no page numbers** — useful
|
||||
only for a name-overlap sanity check, not page-level validation.
|
||||
- The back-of-book "Mục lục tra cứu" (from page ~1529 printed / ~1528
|
||||
0-indexed onward): a proper index with **exact page numbers** per
|
||||
generic-name entry (e.g. `"Abacavir, 101"`), plus brand-name
|
||||
cross-references (`"Ziagen - Abacavir, 101"`, skipped for ground truth).
|
||||
This is the real, page-verifiable ground truth and should be used for
|
||||
any future re-validation, not the front-matter list.
|
||||
- The front matter's own "NỘI DUNG" (table of contents, page 7 0-indexed)
|
||||
also gives exact page ranges for the book's 3 parts: general topic
|
||||
chapters (37-98 printed), individual drug monographs (**99-1496
|
||||
printed**), appendices (1497-1528), back index (1529+). Any monograph-
|
||||
boundary detector should be scoped to the 99-1496 printed page range —
|
||||
scanning the whole book without this scope produces false positives
|
||||
from front-matter/general-chapter bold-caps lines (org names, decree
|
||||
headers, chapter titles) that are not drug monographs.
|
||||
|
||||
## Decision
|
||||
|
||||
- **PyMuPDF is the primary and only general-text extractor.** No TOC
|
||||
dependency, no reliance on the structure tree.
|
||||
- **Section/monograph boundary detection uses bold-font spans** (not font
|
||||
size, not font size + vocabulary alone), scoped to the printed page range
|
||||
of the actual monograph section (99-1496), with all-caps + short length as
|
||||
the additional signal narrowing bold spans down to monograph titles
|
||||
specifically. Multi-line wrapped titles must be merged before matching.
|
||||
- **pdfplumber is retained only for table extraction** (`extract_tables()`),
|
||||
never general reading order, per the confirmed scrambling issue.
|
||||
- **The back-of-book "Mục lục tra cứu" is the ground truth for validation**,
|
||||
not the front-matter drug list.
|
||||
- **Validation is a repeatable, whole-document, automated check**, not a
|
||||
one-time manual read of a handful of pages: a full 1668-page scan runs in
|
||||
under a minute, so re-running it after every heuristic change is cheap and
|
||||
should be standard practice before trusting a change.
|
||||
|
||||
## Validation results (most recent full-document run)
|
||||
|
||||
- Page-verified recall against the back-of-book index: **91.7%** (665/725
|
||||
primary entries had a detected boundary within ±2 physical pages of the
|
||||
expected page).
|
||||
- Remaining misses are overwhelmingly one identified, fixable cause:
|
||||
**multi-line wrapped ALL-CAPS titles** (long Vietnamese drug/vaccine names
|
||||
spanning 2+ physical lines) being matched as fragments rather than merged
|
||||
— not a failure of the bold-span signal itself. A handful of misses are
|
||||
ground-truth extraction noise (the back-index parser occasionally picks up
|
||||
a non-drug appendix/table-of-contents line that happens to match the
|
||||
`"Name, ###"` pattern) rather than real detector failures.
|
||||
- Expected recall after fixing multi-line merging and cleaning non-drug
|
||||
entries out of the ground truth: materially higher than 91.7%, to be
|
||||
re-measured once that fix lands (Phase 1 implementation, not this ADR).
|
||||
|
||||
## Follow-up validation: duplicates and cross-page/column data-loss risk
|
||||
|
||||
Two further questions were raised and empirically tested against the full
|
||||
1405-page monograph range (99-1496 printed):
|
||||
|
||||
1. **Are any drugs detected twice (real content duplication)?** Scanned for
|
||||
normalized-name collisions at physically distant pages. Found exactly
|
||||
**one** candidate: `"GONADOTROPIN"` at physical pages 755 and 1371. On
|
||||
inspection, this is **not** a real duplicate — page 755 is the genuine
|
||||
"GONADOTROPIN" monograph (hCG/menotropin/follitropin), while page 1371 is
|
||||
a different monograph, "THUỐC TƯƠNG TỰ HORMON GIẢI PHÓNG GONADOTROPIN"
|
||||
(GnRH-analog drugs), whose title wraps across two lines — the detector
|
||||
matched only the second line ("GONADOTROPIN"), colliding with the
|
||||
unrelated monograph's normalized key. This is the **same multi-line
|
||||
title-wrapping bug** already identified above, now confirmed with a
|
||||
second concrete example, not a new failure mode. **Conclusion: no real
|
||||
duplicate monographs found in the corpus**; the multi-line merge fix
|
||||
(already required for Phase 1) also resolves this collision.
|
||||
|
||||
2. **Is the PDF two-column, and can content be lost/corrupted across a page
|
||||
or column boundary during chunking?** Confirmed via bounding-box
|
||||
inspection: this document **is** genuinely two-column (left column
|
||||
x≈44-299, right column x≈308-562, same page). PyMuPDF's block-level
|
||||
reading order correctly sequences left-column-then-right-column content
|
||||
(already implicitly validated by the correct Abacavir sample earlier).
|
||||
However, a **separate, real defect** was found and confirmed: on physical
|
||||
page 1373, one short run of text has its glyphs in **reversed
|
||||
(right-to-left) x-order**, producing scrambled output — e.g. `" = tịx 8
|
||||
yàgn gnàh uềil gnổt( uềihc iổub oàv )magorcim 008 = tịx 4( "`, which
|
||||
reverses character-by-character back to the correct
|
||||
`"(4 xịt = 800 microgam) vào buổi chiều (tổng liều hàng ngày 8 xịt = ..."`.
|
||||
This looks like an isolated PDF-authoring artifact (e.g. an accidental
|
||||
RTL/BiDi override on one small span during editing), not a systemic
|
||||
extraction bug. **Initially this scan was scoped to the monograph range
|
||||
only (1405 of 1668 pages) — an oversight, caught and corrected**:
|
||||
re-run across all 1668 pages (front matter, general chapters,
|
||||
monographs, appendices, back index — the entire book, page 0 to the
|
||||
last page), it still found **exactly 1 affected row, on the same page
|
||||
1373, and no others** — confirming the defect is genuinely isolated, not
|
||||
hiding somewhere in the ~260 pages outside the original scan scope.
|
||||
- The same full-book pass also checked for near-empty pages (<20 chars
|
||||
extracted): found exactly **6** — physical pages 3, 37, 99, 1495, 1497,
|
||||
1666 — every one lands exactly at a major section boundary (before
|
||||
"Các chuyên luận chung" at 37, before "Các chuyên luận thuốc" at 99,
|
||||
before "Các phụ lục" at 1497, near the book's end at 1666). These are
|
||||
intentional print-layout blank/separator pages, not lost content —
|
||||
standard practice to force a new part to start on a fresh page.
|
||||
|
||||
**Implications for Phase 1 implementation:**
|
||||
- Build the pipeline as one **continuous cross-page stream** (text + page
|
||||
number + bbox per fragment, in reading order), not per-page-isolated
|
||||
chunks — this is required both for correctly merging multi-line
|
||||
monograph/section titles (see above) and for never truncating a
|
||||
paragraph/sentence that spans a page or column break.
|
||||
- Add an automated **glyph-order sanity check** as a mandatory pass over
|
||||
100% of pages (not sampled): group text fragments into visual rows by
|
||||
y-coordinate, verify x-coordinates are non-decreasing, and either
|
||||
auto-correct (re-sort by x — the fix is deterministic since raw glyph
|
||||
positions are known) or flag for manual QA. This check is cheap
|
||||
(~16 seconds over the full monograph range) and should run before every
|
||||
real ingestion, not just once.
|
||||
- The book's content must ultimately be captured **from page 0 to the last
|
||||
page** — but not all of it as drug-monograph chunks: front matter (pages
|
||||
0-36) is mostly low-value organizational/decree content and can be
|
||||
largely skipped for RAG purposes; general topic chapters (37-98) and
|
||||
appendices (1497-1528) are real, valuable content that must be ingested
|
||||
too, using their own heading-hierarchy-based chunking (not the drug
|
||||
template) — this was already noted in `docs/architecture.md`'s original
|
||||
design and is reaffirmed here, not changed. The back-of-book index
|
||||
(1529+) does not need its own chunks (it's a page-locator, not content)
|
||||
but remains the validation ground truth.
|
||||
|
||||
## Consequences
|
||||
|
||||
- The real ingestion pipeline (Phase 1) should implement the bold-span
|
||||
detector directly (reusing the validated logic, not the exploratory
|
||||
scratch scripts), scoped to the correct page range, with multi-line
|
||||
heading merging as a required fix before first real ingestion run.
|
||||
- Every future change to the segmentation heuristic should be re-validated
|
||||
with the same whole-document + back-index cross-reference script (or its
|
||||
Phase 1 equivalent) before being trusted — this is now the project's
|
||||
standard rigor bar for this pipeline, not an optional nice-to-have.
|
||||
- `opendataloader-pdf` (Java-based) and `pdfplumber`'s table extraction
|
||||
remain candidate tools for the table/formula-handling fallback path
|
||||
described in `docs/architecture.md`; docling's viability is still
|
||||
unresolved pending the environment fix.
|
||||
@@ -0,0 +1,131 @@
|
||||
# ADR 0004: Chunking strategy for drug monographs — validated against real per-section measurements
|
||||
|
||||
## Status
|
||||
|
||||
Accepted for the monograph range (printed pp. 99-1496) only. General
|
||||
chapters (pp. 37-98) and appendices (pp. 1497-1528) are explicitly out of
|
||||
scope — see Consequences.
|
||||
|
||||
## Context
|
||||
|
||||
`docs/architecture.md`'s original "Chunking" paragraph specified `(drug,
|
||||
section)` as the chunk unit, a ~500-800 token budget, and a 400-token/
|
||||
50-overlap sliding window for oversized sections. Those numbers were written
|
||||
before segmentation existed — a plausible guess, never checked against real
|
||||
per-section text length.
|
||||
|
||||
Phase 1 (extract → segment → validate) is now real, tested code producing
|
||||
682 real monographs from the full 1668-page source PDF. This session ran
|
||||
`python -m ingestion.cli run` for real and measured actual per-section
|
||||
length across the whole corpus with a temporary investigation script
|
||||
(`ingestion/scratch/chunking_stats_survey.py`, deleted after this ADR
|
||||
captured its findings, per this project's investigation-script rule) —
|
||||
something that had never been measured before this ADR.
|
||||
|
||||
## What was actually measured (whole corpus, 682 monographs)
|
||||
|
||||
- Sections per monograph: min 11, median 17, max 19 (of ~18-19 known
|
||||
section keys in `segment/vocab.py`'s open taxonomy).
|
||||
- Whole-monograph length: median 11,480 chars, p90 19,068 chars, max 38,786
|
||||
chars.
|
||||
- Per-section length, converted to a **chars/4 token estimate — an
|
||||
estimate, not a real tokenizer count**:
|
||||
- Most of the ~18 section types sit comfortably under 800 estimated
|
||||
tokens even at their p90 (e.g. `chi_dinh` p90≈268 tok, `dang_thuoc_va_
|
||||
ham_luong` p90≈115 tok, `tac_dung_khong_mong_muon` p90≈481 tok).
|
||||
- **Two sections routinely exceed 800 tokens**:
|
||||
`duoc_ly_va_co_che_tac_dung` (242 of 678 monographs that have this
|
||||
section, 35.7%, max ≈3542 tok) and `lieu_luong_va_cach_dung` (200 of
|
||||
675, 29.6%, max ≈3631 tok).
|
||||
- A smaller tail also exceeds it: `than_trong` (25/680, 3.7%),
|
||||
`tuong_tac_thuoc` (22/642, 3.4%).
|
||||
- This means: the original 800-token ceiling is directionally correct
|
||||
(it clears ~16 of 18 section types at their p90 with room to spare),
|
||||
but "sub-chunk in that case" is not a rare hedge as originally implied
|
||||
— it is the routine path for roughly a third of all monographs, on two
|
||||
specific, named, high-clinical-importance sections (mechanism of
|
||||
action and dosing).
|
||||
|
||||
**A separate, blocking bug was found while gathering this data, not fixed
|
||||
by this ADR** (out of scope — belongs to `extract`/`segment`, owned by a
|
||||
parallel session at the time of writing): running header/footer
|
||||
boilerplate ("DTQGVN 2" + page number + repeated drug name, tagged
|
||||
`column="full_width"` in `extract/spans.py`) is never filtered out of
|
||||
section body text before it reaches `SectionSpan.text`. Measured:
|
||||
1,374 of 11,409 sections (12.0%) contain a literal "DTQGVN" string
|
||||
mid-text; 671 of 682 monographs (98.4%) have at least one affected section
|
||||
(e.g. MORPHIN SULFAT's `lieu_luong_va_cach_dung`: `"...Nếu\nDTQGVN 2\n1009\n
|
||||
Morphin sulfat\nuống viên thuốc..."`). This is `docs/pdf-parsing-outlier-
|
||||
catalog.md` item 13's known risk, measured whole-corpus for the first time
|
||||
here. **Chunking must not run against real data until this is fixed** —
|
||||
otherwise boilerplate is baked into embeddings and can surface mid-sentence
|
||||
in a chunk shown to a doctor or pharmacist.
|
||||
|
||||
## Decision
|
||||
|
||||
1. **Chunk unit stays `(drug_id, section_key)`** — matches
|
||||
`segment/models.py`'s existing `Monograph.sections: Dict[str,
|
||||
SectionSpan]`, matches how a doctor/pharmacist would query ("what does
|
||||
it say about liều dùng"), and lets a citation point at one clinical
|
||||
section rather than a whole 2,000-19,000-char monograph.
|
||||
2. **Token budget: keep the 800-token ceiling** (chars/4 estimate) as the
|
||||
split trigger. Below it, a section is one chunk, verbatim. This is now a
|
||||
validated choice, not a guess.
|
||||
3. **Sub-chunking only applies to the long-tail sections above** (~30-36%
|
||||
of monographs for the two named sections, a few percent for the rest).
|
||||
Method: **sentence-boundary-aware sliding window**, replacing the
|
||||
originally-guessed fixed-character window. Target ~600-700 tokens per
|
||||
sub-chunk (headroom under the 800 ceiling), ~1 sentence / 50-80 token
|
||||
overlap between adjacent sub-chunks. Split only at a sentence boundary
|
||||
(`.`/`;`/`:` followed by whitespace + capital letter), explicitly not
|
||||
treating a Vietnamese decimal comma (e.g. "0,425") as a boundary.
|
||||
4. **Why sentence-aware, not line- or character-based**: `assembler.py`
|
||||
joins `body_lines` one line per PyMuPDF *span*, i.e. one PDF visual
|
||||
line-wrap point — not a semantic paragraph or sentence boundary. A blind
|
||||
character/line window can split a sentence mid-way. This is a real,
|
||||
measured risk here, not theoretical: outlier-catalog item 17 found
|
||||
adult/child dosing splits ("Người lớn"/"Trẻ em") appear on 1,121 of
|
||||
~1,400 monograph-range pages — a chunk boundary landing inside one of
|
||||
those sentences would be a patient-safety-relevant defect, not a
|
||||
cosmetic one.
|
||||
5. **Chunk metadata / provenance** (extends the existing `drug_name,
|
||||
section_type, source_page_range, chunk_id` list in `docs/architecture.md`
|
||||
— per CLAUDE.md's provenance rule): `chunk_id`
|
||||
(`{drug_id}__{section_key}__{part_index}`), `drug_id`, `drug_name`,
|
||||
`section_key`, `section_display_name`, `atc_codes` (inherited from the
|
||||
monograph — enables ATC-class-filtered retrieval), exact per-chunk
|
||||
`source_page_range` and `printed_page_range`, `part_index`/`part_count`
|
||||
(`0`/`1` for un-split sections, keeps the schema uniform across all chunks).
|
||||
6. **Schema v4 separates source from retrieval context.** `source_text` is the
|
||||
exact contiguous source span and is the basis for lossless reassembly and
|
||||
page provenance. `text` may prefix repeated route/population labels so a
|
||||
continuation chunk is independently safe to retrieve. Those retrieval-only
|
||||
prefixes are recorded in `context_labels` and may not alter `source_text`.
|
||||
Token counts use `cl100k_base`, not the earlier chars/4 estimate.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **Scope**: this decision covers the monograph range only. General
|
||||
chapters and appendices contain real tables and 2D stacked-fraction
|
||||
formulas (`docs/document-profile.md`, investigation in progress as of
|
||||
this ADR) that need their own structural survey before any chunking rule
|
||||
can be designed for them — do not extend this ADR's rules to those ranges
|
||||
without a fresh investigation.
|
||||
- **Hard prerequisite**: the boilerplate-leakage bug described above must
|
||||
be fixed in `extract`/`segment` before this chunking design is run
|
||||
against real data for ingestion. This ADR does not fix it.
|
||||
- **Known gap — sub-compound tagging inside class-level monographs**: 25.5%
|
||||
of the corpus has more than one ATC code per monograph (outlier item
|
||||
12a), e.g. "VITAMIN D VÀ CÁC THUỐC TƯƠNG TỰ" documents dosing for 7
|
||||
different analogues inside one `lieu_luong_va_cach_dung` section. No
|
||||
reliable structural signal was found in sampled text to split a section
|
||||
by sub-compound — a chunk from this section is tagged with the class
|
||||
name only, not the specific analogue a query might target. Deferred to
|
||||
golden-dataset-driven eval rather than guessed at now.
|
||||
- **Resolved — sub-chunk page precision**: schema v4 derives exact physical
|
||||
support from the contiguous `source_text` span and maps it to verified
|
||||
printed folios. Missing or ambiguous support fails readiness rather than
|
||||
falling back to monograph-level provenance.
|
||||
- **Implemented**: the sentence/label-aware splitter is in
|
||||
`ingestion/ingestion/chunk/` with regression tests for dose continuations,
|
||||
compound label boundaries, parent route context and lossless reassembly.
|
||||
@@ -0,0 +1,216 @@
|
||||
# ADR 0005: `segment/` output contract needed by `chunk/` — structure-preserving, not flattened
|
||||
|
||||
## Status
|
||||
|
||||
Proposed. **Contract/schema only — no implementation.** `segment/models.py`,
|
||||
`segment/assembler.py`, and `segment/io.py` are actively owned by a parallel
|
||||
session on the same checkout at the time of writing; this ADR specifies what
|
||||
`chunk/` needs from `segment/`'s output precisely enough to implement and
|
||||
test, but does not touch those files itself. Supersedes part of ADR 0004
|
||||
(see "Relationship to ADR 0004" below) — ADR 0004's `(drug_id, section_key)`
|
||||
chunk-unit-as-leaf assumption is corrected here to `(drug_id, section_key)`
|
||||
as a **parent**, with sentence-window splitting demoted from primary
|
||||
strategy to fallback.
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0004 designed chunking against `segment/models.py`'s current output:
|
||||
`SectionSpan.text` is a single flattened string per section (`"\n".join(
|
||||
body_line.strip() for ...)`), with all per-line style (`Span.bold`) and
|
||||
per-line page position discarded once the string is built (confirmed by
|
||||
reading `assembler.py`: `body_lines.append(span.text.strip())` keeps only
|
||||
`span.text`, nothing else). Review of ADR 0004 surfaced four real problems
|
||||
that trace back to this flattening, not to the chunking algorithm itself:
|
||||
|
||||
1. **A section is not a single semantic unit.** `liều lượng và cách dùng`
|
||||
and `tương tác thuốc` routinely contain multiple distinct facts (dosing
|
||||
per patient population, dosing per organ-function impairment, multiple
|
||||
separate drug interactions) that a doctor may want to retrieve
|
||||
independently. Measured: an explicit population marker ("Người lớn"/
|
||||
"Trẻ em"/"Trẻ sơ sinh"/"Suy thận"/"Suy gan" immediately followed by `:`
|
||||
or `.`) appears in **303 of 675 monographs (44.9%)** that have a `liều
|
||||
lượng và cách dùng` section — this is common, not an edge case.
|
||||
2. **A blind sentence-boundary sliding window (ADR 0004's original
|
||||
sub-chunking method) can still split two different facts into the same
|
||||
chunk, or split one fact across two chunks**, because it has no way to
|
||||
know a population/interaction boundary exists — that information exists
|
||||
in the source (as a bold or otherwise visually distinct sub-heading, per
|
||||
direct reading of MORPHIN SULFAT/VITAMIN D section text: lines like
|
||||
"Thuốc uống", "Cách dùng:" render as isolated bold short lines in the
|
||||
PDF) but is discarded before `chunk/` ever sees it.
|
||||
3. **Tables inside the monograph range are not addressed at all.** ADR 0004
|
||||
implicitly assumed monograph-range sections are prose. `docs/pdf-parsing-
|
||||
outlier-catalog.md` item 19 already documents a real table (dosing by
|
||||
renal function, HSV/CMV columns) inside a monograph body (Foscarnet
|
||||
natri, physical page 698) — flattening a table's rows into
|
||||
newline-joined body text destroys its row/column structure exactly the
|
||||
way outlier item 7 already describes for the appendix's 2D nomogram
|
||||
table. A whole-range survey to size this properly is in progress
|
||||
alongside this ADR (see "Not yet resolved" below).
|
||||
4. **Provenance is section-level, not chunk-level**, because per-line
|
||||
`physical_page`/`y0` (which `Span` already carries — see
|
||||
`extract/models.py`) is discarded at the same flattening point. For a
|
||||
section spanning several physical pages, a sub-chunk built from its
|
||||
final third currently has no way to know its own real page — it can
|
||||
only inherit the whole monograph's `source_page_range`. For medical
|
||||
citations this is not precise enough.
|
||||
|
||||
**A fifth, independently-found data-quality bug makes precise provenance
|
||||
even more necessary, not less**: the corpus's last-processed monograph
|
||||
(ZOLPIDEM) is never closed until true end-of-stream, and `assembler._classify`
|
||||
calls `match_section()`/`match_section_with_inline_value()` on every span
|
||||
with **no `in_monograph_range` gate** (unlike `_TextEvent` handling, which
|
||||
does check it). A spurious bold-text match on physical page 1655 — deep in
|
||||
the back-of-book "Mục lục tra cứu" brand-name index, confirmed by reading
|
||||
that page directly — overwrote ZOLPIDEM's real `tương tác thuốc`
|
||||
`SectionSpan` with an empty one and corrupted its `source_page_range` to
|
||||
`[1492, 1655]`. This is real content loss (measured: exactly 1 monograph
|
||||
affected, the last one processed — every other monograph is closed on
|
||||
schedule by the next monograph title, which *is* range-gated). Flagged for
|
||||
the session that owns `extract`/`segment`, not fixed here.
|
||||
|
||||
## Decision
|
||||
|
||||
Extend `segment/models.py`'s `SectionSpan` with a structured, line-level
|
||||
representation, additive to (not replacing) the existing flat `text` field
|
||||
— `chunk/` becomes a real, structure-aware consumer instead of re-deriving
|
||||
structure from a flattened string via ad hoc regex.
|
||||
|
||||
### New/changed types (`segment/models.py`)
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class BodyLine:
|
||||
text: str
|
||||
physical_page: int
|
||||
y0: float
|
||||
bold: bool # Span.bold, preserved instead of discarded
|
||||
|
||||
@dataclass
|
||||
class SectionSpan:
|
||||
key: str
|
||||
display_name: str
|
||||
heading: Heading
|
||||
text: str # UNCHANGED meaning, kept for
|
||||
# backward compat (see invariant below)
|
||||
lines: List[BodyLine] = field(default_factory=list) # NEW
|
||||
```
|
||||
|
||||
`lines` carries exactly the per-line signal `chunk/` needs to do its own
|
||||
job (population/subheading detection, precise page provenance) without
|
||||
`segment/` having to know anything about chunking — `segment/`'s
|
||||
responsibility stays "detect boundaries and preserve source structure," not
|
||||
"decide what a retrieval unit is" (Clean Architecture / SoC, per
|
||||
CLAUDE.md). Specifically, this is deliberately **not** a `is_subheading:
|
||||
bool` field computed by `segment/` — classifying "is this line a
|
||||
subheading a chunker should split on" is a chunking-time decision (what
|
||||
counts as a good split point can vary by strategy/eval results), not a
|
||||
segmentation-time one. `segment/` should stop discarding the raw signal
|
||||
(`bold`, `y0`, `physical_page`) it already has per span; it should not also
|
||||
start doing chunk-shaping judgment calls.
|
||||
|
||||
### Invariants
|
||||
|
||||
1. `text == "\n".join(l.text for l in lines).strip()` for every
|
||||
`SectionSpan`, for the lifetime of this contract — `lines` is a strictly
|
||||
additive refinement, never a divergent second source of truth. Any
|
||||
change to how body text is assembled (e.g. the boilerplate-stripping fix
|
||||
already applied by the other session) must update both fields from the
|
||||
same filtered span list, not `text` alone.
|
||||
2. `lines` is in reading order, matching the order `text`'s lines already
|
||||
implicitly have.
|
||||
3. Every `BodyLine.physical_page` satisfies `detector.in_monograph_range`
|
||||
for a `Span` on that page — i.e., **no line in any `SectionSpan.lines`
|
||||
may come from outside the monograph's real printed-page range**. This is
|
||||
the ZOLPIDEM bug's exact failure mode stated as an invariant: it was
|
||||
violated (a spurious section event was accepted from a fully
|
||||
out-of-range page precisely because no such check existed for section
|
||||
*events*, only for body *text* events). Enforcing this invariant closes
|
||||
that bug as a side effect, but the invariant is stated here as a
|
||||
contract requirement independent of any specific fix implementation.
|
||||
4. Every currently-open monograph must be finalized exactly once, at either
|
||||
(a) the next monograph title, or (b) true end-of-stream — with no third
|
||||
path (e.g., a stray out-of-range section match) able to silently mutate
|
||||
an already-"complete" monograph's sections after point (a) would
|
||||
otherwise have applied. (This is a restatement of invariant 3 from the
|
||||
monograph-lifecycle side, not a new requirement.)
|
||||
|
||||
### Migration impact
|
||||
|
||||
- **`segment/io.py`** (`_monograph_to_dict`/`_monograph_from_dict`,
|
||||
`write_monographs_jsonl`/`read_monographs_jsonl`): additive — serialize
|
||||
`lines` alongside the existing `text`/`heading` fields per section.
|
||||
Existing consumers reading only `text` (e.g. `segment/atc.py`'s
|
||||
`extract_atc_codes`, which regexes over `SectionSpan.text`) need no
|
||||
change, per invariant 1.
|
||||
- **`ingestion/data/processed/monographs.jsonl`**: schema grows a new
|
||||
optional-shaped field (`sections[key].lines`). No `schema_version` field
|
||||
currently exists in the serialized dict (checked `io.py` directly) —
|
||||
worth adding as part of this change, both for this migration and because
|
||||
`docs/architecture.md` already assumes "collection aliasing allows
|
||||
re-ingesting with a changed chunking strategy," which implies the
|
||||
ingestion output itself should be able to declare which schema shape it
|
||||
is.
|
||||
- **Existing 110 tests**: unaffected if invariant 1 holds — no assertion in
|
||||
the current suite inspects `lines` (it doesn't exist yet), and `text`'s
|
||||
value/semantics are unchanged.
|
||||
- **New tests required** (this ADR specifies them; implementation and the
|
||||
actual test code are not part of this ADR):
|
||||
1. Regression test reproducing the ZOLPIDEM failure shape: a synthetic
|
||||
span stream — last monograph's title and real sections, followed by
|
||||
spans whose `printed_page` is out of `in_monograph_range` but whose
|
||||
text matches a `vocab.py` section label — asserting the monograph
|
||||
closes with its real sections intact and the out-of-range spurious
|
||||
match is ignored, not accepted.
|
||||
2. `SectionSpan.lines` fixture test: using the real MORPHIN SULFAT
|
||||
boilerplate-fix fixture already in `tests/test_segment_assembler.py`,
|
||||
assert `lines` preserves the correct `bold`/`physical_page`/`y0` per
|
||||
retained line (and that stripped boilerplate lines are absent from
|
||||
`lines` too, not just from `text`).
|
||||
3. Round-trip test: `write_monographs_jsonl` → `read_monographs_jsonl`
|
||||
preserves `lines` exactly (dataclass equality per line).
|
||||
4. Whole-corpus invariant-1 check: for a real `cli run` output, assert
|
||||
`text == "\n".join(l.text for l in lines).strip()` holds for every
|
||||
section of every monograph, not a sample.
|
||||
|
||||
## Relationship to ADR 0004
|
||||
|
||||
ADR 0004's chunk-unit decision (`(drug_id, section_key)`) is **not**
|
||||
discarded — a section is still the natural *parent* grouping (matches how a
|
||||
clinician thinks, matches `Monograph.sections`). What changes: ADR 0004
|
||||
described a section as directly *the* chunk when under the 800-token
|
||||
ceiling, with sentence-window splitting as the fallback for oversized
|
||||
sections. Per the review above, splitting must instead **first** attempt to
|
||||
break at real structural boundaries available in `SectionSpan.lines` (a
|
||||
bold, short, isolated line — the same "subheading" shape already visually
|
||||
confirmed for route-of-administration/population sub-headers — or an
|
||||
explicit population/organ-function marker), with the sentence-window method
|
||||
demoted to a fallback for the remaining prose that has no such marker. The
|
||||
exact splitting algorithm (how a "subheading-shaped line" is defined
|
||||
precisely, in code) is a `chunk/`-side implementation detail *enabled* by
|
||||
this contract, not decided by it.
|
||||
|
||||
## Not yet resolved (explicitly out of scope for this ADR)
|
||||
|
||||
- **Table/formula content blocks.** A separate whole-monograph-range survey
|
||||
(pdfplumber `find_tables()` + PyMuPDF math-symbol scan, physical pages
|
||||
98-1494 excluding blank page 99 — the exact set `detector.
|
||||
in_monograph_range` accepts, not an assumed offset) is in progress at the
|
||||
time of writing, per explicit user instruction to measure before deciding
|
||||
a table/formula chunk-unit strategy. This ADR's `BodyLine`
|
||||
contract covers **text content only**; a table/formula region should
|
||||
*not* currently be flattened into `BodyLine`s (doing so would repeat
|
||||
exactly the "destroys row/column meaning" mistake outlier item 7 already
|
||||
documents) — but the precise `ContentBlock`/table-row/formula-unit shape
|
||||
is deferred to a follow-up revision of this ADR once the survey reports
|
||||
real numbers (how many monographs/sections affected, page-break
|
||||
continuation frequency, multi-tier headers, merged cells, footnotes).
|
||||
- **Paragraph-boundary detection** (grouping consecutive `BodyLine`s into a
|
||||
flowing paragraph vs. a new one) is left to `chunk/`, using the same
|
||||
kind of y-gap heuristic `segment/merge.py` already validates for
|
||||
multi-line title wraps (`_MAX_LINE_GAP_PT`) — `BodyLine.y0` is sufficient
|
||||
raw signal for `chunk/` to compute this itself; `segment/` does not need
|
||||
to pre-compute paragraph grouping.
|
||||
- **The actual `chunk/` splitting implementation** (subheading detector,
|
||||
population-marker regex, sentence-window fallback) is not part of this
|
||||
ADR — this ADR defines the data contract that implementation will consume.
|
||||
@@ -0,0 +1,167 @@
|
||||
# ADR 0006: chunks must carry references to lifted table/formula blocks
|
||||
|
||||
## Status
|
||||
|
||||
Accepted and implemented in schema v4. Resolves the item ADR
|
||||
0005 explicitly deferred ("Table/formula content blocks … the precise
|
||||
`ContentBlock`/table-row/formula-unit shape is deferred to a follow-up
|
||||
revision of this ADR once the survey reports real numbers"). The survey has
|
||||
reported.
|
||||
|
||||
## Context
|
||||
|
||||
`segment/` now lifts table and formula regions out of section prose and
|
||||
quarantines them (ADR 0003 lineage, outlier-catalog items 7, 8, 24, 25).
|
||||
That was the right move — linearised, AMPICILIN VÀ SULBACTAM's
|
||||
Cockcroft-Gault fraction read as `Clcr (ml/phút) = 72 x creatinin huyết
|
||||
thanh`, i.e. a division presented as a multiplication, in a renal-dosing
|
||||
section.
|
||||
|
||||
But `chunk/models.py` has no field that refers to a lifted block. Measured on
|
||||
the current whole-corpus output:
|
||||
|
||||
| quantity | value |
|
||||
|---|---|
|
||||
| lifted blocks represented by descriptor chunks | 151, all quarantined |
|
||||
| sections affected | 103 |
|
||||
| **blocks in `lieu_luong_va_cach_dung`** | **125** |
|
||||
| unverified header rows admitted to embedding text | **0** |
|
||||
|
||||
So three quarters of everything removed from prose was removed from the
|
||||
dosing section, in a drug formulary, for an audience of doctors and
|
||||
pharmacists.
|
||||
|
||||
**The failure this creates is silent, not visible.** A chunk of AMPICILIN VÀ
|
||||
SULBACTAM's `lieu_luong_va_cach_dung` is grammatical, complete-looking prose
|
||||
with the renal-dosing table absent and nothing marking the absence. Retrieval
|
||||
ranks it, the model answers from it, and neither has any way to know a table
|
||||
was taken out. A visible error would be safer than this.
|
||||
|
||||
A second, quieter failure: a table is currently **unreachable**. Nothing in
|
||||
the index represents it, so "bảng liều theo chức năng thận của ampicilin"
|
||||
cannot retrieve it even in principle.
|
||||
|
||||
## Decision
|
||||
|
||||
Chunks reference blocks; blocks' content never becomes embedded text.
|
||||
|
||||
### 1. `Chunk` gains typed attachments
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class ChunkAttachment:
|
||||
block_id: str
|
||||
kind: str # "table" | "formula"
|
||||
shape: str # simple_table | multi_level_or_merged_header |
|
||||
# cross_page_continuation | formula_2d
|
||||
physical_page: int
|
||||
printed_page: int
|
||||
bbox: List[float]
|
||||
quarantined: bool
|
||||
header_row: List[str] = () # always empty until separately verified
|
||||
source_crop: str | None = None
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Chunk:
|
||||
...
|
||||
chunk_kind: str = "prose" # "prose" | "block_descriptor"
|
||||
attachments: List[ChunkAttachment] = ()
|
||||
has_quarantined_content: bool = False
|
||||
```
|
||||
|
||||
`has_quarantined_content` is derivable from `attachments`, and is serialized
|
||||
anyway. A consumer that never looks at `attachments` must still be unable to
|
||||
miss the fact — the whole defect being fixed here is a consumer not knowing
|
||||
what it was not told.
|
||||
|
||||
### 2. One descriptor chunk per block, built from metadata only
|
||||
|
||||
A block also gets its own chunk so it is retrievable at all:
|
||||
|
||||
```
|
||||
chunk_id = "{drug_id}:{section_key}:block:{block_id}"
|
||||
chunk_kind = "block_descriptor"
|
||||
text = "AMPICILIN VÀ SULBACTAM — Liều lượng và cách dùng — bảng,
|
||||
trang in 204."
|
||||
```
|
||||
|
||||
The text is assembled only from verified metadata: drug name, section display
|
||||
name, block kind and printed page. **No cell value or inferred header appears.**
|
||||
The earlier proposal to use `pdfplumber.find_tables()`'s first row was rejected
|
||||
after corpus audit: a guessed first row can be a body row or can merge numeric
|
||||
relationships. Until a separate human-verified header dataset exists,
|
||||
`header_row` is embargoed for every shape and serialized as empty.
|
||||
|
||||
### 3. The answer layer's obligations (binding on `ai-service`)
|
||||
|
||||
These obligations are implemented across `ingestion/` and `ai-service` and are
|
||||
enforced by tests/readiness gates.
|
||||
|
||||
1. A retrieved chunk with `has_quarantined_content: true` **must** cause the
|
||||
answer to state that a table or formula exists at the cited page, and to
|
||||
surface its rendered crop. The answer may not present itself as complete.
|
||||
2. A `block_descriptor` chunk may be answered **only** with the crop. It must
|
||||
never be paraphrased, and its `header_row` must never be presented as the
|
||||
table's content.
|
||||
3. No chunk carrying a quarantined attachment may be used to state a numeric
|
||||
dose. If the dose is in the table, the answer is the crop plus the page.
|
||||
|
||||
### 4. `schema_version`
|
||||
|
||||
`monographs.jsonl` and the chunk output both gain `schema_version`. ADR 0005
|
||||
flagged its absence; a schema that now has two chunk kinds and an attachment
|
||||
list cannot be safely consumed without one.
|
||||
|
||||
## Alternatives rejected
|
||||
|
||||
- **Flatten the block into the chunk text.** This is the defect, not the fix
|
||||
— it reproduces `Clcr = 72 x creatinin` exactly.
|
||||
- **Chunk the block's linearised text as an ordinary chunk.** Worse than
|
||||
flattening: it makes unsafe text independently retrievable *as prose*, with
|
||||
its quarantine flag one dereference away from being ignored.
|
||||
- **Drop the blocks.** Silent loss, and contrary to the standing rule that
|
||||
unreconstructable content is quarantined with full provenance, never
|
||||
deleted.
|
||||
- **Rely on the prose saying "xem bảng".** The prose often does not, and a
|
||||
retrieval layer cannot act on an unstructured hint.
|
||||
- **Wait for row/column reconstruction and do this once.** Reconstruction is
|
||||
days of work and would leave the corpus unchunkable meanwhile; worse, it
|
||||
would make the schema question look answered when the *silent-incompleteness*
|
||||
problem is independent of whether the rows are recovered. Reconstruction
|
||||
later populates `rows` on the same attachment without touching consumers.
|
||||
|
||||
## Why a crop is a legitimate answer, not a placeholder
|
||||
|
||||
For doctors and pharmacists a rendered crop of the source page is the
|
||||
highest-fidelity response available: it *is* the book, and it is verifiable at
|
||||
a glance. Reconstruction earns its keep for a different job — comparing or
|
||||
combining values across drugs, which is the synthesis use case this product
|
||||
exists for — not for single-table lookup.
|
||||
|
||||
## Invariants and gates
|
||||
|
||||
Added to `cli chunk-ready` and to the chunk stage's own tests:
|
||||
|
||||
1. `section_with_lifted_block_but_no_chunk_reference = 0`
|
||||
2. `attachment_block_id_unknown = 0` — every referenced id exists on the
|
||||
monograph
|
||||
3. `attachment_without_page_or_bbox = 0`
|
||||
4. `block_text_leaked_into_chunk_text = 0` — no chunk's embedded text
|
||||
contains a quarantined block's text
|
||||
5. `descriptor_chunk_count == block_count`
|
||||
6. `descriptor_chunk_without_attachment = 0`
|
||||
7. `attachment_header_row_present = 0`
|
||||
8. `descriptor_with_unverified_columns = 0`
|
||||
9. `descriptor_range_not_attachment_page = 0`
|
||||
10. `attachment_without_printed_page = 0`
|
||||
|
||||
## Consequences
|
||||
|
||||
- Prose chunks shrink slightly in trustworthiness terms but grow in honesty:
|
||||
the ones missing a table now say so.
|
||||
- The current candidate index gains 151 descriptor chunks,
|
||||
each cheap and none carrying unsafe text.
|
||||
- `ai-service` cannot answer a dosing question from prose alone for the 103
|
||||
affected sections without violating a stated contract.
|
||||
- The 14 `formula_2d` attachments make the two Cockcroft-Gault formulas
|
||||
answerable as crops today, which they are not now.
|
||||
@@ -0,0 +1,210 @@
|
||||
# ADR 0007: Conversational reasoning RAG — state, bounded loop, and how it is measured
|
||||
|
||||
**Status:** superseded by ADR 0008 (2026-08-07). See the note below before
|
||||
reading this as a description of anything currently running.
|
||||
**Supersedes:** nothing. Extends ADR 0005 (segment output contract) and ADR 0006
|
||||
(quarantined block references) rather than replacing them.
|
||||
|
||||
> **2026-08-07 — why this was superseded, not deleted.** An independent
|
||||
> 7-agent audit on 2026-08-06 found `bootstrap.py` never constructs any of
|
||||
> `rag/conversation.py` / `rag/reasoning.py` / `rag/conversational.py` — the
|
||||
> live agent (`rag/agent.py::RagAgent`, wired in since the F-03 rebuild on
|
||||
> 2026-08-06) is a fixed one-shot pipeline (understand → route → retrieve
|
||||
> once → generate → ≤2 same-claim entailment retries), not the PLAN/RETRIEVE/
|
||||
> ASSESS/REFINE/VERIFY loop or the `Focus`/`ConversationState`/TTL state
|
||||
> design below. This was a real, deliberate pivot mid-implementation, not an
|
||||
> abandoned-but-still-intended plan: `rag/agent.py`'s own module docstring
|
||||
> says outright that `ConversationalLoopService` + `conversation.py` were
|
||||
> replaced because "the LLM reads a plain turn history and resolves
|
||||
> ['thuốc đó' / 'còn liều thì sao'] itself" — simpler than maintaining
|
||||
> `Focus`/TTL/turn-budget state by hand, and proven live across many
|
||||
> multi-turn conversations since. Section 6 below ("Refused: an LLM
|
||||
> confidence score as the loop's uncertainty signal") is the clearest
|
||||
> evidence this is a genuine architecture change, not a gap: the live system
|
||||
> now uses exactly that — an LLM sufficiency/clarify judgment — as its
|
||||
> ask-or-answer signal, the opposite of what this ADR chose.
|
||||
>
|
||||
> The three modules this ADR specified (1,314 lines) and their five dedicated
|
||||
> test files (42 tests) were deleted on 2026-08-07 rather than left as dead
|
||||
> code, once confirmed to have zero live importers anywhere
|
||||
> (`bootstrap.py`/`main.py`/`agent.py`/`answer.py`/`routers/rag.py`). This
|
||||
> document is kept, unedited below this notice, as the historical record of
|
||||
> why that design was chosen and what it traded off — see ADR 0008 for what
|
||||
> actually runs today, including what this ADR got right that ADR 0008
|
||||
> still owes (a real request-scoped time/call budget — F-08, still open; a
|
||||
> durable, cross-worker conversation store — currently an in-process dict).
|
||||
|
||||
## Context
|
||||
|
||||
The service answers one question at a time. `POST /v1/rag/query` carries no
|
||||
conversation id, `apps/chat-service` holds zero source files, and every request
|
||||
re-resolves the drug from scratch. Three consequences, all observed in the UI on
|
||||
2026-08-05:
|
||||
|
||||
- `paracetamol` alone is refused rather than asked about.
|
||||
- `liều dùng paracetamol cho người lớn` returns the identical answer to
|
||||
`liều dùng paracetamol` — the qualifier is not used at any stage.
|
||||
- A follow-up such as *"còn trẻ em thì sao?"* cannot work at all, because
|
||||
nothing carries the drug forward.
|
||||
|
||||
The owner's requirement is a **conversational reasoning RAG**: history, an
|
||||
internal reasoning stage, and a bounded self-improvement loop.
|
||||
|
||||
The binding constraint is that this is a drug formulary for clinicians. Every
|
||||
capability below is designed so that adding it cannot widen what the system is
|
||||
allowed to assert.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Conversation state
|
||||
|
||||
Two stores with different jobs, deliberately not merged.
|
||||
|
||||
**`Focus` — structured, drives routing.** This is what makes *"còn trẻ em thì
|
||||
sao?"* resolvable without an LLM.
|
||||
|
||||
| Field | Purpose |
|
||||
|---|---|
|
||||
| `drug_id`, `drug_name` | The drug under discussion |
|
||||
| `section_key` | The attribute last answered |
|
||||
| `population` | `nguoi_lon` / `tre_em` / `phu_nu_co_thai` / … |
|
||||
| `verbosity` | `concise` \| `detailed`, set when the user asks |
|
||||
| `set_at_turn` | Turn index each field was last set |
|
||||
|
||||
**`ConversationState` — the whole record.**
|
||||
|
||||
```
|
||||
conversation_id
|
||||
recent: tuple[Turn, ...] # last K turns, verbatim
|
||||
summary: str # rolling prose summary of everything older
|
||||
focus: Focus
|
||||
turn_count: int
|
||||
```
|
||||
|
||||
A `Turn` carries `role`, `text`, `at`, and — for assistant turns — the
|
||||
`drug_id`, `section_key` and `evidence_ids` that produced it. Storing the
|
||||
evidence ids is what lets the planner answer a follow-up **from evidence
|
||||
already retrieved** instead of retrieving again.
|
||||
|
||||
**Carry-over is never silent.** An inherited `drug_id` that is wrong is a
|
||||
wrong-drug answer, so any answer built on inherited focus must name what it
|
||||
inherited: *"Về Metformin, ở trẻ em: …"*. This is a hard rule, not a
|
||||
presentation preference.
|
||||
|
||||
**Focus expires.** A field older than `FOCUS_TTL_TURNS` (6) is dropped rather
|
||||
than inherited. Conversations drift, and a drug from ten turns ago is not
|
||||
context, it is a hazard.
|
||||
|
||||
### 2. Recent history and summary
|
||||
|
||||
- `recent` holds the last **K = 6** turns verbatim (three exchanges).
|
||||
- When a turn falls out of `recent`, it is folded into `summary`.
|
||||
- `summary` is regenerated at most every **S = 4** turns, capped at **400
|
||||
tokens**; `recent` is capped at **2000 tokens**, oldest dropped first.
|
||||
- **The summary records what was discussed, never clinical content.** It may
|
||||
say *"đã hỏi liều dùng của Metformin cho người lớn"*; it may not carry a dose.
|
||||
A dose restated from a summary would have no citation and could not be
|
||||
grounding-verified — the check compares against retrieved evidence, and a
|
||||
summary is not evidence.
|
||||
|
||||
### 3. Reasoning loop
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A[User turn] --> B[UNDERSTAND<br/>resolve against Focus]
|
||||
B --> C{Clarify signal?}
|
||||
C -->|ambiguous drug / no attribute /<br/>multi-attribute| Z[ASK — 1 turn, no loop]
|
||||
C -->|no| D{Simple?}
|
||||
D -->|drug + section resolved,<br/>no follow-up ambiguity| E[RETRIEVE]
|
||||
D -->|complex / decomposable| P[PLAN<br/>sub-questions + retrieval set]
|
||||
P --> E
|
||||
E --> F[ASSESS sufficiency]
|
||||
F -->|insufficient AND rounds left| R[REFINE query] --> E
|
||||
F -->|sufficient OR rounds exhausted| G[GENERATE]
|
||||
G --> H[VERIFY<br/>grounding + coverage]
|
||||
H -->|ungrounded / off-target,<br/>repairs left| G
|
||||
H -->|grounded| Y[RESPOND]
|
||||
H -->|repairs exhausted| X[FALL BACK<br/>verbatim source]
|
||||
F -->|exhausted AND still thin| Z
|
||||
```
|
||||
|
||||
**Continue conditions** — a round is spent only when all hold:
|
||||
1. `retrieval_rounds < MAX_RETRIEVAL_ROUNDS` (2)
|
||||
2. the assessor named a *specific* missing thing (a section, a population, a
|
||||
second drug) — "feels incomplete" is not a reason to spend a round
|
||||
3. the refined query differs from every query already tried this turn
|
||||
|
||||
**Stop conditions** — any one ends the loop:
|
||||
- sufficiency satisfied
|
||||
- budget exhausted (rounds, LLM calls, wall-clock, tokens)
|
||||
- a clarify signal fires (these bypass the loop entirely — asking beats guessing)
|
||||
- grounding verification fails after `MAX_REPAIRS` (1) → extractive fallback
|
||||
|
||||
**Fast path.** When the drug resolves and `SectionResolver` returns a section
|
||||
and no clarify signal fires, the loop is skipped: retrieve → generate → verify.
|
||||
This is the majority path and it costs one LLM call.
|
||||
|
||||
### 4. Budgets
|
||||
|
||||
| Limit | Value | Enforced at |
|
||||
|---|---|---|
|
||||
| `MAX_RETRIEVAL_ROUNDS` | 2 | loop guard |
|
||||
| `MAX_REPAIRS` | 1 | loop guard |
|
||||
| `MAX_LLM_CALLS` per turn | 4 | budget object, checked before each call |
|
||||
| `MAX_WALL_CLOCK_MS` | 20000 | checked between stages |
|
||||
| `MAX_EVIDENCE_TOKENS` | 12000 | evidence assembly, oldest-dropped |
|
||||
| `FOCUS_TTL_TURNS` | 6 | state update |
|
||||
|
||||
The budget is a single object threaded through the loop and **decremented
|
||||
before** each call, so exhaustion degrades to the best answer so far rather
|
||||
than to an error.
|
||||
|
||||
### 5. Integration
|
||||
|
||||
New domain modules, no SDK imports:
|
||||
|
||||
- `rag/conversation.py` — `Focus`, `Turn`, `ConversationState`, window and
|
||||
focus-update rules. Pure; the follow-up resolution in it needs no LLM.
|
||||
- `rag/reasoning.py` — the loop, its budget, and its stage protocols.
|
||||
- `rag/ports.py` — `ConversationStore` (load/save), `Summariser`, `Planner`,
|
||||
`SufficiencyAssessor`. Each has a deterministic no-LLM default so the whole
|
||||
loop runs offline.
|
||||
|
||||
New adapter: `adapters/postgres.py` gains `PostgresConversationStore`.
|
||||
|
||||
Unchanged and still binding: `GroundedAnswerService` remains the single-turn
|
||||
engine; `grounding.verify` gates every generated answer; `VERIFY_PDF` evidence
|
||||
is never generated over.
|
||||
|
||||
### 6. Measurement
|
||||
|
||||
A capability that cannot be shown to help does not ship. Three modes are run
|
||||
over the same cases — `single-shot`, `+history`, `+reasoning-loop`:
|
||||
|
||||
| Metric | Answers |
|
||||
|---|---|
|
||||
| follow-up resolution accuracy | does *"còn trẻ em thì sao?"* reach the right drug+section+population |
|
||||
| on-target rate | does the answer contain the population/attribute actually asked for |
|
||||
| grounding rejection rate | does reasoning make fabrication more or less likely |
|
||||
| clarify rate / clarify precision | does it ask when it should, and only then |
|
||||
| median + p95 latency, LLM calls, tokens per answered turn | what the capability costs |
|
||||
|
||||
The evaluation set is a **new multi-turn golden file** — the existing
|
||||
`golden_e2e_v1.csv` is single-turn by construction and cannot measure any of
|
||||
this. Counters land in `rag/metrics.py` and on the existing Grafana dashboard.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Accepted.** More moving parts and more tokens per turn; a stateful service
|
||||
where there was a stateless one; a summary that must be kept free of clinical
|
||||
content by rule rather than by mechanism.
|
||||
|
||||
**Refused.** An LLM confidence score as the loop's uncertainty signal. The
|
||||
signals used are the resolver states that already exist — ambiguous drug,
|
||||
unresolved section, multi-attribute question — because they are deterministic,
|
||||
testable, and explainable to a reviewer. "The model felt 0.73 sure" is not a
|
||||
defensible basis for asking or not asking a clinician a question.
|
||||
|
||||
**Unchanged.** Nothing here lets the system assert a figure absent from the
|
||||
retrieved source. Reasoning chooses *what to look up and how to say it*; it is
|
||||
not a source of facts.
|
||||
@@ -0,0 +1,153 @@
|
||||
# ADR 0008: LLM query understanding + one-shot grounded RAG (what is actually live)
|
||||
|
||||
**Status:** accepted, live since 2026-08-06 (F-03), extended 2026-08-07
|
||||
**Supersedes:** ADR 0007 (conversational reasoning RAG — the `Focus`/
|
||||
`ConversationState`/TTL state design and the PLAN/RETRIEVE/ASSESS/REFINE/
|
||||
VERIFY bounded loop). ADR 0007's own `rag/conversation.py`/`rag/reasoning.py`/
|
||||
`rag/conversational.py` were deleted 2026-08-07 once confirmed unreachable
|
||||
from `bootstrap.py` — see the notice at the top of ADR 0007 for the full
|
||||
reasoning.
|
||||
**Extends:** ADR 0006 (quarantined block references) — unchanged and still
|
||||
binding: a chunk with `has_quarantined_content` still forces `VERIFY_PDF`
|
||||
and is never generated over.
|
||||
|
||||
## Context
|
||||
|
||||
This ADR exists because `docs/architecture.md` and ADR 0007 described a
|
||||
design that was never fully built, and the modules that partially
|
||||
implemented it were never wired into `bootstrap.py`. A 2026-08-06
|
||||
independent 7-agent audit found this the hard way — it cost real time
|
||||
establishing that `QdrantRetriever.search()` (dense vector search) and the
|
||||
entire reasoning-loop module set were dead code, contradicting what the
|
||||
docs claimed was live. The fix is not "finish building ADR 0007" — the
|
||||
project deliberately moved to a simpler design that already works, proven
|
||||
across many real multi-turn conversations (see `docs/progress-log.md`,
|
||||
2026-08-05 through 2026-08-07 entries). This ADR documents that design so
|
||||
the next reader doesn't have to re-discover it by audit.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. One LLM call understands the turn; no separate state object
|
||||
|
||||
`rag/understanding.py::LlmQueryUnderstander.understand(turn, history)` reads
|
||||
the raw current turn plus a **plain list of past turn strings**
|
||||
(`"Người dùng: …"` / `"Trợ lý: …"`, kept by `RagAgent._history`, a
|
||||
per-conversation-id in-process dict) and returns a `QueryFrame`: turn type,
|
||||
resolved `drug_id`s (validated against a candidate set a deterministic
|
||||
fuzzy/alias pass bounds *before* the model runs — F-04), section attribute,
|
||||
population, weight, age, indication, route, and a `needs_clarify`/
|
||||
`clarify_reason`/`quick_replies` triple.
|
||||
|
||||
There is no `Focus` struct, no TTL, no separate summariser. The model
|
||||
re-reads the same history window (last `HISTORY_TURNS * 2` = 12 lines) every
|
||||
turn and re-derives what's still relevant — cheaper to build and, so far,
|
||||
more robust than hand-maintained state: it naturally handles "còn trẻ em thì
|
||||
sao?" and short replies to its own clarify questions (population/route/etc.
|
||||
— the latter only after a 2026-08-07 fix; see progress-log) without a
|
||||
resolver state machine to keep in sync.
|
||||
|
||||
**Known gap, inherited from ADR 0007 and still open:** this history is an
|
||||
in-process dict — lost on restart, not shared across workers if the service
|
||||
ever scales beyond one. ADR 0007's `PostgresConversationStore` was never
|
||||
built either.
|
||||
|
||||
### 2. Routing is a single dispatch, not a loop
|
||||
|
||||
`RagAgent._route()` reads `frame.turn_type` and dispatches once:
|
||||
`interaction` (2+ drugs) → gather each drug's evidence, combine, decide;
|
||||
`drug_attribute`/`drug_overview`/`dosing_calc`/fallback → one drug, one
|
||||
retrieval call; `smalltalk`/`out_of_scope` → canned reply, no retrieval;
|
||||
`symptom_to_drug` with no drug named → an honest "not built yet" clarify.
|
||||
There is no PLAN/REFINE step and no retrieval-round budget, because there is
|
||||
only ever one retrieval call per turn.
|
||||
|
||||
### 3. Retrieval is deterministic routing, not similarity ranking
|
||||
|
||||
`RetrievalService.retrieve_framed(drug_id, section_key, query)`:
|
||||
- `section_key` given (the dominant case, since `understand()` almost always
|
||||
resolves it) → `find_by_section`, an **exact Qdrant payload filter**
|
||||
(`drug_id` + `section_key`), returning the whole section as a scroll.
|
||||
Score is a hardcoded 1.0 — this is a filter, not a ranked search, and nothing
|
||||
here is "confidence" in the sense ADR 0007's retrieval-confidence gate meant.
|
||||
- No section resolved → `find_by_drug` (whole monograph, book order),
|
||||
trimmed to identity sections for a bare name or reranked (Cohere
|
||||
cross-encoder over the ~29 sections of that one drug, not a corpus search)
|
||||
for a free-form question.
|
||||
- `QdrantRetriever.search()` — real dense vector similarity over the whole
|
||||
corpus — exists and is unit-tested, but `RagAgent` never calls it. It is
|
||||
reachable only through the legacy `RetrievalService.retrieve()` entry
|
||||
point, itself only reachable when `ANSWER_PROVIDER=disabled` (no agent
|
||||
configured at all — retrieval-only mode). `docs/architecture.md`'s
|
||||
"Retrieval-confidence gate: below a similarity threshold, skip the LLM
|
||||
call entirely" describes this legacy-only path, not the live one; that
|
||||
section has been corrected to say so.
|
||||
- Measured, and the reason this design was chosen over similarity ranking
|
||||
for the live path: routing by exact `section_key` moved contraindication
|
||||
hit@1 from 0.05 to 1.00 (`[[project-retrieval-quality-gap]]`, 2026-08-04).
|
||||
A quarantined chunk anywhere in the retrieved set still forces the whole
|
||||
result to `VERIFY_PDF` (`RetrievalService.decide`, a public wrapper added
|
||||
2026-08-07 so `RagAgent._interaction` applies the same policy to a
|
||||
combined multi-drug evidence pool instead of hand-rolling it).
|
||||
|
||||
### 4. Generation is one call, verified twice, with no confidence score
|
||||
|
||||
`GroundedAnswerService.answer_from_result`: sufficiency-check (ask instead of
|
||||
guessing when the evidence spans multiple populations/routes and the turn
|
||||
hasn't disambiguated) → generate → `grounding.verify` (every number and
|
||||
citation traces to the block it cites) → `_verify_entailment` (a second LLM
|
||||
pass confirming each cited claim's *content*, not just its numbers, is
|
||||
actually stated by that block; one same-claim retry on a lone reject, since
|
||||
this call is measurably noisy — 2026-08-06 finding). A generation that fails
|
||||
any check **abstains** — it does not fall back to a raw extractive quote
|
||||
when a generator is configured (`[[feedback_no_extractive_fallback_when_llm_configured]]`).
|
||||
|
||||
No `MAX_LLM_CALLS`/`MAX_WALL_CLOCK_MS` budget object exists. Each call is
|
||||
bounded only by its own provider timeout. **This is ADR 0007's F-08 finding,
|
||||
inherited unchanged and still open** — a real end-to-end request deadline
|
||||
threaded through `RagAgent`'s sequence of up to 5 sequential Bedrock calls
|
||||
(understand → sufficiency → generate → ≤2 entailment) is real remaining
|
||||
work, not solved by this ADR. Measured live 2026-08-07: a single answerable
|
||||
turn costs ~8-9s wall clock, ~75-80% of it the 4 sequential LLM calls
|
||||
(understand ~2.6-3.3s dominates — an 80B model doing a classification task
|
||||
that likely doesn't need one); a clarify chain compounds this linearly since
|
||||
each round is a fresh request repeating the same call sequence from scratch.
|
||||
|
||||
### 5. Context resolved across turns is folded into one self-contained string
|
||||
|
||||
Added 2026-08-07, closing a P0 the 2026-08-06 audit named: `frame.population`/
|
||||
`weight_kg`/`age_text`/`route`/`indication` were extracted by `understand()`
|
||||
but never reached `retrieve_framed`/`answer_from_result`, which took only
|
||||
the bare current-turn text — so a reply like "Uống" three turns into a dose
|
||||
conversation reached the sufficiency/generation LLM calls as literally just
|
||||
"Uống", with no notion that population=adult was already established two
|
||||
turns back. `RagAgent._synthesize_query` now folds every resolved field into
|
||||
one string (`"Uống. Đối tượng: người lớn. Đường dùng: uống."`) before it
|
||||
reaches retrieval's rerank signal and generation's `query` argument. No-op
|
||||
for a fresh single-shot question that already states its own context.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Accepted.** No confidence score, no retrieval-round budget, no PLAN/REFINE
|
||||
step — the tradeoff ADR 0007 explicitly refused ("an LLM confidence score...
|
||||
is not a defensible basis for asking or not asking a clinician a question")
|
||||
is exactly what this design uses instead (an LLM sufficiency/clarify
|
||||
judgment), because in practice it has been reliable enough and dramatically
|
||||
simpler to build, extend (route/quick_replies were one schema field + one
|
||||
prompt rule each, not a new state machine), and debug — every session this
|
||||
month that touched the ADR 0007 modules found new bugs in the state-machine
|
||||
edges (TTL boundaries, Focus inheritance correctness) rather than in the
|
||||
domain logic itself.
|
||||
|
||||
**Refused (again, restated from ADR 0007, still true):** an LLM confidence
|
||||
score as a hard gate for retrieval — `RetrievalService.decide`'s
|
||||
`VERIFY_PDF`/`ABSTAIN` decisions remain deterministic (quarantine flag,
|
||||
missing provenance), never a model's self-reported certainty.
|
||||
|
||||
**Still open, named rather than hidden:**
|
||||
- No request-scoped time/call budget (F-08).
|
||||
- Conversation history is in-process, not durable/shared (inherited from
|
||||
ADR 0007, never built either way).
|
||||
- No production-path adversarial regression suite beyond one live-verified
|
||||
end-to-end case (F-10's remaining scope).
|
||||
- `dosing_calc` (a real mg/kg calculator) and `symptom_to_drug` (reverse
|
||||
indication lookup) remain honest "not ready" clarifies, not answers.
|
||||
@@ -0,0 +1,82 @@
|
||||
# ADR 0009: No RAG framework — hand-written orchestration behind ports
|
||||
|
||||
## Status
|
||||
|
||||
Accepted. **Recorded retrospectively** during the 2026-08-12 documentation pass:
|
||||
the decision is unambiguous in the implementation, but no ADR existed for it.
|
||||
|
||||
## Context
|
||||
|
||||
The system performs retrieval-augmented generation with query understanding,
|
||||
multiple retrieval strategies, reranking, prompt construction, structured output
|
||||
parsing, and post-generation verification — the exact feature set LangChain and
|
||||
LlamaIndex exist to provide.
|
||||
|
||||
## Decision
|
||||
|
||||
Neither framework is used. There is no RAG or agent library of any kind.
|
||||
|
||||
Verifiable from the repository:
|
||||
|
||||
- `apps/ai-service/pyproject.toml` declares six runtime dependencies:
|
||||
`fastapi`, `httpx`, `psycopg`, `pydantic-settings`, `qdrant-client`,
|
||||
`uvicorn`. Optional extras add `prometheus-client`, `anthropic` and three
|
||||
OpenTelemetry packages.
|
||||
- `apps/ai-service/Dockerfile` installs that set plus `boto3`.
|
||||
- No file imports `langchain`, `llama_index`, `haystack` or any equivalent.
|
||||
|
||||
Instead:
|
||||
|
||||
- Orchestration is a plain class with an explicit branch table
|
||||
(`rag/agent.py::_route`).
|
||||
- Prompts are module-level constants with JSON schemas (`rag/prompt.py`).
|
||||
- Providers are injected through `typing.Protocol`s (`rag/ports.py`) and
|
||||
implemented in `adapters/`, which is the only package importing an SDK — and
|
||||
always lazily, inside a method.
|
||||
- `bootstrap.py` is the single composition root.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Enabled by this choice**
|
||||
|
||||
- `rag/` imports no SDK, so the entire domain — including every safety check —
|
||||
is unit-testable offline with stub objects. All 278 ai-service tests run in
|
||||
2.6 s with no network.
|
||||
- Behaviour is inspectable: the retrieval route for a given turn is a readable
|
||||
`if` chain, not framework dispatch.
|
||||
- Failure semantics are chosen per call site. The fail-closed/fail-open
|
||||
asymmetry in [02-system-architecture.md](../02-system-architecture.md#failure-boundaries)
|
||||
is deliberate and would be hard to express through a framework's uniform
|
||||
error handling.
|
||||
- Prompt text is reviewable as domain policy in one file, and swapping providers
|
||||
cannot silently change what the model was told.
|
||||
|
||||
**Costs**
|
||||
|
||||
- Retrieval strategies, rank fusion, context packing and evaluation harnesses
|
||||
are all hand-written. Two of them (`fusion.py`, `expansion.py`) were written
|
||||
and never wired ([27-technical-debt.md](../27-technical-debt.md#d-12--dead-code-three-tested-modules-with-no-runtime-caller)).
|
||||
- Optional retriever capabilities are discovered with `getattr` rather than
|
||||
declared, so the real interface is wider than `ports.py` documents (D-14).
|
||||
- No community tooling for tracing, caching or evaluation applies; the
|
||||
observability layer is bespoke.
|
||||
|
||||
## Rationale
|
||||
|
||||
Partially recoverable. The code does not state "we chose not to use a
|
||||
framework", but the ports-and-adapters discipline is documented repeatedly in
|
||||
module docstrings, and one of them makes the intent explicit —
|
||||
`rag/understanding.py`:
|
||||
|
||||
> `rag/` imports no SDK: the LLM is injected as a `JsonLlm` protocol … and a
|
||||
> deterministic stub runs the whole path offline in tests.
|
||||
|
||||
`rag/prompt.py` gives the parallel reason for prompts:
|
||||
|
||||
> This is domain policy, not infrastructure … it lives here so it can be read,
|
||||
> reviewed and tested without an SDK, and so swapping the provider cannot
|
||||
> silently change what the model was told.
|
||||
|
||||
The consistent theme is testability and reviewability of the safety layer.
|
||||
Whether cost, lock-in or framework maturity also weighed in the decision is not
|
||||
recoverable from the repository.
|
||||
@@ -0,0 +1,85 @@
|
||||
# ADR 0010: Single-host Docker Compose as the interim deployment
|
||||
|
||||
## Status
|
||||
|
||||
Accepted. **Recorded retrospectively** during the 2026-08-12 documentation pass.
|
||||
Does **not** supersede [ADR 0002](0002-argocd-gitops.md), whose own status line
|
||||
says it remains the target:
|
||||
|
||||
> **Accepted — still the target, not yet implemented.** Not superseded by the
|
||||
> current production setup.
|
||||
|
||||
## Context
|
||||
|
||||
ADR 0002 chose GitOps on the team's ArgoCD instance. A complete Helm chart
|
||||
(`infra/helm/medical-chatbot/`) and three ArgoCD `Application` manifests exist.
|
||||
Neither has been applied: each `Application` carries three unresolved `TODO`s
|
||||
(project/RBAC scope, repo URL, target cluster), `infra/k8s/base|overlays/` hold
|
||||
only `.gitkeep`, and no image registry is configured anywhere.
|
||||
|
||||
Meanwhile the product is live at `https://realvuxbaro.me`.
|
||||
|
||||
## Decision
|
||||
|
||||
Run production as Docker Compose on a single EC2 host, with Caddy terminating
|
||||
TLS, and deploy by SSH from GitHub Actions.
|
||||
|
||||
Verifiable from the repository:
|
||||
|
||||
- `infra/docker/docker-compose.prod.yml` — postgres, qdrant, ai-service, web,
|
||||
caddy, with named volumes.
|
||||
- `infra/docker/docker-compose.observability.yml` — the OTel/Prometheus/Tempo/
|
||||
Grafana overlay, which also sets `OTEL_ENABLED=true`.
|
||||
- `infra/docker/Caddyfile` — `realvuxbaro.me` → `web:3000`, `/grafana/*` →
|
||||
`grafana:3000`.
|
||||
- `.github/workflows/deploy.yml` — `appleboy/ssh-action`, `git reset --hard`,
|
||||
`docker compose up -d --build`, `caddy reload`, `python -m migrate`, then ~18
|
||||
assertions.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Accepted trade-offs**
|
||||
|
||||
- Images are built on the production host and are untagged, so there is **no
|
||||
artifact to roll back to**; recovery is a revert commit plus a rebuild.
|
||||
- Deploys are in-place, with brief per-service downtime.
|
||||
- No horizontal scaling. That happens to align with the in-process agent state
|
||||
described in [02-system-architecture.md](../02-system-architecture.md#the-stateful-detail-that-constrains-scaling),
|
||||
but the alignment is coincidental, not enforced.
|
||||
- Configuration and secrets live in an uncommitted `.env.prod` on the host, so
|
||||
production configuration cannot be reviewed in Git.
|
||||
- `postgres` and `qdrant` are deliberately absent from the workflow's `up -d`
|
||||
list, so a code deploy never restarts the stateful services — and changes to
|
||||
their service definitions do not take effect until someone restarts them.
|
||||
|
||||
**Preserved despite the simpler runtime**
|
||||
|
||||
The deploy script asserts far more than a Compose deploy usually does: service
|
||||
health, a **real grounded answer** from the real corpus (`decision=answerable`
|
||||
with a `chi_dinh` citation), both Grafana datasources, the provisioned
|
||||
dashboard, public reachability of `/grafana/login`, and end-to-end trace
|
||||
propagation by asserting that a specific `X-Trace-ID` becomes retrievable from
|
||||
Tempo. That verification block is what makes the simpler runtime defensible.
|
||||
|
||||
**Migration path**
|
||||
|
||||
The Helm chart already maps every setting in `config.py` to a ConfigMap, mounts
|
||||
`POSTGRES_DSN` from a Secret, and configures readiness/liveness/startup probes
|
||||
against the same `/ready` and `/health` endpoints Compose uses. Moving to
|
||||
Kubernetes therefore needs: an image registry and tagging, a corpus-load or
|
||||
snapshot-restore step (the chart provisions an **empty** Qdrant, against which
|
||||
`ai-service`'s manifest check refuses to start), the three ArgoCD `TODO`s
|
||||
resolved, and the `bump-image-tag` workflow that
|
||||
`infra/ci/github-actions/README.md` describes but does not contain.
|
||||
|
||||
## Rationale
|
||||
|
||||
**Decision observed; rationale not fully recoverable from the repository.** The
|
||||
Compose header comment records one constraint —
|
||||
|
||||
> No GPU, no team k3s — Bedrock calls go out over the instance's IAM role … so
|
||||
> no AWS access keys live in this file or its env files.
|
||||
|
||||
— and ADR 0002 remaining un-superseded shows the Kubernetes target was not
|
||||
abandoned. Beyond that, whether the driver was cost, cluster access, or time to
|
||||
first deployment is not determinable from the code.
|
||||
@@ -0,0 +1,20 @@
|
||||
# Architecture decision records
|
||||
|
||||
| ADR | Title | Status | Reflected in code? |
|
||||
|---|---|---|---|
|
||||
| [0001](0001-vector-db-qdrant.md) | Use Qdrant as the vector database | Accepted | **Yes** — `adapters/qdrant.py`, `ingestion/load/qdrant_repo.py` |
|
||||
| [0002](0002-argocd-gitops.md) | Use the team's existing ArgoCD instance for deployment (GitOps) | Accepted — target, **not yet implemented** | **No** — production is Docker Compose on EC2 ([20](../20-deployment.md)) |
|
||||
| [0003](0003-pdf-parsing-strategy.md) | PDF parsing strategy, validated empirically | Accepted | **Yes** — `ingestion/extract/`, `ingestion/segment/detector.py` |
|
||||
| [0004](0004-chunking-strategy.md) | Chunking strategy for drug monographs | Accepted (monograph range only) | **Yes** — `ingestion/chunk/chunker.py` |
|
||||
| [0005](0005-segment-output-contract-for-chunking.md) | `segment/` output contract needed by `chunk/` | Proposed; header says "contract only, no implementation" | **Yes, now implemented** — `segment/models.py` + `chunk/` follow it. The status line is stale |
|
||||
| [0006](0006-quarantined-block-references-in-chunks.md) | Chunks must carry references to lifted table/formula blocks | Accepted, implemented in schema v4 | **Yes** — `ChunkAttachment`, `has_quarantined_content`, the ADR-0006 gate set |
|
||||
| [0007](0007-conversational-reasoning-rag.md) | Conversational reasoning RAG (state + bounded loop) | **Superseded by 0008** | **No** — `rag/conversation.py` and `rag/reasoning.py` no longer exist |
|
||||
| [0008](0008-llm-understanding-one-shot-rag.md) | LLM query understanding + one-shot grounded RAG | Accepted, live since 2026-08-06 | **Yes** — `rag/understanding.py`, `rag/agent.py`, `rag/answer.py` |
|
||||
| [0009](0009-no-rag-framework.md) | No RAG framework — hand-written orchestration behind ports | Accepted (recorded retrospectively) | **Yes** |
|
||||
| [0010](0010-interim-single-host-compose-deployment.md) | Single-host Docker Compose as the interim deployment | Accepted (recorded retrospectively) | **Yes** |
|
||||
|
||||
ADRs 0009 and 0010 were written during the documentation pass described in
|
||||
[DOCUMENTATION_PLAN.md](../DOCUMENTATION_PLAN.md). They record decisions that are
|
||||
unambiguously visible in the implementation but had no ADR. Where the rationale
|
||||
could not be recovered from the repository, they say so rather than inventing
|
||||
one.
|
||||
Reference in New Issue
Block a user