Fix live multi-turn: pass the resolved drug, stop did-you-mean garbage

docs/adr/0007-conversational-reasoning-rag.md

@@ -0,0 +1,180 @@
+# ADR 0007: Conversational reasoning RAG — state, bounded loop, and how it is measured
+
+**Status:** accepted, implementation in progress (2026-08-05)
+**Supersedes:** nothing. Extends ADR 0005 (segment output contract) and ADR 0006
+(quarantined block references) rather than replacing them.
+
+## Context
+
+The service answers one question at a time. `POST /v1/rag/query` carries no
+conversation id, `apps/chat-service` holds zero source files, and every request
+re-resolves the drug from scratch. Three consequences, all observed in the UI on
+2026-08-05:
+
+- `paracetamol` alone is refused rather than asked about.
+- `liều dùng paracetamol cho người lớn` returns the identical answer to
+  `liều dùng paracetamol` — the qualifier is not used at any stage.
+- A follow-up such as *"còn trẻ em thì sao?"* cannot work at all, because
+  nothing carries the drug forward.
+
+The owner's requirement is a **conversational reasoning RAG**: history, an
+internal reasoning stage, and a bounded self-improvement loop.
+
+The binding constraint is that this is a drug formulary for clinicians. Every
+capability below is designed so that adding it cannot widen what the system is
+allowed to assert.
+
+## Decision
+
+### 1. Conversation state
+
+Two stores with different jobs, deliberately not merged.
+
+**`Focus` — structured, drives routing.** This is what makes *"còn trẻ em thì
+sao?"* resolvable without an LLM.
+
+| Field | Purpose |
+|---|---|
+| `drug_id`, `drug_name` | The drug under discussion |
+| `section_key` | The attribute last answered |
+| `population` | `nguoi_lon` / `tre_em` / `phu_nu_co_thai` / … |
+| `verbosity` | `concise` \| `detailed`, set when the user asks |
+| `set_at_turn` | Turn index each field was last set |
+
+**`ConversationState` — the whole record.**
+
+```
+conversation_id
+recent: tuple[Turn, ...]     # last K turns, verbatim
+summary: str                 # rolling prose summary of everything older
+focus: Focus
+turn_count: int
+```
+
+A `Turn` carries `role`, `text`, `at`, and — for assistant turns — the
+`drug_id`, `section_key` and `evidence_ids` that produced it. Storing the
+evidence ids is what lets the planner answer a follow-up **from evidence
+already retrieved** instead of retrieving again.
+
+**Carry-over is never silent.** An inherited `drug_id` that is wrong is a
+wrong-drug answer, so any answer built on inherited focus must name what it
+inherited: *"Về Metformin, ở trẻ em: …"*. This is a hard rule, not a
+presentation preference.
+
+**Focus expires.** A field older than `FOCUS_TTL_TURNS` (6) is dropped rather
+than inherited. Conversations drift, and a drug from ten turns ago is not
+context, it is a hazard.
+
+### 2. Recent history and summary
+
+- `recent` holds the last **K = 6** turns verbatim (three exchanges).
+- When a turn falls out of `recent`, it is folded into `summary`.
+- `summary` is regenerated at most every **S = 4** turns, capped at **400
+  tokens**; `recent` is capped at **2000 tokens**, oldest dropped first.
+- **The summary records what was discussed, never clinical content.** It may
+  say *"đã hỏi liều dùng của Metformin cho người lớn"*; it may not carry a dose.
+  A dose restated from a summary would have no citation and could not be
+  grounding-verified — the check compares against retrieved evidence, and a
+  summary is not evidence.
+
+### 3. Reasoning loop
+
+```mermaid
+flowchart TD
+    A[User turn] --> B[UNDERSTAND<br/>resolve against Focus]
+    B --> C{Clarify signal?}
+    C -->|ambiguous drug / no attribute /<br/>multi-attribute| Z[ASK — 1 turn, no loop]
+    C -->|no| D{Simple?}
+    D -->|drug + section resolved,<br/>no follow-up ambiguity| E[RETRIEVE]
+    D -->|complex / decomposable| P[PLAN<br/>sub-questions + retrieval set]
+    P --> E
+    E --> F[ASSESS sufficiency]
+    F -->|insufficient AND rounds left| R[REFINE query] --> E
+    F -->|sufficient OR rounds exhausted| G[GENERATE]
+    G --> H[VERIFY<br/>grounding + coverage]
+    H -->|ungrounded / off-target,<br/>repairs left| G
+    H -->|grounded| Y[RESPOND]
+    H -->|repairs exhausted| X[FALL BACK<br/>verbatim source]
+    F -->|exhausted AND still thin| Z
+```
+
+**Continue conditions** — a round is spent only when all hold:
+1. `retrieval_rounds < MAX_RETRIEVAL_ROUNDS` (2)
+2. the assessor named a *specific* missing thing (a section, a population, a
+   second drug) — "feels incomplete" is not a reason to spend a round
+3. the refined query differs from every query already tried this turn
+
+**Stop conditions** — any one ends the loop:
+- sufficiency satisfied
+- budget exhausted (rounds, LLM calls, wall-clock, tokens)
+- a clarify signal fires (these bypass the loop entirely — asking beats guessing)
+- grounding verification fails after `MAX_REPAIRS` (1) → extractive fallback
+
+**Fast path.** When the drug resolves and `SectionResolver` returns a section
+and no clarify signal fires, the loop is skipped: retrieve → generate → verify.
+This is the majority path and it costs one LLM call.
+
+### 4. Budgets
+
+| Limit | Value | Enforced at |
+|---|---|---|
+| `MAX_RETRIEVAL_ROUNDS` | 2 | loop guard |
+| `MAX_REPAIRS` | 1 | loop guard |
+| `MAX_LLM_CALLS` per turn | 4 | budget object, checked before each call |
+| `MAX_WALL_CLOCK_MS` | 20000 | checked between stages |
+| `MAX_EVIDENCE_TOKENS` | 12000 | evidence assembly, oldest-dropped |
+| `FOCUS_TTL_TURNS` | 6 | state update |
+
+The budget is a single object threaded through the loop and **decremented
+before** each call, so exhaustion degrades to the best answer so far rather
+than to an error.
+
+### 5. Integration
+
+New domain modules, no SDK imports:
+
+- `rag/conversation.py` — `Focus`, `Turn`, `ConversationState`, window and
+  focus-update rules. Pure; the follow-up resolution in it needs no LLM.
+- `rag/reasoning.py` — the loop, its budget, and its stage protocols.
+- `rag/ports.py` — `ConversationStore` (load/save), `Summariser`, `Planner`,
+  `SufficiencyAssessor`. Each has a deterministic no-LLM default so the whole
+  loop runs offline.
+
+New adapter: `adapters/postgres.py` gains `PostgresConversationStore`.
+
+Unchanged and still binding: `GroundedAnswerService` remains the single-turn
+engine; `grounding.verify` gates every generated answer; `VERIFY_PDF` evidence
+is never generated over.
+
+### 6. Measurement
+
+A capability that cannot be shown to help does not ship. Three modes are run
+over the same cases — `single-shot`, `+history`, `+reasoning-loop`:
+
+| Metric | Answers |
+|---|---|
+| follow-up resolution accuracy | does *"còn trẻ em thì sao?"* reach the right drug+section+population |
+| on-target rate | does the answer contain the population/attribute actually asked for |
+| grounding rejection rate | does reasoning make fabrication more or less likely |
+| clarify rate / clarify precision | does it ask when it should, and only then |
+| median + p95 latency, LLM calls, tokens per answered turn | what the capability costs |
+
+The evaluation set is a **new multi-turn golden file** — the existing
+`golden_e2e_v1.csv` is single-turn by construction and cannot measure any of
+this. Counters land in `rag/metrics.py` and on the existing Grafana dashboard.
+
+## Consequences
+
+**Accepted.** More moving parts and more tokens per turn; a stateful service
+where there was a stateless one; a summary that must be kept free of clinical
+content by rule rather than by mechanism.
+
+**Refused.** An LLM confidence score as the loop's uncertainty signal. The
+signals used are the resolver states that already exist — ambiguous drug,
+unresolved section, multi-attribute question — because they are deterministic,
+testable, and explainable to a reviewer. "The model felt 0.73 sure" is not a
+defensible basis for asking or not asking a clinician a question.
+
+**Unchanged.** Nothing here lets the system assert a figure absent from the
+retrieved source. Reasoning chooses *what to look up and how to say it*; it is
+not a source of facts.

docs/progress-log.md

@@ -1,5 +1,414 @@
 # Progress Log
 
+## 2026-08-05 (evening 3) — The LLM cloud is LIVE: DeepSeek generation + Cohere rerank on the real corpus
+
+The owner rejected the $0 offline build as the deliverable and set a hard
+deadline. The chatbot is now a **real LLM RAG**, grounding kept ON, running the
+full HTTP stack (ai-service :8079 ↔ Postgres trace ↔ Qdrant; web :3000). Commit
+`9c4273b` (plus `92497ae`/`9e9cef7`/`1b6f399` earlier this session, which
+committed the previously-uncommitted evening-1/2 work).
+
+**What was turned on** (live via gitignored `.env`; committed defaults stay
+`disabled`/`section-only` so CI/fresh-clone never touches cloud):
+- `EMBEDDING_PROVIDER=cohere-v4` — query now embedded in the corpus's
+  `cohere.embed-v4:0` space (probe: 1024-dim, L2 1.0, ~1.95s). No re-embed; the
+  15,100 vectors already exist.
+- `ANSWER_PROVIDER=bedrock-converse` + `deepseek.v3.2` — new
+  `adapters/bedrock_converse.py` (Bedrock **Converse** API, boto3,
+  model-agnostic; Qwen/GLM = 1 env + 1 ARN). Probe OK. GPT-4o confirmed NOT on
+  Bedrock; OpenAI `gpt-oss`, DeepSeek, Qwen, GLM, Mistral, Kimi ARE (checked live).
+- `RERANK_ENABLED=true` — `cohere.rerank-v3-5` trims the overview/similarity
+  fallback: a free-form drug question no longer dumps all ~29 sections at the
+  model (**measured 29 → 6** on the fever/paracetamol case). Section route never
+  reranks; fail-open (outage → book order, answer survives).
+- `rag/prompt.py` rewritten to current citation-enforced practice: each dose
+  carries its population/condition label (no adult/paediatric mixing), cite only
+  the supporting block, no `[n]` spam, abstain on insufficient evidence.
+
+**IAM:** managed policy `BedrockEmbeddingInvoke` bumped to v4 (invoke on
+titan-embed, cohere.embed-v4, deepseek.v3.2, cohere.rerank-v3-5); repo file
+synced. Codex was off, no collision.
+
+**Verified:** ai-service **111 passed, 3 skipped** (+12 this milestone). Live
+HTTP `POST /v1/rag/query` returns a grounded LLM answer with a citation and a
+Postgres trace id. Golden `golden_e2e` (35 Qs): **19/19 answerable questions
+grounded with the correct drug** (incl. typo `paracetamon`, alias
+`Acetaminophen`, multi-turn inheritance); 14 adversarial correctly abstained
+(fake drugs, weather, symptom→drug reverse-lookup, multi-drug). **Two real
+gaps:** a price question answers from the monograph instead of "no price in the
+formulary", and "should I double the dose?" is not directly warned. Every
+`generated=True` answer passed `grounding.verify`.
+
+**Cost/safety:** Bedrock is pay-per-call — verified **0 EC2** (3 regions) and no
+provisioned throughput; idle = ~$0. A few dozen probe/smoke/eval calls this
+session, cents-scale on the estimate; exact bill not checked.
+
+**Separate track, NOT done (background subagent started, own worktree):**
+reconstruct the 151 quarantined tables with a `needs_expert` flag on uncertain
+cells + parse Part 1 (poisoning/pregnancy/hepatic-renal) & Part 3 (BSA/ATC) +
+re-embed. This is a multi-hour ingestion pass with the whole-doc validation gate
+and will NOT be clinician-validated within the deadline — deliberately kept off
+the deadline path.
+
+## 2026-08-05 (evening 2) — Conversational chat core wired LIVE end-to-end (offline, $0); owner wants the LLM cloud next
+
+The chat core is now **live and serving multi-turn**, not just unit-tested. It
+runs `$0`/no-cloud because the section-route is a payload filter (no query embed)
+and generation is still off (verbatim), but the *conversational* behaviour is
+real and smoke-tested against the running service (ai-service :8079, web :3000).
+
+Built (`rag/conversational.py` `ConversationalLoopService`, wrapping the safe
+`GroundedAnswerService`; wired through `bootstrap.py`/`main.py`/`routers/rag.py`
+with an optional `conversation_id`, plus `route.ts` sending it and a `ChatPanel`
+error state):
+
+- **Multi-turn follow-up inheritance.** "Chống chỉ định Metformin" then "còn trẻ
+  em thì sao?" carries the drug+section forward and names it ("Về metformin: …").
+- **Smalltalk.** "chào bạn" gets a friendly redirect, not a failed-drug-lookup
+  refusal.
+- **Drug-name-only → the whole monograph.** Typing "PARACETAMOL" now returns all
+  18 sections in book order with `【heading】`s and per-section citations
+  (`QdrantRetriever.find_by_drug` + `SECTION_ORDER`; `RetrievalService` uses it
+  when a drug resolves but no attribute is named) — the earlier "specify an
+  attribute" dead-end is gone.
+- **Typo → ask, never threshold-guess.** Only an EXACT drug name auto-resolves;
+  a fuzzy match is offered as a question ("Ý bạn là: Metformin?") via
+  `CatalogDrugResolver.suggest(min_score=0.72)`. A completely-wrong name →
+  "Không có thuốc này trong Dược thư Quốc gia." A formulary must not silently
+  answer about a *different* drug than the one meant.
+- **Autocomplete endpoint** `GET /v1/rag/suggest?q=` (`CatalogDrugResolver.complete`,
+  substring/prefix) — the frontend dropdown that consumes it is still to build.
+- **BSA calculator** `rag/calculators.py` (Appendix 1, DuBois, tested vs the
+  book's own cells).
+
+Verification: **ai-service 99 passed, 3 skipped**; live smoke test of all four
+conversation behaviours plus the monograph/typo/not-supported cases. A
+refine-loop bug (a refined query dropped the inherited drug and abstained,
+discarding a good answer) was found in my own code and removed before shipping —
+clarify + inheritance are the loop's value, retrieval-refine is not, and it is
+gone from the live path.
+
+**Owner's next-session directive (recorded in memory `project-llm-cloud-plan`):**
+stand up the cloud LLM — semantic query embedding (`EMBEDDING_PROVIDER=cohere-v4`,
+already IAM-permitted) and answer generation (a cheap model, non-Anthropic OK, via
+a Bedrock Converse adapter, needs its ARN added to `BedrockEmbeddingInvoke`). The
+offline build was budget/safety-first, not LLM-avoidance; the owner wants the real
+AI experience next, with `grounding.verify` and the quarantine contract kept ON.
+
+## 2026-08-05 (late) — Read the source book's own structure; scope + usage-pattern findings (checkpoint before handoff)
+
+Read the Dược thư 2018 front matter directly (printed p8 "Nội dung", p39
+"Hướng dẫn sử dụng") to understand what the book is *for* and how clinicians use
+it — recorded in memory `reference-duoc-thu-2018-structure`. Key facts that
+reshape the chatbot scope:
+
+- The book has **three parts**. The corpus is **Part 2 (drug monographs, printed
+  99–1496) ONLY**. **Excluded and clinically important:** Part 1 general chapters
+  (printed 37–98: prescribing in the elderly / hepatic-renal impairment /
+  children / pregnancy-lactation; disease-class guidance for asthma, epilepsy,
+  HIV, antibiotics, TB, hepatitis B, antipsychotics; drug allergy; **poisoning &
+  antidotes**; drug-interaction principles) and Part 3 appendices (printed
+  1497–1528: **body-surface-area calc**, IV admixture, ATC classification). So
+  "how to treat asthma", "antidote for X", "BSA-based dosing" have no data in the
+  index today — a coverage limit, not a retrieval bug.
+- The 19 monograph fields are fixed and documented on p39; a field is omitted
+  when the book has no info (so a missing section is not necessarily a parse bug).
+- Field 14 dose is a *general adult+child oral reference dose unless stated*; the
+  clinician adjusts. → the tool supplies reference data, not a prescription.
+
+Data checks run this session (against `chunks.jsonl`), correcting earlier
+pessimism:
+- Indication is searchable: 48 drugs' `chi_dinh` mention "sốt". Reverse lookup
+  (symptom → drugs) is feasible from **content**, but retrieval is drug-first, so
+  not answerable yet.
+- **mg/kg dosing is in PROSE, not tables**: 574 `lieu_luong` chunks contain
+  "mg/kg", all prose, across **295 drugs**, 473 of them mentioning trẻ em. So the
+  *primary* weight/age dosing (incl. pediatric) is answerable; the 83 quarantined
+  dosing tables are mostly the *supplementary* renal-adjustment tables (49 of
+  those 83 drugs also have mg/kg prose).
+- Pregnancy dosing is mostly **qualitative**: 670 drugs have a
+  `thoi_ky_mang_thai` section but only ~23 chunks carry a mg figure — the book
+  rarely gives a separate pregnant dose, so answer = pregnancy caution + standard
+  dose, never a fabricated pregnant-specific number.
+- `drug_id` can be compound (`paracetamol_acetaminophen`); alias resolution must
+  map "paracetamol" → that id.
+
+**Design consequence discussed with the owner (not yet built):** the "understand"
+stage must classify the *turn type* (smalltalk / medical query / multi-drug
+interaction / symptom-indication / out-of-scope / injection-shaped), not just
+resolve a drug. Refusing a clinician's symptom→drug question as
+"recommendation_out_of_scope" was wrong for this audience — such questions are
+indication lookups and should be answered from `chi_dinh`. Multi-drug
+interaction/contraindication questions need a real PLAN → gather both drugs →
+synthesize step (the ADR-0007 PLAN node, still unimplemented), and an
+absence-of-evidence answer must state where it looked, never assert "safe".
+
+**Session state / not yet done (so a fresh session can resume):** the chat
+module's domain glue is built and unit-tested (`rag/conversation.py` ports +
+summariser, `rag/conversational.py` orchestrator + `is_smalltalk`); it is **not**
+wired to the endpoint. Live wiring (turn-type classifier, loop-around-
+GroundedAnswerService, Postgres store, `conversation_id` on `/v1/rag/query`,
+`route.ts`, ChatPanel error state), the P0 audit fixes (§5 context-mixing
+metadata, §8 Qdrant-error degradation), reverse-indication retrieval, and the
+table vision-consensus pipeline all remain to do. No code was wired live this
+session; the behavior spec is still being clarified with the owner before wiring.
+
+**Owner decision: parse the WHOLE book, re-chunk freely** (not just Part 2
+monographs). Current corpus covers physical pages **100–1494** only. To add:
+Part 1 general chapters (physical ~36–97) and Part 3 appendices (~1496–1527);
+front-matter list (13) and index (1529+) are already used as validation ground
+truth. Read `segment/detector.py` to ground the plan — **the machinery already
+generalizes**: a chapter title ("NGỘ ĐỘC VÀ THUỐC GIẢI ĐỘC") has the *same shape*
+as a monograph title (bold + mostly-upper + short), so `is_monograph_title_candidate`
+extends by widening the hardcoded `99–1496` range per `content_type`. Only two
+real changes: (1) parametrize the page range + add a `content_type`
+(`monograph|chapter|appendix`); (2) chapter/appendix sub-headings are **free-form**
+("Hô hấp", "Co giật"), not the 19-key vocab, so `detect_section_headings` needs
+an open-taxonomy mode (bold + short = heading, store the text, no `match_section`
+requirement). Everything downstream (span extraction, table/formula quarantine,
+provenance, chunker) is content-type-agnostic and reused → schema v5 adds
+`content_type` + `chapter_id`. **Gate (CLAUDE.md): the span-routing ledger must
+account for ALL 1668 pages with `unassigned=0`, not just 99–1496.** Then embed
+only the NEW chunks (Cohere, pennies, announce first). This is a focused
+ingestion pass (detector + assembler + chunker + whole-doc re-run + validation),
+not a one-liner — not attempted this session beyond grounding the plan.
+
+**Done this session (App 1, self-contained, validated):** `rag/calculators.py`
+`body_surface_area_m2` replaces Appendix 1's lookup table with the book's DuBois
+formula (`S = W^0.425 × H^0.725 × 71.84`), tested against three of the book's own
+table cells (165cm/60kg→1.66, 90cm/10kg→0.50, 170cm/70kg→1.81) — `tests/test_calculators.py`,
+3 passed. Audit §7 (calculation = tested function, never an LLM).
+
+## 2026-08-05 (evening) — Conversational orchestrator wired to the existing loop; data-quality audit; budget verified live
+
+**Chat module (the glue ADR 0007 specified and nothing had called).** Added the
+two missing conversation ports and their offline defaults to `rag/conversation.py`
+(`ConversationStore`/`InMemoryConversationStore`, `Summariser`/`DeterministicSummariser`)
+and the orchestrator `rag/conversational.py` (`ConversationalRagService`). It owns
+no rules of its own: load state → resolve this turn → inherit gaps from `Focus`
+→ derive clarify signals from resolver state → `reasoning.run_turn` → update
+focus, append turns, summarise overflow, save → name any inherited drug. Runs
+with no LLM/service (collaborators are protocols). `DeterministicSummariser`
+records only drug/section **labels**, never cell values, so the
+no-clinical-content-in-summary rule holds by construction rather than by trust —
+closing the summary-bypasses-grounding hole flagged in review. **11 new tests;
+full ai-service suite 91 passed, 3 skipped.**
+
+**Still NOT wired live:** a `TurnResolver` bridge over `CatalogDrugResolver` +
+`SectionResolver`; bridges from `RetrievalService`→`Retrieve` and the grounded
+generation path→`Generate`; `PostgresConversationStore` + migration; a
+`conversation_id` on `/v1/rag/query`; `route.ts` sending it and dropping the
+hardcoded `intent: fact_lookup`; a `ChatPanel` error state; and the multi-turn
+eval run. So no claim yet that history/loop improves answers — designed and unit-
+proven, not measured end to end.
+
+**Data-quality audit (self-run this session, not quoted from this log).** 684
+drugs; critical-section coverage is strong — dosing missing 0.1% (1), contra-
+indication 0.4% (3), indication 0%. **But 83/684 drugs (12%) have their dosing
+inside a quarantined table**, so a dose query for them returns `VERIFY_PDF`
+(crop, no number) — the largest answer-quality gap for a clinician audience, and
+it lands on the single most-asked query. 125 chunks carry a leading `": "`
+label-leak artifact (93 in `ten_chung_quoc_te`). Vector-path text loss appears
+contained to 22 flagged lines (completeness of detection unverified). Nobody
+clinician-side has validated the 8.2M chars against the book — still the largest
+unmeasured area.
+
+**Table validation — the instrument that text extraction lacked.** Demonstrated
+that vision reads a real quarantined dosing table cell-by-cell: GABAPENTIN's
+renal-adjustment table (printed 706) came back exactly by eye where pdfplumber's
+text layer could not structure it. Found and corrected a page-index off-by-one
+in my own render (data `physical_page` N = `doc[N]`, 0-based) — proof that
+correctness must not depend on trusting coordinates. Strategy, given pharmacists
+are **end-users, not labelers**: reconstruction powers **retrieval only**; the
+displayed answer stays crop + page (clinician verifies at point of use).
+Validation is automated — vision↔geometric consensus + round-trip visual +
+book invariants — with a per-cell precision-first gate (disagreement → stays
+crop-only). Not yet built; 151 blocks is small enough for full census.
+
+**Budget, read live from the billing console** (owner login; `ai-lab-user` has
+no billing API perms): **$138.50 remaining, entirely AWS promotional credit,
+not the owner's card**; August bill $0. Deploy target chosen: team k3s, but
+deferred (mutating a shared cluster). Generation still off (`answer_provider=
+disabled`) — extractive/verbatim, which is defensible for clinicians; wiring a
+cheap model (Nova/Haiku via Bedrock Converse) needs its ARN added to the
+`BedrockEmbeddingInvoke` policy, which today grants invoke on the two embedding
+models only.
+
+## 2026-08-05 — An answer layer that cannot state a number the book does not
+
+Today started by walking the **demo path** rather than the test suite, and the
+two are not the same thing. The suite was green and the demo was broken.
+
+**What the walk found, by running it rather than reading it.** The backend
+answers real Vietnamese questions against the real embedded corpus with real
+citations and **zero cloud cost** — the section route is a payload filter, not
+a vector search. `Chống chỉ định của Metformin là gì?` returns the true
+contraindication text with one citation; `Tương tác thuốc của Warfarin?`
+returns two. But `Tôi sốt cao, uống Paracetamol được không?` returned **HTTP
+500**: the similarity fallback reached Bedrock, which is revoked, and
+`botocore.AccessDeniedException` escaped as an unhandled error. Any question
+whose phrasing is outside the section phrase table takes that path.
+
+That crash also **re-verified the cloud shutdown today, live** — the denial
+came from the service, not from a claim in a document.
+
+**Four defects, all fixed, all at $0.**
+
+1. **The 500.** `adapters/embedding.py` now translates provider failures into
+   the domain error `QueryEmbeddingUnavailable`, and `RetrievalService` catches
+   it and abstains with `reason="query_embedding_unavailable"` — deliberately
+   distinct from `insufficient_retrieval_score`, so an outage never reads as an
+   empty corpus. `rag/` still imports no SDK.
+2. **A default config that does not work.** `config.py` pointed at collection
+   `duoc_thu_chunks`; the real one is `duocthu_v1`. `embedding_provider`
+   defaulted to `disabled`, so `/v1/rag/query` returned 503 on a fresh clone.
+3. **Neither existing provider was a safe default.** `local-smoke` searches a
+   SHA-256 vector against a Cohere collection — confident, meaningless hits.
+   `cohere-v4` spends the boto3 retry budget (~30s) before failing on a revoked
+   account. Added `SectionOnlyQueryEmbedder`: refuses locally and instantly, so
+   retrieval is confined to the route that measured 16/16.
+4. **Safety abstention was incidental, not a gate.** Symptom questions abstain
+   with `reason="drug_not_resolved"` — because no drug name was found, not
+   because anything recognised a symptom question. Recorded, not yet fixed.
+
+**The answer layer now has an LLM, and a check that makes "it does not
+fabricate" measurable rather than promised.** Previously `rag/answer.py` was
+extractive: it concatenated retrieved chunks. That is why
+`Liều Paracetamol cho người lớn?` opened with `5 - 12 tuổi: Trẻ em 12 - 18
+tuổi:` — raw section text, paediatric doses first, for an adult question.
+
+Generation is now three layers, and only the third is load-bearing:
+
+- **Prompt** (`rag/prompt.py`, domain — no SDK): evidence only, figures copied
+  character-for-character, `[n]` citations required, insufficient evidence is a
+  valid answer. Output shape is pinned by `output_config.format`, so a
+  malformed envelope is the provider's error, not our parsing problem.
+- **Verification** (`rag/grounding.py`, pure domain): every numeric token in
+  the generated answer must appear **exactly** in the evidence, and every `[n]`
+  must resolve. Citation markers are stripped before number extraction so `[2]`
+  is never read as the quantity 2.
+- **Fail-closed** (`rag/answer.py`): ungrounded number, invalid citation,
+  malformed output, provider outage, or the model itself reporting insufficient
+  evidence — every one falls back to the verbatim source text, which was
+  computed first and is therefore always available.
+
+**Numbers are compared as strings, and that is the decision worth keeping.**
+No parsing, no normalisation. `1.500` is 1500 under one reading and 1.5 under
+another; a normaliser that strips separators maps `7,5` and `75` to the same
+key, scoring a **tenfold dose error as a match**. Pinned by
+`test_decimal_separators_are_not_interchangeable`. The same rule refuses
+`2 g` → `2000 mg`: arithmetically right, but unit conversion is where dosing
+errors live, so it is refused rather than interpreted.
+
+Quarantined tables and formulas are **never generated over**. `VERIFY_PDF`
+returns before generation — those are precisely the blocks whose numbers were
+not reliably reconstructed, so rephrasing them is the one case where fluency
+could invent a dose. This keeps ADR 0006's contract intact.
+
+**Provider chosen on the owner's instruction: AWS Bedrock + Claude.**
+`adapters/bedrock_claude.py` is the only module naming the `anthropic` SDK,
+imported lazily. Two provider facts taken from the Anthropic API reference
+today, not from memory: Bedrock model ids carry an `anthropic.` prefix
+(`anthropic.claude-opus-5`), and the Messages-API path on Bedrock is
+`AnthropicBedrockMantle`, **not** the legacy `bedrock-runtime` InvokeModel route
+the embedding adapter uses. A `stop_reason: "refusal"` is a successful HTTP
+response with no usable content, so it is routed to the extractive fallback
+rather than allowed to raise on `content[0]`.
+
+**This adapter has never been run against Bedrock.** Cloud access is still
+revoked and no IAM change was made today. `StubAnswerGenerator` exercises the
+entire path — prompt build, schema parse, grounding check, fallback — with no
+cloud call, and that is what the end-to-end run below used.
+
+**Observability, because a dashboard is a better answer than a slide.**
+`rag/metrics.py` defines the counters in the domain; `adapters/prometheus.py`
+is the only module naming `prometheus_client`, imported lazily; `/metrics`
+returns 404 rather than an empty 200 when metrics are off, so a scrape cannot
+succeed silently with no samples. The headline counter is
+`duocthu_generation_rejected_total{reason="ungrounded_number"}` — the measured
+form of the no-fabrication claim. A mismatched label drops the sample instead
+of raising: metrics must not be able to break a clinical answer.
+
+`infra/docker/` gains Prometheus and Grafana with a provisioned datasource and
+dashboard. **Not yet verified running** — the image pull was still in progress
+when this was written.
+
+**A section was being served scrambled, and only using the UI found it.**
+`liều dùng paracetamol` opened mid-sentence on `5 - 12 tuổi:` and buried
+`Liều lượng: Người lớn:` seven hundred words down. `find_by_section` returned
+whatever order Qdrant scrolled, and point ids are `uuid5(chunk_id)`, so
+PARACETAMOL's five dosing parts came back **3, 4, 1, 2, 0** — verified by
+scrolling the real collection, not inferred. `part_index` was in the payload
+all along and simply never used. Now sorted by it; a part missing the field
+sorts last rather than being dropped, because a silently shortened dose list
+is worse than an unordered one. Pinned by `tests/test_section_order.py`,
+including the exact 3,4,1,2,0 case. **This is a clinical defect, not a
+cosmetic one:** a reader who stops partway through stops in the middle of a
+different population's dose. Every section-routed answer given before today —
+including the 16/16 golden result — was assembled in this scrambled order;
+retrieval picked the right chunks, so the measurement stands, but no
+statement about how those answers *read* survives it.
+
+**Conversational reasoning RAG: designed in ADR 0007, domain layer built.**
+`rag/conversation.py` carries `Focus` (drug, section, population, verbosity,
+each stamped with the turn that set it) and the recent-turn window;
+`rag/reasoning.py` is the bounded loop. Both are pure domain and run with no
+provider, which is the point: *which drug is this still about* must be
+deterministic, not inferred.
+
+Three rules make inheritance safe in a formulary, each pinned by a test: an
+explicitly named drug always beats context; focus older than six turns is
+dropped rather than carried, because a stale drug is a wrong-drug answer, not
+context; and any answer built on an inherited drug must name it.
+
+The loop's uncertainty signal is **not** a model confidence score. It is the
+resolver states that already existed and previously dead-ended into `abstain`
+— ambiguous drug, unresolved attribute, multi-attribute question — which now
+produce a clarifying question. Deterministic, testable, and explainable to a
+reviewer in a way that "the model felt 0.73 sure" is not. A clarify signal
+short-circuits before any budget is spent, verified by asserting the budget is
+untouched and neither retriever nor generator was called.
+
+Budgets are decremented **before** the call they pay for, so exhaustion
+degrades to the best answer so far. A retrieval round is bought only by a
+*named* gap with a genuinely new query: `test_an_unnamed_gap_does_not_buy_a_round`
+and `test_a_refinement_that_changes_nothing_stops_the_loop` are the guards
+against a loop that spins on a feeling or re-issues the same query.
+
+`Golden Dataset/golden_multiturn_v1.csv` is new — 8 conversations, 19 turns,
+6 of them inheritance-dependent. The existing golden file is single-turn by
+construction and can measure none of this. Includes the adversarial turns: a
+follow-up after a refused fake drug (must not borrow a drug from elsewhere),
+and a follow-up after a symptom question (must not inherit treatment intent).
+
+**Not yet wired:** the loop is not called by `GroundedAnswerService` or the
+router, there is no `PostgresConversationStore`, and no evaluation run over the
+multi-turn file has been performed — so no claim is made that history or the
+loop improves answers. The design states how that will be measured; it has not
+been measured.
+
+**LangChain was considered and rejected.** The repo already has the ports and
+adapters LangChain would supply, retrieval is already measured, and the
+guardrail is already domain code. Adopting it a week before a review would
+rewrite the working part for no measured capability gain.
+
+Verification actually run: ai-service **56 passed, 3 skipped** (37 + 3 before,
++19); ingestion **296 passed**, checked for regression, unchanged; `duocthu_v1`
+holds **15,100 points** at 1024-dim Cosine, matching the manifest; live service
+against the real collection answered three clinical questions with citations
+and abstained on six of the seven safety probes; `/metrics` scraped and
+returned `duocthu_generation_served_total 2.0` and
+`duocthu_abstention_total{reason="drug_not_resolved"} 1.0`.
+
+Not established, and load-bearing for the demo: **`apps/web` is still entirely
+mocked** — `packages/api-client/src/sendChatMessage.ts:8` returns
+`buildMockResponse(content)` and the whole frontend contains no HTTP call to
+the backend, so the working API and the working UI are not connected;
+`api-gateway`, `chat-service` and `auth-service` hold **0 source files**; the
+Bedrock generator has never been invoked; `intent` is still supplied by the
+caller, so the recommendation gate depends on the client declaring it honestly;
+and the Prometheus/Grafana stack has not been seen running.
+
 ## 2026-08-04 (evening) — Section routing: contraindication retrieval goes from 0.05 to 1.00, at zero cloud cost
 
 The retrieval defect measured earlier today is fixed by routing rather than by