Soften the tone of the docs and comments written today

This commit is contained in:
2026-08-11 10:12:25 +07:00
parent 97cb6d16f4
commit 6b8f7584ed
18 changed files with 921 additions and 64 deletions
+121
View File
@@ -1,5 +1,126 @@
# Progress Log
## 2026-08-11 — Five production bugs found by driving the live site, fixed, deployed and re-verified across 37 live cases
**Starting point: two items in the docs had moved on**, found by reading the
code and driving production rather than by re-reading the docs:
1. The structured-claims refactor that
`coordination/CLAUDE_HANDOFF_2026-08-10.md` describes as in progress
shipped the same day (`dfdbf52`, then `9c3acd0`).
2. Entailment majority-vote (2-of-3) is no longer in the code. `df55af4`
introduced it; `9c3acd0` replaced it with a single pass
(`_ENTAILMENT_MAX_ATTEMPTS = 1`).
The previous entry here (cont. 17) also predates five commits —
`9c3acd0`/`33154d4`/`01e44ad`/`480bd1a`/`4438c5f`, 17:09-17:27 on
2026-08-10. `9c3acd0` is substantial: `QueryFrame` gained
`section_overview`/`standalone_query`/`depends_on_previous_turn`,
`population`/`route` became enum-validated, `CatalogDrugResolver.resolve()`
went from ~10k regexes per query to a token-span index, and `page.tsx`
stopped losing messages on session switch. Worth remembering generally:
entries in this log are written at a point in time, so `git log` is the
reliable check for current state.
### Findings (all from driving `https://realvuxbaro.me`)
- **A 25s client abort against a 40s backend budget.** `ChatPanel.tsx`
aborted every request at 25s; `config.py`'s `max_wall_clock_ms` is 40s and
can overrun by one in-flight call (`read_timeout=20`), putting the
backend's ceiling near 60s. Measured n=8 sequential: 6.2/6.4/8.4/10.9/12.4/
21.7/**25.1**/**40.3**s. The 25.1s case was a correct `answerable`,
grounded, 2-citation answer that never reached the user — the UI showed
"Yêu cầu vượt quá 25 giây... thử lại với câu hỏi cụ thể hơn", which points
at the question when the cause was timing. Caddy and the BFF set no timeout
of their own, so this constant was the only binding limit.
- **Availability failures reaching the user as content failures.**
`_run_entailment_check` returned a bare `None` for budget exhaustion,
provider outage and an unparseable judge reply alike, and the caller mapped
all three to `unsupported_claim`, i.e. "the answer doesn't match the
source", in cases where the judge was never consulted. The
completeness-repair path had the same shape: it fell through to
`incomplete_answer`, whose text tells the clinician the answer was
cancelled for omitting source information. Live example: Isosorbid dinitrat
dosage, 40.3s against a 40s budget, reported as `incomplete_answer`. The
failure taxonomy in `docs/current-rag-pipeline-audit.md` §4 keeps
availability and content failures separate for this reason.
- **The pediatric dosing gate asked again for what the user had given.**
`agent.py`'s fallback was the static "Bé bao nhiêu tuổi và cân nặng bao
nhiêu kg?". Reproduced 5/5: "18 ký", "18 cân", explicit "18 kg", and
"Trẻ 5 tuổi" all received it. Worth noting for anyone revisiting it: this
is not a Vietnamese colloquial-weight parsing issue — explicit "kg"
behaved identically, so the parser is not the place to change. The effect
was also more visible the better `understanding.py` did, since a frame that
parsed the weight and set `needs_clarify=false` reached the static string.
- **A retry constant with no effect.** `_verify_entailment`'s
`for _ in range(_ENTAILMENT_MAX_ATTEMPTS)` returned on its first iteration
on every path, so raising it adds no retries, and the unreachable
`return False` after it returns a `bool` where callers read `.supported`.
- **Citation chips that render identically.** Deduped by `chunkId` but
labelled only drug+section+page, so three distinct chunks appeared as three
identical "METFORMIN · Liều lượng & Cách dùng · tr. 957" chips.
### What was changed, and what deliberately was NOT
The pediatric gate **still requires both age and weight** — the formulary
bands paracetamol by age ("Trẻ em 4-6 tuổi: 240 mg") *and* by mg/kg ("10-50
kg: 15 mg/kg"), so one field alone cannot pick a regimen. Only the question
changed, and it now echoes the known value back so a mis-parse is visible.
Chips are **not** collapsed by label — each opens a different evidence block
and provenance is a hard guardrail — they carry the number the evidence
panel already shows. The completeness judge was **not** relaxed: making that
symptom disappear by loosening it would ship incomplete medical answers.
Reason codes reused are ones the BFF already maps
(`request_budget_exhausted`, `provider_unavailable`, `malformed_output`);
an unmapped code silently reads as "no data in the formulary".
### Verification (37 live cases, not one)
`pytest`: **230 passed** (was 219; 9 added). Ruff, `tsc --noEmit` and the
Next production build all pass. One existing test changed on purpose —
`test_entailment_provider_outage_fails_closed_to_abstain` asserted the old
`unsupported_claim` label; its fail-closed assertions are untouched.
Post-deploy, against production: a **31-case battery** (cases that must
change, cases that must NOT, plus neighbouring behaviour) and a **6-run
repeat** of one flaky query.
- Pediatric clarify verified across 7 variants: "Bé 18 ký…" → "Bé nặng 18
kg, vậy bé bao nhiêu tuổi?"; "Trẻ 5 tuổi…" → "Bé 5 tuổi nặng bao nhiêu
kg?"; "Bé 8 tháng tuổi…" → "Bé 8 tháng tuổi nặng bao nhiêu kg?".
- Multi-turn resolves in **both** directions (clarify→age and clarify→weight
both reach `answerable` with 2 citations).
- Timeout fix proven in the browser: a Metformin adult-dosing question ran
past **33s** — dead 8s earlier under the old limit — and returned a full
grounded answer with 3 citations. Deployed bundle contains `65e3`/`15e3`
and **no** `25e3`.
- Chips render `[1] [2] [3]` matching evidence-panel cards 1/2/3, all three
preserved.
- Regression guards all held: adult dosing untouched by the pediatric gate,
quarantine `verify_pdf` intact (4 and 6 citations), fake drug →
`drug_not_in_formulary`, veterinary → `out_of_scope`, ordinary facet
lookups still answerable.
### Known-remaining, deliberately not claimed as fixed
- **The reason-code split is unit-tested but was NOT observed live**: nothing
in the verification run exhausted the budget (max 24.2s), so no live
`request_budget_exhausted` from the entailment/repair path was seen.
- **Zolpidem ADR is flaky**: 6 repeats gave 5 `answerable`, 1
`ungrounded_number` (~17%). Pre-existing generation variance —
`grounding.verify` runs *before* any code changed here — not a regression.
- **The same "already told you" defect survives in the LLM-generated clarify
question**: "Bé 12 cân uống paracetamol…" is answered with "Đường dùng là
uống hay tiêm ạ?" although the user said "uống". That text comes from
`understanding.py`'s own `clarify_reason`, not the code-level fallback
fixed here.
- Latency is unchanged — the timeout fix stops discarding good answers, it
does not make anything faster. Streaming is still the real fix.
- **Sildenafil ADR is not a deterministic `ungrounded_number` failure**, as
`docs/current-rag-pipeline-audit.md` states: two runs gave 46.8s
`abstain/unsupported_claim` and 25.1s `answerable/grounded`.
- Task #4 (real BM25 via Qdrant native sparse vectors) remains **not
started**.
## 2026-08-10 (cont. 17) — First production deployment: EC2 + Docker + CI/CD, live at realvuxbaro.me
Owner and Codex agreed a work split mid-session