222 lines
11 KiB
Markdown
222 lines
11 KiB
Markdown
# 10 — RAG orchestration
|
||
|
||
Implementation: `apps/ai-service/rag/agent.py` (`RagAgent`, 776 lines).
|
||
Tests: `tests/test_agent.py`, `tests/test_clinical_condition_flow.py`,
|
||
`tests/test_budget.py`.
|
||
Decision record: `docs/adr/0008-llm-understanding-one-shot-rag.md` (supersedes
|
||
ADR 0007).
|
||
|
||
## No framework
|
||
|
||
There is **no** LangChain, LlamaIndex, Haystack, or agent library anywhere in
|
||
the dependency set (`apps/ai-service/pyproject.toml` and the `Dockerfile`'s
|
||
inline pip list both confirm it). Orchestration is a plain Python class with a
|
||
hand-written branch table. `rag/` imports no SDK at all — the LLM arrives as a
|
||
`JsonLlm` / `AnswerGenerator` protocol.
|
||
|
||
## The two operating modes
|
||
|
||
`bootstrap.py::build_runtime` returns different graphs depending on config:
|
||
|
||
| `ANSWER_PROVIDER` | `app.state.conversational` | Live path |
|
||
|---|---|---|
|
||
| `disabled` | `None` | Retrieval-only, single-turn, through `GroundedAnswerService.answer()` + `QueryRoutingService` (fuzzy resolver + keyword section router). Evidence is quoted verbatim. |
|
||
| `stub` / `bedrock-converse` / `bedrock-claude` | `RagAgent` | The full understanding-driven path described below |
|
||
|
||
With `EMBEDDING_PROVIDER=disabled`, `build_runtime` returns `(None, None,
|
||
trace_writer, metrics)` and `/ready` answers 503 only if the embedding provider
|
||
was *not* disabled — so a disabled deployment reports ready while
|
||
`POST /v1/rag/query` returns 503 from the dependency.
|
||
|
||
## `RagAgent.handle()` — one turn
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant R as routers/rag.py
|
||
participant A as RagAgent
|
||
participant B as RequestBudget
|
||
participant S as PostgresConversationStore
|
||
participant U as LlmQueryUnderstander
|
||
participant RS as RetrievalService
|
||
participant GA as GroundedAnswerService
|
||
|
||
R->>A: handle(turn, conversation_id)
|
||
A->>B: start(40_000 ms, 8 calls)
|
||
A->>S: recent(conversation_id, history_turns*2 = 12)
|
||
Note over A,S: fail-open — a store outage means this turn has no memory
|
||
A->>U: understand(turn, history, budget, prior_frame)
|
||
U-->>A: QueryFrame
|
||
A->>A: _route(turn, frame, budget)
|
||
alt retrieval needed
|
||
A->>RS: retrieve_framed / retrieve_by_indication / per-drug interaction
|
||
A->>GA: answer_from_result(..., prechecked=True)
|
||
end
|
||
A->>A: _enforce_clarify_circuit_breaker()
|
||
A->>S: append(conversation_id, lines)
|
||
A->>A: _last_frame[conversation_id] = frame
|
||
A-->>R: AgentReply
|
||
```
|
||
|
||
## The routing table — `_route`
|
||
|
||
Order matters; the first match wins.
|
||
|
||
| # | Condition | Outcome |
|
||
|---|---|---|
|
||
| 1 | `looks_non_human(turn)` | `abstain / out_of_scope` — a deterministic scope guard **before** any conversational clarify, so an out-of-scope request never looks recoverable |
|
||
| 2 | `dosing_calc` + drugs + not a section overview, `population is None` | `clarify / missing_population` |
|
||
| 3 | same, paediatric and (`age_text` or `weight_kg` missing) | `clarify / missing_pediatric_age_or_weight` |
|
||
| 4 | `needs_clarify` + reason, and not `dosing_calc`/`condition_to_drug`/overview | `clarify / needs_more_info`, or `abstain / <system_error>` if the understanding call itself failed |
|
||
| 5 | `condition_to_drug` / `symptom_to_drug`, condition ambiguous | `clarify / ambiguous_condition` |
|
||
| 6 | same, `condition_relation != INDICATION` | `abstain / unsupported_reverse_relation` |
|
||
| 7 | same, condition or indication present | `_condition_to_drug()` |
|
||
| 8 | same, neither present | `clarify / no_condition` or `no_indication` |
|
||
| 9 | `condition_relation` turn type | `abstain / unsupported_reverse_relation` |
|
||
| 10 | `drug_attribute` with drugs but no attribute | `clarify / missing_attribute` |
|
||
| 11 | `smalltalk` | `answerable / smalltalk` (fixed greeting) |
|
||
| 12 | `out_of_scope` | `abstain / out_of_scope` |
|
||
| 13 | no drugs, but `unknown_drugs` | `abstain / drug_not_in_formulary` naming them |
|
||
| 14 | no drugs at all | `clarify / no_drug` |
|
||
| 15 | `interaction` with ≥2 drugs | `_interaction()` |
|
||
| 16 | otherwise | `_single_drug()` |
|
||
|
||
### Why dosing is a state machine, not a model opinion
|
||
|
||
The LLM extracts the fields; **code** decides which are required. Live testing
|
||
caught the model asking an adult's weight repeatedly after the user had supplied
|
||
a route, and previously dumping oral + rectal regimens together.
|
||
|
||
Paediatric turns require **both** age and weight, because the formulary branches
|
||
on both — paracetamol prints an age band (`Trẻ em 4-6 tuổi: 240 mg`) *and* a
|
||
weight rule (`10-50 kg: 15 mg/kg`), so answering with only one means picking a
|
||
regimen the source does not let you pick.
|
||
|
||
What changed on 2026-08-11 is the *question*, not the gate:
|
||
`_pediatric_clarify_question` now asks only for the missing field and echoes back
|
||
the known one (`"Bé nặng 18 kg, vậy bé bao nhiêu tuổi?"`). Reproduced 5/5 before
|
||
the fix: `"Bé 18 ký …"`, `"Bé nặng 18 kg …"` and `"Trẻ 5 tuổi …"` all received
|
||
the same generic sentence.
|
||
|
||
**Route is deliberately not a universal required slot.** Retrieval and the answer
|
||
contract decide from the actual evidence whether omitting it is harmless (one
|
||
applicable route → answer now) or materially ambiguous (several routes → clarify
|
||
with model-proposed quick replies). This prevents a chip funnel for a question
|
||
that was already precise enough.
|
||
|
||
### Clarify circuit breaker
|
||
|
||
```python
|
||
MAX_CONSECUTIVE_CLARIFY = 4
|
||
```
|
||
|
||
Found live 2026-08-07: the understanding model could re-ask the same clarifying
|
||
question forever — reproduced three times independently, one case never
|
||
converging after five real answered turns. `_merge_with_prior_frame` addresses
|
||
most of the cause; this is the code-level bound, because nothing otherwise stops
|
||
a model that keeps deciding `needs_clarify=true`. Any non-clarify decision resets
|
||
the streak. On trip it returns `abstain / clarify_loop_exhausted` with an
|
||
instruction to restate the whole question or start a new session.
|
||
|
||
The streak counter is **in-process only** — see
|
||
[02-system-architecture.md](02-system-architecture.md#the-stateful-detail-that-constrains-scaling).
|
||
|
||
## `_synthesize_query` — the context that reaches generation
|
||
|
||
`GroundedAnswerService.answer_from_result` has **no conversation history of its
|
||
own**; the `query` string it receives *is* the entire context its generation call
|
||
sees. `_synthesize_query` folds the resolved frame into one self-contained
|
||
question:
|
||
|
||
```
|
||
<turn>. Đối tượng: trẻ em. Tuổi: 5 tuổi. Cân nặng: 18 kg. Đường dùng: uống.
|
||
Chỉ định/triệu chứng: …. Bệnh nền: …. Dữ kiện thận: ….
|
||
```
|
||
|
||
Without it, a reply like `"Uống"` three turns into a dose conversation would
|
||
reach generation as just `"Uống"` — the two P0s the 2026-08-06 audit named
|
||
(population/weight/age/route extracted then discarded downstream) are exactly
|
||
this gap. Redundant when the turn is already self-contained; omission is the
|
||
failure mode, not repetition.
|
||
|
||
For a **patient-specific** candidate list, `_patient_generation_query` is used
|
||
instead. It deliberately withholds the raw patient values from the prompt: those
|
||
values have already done their job (selecting safety sections) and are not Dược
|
||
thư evidence, so restating them inside a cited claim would be — correctly —
|
||
rejected by the numeric grounding guard.
|
||
|
||
## Interaction path
|
||
|
||
For each named drug, retrieve its `tuong_tac_thuoc` section; keep parts whose
|
||
decision is `ANSWERABLE` **or** `VERIFY_PDF`; then pass the combined pool through
|
||
`RetrievalService.decide()`.
|
||
|
||
Keeping `VERIFY_PDF` parts is deliberate. Previously only `ANSWERABLE` parts were
|
||
kept, so a quarantined drug's evidence — and the "table exists, verify PDF"
|
||
notice the quarantine contract requires — was silently dropped, and a confident
|
||
interaction answer could omit exactly the unverified contraindication table it
|
||
should have flagged.
|
||
|
||
If no evidence at all: `abstain / no_interaction_evidence`, worded as *"not found
|
||
in each drug's interaction section"* and explicitly **not** as "safe":
|
||
|
||
> Điều này KHÔNG có nghĩa là an toàn khi phối hợp.
|
||
|
||
## Condition → drug path
|
||
|
||
1. Retrieve by indication (keyword, then dense fallback).
|
||
2. Derive matched drugs from `matched_doc_id` (`{drug_id}__chi_dinh__{n}`) — the
|
||
drugs actually found, never `frame.drugs`, which is empty by construction for
|
||
this turn type.
|
||
3. If the patient context requires a safety review, run stage 2
|
||
(`assess_patient_candidates`) and abstain if it produces no safety evidence —
|
||
*"Không suy ra thuốc là phù hợp/an toàn."*
|
||
4. Generate in `list_mode=True` with the candidate `drug_id` set bound into the
|
||
prompt and validated after generation.
|
||
|
||
The docstring is explicit that this is a factual list, not a treatment ranking:
|
||
no drug is preferred over another, and absence is stated plainly rather than as
|
||
"no such drug exists".
|
||
|
||
## Request budget — `rag/budget.py`
|
||
|
||
```python
|
||
max_wall_clock_ms = 40_000 # MAX_WALL_CLOCK_MS
|
||
max_llm_calls_per_turn = 8 # MAX_LLM_CALLS_PER_TURN
|
||
```
|
||
|
||
`budget.require()` is called immediately before each provider call and raises
|
||
`RequestBudgetExhausted` (a subclass of `AnswerGenerationUnavailable`, so every
|
||
existing fail-closed handler already does the right thing).
|
||
|
||
Its stated limit: it is checked **between** calls and cannot cancel a boto3 call
|
||
already in flight. That residual gap is bounded separately by
|
||
`read_timeout=20` with `total_max_attempts=2` in
|
||
`adapters/bedrock_converse.py`. The realistic worst case is therefore ~40 s plus
|
||
one in-flight call ≈ 60 s — which is why the browser timeout in
|
||
`ChatPanel.tsx` is 65 s.
|
||
|
||
## LLM calls per turn
|
||
|
||
| Call | When | Fail behaviour |
|
||
|---|---|---|
|
||
| 1. Understanding | Always (agent path) | Closed |
|
||
| 2. Sufficiency | Only on the legacy path — skipped when `prechecked=True` (i.e. always, on the agent path) or in `list_mode`, or with <2 evidence blocks | **Open** |
|
||
| 3. Generation | When evidence is answerable | Closed |
|
||
| 3b. Generation retry | Only when the model self-reported `evidence_sufficient=false` with no clarifying question | Closed |
|
||
| 4. Entailment | After grounding passes | Closed |
|
||
| 5–6. Completeness repair + re-verify | Only when entailment reports a *grounded* omission | Closed |
|
||
|
||
So a normal answerable agent turn is **3** sequential Bedrock calls; the
|
||
pathological ceiling is 8 (the budget), of which the repair path is the most
|
||
likely to exhaust it — observed live on an Isosorbid dinitrat dosage turn at
|
||
40.3 s against the 40 s budget.
|
||
|
||
## What ADR 0007 described and this replaced
|
||
|
||
ADR 0007's `Focus`/`ConversationState`/TTL design and its
|
||
PLAN/RETRIEVE/ASSESS/REFINE/VERIFY bounded loop, along with
|
||
`rag/conversation.py` and `rag/reasoning.py`, are **gone from the tree**. The
|
||
`LOOP_ROUNDS`, `LOOP_REFINED`, `LOOP_REPAIRED` and `FOLLOWUP_INHERITED` metric
|
||
names in `rag/metrics.py` are leftovers of that design and are no longer
|
||
incremented anywhere — see [27-technical-debt.md](27-technical-debt.md).
|