Add read-only production runtime audit
This commit is contained in:
+67
-201
@@ -1,219 +1,85 @@
|
||||
# Architecture — Dược Thư RAG Medical Chatbot
|
||||
# Kiến trúc và trạng thái hệ thống
|
||||
|
||||
## Overview
|
||||
> Loại chính: Explanation
|
||||
> Đối tượng: developer, reviewer và operator
|
||||
> Kiểm chứng: 2026-08-14
|
||||
|
||||
A medical chatbot grounded in the Vietnamese National Drug Formulary (Dược
|
||||
thư quốc gia Việt Nam 2018), built as a microservices monorepo. Users ask
|
||||
drug-related questions through a web chat UI; answers are generated via
|
||||
retrieval-augmented generation (RAG) over the formulary content, always
|
||||
citing the source drug monograph/section, and always carrying a medical
|
||||
disclaimer.
|
||||
Hệ thống có hai luồng độc lập gặp nhau tại Qdrant: ingestion chạy offline để
|
||||
biến PDF thành corpus; request path chạy online để hiểu câu hỏi, truy hồi bằng
|
||||
chứng, tạo câu trả lời và kiểm tra grounding.
|
||||
|
||||
## Service responsibilities & communication
|
||||
```text
|
||||
Offline
|
||||
PDF -> extract/repair -> monograph -> quarantine table/formula
|
||||
-> chunk schema v4 -> embedding -> Qdrant + manifest
|
||||
|
||||
| Service | Owns | Talks to |
|
||||
|---|---|---|
|
||||
| **api-gateway** (NestJS) | Single public entry point; request routing, JWT validation, rate limiting | Routes to auth-service, user-service, chat-service, ai-service over internal REST |
|
||||
| **auth-service** (NestJS) | Signup/login, password hashing, JWT issuance/refresh | Postgres (users); no dependency on other services |
|
||||
| **user-service** (NestJS) | Profile data, preferences, account settings | Postgres (profiles), called by gateway |
|
||||
| **chat-service** (NestJS) | Chat session lifecycle, message history persistence | Postgres (chat_sessions, chat_messages); calls ai-service per user message, persists both turns |
|
||||
| **ai-service** (Python/FastAPI) | RAG orchestration: understand query (LLM) → route to deterministic section/drug retrieval in Qdrant → generate + verify (LLM) → return answer + citations | Qdrant (payload-filtered retrieval), AWS Bedrock (Cohere embed-v4 for query embedding where used, Qwen3 via the Converse API for understanding/generation/entailment, Cohere rerank); conversation history is an in-process dict per `RagAgent`, not yet durable — see ADR 0008 |
|
||||
| **ingestion** (Python, offline batch) | One-time/periodic job: parse PDF → monographs → chunks → embeddings → upsert to Qdrant | Qdrant (write), AWS Bedrock (`cohere.embed-v4:0`); runs as CLI/CI/k8s Job, never in the live request path |
|
||||
| **web** (Next.js) | Chat UI, auth UI, citation/disclaimer rendering, session list | Calls api-gateway only |
|
||||
Online
|
||||
Browser -> Next.js BFF -> FastAPI RAG
|
||||
-> understand/guard -> Qdrant retrieval -> answer/validate
|
||||
-> PostgreSQL trace + conversation + feedback
|
||||
```
|
||||
|
||||
**Sync vs async**: the live chat path (web → gateway → chat-service →
|
||||
ai-service → Qdrant + AWS Bedrock → back) is synchronous request/response.
|
||||
Ingestion is fully decoupled, offline, batch — it populates Qdrant ahead of
|
||||
time and is never triggered by a chat request, since parsing the 37MB PDF and
|
||||
embedding thousands of chunks takes minutes. Internal protocol is REST/JSON
|
||||
for v1; a future gRPC migration is a documented option (see ADRs), not
|
||||
needed now.
|
||||
## Thành phần trên request path
|
||||
|
||||
## Data stores
|
||||
1. `apps/web` cung cấp Next.js UI và BFF `/api/chat`.
|
||||
2. BFF gọi `POST /v1/rag/query`, chuyển đổi DTO và map reason code cho UI.
|
||||
3. `apps/ai-service` gắn correlation/trace context và điều phối RAG.
|
||||
4. Qdrant giữ vector cùng payload provenance của chunk.
|
||||
5. PostgreSQL giữ retrieval trace, conversation turn và feedback.
|
||||
6. AWS Bedrock cung cấp query embedding; generation và rerank chỉ chạy khi bật.
|
||||
|
||||
- **Vector DB: Qdrant.** Chosen over pgvector because retrieval quality here
|
||||
depends on metadata-filtered ANN search (filter by drug name / section type
|
||||
combined with vector similarity) over a highly structured corpus — Qdrant
|
||||
makes that a first-class, single query. It also scales independently from
|
||||
the transactional Postgres and has a mature Helm chart for the production
|
||||
k8s target. See `docs/adr/0001-vector-db-qdrant.md`.
|
||||
- **Relational DB: PostgreSQL.** One instance, logically separated per
|
||||
service (users/credentials, profiles, chat sessions+messages). *As built,
|
||||
only `ai-service` uses it* — for conversation turns (`rag_conversation_turn`)
|
||||
and retrieval traces (`rag_retrieval_trace`). The users/profiles/sessions
|
||||
tables belong to services that do not exist yet.
|
||||
- **Redis.** Session/refresh-token cache, rate-limit counters, and reserved
|
||||
as the future job-queue backend (BullMQ/Celery) if async admin-triggered
|
||||
re-ingestion or background jobs are added later. **Not deployed** — nothing
|
||||
in the live path reads or writes Redis, so it was left out of
|
||||
`docker-compose.prod.yml` rather than run idle.
|
||||
Các service `api-gateway`, `auth-service`, `user-service` và `chat-service` là
|
||||
scaffold, chưa nằm trên live request path hiện tại.
|
||||
|
||||
## RAG ingestion pipeline (PDF-specific)
|
||||
## Bản đồ repository
|
||||
|
||||
The formulary is a structured per-drug reference, not free prose — the
|
||||
pipeline exploits that structure instead of naive fixed-size chunking. This
|
||||
section reflects an actual empirical investigation of the real PDF (not
|
||||
assumptions) — see `docs/adr/0003-pdf-parsing-strategy.md` for the full
|
||||
methodology, cross-tool comparison, and validation numbers.
|
||||
| Đường dẫn | Trách nhiệm |
|
||||
|---|---|
|
||||
| `apps/web` | UI và BFF Next.js |
|
||||
| `apps/ai-service` | FastAPI, RAG orchestration, adapters, migrations, evals |
|
||||
| `ingestion/ingestion` | PDF extraction, quality gates, chunking và load |
|
||||
| `ingestion/data` | dữ liệu raw/processed và artifact |
|
||||
| `packages/*` | shared types, API client và UI dùng chung |
|
||||
| `infra/docker` | local Compose và observability |
|
||||
| `infra/helm`, `infra/argocd` | target Kubernetes/GitOps |
|
||||
| `.github/workflows` | CI, deploy, rollback và Qdrant migration |
|
||||
| `docs` | bộ tài liệu chuẩn |
|
||||
| `docs-legacy` | raw/legacy để tra lịch sử |
|
||||
|
||||
1. **Extraction**: PyMuPDF (`fitz`) as primary extractor. This document has
|
||||
**no bookmark/outline** (`doc.get_toc()` returns 0 entries — confirmed,
|
||||
do not rely on it) and is a **tagged PDF with only a shallow, unusable
|
||||
structure tree** (~29 generic H1/P elements covering a fraction of 1668
|
||||
pages — also confirmed dead-end, not a data source). PyMuPDF's reading
|
||||
order was cross-validated against `pdfplumber` and `opendataloader-pdf` on
|
||||
real sample pages: pdfplumber's default text order is **unreliable** for
|
||||
this layout (scrambles paragraph order, leaks marked-content artifacts) —
|
||||
use it only for its dedicated table-extraction API, never for body text.
|
||||
Raw per-page extraction is persisted to `ingestion/data/interim/` so
|
||||
re-segmentation doesn't require re-running the expensive extraction step.
|
||||
2. **Segmentation**: drug-entry boundaries are detected via **bold-font
|
||||
spans** (PyMuPDF span `font` containing `"Bold"`), not font-size alone —
|
||||
font size for title/heading spans varies between monographs (confirmed:
|
||||
10.0pt and 9.5pt both occur for genuine drug-title headings), so bold is
|
||||
the reliable signal, all-caps + short length narrows it to monograph
|
||||
titles specifically. Section headings inside a monograph are also bold
|
||||
spans, cross-checked against a canonical taxonomy (`chi_dinh`,
|
||||
`chong_chi_dinh`, `lieu_dung`, `tac_dung_phu`, `tuong_tac_thuoc`, plus
|
||||
real observed extras like `ten_thuong_mai` "Tên thương mại" not in the
|
||||
book's own documented 19-field list — treat the taxonomy as open/
|
||||
extensible, not a fixed enum). Multi-line wrapped titles/headings (long
|
||||
Vietnamese names/vaccine names) must be merged across consecutive
|
||||
bold+all-caps lines before matching — this was the single largest source
|
||||
of missed detections in validation. Output: `{drug_id, drug_name,
|
||||
source_page_range, sections: {...}}` per drug, persisted to
|
||||
`ingestion/data/processed/monographs.jsonl` and validated both
|
||||
automatically (see ADR 0003) and via manual spot-check in
|
||||
`ingestion/notebooks/`.
|
||||
3. **Chunking** (monograph range only, pp. 99-1496 — see
|
||||
`docs/adr/0004-chunking-strategy.md` for the full measured rationale):
|
||||
each `(drug_id, section_key)` pair is the chunk unit; a section stays one
|
||||
chunk if it's under an **800-token ceiling** (chars/4 estimate — a
|
||||
validated line, not a guess: whole-corpus measurement across 682
|
||||
monographs shows ~16 of 18 section types clear it comfortably at their
|
||||
p90). Two sections routinely exceed it — `dược lý và cơ chế tác dụng`
|
||||
(35.7% of monographs that have it) and `liều lượng và cách dùng`
|
||||
(29.6%) — sub-chunking is the **routine** path for those two, not a rare
|
||||
edge case. Oversized sections are split with a **sentence-boundary-aware
|
||||
sliding window** (~600-700 tokens/sub-chunk, ~1 sentence/50-80 token
|
||||
overlap), never a blind character/line window — PDF line-wrap points
|
||||
are not safe cut points, and a mid-sentence split risks separating an
|
||||
adult/child dosing instruction (a measured, common pattern — outlier
|
||||
catalog item 17) into two chunks. Every chunk carries `chunk_id`,
|
||||
`drug_id`, `drug_name`, `section_key`, `section_display_name`,
|
||||
`atc_codes`, `source_page_range`, `part_index`/`part_count` as Qdrant
|
||||
payload — this is what makes citations possible. **Known open gaps**
|
||||
(see ADR 0004): sub-compound tagging inside class-level/multi-ATC
|
||||
monographs (25.5% of the corpus) is not yet solved; `source_page_range`
|
||||
is monograph-level, not sub-chunk-exact; chunking for general chapters/
|
||||
appendices is a separate, not-yet-designed task; a confirmed
|
||||
header/footer-boilerplate leak into section text (98.4% of monographs
|
||||
affected) must be fixed upstream before this design runs against real
|
||||
data.
|
||||
4. **Embedding + load**: AWS Bedrock `cohere.embed-v4:0` in batches
|
||||
(cached by `(model_id, input_kind, text_sha256)` so a reload needs no
|
||||
repeat cloud calls), upserted into Qdrant collection `duocthu_v1`
|
||||
(15,100 points, live) keyed by `uuid5(chunk_id)` for idempotent re-runs; a
|
||||
`<collection>__manifest` sidecar records the corpus sha/model/dimensions
|
||||
and `ai-service` refuses to start against a mismatched one (F-05).
|
||||
5. **Batch job, not synchronous**: runs as a CLI command locally, and as a
|
||||
Kubernetes `Job`/`CronJob` in production — never inside the ai-service
|
||||
request path.
|
||||
## Trạng thái triển khai
|
||||
|
||||
## Safety / guardrails
|
||||
| Thành phần | Trạng thái được xác nhận |
|
||||
|---|---|
|
||||
| Web/BFF | có code, được lint và build trong CI |
|
||||
| FastAPI RAG | có code và test tự động |
|
||||
| PDF → chunk schema v4 | có code, quality gate và test |
|
||||
| Qdrant manifest gate | runtime bắt buộc khi embedding bật |
|
||||
| PostgreSQL trace/hội thoại/feedback | có migration và adapter |
|
||||
| Bedrock Cohere embedding | production provider được runtime hỗ trợ |
|
||||
| Answer generation | tùy chọn; mặc định tắt |
|
||||
| Prometheus/Tempo/Grafana | có cấu hình local và production |
|
||||
| EC2 Compose + Caddy | luồng deploy hiện hành trong GitHub Actions |
|
||||
| Helm + ArgoCD | có manifest; chưa đủ bằng chứng để khẳng định đang phục vụ production |
|
||||
| Frontend automated tests | chưa có test runner; CI chỉ lint/build |
|
||||
| `visual-diff`, `scaffold-golden` | CLI tồn tại nhưng chưa triển khai |
|
||||
|
||||
- **System prompt** instructs the model to answer only from retrieved
|
||||
context, never state a dosage/contraindication/interaction not present in
|
||||
it, always append a disclaimer, and say "not found in the formulary"
|
||||
rather than guess when retrieval is irrelevant.
|
||||
- **Deterministic routing, not a similarity-confidence gate.** The live
|
||||
path resolves drug + section by exact payload filter (`section_key`
|
||||
routing moved contraindication hit@1 from 0.05 to 1.00 — similarity
|
||||
ranking alone was not reliable enough to gate on). A quarantined table/
|
||||
formula in the retrieved evidence, or missing page provenance, forces
|
||||
`VERIFY_PDF`/abstain deterministically — never an LLM-reported confidence
|
||||
score. Dense vector similarity search exists (`QdrantRetriever.search()`)
|
||||
but is reachable only in the legacy no-generator-configured mode, not the
|
||||
live agent path. See ADR 0008.
|
||||
- **Citations from metadata, not LLM prose**: the `citations` list is built
|
||||
directly from retrieved-chunk metadata, independent of what the LLM says,
|
||||
so the frontend can always show verifiable sources.
|
||||
- **Disclaimer enforced at multiple layers**: system prompt + a
|
||||
non-LLM-generated static string always appended to the API response + a
|
||||
persistent, non-dismissible UI banner.
|
||||
- **Scoped refusal**: out-of-scope questions (e.g. general symptom
|
||||
diagnosis) get a scoped refusal directing to a professional, not an
|
||||
ungrounded general-knowledge answer.
|
||||
## Mô hình triển khai
|
||||
|
||||
## Build roadmap
|
||||
Local thường chạy PostgreSQL, Qdrant và observability bằng Compose; web và
|
||||
ai-service chạy trực tiếp trên host. Các app service trong Compose local đang bị
|
||||
comment nên `docker compose up` không tự tạo toàn bộ ứng dụng.
|
||||
|
||||
1. **Ingestion pipeline + populated, queryable vector DB.** Done when a CLI
|
||||
run populates Qdrant and a test script retrieves the correct
|
||||
drug/section chunk for a sample query — no API, no LLM call yet.
|
||||
2. **ai-service (FastAPI) wrapping RAG + AWS Bedrock.** Done when a `curl` to
|
||||
`/v1/rag/query` returns a grounded answer with a traceable citation and an
|
||||
always-present disclaimer. **Done** — live since 2026-08-05, see ADR 0008.
|
||||
3. **auth/user/chat services + api-gateway.** Done when register → login →
|
||||
chat message flows end-to-end through the gateway only, persisted in
|
||||
Postgres. **Not started** — all four directories still hold only a
|
||||
`README.md` and a `package.json`. Phases 4-6 were done around this gap,
|
||||
so the live system has no gateway and no auth (see below).
|
||||
4. **Next.js frontend chat UI.** Done when a browser user can log in, ask a
|
||||
question, and see a grounded answer with citation + disclaimer banner.
|
||||
**Done except the login half** — chat, citations, evidence panel and the
|
||||
disclaimer banner are live; there is no login because Phase 3 does not
|
||||
exist. The browser calls `apps/web`'s own route handlers, which proxy
|
||||
directly to `ai-service`.
|
||||
5. **Containerize + docker-compose local.** Done when `docker compose up`
|
||||
from a clean checkout brings up the full stack and the Phase 4 flow works.
|
||||
**Done** — 2026-08-10. `infra/docker/docker-compose.prod.yml` is what
|
||||
production actually runs.
|
||||
6. **Kubernetes/Helm + Terraform + CI + ArgoCD (GitOps) deployment.** Done
|
||||
when CI builds/tests/pushes an image and bumps the target environment's
|
||||
Helm values file, the team's ArgoCD instance (see `infra/argocd/`,
|
||||
`docs/adr/0002-argocd-gitops.md`) picks up the change and syncs the
|
||||
cluster, and the Phase 4 flow works against the k8s-hosted stack. CI
|
||||
never runs `kubectl`/`helm` directly against a cluster. Cloud provider
|
||||
choice (AWS/GCP/Azure) only affects the Terraform module implementations,
|
||||
not this repo's structure.
|
||||
**Still the destination — not started, not dropped.** Production was
|
||||
shipped ahead of it on an interim single-box setup (see "Deployment as
|
||||
actually built" below), which is a stopgap, not a replacement: ADR 0002
|
||||
remains *Accepted*. Nothing here exists yet — `infra/k8s/`,
|
||||
`infra/helm/medical-chatbot/templates/` and `infra/terraform/` are empty
|
||||
scaffolds (`.gitkeep` only), the chart is version `0.0.0`, and every ArgoCD
|
||||
`Application` manifest still carries unresolved `TODO`s for project, repo
|
||||
URL and destination cluster.
|
||||
Production hiện hành được workflow mô tả là EC2 + Docker Compose + Caddy. CI và
|
||||
deploy là hai workflow độc lập; operator phải chủ động áp gate CI xanh và xác nhận
|
||||
SHA, health cùng synthetic query sau deploy.
|
||||
|
||||
This phase also includes a **repository move to the team's self-hosted
|
||||
Gitea** on the company domain, which is where the GitOps repo is intended
|
||||
to live; the project stays on private GitHub until that move is made
|
||||
deliberately. Hard boundary meanwhile: the team's existing
|
||||
`git.vinmec.tech/ai-team/gitops` repository is **reference-only — never
|
||||
push this project into it**.
|
||||
Helm/ArgoCD là target platform có implementation trong Git. “Manifest render được”
|
||||
không đồng nghĩa “cluster đang phục vụ traffic”; cần xác nhận cluster, secret,
|
||||
image tag, ingress, health và rollback thực tế trước khi đổi trạng thái.
|
||||
|
||||
## Deployment as actually built (2026-08-10)
|
||||
## Biên an toàn kiến trúc
|
||||
|
||||
Production is **not** the Phase 6 design. It is a single AWS EC2 `t3.large`
|
||||
running `infra/docker/docker-compose.prod.yml` — postgres, qdrant,
|
||||
ai-service, web, and Caddy terminating TLS for `realvuxbaro.me` via
|
||||
automatic Let's Encrypt. Bedrock is reached through an IAM instance role, so
|
||||
no long-lived AWS key exists on the box or in any env file.
|
||||
Mỗi evidence phải có provenance về trang in và chunk. Generator không tự quyết
|
||||
claim hợp lệ: code kiểm tra citation, số liệu, entailment và completeness. Khi
|
||||
không chứng minh được, hệ thống trả `clarify`, `verify_pdf` hoặc `abstain`.
|
||||
|
||||
CI/CD is `.github/workflows/deploy.yml`: a push to `master` SSHes in, resets
|
||||
the checkout, rebuilds only `ai-service`/`web`, runs migrations and
|
||||
health-checks both. It does not touch postgres/qdrant/caddy, so the 15,100
|
||||
Qdrant points survive deploys (they live in a named volume).
|
||||
|
||||
This is an **interim setup, not a decision against Phase 6.** It exists
|
||||
because a working public demo was needed sooner than the Kubernetes path
|
||||
could deliver one. The expensive prerequisite for that path — containerising
|
||||
both apps — is exactly what this work produced, so the Dockerfiles and
|
||||
compose services port over when the Gitea + team-ArgoCD migration is
|
||||
actually done. Phase 6 and ADR 0002 both stand as written.
|
||||
|
||||
See `docs/adr/` for architecture decision records. `docs/runbooks/` is still
|
||||
**empty** — the operational knowledge that would live there (restoring a
|
||||
Qdrant snapshot onto a fresh box, what a failed deploy looks like, why
|
||||
`uvicorn --reload` must not be used on Windows here) currently only exists
|
||||
in `docs/progress-log.md`.
|
||||
|
||||
Reference in New Issue
Block a user