122 lines
7.4 KiB
Markdown
122 lines
7.4 KiB
Markdown
# Architecture — Dược Thư RAG Medical Chatbot
|
|
|
|
## Overview
|
|
|
|
A medical chatbot grounded in the Vietnamese National Drug Formulary (Dược
|
|
thư quốc gia Việt Nam 2018), built as a microservices monorepo. Users ask
|
|
drug-related questions through a web chat UI; answers are generated via
|
|
retrieval-augmented generation (RAG) over the formulary content, always
|
|
citing the source drug monograph/section, and always carrying a medical
|
|
disclaimer.
|
|
|
|
## Service responsibilities & communication
|
|
|
|
| Service | Owns | Talks to |
|
|
|---|---|---|
|
|
| **api-gateway** (NestJS) | Single public entry point; request routing, JWT validation, rate limiting | Routes to auth-service, user-service, chat-service, ai-service over internal REST |
|
|
| **auth-service** (NestJS) | Signup/login, password hashing, JWT issuance/refresh | Postgres (users); no dependency on other services |
|
|
| **user-service** (NestJS) | Profile data, preferences, account settings | Postgres (profiles), called by gateway |
|
|
| **chat-service** (NestJS) | Chat session lifecycle, message history persistence | Postgres (chat_sessions, chat_messages); calls ai-service per user message, persists both turns |
|
|
| **ai-service** (Python/FastAPI) | RAG orchestration: embed query → vector search in Qdrant → build grounded prompt → call OpenAI → return answer + citations | Qdrant (vector search), OpenAI API; stateless, does not own chat history |
|
|
| **ingestion** (Python, offline batch) | One-time/periodic job: parse PDF → monographs → chunks → embeddings → upsert to Qdrant | Qdrant (write), OpenAI embeddings API; runs as CLI/CI/k8s Job, never in the live request path |
|
|
| **web** (Next.js) | Chat UI, auth UI, citation/disclaimer rendering, session list | Calls api-gateway only |
|
|
|
|
**Sync vs async**: the live chat path (web → gateway → chat-service →
|
|
ai-service → Qdrant + OpenAI → back) is synchronous request/response.
|
|
Ingestion is fully decoupled, offline, batch — it populates Qdrant ahead of
|
|
time and is never triggered by a chat request, since parsing the 37MB PDF and
|
|
embedding thousands of chunks takes minutes. Internal protocol is REST/JSON
|
|
for v1; a future gRPC migration is a documented option (see ADRs), not
|
|
needed now.
|
|
|
|
## Data stores
|
|
|
|
- **Vector DB: Qdrant.** Chosen over pgvector because retrieval quality here
|
|
depends on metadata-filtered ANN search (filter by drug name / section type
|
|
combined with vector similarity) over a highly structured corpus — Qdrant
|
|
makes that a first-class, single query. It also scales independently from
|
|
the transactional Postgres and has a mature Helm chart for the production
|
|
k8s target. See `docs/adr/0001-vector-db-qdrant.md`.
|
|
- **Relational DB: PostgreSQL.** One instance, logically separated per
|
|
service (users/credentials, profiles, chat sessions+messages).
|
|
- **Redis.** Session/refresh-token cache, rate-limit counters, and reserved
|
|
as the future job-queue backend (BullMQ/Celery) if async admin-triggered
|
|
re-ingestion or background jobs are added later.
|
|
|
|
## RAG ingestion pipeline (PDF-specific)
|
|
|
|
The formulary is a structured per-drug reference, not free prose — the
|
|
pipeline exploits that structure instead of naive fixed-size chunking:
|
|
|
|
1. **Extraction**: PyMuPDF (`fitz`) as primary extractor (font size/style/
|
|
position metadata enables heading detection); pdfplumber as a fallback
|
|
specifically for tabular content (dosing/interaction tables). Raw
|
|
per-page extraction is persisted to `ingestion/data/interim/` so
|
|
re-segmentation doesn't require re-running the expensive extraction step.
|
|
2. **Segmentation**: detect drug-entry boundaries (prefer the PDF's
|
|
bookmark/outline via `doc.get_toc()` when present, else font-size/style
|
|
heuristics), then classify each heading against a canonical section
|
|
taxonomy (`chi_dinh`, `chong_chi_dinh`, `lieu_dung`, `tac_dung_phu`,
|
|
`tuong_tac_thuoc`, etc., Vietnamese diacritic-insensitive matching).
|
|
Output: `{drug_id, drug_name, source_page_range, sections: {...}}` per
|
|
drug, persisted to `ingestion/data/processed/monographs.jsonl` and
|
|
manually spot-checked via `ingestion/notebooks/`.
|
|
3. **Chunking**: each `(drug, section)` pair is the natural chunk unit;
|
|
never split a section unless it exceeds a token budget (~500-800 tokens),
|
|
in which case sub-chunk with a sliding window (400 tokens, 50 overlap),
|
|
tagging the same drug+section metadata plus `part_index`. Every chunk
|
|
carries `drug_name`, `section_type`, `source_page_range`, `chunk_id` as
|
|
Qdrant payload — this is what makes citations possible.
|
|
4. **Embedding + load**: OpenAI `text-embedding-3-small` in batches, upserted
|
|
into a versioned Qdrant collection (`drug_monographs_v1`) keyed by
|
|
`chunk_id` for idempotent re-runs; collection aliasing allows re-ingesting
|
|
with a changed chunking strategy without downtime.
|
|
5. **Batch job, not synchronous**: runs as a CLI command locally, and as a
|
|
Kubernetes `Job`/`CronJob` in production — never inside the ai-service
|
|
request path.
|
|
|
|
## Safety / guardrails
|
|
|
|
- **System prompt** instructs the model to answer only from retrieved
|
|
context, never state a dosage/contraindication/interaction not present in
|
|
it, always append a disclaimer, and say "not found in the formulary"
|
|
rather than guess when retrieval is irrelevant.
|
|
- **Retrieval-confidence gate**: below a similarity threshold, skip the LLM
|
|
call entirely and return a canned "consult a professional" response.
|
|
- **Citations from metadata, not LLM prose**: the `citations` list is built
|
|
directly from retrieved-chunk metadata, independent of what the LLM says,
|
|
so the frontend can always show verifiable sources.
|
|
- **Disclaimer enforced at multiple layers**: system prompt + a
|
|
non-LLM-generated static string always appended to the API response + a
|
|
persistent, non-dismissible UI banner.
|
|
- **Scoped refusal**: out-of-scope questions (e.g. general symptom
|
|
diagnosis) get a scoped refusal directing to a professional, not an
|
|
ungrounded general-knowledge answer.
|
|
|
|
## Build roadmap
|
|
|
|
1. **Ingestion pipeline + populated, queryable vector DB.** Done when a CLI
|
|
run populates Qdrant and a test script retrieves the correct
|
|
drug/section chunk for a sample query — no API, no LLM call yet.
|
|
2. **ai-service (FastAPI) wrapping RAG + OpenAI.** Done when a `curl` to
|
|
`/query` returns a grounded answer with a traceable citation and an
|
|
always-present disclaimer.
|
|
3. **auth/user/chat services + api-gateway.** Done when register → login →
|
|
chat message flows end-to-end through the gateway only, persisted in
|
|
Postgres.
|
|
4. **Next.js frontend chat UI.** Done when a browser user can log in, ask a
|
|
question, and see a grounded answer with citation + disclaimer banner.
|
|
5. **Containerize + docker-compose local.** Done when `docker compose up`
|
|
from a clean checkout brings up the full stack and the Phase 4 flow works.
|
|
6. **Kubernetes/Helm + Terraform + CI + ArgoCD (GitOps) deployment.** Done
|
|
when CI builds/tests/pushes an image and bumps the target environment's
|
|
Helm values file, the team's ArgoCD instance (see `infra/argocd/`,
|
|
`docs/adr/0002-argocd-gitops.md`) picks up the change and syncs the
|
|
cluster, and the Phase 4 flow works against the k8s-hosted stack. CI
|
|
never runs `kubectl`/`helm` directly against a cluster. Cloud provider
|
|
choice (AWS/GCP/Azure) only affects the Terraform module implementations,
|
|
not this repo's structure.
|
|
|
|
See `docs/adr/` for architecture decision records and `docs/runbooks/` for
|
|
operational runbooks (added as they're needed).
|