Dược Thư RAG — Medical Chatbot Platform

Medical chatbot grounded in the Vietnamese National Drug Formulary (Dược thư quốc gia Việt Nam 2018), built as a microservices monorepo.

Start with the canonical documentation set. It is a compact, code-verified set covering architecture, PDF ingestion, RAG/chat, local development, operations, API, configuration, evaluation and documentation governance.

The former numbered 00–29 material and historical plans are retained in docs-legacy/ as raw input only. Architecture decisions also remain there until reviewed. See the canonical documentation policy for source precedence.

Status (2026-08-24): live in production at realvuxbaro.me, running on k3s + ArgoCD since the 2026-08-17 cutover — a real RAG chatbot over the whole formulary, not a scaffold. What exists and what does not:

Part	State
ingestion/	Done — 15,100 chunks embedded and loaded into Qdrant duocthu_v1
apps/ai-service/	Done — live grounded RAG (retrieval, generation, grounding, abstention, citations, traces)
apps/web/	Done — chat UI with citation/evidence panel, optional login
apps/auth-service, apps/api-gateway	Live since 2026-08-19 (real NestJS: register/login/JWT, /auth/* proxy) with a /login page. Verified 2026-08-24: POST /api/auth/login with the seed demo account returns a session. Login stays optional — anonymous chat is unaffected. Seed accounts (admin/demo) still use placeholder passwords and are not safe to expose publicly as-is
apps/user-service, apps/chat-service	Not built — README.md + package.json only
apps/mobile/	Not built — reserved
infra/docker/	Compose is stopped (was production through 2026-08-17); still used for local dev (Postgres/Qdrant/observability), not for deploying anywhere
infra/helm/medical-chatbot/	This is production. Two ArgoCD Applications, medical-chatbot-app and medical-chatbot-data, both tracked in infra/argocd/applications/
Team's Gitea + ArgoCD (ADR 0002)	Still not started, not abandoned — the personal k3s/ArgoCD instance above is a separate, owner-operated cluster, not the team's shared infrastructure

apps/web's chat/history/feedback/sections routes still call apps/ai-service directly — auth-service/api-gateway exist but chat traffic doesn't route through the gateway yet. Login is optional and only gates /admin; anonymous chat is unaffected. See canonical architecture document for the implemented topology (not yet updated for this — verify against code, not that doc, until it is).

Directory map

apps/
  web/            Next.js frontend (also hosts the BFF route the browser calls)
  ai-service/     Python FastAPI — RAG orchestration + AWS Bedrock calls
  api-gateway/    NestJS — public entry point, routes to internal services
  auth-service/   NestJS — signup/login/JWT
  user-service/   NestJS — profile/preferences
  chat-service/   NestJS — chat session + message history
  mobile/         reserved for a future mobile app
packages/
  shared-types/   TS DTOs shared across Node services + web
  api-client/     typed HTTP client for web
  ui/             shared React components
  config/         shared eslint/tsconfig presets
ingestion/        offline batch pipeline: PDF -> monographs -> chunks -> embeddings -> Qdrant
infra/            docker-compose, k8s/Helm, Terraform, CI
docs/             canonical project documentation
docs-legacy/      raw notes, historical plans and ADRs pending review

Prerequisites

• Node.js + pnpm (JS workspace: apps/web, apps/auth-service, apps/api-gateway, packages/*; apps/user-service/apps/chat-service are still unbuilt placeholders)
• Python 3.11+ (apps/ai-service, ingestion)
• Docker (local Postgres + Qdrant via infra/docker/docker-compose.yml)
• AWS credentials with Bedrock invoke permission, for anything that generates an answer. Without them ai-service still starts, but every answer abstains rather than falling back to raw source text.

Running it locally

docker compose -f infra\docker\docker-compose.yml up -d postgres qdrant
cd apps\ai-service
python -m migrate
python -m uvicorn main:app --port 8079     # NOT --reload, see below

pnpm install
pnpm --filter web dev                      # http://localhost:3000

ai-service needs a populated Qdrant collection to serve answers: it verifies a duocthu_v1__manifest sidecar at startup and refuses to run against a corpus whose sha/model/dimensions do not match. A fresh machine either restores a Qdrant snapshot or re-runs ingestion/ (the latter costs real Bedrock spend).

Prefer a plain restart over uvicorn --reload on Windows here. The reloader has been observed serving the previous code after an edit on this project, which makes it hard to tell whether a change took effect.

Tests: cd apps/ai-service && python -m pytest -q — 230 pass. test_api.py and test_live_datastores.py need Postgres and Qdrant actually running; skip them with --ignore when the stack is down. apps/web has no test setup at all, so a green suite says nothing about the frontend — drive it in a browser.

Observability: Prometheus, Grafana and Tempo

The observability stack is provisioned in the repository and has been deployed to the production EC2 instance since 2026-08-11.

• Prometheus scrapes /metrics from ai-service. It records request rate and latency, latency for each RAG stage, routing decisions and reasons, provider failures, trace-write failures and the existing domain counters.
• Grafana is the user interface for dashboards and metric queries. Its datasource and the Dược Thư dashboard are provisioned automatically.
• Tempo stores OpenTelemetry traces. A trace contains the receive, understanding, routing, retrieval, rerank/evidence, generation, grounding/entailment, persistence and response stages. Correlation and trace IDs follow the request from the Next.js BFF into FastAPI.
• OpenTelemetry Collector receives spans from ai-service and exports them to Tempo. Grafana exemplars link aggregate latency metrics to an individual Tempo trace.

For answer lineage, use the three views together:

1. The web citation/evidence panel shows which source chunks, pages and exact evidence text were selected for the answer.
2. Grafana -> Explore -> Tempo shows which pipeline stages ran, their nesting and timing, the final decision/reason, provider failures and the persisted trace ID.
3. PostgreSQL table rag_retrieval_trace is the durable audit record. It stores the query, resolved drug, decision/reason, selected citations/evidence, correlation ID and OpenTelemetry trace ID, so a returned trace_id can be joined to its Tempo trace.

This is provenance and execution tracing, not model chain-of-thought logging. Full prompts/responses, hidden reasoning, every rejected retrieval candidate and every ranking score are deliberately not stored today. If deeper debugging is needed, add bounded audit fields rather than putting sensitive prompt or patient content into metric labels or span names.

Start the local stack from the repository root:

docker compose -f infra\docker\docker-compose.yml up -d prometheus tempo otel-collector grafana

Local endpoints:

Service	Address	Use
Grafana	http://localhost:3002	Dashboards and Explore
Prometheus	http://localhost:9090	Raw targets, PromQL and metrics
Tempo	http://localhost:3200	Trace backend; normally queried through Grafana
ai-service metrics	http://localhost:8079/metrics	Raw OpenMetrics output when ai-service runs on port 8079

Grafana is served through the same k3s ingress as the app, at https://realvuxbaro.me/grafana/. Anonymous access is on but demoted to Viewer (dashboards load with no login; write actions need the admin account). Prometheus has no public URL — use Grafana -> Explore -> Prometheus for PromQL, or Explore -> Tempo with a returned X-Trace-ID to investigate a slow request.

Production

Live at realvuxbaro.me, running on k3s + ArgoCD (a personal, owner-operated cluster — not the team's shared infrastructure) since the 2026-08-17 cutover. Two ArgoCD Applications: medical-chatbot-app (ai-service, web, observability) and medical-chatbot-data (PostgreSQL, Qdrant — a separate release so an app redeploy or prune can never touch persistent data). Both are tracked in infra/argocd/applications/; infra/helm/medical-chatbot/ is the chart both render from. Bedrock is reached through an IAM instance role — no long-lived AWS keys on the box or in any manifest.

Two things deploy independently:

• App code (apps/ai-service/**, apps/web/**, packages/**) — a push to master triggers .github/workflows/build-practice-images.yml, which builds and pushes GHCR images tagged by commit SHA, then repoints medical-chatbot-app at the new tag. ci.yml runs in parallel and does not gate this — a red test suite does not block a deploy.
• Chart/config (infra/helm/**) — helm-chart.yml lints and asserts render invariants on the PR; once merged, ArgoCD's own selfHeal picks up the change automatically. No CI step applies it directly.

Rollback is .github/workflows/rollback-k3s.yml (workflow_dispatch, target_sha) — see docs/operations.md for the full runbook, including its current gaps (no automated rollback for a config-only change, and the Grafana admin password still lives inline on the ArgoCD Application rather than in a real Kubernetes Secret).

The former EC2 Docker Compose deployment (i-039fc8f6102467a54) is stopped, not deleted — see docs/operations.md if it's ever needed as a manual DNS fallback again, though its corpus/schema will drift further out of date the longer it stays off.

The team's self-hosted Gitea + ArgoCD (per docs-legacy/adr/0002-argocd-gitops.md) remains the longer-term target for this project and is still not started — not abandoned, just a separate decision from the personal-cluster cutover above. Until it is deliberately started, the project stays on private GitHub, and the team's existing git.vinmec.tech/ai-team/gitops repository is reference-only: never push this project into it.