duocthu/README.md

# Dược Thư RAG — Medical Chatbot Platform

Medical chatbot grounded in the Vietnamese National Drug Formulary
(Dược thư quốc gia Việt Nam 2018), built as a microservices monorepo.

See [docs/architecture.md](docs/architecture.md) for the full design
(service responsibilities, data stores, RAG ingestion strategy, safety
guardrails), [docs/adr](docs/adr) for architecture decision records,
[docs/pdf-parsing-outlier-catalog.md](docs/pdf-parsing-outlier-catalog.md)
for a reusable checklist of confirmed PDF-parsing risks (useful for this
book and any similarly-structured PDF), and
[docs/progress-log.md](docs/progress-log.md) for a running log of what's
been done and what's next.

Dated planning and audit documents (`docs/v1-delivery-plan.md`,
`docs/rag-rebuild-plan.md`, `docs/current-rag-pipeline-audit.md`,
`docs/answer-experience-implementation-plan.md`) record what was known on
their date and are kept for their reasoning rather than as current status —
this README and `git log` are the better reference for where things stand
today.

> **Status** (2026-08-11): **live in production at
> [realvuxbaro.me](https://realvuxbaro.me)** — a real RAG chatbot over the
> whole formulary, not a scaffold. What exists and what does not:
>
> | Part | State |
> |---|---|
> | `ingestion/` | Done — 15,100 chunks embedded and loaded into Qdrant `duocthu_v1` |
> | `apps/ai-service/` | Done — live grounded RAG (retrieval, generation, grounding, abstention, citations, traces) |
> | `apps/web/` | Done — chat UI with citation/evidence panel |
> | `apps/api-gateway`, `auth-service`, `user-service`, `chat-service` | **Not built** — `README.md` + `package.json` only |
> | `apps/mobile/` | **Not built** — reserved |
> | `infra/docker/` | Done — production runs Compose, including the Prometheus/Grafana/Tempo observability overlay |
> | `infra/helm/medical-chatbot/` | Built and validated as an offline migration kit; **not deployed** to Docker Desktop, k3s or ArgoCD |
> | `infra/k8s`, `terraform`, `argocd` | **Not built yet** — still the target (ADR 0002), not abandoned: the plan is the team's self-hosted Gitea + ArgoCD; the current EC2/Compose setup is an interim stopgap |
>
> Because the gateway and auth services do not exist, `apps/web` talks
> **directly** to `apps/ai-service`; there is no authentication layer. See
> the build roadmap in `docs/architecture.md`.

## Directory map

```
apps/
  web/            Next.js frontend (also hosts the BFF route the browser calls)
  ai-service/     Python FastAPI — RAG orchestration + AWS Bedrock calls
  api-gateway/    NestJS — public entry point, routes to internal services
  auth-service/   NestJS — signup/login/JWT
  user-service/   NestJS — profile/preferences
  chat-service/   NestJS — chat session + message history
  mobile/         reserved for a future mobile app
packages/
  shared-types/   TS DTOs shared across Node services + web
  api-client/     typed HTTP client for web
  ui/             shared React components
  config/         shared eslint/tsconfig presets
ingestion/        offline batch pipeline: PDF -> monographs -> chunks -> embeddings -> Qdrant
infra/            docker-compose, k8s/Helm, Terraform, CI
docs/             architecture docs and ADRs
```

## Prerequisites

- Node.js + pnpm (JS workspace: `apps/web`, `packages/*`; the NestJS service
  directories are unbuilt placeholders)
- Python 3.11+ (`apps/ai-service`, `ingestion`)
- Docker (local Postgres + Qdrant via `infra/docker/docker-compose.yml`)
- AWS credentials with Bedrock invoke permission, for anything that generates
  an answer. Without them `ai-service` still starts, but every answer abstains
  rather than falling back to raw source text.

## Running it locally

```powershell
docker compose -f infra\docker\docker-compose.yml up -d postgres qdrant
cd apps\ai-service
python -m migrate
python -m uvicorn main:app --port 8079     # NOT --reload, see below
```

```powershell
pnpm install
pnpm --filter web dev                      # http://localhost:3000
```

`ai-service` needs a populated Qdrant collection to serve answers: it verifies
a `duocthu_v1__manifest` sidecar at startup and refuses to run against a corpus
whose sha/model/dimensions do not match. A fresh machine either restores a
Qdrant snapshot or re-runs `ingestion/` (the latter costs real Bedrock spend).

> **Prefer a plain restart over `uvicorn --reload` on Windows here.** The
> reloader has been observed serving the previous code after an edit on this
> project, which makes it hard to tell whether a change took effect.

Tests: `cd apps/ai-service && python -m pytest -q` — 230 pass. `test_api.py` and
`test_live_datastores.py` need Postgres and Qdrant actually running; skip them
with `--ignore` when the stack is down. `apps/web` has **no test setup at all**,
so a green suite says nothing about the frontend — drive it in a browser.

## Observability: Prometheus, Grafana and Tempo

The observability stack is provisioned in the repository and has been deployed
to the production EC2 instance since 2026-08-11.

- **Prometheus** scrapes `/metrics` from `ai-service`. It records request rate
  and latency, latency for each RAG stage, routing decisions and reasons,
  provider failures, trace-write failures and the existing domain counters.
- **Grafana** is the user interface for dashboards and metric queries. Its
  datasource and the Dược Thư dashboard are provisioned automatically.
- **Tempo** stores OpenTelemetry traces. A trace contains the receive,
  understanding, routing, retrieval, rerank/evidence, generation,
  grounding/entailment, persistence and response stages. Correlation and trace
  IDs follow the request from the Next.js BFF into FastAPI.
- **OpenTelemetry Collector** receives spans from `ai-service` and exports them
  to Tempo. Grafana exemplars link aggregate latency metrics to an individual
  Tempo trace.

For answer lineage, use the three views together:

1. The web citation/evidence panel shows which source chunks, pages and exact
   evidence text were selected for the answer.
2. **Grafana -> Explore -> Tempo** shows which pipeline stages ran, their
   nesting and timing, the final decision/reason, provider failures and the
   persisted trace ID.
3. PostgreSQL table `rag_retrieval_trace` is the durable audit record. It stores
   the query, resolved drug, decision/reason, selected citations/evidence,
   correlation ID and OpenTelemetry trace ID, so a returned `trace_id` can be
   joined to its Tempo trace.

This is provenance and execution tracing, not model chain-of-thought logging.
Full prompts/responses, hidden reasoning, every rejected retrieval candidate
and every ranking score are deliberately not stored today. If deeper debugging
is needed, add bounded audit fields rather than putting sensitive prompt or
patient content into metric labels or span names.

Start the local stack from the repository root:

```powershell
docker compose -f infra\docker\docker-compose.yml up -d prometheus tempo otel-collector grafana
```

Local endpoints:

| Service | Address | Use |
|---|---|---|
| Grafana | `http://localhost:3002` | Dashboards and Explore |
| Prometheus | `http://localhost:9090` | Raw targets, PromQL and metrics |
| Tempo | `http://localhost:3200` | Trace backend; normally queried through Grafana |
| ai-service metrics | `http://localhost:8079/metrics` | Raw OpenMetrics output when ai-service runs on port 8079 |

For the existing EC2 Compose deployment, the optional overlay is
`infra/docker/docker-compose.observability.yml`. It leaves
`docker-compose.prod.yml` unchanged. A deployment, when explicitly approved,
uses both files:

```powershell
docker compose `
  -f infra/docker/docker-compose.prod.yml `
  -f infra/docker/docker-compose.observability.yml `
  up -d
```

Only Grafana is mapped to the EC2 host (`3002:3000`) by the production overlay;
Prometheus and Tempo stay on the internal Compose network. The EC2 security
group does not expose port 3002 publicly. View Grafana through an SSH tunnel:

```powershell
ssh -L 3002:127.0.0.1:3002 <ssh-user>@52.0.158.61
```

Keep that session open and visit `http://localhost:3002`. Set
`GRAFANA_ADMIN_USER` and `GRAFANA_ADMIN_PASSWORD` in the production environment
before deployment; do not use the fallback password in production. Prometheus
metrics are available in **Grafana -> Explore -> Prometheus**. To investigate a
slow request, open the request-latency panel, follow its exemplar/trace link, or
paste the returned `X-Trace-ID` into **Explore -> Tempo**.

## Production

Live at [realvuxbaro.me](https://realvuxbaro.me): a single EC2 `t3.large`
running `infra/docker/docker-compose.prod.yml` (postgres, qdrant, ai-service,
web, Caddy for automatic Let's Encrypt TLS). Bedrock is reached through an IAM
instance role — there are no long-lived AWS keys on the box or in any env file.

Pushing to `master` deploys: `.github/workflows/deploy.yml` SSHes in, resets to
the pushed commit, rebuilds only `ai-service`/`web`, runs migrations and
health-checks both. Postgres/Qdrant/Caddy are left untouched, so the vector
data survives deploys (it lives in a named volume, not the container).

This is **interim infrastructure**, not the end state. The intended target is
still the team's self-hosted **Gitea** (company domain) plus their **ArgoCD**
instance, per `docs/adr/0002-argocd-gitops.md` — that work is *not started*,
not cancelled. Until it is deliberately started, the project stays on private
GitHub, and the team's existing `git.vinmec.tech/ai-team/gitops` repository is
reference-only: never push this project into it.