Initial monorepo scaffold for Duoc Thu RAG medical chatbot
This commit is contained in:
@@ -0,0 +1,15 @@
|
|||||||
|
root = true
|
||||||
|
|
||||||
|
[*]
|
||||||
|
indent_style = space
|
||||||
|
indent_size = 2
|
||||||
|
end_of_line = lf
|
||||||
|
charset = utf-8
|
||||||
|
trim_trailing_whitespace = true
|
||||||
|
insert_final_newline = true
|
||||||
|
|
||||||
|
[*.py]
|
||||||
|
indent_size = 4
|
||||||
|
|
||||||
|
[*.md]
|
||||||
|
trim_trailing_whitespace = false
|
||||||
+44
@@ -0,0 +1,44 @@
|
|||||||
|
# Node
|
||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
.next/
|
||||||
|
.turbo/
|
||||||
|
*.tsbuildinfo
|
||||||
|
npm-debug.log*
|
||||||
|
pnpm-debug.log*
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
*.egg-info/
|
||||||
|
|
||||||
|
# Env / secrets
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
|
||||||
|
# Ingestion large/derived artifacts (regeneratable — never commit)
|
||||||
|
ingestion/data/interim/*
|
||||||
|
!ingestion/data/interim/.gitkeep
|
||||||
|
ingestion/data/processed/*
|
||||||
|
!ingestion/data/processed/.gitkeep
|
||||||
|
|
||||||
|
# Terraform
|
||||||
|
**/.terraform/
|
||||||
|
*.tfstate
|
||||||
|
*.tfstate.*
|
||||||
|
*.tfvars
|
||||||
|
|
||||||
|
# OS / editor
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
|
||||||
|
# Claude Code personal/local settings (bypass permissions mode etc. — not shared)
|
||||||
|
.claude/settings.local.json
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# Dược Thư RAG — Medical Chatbot Platform
|
||||||
|
|
||||||
|
Medical chatbot grounded in the Vietnamese National Drug Formulary
|
||||||
|
(Dược thư quốc gia Việt Nam 2018), built as a microservices monorepo.
|
||||||
|
|
||||||
|
See [docs/architecture.md](docs/architecture.md) for the full design
|
||||||
|
(service responsibilities, data stores, RAG ingestion strategy, safety
|
||||||
|
guardrails), [docs/adr](docs/adr) for architecture decision records, and
|
||||||
|
[docs/progress-log.md](docs/progress-log.md) for a running log of what's
|
||||||
|
been done and what's next.
|
||||||
|
|
||||||
|
> **Status**: directory scaffold only — no business logic implemented yet.
|
||||||
|
> See the build roadmap in `docs/architecture.md` for the phased plan.
|
||||||
|
|
||||||
|
## Directory map
|
||||||
|
|
||||||
|
```
|
||||||
|
apps/
|
||||||
|
web/ Next.js frontend
|
||||||
|
ai-service/ Python FastAPI — RAG orchestration + OpenAI calls
|
||||||
|
api-gateway/ NestJS — public entry point, routes to internal services
|
||||||
|
auth-service/ NestJS — signup/login/JWT
|
||||||
|
user-service/ NestJS — profile/preferences
|
||||||
|
chat-service/ NestJS — chat session + message history
|
||||||
|
mobile/ reserved for a future mobile app
|
||||||
|
packages/
|
||||||
|
shared-types/ TS DTOs shared across Node services + web
|
||||||
|
api-client/ typed HTTP client for web
|
||||||
|
ui/ shared React components
|
||||||
|
config/ shared eslint/tsconfig presets
|
||||||
|
ingestion/ offline batch pipeline: PDF -> monographs -> chunks -> embeddings -> Qdrant
|
||||||
|
infra/ docker-compose, k8s/Helm, Terraform, CI
|
||||||
|
docs/ architecture docs and ADRs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prerequisites (once implementation starts)
|
||||||
|
|
||||||
|
- Node.js + pnpm (JS workspace: `apps/web`, `apps/api-gateway`, `apps/auth-service`,
|
||||||
|
`apps/user-service`, `apps/chat-service`, `packages/*`)
|
||||||
|
- Python 3.11+ (`apps/ai-service`, `ingestion`)
|
||||||
|
- Docker (local Postgres/Qdrant/Redis via `infra/docker/docker-compose.yml`)
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# ai-service
|
||||||
|
|
||||||
|
Python/FastAPI. RAG orchestration: embed query -> vector search in Qdrant ->
|
||||||
|
build grounded prompt -> call OpenAI chat completion -> return answer +
|
||||||
|
citations. Stateless — does not own chat history itself.
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
[project]
|
||||||
|
name = "ai-service"
|
||||||
|
version = "0.0.0"
|
||||||
|
description = "RAG orchestration + OpenAI calls for the Duoc Thu medical chatbot"
|
||||||
|
requires-python = ">=3.11"
|
||||||
|
dependencies = []
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# api-gateway
|
||||||
|
|
||||||
|
NestJS. Single public entry point: request routing, JWT validation, rate
|
||||||
|
limiting. Routes to `auth-service`, `user-service`, `chat-service` over
|
||||||
|
internal REST. Frontend and clients only ever talk to this service.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/api-gateway",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# auth-service
|
||||||
|
|
||||||
|
NestJS. User signup/login, password hashing, JWT issuance/refresh. Owns the
|
||||||
|
users/credentials table in Postgres. No dependency on other services.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/auth-service",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
# chat-service
|
||||||
|
|
||||||
|
NestJS. Chat session lifecycle and message history persistence (Postgres).
|
||||||
|
Calls `ai-service` synchronously per user message to get the assistant reply,
|
||||||
|
then persists both turns.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/chat-service",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# mobile
|
||||||
|
|
||||||
|
Reserved for a future mobile app (React Native/Flutter, TBD). Not built yet —
|
||||||
|
placeholder only so the monorepo layout doesn't need restructuring later.
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# user-service
|
||||||
|
|
||||||
|
NestJS. Profile data, preferences (language, saved drugs), account settings.
|
||||||
|
Owns profile tables in Postgres, called by the gateway.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/user-service",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# web
|
||||||
|
|
||||||
|
Next.js frontend. Chat UI, auth pages, citation/disclaimer rendering, session
|
||||||
|
history. Talks only to `api-gateway` — never calls internal services directly.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/web",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,45 @@
|
|||||||
|
# ADR 0001: Use Qdrant as the vector database
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The RAG pipeline needs a vector store for drug-monograph chunks. The main
|
||||||
|
alternative considered was **pgvector** (a Postgres extension), which would
|
||||||
|
let us reuse the Postgres instance already needed for users/chat history —
|
||||||
|
one fewer moving part to operate.
|
||||||
|
|
||||||
|
The corpus is not free-flowing prose: it's a structured per-drug reference
|
||||||
|
with rich per-chunk metadata (drug name, section type, page range). The
|
||||||
|
common retrieval pattern this domain calls for is "vector similarity search,
|
||||||
|
filtered by metadata" — e.g. "search only within chỉ định sections" or
|
||||||
|
"filter to a specific drug the user named" combined with the semantic query.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Use **Qdrant** as a dedicated vector database, separate from Postgres.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
- Qdrant gives first-class combined payload-filtering + ANN search in a
|
||||||
|
single query, which is exactly the retrieval pattern this structured
|
||||||
|
corpus needs — pgvector supports filtering too, but it's a less natural
|
||||||
|
fit layered on top of a general-purpose relational engine.
|
||||||
|
- Vector search becomes its own independent scaling axis, separate from the
|
||||||
|
transactional Postgres workload (users/chat) — re-indexing or re-ingesting
|
||||||
|
the formulary doesn't contend with transactional traffic.
|
||||||
|
- Mature standalone Docker image for local dev, a well-supported Python
|
||||||
|
client, and a Helm chart for the production Kubernetes deployment target.
|
||||||
|
- Corpus size (tens of thousands of chunks) is trivial for Qdrant's HNSW
|
||||||
|
indexing.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- One additional service to operate/deploy/monitor compared to pgvector
|
||||||
|
(which would ride on the existing Postgres).
|
||||||
|
- Revisit if operational overhead becomes a real burden at our actual scale,
|
||||||
|
or if we want tighter transactional consistency between chat data and
|
||||||
|
retrieval — pgvector remains a viable fallback documented here for that
|
||||||
|
case.
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
# ADR 0002: Use the team's existing ArgoCD instance for deployment (GitOps)
|
||||||
|
|
||||||
|
## Status
|
||||||
|
|
||||||
|
Accepted
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Phase 6 of the build roadmap needs a way to actually deploy the Helm chart to
|
||||||
|
Kubernetes across dev/staging/prod. The original scaffold (`infra/ci/github-actions/deploy-cd.yml`)
|
||||||
|
assumed a push-based CI deploy step (CI runs `helm upgrade`/`kubectl apply`
|
||||||
|
directly against the cluster). The team already runs an ArgoCD instance used
|
||||||
|
by other projects.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Deploy via **GitOps through the team's existing ArgoCD instance** instead of
|
||||||
|
building a custom push-based CD pipeline. ArgoCD Applications
|
||||||
|
(`infra/argocd/applications/{dev,staging,prod}/app.yaml`) point at
|
||||||
|
`infra/helm/medical-chatbot` in this repo; ArgoCD watches the repo and
|
||||||
|
reconciles the cluster to match.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
- Reuses infrastructure the team already operates and trusts, instead of
|
||||||
|
standing up a parallel deploy mechanism.
|
||||||
|
- GitOps gives an auditable history of every deploy (it's just git commits
|
||||||
|
changing values files/image tags) and a built-in rollback path (revert the
|
||||||
|
commit).
|
||||||
|
- Removes the need for CI to hold cluster credentials — CI's job shrinks to
|
||||||
|
"build, test, push image, bump tag," which is a smaller security surface
|
||||||
|
than "CI can directly mutate the production cluster."
|
||||||
|
- Prod uses a non-automated `syncPolicy` (manual approval in ArgoCD) while
|
||||||
|
dev/staging auto-sync, matching normal caution around production changes.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- CI workflows (`infra/ci/github-actions/*.yml`) build/test/push images and
|
||||||
|
bump the relevant `values-<env>.yaml` image tag + push that commit; they do
|
||||||
|
**not** call `kubectl`/`helm` against any cluster directly.
|
||||||
|
- Actual deploy execution and health/sync status live in the team's ArgoCD
|
||||||
|
UI/CLI, outside this repo — runbooks in `docs/runbooks/` should document how
|
||||||
|
to check sync status and roll back once the team's ArgoCD instance details
|
||||||
|
(cluster/server, project, repo URL) are confirmed (see TODOs in
|
||||||
|
`infra/argocd/README.md`).
|
||||||
|
- If the team's ArgoCD instance becomes unavailable or this project needs to
|
||||||
|
fully own its own deploy tooling later, the push-based `deploy-cd.yml`
|
||||||
|
approach remains a documented fallback.
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
# Architecture — Dược Thư RAG Medical Chatbot
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
A medical chatbot grounded in the Vietnamese National Drug Formulary (Dược
|
||||||
|
thư quốc gia Việt Nam 2018), built as a microservices monorepo. Users ask
|
||||||
|
drug-related questions through a web chat UI; answers are generated via
|
||||||
|
retrieval-augmented generation (RAG) over the formulary content, always
|
||||||
|
citing the source drug monograph/section, and always carrying a medical
|
||||||
|
disclaimer.
|
||||||
|
|
||||||
|
## Service responsibilities & communication
|
||||||
|
|
||||||
|
| Service | Owns | Talks to |
|
||||||
|
|---|---|---|
|
||||||
|
| **api-gateway** (NestJS) | Single public entry point; request routing, JWT validation, rate limiting | Routes to auth-service, user-service, chat-service, ai-service over internal REST |
|
||||||
|
| **auth-service** (NestJS) | Signup/login, password hashing, JWT issuance/refresh | Postgres (users); no dependency on other services |
|
||||||
|
| **user-service** (NestJS) | Profile data, preferences, account settings | Postgres (profiles), called by gateway |
|
||||||
|
| **chat-service** (NestJS) | Chat session lifecycle, message history persistence | Postgres (chat_sessions, chat_messages); calls ai-service per user message, persists both turns |
|
||||||
|
| **ai-service** (Python/FastAPI) | RAG orchestration: embed query → vector search in Qdrant → build grounded prompt → call OpenAI → return answer + citations | Qdrant (vector search), OpenAI API; stateless, does not own chat history |
|
||||||
|
| **ingestion** (Python, offline batch) | One-time/periodic job: parse PDF → monographs → chunks → embeddings → upsert to Qdrant | Qdrant (write), OpenAI embeddings API; runs as CLI/CI/k8s Job, never in the live request path |
|
||||||
|
| **web** (Next.js) | Chat UI, auth UI, citation/disclaimer rendering, session list | Calls api-gateway only |
|
||||||
|
|
||||||
|
**Sync vs async**: the live chat path (web → gateway → chat-service →
|
||||||
|
ai-service → Qdrant + OpenAI → back) is synchronous request/response.
|
||||||
|
Ingestion is fully decoupled, offline, batch — it populates Qdrant ahead of
|
||||||
|
time and is never triggered by a chat request, since parsing the 37MB PDF and
|
||||||
|
embedding thousands of chunks takes minutes. Internal protocol is REST/JSON
|
||||||
|
for v1; a future gRPC migration is a documented option (see ADRs), not
|
||||||
|
needed now.
|
||||||
|
|
||||||
|
## Data stores
|
||||||
|
|
||||||
|
- **Vector DB: Qdrant.** Chosen over pgvector because retrieval quality here
|
||||||
|
depends on metadata-filtered ANN search (filter by drug name / section type
|
||||||
|
combined with vector similarity) over a highly structured corpus — Qdrant
|
||||||
|
makes that a first-class, single query. It also scales independently from
|
||||||
|
the transactional Postgres and has a mature Helm chart for the production
|
||||||
|
k8s target. See `docs/adr/0001-vector-db-qdrant.md`.
|
||||||
|
- **Relational DB: PostgreSQL.** One instance, logically separated per
|
||||||
|
service (users/credentials, profiles, chat sessions+messages).
|
||||||
|
- **Redis.** Session/refresh-token cache, rate-limit counters, and reserved
|
||||||
|
as the future job-queue backend (BullMQ/Celery) if async admin-triggered
|
||||||
|
re-ingestion or background jobs are added later.
|
||||||
|
|
||||||
|
## RAG ingestion pipeline (PDF-specific)
|
||||||
|
|
||||||
|
The formulary is a structured per-drug reference, not free prose — the
|
||||||
|
pipeline exploits that structure instead of naive fixed-size chunking:
|
||||||
|
|
||||||
|
1. **Extraction**: PyMuPDF (`fitz`) as primary extractor (font size/style/
|
||||||
|
position metadata enables heading detection); pdfplumber as a fallback
|
||||||
|
specifically for tabular content (dosing/interaction tables). Raw
|
||||||
|
per-page extraction is persisted to `ingestion/data/interim/` so
|
||||||
|
re-segmentation doesn't require re-running the expensive extraction step.
|
||||||
|
2. **Segmentation**: detect drug-entry boundaries (prefer the PDF's
|
||||||
|
bookmark/outline via `doc.get_toc()` when present, else font-size/style
|
||||||
|
heuristics), then classify each heading against a canonical section
|
||||||
|
taxonomy (`chi_dinh`, `chong_chi_dinh`, `lieu_dung`, `tac_dung_phu`,
|
||||||
|
`tuong_tac_thuoc`, etc., Vietnamese diacritic-insensitive matching).
|
||||||
|
Output: `{drug_id, drug_name, source_page_range, sections: {...}}` per
|
||||||
|
drug, persisted to `ingestion/data/processed/monographs.jsonl` and
|
||||||
|
manually spot-checked via `ingestion/notebooks/`.
|
||||||
|
3. **Chunking**: each `(drug, section)` pair is the natural chunk unit;
|
||||||
|
never split a section unless it exceeds a token budget (~500-800 tokens),
|
||||||
|
in which case sub-chunk with a sliding window (400 tokens, 50 overlap),
|
||||||
|
tagging the same drug+section metadata plus `part_index`. Every chunk
|
||||||
|
carries `drug_name`, `section_type`, `source_page_range`, `chunk_id` as
|
||||||
|
Qdrant payload — this is what makes citations possible.
|
||||||
|
4. **Embedding + load**: OpenAI `text-embedding-3-small` in batches, upserted
|
||||||
|
into a versioned Qdrant collection (`drug_monographs_v1`) keyed by
|
||||||
|
`chunk_id` for idempotent re-runs; collection aliasing allows re-ingesting
|
||||||
|
with a changed chunking strategy without downtime.
|
||||||
|
5. **Batch job, not synchronous**: runs as a CLI command locally, and as a
|
||||||
|
Kubernetes `Job`/`CronJob` in production — never inside the ai-service
|
||||||
|
request path.
|
||||||
|
|
||||||
|
## Safety / guardrails
|
||||||
|
|
||||||
|
- **System prompt** instructs the model to answer only from retrieved
|
||||||
|
context, never state a dosage/contraindication/interaction not present in
|
||||||
|
it, always append a disclaimer, and say "not found in the formulary"
|
||||||
|
rather than guess when retrieval is irrelevant.
|
||||||
|
- **Retrieval-confidence gate**: below a similarity threshold, skip the LLM
|
||||||
|
call entirely and return a canned "consult a professional" response.
|
||||||
|
- **Citations from metadata, not LLM prose**: the `citations` list is built
|
||||||
|
directly from retrieved-chunk metadata, independent of what the LLM says,
|
||||||
|
so the frontend can always show verifiable sources.
|
||||||
|
- **Disclaimer enforced at multiple layers**: system prompt + a
|
||||||
|
non-LLM-generated static string always appended to the API response + a
|
||||||
|
persistent, non-dismissible UI banner.
|
||||||
|
- **Scoped refusal**: out-of-scope questions (e.g. general symptom
|
||||||
|
diagnosis) get a scoped refusal directing to a professional, not an
|
||||||
|
ungrounded general-knowledge answer.
|
||||||
|
|
||||||
|
## Build roadmap
|
||||||
|
|
||||||
|
1. **Ingestion pipeline + populated, queryable vector DB.** Done when a CLI
|
||||||
|
run populates Qdrant and a test script retrieves the correct
|
||||||
|
drug/section chunk for a sample query — no API, no LLM call yet.
|
||||||
|
2. **ai-service (FastAPI) wrapping RAG + OpenAI.** Done when a `curl` to
|
||||||
|
`/query` returns a grounded answer with a traceable citation and an
|
||||||
|
always-present disclaimer.
|
||||||
|
3. **auth/user/chat services + api-gateway.** Done when register → login →
|
||||||
|
chat message flows end-to-end through the gateway only, persisted in
|
||||||
|
Postgres.
|
||||||
|
4. **Next.js frontend chat UI.** Done when a browser user can log in, ask a
|
||||||
|
question, and see a grounded answer with citation + disclaimer banner.
|
||||||
|
5. **Containerize + docker-compose local.** Done when `docker compose up`
|
||||||
|
from a clean checkout brings up the full stack and the Phase 4 flow works.
|
||||||
|
6. **Kubernetes/Helm + Terraform + CI + ArgoCD (GitOps) deployment.** Done
|
||||||
|
when CI builds/tests/pushes an image and bumps the target environment's
|
||||||
|
Helm values file, the team's ArgoCD instance (see `infra/argocd/`,
|
||||||
|
`docs/adr/0002-argocd-gitops.md`) picks up the change and syncs the
|
||||||
|
cluster, and the Phase 4 flow works against the k8s-hosted stack. CI
|
||||||
|
never runs `kubectl`/`helm` directly against a cluster. Cloud provider
|
||||||
|
choice (AWS/GCP/Azure) only affects the Terraform module implementations,
|
||||||
|
not this repo's structure.
|
||||||
|
|
||||||
|
See `docs/adr/` for architecture decision records and `docs/runbooks/` for
|
||||||
|
operational runbooks (added as they're needed).
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
# Progress Log
|
||||||
|
|
||||||
|
Chronological record of work done on this project, newest entry on top. The
|
||||||
|
goal is continuity across sessions: if a work session ends unexpectedly
|
||||||
|
(context/token limit, interruption), whoever picks this up next — human or
|
||||||
|
Claude — should be able to read the latest entry and know exactly what's
|
||||||
|
done and what's next, without having to reconstruct it from git history.
|
||||||
|
|
||||||
|
**Convention**: add a new entry at the top before ending a session whenever
|
||||||
|
meaningful progress was made, and proactively the moment it looks like the
|
||||||
|
session might run out of context/tokens mid-task — don't wait until the very
|
||||||
|
end if that risk is showing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2026-07-30 — Initial monorepo scaffold
|
||||||
|
|
||||||
|
**Done:**
|
||||||
|
- Designed the microservices architecture (see `docs/architecture.md`):
|
||||||
|
Python/FastAPI `ai-service` for RAG, NestJS for `api-gateway`/`auth-service`/
|
||||||
|
`user-service`/`chat-service`, Next.js `web`, Qdrant for vectors, Postgres
|
||||||
|
for relational data, Redis reserved for caching/queues.
|
||||||
|
- Scaffolded the full monorepo directory tree (`apps/`, `packages/`,
|
||||||
|
`ingestion/`, `infra/`, `docs/`) with baseline config (package.json/
|
||||||
|
pyproject.toml stubs, pnpm workspace, docker-compose topology stub).
|
||||||
|
- Moved `duoc-thu-quoc-gia-viet-nam-2018.pdf` into `ingestion/data/raw/`.
|
||||||
|
- Decided vector DB: **Qdrant** over pgvector (`docs/adr/0001-vector-db-qdrant.md`).
|
||||||
|
- Decided deployment: GitOps via the **team's existing ArgoCD instance**,
|
||||||
|
not a custom push-based CD pipeline (`docs/adr/0002-argocd-gitops.md`,
|
||||||
|
`infra/argocd/`). CI's job is build/test/push image + bump the Helm values
|
||||||
|
image tag; ArgoCD does the actual sync.
|
||||||
|
- `git init` + initial commit (this scaffold).
|
||||||
|
- Created a private GitHub repo and pushed the initial commit.
|
||||||
|
|
||||||
|
**Not done yet / next up (Phase 1 of the build roadmap in `docs/architecture.md`):**
|
||||||
|
- No business logic exists yet anywhere — this was scaffold only.
|
||||||
|
- Phase 1: build the `ingestion/` pipeline for real (PDF extraction via
|
||||||
|
PyMuPDF, monograph/section segmentation, section-aware chunking, OpenAI
|
||||||
|
embeddings, Qdrant upsert) and validate retrieval quality via the
|
||||||
|
`ingestion/notebooks/` QA step.
|
||||||
|
- Still pending/TBD: which cloud provider (AWS/GCP/Azure) for Terraform
|
||||||
|
(`infra/terraform/README.md`), and the team's ArgoCD instance's actual
|
||||||
|
cluster/server + project details (`infra/argocd/README.md` TODOs).
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
# ArgoCD (GitOps deployment)
|
||||||
|
|
||||||
|
Deployment uses the **team's existing ArgoCD instance** (not self-hosted by
|
||||||
|
this project) rather than a custom push-based CD pipeline. See
|
||||||
|
`docs/adr/0002-argocd-gitops.md` for the rationale.
|
||||||
|
|
||||||
|
## Flow
|
||||||
|
|
||||||
|
1. CI (`infra/ci/github-actions/*-ci.yml`) builds and pushes a container image
|
||||||
|
per app on merge to main, then bumps that app's image tag in
|
||||||
|
`infra/helm/medical-chatbot/values-<env>.yaml` (or a per-app values file)
|
||||||
|
and pushes that commit back to the repo. CI never runs `kubectl apply` or
|
||||||
|
`helm upgrade` directly.
|
||||||
|
2. ArgoCD (team-managed, pointed at this repo) watches `infra/argocd/applications/<env>/`
|
||||||
|
and `infra/helm/medical-chatbot/`, detects the values-file change, and
|
||||||
|
syncs the cluster to match — this is the actual deploy step, owned by
|
||||||
|
ArgoCD, not by our CI.
|
||||||
|
3. Promotion between environments (dev -> staging -> prod) is a Git operation
|
||||||
|
(merge/PR that changes the target values file or image tag for that env),
|
||||||
|
not a manual `kubectl`/`helm` command.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `applications/dev/app.yaml`, `applications/staging/app.yaml`,
|
||||||
|
`applications/prod/app.yaml` — one ArgoCD `Application` CR per environment,
|
||||||
|
each pointing at this repo + the `infra/helm/medical-chatbot` chart with
|
||||||
|
that environment's values file.
|
||||||
|
|
||||||
|
## TODO once the team's ArgoCD instance details are known
|
||||||
|
|
||||||
|
- Fill in `spec.destination.server` (target cluster API server / context name)
|
||||||
|
in each `app.yaml` — currently a placeholder.
|
||||||
|
- Confirm which ArgoCD `project` (RBAC scoping) these Applications should
|
||||||
|
belong to, instead of the placeholder `default`.
|
||||||
|
- Confirm the repo URL placeholder in each `app.yaml` once the GitHub repo
|
||||||
|
exists (filled in as part of the initial scaffold commit/push).
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: medical-chatbot-dev
|
||||||
|
namespace: argocd
|
||||||
|
spec:
|
||||||
|
project: default # TODO: confirm the team's ArgoCD project/RBAC scope for this app
|
||||||
|
source:
|
||||||
|
repoURL: https://github.com/BaoVu2k4/vsf-duocthu.git # TODO: confirm once repo is created
|
||||||
|
targetRevision: main
|
||||||
|
path: infra/helm/medical-chatbot
|
||||||
|
helm:
|
||||||
|
valueFiles:
|
||||||
|
- values.yaml
|
||||||
|
- values-dev.yaml
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc # TODO: point at the team's target cluster/context
|
||||||
|
namespace: medical-chatbot-dev
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: medical-chatbot-prod
|
||||||
|
namespace: argocd
|
||||||
|
spec:
|
||||||
|
project: default # TODO: confirm the team's ArgoCD project/RBAC scope for this app
|
||||||
|
source:
|
||||||
|
repoURL: https://github.com/BaoVu2k4/vsf-duocthu.git # TODO: confirm once repo is created
|
||||||
|
targetRevision: main
|
||||||
|
path: infra/helm/medical-chatbot
|
||||||
|
helm:
|
||||||
|
valueFiles:
|
||||||
|
- values.yaml
|
||||||
|
- values-prod.yaml
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc # TODO: point at the team's target cluster/context
|
||||||
|
namespace: medical-chatbot-prod
|
||||||
|
syncPolicy: {} # intentionally NOT automated — prod sync requires manual approval in the ArgoCD UI/CLI
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
apiVersion: argoproj.io/v1alpha1
|
||||||
|
kind: Application
|
||||||
|
metadata:
|
||||||
|
name: medical-chatbot-staging
|
||||||
|
namespace: argocd
|
||||||
|
spec:
|
||||||
|
project: default # TODO: confirm the team's ArgoCD project/RBAC scope for this app
|
||||||
|
source:
|
||||||
|
repoURL: https://github.com/BaoVu2k4/vsf-duocthu.git # TODO: confirm once repo is created
|
||||||
|
targetRevision: main
|
||||||
|
path: infra/helm/medical-chatbot
|
||||||
|
helm:
|
||||||
|
valueFiles:
|
||||||
|
- values.yaml
|
||||||
|
- values-staging.yaml
|
||||||
|
destination:
|
||||||
|
server: https://kubernetes.default.svc # TODO: point at the team's target cluster/context
|
||||||
|
namespace: medical-chatbot-staging
|
||||||
|
syncPolicy:
|
||||||
|
automated:
|
||||||
|
prune: true
|
||||||
|
selfHeal: true
|
||||||
|
syncOptions:
|
||||||
|
- CreateNamespace=true
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# CI workflows (placeholder)
|
||||||
|
|
||||||
|
Not yet functional — filled in during Phase 6. Deployment is GitOps via the
|
||||||
|
team's existing ArgoCD instance (see `infra/argocd/` and
|
||||||
|
`docs/adr/0002-argocd-gitops.md`) — CI never runs `kubectl`/`helm` against a
|
||||||
|
cluster directly. Planned workflows:
|
||||||
|
|
||||||
|
- `ai-service-ci.yml` — lint/test/build/push image for `apps/ai-service`
|
||||||
|
- `node-services-ci.yml` — lint/test/build for the NestJS services
|
||||||
|
- `web-ci.yml` — lint/test/build for `apps/web`
|
||||||
|
- `ingestion-ci.yml` — lint/test for the `ingestion` pipeline
|
||||||
|
- `bump-image-tag.yml` — on image push, updates the image tag in the
|
||||||
|
relevant `infra/helm/medical-chatbot/values-<env>.yaml` and commits/pushes
|
||||||
|
that change; ArgoCD picks it up from there
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
# Local development topology. App services are commented out until their
|
||||||
|
# Dockerfiles exist (Phase 5) — infra services can be started standalone
|
||||||
|
# today for Phase 1 (ingestion) development, e.g.:
|
||||||
|
# docker compose up postgres qdrant redis
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: duoc_thu
|
||||||
|
POSTGRES_PASSWORD: duoc_thu
|
||||||
|
POSTGRES_DB: duoc_thu
|
||||||
|
ports:
|
||||||
|
- "5432:5432"
|
||||||
|
volumes:
|
||||||
|
- postgres-data:/var/lib/postgresql/data
|
||||||
|
|
||||||
|
qdrant:
|
||||||
|
image: qdrant/qdrant:latest
|
||||||
|
ports:
|
||||||
|
- "6333:6333"
|
||||||
|
- "6334:6334"
|
||||||
|
volumes:
|
||||||
|
- qdrant-data:/qdrant/storage
|
||||||
|
|
||||||
|
redis:
|
||||||
|
image: redis:7-alpine
|
||||||
|
ports:
|
||||||
|
- "6379:6379"
|
||||||
|
volumes:
|
||||||
|
- redis-data:/data
|
||||||
|
|
||||||
|
# ai-service:
|
||||||
|
# build: ../../apps/ai-service
|
||||||
|
# env_file: ../../apps/ai-service/.env
|
||||||
|
# ports: ["8000:8000"]
|
||||||
|
# depends_on: [qdrant]
|
||||||
|
#
|
||||||
|
# api-gateway:
|
||||||
|
# build: ../../apps/api-gateway
|
||||||
|
# env_file: ../../apps/api-gateway/.env
|
||||||
|
# ports: ["3000:3000"]
|
||||||
|
# depends_on: [auth-service, user-service, chat-service]
|
||||||
|
#
|
||||||
|
# auth-service:
|
||||||
|
# build: ../../apps/auth-service
|
||||||
|
# env_file: ../../apps/auth-service/.env
|
||||||
|
# depends_on: [postgres]
|
||||||
|
#
|
||||||
|
# user-service:
|
||||||
|
# build: ../../apps/user-service
|
||||||
|
# env_file: ../../apps/user-service/.env
|
||||||
|
# depends_on: [postgres]
|
||||||
|
#
|
||||||
|
# chat-service:
|
||||||
|
# build: ../../apps/chat-service
|
||||||
|
# env_file: ../../apps/chat-service/.env
|
||||||
|
# depends_on: [postgres, ai-service]
|
||||||
|
#
|
||||||
|
# web:
|
||||||
|
# build: ../../apps/web
|
||||||
|
# ports: ["3001:3000"]
|
||||||
|
# depends_on: [api-gateway]
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
postgres-data:
|
||||||
|
qdrant-data:
|
||||||
|
redis-data:
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
apiVersion: v2
|
||||||
|
name: medical-chatbot
|
||||||
|
description: Umbrella Helm chart for the Duoc Thu RAG medical chatbot platform
|
||||||
|
type: application
|
||||||
|
version: 0.0.0
|
||||||
|
appVersion: "0.0.0"
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
# dev environment overrides (TBD, Phase 6)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
# prod environment overrides (TBD, Phase 6)
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
# staging environment overrides (TBD, Phase 6)
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
# Base values — filled in during Phase 6. Overridden per-environment by
|
||||||
|
# values-dev.yaml / values-staging.yaml / values-prod.yaml.
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
# Terraform (cloud-agnostic scaffold)
|
||||||
|
|
||||||
|
**Cloud provider decision is pending** (AWS vs GCP vs Azure). Modules below are
|
||||||
|
placeholder interfaces — the concrete resource implementations (EKS/GKE/AKS,
|
||||||
|
managed Postgres, object storage, secrets manager) get filled in once the
|
||||||
|
provider is chosen, in Phase 6 of the build roadmap. Directory shape is
|
||||||
|
provider-agnostic so no restructuring is needed once decided.
|
||||||
|
|
||||||
|
- `modules/k8s-cluster` — managed Kubernetes cluster
|
||||||
|
- `modules/networking` — VPC/subnets/ingress networking
|
||||||
|
- `modules/managed-postgres` — managed Postgres instance
|
||||||
|
- `modules/object-storage` — bucket for PDF/raw artifacts, index backups
|
||||||
|
- `modules/secrets` — secrets manager integration
|
||||||
|
- `envs/{dev,staging,prod}` — per-environment root modules wiring the above
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
# ingestion
|
||||||
|
|
||||||
|
Offline batch pipeline (never part of the live `ai-service` request path):
|
||||||
|
extract -> segment -> chunk -> embed -> load. Parses
|
||||||
|
`data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf` into per-drug, per-section
|
||||||
|
chunks and upserts embeddings into Qdrant. Run via
|
||||||
|
`python -m ingestion.cli run --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf`
|
||||||
|
(once implemented).
|
||||||
Binary file not shown.
@@ -0,0 +1,10 @@
|
|||||||
|
[project]
|
||||||
|
name = "ingestion"
|
||||||
|
version = "0.0.0"
|
||||||
|
description = "Offline batch pipeline: PDF -> monographs -> chunks -> embeddings -> Qdrant"
|
||||||
|
requires-python = ">=3.11"
|
||||||
|
dependencies = []
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"name": "duoc-thu-rag",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0",
|
||||||
|
"packageManager": "pnpm@9.0.0",
|
||||||
|
"scripts": {
|
||||||
|
"build": "turbo run build",
|
||||||
|
"dev": "turbo run dev",
|
||||||
|
"lint": "turbo run lint",
|
||||||
|
"test": "turbo run test"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"turbo": "^2.0.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# api-client
|
||||||
|
|
||||||
|
Typed HTTP client for `web` to call `api-gateway`, built on `shared-types`.
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/api-client",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0",
|
||||||
|
"main": "src/index.ts"
|
||||||
|
}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
export {};
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# config
|
||||||
|
|
||||||
|
Shared ESLint/TSConfig/Prettier base configs reused across Node apps and packages.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/config",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0"
|
||||||
|
}
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2022",
|
||||||
|
"module": "commonjs",
|
||||||
|
"moduleResolution": "node",
|
||||||
|
"strict": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"declaration": true
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# shared-types
|
||||||
|
|
||||||
|
TypeScript DTOs and event payload contracts shared between the Node services
|
||||||
|
(`api-gateway`, `auth-service`, `user-service`, `chat-service`) and `web`.
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/shared-types",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0",
|
||||||
|
"main": "src/index.ts"
|
||||||
|
}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
export {};
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
# ui
|
||||||
|
|
||||||
|
Shared React components: chat bubble, citation card, disclaimer banner. Used
|
||||||
|
by `web` (and future `mobile` where applicable).
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
{
|
||||||
|
"name": "@duoc-thu/ui",
|
||||||
|
"private": true,
|
||||||
|
"version": "0.0.0",
|
||||||
|
"main": "src/index.ts"
|
||||||
|
}
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
export {};
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
packages:
|
||||||
|
- "apps/web"
|
||||||
|
- "apps/api-gateway"
|
||||||
|
- "apps/auth-service"
|
||||||
|
- "apps/user-service"
|
||||||
|
- "apps/chat-service"
|
||||||
|
- "packages/*"
|
||||||
+15
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"$schema": "https://turbo.build/schema.json",
|
||||||
|
"tasks": {
|
||||||
|
"build": {
|
||||||
|
"dependsOn": ["^build"],
|
||||||
|
"outputs": ["dist/**", ".next/**"]
|
||||||
|
},
|
||||||
|
"dev": {
|
||||||
|
"cache": false,
|
||||||
|
"persistent": true
|
||||||
|
},
|
||||||
|
"lint": {},
|
||||||
|
"test": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user