Add read-only production runtime audit
This commit is contained in:
@@ -0,0 +1,204 @@
|
||||
# 23 — Local development
|
||||
|
||||
Every command below is taken from a file in the repository. Where a step is
|
||||
undocumented in the repo, that is stated rather than invented.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Tool | Version | Why |
|
||||
|---|---|---|
|
||||
| Python | ≥3.11 (the image uses 3.12) | `apps/ai-service/pyproject.toml` |
|
||||
| Node.js | 20 | `apps/web/Dockerfile` |
|
||||
| pnpm | 9.0.0 | `package.json` `packageManager` |
|
||||
| Docker + Compose | any recent | `infra/docker/docker-compose.yml` |
|
||||
| AWS credentials | optional | Only for live embedding/generation — **costs money** |
|
||||
|
||||
## 1. Clone and install
|
||||
|
||||
```bash
|
||||
git clone <repo> && cd VSF-DUOCTHU
|
||||
|
||||
# JavaScript workspace
|
||||
pnpm install
|
||||
|
||||
# Python — no lockfile exists; install the declared dependencies
|
||||
pip install fastapi httpx "psycopg[binary]" pydantic-settings qdrant-client uvicorn \
|
||||
prometheus-client opentelemetry-api opentelemetry-sdk \
|
||||
opentelemetry-exporter-otlp-proto-http boto3 pytest
|
||||
pip install -e ingestion # or add ingestion/ to PYTHONPATH
|
||||
```
|
||||
|
||||
> There is no `requirements.txt`, no Poetry/uv lockfile, and
|
||||
> `apps/ai-service` is not `pip install`-able (its flat module layout makes
|
||||
> setuptools reject it — the `Dockerfile` says so). The list above mirrors the
|
||||
> Dockerfile's inline install.
|
||||
|
||||
## 2. Start the infrastructure
|
||||
|
||||
```bash
|
||||
cd infra/docker
|
||||
docker compose up -d postgres qdrant
|
||||
# optional observability:
|
||||
docker compose up -d prometheus grafana tempo otel-collector
|
||||
```
|
||||
|
||||
Ports: PostgreSQL `5432`, Qdrant `6333`/`6334`, Prometheus `9090`, Grafana
|
||||
`3002` (anonymous admin, local only), Tempo `3200`, OTLP `4317`/`4318`.
|
||||
|
||||
The app services in that file are commented out; `ai-service` and `web` run on
|
||||
the host during development, which is why the local Prometheus config scrapes
|
||||
`host.docker.internal`.
|
||||
|
||||
## 3. Configure `ai-service`
|
||||
|
||||
Copy the maintained example, then edit the local file:
|
||||
|
||||
```bash
|
||||
cp apps/ai-service/.env.example apps/ai-service/.env
|
||||
```
|
||||
|
||||
`config.py` remains the authority; `.env.example` documents its code defaults.
|
||||
Two useful shapes:
|
||||
|
||||
**(a) Offline — no AWS, no corpus needed.** Everything except retrieval and
|
||||
generation works; `/v1/rag/query` returns 503.
|
||||
|
||||
```dotenv
|
||||
EMBEDDING_PROVIDER=disabled
|
||||
ANSWER_PROVIDER=disabled
|
||||
POSTGRES_DSN=postgresql://duoc_thu:duoc_thu@localhost:5432/duoc_thu
|
||||
```
|
||||
|
||||
**(b) Full local RAG — requires a loaded Qdrant collection *and* AWS Bedrock
|
||||
access (real spend).**
|
||||
|
||||
```dotenv
|
||||
EMBEDDING_PROVIDER=cohere-v4
|
||||
ANSWER_PROVIDER=bedrock-converse
|
||||
ANSWER_MODEL_ID=<a Bedrock model id you have access to>
|
||||
RERANK_ENABLED=true
|
||||
AWS_REGION=us-east-1
|
||||
QDRANT_URL=http://localhost:6333
|
||||
QDRANT_COLLECTION=duocthu_v1
|
||||
```
|
||||
|
||||
See [15-configuration.md](15-configuration.md) for every setting.
|
||||
|
||||
## 4. Apply migrations
|
||||
|
||||
```bash
|
||||
cd apps/ai-service
|
||||
python -m migrate # applies migrations/*.sql in sorted order, idempotent
|
||||
```
|
||||
|
||||
## 5. Get a corpus into Qdrant
|
||||
|
||||
`ai-service` **refuses to start** in mode (b) against a collection with no
|
||||
manifest. Three options:
|
||||
|
||||
- **Snapshot/restore an existing `duocthu_v1`** — `ingestion/README.md`
|
||||
recommends this for moving a corpus between machines: it is free and exact.
|
||||
- **Run the loader from the committed `chunks.jsonl`** — this re-embeds and
|
||||
**costs real Bedrock spend on a personal account**; `ingestion/README.md` says
|
||||
not to start a corpus run without explicit approval:
|
||||
|
||||
```bash
|
||||
cd ingestion
|
||||
python -m ingestion.load.run \
|
||||
--chunks data/processed/chunks.jsonl \
|
||||
--provider cohere-v4 \
|
||||
--collection duocthu_v1 \
|
||||
--qdrant-url http://localhost:6333
|
||||
```
|
||||
|
||||
- **Use mode (a)** and skip retrieval entirely.
|
||||
|
||||
## 6. Rebuild the corpus from the PDF (optional, no cloud cost)
|
||||
|
||||
```bash
|
||||
cd ingestion
|
||||
python -m ingestion.cli detect-tables --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf
|
||||
python -m ingestion.cli run --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf
|
||||
python -m ingestion.cli chunk --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf
|
||||
python -m ingestion.cli chunk-ready
|
||||
# diagnostics
|
||||
python -m ingestion.cli validate --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf
|
||||
python -m ingestion.cli coverage --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf
|
||||
python -m ingestion.cli residual-ink --pdf data/raw/duoc-thu-quoc-gia-viet-nam-2018.pdf --pages 200-210
|
||||
```
|
||||
|
||||
Defaults write to `data/processed/`. `detect-tables` is slow and its output is
|
||||
cached and reused.
|
||||
|
||||
## 7. Run the backend
|
||||
|
||||
```bash
|
||||
cd apps/ai-service
|
||||
uvicorn main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
> **Do not use `--reload` on Windows.** The reloader has been unreliable in this
|
||||
> project; restart the process after edits instead. Also check for an orphaned
|
||||
> process on port 8000 from a previous run before starting.
|
||||
|
||||
Docs at `http://localhost:8000/docs`.
|
||||
|
||||
## 8. Run the frontend
|
||||
|
||||
```bash
|
||||
cd apps/web
|
||||
AI_SERVICE_URL=http://localhost:8000 pnpm dev
|
||||
# or from the repo root: pnpm dev (turbo run dev)
|
||||
```
|
||||
|
||||
`http://localhost:3000`.
|
||||
|
||||
## 9. Run the tests
|
||||
|
||||
```bash
|
||||
cd ingestion && python -m pytest tests -q
|
||||
# → 277 passed, 12 skipped
|
||||
|
||||
cd apps/ai-service && EMBEDDING_PROVIDER=disabled python -m pytest tests -q
|
||||
# → 278 passed, 6 skipped
|
||||
```
|
||||
|
||||
Without `EMBEDDING_PROVIDER=disabled` (and with a `.env` present) collection
|
||||
fails because `tests/test_api.py` imports `main`, which builds the runtime and
|
||||
contacts Qdrant. See [18-testing.md](18-testing.md).
|
||||
|
||||
Integration tests against real datastores:
|
||||
|
||||
```bash
|
||||
cd apps/ai-service
|
||||
RUN_INTEGRATION=1 python -m pytest tests/test_live_datastores.py -q
|
||||
```
|
||||
|
||||
## 10. Manual evaluation against a running service
|
||||
|
||||
```bash
|
||||
cd apps/ai-service
|
||||
python scripts/run_manual_battery.py --help
|
||||
```
|
||||
|
||||
Posts each case in `evals/production_manual_60.jsonl` to a live endpoint and
|
||||
records full responses for human review ([19](19-rag-evaluation.md)).
|
||||
|
||||
## Windows notes
|
||||
|
||||
The project is developed on Windows and several practicalities are baked in:
|
||||
|
||||
- `ingestion/cli.py::main` calls `sys.stdout.reconfigure(encoding="utf-8")`
|
||||
because the console cannot print Vietnamese otherwise. For other scripts, set
|
||||
`PYTHONIOENCODING=utf-8`.
|
||||
- Long-running cloud jobs should be started in the background with flushed
|
||||
output rather than held in an interactive shell.
|
||||
- The repository root accumulates `.codex-*.log`/`.png` scratch files; they are
|
||||
untracked and safe to delete.
|
||||
|
||||
## Working conventions found in the repository
|
||||
|
||||
`coordination/` contains hand-off notes between two AI agents working this repo
|
||||
in parallel, including ownership claims per directory. If you see a claim file
|
||||
for a path you are about to edit, read it first — the convention is to claim
|
||||
ownership before editing shared files.
|
||||
Reference in New Issue
Block a user