Add read-only production runtime audit

docs-legacy/14-data-stores.md

@@ -0,0 +1,92 @@
+# 14 — Data stores
+
+Detail on schema and indexing is in
+[07-indexing-and-storage.md](07-indexing-and-storage.md). This page covers
+operational shape: what is deployed, who touches it, and what is missing.
+
+## Deployed stores
+
+| Store | Image | Deployed in | Volume | Host port |
+|---|---|---|---|---|
+| Qdrant | `qdrant/qdrant:latest` | `docker-compose.prod.yml` | `qdrant-data` | none in prod; `6333`/`6334` in local dev |
+| PostgreSQL 16 | `postgres:16-alpine` | `docker-compose.prod.yml` | `postgres-data` | none in prod; `5432` in local dev |
+| Prometheus TSDB | `prom/prometheus:v3.3.0` | observability overlay | `prometheus-data` | `127.0.0.1:9090` |
+| Tempo | `grafana/tempo:2.7.2` | observability overlay | `tempo-data` | none |
+| Grafana | `grafana/grafana:11.5.2` | observability overlay | `grafana-data` | `127.0.0.1:3002` |
+| Caddy | `caddy:2-alpine` | prod | `caddy-data`, `caddy-config` | `80`, `443` |
+
+`qdrant/qdrant:latest` is an unpinned tag — a rebuild can silently move the
+Qdrant version underneath a loaded collection. Every other image is pinned.
+
+## Redis — declared, never used
+
+Redis appears in three places and is used by none of them:
+
+- `infra/docker/docker-compose.yml` (local dev) starts `redis:7-alpine`.
+- `infra/k8s/base/redis/` is an empty directory.
+- The pre-existing `docs/architecture.md` reserves it for session cache,
+  rate-limit counters and a future job queue.
+
+**No source file in the repository imports a Redis client**, and it is absent
+from `docker-compose.prod.yml` and from the Helm chart. `middleware.ts` names
+Redis as where its in-memory rate limiter *should* move when `web` scales past
+one replica.
+
+## Who touches what
+
+| Component | Qdrant | PostgreSQL | Local disk |
+|---|---|---|---|
+| `ai-service` startup | read (manifest, collection list) | — | reads `ENTITIES_PATH` JSON |
+| `ai-service` query path | read (scroll + query_points) | write trace, read/write conversation turns | — |
+| `ai-service` `/v1/rag/feedback` | — | upsert feedback | — |
+| `ingestion` load | create collection, create indexes, upsert, count | — | reads `chunks.jsonl`, reads/writes embedding cache |
+| `web` | — | — | reads the source PDF for `/api/pdf` |
+
+## Consistency and idempotency
+
+- **Qdrant writes are idempotent.** Point ids are `uuid5(namespace, chunk_id)`,
+  so re-loading the same corpus converges.
+- **Migrations are idempotent.** All four are `CREATE TABLE IF NOT EXISTS` /
+  `ADD COLUMN IF NOT EXISTS` / `CREATE INDEX IF NOT EXISTS`. There is no
+  migration-version table and no down-migration; `migrate.py` simply replays all
+  four every deploy.
+- **No transactions span stores.** A trace row and a Qdrant read are unrelated;
+  a failed trace write leaves the answer already returned.
+- **No cache layer.** The only cache in the system is the offline embedding
+  cache on disk. Query embeddings, retrieval results and generations are **not**
+  cached — every identical question re-pays for every model call.
+
+## Connection handling
+
+`adapters/postgres.py` opens a **new connection per call** with
+`connect_timeout=5` and no pool. Both classes document this as a known
+simplification (F-09: "a real pool, with startup-time lifecycle, is a further
+improvement not made here"). The timeout is load-bearing: an unreachable but
+non-refusing host otherwise hangs on the OS TCP timeout, which defeats the
+caller's fail-open `try/except` just as completely as no `try/except` at all.
+
+The Qdrant client is a single long-lived `QdrantClient(timeout=30)` built in
+`bootstrap.py`.
+
+## Backup, restore, retention
+
+| Concern | State |
+|---|---|
+| PostgreSQL backup | **Not found** — no dump job, no cron, no snapshot automation |
+| Qdrant backup | **Not found** in code; `ingestion/README.md` recommends snapshot + restore for moving a corpus, done manually |
+| EBS snapshots | Unverifiable from the repository |
+| `rag_conversation_turn` retention | **None** — append-only, grows without bound |
+| `rag_retrieval_trace` retention | **None** |
+| Prometheus retention | `7d` in Helm values; the Compose overlay sets no `--storage.tsdb.retention` flag, so the Prometheus default applies |
+| Tempo retention | `24h` in Helm values; Compose uses whatever `infra/docker/tempo/tempo.yml` specifies |
+
+## Data classification
+
+`rag_retrieval_trace.query_text` and `rag_conversation_turn.line` store the raw
+user turn. Because the product asks clinicians to supply patient context — age,
+weight, comorbidities, allergies, current medications, eGFR/CrCl, Child-Pugh,
+pregnancy status, lab values (`rag/clinical.py::PatientContext`) — those columns
+can contain clinical detail about a third party. There is no redaction, no
+encryption at rest beyond whatever the host volume provides, no access control
+on the database, and no retention limit. See
+[16-security.md](16-security.md#data-privacy).