# 14 — Data stores

Detail on schema and indexing is in
[07-indexing-and-storage.md](07-indexing-and-storage.md). This page covers
operational shape: what is deployed, who touches it, and what is missing.

## Deployed stores

| Store | Image | Deployed in | Volume | Host port |
|---|---|---|---|---|
| Qdrant | `qdrant/qdrant:latest` | `docker-compose.prod.yml` | `qdrant-data` | none in prod; `6333`/`6334` in local dev |
| PostgreSQL 16 | `postgres:16-alpine` | `docker-compose.prod.yml` | `postgres-data` | none in prod; `5432` in local dev |
| Prometheus TSDB | `prom/prometheus:v3.3.0` | observability overlay | `prometheus-data` | `127.0.0.1:9090` |
| Tempo | `grafana/tempo:2.7.2` | observability overlay | `tempo-data` | none |
| Grafana | `grafana/grafana:11.5.2` | observability overlay | `grafana-data` | `127.0.0.1:3002` |
| Caddy | `caddy:2-alpine` | prod | `caddy-data`, `caddy-config` | `80`, `443` |

`qdrant/qdrant:latest` is an unpinned tag — a rebuild can silently move the
Qdrant version underneath a loaded collection. Every other image is pinned.

## Redis — declared, never used

Redis appears in three places and is used by none of them:

- `infra/docker/docker-compose.yml` (local dev) starts `redis:7-alpine`.
- `infra/k8s/base/redis/` is an empty directory.
- The pre-existing `docs/architecture.md` reserves it for session cache,
  rate-limit counters and a future job queue.

**No source file in the repository imports a Redis client**, and it is absent
from `docker-compose.prod.yml` and from the Helm chart. `middleware.ts` names
Redis as where its in-memory rate limiter *should* move when `web` scales past
one replica.

## Who touches what

| Component | Qdrant | PostgreSQL | Local disk |
|---|---|---|---|
| `ai-service` startup | read (manifest, collection list) | — | reads `ENTITIES_PATH` JSON |
| `ai-service` query path | read (scroll + query_points) | write trace, read/write conversation turns | — |
| `ai-service` `/v1/rag/feedback` | — | upsert feedback | — |
| `ingestion` load | create collection, create indexes, upsert, count | — | reads `chunks.jsonl`, reads/writes embedding cache |
| `web` | — | — | reads the source PDF for `/api/pdf` |

## Consistency and idempotency

- **Qdrant writes are idempotent.** Point ids are `uuid5(namespace, chunk_id)`,
  so re-loading the same corpus converges.
- **Migrations are idempotent.** All four are `CREATE TABLE IF NOT EXISTS` /
  `ADD COLUMN IF NOT EXISTS` / `CREATE INDEX IF NOT EXISTS`. There is no
  migration-version table and no down-migration; `migrate.py` simply replays all
  four every deploy.
- **No transactions span stores.** A trace row and a Qdrant read are unrelated;
  a failed trace write leaves the answer already returned.
- **No cache layer.** The only cache in the system is the offline embedding
  cache on disk. Query embeddings, retrieval results and generations are **not**
  cached — every identical question re-pays for every model call.

## Connection handling

`adapters/postgres.py` opens a **new connection per call** with
`connect_timeout=5` and no pool. Both classes document this as a known
simplification (F-09: "a real pool, with startup-time lifecycle, is a further
improvement not made here"). The timeout is load-bearing: an unreachable but
non-refusing host otherwise hangs on the OS TCP timeout, which defeats the
caller's fail-open `try/except` just as completely as no `try/except` at all.

The Qdrant client is a single long-lived `QdrantClient(timeout=30)` built in
`bootstrap.py`.

## Backup, restore, retention

| Concern | State |
|---|---|
| PostgreSQL backup | **Not found** — no dump job, no cron, no snapshot automation |
| Qdrant backup | **Not found** in code; `ingestion/README.md` recommends snapshot + restore for moving a corpus, done manually |
| EBS snapshots | Unverifiable from the repository |
| `rag_conversation_turn` retention | **None** — append-only, grows without bound |
| `rag_retrieval_trace` retention | **None** |
| Prometheus retention | `7d` in Helm values; the Compose overlay sets no `--storage.tsdb.retention` flag, so the Prometheus default applies |
| Tempo retention | `24h` in Helm values; Compose uses whatever `infra/docker/tempo/tempo.yml` specifies |

## Data classification

`rag_retrieval_trace.query_text` and `rag_conversation_turn.line` store the raw
user turn. Because the product asks clinicians to supply patient context — age,
weight, comorbidities, allergies, current medications, eGFR/CrCl, Child-Pugh,
pregnancy status, lab values (`rag/clinical.py::PatientContext`) — those columns
can contain clinical detail about a third party. There is no redaction, no
encryption at rest beyond whatever the host volume provides, no access control
on the database, and no retention limit. See
[16-security.md](16-security.md#data-privacy).