# 02 — System architecture

## Architectural style

**As built:** a two-service application (`web` + `ai-service`) plus an offline
batch pipeline, deployed as Docker Compose services on one host. Communication
is synchronous HTTP/JSON. There is no message broker, no queue, no async
worker, and no service mesh.

**As designed on paper:** a seven-service microservices platform
(`api-gateway`, `auth-service`, `user-service`, `chat-service`, `ai-service`,
`web`, `ingestion`), described in the pre-existing `docs/architecture.md`. Four
of those seven do not exist — their directories hold a `README.md` and a
four-line `package.json` with no `dependencies` and no source files. The
monorepo scaffolding (pnpm workspace entries, `infra/k8s/base/<service>/`
directories) still reserves their names.

Both facts matter: the second explains why `apps/`, `pnpm-workspace.yaml` and
the Helm chart look bigger than the running system.

## Component diagram — what actually runs

```mermaid
flowchart TB
    subgraph browser["Browser"]
        UI["Chat UI<br/>ChatPanel.tsx · 65s abort"]
    end

    subgraph ec2["EC2 host — docker compose"]
        CADDY["caddy:2-alpine<br/>:80 :443 · ACME TLS"]
        subgraph webc["web (Next.js 14, :3000)"]
            MW["middleware.ts<br/>in-memory IP rate limit"]
            BFF["/api/chat · /api/suggest<br/>/api/feedback · /api/pdf"]
        end
        subgraph aic["ai-service (FastAPI, :8000)"]
            HTTP["routers/rag.py"]
            AGENT["RagAgent"]
            RET["RetrievalService"]
            ANS["GroundedAnswerService"]
        end
        PG[("postgres:16-alpine")]
        QD[("qdrant/qdrant")]
        subgraph obs["observability overlay"]
            OTELC["otel-collector"]
            TEMPO["tempo"]
            PROM["prometheus"]
            GRAF["grafana :3002 (127.0.0.1)"]
        end
    end

    BEDROCK["AWS Bedrock<br/>embed-v4 · rerank-v3.5 · Converse"]

    UI -->|HTTPS| CADDY
    CADDY -->|"/*"| MW --> BFF
    CADDY -->|"/grafana/*"| GRAF
    BFF -->|"POST /v1/rag/query"| HTTP
    HTTP --> AGENT
    AGENT --> RET
    AGENT --> ANS
    RET --> QD
    ANS -->|generation + entailment| BEDROCK
    AGENT -->|understanding| BEDROCK
    RET -->|embed + rerank| BEDROCK
    HTTP --> PG
    AGENT --> PG
    aic -->|OTLP/HTTP| OTELC --> TEMPO
    PROM -->|scrape /metrics| aic
    GRAF --> PROM
    GRAF --> TEMPO
```

`ai-service` publishes **no host port** in `docker-compose.prod.yml`; it is
reachable only on the Compose network. Caddy proxies `web` and `grafana` only.

## Service boundaries

| Service | Owns | Depends on | Stateless? |
|---|---|---|---|
| `web` | Rendering, reason-code → Vietnamese message mapping, citation grouping, rate limiting | `ai-service` over HTTP; the source PDF on a bind-mounted path | **No** — rate-limit counters are per-process in memory |
| `ai-service` | Understanding, retrieval, generation, grounding, citations, traces | Qdrant, PostgreSQL, Bedrock | **Mostly** — `RagAgent` keeps two in-process dicts (`_last_frame`, `_clarify_streak`); conversation *text* is in PostgreSQL |
| `ingestion` | Turning the PDF into `chunks.jsonl` and Qdrant points | Qdrant, Bedrock, local disk | N/A — batch |

### The stateful detail that constrains scaling

`RagAgent` (`rag/agent.py`) holds three dicts:

```python
self._history: dict[str, list[str]]        # unused when a ConversationStore is configured
self._last_frame: dict[str, QueryFrame]    # ALWAYS in-process
self._clarify_streak: dict[str, int]       # ALWAYS in-process
```

`PostgresConversationStore` replaces `_history` only. `_last_frame` carries the
structured merge that stops the model re-asking an answered clarify question,
and `_clarify_streak` drives the clarify circuit breaker. Both are lost on
restart and **not shared between replicas**. Running more than one `ai-service`
replica therefore degrades multi-turn quality in a way nothing detects. The Helm
chart's `aiService.replicaCount` defaults to `1`; nothing enforces it.

## Module boundaries inside `ai-service`

Ports-and-adapters, enforced by import discipline rather than by tooling:

- `rag/ports.py` declares `Retriever`, `SectionRetriever`, `ParentStore`,
  `AnswerGenerator`, `Reranker`, plus three exception types
  (`QueryEmbeddingUnavailable`, `AnswerGenerationUnavailable`,
  `RerankUnavailable`) that adapters raise and the domain catches.
- `adapters/` is the only place `qdrant_client`, `boto3`, `psycopg` and
  `prometheus_client` are imported — and always **lazily**, inside a method, so
  the domain imports cleanly on a machine with none of them installed.
- `bootstrap.py` is the composition root. Nothing else constructs an adapter.

One boundary is looser than the protocol suggests: `RetrievalService` reaches
optional retriever capabilities with `getattr(self._retriever, "find_by_section",
None)` rather than through a declared protocol. `find_by_indication`,
`search_indication`, `search_lexical` and `find_by_drug` are all discovered this
way and none of them appear in `ports.py`. A retriever missing one silently
disables a whole route instead of failing a type check.

## Synchronous communication

Every hop is a blocking HTTP or SDK call. One answerable turn issues, in
sequence:

1. `POST /api/chat` (browser → web)
2. `POST /v1/rag/query` (web → ai-service)
3. Bedrock Converse — understanding
4. Qdrant `scroll`/`query_points` — retrieval (1–N calls)
5. Bedrock Converse — generation (plus one retry on `evidence_sufficient=false`)
6. Bedrock Converse — entailment
7. optional Bedrock Converse ×2 — completeness repair + its re-verification
8. PostgreSQL insert — trace

Measured production latencies recorded in `ChatPanel.tsx` (n=8, 2026-08-11):
6.2 / 6.4 / 8.4 / 10.9 / 12.4 / 21.7 / 25.1 / 40.3 seconds.

## Asynchronous communication

**Not found.** No broker, no queue, no background worker, no SSE, no
WebSocket, no streaming response. `web`'s `/api/chat` awaits the full upstream
response before replying.

## Failure boundaries

| Boundary | Policy | Implemented in |
|---|---|---|
| Corpus/model manifest mismatch at startup | **Fail closed, crash the process** | `bootstrap.py::_verify_corpus_manifest` → `rag/manifest.py` |
| Query embedder unreachable | **Fail closed** — abstain, never a 500 | `rag/service.py`, `rag/ports.py` |
| Generator unreachable / malformed / budget exhausted | **Fail closed** — abstain with a specific reason code | `rag/answer.py::_generate` |
| Understanding call fails | **Fail closed** — abstain, tagged `system_error` | `rag/understanding.py::understand` |
| Grounding or entailment rejects | **Fail closed** — abstain | `rag/answer.py` |
| Reranker unreachable | **Fail open** — keep original order | `rag/service.py::_rerank` |
| Sufficiency check unreachable | **Fail open** — proceed to generate | `rag/answer.py::_check_sufficiency` |
| PostgreSQL trace write fails | **Fail open** — answer returned, `TRACE_WRITE_FAILED` counter | `routers/rag.py` |
| Conversation store read/write fails | **Fail open** — this turn has no memory | `rag/agent.py::_get_history` / `_remember` |
| Metrics package missing | **Degrade** — `NullMetrics` | `bootstrap.py::_build_metrics` |
| OpenTelemetry packages missing / `OTEL_ENABLED=false` | **Degrade** — no-op tracer | `rag/telemetry.py` |

The asymmetry is deliberate and documented in-code: anything that could change
*what is stated* fails closed; anything that only affects quality or
observability fails open.

## Deployment units

| Unit | Image | Built by |
|---|---|---|
| `ai-service` | `apps/ai-service/Dockerfile` — `python:3.12-slim`, deps pinned inline (not from `pyproject.toml`) | `docker compose up --build` on the EC2 host |
| `web` | `apps/web/Dockerfile` — 3-stage node:20-slim, `pnpm --filter @duoc-thu/web build` | same |
| `postgres`, `qdrant`, `caddy`, `prometheus`, `tempo`, `grafana`, `otel-collector` | Upstream images | pulled |

There is **no container registry**. Images are built on the production host at
deploy time. The Helm chart assumes registry images
(`duocthu-ai-service:<tag>`) that nothing currently produces.

## Scaling implications

- `ai-service` is CPU-light and latency-bound on Bedrock. Horizontal scaling is
  blocked by the in-process `_last_frame`/`_clarify_streak` state above.
- `web`'s rate limiter is per-process; a second replica doubles the effective
  allowance. `middleware.ts` says so explicitly.
- Qdrant and PostgreSQL are single containers with named Docker volumes on one
  EBS-backed host. No replication, no backup job in the repository.
- The `evidence_limit`/`max_context_tokens` policy (`rag/service.py`,
  `EvidencePolicy`) bounds prompt size; nothing bounds concurrent Bedrock calls
  beyond the per-request budget.