ADR 0009: No RAG framework — hand-written orchestration behind ports

Status

Accepted. Recorded retrospectively during the 2026-08-12 documentation pass: the decision is unambiguous in the implementation, but no ADR existed for it.

Context

The system performs retrieval-augmented generation with query understanding, multiple retrieval strategies, reranking, prompt construction, structured output parsing, and post-generation verification — the exact feature set LangChain and LlamaIndex exist to provide.

Decision

Neither framework is used. There is no RAG or agent library of any kind.

Verifiable from the repository:

apps/ai-service/pyproject.toml declares six runtime dependencies: fastapi, httpx, psycopg, pydantic-settings, qdrant-client, uvicorn. Optional extras add prometheus-client, anthropic and three OpenTelemetry packages.
apps/ai-service/Dockerfile installs that set plus boto3.
No file imports langchain, llama_index, haystack or any equivalent.

Instead:

Orchestration is a plain class with an explicit branch table (rag/agent.py::_route).
Prompts are module-level constants with JSON schemas (rag/prompt.py).
Providers are injected through typing.Protocols (rag/ports.py) and implemented in adapters/, which is the only package importing an SDK — and always lazily, inside a method.
bootstrap.py is the single composition root.

Consequences

Enabled by this choice

rag/ imports no SDK, so the entire domain — including every safety check — is unit-testable offline with stub objects. All 278 ai-service tests run in 2.6 s with no network.
Behaviour is inspectable: the retrieval route for a given turn is a readable if chain, not framework dispatch.
Failure semantics are chosen per call site. The fail-closed/fail-open asymmetry in 02-system-architecture.md is deliberate and would be hard to express through a framework's uniform error handling.
Prompt text is reviewable as domain policy in one file, and swapping providers cannot silently change what the model was told.

Costs

Retrieval strategies, rank fusion, context packing and evaluation harnesses are all hand-written. Two of them (fusion.py, expansion.py) were written and never wired (27-technical-debt.md).
Optional retriever capabilities are discovered with getattr rather than declared, so the real interface is wider than ports.py documents (D-14).
No community tooling for tracing, caching or evaluation applies; the observability layer is bespoke.

Rationale

Partially recoverable. The code does not state "we chose not to use a framework", but the ports-and-adapters discipline is documented repeatedly in module docstrings, and one of them makes the intent explicit — rag/understanding.py:

rag/ imports no SDK: the LLM is injected as a JsonLlm protocol … and a deterministic stub runs the whole path offline in tests.

rag/prompt.py gives the parallel reason for prompts:

This is domain policy, not infrastructure … it lives here so it can be read, reviewed and tested without an SDK, and so swapping the provider cannot silently change what the model was told.

The consistent theme is testability and reviewability of the safety layer. Whether cost, lock-in or framework maturity also weighed in the decision is not recoverable from the repository.

3.5 KiB Raw Blame History