duocthu/CLAUDE.md

# Instructions for Claude working in this repo

## Never fabricate, never bluff

Do not state a number, a test result, a "verified" claim, or a capability
estimate unless it is backed by something you actually ran or actually
read. If you haven't checked something, say so explicitly ("not verified
yet", "estimate, not measured") instead of presenting a guess as fact.

**Why:** this project involves parsing a medical reference book into a
chatbot's knowledge base — false confidence here is not a cosmetic bug, it
propagates into medical answers. During the ingestion-strategy
investigation, a size-based heading threshold silently dropped ~15% of real
monographs before whole-document validation caught it; a monograph-boundary
scan was initially run on 1405 of 1668 pages before being caught and
corrected. Confident-sounding claims that turn out wrong cost real rework
and could cost real answer quality once this is live.

**How to apply:**
- Prefer "I ran X and got Y" over "X should work" — run the check.
- When asked something you don't know for certain (throughput estimates,
  whether a tool/library works on this environment, whether a heuristic
  holds at scale), say what's measured vs. estimated, explicitly.
- Whole-document / whole-scope validation over small-sample claims — if the
  user states a total (e.g. "1668 pages"), any check must cover that literal
  total before being reported as done, not a convenient subset.
- When a claim turns out wrong after fuller checking, say so plainly and
  show the corrected result — don't quietly smooth over the miss.

See `docs/pdf-parsing-outlier-catalog.md` and
`docs/adr/0003-pdf-parsing-strategy.md` for the concrete track record this
rule comes from.

**What "verified" means:** when reporting something as verified, state (1)
the command/test/script/manual check that was run, (2) the exact input
scope, (3) the expected invariant or acceptance condition, (4) the observed
result, and (5) any part of the requested scope that was *not* covered.
Distinct scopes (unit test, regression fixture, selected-page sample,
selected monographs, all detected monographs, full 1668-page document) are
not interchangeable — don't describe one as another. Avoid words like
"fully verified", "complete", "all", "no data lost", or "production-ready"
unless the checks actually performed support that literal claim.

## Real code follows Clean Code / Clean Architecture / SoC / DRY / SOLID

Applies to anything meant to be committed as part of the actual system
(`apps/*`, `ingestion/*`, `packages/*`) — not throwaway investigation
scripts (e.g. a one-off scan to check a hypothesis), which may stay quick
and disposable as long as they're never confused for production code and
get deleted once their finding is written down.

**Why this is a written rule, not just an intention:** intentions from one
conversation don't carry into the next session, and under time pressure or
mid-refactor it's easy to let a principle slip without noticing — a written
checklist is what actually catches that, the same reasoning behind the
"never fabricate" rule above.

**How to apply, concretely, in this repo:**
- **SoC**: keep the `ingestion/` pipeline stages (`extract/`, `segment/`,
  `chunk/`, `embed/`, `load/`) genuinely independent — extraction code must
  not know about chunking, chunking must not call OpenAI, etc.
- **DRY**: shared logic (e.g. the bold-span heading/boundary detector) lives
  in exactly one module that both the real pipeline and any validation
  script import — never re-implemented per script, which is what happened
  during exploratory investigation and is fine there, but must not carry
  into real code.
- **SOLID**: single-responsibility modules/classes (a detector detects, it
  doesn't also chunk); open/closed section taxonomy (adding a new section
  label — e.g. a field like "Tên thương mại" not in the book's own
  documented list — must not require editing existing matching code, only
  adding an entry); dependency inversion at infrastructure boundaries
  (`ai-service`'s domain/retrieval logic depends on an interface, not a
  hard import of the Qdrant SDK or OpenAI client directly, so it stays
  testable without live services).
- **Clean Architecture**: domain/business logic (parsing rules, chunking
  rules, retrieval/grounding logic) stays independent of infrastructure
  (OpenAI SDK, Qdrant client, filesystem, NestJS framework details) so it's
  testable in isolation.
- **Clean Code**: meaningful names, small functions, minimal comments (only
  where the *why* isn't obvious from the code itself) — matches the
  no-comments-unless-non-obvious style already used throughout this
  project's docs and ADRs.

## Preserve provenance

Every extracted or transformed unit must retain enough provenance to trace
it back to the source document — depending on the data type, this may
include document id, page number, source block/span id, bounding box,
reading-order position, table id and row/column coordinates, formula
source span, monograph id, section path, and extraction method/parser
version.

**Why:** this is a medical reference book being turned into a chatbot's
knowledge base — if an answer is wrong, being able to trace a chunk back to
the exact page/span it came from is how it gets debugged and corrected.
Normalized text that "looks right" is not the same guarantee as text that
is traceable.

**How to apply:**
- Don't discard provenance fields just because the normalized text appears
  correct — a text value that can't be traced back to its source is not a
  fully validated extraction result.
- When adding a new pipeline stage or record type, carry existing
  provenance fields through rather than dropping them at the boundary.

## Investigation scripts are evidence tools, not production code

One-off investigation scripts (e.g. scanning the corpus to check a
hypothesis) may optimize for speed, but they must:
- be clearly named or located as temporary investigation code;
- state or record the scope they scanned;
- output enough information to reproduce or inspect the finding;
- not be imported by production code, and not become the only
  implementation of a parsing rule;
- not be cited as whole-document evidence unless they actually covered the
  whole document;
- be deleted after their finding is captured in a regression test, fixture,
  ADR, or the outlier catalog.

When an investigation uncovers a real parsing rule, move that rule into the
production implementation and validate both the production code and the
regression fixture — per [[DRY]] above, the rule should end up living in
exactly one place.

## Definition of done

A task is not complete merely because code was written. Before reporting
completion:
- run the most relevant available tests and validation commands, and
  report the exact commands/checks run and whether each passed or failed;
- state the validation scope (see "What 'verified' means" above);
- add or update a regression fixture for each parser bug fixed;
- confirm intended provenance fields remain present;
- check for silent loss of expected monographs, sections, tables, formulas,
  or source references when the task could affect them;
- avoid whole-document claims when only sample validation was performed;
- list anything not tested, not measured, blocked, or still uncertain.

If only part of the task is complete, report the completed and incomplete
parts separately — don't hide failing tests, unexpected counts, incomplete
coverage, or contradictory evidence to present a cleaner status.