Instructions for Claude working in this repo

Never fabricate, never bluff

Do not state a number, a test result, a "verified" claim, or a capability estimate unless it is backed by something you actually ran or actually read. If you haven't checked something, say so explicitly ("not verified yet", "estimate, not measured") instead of presenting a guess as fact.

Why: this project involves parsing a medical reference book into a chatbot's knowledge base — false confidence here is not a cosmetic bug, it propagates into medical answers. During the ingestion-strategy investigation, a size-based heading threshold silently dropped ~15% of real monographs before whole-document validation caught it; a monograph-boundary scan was initially run on 1405 of 1668 pages before being caught and corrected. Confident-sounding claims that turn out wrong cost real rework and could cost real answer quality once this is live.

How to apply:

Prefer "I ran X and got Y" over "X should work" — run the check.
When asked something you don't know for certain (throughput estimates, whether a tool/library works on this environment, whether a heuristic holds at scale), say what's measured vs. estimated, explicitly.
Whole-document / whole-scope validation over small-sample claims — if the user states a total (e.g. "1668 pages"), any check must cover that literal total before being reported as done, not a convenient subset.
When a claim turns out wrong after fuller checking, say so plainly and show the corrected result — don't quietly smooth over the miss.

See docs/pdf-parsing-outlier-catalog.md and docs/adr/0003-pdf-parsing-strategy.md for the concrete track record this rule comes from.

What "verified" means: when reporting something as verified, state (1) the command/test/script/manual check that was run, (2) the exact input scope, (3) the expected invariant or acceptance condition, (4) the observed result, and (5) any part of the requested scope that was not covered. Distinct scopes (unit test, regression fixture, selected-page sample, selected monographs, all detected monographs, full 1668-page document) are not interchangeable — don't describe one as another. Avoid words like "fully verified", "complete", "all", "no data lost", or "production-ready" unless the checks actually performed support that literal claim.

Real code follows Clean Code / Clean Architecture / SoC / DRY / SOLID

Applies to anything meant to be committed as part of the actual system (apps/*, ingestion/*, packages/*) — not throwaway investigation scripts (e.g. a one-off scan to check a hypothesis), which may stay quick and disposable as long as they're never confused for production code and get deleted once their finding is written down.

Why this is a written rule, not just an intention: intentions from one conversation don't carry into the next session, and under time pressure or mid-refactor it's easy to let a principle slip without noticing — a written checklist is what actually catches that, the same reasoning behind the "never fabricate" rule above.

How to apply, concretely, in this repo:

SoC: keep the ingestion/ pipeline stages (extract/, segment/, chunk/, embed/, load/) genuinely independent — extraction code must not know about chunking, chunking must not call OpenAI, etc.
DRY: shared logic (e.g. the bold-span heading/boundary detector) lives in exactly one module that both the real pipeline and any validation script import — never re-implemented per script, which is what happened during exploratory investigation and is fine there, but must not carry into real code.
SOLID: single-responsibility modules/classes (a detector detects, it doesn't also chunk); open/closed section taxonomy (adding a new section label — e.g. a field like "Tên thương mại" not in the book's own documented list — must not require editing existing matching code, only adding an entry); dependency inversion at infrastructure boundaries (ai-service's domain/retrieval logic depends on an interface, not a hard import of the Qdrant SDK or OpenAI client directly, so it stays testable without live services).
Clean Architecture: domain/business logic (parsing rules, chunking rules, retrieval/grounding logic) stays independent of infrastructure (OpenAI SDK, Qdrant client, filesystem, NestJS framework details) so it's testable in isolation.
Clean Code: meaningful names, small functions, minimal comments (only where the why isn't obvious from the code itself) — matches the no-comments-unless-non-obvious style already used throughout this project's docs and ADRs.

Preserve provenance

Every extracted or transformed unit must retain enough provenance to trace it back to the source document — depending on the data type, this may include document id, page number, source block/span id, bounding box, reading-order position, table id and row/column coordinates, formula source span, monograph id, section path, and extraction method/parser version.

Why: this is a medical reference book being turned into a chatbot's knowledge base — if an answer is wrong, being able to trace a chunk back to the exact page/span it came from is how it gets debugged and corrected. Normalized text that "looks right" is not the same guarantee as text that is traceable.

How to apply:

Don't discard provenance fields just because the normalized text appears correct — a text value that can't be traced back to its source is not a fully validated extraction result.
When adding a new pipeline stage or record type, carry existing provenance fields through rather than dropping them at the boundary.

Investigation scripts are evidence tools, not production code

One-off investigation scripts (e.g. scanning the corpus to check a hypothesis) may optimize for speed, but they must:

be clearly named or located as temporary investigation code;
state or record the scope they scanned;
output enough information to reproduce or inspect the finding;
not be imported by production code, and not become the only implementation of a parsing rule;
not be cited as whole-document evidence unless they actually covered the whole document;
be deleted after their finding is captured in a regression test, fixture, ADR, or the outlier catalog.

When an investigation uncovers a real parsing rule, move that rule into the production implementation and validate both the production code and the regression fixture — per DRY above, the rule should end up living in exactly one place.

Definition of done

A task is not complete merely because code was written. Before reporting completion:

run the most relevant available tests and validation commands, and report the exact commands/checks run and whether each passed or failed;
state the validation scope (see "What 'verified' means" above);
add or update a regression fixture for each parser bug fixed;
confirm intended provenance fields remain present;
check for silent loss of expected monographs, sections, tables, formulas, or source references when the task could affect them;
avoid whole-document claims when only sample validation was performed;
list anything not tested, not measured, blocked, or still uncertain.

If only part of the task is complete, report the completed and incomplete parts separately — don't hide failing tests, unexpected counts, incomplete coverage, or contradictory evidence to present a cleaner status.

7.4 KiB Raw Blame History