Fix every real lint finding and drop degenerate splice fragments
This commit is contained in:
@@ -31,6 +31,16 @@ See `docs/pdf-parsing-outlier-catalog.md` and
|
||||
`docs/adr/0003-pdf-parsing-strategy.md` for the concrete track record this
|
||||
rule comes from.
|
||||
|
||||
**What "verified" means:** when reporting something as verified, state (1)
|
||||
the command/test/script/manual check that was run, (2) the exact input
|
||||
scope, (3) the expected invariant or acceptance condition, (4) the observed
|
||||
result, and (5) any part of the requested scope that was *not* covered.
|
||||
Distinct scopes (unit test, regression fixture, selected-page sample,
|
||||
selected monographs, all detected monographs, full 1668-page document) are
|
||||
not interchangeable — don't describe one as another. Avoid words like
|
||||
"fully verified", "complete", "all", "no data lost", or "production-ready"
|
||||
unless the checks actually performed support that literal claim.
|
||||
|
||||
## Real code follows Clean Code / Clean Architecture / SoC / DRY / SOLID
|
||||
|
||||
Applies to anything meant to be committed as part of the actual system
|
||||
@@ -70,3 +80,62 @@ checklist is what actually catches that, the same reasoning behind the
|
||||
where the *why* isn't obvious from the code itself) — matches the
|
||||
no-comments-unless-non-obvious style already used throughout this
|
||||
project's docs and ADRs.
|
||||
|
||||
## Preserve provenance
|
||||
|
||||
Every extracted or transformed unit must retain enough provenance to trace
|
||||
it back to the source document — depending on the data type, this may
|
||||
include document id, page number, source block/span id, bounding box,
|
||||
reading-order position, table id and row/column coordinates, formula
|
||||
source span, monograph id, section path, and extraction method/parser
|
||||
version.
|
||||
|
||||
**Why:** this is a medical reference book being turned into a chatbot's
|
||||
knowledge base — if an answer is wrong, being able to trace a chunk back to
|
||||
the exact page/span it came from is how it gets debugged and corrected.
|
||||
Normalized text that "looks right" is not the same guarantee as text that
|
||||
is traceable.
|
||||
|
||||
**How to apply:**
|
||||
- Don't discard provenance fields just because the normalized text appears
|
||||
correct — a text value that can't be traced back to its source is not a
|
||||
fully validated extraction result.
|
||||
- When adding a new pipeline stage or record type, carry existing
|
||||
provenance fields through rather than dropping them at the boundary.
|
||||
|
||||
## Investigation scripts are evidence tools, not production code
|
||||
|
||||
One-off investigation scripts (e.g. scanning the corpus to check a
|
||||
hypothesis) may optimize for speed, but they must:
|
||||
- be clearly named or located as temporary investigation code;
|
||||
- state or record the scope they scanned;
|
||||
- output enough information to reproduce or inspect the finding;
|
||||
- not be imported by production code, and not become the only
|
||||
implementation of a parsing rule;
|
||||
- not be cited as whole-document evidence unless they actually covered the
|
||||
whole document;
|
||||
- be deleted after their finding is captured in a regression test, fixture,
|
||||
ADR, or the outlier catalog.
|
||||
|
||||
When an investigation uncovers a real parsing rule, move that rule into the
|
||||
production implementation and validate both the production code and the
|
||||
regression fixture — per [[DRY]] above, the rule should end up living in
|
||||
exactly one place.
|
||||
|
||||
## Definition of done
|
||||
|
||||
A task is not complete merely because code was written. Before reporting
|
||||
completion:
|
||||
- run the most relevant available tests and validation commands, and
|
||||
report the exact commands/checks run and whether each passed or failed;
|
||||
- state the validation scope (see "What 'verified' means" above);
|
||||
- add or update a regression fixture for each parser bug fixed;
|
||||
- confirm intended provenance fields remain present;
|
||||
- check for silent loss of expected monographs, sections, tables, formulas,
|
||||
or source references when the task could affect them;
|
||||
- avoid whole-document claims when only sample validation was performed;
|
||||
- list anything not tested, not measured, blocked, or still uncertain.
|
||||
|
||||
If only part of the task is complete, report the completed and incomplete
|
||||
parts separately — don't hide failing tests, unexpected counts, incomplete
|
||||
coverage, or contradictory evidence to present a cleaner status.
|
||||
|
||||
Reference in New Issue
Block a user