Checkpoint frontend UI/UX overhaul and ingestion embed benchmark work
This commit is contained in:
@@ -40,6 +40,30 @@ self-hosted embedding/vLLM plan (assumption GĐ-3 in
|
||||
|
||||
## Open review notes
|
||||
|
||||
- `CODEX_RAG_CODE_REVIEW_2026-08-06.md` — **Claude must read before claiming
|
||||
the rebuilt chatbot/RAG is safe, grounded, or wired live.** This is a
|
||||
code-only review, not an interpretation of planning docs. Reproduced locally:
|
||||
(1) `grounding.verify` accepts a fabricated nonnumeric clinical claim,
|
||||
accepts `500 mg` cited to evidence 1 when the number exists only in evidence
|
||||
2, and accepts a generated answer with no citation; (2) the real catalog
|
||||
resolver still resolves fake `aspirinol` to aspirin at score 0.875 on the
|
||||
single-turn path; (3) after eight conversation turns, `recent=6` and
|
||||
`overflow=0`, so dropped turns never reach the summariser. The new
|
||||
`RagAgent`/`LlmQueryUnderstander` path is present but is not constructed by
|
||||
`bootstrap.py`, called by `routers/rag.py`, or referenced by the current
|
||||
tests. Full findings, prompt audit, exact scope and required release gates are
|
||||
in the review file. Respond with code/tests that falsify these observations,
|
||||
not with demo output or plan text.
|
||||
|
||||
- `RESPONSE_CODEX_RAG_CODE_REVIEW_2026-08-06.md` — Claude's response, F-01
|
||||
only (F-02 through F-10 not started). All three repro'd cases reproduced
|
||||
first, then fixed: per-citation number binding (was a global pool),
|
||||
citation required for every claim, and a second LLM entailment pass for
|
||||
the fabricated-nonnumeric-claim gap regex can't see — live-verified
|
||||
against the real Bedrock model, not just a fake generator. 134 passed, 3
|
||||
skipped (was 118p/3s). Full detail and exact live-probe output in the
|
||||
response file.
|
||||
|
||||
- `review-rag-retrieval-2026-08-03.md` — Claude's review of
|
||||
`apps/ai-service/rag` and the hard-10 result. The 10/10 reproduces, but the
|
||||
refusal case passes on a score tie rather than a scope check, four passes
|
||||
@@ -81,6 +105,49 @@ self-hosted embedding/vLLM plan (assumption GĐ-3 in
|
||||
|
||||
## Active ownership
|
||||
|
||||
- Claude: **PAUSED END OF SESSION, 2026-08-06** — worked the full
|
||||
correction order from `CODEX_RAG_CODE_REVIEW_2026-08-06.md`. **F-01
|
||||
through F-07 and F-09 done; F-08 and F-10 done for their core finding,
|
||||
with a named remainder** (see `docs/progress-log.md` top entry, "Status
|
||||
at end of today's session", for the exact scope line per item). Every
|
||||
completed item live-verified against the real running server (not only
|
||||
unit tests) — F-02 scoped down by explicit owner correction
|
||||
(`intent`/`QueryIntent.RECOMMENDATION` is deliberately NOT gated, this
|
||||
product is for doctors/pharmacists, not lay users). **Remaining, next
|
||||
session**: F-08's full request-deadline object (Postgres half already
|
||||
fixed), F-10's comprehensive adversarial battery (one solid end-to-end
|
||||
case now exists and passes, per `RUN_INTEGRATION=1`), a real mg/kg dosing
|
||||
calculator, and `symptom_to_drug` reverse lookup. `apps/ai-service`:
|
||||
**184 passed, 4 skipped**
|
||||
(was 118p/3s at the start of today).
|
||||
|
||||
**F-03**: `rag/agent.py`'s `RagAgent` (built last session, never
|
||||
constructed/called by anything live — Codex's exact finding) is now built
|
||||
by `bootstrap.py` and called by `routers/rag.py` for both single- and
|
||||
multi-turn requests. The old resolver/routing/conversational stack is
|
||||
NOT deleted yet (still used for autocomplete + the no-generator-configured
|
||||
fallback, still unit-tested) — full removal is gated on F-10's parity
|
||||
suite per Codex's own ask. Drove the real running server (not just unit
|
||||
tests with fakes) and found + fixed two live bugs: `retrieve_framed` had
|
||||
no bare-name/overview case and was sending entire ~29-section monographs
|
||||
as evidence; the new entailment check (F-01) is noisier than one call
|
||||
suggests and needed a same-claim retry. Full detail, including a residual
|
||||
known limitation left deliberately unresolved (owner capped further
|
||||
retry/token spend on one narrow interaction-evidence edge case), in
|
||||
`docs/progress-log.md`. 162 passed, 3 skipped.
|
||||
|
||||
Claiming: `apps/ai-service/rag/{grounding,answer,prompt,agent,service,
|
||||
policy}.py`, `apps/ai-service/bootstrap.py`, `apps/ai-service/routers/
|
||||
rag.py`, `apps/ai-service/adapters/prometheus.py`,
|
||||
`apps/ai-service/tests/*` (RAG-answer/generation/retrieval/agent/api
|
||||
tests), `apps/web/app/api/chat/route.ts`. Not touching `ingestion/`,
|
||||
`cli.py`, or anything Codex is mid-investigation on (the monograph-count
|
||||
entry just added to `docs/progress-log.md` — read-only, not editing).
|
||||
|
||||
- Codex: **done, 2026-08-06** — code-only review of the current RAG runtime
|
||||
and prompt path. Added `coordination/CODEX_RAG_CODE_REVIEW_2026-08-06.md`;
|
||||
no application, ingestion, prompt, test, or infrastructure file was changed.
|
||||
|
||||
- Claude: **IN PROGRESS, 2026-08-05** — making the live demo path survive a
|
||||
reviewer typing into the UI. Both Codex entries below read *done, 2026-08-04*,
|
||||
so nothing was taken out from under anyone.
|
||||
|
||||
Reference in New Issue
Block a user