Remove corpus counts from chat chrome

This commit is contained in:
2026-08-10 17:26:58 +07:00
parent 46469468bb
commit 97cb6d16f4
31 changed files with 2192 additions and 424 deletions
+8 -9
View File
@@ -1,21 +1,20 @@
"""A per-request LLM-call budget — F-08.
`RagAgent.handle()` makes up to ~5 sequential Bedrock calls per turn
(understand, sufficiency, generate, up to 2 entailment retries) with no
aggregate deadline before this: each call is bounded only by its own fixed
provider timeout (`read_timeout=60` in `adapters/bedrock_converse.py`, times
up to 3 retries at "standard" backoff — worst case several minutes for one
stuck call, let alone five). Measured live 2026-08-07: a normal answerable
(understand, sufficiency, generate, entailment) with no aggregate deadline
before this. Each call now has at most two bounded provider attempts
(`read_timeout=20` in `adapters/bedrock_converse.py`) because this budget is
checked between calls and cannot cancel boto3 while it is already in flight.
Measured live 2026-08-07: a normal answerable
turn costs ~8-9s total; nothing bounds the pathological case.
Checked before each call, not wrapped around an already-running one — this
bounds how many MORE calls get a chance to start once time/calls run out. It
does not cancel a call already in flight past its own provider timeout; a
hard per-call cancellation would need cooperative cancellation support from
`adapters/bedrock_converse.py`'s boto3 client, a larger change than this
budget object alone. Still a real improvement: five calls each capable of
running to their own 60s+ limit, one after another, is the actual gap this
closes.
`adapters/bedrock_converse.py`'s boto3 client. The adapter-level timeout bounds
that residual gap; this object prevents any later call from starting after
the aggregate deadline.
"""
from __future__ import annotations