Remove corpus counts from chat chrome
This commit is contained in:
@@ -1,21 +1,20 @@
|
||||
"""A per-request LLM-call budget — F-08.
|
||||
|
||||
`RagAgent.handle()` makes up to ~5 sequential Bedrock calls per turn
|
||||
(understand, sufficiency, generate, up to 2 entailment retries) with no
|
||||
aggregate deadline before this: each call is bounded only by its own fixed
|
||||
provider timeout (`read_timeout=60` in `adapters/bedrock_converse.py`, times
|
||||
up to 3 retries at "standard" backoff — worst case several minutes for one
|
||||
stuck call, let alone five). Measured live 2026-08-07: a normal answerable
|
||||
(understand, sufficiency, generate, entailment) with no aggregate deadline
|
||||
before this. Each call now has at most two bounded provider attempts
|
||||
(`read_timeout=20` in `adapters/bedrock_converse.py`) because this budget is
|
||||
checked between calls and cannot cancel boto3 while it is already in flight.
|
||||
Measured live 2026-08-07: a normal answerable
|
||||
turn costs ~8-9s total; nothing bounds the pathological case.
|
||||
|
||||
Checked before each call, not wrapped around an already-running one — this
|
||||
bounds how many MORE calls get a chance to start once time/calls run out. It
|
||||
does not cancel a call already in flight past its own provider timeout; a
|
||||
hard per-call cancellation would need cooperative cancellation support from
|
||||
`adapters/bedrock_converse.py`'s boto3 client, a larger change than this
|
||||
budget object alone. Still a real improvement: five calls each capable of
|
||||
running to their own 60s+ limit, one after another, is the actual gap this
|
||||
closes.
|
||||
`adapters/bedrock_converse.py`'s boto3 client. The adapter-level timeout bounds
|
||||
that residual gap; this object prevents any later call from starting after
|
||||
the aggregate deadline.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
Reference in New Issue
Block a user