Attach real input/output/token usage to provider spans for Langfuse cost tracking
Langfuse showed $0.00 cost on every trace even after tracing was wired up: provider.bedrock_converse.* spans carried only latency, no content or usage. Converse's own response already includes token usage -- it just wasn't being read. Captured on the adapter (last_usage, no interface change) and attached to the span using Langfuse's own OTel attribute convention (langfuse.observation.*), which its docs say takes precedence over generic GenAI attributes. Verified live: a fresh trace now shows real prompt/completion tokens and Langfuse computes real cost once given the model's actual per-token price.
This commit is contained in:
@@ -78,6 +78,10 @@ class BedrockConverseAnswerGenerator:
|
||||
self._client = client
|
||||
self._model_id = model_id
|
||||
self._max_tokens = max_tokens
|
||||
# Converse's own response usage, captured for the observability layer
|
||||
# (rag/instrumentation.py) to attach to its span. Not part of the
|
||||
# `generate()` return contract -- every existing caller is unaffected.
|
||||
self.last_usage: dict[str, int] | None = None
|
||||
|
||||
@property
|
||||
def model_id(self) -> str:
|
||||
@@ -126,6 +130,8 @@ class BedrockConverseAnswerGenerator:
|
||||
f"{self._model_id} could not be invoked: {type(error).__name__}"
|
||||
) from error
|
||||
|
||||
self.last_usage = dict(response.get("usage") or {}) or None
|
||||
|
||||
if response.get("stopReason") in _EMPTY_STOP_REASONS:
|
||||
raise AnswerGenerationUnavailable(
|
||||
f"{self._model_id} produced no usable content "
|
||||
|
||||
Reference in New Issue
Block a user