Skip to content

[Improve] Track Brain inference usage and cost - #1663

Draft
roomote-roomote[bot] wants to merge 1 commit into
developfrom
feature/brain-inference-cost-04fp2u1ta6z2c
Draft

[Improve] Track Brain inference usage and cost#1663
roomote-roomote[bot] wants to merge 1 commit into
developfrom
feature/brain-inference-cost-04fp2u1ta6z2c

Conversation

@roomote-roomote

Copy link
Copy Markdown
Contributor

​Opened on behalf of @mrubens. Follow up by mentioning @roomote-roomote, in the web UI, or in Slack.

What changed

  • Record every Brain provider attempt through the existing LLM usage ledger, including provider, resolved model, operation, status, and full-response latency.
  • Normalize embedding, rerank, Chat Completions, and Responses token metadata from JSON and SSE responses, and distinguish their embedding, rerank, and inference usage types.
  • Capture provider-reported cost as costMicroUsd with a provider_response source, while retaining failures and metadata-free responses with explicit missing-cost attribution.
  • Request final usage chunks for OpenAI-compatible chat streams without replacing caller-supplied stream options.

Why this change was made

Brain inference previously passed through the provider gateway without entering Roomote's usage ledger, leaving embeddings, reranking, query expansion, and synthesis absent from cost and token reporting.

Impact

Operators can account for Brain inference alongside other LLM usage and separate it by operation, model, provider, status, and usage type. OpenRouter responses can include monetary cost; direct OpenAI responses generally provide token counts but not cost, so those rows retain costSource: missing unless the provider returns cost metadata.

@roomote-community

roomote-community Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

No code issues found. See task

Reviewed 630c359

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant