Skip to content

Improve Langfuse span labels for artifact generation and revisions #138

Description

@eshulman2

Problem

The labels attached to spans we send to Langfuse are missing useful context, which makes Grafana dashboards misleading and hard to interpret.

For example, when we break down epics, the span is currently labeled as spec approval gate. In dashboards this makes it look like the generation cost for epic breakdown happened in the spec stage, even though the work belongs to the epic breakdown flow.

There is also no clear separation between:

  • initial artifact generation
  • artifact revisions/regeneration
  • question asking or clarification flows
  • approval gate activity
  • downstream breakdown/generation work

As a result, the data is flattened into broad labels and costs/tokens are attributed to confusing stages.

Impact

Grafana views based on Langfuse labels can lead users to incorrect conclusions about where generation cost and latency are coming from. It is especially hard to distinguish stage overhead from actual artifact generation, revisions, and follow-up question handling.

Expected behavior

Langfuse span labels should include enough structured context to make dashboards meaningful. At minimum, we should be able to distinguish:

  • workflow stage or node
  • operation type, such as approval gate, initial generation, revision, question asking, breakdown
  • artifact type, such as spec, plan, tasks, epic breakdown
  • attempt/revision context when applicable

Acceptance criteria

  • Epic breakdown spans are not labeled as spec approval gate.
  • Initial generation, revisions, and question asking are distinguishable in Langfuse/Grafana.
  • Labels remain stable enough for dashboard grouping and filtering.
  • Existing dashboards can attribute token/cost/latency metrics to the correct workflow operation.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions