Skip to content

Post error comments regardless of trigger - #843

Merged
lbarcziova merged 2 commits into
packit:mainfrom
lbarcziova:comment-on-failures-always
Sep 23, 2026
Merged

lbarcziova merged 2 commits into
packit:mainfrom
lbarcziova:comment-on-failures-always

Conversation

@lbarcziova

Copy link
Copy Markdown
Member

Users on the forum channel regularly ask about failed issues because the only visible signal was an errored label with no explanation.

Error comments were previously gated on user_triggered, meaning automatic runs failed silently. This makes error feedback unconditional:

  • Crash-based errors (exceptions): a new post_terminal_error_comment() function bypasses the trigger gate and is called from each agent's final-retry path, which already only runs after retries are exhausted.
  • Workflow ERROR resolutions (triage): added to the unbidden no-MR list in _should_update_jira(), so the result comment is posted immediately like not-affected or postponed — there is no retry for these.
  • Mid-workflow errors (e.g. consolidation failures inside run_workflow) remain trigger-gated in comment_in_jira() since they are intermediate, not terminal.

The comment_in_jira / _post_jira_comment split keeps the trace-link logic in one place while giving callers a clear choice: comment_in_jira for workflow results (trigger-gated on errors), post_terminal_error_comment for terminal failures (always posts).

Assisted-by: Claude Opus 4.6

TODO:

  • Write new tests or update the old ones to cover new functionality.
  • Update doc-strings where appropriate.
  • Update or write new documentation in packit/packit.dev.
  • ‹fill in›

RELEASE NOTES BEGIN

Ymir now comments on errors even for issues that are autoprocessed (not triggered via ymir_todo).

RELEASE NOTES END

@lbarcziova

Copy link
Copy Markdown
Member Author

/agentic_review

@qodo-for-packit

qodo-for-packit Bot commented Sep 23, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Error retries post duplicate comments ✓ Resolved 🐞 Bug ≡ Correctness
Description
_should_update_jira now posts every Resolution.ERROR result before the queue handler calls
retry for that same result. Each repeated workflow attempt therefore adds another result comment,
and the exhausted attempt also adds the new terminal failure comment.
Code

ymir/agents/triage_agent.py[117]

+        Resolution.ERROR,
Relevance

●● Moderate

PR intent says ERROR resolutions do not retry, but duplicate-comment behavior lacks a close
historical precedent.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The workflow posts the newly admitted error resolution in comment_in_jira, then dispatch
subsequently routes it through the retry mechanism. The final retry branch independently invokes
post_terminal_error_comment, with no deduplication for error resolutions.

ymir/agents/triage_agent.py[111-119]
ymir/agents/triage_agent.py[1214-1260]
ymir/agents/triage_agent.py[1509-1559]
ymir/agents/triage_agent.py[1796-1813]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Triage error resolutions are commented before being requeued, so every retry posts another comment and final exhaustion posts an additional terminal error.

## Fix Focus Areas
- ymir/agents/triage_agent.py[111-119]
- ymir/agents/triage_agent.py[1214-1260]
- ymir/agents/triage_agent.py[1509-1559]
- ymir/agents/triage_agent.py[1796-1813]

## Recommended Fix
Make `Resolution.ERROR` handling consistently terminal or retryable. If retries remain, defer its Jira comment until exhaustion or persist a delivery marker so only one comment is posted; if it is intended to be terminal immediately, stop requeueing it and record the failure once.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Automatic failures expose stack traces ✓ Resolved 🐞 Bug ⛨ Security
Description
The new triage finalizer interpolates error.details directly into a Jira comment, while its
exception path populates that field using traceback.format_exception. Automatic crash failures can
therefore disclose local source paths, internal frames, and exception text to issue viewers instead
of keeping diagnostics behind the linked trace viewer.
Code

ymir/agents/triage_agent.py[R1546-1550]

+                                await tasks.post_terminal_error_comment(
+                                    jira_issue=input.issue,
+                                    agent_type="Triage",
+                                    comment_text=f"Agent failed to perform triage: {error.details}",
+                                    available_tools=gateway_tools,
Relevance

●●● Strong

Direct traceback disclosure in unconditional Jira comments is a concrete security issue; terminal
comments otherwise emphasize trace links.

PR-#726

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
Triage constructs ErrorData.details from the complete formatted exception traceback, and the newly
added final-retry branch passes it to the unconditional helper. That helper templates the supplied
text directly into a private Jira comment without redaction.

ymir/agents/triage_agent.py[1540-1551]
ymir/agents/triage_agent.py[1627-1638]
ymir/agents/tasks.py[686-723]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Automatic triage crash comments publish the complete Python traceback stored in `ErrorData.details`, exposing internal diagnostic information in Jira.

## Fix Focus Areas
- ymir/agents/triage_agent.py[1540-1551]
- ymir/agents/triage_agent.py[1627-1638]
- ymir/agents/tasks.py[686-723]

## Recommended Fix
Post a concise, sanitized failure summary and the existing trace-viewer link instead of `error.details`. Keep the full traceback in internal logs and trace storage, and sanitize exception messages before including any portion in Jira.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Error comments show unrelated results ✓ Resolved 🐞 Bug ≡ Correctness
Description
_should_update_jira now sends an error result before the dispatch code checks that output.data
is actually ErrorData. Because the schema permits any data union member and formatting dispatches
on its runtime type, a mismatched model response can publish rebase, backport, or rebuild details as
an error before retry handling synthesizes the actual failure.
Code

ymir/agents/triage_agent.py[117]

+        Resolution.ERROR,
Relevance

●● Moderate

Potential schema mismatch is plausible, but no repository precedent addresses independently modeled
resolution and data types.

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
TriageOutputSchema models resolution and data independently, and format_for_comment selects
content from the data object's type. The mismatch check occurs only after the workflow has already
formatted and posted the result.

ymir/common/models.py[563-585]
ymir/common/models.py[587-619]
ymir/agents/triage_agent.py[1214-1260]
ymir/agents/triage_agent.py[1796-1813]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
An error resolution can be paired with another result-data type, and the comment is formatted and posted before that mismatch is detected.

## Fix Focus Areas
- ymir/agents/triage_agent.py[1214-1260]
- ymir/agents/triage_agent.py[1796-1813]
- ymir/common/models.py[563-619]

## Recommended Fix
Validate or normalize the resolution/data pairing before `format_for_comment` runs. For `Resolution.ERROR`, require `ErrorData` or replace mismatched data with the synthesized error before constructing any Jira comment.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Informational

4. Automatic workflow failures stay silent ✓ Resolved 🐞 Bug ≡ Correctness
Description
finalize_failure only posts through post_terminal_error_comment when comment_text is nonempty,
while graceful backport, rebase, and rebuild failures retry without supplying it because their
earlier comment is assumed sufficient. On non-user-triggered runs that earlier error comment is
suppressed, so retry exhaustion still leaves users with labels but no explanation.
Code

ymir/agents/backport_agent.py[2035]

+                if comment_text and not dry_run:
Relevance

● Weak

Accepted precedent explicitly preserves empty comment_text for graceful failures to avoid duplicate
comments.

PR-#611

ⓘ Recommendations generated based on similar findings in past PRs

Evidence
The new final paths require comment_text, but each graceful failure path deliberately invokes
retry without it. Their earlier workflow comments call the trigger-gated helper, which returns
without posting for automatic runs.

ymir/agents/tasks.py[660-684]
ymir/agents/backport_agent.py[2032-2045]
ymir/agents/backport_agent.py[2161-2177]
ymir/agents/rebase_agent.py[622-656]
ymir/agents/rebase_agent.py[895-912]
ymir/agents/rebuild_agent.py[677-696]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Automatic graceful backport, rebase, and rebuild failures remain silent because their final-retry calls omit `comment_text`, while their in-workflow error comments are suppressed for non-user-triggered runs.

## Fix Focus Areas
- ymir/agents/backport_agent.py[2032-2045]
- ymir/agents/backport_agent.py[2161-2177]
- ymir/agents/rebase_agent.py[787-802]
- ymir/agents/rebase_agent.py[895-912]
- ymir/agents/rebuild_agent.py[567-582]
- ymir/agents/rebuild_agent.py[677-696]

## Recommended Fix
Ensure each graceful failure supplies a suitable terminal comment when its retries are exhausted on an automatic run. Preserve the intended intermediate-comment behavior for user-triggered runs without adding a second final copy.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Context sources
✅ Compliance rules (platform): 8 rules
Review mode: ⚖️ Balanced: This changes terminal error-comment behavior across multiple agents and triage paths, with meaningful notification and retry-path risk, but is not dense enough to warrant redundant extended review.

Grey Divider

Tip of the day
💡 Did you know, you can keep summaries lean with Finding overflow, which tucks the rest behind 'View more'

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

Comment thread ymir/agents/triage_agent.py Outdated
Comment thread ymir/agents/triage_agent.py Outdated
Comment thread ymir/agents/triage_agent.py
@lbarcziova
lbarcziova force-pushed the comment-on-failures-always branch from 958ca5c to 83303dd Compare September 23, 2026 15:11

@nforro nforro left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Users on the forum channel regularly ask about failed issues because the
only visible signal was an errored label with no explanation.

Error comments were previously gated on user_triggered, meaning automatic
runs failed silently. This makes error feedback unconditional:

- Crash-based errors (exceptions): a new post_terminal_error_comment()
  function bypasses the trigger gate and is called from each agent's
  final-retry path, which already only runs after retries are exhausted.
- Workflow ERROR resolutions (triage): added to the unbidden no-MR list
  in _should_update_jira(), so the result comment is posted immediately
  like not-affected or postponed — there is no retry for these.
- Mid-workflow errors (e.g. consolidation failures inside run_workflow)
  remain trigger-gated in comment_in_jira() since they are intermediate,
  not terminal.

The comment_in_jira / _post_jira_comment split keeps the trace-link
logic in one place while giving callers a clear choice: comment_in_jira
for workflow results (trigger-gated on errors), post_terminal_error_comment
for terminal failures (always posts).

Assisted-by: Claude Opus 4.6
Resolution.ERROR from run_workflow() is dispatched to retry(), so posting
the comment inside the workflow would duplicate it on every attempt.
Suppress ERROR in _should_update_jira() and instead have retry() post a
single terminal comment after retries are exhausted.

Also use concise error messages (e.explain() / str(e)) for Jira comments
instead of the full traceback, matching the pattern in the other agents.
The full traceback stays in ErrorData.details and the trace viewer.

Assisted-by: Claude Opus 4.6
@lbarcziova
lbarcziova force-pushed the comment-on-failures-always branch from 83303dd to d4ef8cc Compare September 23, 2026 16:15
@lbarcziova
lbarcziova merged commit 3771427 into packit:main Sep 23, 2026
15 checks passed
@lbarcziova
lbarcziova deleted the comment-on-failures-always branch September 23, 2026 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants