Repo Auditor: add job failure rate metric - #149
Merged
Merged
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Add GitHubClient.get_workflow_runs() to fetch Actions workflow runs (last 30 days) via REST with pagination. - Add job_metrics.py: computes overall and per-workflow Job Failure Rate, counting failure/timed_out/action_required/startup_failure as failures and excluding cancelled/skipped/neutral runs. - Wire get_job_failure_metrics() into auditor.run_audit(). - Add tests for the new client method and metrics module. - Document the new metric in README.md and CLAUDE.md. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Fix off-by-~1-day error in the 30-day window boundary check - Prevent one repo's workflow-run API failure from aborting the whole audit batch - Render None metric values as "N/A (no data)" instead of the literal string "None" in the job summary - Validate per_page in get_workflow_runs to avoid an infinite pagination loop - Document the double date-filtering rationale and the name-based workflow grouping limitation - Add tests covering the auditor.py wiring, malformed/boundary timestamps, rounding, and pagination edge cases Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Test fixtures used hardcoded absolute dates (2026-03-21) that became stale relative to the current clock, causing the real mergedAt-based early pagination exit to trigger prematurely and truncate results. Switched to dates computed relative to datetime.now(timezone.utc). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
stephenfuqua
marked this pull request as ready for review
August 7, 2026 13:44
…sult pagination cap GitHub's actions/runs endpoint stops returning results once page * per_page exceeds 1000, even though total_count reports a larger true total. Since results are newest-first, this silently dropped the oldest (often successful) runs for active repos, inflating the Job Failure Rate metric. Detected via a live mismatch between manual and computed rates on ODS-Admin-API. When total_count exceeds 1000, get_workflow_runs now re-fetches one calendar day at a time instead of trusting the single windowed query. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
roberthunterjr
approved these changes
Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new "job failure rate" metric for the audited repository
Example; I spot checked the new calculations for correctness.