Skip to content

feat: implement execution gap reporting, redundant test detection, CI - #2

Merged
trancee merged 4 commits into
mainfrom
feature/gap-reporting-redundant-detection-ci
Aug 27, 2026
Merged

feat: implement execution gap reporting, redundant test detection, CI#2
trancee merged 4 commits into
mainfrom
feature/gap-reporting-redundant-detection-ci

Conversation

@trancee

@trancee trancee commented Aug 26, 2026

Copy link
Copy Markdown
Owner

Implements three Scott-CC→OMP feature gaps:

T6 — Confidence Intervals

  • Wilson score 95% CI formula in wilsonInterval(), clamped to [0,1]
  • Added confidenceIntervalLow/High to MutationStats, serialized via encodeDefaults=true
  • Score is nullable; CI null when no mutations are evaluable

D3/T7 — Execution Gap Reporting

  • ExecutionGap data class (5 gap types: NO_OUTPUT, PARTIAL_RUN, COMPILATION_FAILURE, IR_TRANSFORMATION_ERROR, BACKSTOP_TIMEOUT)
  • detectGaps(): parser-level detection for empty stdout (NO_OUTPUT) and footer count mismatch (PARTIAL_RUN)
  • Gradle task checks JUnit XML presence → COMPILATION_FAILURE gap when missing
  • Score = killed/(total - gaps), null when denominator == 0
  • TimedOut is NOT a gap — it's a valid mutation result
  • 5 parser tests + 3 serializer tests

D1 — Redundant Test Detection

  • RedundantGroup data class (tests, count, failureSignature)
  • detectRedundantTestGroups(): groups tests by failure signature, threshold >5, sorted by count desc, excludes zombies
  • 5 parser tests

D2 — Modes, Focus, Auto-Approve (agent-level docs)

  • --mode quick|standard|deep → mutflow maxRuns (10/30/all)
  • --focus → Gradle includeTargets/excludeTargets bridge
  • --auto-approve gate: zombie/redundant deletion requires explicit approval
  • Updated SKILL.md, test-quality-reviewer.md, test-refactor-specialist.md, test-auditor.md, test-executor.md

Tests

35 tests (18 parser + 7 serializer + 10 stats), all passing.

@trancee
trancee force-pushed the feature/gap-reporting-redundant-detection-ci branch from 740ab39 to 82071e4 Compare August 26, 2026 11:34
@trancee trancee changed the title Implement execution gap reporting, redundant test detection, confidence intervals feat: implement execution gap reporting, redundant test detection, CI Aug 26, 2026
Kotlin module:
- ExecutionGap data class (5 gap types: NO_OUTPUT, PARTIAL_RUN,
  COMPILATION_FAILURE, IR_TRANSFORMATION_ERROR, BACKSTOP_TIMEOUT)
- RedundantGroup data class with tests, count, failureSignature
- MutationStats.score → Double? (null when no mutations evaluable)
- MutationStats: added gaps, mutationsEvaluated, confidenceIntervalLow/High
- MutationResults: added gaps, mutationsEvaluated, CI fields, executionGaps,
  redundantGroups, mutationScore as nullable Double?
- calculateMetrics: score = killed/(total-gaps), null when denominator == 0
- wilsonInterval(): 95% Wilson score CI, clamped to [0,1]
- detectGaps(): NO_OUTPUT, PARTIAL_RUN, preserves build-level gaps
- detectRedundantTestGroups(): signature grouping, threshold >5, zombie exclusion
- assembleResults: accepts gaps + redundantGroups params

Gradle task:
- Checks JUnit XML → COMPILATION_FAILURE gap when missing
- Calls detectGaps() with stdout + build-level gaps
- Handles nullable mutationScore (N/A) in logger, logs gap count

Agent docs:
- test-auditor.md: score formula, gap/redundant reporting, JSON schema
- test-executor.md: gap detection documentation
- test-refactor-specialist.md: redundantGroups, @ParameterizedTest,
  --auto-approve gate, diff + rollback output
- test-quality-reviewer.md: modes (quick/standard/deep → maxRuns),
  --focus, --auto-approve orchestration
- SKILL.md: modes, --focus, --auto-approve flags

Tests: 35 total (18 parser + 7 serializer + 10 stats), all passing
@trancee
trancee force-pushed the feature/gap-reporting-redundant-detection-ci branch from 82071e4 to 0ebd53b Compare August 26, 2026 12:06
@trancee
trancee force-pushed the feature/gap-reporting-redundant-detection-ci branch 7 times, most recently from 1540cea to fc82cc7 Compare August 26, 2026 13:18
…place markdownlint config; fix CONTEXT.md headings
@trancee
trancee force-pushed the feature/gap-reporting-redundant-detection-ci branch from fc82cc7 to 9cad14a Compare August 26, 2026 13:24
@trancee
trancee merged commit ac30b0a into main Aug 27, 2026
2 checks passed
@trancee
trancee deleted the feature/gap-reporting-redundant-detection-ci branch August 27, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant