Skip to content

feat(oracle): enforce median-deviation thresholds and quarantine anomalies from auto-finalization (#1611) - #1745

Merged
Olowodarey merged 1 commit into
Arena1X:mainfrom
meem08:feat/1611-oracle-anomaly-median-consensus
Aug 27, 2026
Merged

feat(oracle): enforce median-deviation thresholds and quarantine anomalies from auto-finalization (#1611)#1745
Olowodarey merged 1 commit into
Arena1X:mainfrom
meem08:feat/1611-oracle-anomaly-median-consensus

Conversation

@meem08

@meem08 meem08 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Oracle anomaly detection existed in the schema (#1364) but thresholds were not enforced when a submission deviated sharply from consensus. This PR closes that gap:

  1. Median-based flagging ([Backend] — Oracle Anomaly Detection Thresholds #1611) — every oracle submission is now evaluated not only by its historical z-score rule but also by its absolute deviation from the consensus median of its baseline window. A submission deviating beyond ORACLE_MEDIAN_DEVIATION_THRESHOLD (default 15 confidence-score units) is flagged as an anomaly even when its z-score alone would pass because the source's history is naturally wide or skewed.
  2. Quarantine from auto-finalization — flagged submissions are held for review (review_status = HELD) before any on-chain use, and a new consensus evaluation excludes quarantined submissions (HELD pending review or REJECTED) from the outcome vote and the confidence median. Admin-APPROVED submissions return to the consensus pool.
  3. Reviewable evidence + gating endpoint — flags now record baseline_median, and consumers/reviewers can query whether a match may be auto-finalized via GET /oracle/matches/:matchId/consensus.

Closes #1611

Changes

File Change
backend/src/oracle/submission-history.service.ts Adds the median-deviation rule to evaluateAnomaly (new reasons: median_deviation_exceeds_threshold; docs updated), records baseline_median on flag rows and enriches flag reason text with median evidence
backend/src/oracle/dto/anomaly-detection.dto.ts Extends AnomalyEvaluation with baselineMedian / medianDeviation; adds baseline_median to FlagResponse
backend/src/oracle/entities/oracle-submission-flag.entity.ts Nullable baseline_median audit column
backend/src/migrations/1776800000000-AddOracleBaselineMedian.ts Migration adding oracle_submission_flags.baseline_median (with revert)
backend/src/oracle/dto/match-consensus.dto.ts (new) MatchConsensusResponse / ConsensusSubmissionSummary Swagger DTOs
backend/src/oracle/oracle.service.ts New getMatchConsensus(matchId): partitions submissions into eligible vs quarantined, tallies outcome votes, computes confidence median, requires an outright majority plus ORACLE_CONSENSUS_MIN_SOURCES (default 2) eligible sources before reporting can_auto_finalize = true
backend/src/oracle/oracle.controller.ts GET /oracle/matches/:matchId/consensus behind OracleAuthGuard
backend/src/config/env.validation.ts Documents ORACLE_MEDIAN_DEVIATION_THRESHOLD and ORACLE_CONSENSUS_MIN_SOURCES

How thresholds are enforced

  • Intake: WebhookService.processMatchResult screens every submission through screenSubmission; when flagged and holding is enabled (ORACLE_ANOMALY_HOLD=true) the result is never queued on-chain — it is quarantined until /submissions/:id/review approves/rejects it (existing [Backend] — Oracle Submission Anomaly Detection #1364 flow, now also fed by the median rule).
  • Consensus: getMatchConsensus never counts a HELD/REJECTED submission toward the finalized outcome — a deviant source can neither decide nor veto a match result, and auto-finalization stays blocked while quarantine would leave too few eligible sources.

Tests

All under src/oracle/*.spec.ts (75 passing in the module):

  • evaluateAnomaly:
    • flags a value whose z-score passes but whose deviation from consensus median exceeds the threshold (median_deviation_exceeds_threshold)
    • does not flag within the configurable threshold band; exact boundary respected
    • reports null median deviation with insufficient baseline samples
    • still records baseline_median evidence on the flag row
  • getMatchConsensus:
    • outlier/held submission is excluded from the outcome vote and confidence median ("outlier flagged and excluded from consensus")
    • auto-finalization blocked when quarantine leaves fewer than the minimum sources
    • admin-approved anomalies rejoin consensus; rejected ones stay excluded
    • tie votes block finalization; higher configured minimum is honored

Verification

  • pnpm install && npx jest src/oracle → 5 suites, 75 tests passed
  • npx tsc --noEmit → no new errors (remaining ones pre-exist outside this change)
  • npx eslint src/oracle src/config env.validation → 0 errors

🤖 Generated with Codebuff
Co-Authored-By: Codebuff noreply@codebuff.com

…alies from auto-finalization (Arena1X#1611)

Oracle anomaly detection existed in the schema but thresholds were not
enforced when a submission deviated sharply from consensus. This:

- Enforces a configurable median-deviation rule (ORACLE_MEDIAN_DEVIATION_THRESHOLD,
  default 15) in SubmissionHistoryService.evaluateAnomaly alongside the existing
  z-score rule, so outliers versus the consensus median are flagged even when
  source history is naturally wide or skewed.
- Records baseline_median evidence on oracle_submission_flags via migration
  1776800000000-AddOracleBaselineMedian for admin auditability.
- Adds OracleService.getMatchConsensus to gate auto-finalization: quarantined
  submissions (HELD pending review or REJECTED) are excluded from the outcome
  vote and confidence median; APPROVED submissions return to consensus. A match
  is finalizable only with ORACLE_CONSENSUS_MIN_SOURCES eligible sources and an
  outright majority.
- Exposes GET /oracle/matches/:matchId/consensus for consumers/reviewers.
- Documents ORACLE_MEDIAN_DEVIATION_THRESHOLD and ORACLE_CONSENSUS_MIN_SOURCES
  in env validation.

Covers with tests: outlier flagged by median-only trip, threshold-boundary
behavior, flag row evidence, and consensus exclusion of held/rejected
submissions.

🤖 Generated with Codebuff
Co-Authored-By: Codebuff <noreply@codebuff.com>
@drips-wave

drips-wave Bot commented Aug 27, 2026

Copy link
Copy Markdown

@meem08 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@vercel

vercel Bot commented Aug 27, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
insight-arena-4rll Ready Ready Preview Aug 27, 2026 6:24am

@Olowodarey
Olowodarey merged commit aac2ca1 into Arena1X:main Aug 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Backend] — Oracle Anomaly Detection Thresholds

2 participants