feat(oracle): enforce median-deviation thresholds and quarantine anomalies from auto-finalization (#1611) - #1745
Merged
Olowodarey merged 1 commit intoAug 27, 2026
Conversation
…alies from auto-finalization (Arena1X#1611) Oracle anomaly detection existed in the schema but thresholds were not enforced when a submission deviated sharply from consensus. This: - Enforces a configurable median-deviation rule (ORACLE_MEDIAN_DEVIATION_THRESHOLD, default 15) in SubmissionHistoryService.evaluateAnomaly alongside the existing z-score rule, so outliers versus the consensus median are flagged even when source history is naturally wide or skewed. - Records baseline_median evidence on oracle_submission_flags via migration 1776800000000-AddOracleBaselineMedian for admin auditability. - Adds OracleService.getMatchConsensus to gate auto-finalization: quarantined submissions (HELD pending review or REJECTED) are excluded from the outcome vote and confidence median; APPROVED submissions return to consensus. A match is finalizable only with ORACLE_CONSENSUS_MIN_SOURCES eligible sources and an outright majority. - Exposes GET /oracle/matches/:matchId/consensus for consumers/reviewers. - Documents ORACLE_MEDIAN_DEVIATION_THRESHOLD and ORACLE_CONSENSUS_MIN_SOURCES in env validation. Covers with tests: outlier flagged by median-only trip, threshold-boundary behavior, flag row evidence, and consensus exclusion of held/rejected submissions. 🤖 Generated with Codebuff Co-Authored-By: Codebuff <noreply@codebuff.com>
|
@meem08 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Oracle anomaly detection existed in the schema (#1364) but thresholds were not enforced when a submission deviated sharply from consensus. This PR closes that gap:
ORACLE_MEDIAN_DEVIATION_THRESHOLD(default15confidence-score units) is flagged as an anomaly even when its z-score alone would pass because the source's history is naturally wide or skewed.review_status = HELD) before any on-chain use, and a new consensus evaluation excludes quarantined submissions (HELDpending review orREJECTED) from the outcome vote and the confidence median. Admin-APPROVEDsubmissions return to the consensus pool.baseline_median, and consumers/reviewers can query whether a match may be auto-finalized viaGET /oracle/matches/:matchId/consensus.Closes #1611
Changes
backend/src/oracle/submission-history.service.tsevaluateAnomaly(new reasons:median_deviation_exceeds_threshold; docs updated), recordsbaseline_medianon flag rows and enriches flag reason text with median evidencebackend/src/oracle/dto/anomaly-detection.dto.tsAnomalyEvaluationwithbaselineMedian/medianDeviation; addsbaseline_mediantoFlagResponsebackend/src/oracle/entities/oracle-submission-flag.entity.tsbaseline_medianaudit columnbackend/src/migrations/1776800000000-AddOracleBaselineMedian.tsoracle_submission_flags.baseline_median(with revert)backend/src/oracle/dto/match-consensus.dto.ts(new)MatchConsensusResponse/ConsensusSubmissionSummarySwagger DTOsbackend/src/oracle/oracle.service.tsgetMatchConsensus(matchId): partitions submissions into eligible vs quarantined, tallies outcome votes, computes confidence median, requires an outright majority plusORACLE_CONSENSUS_MIN_SOURCES(default2) eligible sources before reportingcan_auto_finalize = truebackend/src/oracle/oracle.controller.tsGET /oracle/matches/:matchId/consensusbehindOracleAuthGuardbackend/src/config/env.validation.tsORACLE_MEDIAN_DEVIATION_THRESHOLDandORACLE_CONSENSUS_MIN_SOURCESHow thresholds are enforced
WebhookService.processMatchResultscreens every submission throughscreenSubmission; when flagged and holding is enabled (ORACLE_ANOMALY_HOLD=true) the result is never queued on-chain — it is quarantined until/submissions/:id/reviewapproves/rejects it (existing [Backend] — Oracle Submission Anomaly Detection #1364 flow, now also fed by the median rule).getMatchConsensusnever counts aHELD/REJECTEDsubmission toward the finalized outcome — a deviant source can neither decide nor veto a match result, and auto-finalization stays blocked while quarantine would leave too few eligible sources.Tests
All under
src/oracle/*.spec.ts(75 passing in the module):evaluateAnomaly:median_deviation_exceeds_threshold)nullmedian deviation with insufficient baseline samplesbaseline_medianevidence on the flag rowgetMatchConsensus:Verification
pnpm install && npx jest src/oracle→ 5 suites, 75 tests passednpx tsc --noEmit→ no new errors (remaining ones pre-exist outside this change)npx eslint src/oracle src/config env.validation→ 0 errors🤖 Generated with Codebuff
Co-Authored-By: Codebuff noreply@codebuff.com