Composite & Anomaly-Detection Alert Engine

[robertocarlous]

Problem Statement

Alert rules (#289) are single-condition and single-threshold: "APY < 5" or "drawdown > 10%". Real monitoring needs composition ("warn me if either the APY drops below 4% and my position is in a drawdown, or the protocol's risk score collapses") and it needs to notice problems that no fixed threshold can express — a protocol whose yield has quietly decayed for three weeks, or a portfolio whose volatility regime has changed. This issue extends the alert engine into a composite, anomaly-aware system: nested boolean rules, statistical drift/anomaly detection on the series the platform already keeps, backtesting of a rule before it's saved, and delivery semantics that stay sane when composite rules fire.

Current State

• AlertRule (prisma/schema.prisma): one metric (PROTOCOL_APY | PORTFOLIO_VALUE | POSITION_DRAWDOWN), one comparator, one threshold, one deliveryChannel, one cooldownMinutes. src/jobs/alertRules.ts evaluates rules on a schedule; src/services/alertEvaluator.ts evaluates a single condition; src/jobs/alertRules.ts's valueByInstant aggregates the portfolio series (the same aggregation bucketByInstant generalizes).
• Delivery handles missing phone (warn-and-skip), and cooldown suppresses repeat fires. src/whatsapp/alertManager.ts and the webhook dispatcher are the delivery rails.

Proposed Solution

1. Composite rule model

Extend AlertRule (or add AlertRuleGroup/nodes) so a rule is a tree, not a scalar:

model AlertRuleCondition {
  id         String   @id @default(uuid())
  ruleId     String?  // top-level rule when this is a root
  parentId   String?  // nesting
  operator   String   // AND | OR | NOT | CONDITION
  // when operator = CONDITION:
  metric     AlertMetric?
  comparator Comparator?
  threshold  Decimal? @db.Decimal(36, 18)
  windowMinutes Int?  // for windowed/anomaly metrics
  // when operator = AND|OR|NOT: children[]
}

• Semantics: a tree evaluates bottom-up; NOT inverts a sub-tree; every leaf is a concrete condition on one of the existing metrics (plus the new ones below). Depth is bounded (validation), cycles impossible by construction (a leaf has no children; a parent is required to reference only previously-created nodes — enforce at the API/validator layer and test it).
• Cooldown semantics for composites: the rule-level cooldownMinutes suppresses all fires of the tree, not per-leaf — a composite firing at midnight then again at midnight+1s with a changed leaf must obey one cooldown (define and test the "fire or not" decision at the root, with the leaf breakdown attached to the notification so the user sees why).
• Backward compatibility: existing single-condition rules are trees with one CONDITION node — the evaluation path is shared, the stored shape migrates, and the current API responses stay valid.

2. Statistical / anomaly conditions

New metric families computed from the series the platform already retains (with the standard honesty rules — never compute drift from the smoothed cumulative YieldSnapshot.apy column; use period-return/value series):

• DRIFT (per-protocol APY, windowed): the current APY vs. the trailing window's EWMA/z-score — fires when the deviation exceeds a configurable number of robust standard deviations (MAD-based, consistent with the AML-scoring conventions). Requires windowMinutes.
• VOLATILITY_REGIME (portfolio series): fires when rolling volatility crosses into a new regime band for a sustained duration (e.g. above the 90th percentile of the trailing window for N consecutive evaluations) — sustained-state detection, not a one-tick blip.
• ANOMALY (portfolio value / yield series): a simple, documented residual-based anomaly (e.g. observation more than k× robust-scaled-deviation from an EWMA forecast), configurable sensitivity. The model must be versioned in the rule so future algorithms don't silently reinterpret old rules.
• Composite cross-metric: e.g. APY < 4 AND (DRIFT APY > 2σ OR POSITION_DRAWDOWN > 10) — the reason for the current separate-metrics limitation (one metric per rule) disappears.

3. Rule backtesting / dry-run

• POST /api/v1/alerts/test (or ?preview=true on rule create/update): evaluate the proposed tree against the last N days of retained history and return: how many times it would have fired, when, and the leaf values at each fire — so a user can see their threshold is realistic (or that it would fire hourly). Must be honest about data availability: an anomaly rule over a window longer than retained history (90-day YieldSnapshot hard-delete) is rejected or truncated with a clear label.
• POST /api/v1/alerts/:id/trigger-now (admin or owner): force a single manual evaluation against live data without waiting for the schedule (used to verify delivery).

4. Delivery enhancements

• Fire payload includes the full decision tree trace: the composite rule's notification must carry the per-leaf values + which branches matched, formatted per channel (src/whatsapp/formatters.ts / src/telegram/formatters.ts), so "your rule fired" is always explainable.
• Escalation: optional, documented escalation list (a second channel/number/email to try after a rule fires N times within a window un-acknowledged). Keep v1 simple: a per-rule escalationChannel that activates on consecutive fires within a window.
• Scheduling pressure: anomaly evaluation over windows must be incremental (in-memory/Redis-cached series tails per user+metric), not a full-history recompute per tick — bound and document the compute budget.

5. API + docs (src/routes/alerts.ts, src/validators/alert-validators.ts, docs/openapi.yaml)

• Create/update accepts the tree shape with a depth cap and cycle-free validation; response mirrors the tree so clients can render it.
• New metrics (DRIFT, VOLATILITY_REGIME, ANOMALY) with windowMinutes requirements validated (a missing window on an anomaly metric is a named 400).
• GET /api/v1/alerts/:id/evaluations — recent evaluation results per node (what the scheduler actually saw), paginated.

Edge Cases & Failure Modes

• Tree with a permanently-false branch: allowed, but the test endpoint must surface "this branch never fired in the last N days" so the user knows it's dead weight.
• Data missing for a leaf (protocol delisted): the leaf evaluates to unknown; AND/OR truth tables must define unknown propagation explicitly (recommended: unknown → false for AND-context safety, but a fire with a missing leaf must be labeled partial_data and never silent).
• Cooldown vs. composite drift: a drift leaf that stays out-of-band must not keep the root from re-firing on the other branch — cooldown is per-root-fire, and consecutive distinct-branch fires inside the cooldown are recorded as suppressed (visible in evaluations).
• Anomaly alert storms during a real crash: sustained-regime conditions with N consecutive guards; document and test the debounce.
• Retention boundary: an anomaly window longer than retained data returns the honest span + flag, never silently recomputes from a truncated series.
• Legacy rule migration: all existing rules migrate losslessly to the tree shape; the migration is data-safe and reversible (down-migration).

Security & Privacy Considerations

• Composite rule trees are user data; owner-scoped reads/writes (enforceUserAccess), parent/child sub-account semantics preserved.
• The test/dry-run endpoint must not leak other users' series — it evaluates against the caller's own data and public ProtocolRate only.
• Anomaly detection runs on the user's own series + public protocol history; no cross-user inference.
• Notifications triggered by composite rules must not reveal another user's data (per-user fire payloads as today).

Out of Scope

• ML-based anomaly detection (the v1 algorithms are explicit, documented, rules-based — a modelVersion seam is included so they can be upgraded).
• Cross-user/community alert signals.
• Email/SMS channels beyond the existing webhook/WhatsApp/Telegram rails.

Suggested Implementation Plan

1. Schema: condition nodes + migration (lossless tree migration for existing rules) + validator for tree shape/depth/cycles.
2. Evaluation engine: bottom-up tree evaluation with unknown-propagation truth tables (unit-tested exhaustively).
3. Drift/regime/anomaly conditions with windowed, incremental series tails + cache.
4. Test/dry-run endpoint + trigger-now + evaluations history.
5. Delivery: fire-trace payloads, escalation, formatter updates, docs/openapi.yaml.

Acceptance Criteria

• Rules are trees (AND/OR/NOT + condition leaves) with bounded depth and cycle-free validation; legacy rules migrate losslessly
• Cooldown suppresses root fires, not leaves; suppressed fires visible in evaluations history
• DRIFT, VOLATILITY_REGIME, and ANOMALY conditions implemented with documented, versioned algorithms; window requirements validated
• Unknown/missing leaf semantics defined (AND/OR/NOT truth tables) with partial_data labeling, never silent
• POST /api/v1/alerts/test evaluates against retained history with honest retention handling; trigger-now forces live evaluation
• Fire notifications carry the full decision-tree trace per channel; optional escalation supported
• Anomaly evaluation is incremental/cached with a documented compute budget
• docs/openapi.yaml updated; unit + integration tests green

[Jenix777]

Hello Maintainer
I'd like to take on issue #324. I understand the core problem: alert rules are currently single-condition/single-threshold (one metric, one comparator, one threshold), so users can't express composite logic like "APY < 4% AND drawdown > 10%" or catch problems no fixed threshold can express, like a slow multi-week yield decay or a shifted volatility regime. My proposed approach: model rules as a tree of AlertRuleCondition nodes (AND/OR/NOT operators with condition leaves) evaluated bottom-up with depth-bounded, cycle-free validation, migrating existing single-condition rules losslessly into single-leaf trees; add the new DRIFT (MAD/z-score based), VOLATILITY_REGIME (sustained-band detection), and ANOMALY (EWMA residual-based, versioned) metrics computed incrementally off cached series tails rather than full-history recomputes; implement root-level cooldown semantics with explicit unknown-propagation truth tables and partial_data labeling for missing leaves; and build the /alerts/test dry-run and /alerts/:id/trigger-now endpoints plus fire-trace payloads so notifications explain exactly which branches matched. Could you assign this to me? Given the schema/migration/evaluation-engine scope here, I'd like to confirm the tree-node schema and truth-table conventions with you before implementing.
Thanks
Jennickson

[xeladev4]

Hi maintainer, I’d like to work on this issue.
I have experience building Stellar projects. I focus on clean implementations.
I’ll ensure the solution follows the project’s architecture and security best practices.

[sketcherzee]

Hi Maintainer, I’d like to work on issue #324. I understand the composite alert tree, anomaly detection, backtesting, cooldown, and delivery requirements. I have experience with Stellar/backend projects and will follow the existing architecture with proper validation and tests. I’d be happy to take this on.

[grantfox-oss]

🦊 GrantFox — @sketcherzee has been assigned to this issue as part of the Third Campaign campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #324)
2. Your PR will be reviewed by the Neurowealth maintainers

Good luck! Track your progress on GrantFox.