Skip to content

feat: add circuit breaker protection for Horizon SDK calls - #583

Merged
Abdulazeem-code merged 1 commit into
Abdulazeem-code:mainfrom
muffti123:circuit-breaker-pattern
Aug 27, 2026
Merged

feat: add circuit breaker protection for Horizon SDK calls#583
Abdulazeem-code merged 1 commit into
Abdulazeem-code:mainfrom
muffti123:circuit-breaker-pattern

Conversation

@muffti123

Copy link
Copy Markdown
Contributor

Summary

This PR wraps all Stellar Horizon SDK interactions with opossum circuit breakers to prevent cascading timeouts and crashes when Horizon goes down.

What changed

New: src/services/stellarService.js

A centralized Horizon SDK wrapper that provides circuit-breaker-protected versions of all Stellar operations:

Operation Circuit Breaker Config
loadAccount threshold: 5, timeout: 10s, reset: 30s
fetchTransaction threshold: 5, timeout: 10s, reset: 30s
fetchPaymentsForTransaction threshold: 5, timeout: 15s, reset: 30s
fetchPaymentsForAccount threshold: 5, timeout: 10s, reset: 30s
paginated export threshold: 5, timeout: 10s, reset: 30s
stream health check threshold: 3, timeout: 5s, reset: 15s

Updated: horizonListener.js

  • Added health-check breaker that gates stream sync when Horizon is unreachable
  • Improved stream resilience: dead streams are removed from the active map and auto-reconnect on the next poll cycle

Updated: Route handlers (exportRoutes.js, historyRoutes.js, receiptRoutes.js)

  • Detect EOPENBREAKER errors from circuit breakers
  • Return 503 SERVICE_UNAVAILABLE with graceful error messages instead of crashing

Updated: multisigner-verifier.js

  • Refactored to use the new stellarService wrapper

New: tests/stellarService.test.js

  • 18 new tests covering circuit breaker behavior, fallback paths, and error handling

Why

When Stellar Horizon experiences downtime or high latency, the SDK calls would hang indefinitely or throw unhandled errors, causing cascading failures across the entire application. Circuit breakers provide:

  1. Fast failure — requests fail fast instead of hanging for 30+ seconds
  2. Automatic recovery — once Horizon recovers, the breaker resets and traffic flows normally
  3. Graceful degradation — users see a friendly 503 message instead of crashes
  4. Monitoring — breaker state transitions are logged for observability

Testing

  • All 370 existing tests pass
  • 18 new tests added for circuit breaker behavior
  • Verified that breakers trip after threshold failures and reset after the configured timeout

Closes #499

Wrap all Stellar Horizon SDK interactions with opossum circuit breakers
to prevent cascading timeouts and crashes when Horizon goes down.

- Create src/services/stellarService.js as centralized Horizon wrapper
  with configurable circuit breakers (threshold, timeout, reset)
- Add health-check breaker in horizonListener.js that gates stream sync
  when Horizon is unreachable
- Improve stream resilience: dead streams are removed from active map
  and auto-reconnect on next poll cycle
- Wrap loadAccount, fetchTransaction, fetchPaymentsForTransaction,
  fetchPaymentsForAccount, and paginated export with breakers
- Export paginated results with circuit-breaker-protected next() calls
- Detect EOPENBREAKER errors in route handlers and return 503
  SERVICE_UNAVAILABLE with graceful error messages
- All existing tests pass (370/370)
- Add 18 new tests for circuit breaker behavior

Closes Abdulazeem-code#499
@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

@muffti123 is attempting to deploy a commit to the Abdulazeem's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Aug 26, 2026

Copy link
Copy Markdown

@muffti123 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@Abdulazeem-code
Abdulazeem-code merged commit c40e91f into Abdulazeem-code:main Aug 27, 2026
7 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a circuit breaker pattern for external Stellar Horizon API calls

2 participants