Feat Implemented Advanced Stellar Ledger Indexer with Real-time Event Streaming - #54
Merged
Conversation
…ipeline Add configuration options for WebSocket reconnection, in-memory event buffer, batched persistence, and configurable retention policies that drive the new real-time ledger indexer. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…nt types Add soroban_contract_invocation, soroban_contract_event, soroban_system_event, and soroban_diagnostic_event to support Soroban contract event indexing alongside native Stellar operations. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…torage Create a unified event entity that stores both Stellar native operations and Soroban contract events in a single table with optimized indexes on sequence number, ledger, event type, contract ID, and account fields to support temporal queries and high-throughput writes. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…version Normalize raw Horizon operations and Soroban contract events into a unified IndexedEvent shape with monotonic sequence number assignment, enabling consistent ordering and deduplication across event sources. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…streaming Replace polling with a persistent SSE connection to the Stellar Horizon API for sub-second ledger close event detection. Includes automatic reconnection with exponential backoff and cursor-based resumption. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…nd ordering In-memory event buffer that absorbs bursts while maintaining sequence-number ordering guarantees and O(1) deduplication. Includes TTL-based expiry for stale events and FIFO batch extraction for the persistence layer. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…ughput writes Batched PostgreSQL persistence layer using bulk INSERT with ON CONFLICT DO NOTHING for idempotency. Configurable flush interval and batch size to achieve >10k events/second throughput with a retention-based purge policy. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…istribution Client-side pub/sub system filtering events by type, contract ID, account, and ledger range. Maintains reverse index for efficient per-client cleanup and emits matched events for WebSocket delivery. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…criptions Socket.io WebSocket gateway at /blockchain-indexer namespace allowing frontend clients to subscribe to real-time trade executions, balance updates, liquidations, and Soroban contract events with typed filters. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
…full pipeline Orchestrates Horizon SSE streaming, operation fetching, normalization, in-memory buffering, batched persistence, and WebSocket distribution into a single real-time indexing pipeline with sub-second latency. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
Add SubscribeEventsDto and TemporalQueryDto for typed event subscription and state-at-block queries. Update controller with /realtime/status, /realtime/events, and /realtime/temporal endpoints. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
… unit tests Register all new services, IndexedEvent entity, and WebSocket gateway in BlockchainIndexerModule. Add cursor and sequence state helpers to IndexingStateService. Add 36 unit tests for EventNormalizer, EventBufferService, and SubscriptionManager. 🤖 Generated with Codebuff Co-Authored-By: DevMuhdishaq <noreply@codebuff.com>
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #49
Summary
Implemented Advanced Stellar Ledger Indexer with Real-time Event Streaming
Related Issues
Closes #49
Type of Change
Changes
new feature module
Checklist
npm run lintpasses.npx tsc --noEmitpasses (no type errors).npm run buildsucceeds.npm testpasses, and I added/updated tests for my change where itmade sense.
stellarmoduleservices (no direct Horizon/RPC calls from feature modules).
Notes for Reviewers