Skip to content

feat(events): add execution-backed event pipeline - #21

Open
psteinroe wants to merge 1 commit into
mainfrom
feat/event-pipeline
Open

psteinroe wants to merge 1 commit into
mainfrom
feat/event-pipeline

Conversation

@psteinroe

@psteinroe psteinroe commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Make custom events durable by representing each emission as an internal execution and fanning it out asynchronously through a hidden dispatcher worker. This removes shared-inbox and per-worker event polling overhead while preserving typed event APIs.

PostgreSQL-native fan-out

Custom-event filters compile into normalized matcher tables. Claim-fenced dispatch uses indexed candidate discovery plus complete residual verification, deduplicates subscriptions, and atomically writes destination executions with a completion marker. Destination uniqueness makes retries idempotent, while per-source validation failures do not roll back valid peers in the same batch.

Application-owned database triggers

Pgconductor no longer creates, synchronizes, or manages triggers on application tables. Applications that emit events from PostgreSQL triggers own those triggers and can call pgconductor.emit_event() transactionally, so source changes and event persistence still commit or roll back together.

Worker lifecycle and bounded matching

The hidden dispatcher reuses the normal claim, execution, retry, and result-flush lifecycle with an initial batch size of 10. Registration precedes dispatcher polling, recursive drain passes process newly emitted work, and registration/dispatch enforce documented limits for subscriptions, filters, scalar event fields, and destinations.

Refs #14

@psteinroe
psteinroe added this pull request to stack #25 September 15, 2026 03:35
@psteinroe
psteinroe force-pushed the feat/event-pipeline branch 2 times, most recently from 5bac5a9 to ac6d4f2 Compare September 15, 2026 04:00
@psteinroe
psteinroe force-pushed the feat/event-pipeline branch 2 times, most recently from 21c7920 to 2786766 Compare September 15, 2026 04:22
@psteinroe
psteinroe removed this pull request from stack #25 September 15, 2026 11:41
@psteinroe
psteinroe added this pull request to stack #27 September 15, 2026 13:27
Base automatically changed from feat/dlq to main September 18, 2026 14:26
@psteinroe
psteinroe force-pushed the feat/event-pipeline branch 2 times, most recently from f64409a to 143a962 Compare September 18, 2026 21:41
@psteinroe psteinroe changed the title feat(events): add persistent batch processing feat(events): add execution-backed event pipeline Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant