Skip to content

Parser: batch Vert.x eventbus messages to cut per-line envelope allocation #569

Description

@brunoborges

Summary

The parser ships one Vert.x eventbus message per parsed line. The per-message envelope objects (MessageImpl, DeliveryOptions, InboundDeliveryContext) add a measurable, constant allocation tax proportional to line count.

Evidence (JFR settings=profile, ~28s run)

Workload: sample Main parsing a 195 MB JDK8 ParallelGC rolling log (gclogs/rolling/jdk8/aagl_prd/gc.log), -Xmx1500m, G1, Java 25 (GraalVM 25.0.3). ~19,985 MB sampled allocations.

Allocation by class / site:

Item Share
io.vertx.core.eventbus.impl.MessageImpl 3.43% (~685 MB)
io.vertx.core.eventbus.DeliveryOptions 1.34%
io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext 1.30%
alloc site EventBusImpl.checkStarted() 3.94%

Combined ≈6% of sampled allocations is eventbus envelope overhead, one instance per line.

Suggested fixes

  • Batch multiple parsed lines/events into a single eventbus message to amortize the envelope cost (e.g. send chunks of N lines/events instead of one per line).
  • Consider a local (non-clustered) fast path for the single-process analysis case, avoiding full eventbus delivery machinery where a direct handler call would do.

Expected impact

Removes a large fraction of the ~6% eventbus-envelope allocation, and reduces checkStarted()/delivery-context CPU on every line. GC is already healthy (98.76% throughput); this is an allocation-throughput / scalability improvement, most visible on very large logs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions