api: reduce syscall overhead in client and server by rjarry · Pull Request #714 · DPDK/grout

rjarry · 2026-08-26T13:28:26Z

Reduce the number of syscalls per API transaction on both client and server sides.

On the server, use BEV_OPT_DEFER_CALLBACKS so that libevent accumulates header and payload writes in the evbuffer and flushes them in a single writev(). Remove the no-op bufferevent_flush() calls and the manual read_cb re-trigger which libevent already handles.

On the client, replace two separate send() calls with a single sendmsg() using an iovec, and add a BUFSIZ read-ahead buffer so that recv_all() can serve multiple small reads from one recv() syscall. This is particularly effective during stream iterations where many small messages arrive back-to-back.

A GR_PING request type and accompanying ping_perf benchmark tool are added to measure the resulting transaction rate.

Different approach from https://patches.dpdk.org/project/grout/patch/20260720224924.3299328-1-mb@smartsharesystems.com/

Cc: @MortenBroerup

qodo-free-for-open-source-projects · 2026-08-26T13:30:38Z

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

MortenBroerup · 2026-08-26T13:46:23Z

Deeply buried in DPDK patches. Will review in a few days. Venlig hilsen / Kind regards,

…

-Morten Brørup From: Robin Jarry ***@***.*** Sent: Wednesday, 26 August 2026 15.29 To: DPDK/grout Cc: Morten Brørup; Mention Subject: [DPDK/grout] api: reduce syscall overhead in client and server (PR #714) Reduce the number of syscalls per API transaction on both client and server sides. On the server, use BEV_OPT_DEFER_CALLBACKS so that libevent accumulates header and payload writes in the evbuffer and flushes them in a single writev(). Remove the no-op bufferevent_flush() calls and the manual read_cb re-trigger which libevent already handles. On the client, replace two separate send() calls with a single sendmsg() using an iovec, and add a BUFSIZ read-ahead buffer so that recv_all() can serve multiple small reads from one recv() syscall. This is particularly effective during stream iterations where many small messages arrive back-to-back. A GR_PING request type and accompanying ping_perf benchmark tool are added to measure the resulting transaction rate. Different approach from ***@***.***/ Cc: @MortenBroerup <https://github.com/MortenBroerup>

The bufferevent_sock implementation of flush is a no-op, so remove the bufferevent_flush() calls. The manual re-trigger of read_cb when input data remains is also unnecessary since libevent will invoke the callback as long as data is available in the input buffer. Signed-off-by: Robin Jarry <rjarry@redhat.com>

MortenBroerup

Deferred bufferevent callbacks look like good solution.
I have a concern about huge responses...
For reference, PHP has functions to enable/disable output buffering.
Is there any means to limit libevent buffering for huge responses? Otherwise, they may consume huge amounts of memory in the server (Grout), instead of streaming in chunks to the client.
I'm asking for either a configurable threshold in libevent, or a function to temporarily disable/enable deferring the callbacks, or a flush-like function that invokes the callback.

rjarry · 2026-08-26T20:55:32Z

Deferred bufferevent callbacks look like good solution. I have a concern about huge responses... For reference, PHP has functions to enable/disable output buffering. Is there any means to limit libevent buffering for huge responses? Otherwise, they may consume huge amounts of memory in the server (Grout), instead of streaming in chunks to the client. I'm asking for either a configurable threshold in libevent, or a function to temporarily disable/enable deferring the callbacks, or a flush-like function that invokes the callback.

That's a good point. I have capped the amount of buffering before flushing to the socket.

// 64K allows buffering up to 16 * 4K evbuffer chains before flushing
#define HIGH_WATERMARK (1 << 16)

void api_send(struct api_ctx *ctx, uint32_t len, const void *payload) {
	...

	if (bufferevent_write(ctx->bev, &resp, sizeof(resp)) < 0)
		LOG(ERR, "pid=%d cannot write header", ctx->pid);
	if (bufferevent_write(ctx->bev, payload, len) < 0)
		LOG(ERR, "pid=%d cannot write payload", ctx->pid);

	// Force flush when buffer grows past HIGH_WATERMARK to avoid runaway memory use
	struct evbuffer *output = bufferevent_get_output(ctx->bev);
	if (evbuffer_get_length(output) >= HIGH_WATERMARK) {
		evutil_socket_t fd = bufferevent_getfd(ctx->bev);
		evbuffer_unfreeze(output, true);
		evbuffer_write_atmost(output, fd, -1);
		evbuffer_freeze(output, true);
	}
}

MortenBroerup · 2026-08-27T14:09:21Z

Deferred bufferevent callbacks look like good solution. I have a concern about huge responses... For reference, PHP has functions to enable/disable output buffering. Is there any means to limit libevent buffering for huge responses? Otherwise, they may consume huge amounts of memory in the server (Grout), instead of streaming in chunks to the client. I'm asking for either a configurable threshold in libevent, or a function to temporarily disable/enable deferring the callbacks, or a flush-like function that invokes the callback.

That's a good point. I have capped the amount of buffering before flushing to the socket.
// 64K allows buffering up to 16 * 4K evbuffer chains before flushing
#define HIGH_WATERMARK (1 << 16)

void api_send(struct api_ctx *ctx, uint32_t len, const void *payload) {
	...

	if (bufferevent_write(ctx->bev, &resp, sizeof(resp)) < 0)
		LOG(ERR, "pid=%d cannot write header", ctx->pid);
	if (bufferevent_write(ctx->bev, payload, len) < 0)
		LOG(ERR, "pid=%d cannot write payload", ctx->pid);

	// Force flush when buffer grows past HIGH_WATERMARK to avoid runaway memory use
	struct evbuffer *output = bufferevent_get_output(ctx->bev);
	if (evbuffer_get_length(output) >= HIGH_WATERMARK) {
		evutil_socket_t fd = bufferevent_getfd(ctx->bev);
		evbuffer_unfreeze(output, true);
		evbuffer_write_atmost(output, fd, -1);
		evbuffer_freeze(output, true);
	}
}

I was wondering if HIGH_WATERMARK and BUFSIZ in API read-ahead should be similar? Maybe not, because they execute in two different applications. But - considering the value of HIGH_WATERMARK (64 KB)- maybe read-ahead should be higher than BUFSIZ (8 KB).
Should we use GR_API_MAX_MSG_LEN?
Regardless, I think HIGH_WATERMARK should be at least GR_API_MAX_MSG_LEN.
No good reasons for changing these; just my initial thoughts - gut feeling.

MortenBroerup · 2026-08-27T16:00:40Z

+	if (evbuffer_get_length(output) >= HIGH_WATERMARK) {
+		evutil_socket_t fd = bufferevent_getfd(ctx->bev);
+		evbuffer_unfreeze(output, true);
+		evbuffer_write_atmost(output, fd, -1);


When crossing the HIGH_WATERMARK, the flush calls deeply inside libevent, to replicate this:
https://github.com/libevent/libevent/blob/master/bufferevent_sock.c#L301
But the error handling is missing here.
Would it be possible to call the libevent write callback instead?

I have double checked and we don't need this flag after all. All bufferevent_write() calls made synchronously (without yielding back to the event loop) are coalesced into a single EV_WRITE event. When the read callback returns (after writing any number of times), up to 512K bytes will be written in a single system call (using writev() + iovecs).

I have removed the commit that added BEV_OPT_DEFER_CALLBACKS entirely.

MortenBroerup · 2026-08-27T17:17:13Z

I was wondering if HIGH_WATERMARK and BUFSIZ in API read-ahead should be similar? Maybe not, because they execute in two different applications. But - considering the value of HIGH_WATERMARK (64 KB)- maybe read-ahead should be higher than BUFSIZ (8 KB).
Should we use GR_API_MAX_MSG_LEN?

After further consideration: No, they serve different purposes. Let's keep them individual.

Regardless, I think HIGH_WATERMARK should be at least GR_API_MAX_MSG_LEN.

If we consider the HIGH_WATERMARK a safety guard against runaway buffering, it can be much higher, such as 2 MB. We don't want to trigger it unnecessarily.
(GR_API_MAX_MSG_LEN is 128 KB.)

No good reasons for changing these; just my initial thoughts - gut feeling.

Handle EAGAIN/EWOULDBLOCK in recv_all() and the sendmsg() loop so that callers using non-blocking sockets do not get spurious errors. When the error occurs mid-transfer, poll() and retry. When nothing has been read yet, propagate the error so the caller can distinguish "no data available" from a real failure. Signed-off-by: Robin Jarry <rjarry@redhat.com>

Set O_NONBLOCK on the notification sockets after subscribing. When gr_api_client_event_recv() returns EWOULDBLOCK, re-arm event_add_read() using the socket fd. After successfully reading one event, re-arm event_add_read() without any file descriptor to schedule the read callback immediately. Signed-off-by: Robin Jarry <rjarry@redhat.com>

Use sendmsg() with an iovec instead of two separate send() calls for header and payload. This halves the number of syscalls per API request on the client side. Signed-off-by: Robin Jarry <rjarry@redhat.com>

Add a read-ahead buffer to the API client so that recv_all() can serve multiple small reads from a single recv() syscall. Size this buffer so it can receive one full-size message (very unlikely to exist). When the remaining bytes to read fit in the buffer, recv() reads ahead into it; subsequent calls drain the buffer without any syscall. This is especially effective during stream iterations where many small header+payload messages arrive back-to-back. Signed-off-by: Morten Brørup <mb@smartsharesystems.com> Signed-off-by: Robin Jarry <rjarry@redhat.com>

Add a ping request that accepts an optional payload and echoes it back. The accompanying ping_perf tool sends a configurable number of ping calls in a tight loop and reports the transaction rate, useful for benchmarking API overhead. Signed-off-by: Morten Brørup <mb@smartsharesystems.com> Signed-off-by: Robin Jarry <rjarry@redhat.com>

rjarry · 2026-08-27T22:55:05Z

Following up on what I saw in the iovec implementation of evbuffer write callbacks, they can deal with up to 512K bytes in a single writev() (128 * 4K chains).

So we could size the client side buffer accordingly. But that looks a bit ridiculous. I have gone with GR_API_MAX_MSG_SIZE which seems reasonable.

About runaway memory consumption concerns: the current implementation will already buffer everything in streaming responses before writing all at once. The only way to prevent infinite memory allocation with unbounded streams would be to allow pausing/resuming streaming responses.

This is something I had already attempted in #548 but it seemed too complicated and over engineered at the moment.

MortenBroerup · 2026-08-28T10:32:40Z

+			continue;
+		}
+
+		if (remaining < sizeof(c->recv_buf)) {


With recv_buf[sizeof(struct gr_api_response) + GR_API_MAX_MSG_LEN]], this branch will always be taken. (Except when receiving a max-size message.)
In other words: We could consider changing the algorithm from conditional read-ahead, to just always read via the recv_buf.

rjarry force-pushed the api-syscall-optim branch from d82f3d7 to c7c3ea5 Compare August 26, 2026 13:40

rjarry force-pushed the api-syscall-optim branch from c7c3ea5 to f63b50a Compare August 26, 2026 14:31

MortenBroerup reviewed Aug 26, 2026

View reviewed changes

rjarry force-pushed the api-syscall-optim branch from f63b50a to 78ce613 Compare August 26, 2026 20:53

rjarry force-pushed the api-syscall-optim branch 2 times, most recently from fe047e5 to 93e7ba8 Compare August 26, 2026 21:44

MortenBroerup reviewed Aug 27, 2026

View reviewed changes

rjarry and others added 5 commits August 28, 2026 00:33

api: send request header and payload in a single syscall

5684872

Use sendmsg() with an iovec instead of two separate send() calls for header and payload. This halves the number of syscalls per API request on the client side. Signed-off-by: Robin Jarry <rjarry@redhat.com>

rjarry force-pushed the api-syscall-optim branch from 93e7ba8 to 77a56f1 Compare August 27, 2026 22:40

MortenBroerup reviewed Aug 28, 2026

View reviewed changes

Navigation Menu

Uh oh!

api: reduce syscall overhead in client and server - #714

api: reduce syscall overhead in client and server#714
rjarry wants to merge 6 commits into
DPDK:mainfrom
rjarry:api-syscall-optim

rjarry commented Aug 26, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Aug 26, 2026

Uh oh!

MortenBroerup commented Aug 26, 2026 via email

Uh oh!

MortenBroerup left a comment

Uh oh!

rjarry commented Aug 26, 2026 •

edited

Loading

Uh oh!

MortenBroerup commented Aug 27, 2026 •

edited

Loading

Uh oh!

MortenBroerup Aug 27, 2026

Uh oh!

rjarry Aug 27, 2026

Uh oh!

MortenBroerup commented Aug 27, 2026 •

edited

Loading

Uh oh!

rjarry commented Aug 27, 2026

Uh oh!

MortenBroerup Aug 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

rjarry commented Aug 26, 2026

Uh oh!

qodo-free-for-open-source-projects Bot commented Aug 26, 2026

Code Review by Qodo

Great, no issues found!

Uh oh!

MortenBroerup commented Aug 26, 2026 via email

Uh oh!

MortenBroerup left a comment

Choose a reason for hiding this comment

Uh oh!

rjarry commented Aug 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MortenBroerup commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MortenBroerup Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

rjarry Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

MortenBroerup commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rjarry commented Aug 27, 2026

Uh oh!

MortenBroerup Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rjarry commented Aug 26, 2026 •

edited

Loading

MortenBroerup commented Aug 27, 2026 •

edited

Loading

MortenBroerup commented Aug 27, 2026 •

edited

Loading