Skip to content

Add deadline, timeout, and hot-path controls to the connection loop - #7

Merged
lpgauth merged 3 commits into
masterfrom
bidder-workload-features
Jul 27, 2026
Merged

lpgauth merged 3 commits into
masterfrom
bidder-workload-features

Conversation

@lpgauth

@lpgauth lpgauth commented Jul 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Extends whitecap with the controls a latency-sensitive workload needs to honor a per-request deadline and stay resilient under load, plus two hot-path improvements. All new timeouts default to infinity and fall back to a set receive_timeout, so behaviour is unchanged unless configured.

  • Per-request arrival timestamp. #whitecap_req{} gains a received field (os:system_time(), native units), stamped when the first byte arrives, so a handler can measure time already spent on the wire and shed against a deadline.
  • Split idle vs. in-request timeouts. receive_timeout splits into keepalive_timeout (idle wait for the next request) and request_timeout (per-read wait once a request has started arriving). A short request_timeout gives slowloris protection without dropping healthy idle keep-alive connections. receive_timeout, if set, remains the back-compat default for both.
  • Optional handler deadline. handler_timeout (default infinity). A finite value runs the handler in a monitored process; an overrun is killed and answered 504, emitting [whitecap, handler, timeout]. infinity keeps the handler inline with no per-request process — the right choice when the handler enforces its own deadline.
  • Hoisted config. handler_timeout, keepalive_timeout, max_keepalive, and request_timeout are read once at connection start rather than on every request/read.
  • Length-aware body reads. A body with a known Content-Length is read with an exact-length gen_tcp:recv, landing in one read and one concat instead of repeatedly re-buffering partial reads (previously O(n²) on a body fragmented across TCP segments).

Docs (README config + telemetry tables, received usage note) and CHANGELOG updated under ## Unreleased. No version bump.

Verified: strict compile (warnings-as-errors), xref, and dialyzer clean; eunit 32 passed, including 4 new tests covering the arrival timestamp, split-body reassembly, 504 on handler overrun (~100ms), and 408 on a stalled request (~300ms).

@lpgauth lpgauth changed the title Add deadline, timeout, and hot-path controls for bidders Add deadline, timeout, and hot-path controls to the connection loop Jul 10, 2026
@lpgauth
lpgauth force-pushed the bidder-workload-features branch from 3baa122 to e8306fd Compare July 10, 2026 10:47
Extend whitecap with the controls a latency-sensitive workload needs
to honor a per-request deadline and stay resilient under load, plus
two hot-path improvements.

Requests now carry a `received` arrival timestamp so a handler can
measure time already spent on the wire and shed against its deadline.
The single `receive_timeout` is split into `keepalive_timeout` (idle
wait for the next request) and `request_timeout` (per-read wait once a
request has started), allowing a short in-request deadline for
slowloris protection without dropping healthy idle keep-alive
connections. An optional `handler_timeout` bounds handler run time,
answering 504 and emitting a `[whitecap, handler, timeout]` event; it
defaults to infinity and stays inline with no per-request process.

Per-connection config is read once at connection start rather than on
every request, and a body with a known Content-Length is read with an
exact-length recv instead of repeatedly re-buffering partial reads.

All new timeouts default to infinity and fall back to a set
`receive_timeout`, so behaviour is unchanged unless configured.
@lpgauth
lpgauth force-pushed the bidder-workload-features branch from e8306fd to ea73b6b Compare July 24, 2026 13:50
lpgauth added 2 commits July 27, 2026 09:27
The timeout branch demonitored, killed the handler, then drained its
mailbox with a non-blocking receive. The kill and the handler's result
travel in opposite directions, so there is no ordering between them: a
result sent in the kill race can arrive after the drain and then linger
forever under a fresh per-request ref. On a long-lived keep-alive
connection that is a slow memory leak and, via selective-receive
scanning, a creeping latency regression.

Kill first, then block for either the result or the guaranteed DOWN.
Same-sender signal ordering delivers a raced result before the DOWN, so
both are consumed and no message is orphaned.
Document that a finite handler_timeout kills the handler with
exit(kill), which does not release resources it held (pool checkouts,
locks), so such handlers must be side-effect-safe.
@lpgauth
lpgauth merged commit f5e4b2f into master Jul 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant