Skip to content

Idle connections cost 74 KB each, 28% more than haproxy, mostly OpenSSL buffers #72

Description

@novotimo

Measured with benchmark/bench.sh -m idle at 6e847f2, holding connections
open rather than churning them, on four physical cores with the same certificate
and TLS parameters for all three subjects:

connections held tlsproxy nginx stream haproxy tcp
1,000 86 MB 184 MB 99 MB
5,000 417 MB 430 MB 334 MB
20,000 1,459 MB 1,355 MB 1,175 MB

Taking the slope between 1k and 20k, so that the fixed footprint drops out, the
marginal cost of one held connection is 74 KB for tlsproxy, 63 KB for nginx
and 58 KB for haproxy
. We are the most expensive of the three.

This only shows up when connections are held. A churn benchmark has
few live at any instant, and by that measure we looked several times better
than we are, so the honest number is this one.

The composition is worth separating, because the two halves point different
ways. Our fixed footprint is the smallest by a wide margin: 15 MB against
haproxy's 48 MB and nginx's 126 MB. It is the per-connection part that is
worst. 74 KB is roughly OpenSSL's read and write buffers, which default to
about 16 KB each plus overhead, on top of the two TPX_NET_BUFSIZE chunks in
the bufq pair.

SSL_MODE_RELEASE_BUFFERS tells OpenSSL to free the read and write buffers
whenever a connection has no data in flight, at the cost of reallocating them
when it next does. nginx sets it. For a proxy whose stated use is many
long-lived and mostly idle connections, an MQTT broker or a game server or an
RPC service, that trade is the right way round, since the buffers are idle
almost all the time.

Worth measuring rather than assuming, since it costs an allocation per wake-up
on a busy connection: bench.sh -m idle gives the memory side and bench.sh -m bulk and -m rate give whatever it costs the data path, so the trade is
directly observable before and after.

The other component is ours. Two 16 KB bufq chunks per proxy are allocated
whether or not the connection is carrying anything; allocating the first chunk
lazily on first use would take an idle pair down to the proxy_t itself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions