Does connection_acquisition_timeout cover initial DNS resolution, or is that phase unbounded?

[harshitayadavv]

Following the changes in #1215 and #1275 (connection_acquisition_timeout now covering TLS handshake, routing table fetches, etc.), I wanted to check whether the very first DNS resolution step is included in that coverage or not.

Looking at _pool.py, resolve_address() is called in a plain loop (both in the direct acquire path and in _update_routing_table_from) with no wait_for or deadline check wrapping it. The subsequent connection attempt (fetch_routing_table / socket connect) does receive acquisition_timeout, but resolution itself doesn't appear to.

In practice this means if DNS resolution hangs (e.g. a resolver silently dropping the query, rather than returning NXDOMAIN quickly), there's no bound on that phase at all, regardless of connection_acquisition_timeout or connection_timeout settings. We hit this deploying to a platform where the DNS path to an AuraDB instance was temporarily unreachable: instead of failing fast, verify_connectivity() hung for a long time with no timeout applying, which broke a startup health check with a short deadline.

Is this expected, or is DNS resolution meant to be covered by one of the existing timeout settings and I'm missing where that's enforced? If it's genuinely unbounded, would a dedicated short timeout for just the DNS phase (independent of connection_timeout, which should only start once resolution succeeds) be a reasonable ask, or is this considered out of scope given the recent timeout consolidation work?

[robsdedude]

Hi and thanks for reaching out!

This is a very good point. Ideally, connection_acquisition_timeout should also cap the time the driver waits for DNS resolution. The issue is that getaddrinfo doesn't accept a timeout configuration, neither in Python nor in the underlying libc API. For async, I could wrap it in a wait_for, but for the sync driver there'd be no equivalent thing. And the sync and async APIs (including their behavior) should deviate as little as possible with the sync API being the blueprint. It was there first and also is the simpler one in terms of machinery going on under the hood.

For the time being, and if your system is running on Linux, you might be able to adjust /etc/resolv.conf to achieve an earlier timeout: https://man7.org/linux/man-pages/man5/resolver.5.html

I will raise this with the other driver authors in the team to see how/whether other drivers handle this.

N.B. I'll be on leave for the next few weeks, so responses might be delayed.

[harshitayadavv]

Thanks for the detailed explanation, that makes sense re: getaddrinfo not supporting a timeout and keeping sync as the source of truth. Appreciate you flagging it to the team, no rush given you're heading out. Will try the resolv.conf adjustment in the meantime for our case.

[robsdedude]

I will raise this with the other driver authors in the team to see how/whether other drivers handle this.

So it turns out this is a wild mix. We have some drivers which apply the connection_acquisition_timeout to the DNS/hostname resolution other don't. Of the former, some cancel the resolution on timeout, others leave it running in the backgroud ignoring when it finishes.

I'm torn on this. As said, ideally I'd like the driver to respect the acquisition timeout even during hostname resolution. But technically that'd mean introducing background threads (which the driver is currently free of). This adds complexity and crucially overhead. Did editing resolv.conf alleviate the problem for you?