Skip to content

feat: fail fast on unrecoverable auth errors during reconnect - #22

Merged
KARTIKrocks merged 1 commit into
mainfrom
feat/fail-fast-auth-reconnect
Jul 27, 2026
Merged

feat: fail fast on unrecoverable auth errors during reconnect#22
KARTIKrocks merged 1 commit into
mainfrom
feat/fail-fast-auth-reconnect

Conversation

@KARTIKrocks

@KARTIKrocks KARTIKrocks commented Jul 27, 2026

Copy link
Copy Markdown
Owner

When a reconnection dial is rejected for bad credentials, an unusable SASL mechanism, or no vhost access (AMQP 403 AccessRefused / 530 NotAllowed), handleReconnect now surfaces the error via OnDisconnect and stops instead of backing off and re-submitting the same rejected parameters forever. Transient failures (network errors, 320 ConnectionForced from a broker restart) keep retrying with the existing exponential backoff.

Classification is by AMQP reply code, not amqp.Error.Recoverable(): the dial-time auth sentinels (ErrCredentials, ErrSASL, ErrVhost) are struct literals whose Recover field is false, so Recoverable() reports false for exactly these errors. Dial errors are now wrapped with %w so callers can errors.As them to *amqp.Error; errors.Is(err, ErrConnectionClosed) still matches.

OnDisconnect now fires on any terminal give-up: both the auth abort and the pre-existing MaxReconnectAttempts-exhausted path invoke it once more with the terminal error (a *amqp.Error, or the previously-unused ErrMaxReconnects sentinel), so a permanently dead connection is observable, not just logged.

Also fixes a pre-existing latent panic: a clean broker close delivers a nil *amqp.Error, which as an interface value is non-nil but panics when a handler calls err.Error(); it is now normalized to ErrConnectionClosed before the callback runs.

Bumps amqp091-go to v1.13.0 (data-race, ack, and TLS/SASL hardening fixes consumed transparently). Verified with -race unit tests and the full integration suite on RabbitMQ 4 and 3.13.

Summary

Motivation

Fixes #

Changes

Checklist

  • fmt, vet, lint, test, build passes (make all)
  • New code has tests where appropriate
  • Breaking changes are documented

Summary by CodeRabbit

  • Bug Fixes

    • Reconnection now stops promptly for unrecoverable authentication and authorization failures.
    • OnDisconnect now reports terminal authentication errors and exhausted reconnect attempts.
    • Clean disconnects consistently report ErrConnectionClosed instead of an invalid nil error.
    • Connection errors now preserve their underlying details for reliable inspection.
  • Documentation

    • Clarified reconnection behavior, retry limits, and OnDisconnect callback outcomes.
    • Updated release notes for version 0.9.0.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@KARTIKrocks, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 35 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 10485294-4669-4a6a-bbc7-35d807695bd0

📥 Commits

Reviewing files that changed from the base of the PR and between 73611b9 and 44b6ba5.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • README.md
  • config_test.go
  • integration_test.go
  • rabbitmq.go
📝 Walkthrough

Walkthrough

Reconnection now stops on permanent authentication or authorization failures, reports terminal errors through OnDisconnect, normalizes typed-nil disconnect errors, and reports ErrMaxReconnects after bounded retries. Tests, documentation, and changelog entries cover the updated behavior.

Changes

Reconnect error handling

Layer / File(s) Summary
Dial and disconnect error contracts
rabbitmq.go, config_test.go
Permanent AMQP dial errors are classified, disconnect errors are normalized to ErrConnectionClosed, and wrapped errors preserve unwrapping behavior.
Reconnect termination and callback flow
rabbitmq.go, integration_test.go
Reconnect aborts on unrecoverable authentication failures and invokes OnDisconnect; exhausted attempts report ErrMaxReconnects, with integration coverage for both paths.
Callback documentation and behavior validation
rabbitmq.go, README.md, CHANGELOG.md, config_test.go, integration_test.go
OnDisconnect semantics and backoff behavior are documented, and release notes record the reconnection changes and dependency update.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Connection
  participant RabbitMQ
  participant OnDisconnect
  Connection->>RabbitMQ: attempt reconnect
  RabbitMQ-->>Connection: dial result
  Connection->>Connection: classify permanent error
  Connection->>OnDisconnect: terminal error or ErrMaxReconnects
Loading

Poem

Backoff winds through queues of rain,
A stubborn login breaks the chain.
Typed nils turn safely into light,
Max attempts ends the retry night.
OnDisconnect tells the tale.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: fail-fast reconnect behavior for unrecoverable auth errors.
Description check ✅ Passed The description covers the required sections and includes summary, motivation, checklist, and verification details.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/fail-fast-auth-reconnect

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov-commenter

codecov-commenter commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.00000% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rabbitmq.go 64.00% 9 Missing ⚠️

📢 Thoughts on this report? Let us know!

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rabbitmq.go`:
- Around line 341-343: Protect all three onDisconnect invocation sites in
handleReconnect from panics and blocking callback execution, while preserving
the documented before-reconnection ordering. Add a safeOnDisconnect helper that
recovers callback panics and invoke it instead of calling onDisconnect directly
at each site; ensure the reconnect loop can still observe closeCh without an
unbounded user callback blocking it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 023b050a-4f70-45ba-a0c2-b5756e53ab34

📥 Commits

Reviewing files that changed from the base of the PR and between 2d62c97 and 73611b9.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • README.md
  • config_test.go
  • integration_test.go
  • rabbitmq.go

Comment thread rabbitmq.go Outdated
When a reconnection dial is rejected for bad credentials, an unusable SASL
mechanism, or no vhost access (AMQP 403 AccessRefused / 530 NotAllowed),
handleReconnect now surfaces the error via OnDisconnect and stops instead of
backing off and re-submitting the same rejected parameters forever. Transient
failures (network errors, 320 ConnectionForced from a broker restart) keep
retrying with the existing exponential backoff.

Classification is by AMQP reply code, not amqp.Error.Recoverable(): the
dial-time auth sentinels (ErrCredentials, ErrSASL, ErrVhost) are struct
literals whose Recover field is false, so Recoverable() reports false for
exactly these errors. Dial errors are now wrapped with %w so callers can
errors.As them to *amqp.Error; errors.Is(err, ErrConnectionClosed) still matches.

OnDisconnect now fires on any terminal give-up: both the auth abort and the
pre-existing MaxReconnectAttempts-exhausted path invoke it once more with the
terminal error (a *amqp.Error, or the previously-unused ErrMaxReconnects
sentinel), so a permanently dead connection is observable, not just logged.

Hardening of the OnDisconnect path:
- A clean broker close delivers a nil *amqp.Error, which as an interface value
  is non-nil but panics when a handler calls err.Error(); it is normalized to
  ErrConnectionClosed before the callback runs.
- All callback invocations are routed through safeOnDisconnect, which recovers
  and logs a panicking callback instead of letting it crash the reconnect
  goroutine (and the process). Callbacks remain synchronous to preserve the
  documented before-reconnection ordering.

Bumps amqp091-go to v1.13.0 (data-race, ack, and TLS/SASL hardening fixes
consumed transparently). Verified with -race unit tests and the full
integration suite on RabbitMQ 4 and 3.13.
@KARTIKrocks
KARTIKrocks force-pushed the feat/fail-fast-auth-reconnect branch from 73611b9 to 44b6ba5 Compare July 27, 2026 11:33
@KARTIKrocks
KARTIKrocks merged commit 6b473be into main Jul 27, 2026
10 checks passed
@KARTIKrocks
KARTIKrocks deleted the feat/fail-fast-auth-reconnect branch July 27, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants