You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
With the Mint adapter, a server GOAWAY that races in-flight requests crashes the whole ConnectionProcess with a FunctionClauseError instead of failing just the raced requests. The
crash is an abnormal exit, so it bypasses both the adapter's retry/attempt_reconnect loop and the {:elixir_grpc, :connection_down, pid} parent notification (both only run on the
graceful-disconnect path). The channel owner only gets a linked {:EXIT, pid, {:function_clause, ...}} message (connect/2 traps exits on the caller), so unless it handles that
explicitly it keeps serving a channel whose conn_pid is dead — every subsequent call fails with :noproc, and callers that had requests in flight exit or hang.
Mechanism: Mint.HTTP2.stream/2 returns requests that were in flight when GOAWAY arrived as {:error, request_ref, %Mint.HTTPError{reason: :unprocessed}} entries in the responses list
(documented Mint behavior). ConnectionProcess.process_response/2 only has clauses for :status, :headers, :data and :done, so the error entry raises.
To Reproduce
Steps to reproduce the behavior:
Connect with the Mint adapter to an HTTP/2 server that recycles idle connections with GOAWAY (e.g. an nginx ingress with default keepalive settings, ~60s idle).
Keep a steady stream of unary calls so requests are in flight when GOAWAY lands.
Expected behavior
The raced requests fail with a per-request error delivered to their stream response processes (:unprocessed is retryable by contract — the server did not process the request), and the
connection process stays alive so connection-level GOAWAY handling goes through the existing graceful paths.
Logs
** (FunctionClauseError) no function clause matching in GRPC.Client.Adapters.Mint.ConnectionProcess.process_response/2
(grpc 1.0.2) lib/grpc/client/adapters/mint/connection_process/connection_process.ex:248:
GRPC.Client.Adapters.Mint.ConnectionProcess.process_response(
{:error, #Reference<...>, %Mint.HTTPError{reason: :unprocessed, module: Mint.HTTP2}},
%GRPC.Client.Adapters.Mint.ConnectionProcess.State{conn: %Mint.HTTP2{state: {:goaway, :no_error, ""}, ...}, ...})
(grpc 1.0.2) lib/grpc/client/adapters/mint/connection_process/connection_process.ex:212:
GRPC.Client.Adapters.Mint.ConnectionProcess.handle_info/2
Protos
Not proto-specific — any unary call in flight at GOAWAY time triggers it.
Versions:
OS: Linux (Docker on AWS EKS)
Erlang: OTP 28
Elixir: 1.20.2
mix.lock(grpc, gun, cowboy, cowlib): grpc 1.0.2 (hex), grpc_core 1.0.2, gun not installed (Mint adapter), cowboy 2.17.0, cowlib 2.18.0
Additional context
In production behind an nginx ingress this fires several times per hour under steady traffic; each occurrence kills or wedges the background jobs whose calls were in flight. Fix proposed
in #562 (adds the missing process_response({:error, request_ref, error}, state) clause, mirroring the connection-close path; includes a deterministic regression test).
Describe the bug
With the Mint adapter, a server GOAWAY that races in-flight requests crashes the whole
ConnectionProcesswith aFunctionClauseErrorinstead of failing just the raced requests. Thecrash is an abnormal exit, so it bypasses both the adapter's
retry/attempt_reconnectloop and the{:elixir_grpc, :connection_down, pid}parent notification (both only run on thegraceful-disconnect path). The channel owner only gets a linked
{:EXIT, pid, {:function_clause, ...}}message (connect/2traps exits on the caller), so unless it handles thatexplicitly it keeps serving a channel whose
conn_pidis dead — every subsequent call fails with:noproc, and callers that had requests in flight exit or hang.Mechanism:
Mint.HTTP2.stream/2returns requests that were in flight when GOAWAY arrived as{:error, request_ref, %Mint.HTTPError{reason: :unprocessed}}entries in the responses list(documented Mint behavior).
ConnectionProcess.process_response/2only has clauses for:status,:headers,:dataand:done, so the error entry raises.To Reproduce
Steps to reproduce the behavior:
regression test in fix(mint): handle stream-level errors in process_response instead of crashing #562).
Expected behavior
The raced requests fail with a per-request error delivered to their stream response processes (
:unprocessedis retryable by contract — the server did not process the request), and theconnection process stays alive so connection-level GOAWAY handling goes through the existing graceful paths.
Logs
Protos
Not proto-specific — any unary call in flight at GOAWAY time triggers it.
Versions:
Additional context
In production behind an nginx ingress this fires several times per hour under steady traffic; each occurrence kills or wedges the background jobs whose calls were in flight. Fix proposed
in #562 (adds the missing
process_response({:error, request_ref, error}, state)clause, mirroring the connection-close path; includes a deterministic regression test).