Client connection recovery

A network failure can be silent. No error arrives, and the channel looks open, but nothing will ever come back. A program that waits on such a channel hangs forever. This appendix explains how transport pings detect a dead connection, and how to tune them. Most programs can keep the defaults and skip this page.

get_authenticated_channel() sends a transport ping every 900 seconds by default. If the broker does not acknowledge a ping in time, the transport closes the channel. The pending calls then fail with an error instead of hanging, and your code can open a new channel. Three parameters control this:

  • keepalive_time is the ping interval in seconds. Pass None to turn pings off.

  • keepalive_timeout is how long to wait for the acknowledgement. It defaults to the value in the supplied grpclib Configuration, or to 20 seconds.

  • keepalive_permit_without_calls also pings when no call is active. It defaults to the Configuration value, or to off.

Both durations must be positive and finite. Zero, negative, non-finite, and boolean values are rejected.

Added in version 6.0.1: The keepalive_timeout, keepalive_permit_without_calls, and config parameters of get_authenticated_channel().

Choose faster detection

Pass keepalive_time=15, keepalive_timeout=10 to detect a silent failure within at most 25 seconds, subject to event-loop scheduling. Reconnect delay and authentication time come on top of that. A worker with a 5-second retry cap and a 10-second authentication timeout can make its next connection attempt within 30 seconds, and complete an available connection within 40 seconds. The broker’s normal session timeout is 60 seconds, so the worker returns before its session expires.

Details for channel owners

An outstanding getTask call counts as an active call even when no work matches. Pings therefore continue during a healthy long wait. A channel with no active calls sends no pings unless keepalive_permit_without_calls is on.

A ping proves that the transport is alive. It does not prove that the broker application makes progress, and it says nothing about task ownership. When a channel fails, the caller must replace the authenticated channel and cancel its old work.

A supplied config=Configuration(...) provides window sizes, TLS hostname settings, the acknowledgement timeout, and the idle-call policy. Explicit keepalive arguments override those policy values. keepalive_time always wins, including its default of 900. The helper permits unlimited pings without data frames. It sets the minimum ping spacing to the smaller of one second and the ping interval. Confirm that the broker and any proxies between you and the broker permit your chosen ping rate. A peer that rejects frequent pings can close the connection instead.

Transport tests exercise silent loss, explicit disconnect, idle channels, and long calls over TCP, including certificate-verified TLS, against grpclib in the supported >=0.4.9,<0.5 range. The minimum excludes 0.4.7, whose stream reset after a TLS disconnect can raise AttributeError. Policy log lines contain durations and flags, never authentication credentials. Debug logging on the grpclib.protocol logger shows transport close reasons, including keepalive expiry.