Grpc long connection keepalive

background

Encountered in the project grpc The two-way flow and one-way flow appear the service end flow context Report errors ：context cancel error, And the client is not aware of the disconnection , Still think it's still connected , Cause data transmission interruption .

In this case, think about the possibility of the problem from the following dimensions ：

• Is the network stable , Can you reproduce
• Whether there is a connection between the client and the server context Conduct cancel
• Whether the client and server have to deal with context Of cancel The signal

Because the problem is not in the development environment , Therefore, you can only check the log at first , But only the server will receive in the log context Of cancel The signal , The client does not have any exceptions . So try to replicate it in the development environment , This exception is reproduced by unplugging the client network cable . The solution is simple , You can add heartbeat to the streaming interface , If you haven't received your heartbeat for a long time, you'll be disconnected and reconnected . But is there a more concise and elegant way ？ yes , we have , The answer is grpc Of keepalive Parameters .

KeepAlive

keepalive ping It is a kind of transmission HTTP2 ping To check whether the channel is currently working . It is sent periodically , If ping No confirmation from the peer within a certain timeout , Then the transmission is disconnected .

Client's grpc keepalive Parameters

// ClientParameters is used to set keepalive parameters on the client-side.
// These configure how the client will actively probe to notice when a
// connection is broken and send pings so intermediaries will be aware of the
// liveness of the connection. Make sure these parameters are set in
// coordination with the keepalive policy on the server, as incompatible
// settings can result in closing of connection.
type ClientParameters struct {
    
	// After a duration of this time if the client doesn't see any activity it
	// pings the server to see if the transport is still alive.
	// If set below 10s, a minimum value of 10s will be used instead.
	Time time.Duration // The current default value is infinity.
	// After having pinged for keepalive check, the client waits for a duration
	// of Timeout and if no activity is seen even after that the connection is
	// closed.
	Timeout time.Duration // The current default value is 20 seconds.
	// If true, client sends keepalive pings even with no active RPCs. If false,
	// when there are no active RPCs, Time and Timeout will be ignored and no
	// keepalive pings will be sent.
	PermitWithoutStream bool // false by default.
}

keepalive.ClientParameters The meaning of the parameter is as follows :

• Time： without activity, Then send one... Every this value ping package
• Timeout： If ping ack If it is not returned within this value, the connection is considered disconnected
• PermitWithoutStream： without active Of stream, Is it allowed to send ping

Server side grpc keepalive Parameters

// ServerParameters is used to set keepalive and max-age parameters on the
// server-side.
type ServerParameters struct {
    
	// MaxConnectionIdle is a duration for the amount of time after which an
	// idle connection would be closed by sending a GoAway. Idleness duration is
	// defined since the most recent time the number of outstanding RPCs became
	// zero or the connection establishment.
	MaxConnectionIdle time.Duration // The current default value is infinity.
	// MaxConnectionAge is a duration for the maximum amount of time a
	// connection may exist before it will be closed by sending a GoAway. A
	// random jitter of +/-10% will be added to MaxConnectionAge to spread out
	// connection storms.
	MaxConnectionAge time.Duration // The current default value is infinity.
	// MaxConnectionAgeGrace is an additive period after MaxConnectionAge after
	// which the connection will be forcibly closed.
	MaxConnectionAgeGrace time.Duration // The current default value is infinity.
	// After a duration of this time if the server doesn't see any activity it
	// pings the client to see if the transport is still alive.
	// If set below 1s, a minimum value of 1s will be used instead.
	Time time.Duration // The current default value is 2 hours.
	// After having pinged for keepalive check, the server waits for a duration
	// of Timeout and if no activity is seen even after that the connection is
	// closed.
	Timeout time.Duration // The current default value is 20 seconds.
}

// EnforcementPolicy is used to set keepalive enforcement policy on the
// server-side. Server will close connection with a client that violates this
// policy.
type EnforcementPolicy struct {
    
	// MinTime is the minimum amount of time a client should wait before sending
	// a keepalive ping.
	MinTime time.Duration // The current default value is 5 minutes.
	// If true, server allows keepalive pings even when there are no active
	// streams(RPCs). If false, and client sends ping when there are no active
	// streams, server will send GOAWAY and close the connection.
	PermitWithoutStream bool // false by default.
}

keepalive.EnforcementPolicy：

• MinTime： If the client twice ping The interval between is less than this value , Close the connection
• PermitWithoutStream： Even without active stream, It is also allowed to ping

keepalive.ServerParameters：

• MaxConnectionIdle： If one client Idle exceeds this value , Send a GOAWAY, To prevent sending a large number of at the same time GOAWAY, Will float up and down this time interval 10%, For example, set to 15s, namely 15+1.5 perhaps 15-1.5
• MaxConnectionAge： If the lifetime of any connection exceeds this value , Send a GOAWAY
• MaxConnectionAgeGrace： Between forcibly closing the connection , Allow time with this value to complete pending Of rpc request
• Time： If one client Idle exceeds this value , Send a ping request
• Timeout： If ping Request no reply received within this time period , The connection is considered disconnected

Official sample code

server

https://github.com/grpc/grpc-go/blob/master/examples/features/keepalive/server/main.go

client

https://github.com/grpc/grpc-go/blob/master/examples/features/keepalive/client/main.go