Too many pings and one client always disconnects

[ajulyav]

Describe the bug

grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "Too many pings"
	debug_error_string = "UNKNOWN:Error received from peer ipv4:192.168.229.99:5040 {grpc_message:"Too many pings", grpc_status:14, created_time:"2024-10-07T15:40:46.164225255+02:00"}"
>

I've got my grpc server settings as:

        ("grpc.http2.max_pings_without_data", 0),
        # Is it permissible to send keepalive pings from the client without
        # any outstanding streams. More explanation here:
        # https://github.com/adap/flower/pull/2197
        ("grpc.keepalive_permit_without_calls", 0),

but it does not help though

later, i added up two options:

        ("grpc.http2.max_ping_strikes", 0),
        ("grpc.http2.min_ping_interval_without_data_ms", 10)

it allowed me escape the initial error, but then I have:

    raise GrpcBridgeClosed()
flwr.server.superlink.fleet.grpc_bidi.grpc_bridge.GrpcBridgeClosed

Steps/Code to Reproduce

I use basic FedAvg strategy except that i send additional round of evaluation on each client during aggregate_fit
EvaluateRes = client_proxy.evaluate(ins = evaluate_ins, timeout = None, group_id=rnd) . Sometimes when rerun the clients and server, the error happens after 1 successful round, so it is not always happens the same moment.

Expected Results

Client stays alive

Actual Results

Client disconnects

[oabuhamdan]

Did you come to a solution?

[ajulyav]

Did you come to a solution?

Hello, I am still encountering this problem, and it occurs quite randomly. A few things have helped me reduce the frequency of this issue:

1. Run the server and clients on the same machine, so you can use "localhost" as the server address.
2. If you're using loops that send messages to clients, try replacing the loop with non-loop code.

[WilliamLindskog]

Hi @ajulyav,

Thanks for raising this. Are you still experiencing this issue?