Optionally terminate connections after an idle period

[jsvd]

The beats input will send an Ack(0) whenever the writer is idle, which is handled by the IdleStateHandler.
This happens when Logstash can't send data downstream, and the Ack(0) is sent to keep the connection alive.

However when beats are sending data to multiple Logstashes it'd be best if beats were able to give up on writing a batch to a blocked Logstash and to try a different one. For this the connection must be terminated by either Logstash or Beats.

On the beats side elastic/beats#41534 (comment) was created to explore the termination of connections after a period of idleness.

This issue is to evaluate if the beats input should also have a setting to terminate the connection after an idle period.

Overall this responsibility should fall on the client as the server will provide the necessary information - Ack(0) - to the client, for it to make a decision to wether keep waiting or to give up and try again.
A couple of arguments to also implement it on the Logstash side are a) to have this feature on Logstash lumberjack to Logstash beats and b) to reduce the effort of a user to benefit from this reconnection, upgrading only a few Logstash instances and not having to upgrade a fleet of beats.
Neither of these is a very strong argument since Logstash to Logstash is moving to HTTP, and the effort argument is mostly a tactical solution and not a solid architectural solution.

That said, we can still implement this here as it would be low effort, so I'm looking for more arguments or opinions if anyone has them :)

[VihasMakwana]

I think we are going to implement this from the client side based on Craig's comment.

From the server side, it also makes sense for Logstash to close the connection after a period of inactivity. Fundamentally, neither side (client or server) should make assumptions and should simply do their best.