Replication drops connections and gives up on peers without logging a cause

[vharseko]

Three paths around the replication port drop a connection, or give up on a peer, without logging a message an administrator can act on. They are the complement of #906, which covers the inbound SSL handshake failure and a certificate nickname missing from the ads-truststore.

The accept loop swallows a whole failure class

ReplicationServer.java:307:

catch (Exception e)
{
  // If problems happen during the SSL handshake, it is necessary
  // to close the socket to free the associated resources.
  if (newSocket != null)
  {
    newSocket.close();
  }
  continue;
}

Nothing is logged, at any level. createServerSession() handles SSLException itself, so what lands here is everything else — in particular ConfigException from CryptoManagerImpl.getSslContext(), which is reachable at runtime: TrustStoreBackend re-reads the store file on every getKeyManagers() and getTrustManagers() call, so an ads-truststore deleted, truncated or chmod'ed after startup makes the server drop every inbound replication connection in silence.

The outbound RS to RS connect logs no message

ReplicationServer.java:490:

catch (Exception e)
{
  logger.traceException(e);
  close(session);
  close(socket);
  return false;
}

traceException carries no message, so a replication server whose outgoing handshake fails logs nothing in logs/errors. The failure is only visible in the log of the peer, which is the machine whose configuration is correct.

The data server side names no cause

When a data server cannot connect to any replication server it logs WARN_COULD_NOT_FIND_CHANGELOG / WARN_NO_AVAILABLE_CHANGELOGS (ReplicationBroker.java:858 and :863). Neither message carries the underlying exception, so a certificate rejection, a network failure and a wrong port all read as "no changelog".

Suggestion

• Log the swallowed exception in the accept loop, throttled the way ReplSessionSecurity throttles handshake failures — every TCP probe reaching port 8989 goes through this loop, so the rate has to be bounded.
• Give the RS to RS connect failure a message naming the remote server and the cause.
• Carry the last connection error into the "no changelog" warnings.

Raised in the review of #906 by @maximthomas.