Disabling allowLinkStealing not working in HA mode

[eanbiso]

Hi team, I found an unexpected behavior that I would like to discuss with you.

Test environment:

• Artemis broker in k8s deployment in HA mode, with 2 replicas up and running (spec.deploymentPlan.size=2 set in the Artemis CRD)
• versions:
  • Artemis Operator ver. 1.0.16
  • Artemis Broker Image ver. 1.0.23
  • Artemis Broker Init Image ver. 1.0.23

Test scenario:

• 2 MQTT clients (A and B) that want to connect with the same clientId to an MQTT acceptor exposed by the Artemis Broker in HA mode (2 replicas) and reachable via a Load Balancer that balances the load on the two pods
• allowLinkStealing option disabled on the MQTT acceptor via broker properties acceptorConfigurations.<ACCEPTOR_NAME>.extraParams.allowLinkStealing=false
• MQTT Client A already connected with clientId DemoClient1
• MQTT Client B that tries to connect with the same clientId DemoClient1 to the same acceptor

Observed behavior:

• despite the option allowLinkStealing=false, the second client still manages to open an mqtt connection under certain conditions, stealing the connection from the first client. In practice, the behavior seems to depend on the replica to which the connection is redirected:
  -- the connection is rejected if the connection attempt coming from the MQTT Client B is routed to the same replica that manages the connection of the MQTT Client A; in this case we observed the (expected) log
  2023-12-11 14:19:16,158 DEBUG [org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler] Incoming MQTT session from 127.0.0.6:49391 closed due to existing session from 127.0.0.6:42435 with the same client ID: DemoClient1
  -- the connection is accepted if the connection attempt coming from the MQTT Client B is routed to a different replica (i.e. not that one managing the connection of the MQTT Client A): in this case the MQTT Client B is able to connect to the broker, stealing the connection to the MQTT Client A (that is disconnected), and no log is shown

Behavior I would expect:
like in the case we are not in HA mode (i.e. we have a single broker replica, with spec.deploymentPlan.size=1), I would expect the second MQTT client (MQTT Client B in this case) is never able to connect to the broker if a client with the same clientId is already connected and the broker property acceptorConfigurations.<ACCEPTOR_NAME>.extraParams.allowLinkStealing=false is configured, despite the number of replicas

Can you help here?
Many thanks in advance

[brusdev]

@eanbiso what do you mean with Artemis broker in k8s deployment in HA mode? Did you use broker properties to enable Apache ActiveMQ Artermis HA?

[eanbiso]

Hi @brusdev , I meant deploying the broker with a number of pods bigger than 1, setting the property spec.deploymentPlan.size to a value >= 2, and setting the property spec.deploymentPlan.persistenceEnabled to true on the Artemis CRD.

Note: the broker seems to be properly configured (with the cluster connection bridge working fine) since we see the log:

INFO  [org.apache.activemq.artemis.core.server] AMQ221027: Bridge ClusterConnectionBridge@7edd8bf6 [name=$.artemis.internal.sf.my-cluster.3bc7c846-99d0-11ee-839f-02f320d447a2, queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.3bc7c846-99d0-11ee-839f-02f320d447a2, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], temp=false]@7082d9ae targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@7edd8bf6 [name=$.artemis.internal.sf.my-cluster.3bc7c846-99d0-11ee-839f-02f320d447a2, queue=QueueImpl[name=$.artemis.internal.sf.my-cluster.3bc7c846-99d0-11ee-839f-02f320d447a2, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=amq-broker], temp=false]@7082d9ae targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=amq-broker-ss-1-amq-broker-hdls-svc-demo-svc-cluster-local], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@926188689[nodeUUID=319ec50a-99d0-11ee-b22b-1e89dc031c98, connector=TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=amq-broker-ss-0-amq-broker-hdls-svc-demo-svc-cluster-local, address=, server=ActiveMQServerImpl::name=amq-broker])) [initialConnectors=[TransportConfiguration(name=artemis, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=amq-broker-ss-1-amq-broker-hdls-svc-demo-svc-cluster-local], discoveryGroupConfiguration=null]] is connected

[jbertram]

Here's the related thread on the ActiveMQ users mailing list. As far as I can tell this isn't an issue with the operator and therefore should not be addressed here.

[eanbiso]

Thanks, I will follow that discussion thread