Additional Causes for Disconnect Code 142: SessionTakenOver?

[EandrewJones]

I'm trying to create a socket connection pool for celery tasks to use for streaming purposes. Whenever I create a new socket connection, it kills all existing connections with disconnect code: 142 (SessionTakenOver).

The docs state the reason as:

Returned when the server has established another connection with the same client ID as a client's current connection, causing the current client to become disconnected.

However, that cannot be the cause in my case. Every connection in the pool has a randomly generated input_client_id.

Code excerpt for socket connection class

class SocketConnection:
    def __init__(
        self,
        input_client_name: str,
        input_proxy_host: str,
        input_proxy_port: int,
        input_signing_region: str,
        input_endpoint: str,
        input_cognito_identity: str,
        input_client_id: str = f"connection-{str(uuid4())}",
        callback: callable = None,
        subscriptions: List[mqtt5.Subscription] = None,
        session_behavior=mqtt5.ClientSessionBehaviorType.CLEAN,
        keep_alive_secs=30,
        session_expiry_interval_secs=60,
        logger: Optional[logging.Logger] = None,
    ):
    ...
        try:
            self.logger.info(
                f"[{self.name}]: Creating MQTT client with client id: {input_client_id}"
            )
            client_options = mqtt5.ClientOptions(
                host_name=input_endpoint,
                http_proxy_options=self.proxy_options,
                session_behavior=session_behavior,
            )
            self.client = mqtt5_client_builder.websockets_with_default_aws_signing(
                endpoint=input_endpoint,
                region=input_signing_region,
                credentials_provider=self.credentials_provider,
                client_id=input_client_id,
                client_options=client_options,
                on_publish_received=self.on_publish_received,
                on_lifecycle_stopped=self.on_lifecycle_stopped,
                on_lifecycle_connection_success=self.on_lifecycle_connection_success,
                on_lifecycle_connection_failure=self.on_lifecycle_connection_failure,
                on_lifecycle_disconnection=self.on_lifecycle_disconnection,
            )
        except Exception as e:
            self.logger.error(f"Error creating MQTT client: {e}")
            raise e

Log Excerpt

services-agents_service-1      | [2023-12-24 16:29:33,787 | socket.py | __init__ | INFO ] [94cd4d91-0dec-4985-9d6d-a9d2ec5fed10]: Creating MQTT client with client id: connection-95f38079-331f-4a4c-91b2-8933a3ee9e02
services-celery_agents-1       | [2023-12-24 16:29:33,831 | socket.py | __init__ | INFO ] [68dc7f7b-6d28-4e8e-ada3-d6f196d04330]: Creating MQTT client with client id: connection-5575dd6b-8021-423a-a091-f28df11ffe15
services-agents_service-1      | [2023-12-24 16:29:34,440 | socket.py | on_lifecycle_disconnection | INFO ] [3f196c5c-4c1e-4381-afc7-8c1cfe06782b]: Lifecycle Disconnected
services-agents_service-1      | [2023-12-24 16:29:34,440 | socket.py | on_lifecycle_disconnection | INFO ]     Disconnection packet code: 142
services-agents_service-1      | [2023-12-24 16:29:34,440 | socket.py | on_lifecycle_disconnection | INFO ]     Disconnection packet reason: None

So something else must be the culprit or there's a bug, but I have no clue. Ideas?

[bretambrose]

Please attach CRT logs for the relevant time period. A one file repro sample could be helpful as well.

In the short-term, you could step into client instantiation to this point here: https://github.com/aws/aws-iot-device-sdk-python-v2/blob/main/awsiot/mqtt5_client_builder.py#L359
and look at what client_options.connect_options.client_id is. Its final value is a function of a number of things.

[EandrewJones]

Thank you and happy holidays! Will do and get back with logs + one-file repro.

[EandrewJones]

I worked through a one file repro and discovered the problem.

class SocketConnection:
    def __init__(
        self,
        input_client_name: str,
        input_proxy_host: str,
        input_proxy_port: int,
        input_signing_region: str,
        input_endpoint: str,
        input_cognito_identity: str,
        input_client_id: str = f"connection-{str(uuid4())}", <---- This does not work
        callback: callable = None,
        subscriptions: List[mqtt5.Subscription] = None,
        session_behavior=mqtt5.ClientSessionBehaviorType.CLEAN,
        keep_alive_secs=30,
        session_expiry_interval_secs=60,
        logger: Optional[logging.Logger] = None,
    ):
    ...

Calling uuidv4 in the class definition leads to a single hard-coded default rather than calling the function on every instantiation. Passing a callable function also does not work. I had to move the randomization into __init__ itself.

Though the logs showed randomized client ids, they were not randomizing under the hood. Not sure why the discrepancy.

Closing as the culprit is unrelated to this lib.

[bretambrose]

Interesting and good find. I've seen issues with lambda captures leading to similar behavior, but didn't realize default parameter assignment could give similar unexpected results.