Concurrent use of `PacketIdManager`

[smaeul]

With the current design, allocating a packet ID requires a mutable reference to the Connection, so it is difficult to generate MQTT packets from multiple threads (or async contexts) and send them over a channel to the thread running the MQTT client event loop (especially if the Connection object is local to the MQTT client thread). Some solutions I can think of:

1. Pass my own message struct over the channel, and generate Publish/Subscribe/Unsubscribe packets on the MQTT client thread.
2. Pass the packet builder over the channel, and call builder.packet_id(foo).build() on the MQTT client thread.
3. Add an option for the MQTT client to allocate and overwrite the packet ID when serializing the packet inside checked_send.
4. Allow Connection to use an external PacketIdManager trait object, so I can wrap the PacketIdManager in a mutex and access it directly from the other threads.

Do you have any suggestions?

One reason I want to pass the pre-constructed packets over the channel, beyond having them pre-validated, is that they own the topic string/payload data, so they can be sent by value without additional boxing. But if I send my own message struct over the channel, at least the topic name will be copied again when constructing the MqttString. Would it make sense to change the builder::topic_name() methods to take an Into<MqttString> (like payloads take IntoPayload), so the channel sender can pre-create the MqttString?

[redboltz]

Thank you for the detailed feedback regarding multi-threaded usage scenarios. I'd like to address your concerns:

Why Solution 4 is Unrealistic

Solution 4 (exposing PacketIdManager externally) is not practical for the following reasons:

1. Tight Coupling with Internal Protocol Logic: PacketIdManager is deeply integrated with Connection's packet handling. The library automatically releases packet IDs in various situations:

   • When receiving PUBACK, PUBREC, SUBACK, or UNSUBACK packets (src/mqtt/connection/core.rs:758-771)
   • When send operations fail (src/mqtt/connection/core.rs:1528-1530)
   • When handling protocol errors and disconnections

2. State Synchronization Complexity: Exposing PacketIdManager externally would require complex synchronization between user code and internal Connection state, leading to potential race conditions and inconsistent state.

3. Breaking Encapsulation: The automatic packet ID lifecycle management is a core feature that simplifies the API. Externalizing it would shift significant complexity to users and increase the risk of misuse.

Recommended Solution: Option 1

I recommend Solution 1: Prepare message structures on worker threads, then pass them to the Connection thread for packet generation.

To support this approach efficiently, I've implemented (#54) the following enhancements:

Implementation: IntoMqttString and IntoMqttBinary Traits

All builder methods that previously accepted AsRef or AsRef<[u8]> now accept these new traits:

Updated Methods:

• v3.1.1 & v5.0 Connect: client_id(), user_name(), will_message() (topic & payload), password()
• v3.1.1 & v5.0 Publish: topic_name()
• SubEntry: new()
• Properties: ContentType, ResponseTopic, CorrelationData, AuthenticationData, UserProperty, etc.

How This Solves "Additional Boxing"

Before:
// Worker thread constructs a String
let topic = format!("device/{}/telemetry", device_id);

// Passing to Connection thread requires the builder to call MqttString::new()
// internally, creating an additional heap allocation
publish_builder.topic_name(topic) // String -> MqttString (allocation!)

After:
// Worker thread pre-constructs MqttString
let topic = MqttString::new(format!("device/{}/telemetry", device_id)).unwrap();

// Passing to Connection thread - NO additional allocation!
publish_builder.topic_name(topic) // MqttString -> MqttString (passthrough)

Key Benefits

✅ Zero-Copy Optimization: Pre-constructed MqttString/MqttBinary instances are passed through without additional heap allocations

✅ Backward Compatible: Existing code using &str or String continues to work unchanged

✅ Consistent API: All packets and properties use the same pattern

✅ Multi-Threading Friendly: Worker threads can construct validated MqttString/MqttBinary objects once, then pass them efficiently to the Connection thread

Example Usage

// Worker thread: prepare validated message components
let topic = MqttString::new("sensor/temperature").unwrap();
let payload = MqttBinary::new(b"23.5").unwrap();

// Send to Connection thread (e.g., via channel)
tx.send((topic, payload, qos, retain)).await;

// Connection thread: build packet without additional allocations
let publish = Publish::builder()
.topic_name(topic)      // No boxing!
.payload(payload)       // No boxing!
.qos(qos)
.retain(retain)
.build()?;

This approach provides the efficiency you need for multi-threaded scenarios while maintaining the library's design principles and automatic packet ID management.

In addtion, there is no breaking changes. Existing all tests are passed without any modification.

[redboltz]

Solved by #54.

@smaeul I’ll cut a new release from the current HEAD (+ CHANGELOG update). If you run into any other issues, please let me know.