v.1.8.0-beta.2

[Gsantomaggio]

Enhancements

• Improve the rawProducer and rawSuperStreamProducer status by @Gsantomaggio in Improve the rawProducer and rawSuperStreamProducer status #337
• Handle cancellation token during the consumer close by @Gsantomaggio in Handle cancellation token during the consumer close #339
• Update license to Broadcom by @Gsantomaggio in Update license to Broadcom #341
• Improve Reconnection by @Gsantomaggio in Improve Reconnection #343
• Improve the super stream reconnection by @Gsantomaggio in Improve the super stream reconnection #344
• Increase the backoff strategy time by @Gsantomaggio in Increase the backoff strategy time #345

Full Changelog: v1.8.0-beta.1...v1.8.0-beta.2

What's new in 1.8

The 1.8 focus are:

• Multiple Consumers and Producers per connection
• Improve the reconnection for stream and super stream.

The high-level classes Consumer and Producer don't introduce breaking changes.
The RawSuperStream* classes change the default behaviour. Please take a look at the section 3.

1. Multiple Consumers and Producers per connection

The RabbitMQ stream protocol supports multi-producers and multi-consumers per TCP Connection.
This version introduces the connection pool for Consumers and Producers.

There is a new ConnectionPoolConfig setting:

new StreamSystemConfig {
  ConnectionPoolConfig = new ConnectionPoolConfig()
 {
		ConsumersPerConnection = 10, 
		ProducersPerConnection = 10, 
 }
};

ConsumersPerConnection == The number of consumers per connection min 1 max 200 default is 1
ProducersPerConnection == The number of producers per connection min 1 max 200 default is 1

Each connection can handle different streams; see the image:

Performances

Sharing the same connection for multiple streams reduces the number of connections, but it could impact the performances:

• Consumer side. If one consumer is slow, it can also affect the other consumers
• Producer side: If all the producers are at full rate, it can reduce the performances

The proper parameter depends on your environment.

Tip

You can use different StreamSystemConfig like:

configToReduceTheConnections = new StreamSystemConfig{
 ConnectionPoolConfig = new ConnectionPoolConfig() {
    ConsumersPerConnection = 50, // high value 
    ProducersPerConnection = 50,  // high value
  }
}

configToIncreaseThePerformances = new StreamSystemConfig{
 ConnectionPoolConfig = new ConnectionPoolConfig() {
		ConsumersPerConnection = 1, // low value 
		ProducersPerConnection = 1,  // low value
  }
}

There are many combinations from 1 to 200.

2. Improve the reconnections

Handle streamNotAvailable, Add disconnection Info: #343

Improve the super stream reconnection: #344

Increase the backoff strategy time: #345

Please follow this document If you want to know more about what happens during a broker restart.

The focus is to improve the reconnection during the cluster restart.

3. Raw Super stream events

Removed the auto-reconnect. The RawSuperStreamProducer and RawSuperStreamConsumer classes now expose two events:

#344

**NOTE: If you are using these classes, the auto-reconnect is removed to be compliant with all the Raw* classes. **

You should use Consumer and Producer unless for a specific use case.

For Raw* users:

• Super Stream: during the disconnection, it is possible to understand the disconnection cause and reconnect the stream like:

var consumer = await system.CreateSuperStreamConsumer(configuration);
        var completed = new TaskCompletionSource<bool>();
        configuration.ConnectionClosedHandler = async (reason, stream) =>
        {
            if (reason == ConnectionClosedReason.Unexpected)
            {
                await consumer.ReconnectPartition(
                    await system.StreamInfo(stream).ConfigureAwait(false)
                );
                completed.SetResult(true);
            }
        };

The same is true for the standard consumer.

• Metadata update

 MetadataHandler = async update =>
 {
    await consumer.ReconnectPartition(
                    await system.StreamInfo(stream).ConfigureAwait(false));
                        
 }   

---

This discussion was created from the release v.1.8.0-beta.2.