Error connecting to Sentinel

[mohammed-essa]

Hi all

I am trying to connect to a redis sentinel topology but as of now I have tried StackExchangeDistributedCache and now I am trying the IRedisCacheClient but I am still having the same issue. The redis sentinel is up and I can connect to it from Redis-commander as well but the error I get is:

StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). UnableToConnect on redis-sentinel:26379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.2.4.27433

The code I am using to connect to the client is:

private RedisConfiguration LoadSentinel()
{
return new RedisConfiguration()
{
AbortOnConnectFail = true,
KeyPrefix = "my_key_prefix",
Hosts = new RedisHost[]
{
new RedisHost(){Host = "redis-sentinel", Port = 26379}
},
ServiceName = "redis-sentinel", // In case you are using Sentinel
AllowAdmin = true,
ConnectTimeout = 3000,
Database = 0,
Ssl = true,
Password = "str0ng_passw0rd",
ServerEnumerationStrategy = new ServerEnumerationStrategy()
{
Mode = ServerEnumerationStrategy.ModeOptions.All,
TargetRole = ServerEnumerationStrategy.TargetRoleOptions.Any,
UnreachableServerAction = ServerEnumerationStrategy.UnreachableServerActionOptions.Throw
},
MaxValueLength = 1024,
PoolSize = 5
};
}

I copied the code StackExchangeRedisExtensions

services.AddStackExchangeRedisExtensions<SystemTextJsonSerializer>((options) => { return appConfig.RedisOptions; });

Can anyone offer any advice on this as I have tried almost everything to get this to work but

[imperugo]

Hi @mohammed-essa I think your problem is not related to this library (exactly like is not related to StackExchangeDistributedCache I think).

The connection is something that is into the main library and here you just send the connection string (or parameters), there is nothing special into the connection with this library

[mohammed-essa]

possibly, agreed, my main issue is that I am looking for a working sample reference using the redis-sentinel topology anywhere with no luck.

[Namoshek]

@imperugo I can confirm that the issue does not come from this library directly, but is triggered by it.

Today I was in the exact same situation as @mohammed-essa and after looking carefully through my logs, I noticed something odd. But first, here is the log - try to find the issue yourself, it is quite obvious to be honest:

[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Some unrelated task which accesses Redis the first time after applciation boot has been started [Foo.Bar.Queries.SomeQueryHandler]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating the redis connection pool with 5 connections. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating the redis connection pool with 5 connections. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating the redis connection pool with 5 connections. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating the redis connection pool with 5 connections. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating the redis connection pool with 5 connections. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:17 DBG] [0HM86A46G8KCG:00000001] Creating new Redis connection. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:18 DBG] [0HM86A46G8KCG:00000001] Creating new Redis connection. [StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager]
[2021-04-23 14:13:19 ERR] [0HM86A46G8KCG:00000001] Some business logic failed [Foo.Bar.Buz]
Foo.Bar.Exceptions.ServiceException: Some business logic failed.
 ---> StackExchange.Redis.RedisConnectionException: Sentinel: The ConnectionMultiplexer is not a Sentinel connection.
   at StackExchange.Redis.ConnectionMultiplexer.GetSentinelMasterConnection(ConfigurationOptions config, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 2358
   at StackExchange.Redis.ConnectionMultiplexer.SentinelMasterConnect(ConfigurationOptions configuration, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1100
   at StackExchange.Redis.ConnectionMultiplexer.Connect(ConfigurationOptions configuration, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1032
   at StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager.<EmitConnection>b__7_0()
   at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
   at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
   at System.Lazy`1.CreateValue()
   at StackExchange.Redis.Extensions.Core.Implementations.RedisCacheConnectionPoolManager.GetConnection()
   at StackExchange.Redis.Extensions.Core.Implementations.RedisDatabase.get_Database()
   at StackExchange.Redis.Extensions.Core.Implementations.RedisDatabase.HashGetAsync[T](String hashKey, String key, CommandFlags commandFlags)

In case it isn't as obvious as I think (now after I found it 😄):
Creating the second Redis connection fails. And it is fixable by setting RedisConfiguration.PoolSize to 1. So my best guess is that ConnectionMultiplexer.Connect() changes the internal state of the multiplexer and it therefore doesn't like being called twice with the same Sentinel connection string.

My suggestion would therefore be to add this to the documentation of this library. Not sure if we call it a solution or a workaround, but as far as I understood, the multiplexer already performs load balancing (since Sentinel triggers a managed connection) and a connection pool is not necessary anyway. But I might be wrong.

[guillaumer1313]

Hello,
I think I have the same problem, if i use Sentinel, when creating the next connections (after the PoolSize) I get a "The ConnectionMultiplexer is not a Sentinel connection" error.
I saw that the variable with the global configuration is modified after the 1st connection by replacing the sentinel hosts by those of Redis, so when creating the new connection (EmitConnection) we no longer have the Sentinel hosts but only the redis endpoints.

[GiacomoBruno]

Hi,
I run into this problem myself and found out some possible solutions to this.
The issue seems to stem from having a specified port on the hosts whilst connecting to a redis sentinel and this is triggered on the call to ConnectionMultiplexer.Connect after the first connection for the pool has been made.

As it can be seen, after the call to ConnectionMultiplexer.Connect the hosts of the given configuration are populated with the redis sentinel instances' ips and ports.

However if instead of "ConnectionMultiplexer.Connect", "ConnectionMultiplexer.SentinelConnect" is used when the ServiceName property is set then the modified configuration seems to not generate any error and connect to the redis sentinel succesfully.

StackExchange.Redis.Extensions/src/core/StackExchange.Redis.Extensions.Core/Implementations/RedisConnectionPoolManager.cs

Line 128 in 451f93c

var multiplexer = ConnectionMultiplexer.Connect(redisConfiguration.ConfigurationOptions);