Does Ocelot support sharing routes and rate limit configurations between its instances if we deploy more than one Ocelot instance?

[MMoussaSURE]

Expected Behavior / New Feature

I want to publish two instance of Ocelot at different servers but the under the same domain
and if the user called my domain /{upstreamRoute} one of Ocelot instances should achieve the request but also the two instances should achieve the rate limit of any one of them called, for example:

  "Routes": [
          {
            "DownstreamPathTemplate": "/product",
            "DownstreamScheme": "https",
             "DownstreamHostAndPorts": [
               {
                 "Host": "products.com",
                 "Port": 44322
               }
             ],
             "UpstreamPathTemplate": "/api/product",
             "UpstreamHttpMethod": [ "POST" ],
             "RouteIsCaseSensitive": true,
             "RateLimitOptions": {
                      "ClientWhitelist": [],
                       "EnableRateLimiting": true,
                       "Period": "60s",
                       "PeriodTimespan": 60,
                       "Limit": 1
                     } 
               }
]
"GlobalConfiguration": {
  "BaseUrl": "https://api.products",
  "RateLimitOptions": {
    "DisableRateLimitHeaders": false,
    "QuotaExceededMessage": "Too Many Requests At Short Duration!",
    "HttpStatusCode": 429
  }
}

What should happen is

• if there is more than one request reach my Ocelot domain during current 60s
• one of ocelot instance should achieve the request and also at the same time the two ocelot instances should know that the remaining allowed request is 0

[raman-m]

The short answer is NO.

Each Ocelot instance counts requests independently for rate limiting due to its internal cache storage. However, if you configure your instance to use a distributed cache, then the rate limiting counters will store values in this distributed cache storage.

Therefore, you should use a distributed cache, as Ocelot does not have the capability to allow one instance to share its internal cache with other instances. Nonetheless, this could be an interesting feature to explore further.

[raman-m]

The second design pertains to the storage of Rate Limiting

The Rate Limiting feature is incorporated into DI via the AddRateLimiting method.
The objective is to substitute the following interface object:

Ocelot/src/Ocelot/DependencyInjection/Features.cs

Line 22 in 8c0180a

.AddSingleton<IRateLimitStorage, MemoryCacheRateLimitStorage>();

Rate limiting storage relies heavily on the IRateLimitStorage interface for saving and reading rate limiting counters to and from persistent storage, which in practice is a memory cache: MemoryCacheRateLimitStorage which inherits from the Microsoft IMemoryCache interface.

There is another undocumented class, DistributedCacheRateLimitStorage:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Line 12 in 8c0180a

public class DistributedCacheRateLimitStorage : IRateLimitStorage

This class incorporates the Microsoft.Extensions.Caching.Distributed.IDistributedCache object as seen in the provided code segment from the Ocelot library:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Lines 12 to 16 in 8c0180a

	public class DistributedCacheRateLimitStorage : IRateLimitStorage
	{
	private readonly IDistributedCache _memoryCache;
	public DistributedCacheRateLimitStorage(IDistributedCache memoryCache) => _memoryCache = memoryCache;

Our objectives include:

• Integrating a native IDistributedCache object through native storage driver helpers, see MS docs
• Substituting the IRateLimitStorage object with an actual DistributedCacheRateLimitStorage object

The pseudocode could be structured as follows:

    var builder = WebApplication.CreateBuilder(args);
    builder.Configuration.AddJsonFile("ocelot.json");
    builder.Services.AddOcelot()
        .Services.RemoveAll<Ocelot.RateLimiting.IRateLimitStorage>()
        .AddSingleton(typeof(Ocelot.RateLimiting.IRateLimitStorage), typeof(Ocelot.RateLimiting.DistributedCacheRateLimitStorage));
        // Add Microsoft.Extensions.Caching.Distributed.IDistributedCache services

Please be aware that incorporating the Microsoft.Extensions.Caching.Distributed.IDistributedCache object falls outside the scope of Ocelot's responsibilities.
As a developer, you should consult the excellent official Microsoft documentation:

• Distributed caching in ASP.NET Core | Microsoft Learn

This superb guide will assist you in configuring any distributed persistent storage solutions, including Redis, SQL Server, Azure CosmosDB, and others.

Hope it helps!

[MMoussaSURE]

For how many years have you been developing software using the .NET framework? For how many years have you been using Ocelot as a gateway? What's your full name? And what's your LinkedIn?

i'm mohamed ragab
net developer with 6 years of experience
and that is first time to use ocelot
(i write you from my second github account)
the main one is https://github.com/MohamedRagab10793-design

[MMoussaSURE]

🆗 Let's research the Ocelot code together...

We have this Ocelot feature which is a part of Ocelot kernel: AddOcelotCache So, we have these services in DI-container:

Ocelot/src/Ocelot/DependencyInjection/Features.cs

Lines 32 to 36 in 8c0180a

	public static IServiceCollection AddOcelotCache(this IServiceCollection services) => services
	.AddSingleton<IOcelotCache<FileConfiguration>, DefaultMemoryCache<FileConfiguration>>()
	.AddSingleton<IOcelotCache<CachedResponse>, DefaultMemoryCache<CachedResponse>>()
	.AddSingleton<ICacheKeyGenerator, DefaultCacheKeyGenerator>()
	.AddSingleton<ICacheOptionsCreator, CacheOptionsCreator>();

The 3rd and 4th services should not be changed at all!
Our goal to replace 1st and the seconds services of the IOcelotCache<T> type.
There is default service implementation DefaultMemoryCache<T> which is added to DI. This is standard In-Memory cache:

Ocelot/src/Ocelot/Cache/DefaultMemoryCache.cs

Line 5 in 8c0180a

public class DefaultMemoryCache<T> : IOcelotCache<T>

This class implements target IOcelotCache<T> interface which is added to DI 👇 Also pay attention to IMemoryCache which is Microsoft interface:

Ocelot/src/Ocelot/Cache/DefaultMemoryCache.cs

Lines 5 to 14 in 8c0180a

	public class DefaultMemoryCache<T> : IOcelotCache<T>
	{
	private readonly IMemoryCache _memoryCache;
	private readonly Dictionary<string, List<string>> _regions;
	public DefaultMemoryCache(IMemoryCache memoryCache)
	{
	_memoryCache = memoryCache;
	_regions = new Dictionary<string, List<string>>();
	}

If you possess sufficient experience with DI and MS .NET, you might consider two designs:

• Replacing the .NET framework IMemoryCache object
• Replacing the Ocelot IOcelotCache<T> object

Additionally, it's important to note that replacing either of these interfaces in DI will result in Ocelot utilizing your custom caching storage❗

thank you for your clearness
this may be useful if i want to store all configurations at distributed cache like (redis) instead of standard cache (memory cache)

[MMoussaSURE]

The second design pertains to the storage of Rate Limiting

The Rate Limiting feature is incorporated into DI via the AddRateLimiting method. The objective is to substitute the following interface object:

Ocelot/src/Ocelot/DependencyInjection/Features.cs

Line 22 in 8c0180a

.AddSingleton<IRateLimitStorage, MemoryCacheRateLimitStorage>();

Rate limiting storage relies heavily on the IRateLimitStorage interface for saving and reading rate limiting counters to and from persistent storage, which in practice is a memory cache: MemoryCacheRateLimitStorage which inherits from the Microsoft IMemoryCache interface.

There is another undocumented class, DistributedCacheRateLimitStorage:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Line 12 in 8c0180a

public class DistributedCacheRateLimitStorage : IRateLimitStorage

This class incorporates the Microsoft.Extensions.Caching.Distributed.IDistributedCache object as seen in the provided code segment from the Ocelot library:

Ocelot/src/Ocelot/RateLimiting/DistributedCacheRateLimitStorage.cs

Lines 12 to 16 in 8c0180a

	public class DistributedCacheRateLimitStorage : IRateLimitStorage
	{
	private readonly IDistributedCache _memoryCache;
	public DistributedCacheRateLimitStorage(IDistributedCache memoryCache) => _memoryCache = memoryCache;

Our objectives include:

• Integrating a native IDistributedCache object through native storage driver helpers, see MS docs
• Substituting the IRateLimitStorage object with an actual DistributedCacheRateLimitStorage object

The pseudocode could be structured as follows:

    var builder = WebApplication.CreateBuilder(args);
    builder.Configuration.AddJsonFile("ocelot.json");
    builder.Services.AddOcelot()
        .Services.RemoveAll<Ocelot.RateLimiting.IRateLimitStorage>()
        .AddSingleton(typeof(Ocelot.RateLimiting.IRateLimitStorage), typeof(Ocelot.RateLimiting.DistributedCacheRateLimitStorage));
        // Add Microsoft.Extensions.Caching.Distributed.IDistributedCache services

Please be aware that incorporating the Microsoft.Extensions.Caching.Distributed.IDistributedCache object falls outside the scope of Ocelot's responsibilities. As a developer, you should consult the excellent official Microsoft documentation:

• Distributed caching in ASP.NET Core | Microsoft Learn

This superb guide will assist you in configuring any distributed persistent storage solutions, including Redis, SQL Server, Azure CosmosDB, and others.

Hope it helps!

this what i searching for
store rate limit values at distributed cache (redis) to make all ocelot hosted instances have the same rate limit values at the same time

i apply that after reading your solution throw the following

i just add Microsoft.Extensions.Caching.StackExchangeRedis package version 8.0.6
and update Ocelot to version 23.3.3

then configure my services like
context.Services.AddStackExchangeRedisCache(options =>
{
options.Configuration = configuration["Redis:ConnectionString"];
options.InstanceName = "BaseCacheKey:OcelotRateLimit:";
});

context.Services.AddOcelot(context.Services.GetConfiguration())
.AddAdministration("/ocelotAdministration", "secretKey")
.Services.RemoveAll()
.AddSingleton(typeof(IRateLimitStorage), typeof(DistributedCacheRateLimitStorage));

@raman-m is any thing at my the above configuration will Inconsistent with my needs

[raman-m]

Good luck! Once all development is complete, it would be beneficial to demonstrate a real, working solution. Perhaps we could prepare a sample project to publish in our repository.
Do you have anything else to add, Mohamed?