Ocelot Load Balance using Web Socket Issue

[rycho27]

Hi Ocelot Team,

Please give advice for my current issue when using Load Balance configuration on Ocelot Gateway. My websocket implementation is for Microsof SignalR on my web client

## Expected Behavior / New Feature
When one of server is down, not causing error and able redirect request to connect websocket to another server

Actual Behavior / Motivation for New Feature

1 Backend Server down causing Bad Gateway request and unable to a connect websocket to another backend server

Steps to Reproduce the Problem

1. Server A installed with 1 Backend service (using .net core 3.1) and Gateway (Ocelot) service
2. Server B installed with 1 Backend service
3. When All service running, client able to connect websocket using gateway
4. When Backend service is turned off on Server A and backend in server B is running, client unable to connect websocket via gateway
5. When Backend service is turned off on Server B and backend in server A is running, client unable to connect websocket via gateway
6. Http request is running fine.

Specifications

• Version: Ocelot 16.0.1
• Platform: .net Core 3.1
• Subsystem:
• Please see my config on my ocelot.json

 {
    "Routes": [
        {
            "DownstreamPathTemplate": "/OPUIAPI/api/{everything}",
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8001
                },
                {
                    "Host": "bthtsa1030.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/opui/api/{everything}",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        },
        {
            "DownstreamPathTemplate": "/eqstatoui/api/{everything}",
            "DownstreamScheme": "http",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/eqstat/api/{everything}",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        },
        {
            "DownstreamPathTemplate": "/eqstatoui/signalr",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/eqstat/signalr",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        },
        {
            "DownstreamPathTemplate": "/eqstatoui/signalr/{catchAll}",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/eqstat/signalr/{catchAll}",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        },
        {
            "DownstreamPathTemplate": "/OPUIAPI/signalr",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8001
                },
                {
                    "Host": "bthtsa1030.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/opui/signalr",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        },
        {
            "DownstreamPathTemplate": "/OPUIAPI/signalr/{catchAll}",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8001
                },
                {
                    "Host": "bthtsa1030.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/opui/signalr/{catchAll}",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        }
    ],
    "GlobalConfiguration": {
        "BaseUrl": "https://localhost:5011"
    }
}

[nhochamvui]

so sad bro, i'm facing this issue too

[raman-m]

@ggnaegi What do you think?
The author wrote strange scenarios... He switched off downstream service and then he wanted to connect successfully. 😂

[raman-m]

Hi @rycho27 !
Most of routes are correct, but some routes are incorrectly defined.
You cannot load balance if 1 service, aka DownstreamHostAndPorts object!
You CAN load balance if 2+ services, or service discovery is enabled.

[ggnaegi]

{
            "DownstreamPathTemplate": "/OPUIAPI/signalr",
            "DownstreamScheme": "ws",
            "DownstreamHostAndPorts": [
                {
                    "Host": "bthsa1163.infineon.com",
                    "Port": 8001
                },
                {
                    "Host": "bthtsa1030.infineon.com",
                    "Port": 8000
                }
            ],
            "UpstreamPathTemplate": "/gateway/opui/signalr",
            "UpstreamHttpMethod": [
                "Get",
                "Put",
                "Post",
                "Delete",
                "Options"
            ],
            "AuthenticationOptions": null,
            "FileCacheOptions": {
                "TtlSeconds": 0
            },
            "LoadBalancerOptions": {
                "Type": "LeastConnection"
            }
        }

Yes, I would expect the same behavior too, if one of the two servers is down, then the requests should be sent to the available service... We should check that.

[raman-m]

@ggnaegi OK I've got it. Wonder if it was not implemented in the past... I guess also that the problem is not related to ws scheme, the issue can be reproduced with http scheme too.
I see Load Balancer limitation here when one service instance in the list is down, then Ocelot is not smart enough to find the next online service instance. Yeap, we have to filter offline service instances making requests to online ones, so making "redirects".

I see this is by design issue. And because Ocelot has no servicing Health Status feature we get this problem when Load Balancer service instances are not healthy. It will be great if we will think in terms of Microservices architecture which recommends to make quick short regular Health Check requests to all registered domain services. So, we need to design very simple Health feature... That will be great feature of Ocelot respecting Microservices philosophy.

[raman-m]

Duplicate of #1041

[raman-m]

Delivery after upgrade to NET 9 framework ❗

There is no team capacity to develop the feature in 2024... Asking community help...