add proposal for Locality LoadBalance #574

derekwin · 2024-07-15T06:53:14Z

What type of PR is this?
/kind enhancement

What this PR does / why we need it:
add proposal for Locality LB

kmesh-bot · 2024-07-15T06:53:25Z

Welcome @derekwin! It looks like this is your first PR to kmesh-net/kmesh 🎉

LiZhenCheng9527 · 2024-07-15T07:47:20Z

Would you like to share your issue at Thursday's community meeting?

LiZhenCheng9527 · 2024-07-15T07:56:05Z

docs/proposal/kmesh_support_localityLB.md

+
+### Motivation
+
+Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.


Suggested change

Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

Currently, Kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

Unified capitalisation of initial letters in Kmesh

LiZhenCheng9527 · 2024-07-15T07:58:11Z

docs/proposal/kmesh_support_localityLB.md

+
+#### case 1. locality failover
+1. Destination Rule
+Same as Istion. Parse rules specify configuration for Locality load balancing. (todo: outlier detection settings to detect and evict unhealthy hosts from the load balancing pool.)


what is istion? Istio?

hzxuzhonghu · 2024-07-16T01:40:50Z

/ok-to-test

codecov · 2024-07-16T01:53:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.80%. Comparing base (433592b) to head (b2caa7c).
Report is 217 commits behind head on main.

see 29 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dda7049...b2caa7c. Read the comment docs.

hzxuzhonghu · 2024-07-16T02:35:49Z

docs/proposal/kmesh_support_localityLB.md

+
+Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.
+
+#### Goals


Suggested change

#### Goals

### Goals

hzxuzhonghu · 2024-07-16T02:37:16Z

docs/proposal/kmesh_support_localityLB.md

+
+1. prioritize add locality load balancing capabilities in the workload mode.
+
+2. two types of locality load balancing : locality failover, locality weighted distribution.


I am not sure how locality weighted distribution can be implemented in workload mode. The workload api does not support weight actually

hzxuzhonghu · 2024-07-16T02:41:49Z

docs/proposal/kmesh_support_localityLB.md

+#### case 1. locality failover
+1. Destination Rule
+Same as Istion. Parse rules specify configuration for Locality load balancing. (todo: outlier detection settings to detect and evict unhealthy hosts from the load balancing pool.)
+    - Outlier detection should occur before load balancing.


This does not suite worklaod mode as workload api does not include outlier setting. It do LB based on where the endpoint resides.

derekwin · 2024-07-16T03:45:39Z

Would you like to share your issue at Thursday's community meeting?

yes

derekwin · 2024-07-25T05:53:55Z

I have updated the proposal.

derekwin · 2024-07-26T11:58:02Z

Propose a new implementation for a location matching algorithm that avoids circular computations while also reducing the amount of data needed to be stored in BPF maps. detail: https://github.com/derekwin/treemap/tree/master
Welcome to offer suggestions to further improve the approach.

Okabe-Rintarou-0 · 2024-07-30T09:24:02Z

if no conflict, there is no need to merge main branch.
If there are some conflicts, to get a clearer commit history, you should:

git rebase main

then fix some conflicts, and then

git rebase --continue
git push --force

the DCO github action failed, because it asks you to commit with your signature, which can be attached with -s flag:

git commit -s -m 'something to say'

hzxuzhonghu

I wish to see more api design instead of function implement in the proposal

How do you express the priority level, and how do you match the client locality with the endpoints

hzxuzhonghu · 2024-08-02T09:03:00Z

docs/proposal/kmesh_support_localityLB.md

+
+1. prioritize add locality load balancing capabilities in the workload mode.
+
+2. locality load balancing mode: locality failover.


how about strict mode

hzxuzhonghu · 2024-08-02T09:06:06Z

docs/proposal/kmesh_support_localityLB.md

+```
+https://pkg.go.dev/istio.io/istio/pkg/workloadapi#LoadBalancing_Scope
+
+2. calculate locality match rank


group endpoints with prority

hzxuzhonghu · 2024-08-02T09:07:20Z

docs/proposal/kmesh_support_localityLB.md

+
+3. choose endpoint
+
+Randomly select one endpoint from the group with the highest rank as the service backend.


Suggested change

Randomly select one endpoint from the group with the highest rank as the service backend.

Randomly select one endpoint from the group with the highest priority

And add more comments what we do if all the endpoints of high priority is unhealthy

And for the strict mode, how would you select the endpoint, i would like to see that

hzxuzhonghu · 2024-08-02T09:08:43Z

docs/proposal/kmesh_support_localityLB.md

+
+4. maybe more？ Panic threshold
+
+When the proportion of healthy endpoints in the high-rank group falls below the panic threshold, select endpoints from the next rank group.


I donot care about this at first. First respect workload healthy status

hzxuzhonghu · 2024-08-02T09:09:34Z

docs/proposal/kmesh_support_localityLB.md

+    __u32 waypoint_addr;
+    __u32 waypoint_port;
+    // 增加健康状态 healthStatus
+    // 增加locality信息


Please supplement what does this field look like.

derekwin · 2024-11-23T08:36:50Z

Yeah, we need user guide for locality loadbalancer, you should file it into website repo. BTW, this is a refer for you https://istio.io/latest/docs/tasks/traffic-management/locality-load-balancing/是的，我们需要 Locality LoadBalancer 的用户指南，您应该将其归档到 Website Repo 中。顺便说一句，这是为您推荐的 https://istio.io/latest/docs/tasks/traffic-management/locality-load-balancing/

ok.

derekwin · 2024-11-26T03:19:03Z

I have updated the content of this proposal, and the documentation in the website repo will be updated soon.

derekwin · 2024-11-26T03:19:39Z

@hzxuzhonghu

LiZhenCheng9527

almost lgtm

docs/proposal/kmesh_support_localityLB.md

LiZhenCheng9527 · 2024-11-27T02:57:08Z

docs/proposal/kmesh_support_localityLB.md

+7. For the random policy, all endpoints are marked with a priority of 0. For failover or strict policy, the priority is set to 0 for the endpoint with the highest match according to the `routingPreference`.
+
+#### control flow
+![locality_lb_pic](pics/locality_lb.svg)


Please center the image

LiZhenCheng9527 · 2024-11-27T08:27:50Z

/lgtm

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

Signed-off-by: derekwin <[email protected]>

Co-authored-by: lizhencheng <[email protected]> Signed-off-by: derekwin <[email protected]>

Signed-off-by: derekwin <[email protected]>

LiZhenCheng9527 · 2024-11-27T08:34:25Z

/lgtm

LiZhenCheng9527 · 2024-12-12T11:40:05Z

@hzxuzhonghu Can this PR be merged?

hzxuzhonghu

/lgtm

kmesh-bot · 2024-12-13T06:52:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hzxuzhonghu]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kmesh-bot added the kind/enhancement New feature or request label Jul 15, 2024

kmesh-bot requested review from bitcoffeeiux and kevin-wangzefeng July 15, 2024 06:53

kmesh-bot added size/M do-not-merge/contains-merge-commits labels Jul 15, 2024

LiZhenCheng9527 reviewed Jul 15, 2024

View reviewed changes

kmesh-bot added the ok-to-test label Jul 16, 2024

hzxuzhonghu reviewed Jul 16, 2024

View reviewed changes

kmesh-bot added size/L and removed size/M labels Jul 25, 2024

hzxuzhonghu reviewed Aug 2, 2024

View reviewed changes

derekwin force-pushed the main branch from 48a0b47 to a19444b Compare August 8, 2024 12:41

kmesh-bot removed the do-not-merge/contains-merge-commits label Aug 8, 2024

derekwin deleted the branch kmesh-net:main August 8, 2024 12:47

derekwin closed this Aug 8, 2024

derekwin deleted the main branch August 8, 2024 12:47

derekwin restored the main branch August 8, 2024 12:49

derekwin reopened this Aug 8, 2024

derekwin force-pushed the main branch from a19444b to b6583c6 Compare August 10, 2024 10:57

derekwin requested review from hzxuzhonghu and LiZhenCheng9527 August 10, 2024 23:13

derekwin dismissed hzxuzhonghu’s stale review via e663622 November 26, 2024 03:11

kmesh-bot removed the approved label Nov 26, 2024

derekwin force-pushed the main branch from df2aa74 to 5a1de9b Compare November 26, 2024 03:16

derekwin mentioned this pull request Nov 26, 2024

Add documents on kmesh locality failover features #1061

Closed

LiZhenCheng9527 reviewed Nov 27, 2024

View reviewed changes

kmesh-bot assigned LiZhenCheng9527 Nov 27, 2024

kmesh-bot added the lgtm label Nov 27, 2024

derekwin and others added 9 commits November 27, 2024 16:29

add proposal for Locality LB

5299c6e

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

Revise the proposal based on the recommendations.

870094a

Signed-off-by: seclee <[email protected]> Signed-off-by: derekwin <[email protected]>

locality LB logic in user-space

804747b

Signed-off-by: derekwin <[email protected]>

new proposal for locality loadbalance

19b3358

Signed-off-by: derekwin <[email protected]>

change prio from 0 to 6

5b600f6

Signed-off-by: derekwin <[email protected]>

update locality LB proposal

fa21d9a

Signed-off-by: derekwin <[email protected]>

ent

39b1677

Signed-off-by: derekwin <[email protected]>

Update docs/proposal/kmesh_support_localityLB.md

76be599

Co-authored-by: lizhencheng <[email protected]> Signed-off-by: derekwin <[email protected]>

center the img

c3d1af2

Signed-off-by: derekwin <[email protected]>

derekwin force-pushed the main branch from 0d9df36 to c3d1af2 Compare November 27, 2024 08:30

kmesh-bot removed the lgtm label Nov 27, 2024

kmesh-bot added the lgtm label Nov 27, 2024

hzxuzhonghu approved these changes Dec 13, 2024

View reviewed changes

kmesh-bot assigned hzxuzhonghu Dec 13, 2024

kmesh-bot added the approved label Dec 13, 2024

kmesh-bot merged commit 1727d3c into kmesh-net:main Dec 13, 2024
3 checks passed


		### Motivation

		Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.


		Currently, kmesh does not support locality topology-aware load balancing. Locality Load Balancing optimizes performance and reliability in distributed systems by directing traffic to the nearest service instances. This reduces latency, enhances availability, and lowers costs associated with cross-region data transfers. It also helps ensure compliance with data sovereignty regulations and improves overall user experience by providing faster and more reliable service responses.

		#### Goals


		1. prioritize add locality load balancing capabilities in the workload mode.

		2. two types of locality load balancing : locality failover, locality weighted distribution.


		1. prioritize add locality load balancing capabilities in the workload mode.

		2. locality load balancing mode: locality failover.


		3. choose endpoint

		Randomly select one endpoint from the group with the highest rank as the service backend.


		4. maybe more？ Panic threshold

		When the proportion of healthy endpoints in the high-rank group falls below the panic threshold, select endpoints from the next rank group.

add proposal for Locality LoadBalance #574

add proposal for Locality LoadBalance #574

Uh oh!

Conversation

derekwin commented Jul 15, 2024

Uh oh!

kmesh-bot commented Jul 15, 2024

Uh oh!

LiZhenCheng9527 commented Jul 15, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hzxuzhonghu commented Jul 16, 2024

Uh oh!

codecov bot commented Jul 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

derekwin commented Jul 16, 2024

Uh oh!

derekwin commented Jul 25, 2024

Uh oh!

derekwin commented Jul 26, 2024

Uh oh!

Okabe-Rintarou-0 commented Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hzxuzhonghu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

derekwin commented Nov 23, 2024

Uh oh!

derekwin commented Nov 26, 2024

Uh oh!

derekwin commented Nov 26, 2024

Uh oh!

LiZhenCheng9527 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LiZhenCheng9527 commented Nov 27, 2024

Uh oh!

LiZhenCheng9527 commented Nov 27, 2024

Uh oh!

LiZhenCheng9527 commented Dec 12, 2024

Uh oh!

hzxuzhonghu left a comment

Choose a reason for hiding this comment

Uh oh!

kmesh-bot commented Dec 13, 2024

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jul 16, 2024 •

edited

Loading

Okabe-Rintarou-0 commented Jul 30, 2024 •

edited

Loading