Implement zone aware load balancing for XdsEndpointGroup

[jrhee17]

Motivation:

This changeset attempts to implement envoy's zone aware routing.
A small sample on how this can be configured can be seen here as well.

The basic algorithm in layman's terms is the following:

1. A client supplies a locality and a local cluster. The corresponding cluster should also be added.
2. The percentage of hosts for each locality is computed for both upstream and local clusters.
3. The percentage is compared and a new "weight" is computed to determine which locality to route to when selecting an endpoint.

The code has been mostly adapted from envoy's implementation:

• Computing the locality routing state
  • https://github.com/envoyproxy/envoy/blob/ae1811aaf8736e2a4ca9fc0d045f6696f58e45e2/source/extensions/load_balancing_policies/common/load_balancer_impl.cc#L450-L542
• Routing the locality for each request
  • https://github.com/envoyproxy/envoy/blob/ae1811aaf8736e2a4ca9fc0d045f6696f58e45e2/source/extensions/load_balancing_policies/common/load_balancer_impl.cc#L772-L816

Modifications:

• (Breaking) Previously we had been accepting a ListenerRoot when creating an XdsEndpointGroup. However, we need more information like local locality from the bootstrap. To make this intention clear, XdsEndpointGroup now accepts an XdsBootstrap. Additionally, XdsBootstrap#bootstrap has been added to access the local cluster name and locality.
• ClusterManager now maintains a LocalCluster data structure which watches changes in the local cluster if exists. When a ClusterEntry is created, the existing LocalCluster is passed.
• The upstream ClusterEntry needs to listen to the local ClusterEntry for updates in order to pre-compute the state. ClusterEntry is now an AbstractListenable and notifies the current LoadBalancer.
  • Following this change, the PrioritySet is required to compute the locality state. Added a LoadBalancer#prioritySet method to retrieve the current PrioritySet.
  • XdsLoadBalancer has been added as an extra abstraction layer to avoid conflicts with Add LoadBalancer for generalizing EndpointSelector #5779
  • ClusterEntry now has two update points: 1) when the endpoints are updated 2) when the local cluster is updated. Now, the two entry points call a single tryRefresh to refresh the load balancer.
• Added LocalityRoutingStateFactory and modified DefaultLoadBalancer to support zone aware routing. The implementation should look very similar to envoy's source.
• Testing was difficult due to multiple locations where random was used. To make the tests deterministic, I've added a XdsRandom interface for easier testing. Additionally, to guarantee that the local cluster was fully loaded before proceeding with tests, ClusterEntry#initialLocalEntryStateFuture has been added.
  • In order to access XdsEndpointGroup#clusterEntries, the factory methods for XdsEndpointGroup now return the concrete type.

Result:

• XdsEndpointGroup now supports zone-aware load balancing.

[github-actions]

🔍 Build Scan® (commit: 9255344)

Job name	Status	Build Scan®
build-self-hosted-unsafe-jdk-8	✅	https://ge.armeria.dev/s/xbxdlai23m4fw
build-self-hosted-unsafe-jdk-21-snapshot-blockhound	✅	https://ge.armeria.dev/s/sewckikrak6iw
build-self-hosted-unsafe-jdk-17-min-java-17-coverage	✅	https://ge.armeria.dev/s/evyxr2tnpy4g2
build-self-hosted-unsafe-jdk-17-min-java-11	✅	https://ge.armeria.dev/s/sgtxgeeq5zfio
build-self-hosted-unsafe-jdk-17-leak	❌ (failure)	https://ge.armeria.dev/s/gyyrwegeo546o
build-self-hosted-unsafe-jdk-11	✅	https://ge.armeria.dev/s/lc2kzbmbx2eb2
build-macos-12-jdk-21	✅	https://ge.armeria.dev/s/conoht7ichny2

[ikhoon]

Nice work! 👍

In fact, I approve this PR without reviewing it in detail so that you can continue working on your tasks. I will separately check and compare the Envoy implementation and ours.

[minwoox]

Looks very promising. 👍

[minwoox]

So this localLocality will never be chosen. Is that correct?

[jrhee17]

localLocality could be chosen in the extreme case if all ResidualCapacity have 0

armeria/xds/src/main/java/com/linecorp/armeria/xds/client/endpoint/LocalityRoutingStateFactory.java

Lines 174 to 177 in 885f7f8

	// This is *extremely* unlikely but possible due to rounding errors when calculating
	// locality percentages. In this case just select random locality.
	final int idx = random.nextInt(residualCapacities.size(), RandomHint.ALL_RESIDUAL_ZERO);
	return residualCapacities.get(idx).locality;

Having said this, the fact that this code is called implies that upstream already has healthy/degraded localLocality hosts so I guess this won't be a big deal.

[minwoox]

I missed the code. Thanks for the explanation. 👍

[minwoox]

Thanks a lot for adding this excellent feature.
You're the MVP. 👍 👍 👍

[minwoox]

nit:

Suggested change

	closeFuturesBuilder.addAll(clusterEntries.clusterEntriesMap
	closeFuturesBuilder.addAll(clusterEntriesMap()