Grove proposal/topology #224

Ronkahn21 · 2025-10-21T11:31:51Z

What type of PR is this?

/kind documentation
/kind feature

What this PR does / why we need it:

This PR adds a comprehensive design document for topology-aware scheduling in the Grove operator. The design introduces a flexible topology system that enables optimal placement of multinode inference
workloads based on cluster network topology.

Key components:

TopologyDomain CRD: Admin-configured cluster topology hierarchy mapping friendly names to node labels
Operator Configuration: Selects active topology via --topology-domain-name argument
TopologyConstraint API: User-specified packing requirements in workloads (PodCliqueSet, PodCliqueScalingGroup, PodClique)
Automatic Optimization: Out-of-box topology optimization via auto-generated preferred constraints
KAI Scheduler Integration: Automatic Kueue Topology generation and three-level constraint translation

The design addresses critical requirements for multinode inference workloads including network locality, coordinated placement, and latency optimization.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

This is a design document only - no implementation changes
Design includes detailed API specifications, validation rules, and scheduler contract
Addresses review feedback including RBAC, runtime behavior, and edge cases
Immutability enforced at multiple levels to ensure scheduling consistency
Explicit failure modes prevent silent degradation of topology features

Does this PR introduce a API change?

suggest change

…ator Signed-off-by: Ron Kahn <[email protected]>

Signed-off-by: Ron Kahn <[email protected]>

docs/designs/topology.md

romanbaron · 2025-10-21T17:14:09Z

docs/designs/topology.md

+**Motivation**: Topology-aware scheduling is critical for Grove's multinode inference workloads because these
+applications require:
+
+- **Network Locality**: High-bandwidth communication between prefill and decode workers benefits from proximity


I was under the impression we mainly aim to pack different roles of the same inference pipeline phase: workers and leader. At the moment, we were not requested to specifically pack prefill and decode workers.

its was left over from the support of group affinities, before we decided to postpone this required

romanbaron · 2025-10-21T17:16:50Z

docs/designs/topology.md

+       PodClique)
+
+**Key Feature**: Grove provides automatic out-of-box topology optimization by generating preferred packing constraints
+at all levels, even without user configuration. Users can optionally specify required constraints for strict placement


Shouldn't it say something like "at the global/workload level"?

we are adding preferred to each level, so it will as close it can be, and not only at workload level

fixed description

what is the out-of-box topology optimization that is generated by Grove especially when no pack configuration is defined by the user? Ideally if no pack configuration is specified then should it not mean that there is no strict requirement or in other words there is no packing constraints that is observed by the scheduler while taking scheduling decisions?

If no configuration is provided, and the user has no real constraints that they need to follow, how would they be able to opt out if Grove always tries for the optimal by default?

If by default Grove tries for the optimal placement, and the scheduler is unable to schedule pods with these constraints, then the workload will never fully be up even though the user has no requirement for it.

Perhaps we can provide a field in the API through which the user indicates that they want out-of-box topology optimization. Or, we can provide a way for user to opt out of these out-of-box optimizations if they do not really care?

typo fix Co-authored-by: Roman Baron <[email protected]> Signed-off-by: Ron Kahn <[email protected]>

romanbaron · 2025-10-21T17:55:37Z

docs/designs/topology.md

+**Out-of-Box Optimization:**
+
+- Operator automatically generates **preferred** constraints using strictest topology level (e.g., "host")
+    - Applied at all three levels (PodGang, NetworkPackGroup, PodGroup) during translation to scheduler API


Why not only at the PodGang level?

The intention is to make the scheduler prioritize keeping pods with a shared rule or relationship (like a 'leader' and its 'workers') together.

To illustrate: Imagine a host has capacity for one more pod. The scheduler has two choices:

An unrelated pod from the same workload.

A 'worker' pod that belongs to a service already running on that host.

With this change, the scheduler should strongly prefer option 2.

Typically prefill/decode worker/leader will be realized as a PCSG. Each replica of PCSG will be translated to a PodGang, so what you want @romanbaron will already happen for the case that @Ronkahn21 is describing above.

However this cannot be a generic rule, you would need flexibility to define pack constraints at every level.

Signed-off-by: Ron Kahn <[email protected]>

sanjaychatterjee

Thanks for the PR. Added some comments to simplify the implementation.

sanjaychatterjee · 2025-10-22T03:48:05Z

docs/designs/topology.md

+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels
+    2. **Operator Configuration**: Selects active topology via `--topology-domain-name` argument


Can we fix a default topology CR name for the controller, e.g. grove-topology?

we could but what will the value of this? the admin create the CR, He could choose any name he want,
if we were the one to create the crd I will agree with you

What would happen if the user creates multiple instances of this CustomResource? How will Grove decide which to pick?

During runtime, there is only one CR that Grove must use for specifying topologies.

If we allow any name, we will then have to implement a validating webhook that ensures that only one instance of this CustomResource exists in the entire cluster. This seems more complicated the enforcing the name of the CustomResource that is to be created, which Grove will always look for.

If the consumer chooses to change the topology, then they will edit this configuration CR, which Grove will make use of by fetching it and making use of the newly specified mappings before creating any resources.

sanjaychatterjee · 2025-10-22T04:04:06Z

docs/designs/topology.md

+### Component Interactions
+
+```
+TopologyDomain CRD ──┐


sanjaychatterjee · 2025-10-22T04:06:24Z

docs/designs/topology.md

+
+The TopologyDomain controller manages:
+
+- **Kueue Topology Generation**: Auto-creates Kueue Topology CRD for KAI scheduler integration


CRD -> CR

I don't think this should be Grove operator responsibility. Instead, KAI PodGrouper's Grove plugin can do this.

I have to say I am a bit perplexed why Grove and KAI wouldnt use same CRD

sanjaychatterjee · 2025-10-22T04:07:02Z

docs/designs/topology.md

+│  Admin Layer:                                                           │
+│  ┌──────────────────┐              ┌────────────────────┐               │
+│  │ TopologyDomain   │─────────────▶│ TopologyDomain     │               │
+│  │ CRD              │              │ Controller         │               │


sanjaychatterjee · 2025-10-22T04:13:33Z

docs/designs/topology.md

+**Example TopologyDomain:**
+
+```yaml
+apiVersion: grove.run.ai/v1alpha1


grove.run.ai/v1alpha1 -> grove.io/v1alpha1

sanjaychatterjee · 2025-10-22T04:16:28Z

docs/designs/topology.md

+// TopologyDomain defines the topology hierarchy for the cluster
+// This resource is immutable after creation
+// Multiple TopologyDomain resources can exist; Grove operator references one via --topology-domain-name argument
+type TopologyDomain struct {


Can we rename this to just Topology ?

As we need to use both our topology and kueue topology and kueue already taken the name Topology,
I was thinking it would make confusion and will annoying to add the full name (include api group and version) of each topology.
what do you think?

sanjaychatterjee · 2025-10-22T04:18:01Z

docs/designs/topology.md

+Spec TopologyDomainSpec `json:"spec,omitempty"`
+}
+
+type TopologyDomainSpec struct {


Can we rename this to TopologySpec

sanjaychatterjee · 2025-10-22T04:34:01Z

docs/designs/topology.md

+
+The TopologyDomain controller manages the TopologyDomain resource lifecycle with two primary responsibilities:
+
+**1. Kueue Topology Generation**


This responsibility should be with KAI PodGrouper.

its depend who you ask, some the kai team didnt agree about this
(we could do it for kubecon, but overall I believe we need to integrate with other and not expect other to do it for us), some day vlolcano might support this api, They will never create the topology they using for us

sanjaychatterjee · 2025-10-22T05:33:21Z

docs/designs/topology.md

+// TopologyRef references the Kueue Topology resource
+// Points to Kueue Topology CRD auto-generated by TopologyDomain controller
+// +optional
+TopologyRef *string `json:"topologyRef,omitempty"`


We won't need this if we move the Kueue topology generation to KAI PodGrouper.

we still need it, if the name of the topology isn't fixed, as it will need to know the name the user topology and create kueue topology out of it

shayasoolin · 2025-10-22T08:49:43Z

docs/designs/topology.md

+
+- **Network Locality**: Proximity improves high-bandwidth communication between leaders and their respective workers (
+  prefill and decode, etc)
+    - **Coordinated Placement**: Related components (e.g., model shards) perform better when co-located within the same


All the indentations in the file should be fixed, they're not formatted properly. Both bullet and numbered ones.

shayasoolin · 2025-10-22T08:51:59Z

docs/designs/topology.md

+    - Enable packing constraints for network locality across all Grove scalable resources
+    - Support multiple topology configurations for different environments
+    - Automatic Kueue Topology generation for KAI scheduler integration
+    - Immutable topology configuration ensuring scheduling consistency


That means that in order to fix or edit an existing configuration, the admin will have to create a new one, switch to it and delete the previous one.
Can we consider allowing edits if no PCS is currently using the topology? Similarly to how deletions are handled.

you are correct, it a bit wired, I will change this section and clearfy that deletion fo topology is allowed only if no one ref them, it will be easier and more logical

If a PCS does end up using this CR, then we are stuck with it forever/until that PCS is deleted.

An easier way to tackle this is to specify a reference to an instance of the configuration CR? Newly created PCS-s can rely on a different instance of the CR, which has the correct configuration?

shayasoolin · 2025-10-22T09:48:30Z

docs/designs/topology.md

+    2. **Operator Configuration**: References TopologyDomain by name
+        - Operator argument `--topology-domain-name=default` selects which TopologyDomain to use
+        - All workload validation performed against configured TopologyDomain
+        - Enables switching between topologies without changing workloads


What do you mean here? Maybe "without changing the workload specs"? That's also not always true, it depends on the nature of the change in topology configuration (e.g. whether level names were changed.)

shayasoolin · 2025-10-22T09:53:07Z

docs/designs/topology.md

+
+- Cluster administrators are responsible for ensuring that node labels specified in `topologyKey` fields exist on
+  cluster nodes
+    - TopologyDomain creation succeeds even if labels don't exist yet (allows pre-configuration)


I understand the requirement for pre-configuration, but I believe we should at least offer the admin to provide validation for the TopologyDomain resource. Leaving misconfiguration errors to the user, upon workload creation, is (a) too late and (b) not the right persona to handle.
I suggest to allow some annotation for example (either opt-in or out) that instructs the operator to validate the topology resource against the nodes in the cluster.
Sample basic validation we can apply there: each of the node labels must exist on at least one node in the cluster. Perhaps we can think of a stronger validation, but that's a start and will catch simple typos.

shayasoolin · 2025-10-22T09:54:14Z

docs/designs/topology.md

+Configuration:
+
+- `--topology-domain-name`: Specifies TopologyDomain resource name for validation
+    - Operator loads referenced TopologyDomain at startup


Please mention here that this is an optional argument, the operator can run without it.

shayasoolin · 2025-10-22T09:58:42Z

docs/designs/topology.md

+
+**Immutability:**
+
+- All TopologyConstraint fields immutable after resource creation


Why is that? Do we plan to lift this limitation in the future, and describing here the contents of the first phase?
I mean, we can allow changing the topology constraint of a component in two cases:

It doesn't conflict with the overall topology spec of the PCS as a whole.

The rolling update is for the whole PCS (rather than individual components), a case in which a complete PCS replica is going to be created, and it can use the new topology constraints.

But again, if this note is mentioned here as "out of scope for now" I definitely agree we better not deal with this complication at this stage.

shayasoolin · 2025-10-22T10:01:25Z

docs/designs/topology.md

+
+**Preferred Constraints (Auto-Generated):**
+
+- Operator ALWAYS generates preferred constraint at all three levels


Nitpick, but this is true unless the user provided a "Required" constraint for the same (strictest) level. Or we don't care and set preferred in addition, as it's harmless?

omer-dayan · 2025-10-22T10:05:52Z

docs/designs/topology.md

+    - Multiple TopologyDomains supported for different environments
+
+    2. **Operator Configuration**: References TopologyDomain by name
+        - Operator argument `--topology-domain-name=default` selects which TopologyDomain to use


Another option is to have a annotation in the TopologyDomain CR "is-default: true" that only one topology has, and grove knows which one is the default.

A benefit of that, is that you can change the used topology without restarting the grove controller.
The bad side is that you need to handle what happen if there are 2 instances or more with that annotation.

Thats how default StorageClass is implemented in K8S: https://kubernetes.io/docs/tasks/administer-cluster/change-default-storage-class/

omer-dayan · 2025-10-22T10:06:31Z

docs/designs/topology.md

+    2. **Operator Configuration**: References TopologyDomain by name
+        - Operator argument `--topology-domain-name=default` selects which TopologyDomain to use
+        - All workload validation performed against configured TopologyDomain
+        - Enables switching between topologies without changing workloads


Does the user has the ability to override that? Or will have in the future?

unmarshall

I have only partly done the review. Posting comments in batches.

unmarshall · 2025-10-22T09:03:01Z

docs/designs/topology.md

+
+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels


Is the TopologyDomain CRD required?

I'm proposing a CRD for cluster topology to support complex scenarios, like a single cluster with multiple network layouts (e.g., for hybrid on-prem/cloud environments).

This lets admins dynamically define multi-level topologies with user-friendly names (rack, spine) and is far more extensible than a hard-coded list or ConfigMap, allowing for a future status field.

Can friendly names be expanded on here? Perhaps examples like rack, spine, and host can be added in brackets so the sentence is clearer in a first pass?

Or we can instead write cluster topology hierarchy mapping Grove defined cluster topologies to node labels?

unmarshall · 2025-10-22T09:05:01Z

docs/designs/topology.md

+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels
+    2. **Operator Configuration**: Selects active topology via `--topology-domain-name` argument


what is active topology?

the default topology configured by admin, and applied to all the pcs submitted

We do not provide configuration flags like --topology-domain-name in Grove. The operator can only be configured through OperatorConfiguration, where each configuration is a field/subfield in the manfiest file provided. So this could be OperatorConfiguration.TopologyDomainName.

unmarshall · 2025-10-22T09:10:00Z

docs/designs/topology.md

+       PodClique)
+
+**Key Feature**: Grove provides automatic out-of-box topology optimization by generating preferred packing constraints
+at all levels, even without user configuration. Users can optionally specify required constraints for strict placement


what is the out-of-box topology optimization that is generated by Grove especially when no pack configuration is defined by the user? Ideally if no pack configuration is specified then should it not mean that there is no strict requirement or in other words there is no packing constraints that is observed by the scheduler while taking scheduling decisions?

unmarshall · 2025-10-22T09:14:42Z

docs/designs/topology.md

+
+- Provide flexible, cluster-agnostic topology hierarchy definition via TopologyDomain CRD
+    - Enable packing constraints for network locality across all Grove scalable resources
+    - Support multiple topology configurations for different environments


what does environment mean here?

cluster with hybrid on-prem/cloud environments.

unmarshall · 2025-10-22T09:15:48Z

docs/designs/topology.md

+- Provide flexible, cluster-agnostic topology hierarchy definition via TopologyDomain CRD
+    - Enable packing constraints for network locality across all Grove scalable resources
+    - Support multiple topology configurations for different environments
+    - Automatic Kueue Topology generation for KAI scheduler integration


who generates the keueu topology CR. It should not be done in Grove IMHO.

Some of KAI maintains prefer that grove will integrate with KAI and not the other way around, I belive we are the one that support to integrate with other schedulers today it's KAI but tomorrow it might be Volcano, so if KAI work in a cerain way we are the one that should create it IMHO, (also KAI need to agree for this)

unmarshall · 2025-10-22T10:08:07Z

docs/designs/topology.md

+
+- Operator automatically generates **preferred** constraints using strictest topology level (e.g., "host")
+    - Applied at all three levels (PodGang, NetworkPackGroup, PodGroup) during translation to scheduler API
+    - Users get optimal packing without configuration


I am not sure this is the correct as described above. Grove does not understand the workload semantics and therefore is in no position to determine the optimal packing configuration for it.

I describe it to optimal, it better to say it provides an opportunistic performance boost

unmarshall · 2025-10-22T10:09:03Z

docs/designs/topology.md

+
+**User Control:**
+
+- Users can specify **required** constraints via `packDomain` for strict placement requirements


Maybe you can remove the word strict here as users can also define relaxed pack constraints as well.

unmarshall · 2025-10-22T10:11:24Z

docs/designs/topology.md

+// Description provides human-readable information about this level
+// +kubebuilder:validation:MaxLength=1024
+// +optional
+Description string `json:"description,omitempty"`


why is this required? who will use it?

I thought i will helpful for users who want to understand better the levels but it only suggestion

unmarshall · 2025-10-22T10:12:50Z

docs/designs/topology.md

+├─────────────────────────────────────────────────────────────────────────┤
+│                                                                         │
+│  Admin Layer:                                                           │
+│  ┌──────────────────┐              ┌────────────────────┐               │


why do you recommend a different controller for it when it is part of the creation flow of a PCS?

unmarshall · 2025-10-22T10:17:03Z

docs/designs/topology.md

+
+### Controller Responsibilities
+
+The TopologyDomain controller manages:


Why should Grove manage any Kueue CRs? Today KAI scheduler decides to have dependency on Kueue but that is the choice that KAI makes. So while Grove today uses KAI scheduler its responsibility should not be tied to a single scheduler (in this case KAI scheduler). Therefore creation of CRs that are required by KAI should be done in KAI using the enhancements to the PodGang scheduler API that Grove defines.

I don't agree with you about this,KAI are doing us a favor here (they would not agree it on them), if you want to integrate with them not the other way around, in the feature we would want to integrate Volcano/unicron/ or even the default scheduler you need to work with the API they agree upon

renormalize

thanks for the proposal!

1/n as I have yet to go through most of the document.

renormalize · 2025-10-22T12:22:22Z

docs/designs/topology.md

+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels
+    2. **Operator Configuration**: Selects active topology via `--topology-domain-name` argument
+    3. **TopologyConstraint**: User-specified packing requirements in workloads (PodCliqueSet, PodCliqueScalingGroup,
+       PodClique)


Each numbered point is not being rendered in its own line because of the indentation. Can this be corrected? There are also multiple other places in the file where there are unnecessary newlines/indentation which causes the rendered markdown to look weird.

docs/designs/topology.md

renormalize · 2025-10-22T12:23:55Z

docs/designs/topology.md

+
+## Overview
+
+This document defines the design for implementing topology-aware scheduling in the Grove operator.


nit:

Suggested change

This document defines the design for implementing topology-aware scheduling in the Grove operator.

This document defines the design for supporting topology-aware scheduling in the Grove operator.

renormalize · 2025-10-22T12:25:32Z

docs/designs/topology.md

+applications require:
+
+- **Network Locality**: Proximity improves high-bandwidth communication between leaders and their respective workers (
+  prefill and decode, etc)


nit:

A period is typically added after the abbreviation etc.. Also, unnecessary new line.

Suggested change

prefill and decode, etc)

prefill and decode, etc.)

renormalize · 2025-10-22T12:34:07Z

docs/designs/topology.md

+
+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels


Can friendly names be expanded on here? Perhaps examples like rack, spine, and host can be added in brackets so the sentence is clearer in a first pass?

Or we can instead write cluster topology hierarchy mapping Grove defined cluster topologies to node labels?

renormalize · 2025-10-22T12:59:46Z

docs/designs/topology.md

+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels
+    2. **Operator Configuration**: Selects active topology via `--topology-domain-name` argument


What would happen if the user creates multiple instances of this CustomResource? How will Grove decide which to pick?

During runtime, there is only one CR that Grove must use for specifying topologies.

If we allow any name, we will then have to implement a validating webhook that ensures that only one instance of this CustomResource exists in the entire cluster. This seems more complicated the enforcing the name of the CustomResource that is to be created, which Grove will always look for.

If the consumer chooses to change the topology, then they will edit this configuration CR, which Grove will make use of by fetching it and making use of the newly specified mappings before creating any resources.

renormalize · 2025-10-22T13:03:56Z

docs/designs/topology.md

+**Design Approach**: This design introduces a flexible topology system with three main components:
+
+1. **TopologyDomain CRD**: Admin-configured cluster topology hierarchy mapping friendly names to node labels
+    2. **Operator Configuration**: Selects active topology via `--topology-domain-name` argument


We do not provide configuration flags like --topology-domain-name in Grove. The operator can only be configured through OperatorConfiguration, where each configuration is a field/subfield in the manfiest file provided. So this could be OperatorConfiguration.TopologyDomainName.

renormalize · 2025-10-22T13:12:52Z

docs/designs/topology.md

+       PodClique)
+
+**Key Feature**: Grove provides automatic out-of-box topology optimization by generating preferred packing constraints
+at all levels, even without user configuration. Users can optionally specify required constraints for strict placement


If no configuration is provided, and the user has no real constraints that they need to follow, how would they be able to opt out if Grove always tries for the optimal by default?

If by default Grove tries for the optimal placement, and the scheduler is unable to schedule pods with these constraints, then the workload will never fully be up even though the user has no requirement for it.

Perhaps we can provide a field in the API through which the user indicates that they want out-of-box topology optimization. Or, we can provide a way for user to opt out of these out-of-box optimizations if they do not really care?

renormalize · 2025-10-22T13:17:46Z

docs/designs/topology.md

+    - Enable packing constraints for network locality across all Grove scalable resources
+    - Support multiple topology configurations for different environments
+    - Automatic Kueue Topology generation for KAI scheduler integration
+    - Immutable topology configuration ensuring scheduling consistency


If the configuration is changed, and the PCS is mutated, will Grove then go ahead and recreate all the children PCSG and PCLQs because the previous configurations were less strict, and now Grove enforces the more strict configuration?

renormalize · 2025-10-22T13:20:40Z

docs/designs/topology.md

+    - Enable packing constraints for network locality across all Grove scalable resources
+    - Support multiple topology configurations for different environments
+    - Automatic Kueue Topology generation for KAI scheduler integration
+    - Immutable topology configuration ensuring scheduling consistency


If a PCS does end up using this CR, then we are stuck with it forever/until that PCS is deleted.

An easier way to tackle this is to specify a reference to an instance of the configuration CR? Newly created PCS-s can rely on a different instance of the CR, which has the correct configuration?

Signed-off-by: Ron Kahn <[email protected]>

Co-authored-by: Saketh Kalaga <[email protected]> Signed-off-by: Ron Kahn <[email protected]>

shayasoolin · 2025-10-26T11:22:08Z

docs/designs/topology.md

+- Provide flexible, cluster-agnostic topology hierarchy definition via TopologyDomain CRD
+- Enable packing constraints for network locality across all Grove scalable resources
+- Support multiple topology configurations for different environments
+- Automatic Kueue Topology generation for KAI scheduler integration


AFAIU this is not a goal anymore, so please remove it. Also appears in other parts of the document, search for "kueue" to find them.

shayasoolin · 2025-10-26T11:23:16Z

docs/designs/topology.md

+- Enable packing constraints for network locality across all Grove scalable resources
+- Support multiple topology configurations for different environments
+- Automatic Kueue Topology generation for KAI scheduler integration
+- Immutable topology configuration ensuring scheduling consistency


Please be more explicit here and clarify that the immutable topology is of the global configuration, not the require topology of a certain workload (as there it's now mutable.)

shayasoolin · 2025-10-26T11:24:49Z

docs/designs/topology.md

+
+**Out-of-Box Optimization:**
+
+- Operator automatically generates **preferred** constraints using strictest topology level (e.g., "host")


Is this agreed or still under discussion? @unmarshall commented about it in a different place.

Ronkahn21 added 2 commits October 21, 2025 12:58

feat: add design document for topology-aware scheduling in Grove oper…

7a3448d

…ator Signed-off-by: Ron Kahn <[email protected]>

fix some structs issues

d53736e

Signed-off-by: Ron Kahn <[email protected]>

Ronkahn21 requested review from sanjaychatterjee and unmarshall as code owners October 21, 2025 11:31

romanbaron reviewed Oct 21, 2025

View reviewed changes

docs/designs/topology.md Outdated Show resolved Hide resolved

romanbaron reviewed Oct 21, 2025

View reviewed changes

Update docs/designs/topology.md

cdf2b67

typo fix Co-authored-by: Roman Baron <[email protected]> Signed-off-by: Ron Kahn <[email protected]>

romanbaron reviewed Oct 21, 2025

View reviewed changes

docs: improve network locality description in topology design

9012c4a

Signed-off-by: Ron Kahn <[email protected]>

sanjaychatterjee reviewed Oct 22, 2025

View reviewed changes

shayasoolin reviewed Oct 22, 2025

View reviewed changes

omer-dayan reviewed Oct 22, 2025

View reviewed changes

unmarshall requested changes Oct 22, 2025

View reviewed changes

renormalize reviewed Oct 22, 2025

View reviewed changes

Ronkahn21 and others added 4 commits October 22, 2025 18:04

CR Fixes

4f5b0c4

Signed-off-by: Ron Kahn <[email protected]>

CR Fixes

63158be

Signed-off-by: Ron Kahn <[email protected]>

restore deleted part

6ec6b98

Signed-off-by: Ron Kahn <[email protected]>

Update docs/designs/topology.md

ef135ea

Co-authored-by: Saketh Kalaga <[email protected]> Signed-off-by: Ron Kahn <[email protected]>

shayasoolin reviewed Oct 26, 2025

View reviewed changes


		The TopologyDomain controller manages:

		- Kueue Topology Generation: Auto-creates Kueue Topology CRD for KAI scheduler integration


		The TopologyDomain controller manages the TopologyDomain resource lifecycle with two primary responsibilities:

		1. Kueue Topology Generation


		Immutability:

		- All TopologyConstraint fields immutable after resource creation


		Preferred Constraints (Auto-Generated):

		- Operator ALWAYS generates preferred constraint at all three levels


		Design Approach: This design introduces a flexible topology system with three main components:

		1. TopologyDomain CRD: Admin-configured cluster topology hierarchy mapping friendly names to node labels


		User Control:

		- Users can specify required constraints via `packDomain` for strict placement requirements


		### Controller Responsibilities

		The TopologyDomain controller manages:


		## Overview

		This document defines the design for implementing topology-aware scheduling in the Grove operator.

	This document defines the design for implementing topology-aware scheduling in the Grove operator.
	This document defines the design for supporting topology-aware scheduling in the Grove operator.


		Out-of-Box Optimization:

		- Operator automatically generates preferred constraints using strictest topology level (e.g., "host")

Grove proposal/topology #224

Are you sure you want to change the base?

Grove proposal/topology #224

Conversation

Ronkahn21 commented Oct 21, 2025

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a API change?

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sanjaychatterjee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment