Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IntelRdt to CDI spec #164

Merged
merged 1 commit into from
Jan 10, 2024
Merged

Add IntelRdt to CDI spec #164

merged 1 commit into from
Jan 10, 2024

Conversation

marquiz
Copy link
Contributor

@marquiz marquiz commented Sep 26, 2023

Add support for specifying the OCI Linux.IntelRdt configuration that is the control point for cache and memory bandwidth allocation technologies sucn as Intel RDT (Resource Director Technology). There can only be one IntelRdt configuration per container so in case multiple specs happened to specify it the last one applied prevails. Also, the IntelRdt configuration is always applied as a whole - editing specific sub-fields is not supported.

@marquiz marquiz force-pushed the devel/intel-rdt branch 3 times, most recently from 66b31ca to c4b03ce Compare September 29, 2023 06:11
@marquiz marquiz changed the title WIP: Add IntelRdt to CDI spec Add IntelRdt to CDI spec Sep 29, 2023
@marquiz
Copy link
Contributor Author

marquiz commented Sep 29, 2023

Updated PR:

  • Rebased
  • Aligned schema fields, now they correspond one-to-one with the OCI spec
  • Removed WIP status

@zvonkok
Copy link
Contributor

zvonkok commented Sep 29, 2023

@marquiz You may want to enclose CLOS as CLOS so that spell check succeeds.

@marquiz
Copy link
Contributor Author

marquiz commented Sep 29, 2023

You may want to enclose CLOS as CLOS so that spell check succeeds.

Thx, did that. Lovely smart spell checker...

Copy link
Contributor

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work here @marquiz.

I have some questions / comments below.

One thing that is also missing is automatically determining spec version 0.7.0 when calling MinimumRequiredVersion in version.go.

SPEC.md Outdated Show resolved Hide resolved
@@ -202,7 +217,7 @@ func (e *ContainerEdits) isEmpty() bool {
if e == nil {
return false
}
return len(e.Env)+len(e.DeviceNodes)+len(e.Hooks)+len(e.Mounts) == 0
return len(e.Env)+len(e.DeviceNodes)+len(e.Hooks)+len(e.Mounts) == 0 && e.IntelRdt == nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Should we explicitly check for an empty IntelRdt object here, or is checking for a nil sufficient?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think checking nil is the right thing to do. From the OCI runtime-spec perspective (or the crun implementation, on it's behalf, IIRC) it's valid to have a non-nil IntelRdt in which all fields are empty. (In this case the runtime should create a container-specific CLOS with default settings).

pkg/cdi/container-edits_test.go Show resolved Hide resolved
pkg/cdi/container-edits.go Show resolved Hide resolved
@@ -192,6 +204,9 @@ func (e *ContainerEdits) Append(o *ContainerEdits) *ContainerEdits {
e.DeviceNodes = append(e.DeviceNodes, o.DeviceNodes...)
e.Hooks = append(e.Hooks, o.Hooks...)
e.Mounts = append(e.Mounts, o.Mounts...)
if o.IntelRdt != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the expected behaviour is both are non-nil? Here we overwrite the previous value if the input value is non-nil.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

intelRDT fiels in OCI spec can't have multiple values, so overwriting already set values theoretically expected behaviour. One thing which is questionable, is to go over individual values, instead of overwriting as a whole struct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's like @kad explained. There can only be single RDT configuration per container. Also, the config must be applied as a whole (merging multiple RDT configs together is not something that we want).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the COD working group meeting, I'm happy as long as this is consistent with what is done for other edits (Env, Mounts). It seems to be the case in that the last value applied takes precedence.

We also mentioned that if we do want to expose errors due to conflicts, we can do so in a follow-up.

One thing that would be good would be to add a unit test to capture this behaviour to prevent regressions.

@marquiz marquiz force-pushed the devel/intel-rdt branch 4 times, most recently from 3419299 to 6392a5f Compare January 9, 2024 13:53
@marquiz
Copy link
Contributor Author

marquiz commented Jan 9, 2024

TODO:

  • support version validation
  • add unit test(s) for overrides

@elezar
Copy link
Contributor

elezar commented Jan 9, 2024

@marquiz here is the code to determine the automatic version:

func requiresV060(spec *cdi.Spec) bool {

Something like:

diff --git a/pkg/cdi/version.go b/pkg/cdi/version.go
index a617812..d2f024b 100644
--- a/pkg/cdi/version.go
+++ b/pkg/cdi/version.go
@@ -39,6 +39,7 @@ const (
 	v040 version = "v0.4.0"
 	v050 version = "v0.5.0"
 	v060 version = "v0.6.0"
+	v070 version = "v0.7.0"
 
 	// vEarliest is the earliest supported version of the CDI specification
 	vEarliest version = v030
@@ -54,6 +55,7 @@ var validSpecVersions = requiredVersionMap{
 	v040: requiresV040,
 	v050: requiresV050,
 	v060: requiresV060,
+	v070: requiresV070,
 }
 
 // MinimumRequiredVersion determines the minimum spec version for the input spec.
@@ -118,6 +120,20 @@ func (r requiredVersionMap) requiredVersion(spec *cdi.Spec) version {
 	return minVersion
 }
 
+func requiresV070(spec *cdi.Spec) bool {
+	if spec.ContainerEdits.IntelRdt != nil {
+		return true
+	}
+
+	for _, d := range spec.Devices {
+		if d.ContainerEdits.IntelRdt != nil {
+			return true
+		}
+	}
+
+	return false
+}
+
 // requiresV060 returns true if the spec uses v0.6.0 features
 func requiresV060(spec *cdi.Spec) bool {
 	// The v0.6.0 spec allows annotations to be specified at a spec level

@elezar
Copy link
Contributor

elezar commented Jan 10, 2024

@marquiz I have added the AdditionalGIDs extension as discussed yesterday in #179. I would expect some conflicts with your changes, so I'm happy to wait for your PR to land first.

@marquiz
Copy link
Contributor Author

marquiz commented Jan 10, 2024

Thanks @elezar. I now updated the PR with the TODO items resolved:

  1. Support for v0.7.0 version check was added (with unit tests)
  2. Unit tests (TestApply) extended to cover overriding previous data

In addition I updated the TestAppend unit test to cover intelRdt field.

Copy link
Contributor

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last question / comment.

Otherwise LGTM.

pkg/cdi/container-edits.go Outdated Show resolved Hide resolved
Add support for specifying the OCI Linux.IntelRdt configuration that
is the control point for cache and memory bandwidth allocation
technologies sucn as Intel RDT (Resource Director Technology). There can
only be one IntelRdt configuration per container so in case multiple
specs happened to specify it the last one applied prevails. Also, the
IntelRdt configuration is always applied as a whole - editing specific
sub-fields is not supported.

Signed-off-by: Markus Lehtonen <[email protected]>
@elezar elezar self-requested a review January 10, 2024 16:06
Copy link
Contributor

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @marquiz.

LGTM!

@elezar elezar merged commit 57c92f8 into cncf-tags:main Jan 10, 2024
7 checks passed
@marquiz marquiz deleted the devel/intel-rdt branch January 10, 2024 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants