Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a hostNetwork setting to the Driver and ControllerPlugin Specs #194

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

obnoxxx
Copy link
Collaborator

@obnoxxx obnoxxx commented Jan 13, 2025

This bool setting can be added to the controllerPlugin section of the Driver Spec.

It will be propagated to all controller plugin pods.

This implements the following design:

https://github.com/ceph/ceph-csi-operator/blob/main/docs/design/hostNetwork.md

see issue #157 for background and context.

Describe what this PR does

This work in progress is a draft PR for the implementation of host Networking for controller plugin pods as described in https://github.com/ceph/ceph-csi-operator/blob/main/docs/design/hostNetwork.md

Is there anything that requires special attention

This needs thorough testing.

Is the change backward compatible?

should be but this needs testing.

Are there concerns around backward compatibility?

none right now

For example:

Related issues

Fixes: #157

Future concerns

none.

Checklist:

  • Commit Message Formatting: Commit titles and messages follow
    guidelines in the developer
    guide
    .
  • Reviewed the developer guide on Submitting a Pull
    Request
  • Pending release
    notes

    updated with breaking and/or notable changes for the next major release.
  • Documentation has been updated, if necessary.
  • Unit tests have been added, if necessary.
  • Integration tests have been added, if necessary.

This bool setting can be added to the controllerPlugin section of the
Driver Spec.

It will be propagated to all controller plugin pods.

This implements the following design:

https://github.com/ceph/ceph-csi-operator/blob/main/docs/design/hostNetwork.md

Signed-off-by: Michael Adam <[email protected]>
@obnoxxx
Copy link
Collaborator Author

obnoxxx commented Jan 13, 2025

@nb-ohad, @Madhu-1 , @rohan47, this is a replacement for the closed stale PR #176
this is a replacement

@obnoxxx
Copy link
Collaborator Author

obnoxxx commented Feb 5, 2025

PR updated to address the port collisions that @rohan47 observed in testing.

I have built an image and pushed it to quay:

$ docker pull quay.io/madam/ceph-csi-operator:host-net`

or:

 $ podman pull quay.io/madam/ceph-csi-operator:host-net

@@ -771,15 +772,15 @@ func (r *driverReconcile) reconcileControllerPluginDeployment() error {
utils.PodContainerArg,
utils.PodUidContainerArg,
utils.CsiAddonsAddressContainerArg,
utils.ControllerPortContainerArg,
utils.ControllerPort2ContainerArg,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better name for the variable is required, we have 2 deployments where csi-addons is enabled (cephfs and rbd), don't we going to get a port conflict when both deployments(pods) run on the same node?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for better name, I could do something like PortDeployment and PortDeamonSet. @Madhu-1 , wdyt?

// CSI Addons container port definition
var CsiAddonsContainerPort = corev1.ContainerPort{
// CSI Addons container port definitions
var CsiAddonsContainerPort1 = corev1.ContainerPort{
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have few question regarding the port in use here @Madhu-1

  1. What is the use of this port?
  2. Is this port information required anywhere else?
  3. Since we know that we need to mitigate port collision when we use hostNetwork, can we hard code new ports?

Copy link
Collaborator Author

@obnoxxx obnoxxx Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rohan47 , the ports are in fact hard-coded in internal/utils/csi.go

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have few question regarding the port in use here @Madhu-1

  1. What is the use of this port?
  2. Is this port information required anywhere else?

These ports are used for csi-addons server

  1. Since we know that we need to mitigate port collision when we use hostNetwork, can we hard code new ports?

Yes, it can be hardcoded in this PR but we need to have a new PR to make it configurable as we support creating multiple instances of the csi drives. For 2 instances of driver will get in to the same problem of port (This bug we have today as well for daemonset)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Madhu-1 : do you mean a PR as preparation for this one?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes to make the port configurable in CR in a new PR

Usubg host network produces port collisions.

So we use different ports for controller plugin deployments and
node plugin deamonsets  to avoid collisions.

Signed-off-by: Michael Adam <[email protected]>
@obnoxxx
Copy link
Collaborator Author

obnoxxx commented Feb 6, 2025

@Madhu-1 : updated with names ControllerPluginCsiAddonsContainerPort and NodePluginCsiAddonsContainerPort. We can dtill add logic to differetiate between drivers as follow-up.

@obnoxxx
Copy link
Collaborator Author

obnoxxx commented Feb 6, 2025

CI failures seem to be mostly due to problems with setup-go (network gateway timeouts)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Enforce host networking for Ceph CSI controller plugin pods
3 participants