Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Configure Manila with a Storage NFS network #2273

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

fultonj
Copy link
Contributor

@fultonj fultonj commented Aug 27, 2024

Manila Tempest tests need to connect to the NFS share
for Ganesha tests, and they use a special (openstack)
network for that [1].

This patch adds an NFS network with VLAN 24 and range
172.21.0.0/24 in reproducers networking-definition.yml.
It also adds a multus range for this network so that the
Tempest pod can access this network for testing. The NFS
network is added to the OCP nodes for the same reason.
The podified-multinode-hci-deployment-crc job is updated
to not deploy manila since it was never tested by tempest.

This patch allows the RGW VIP and NFS VIP to not be the same.
The variables cifmw_cephadm_rgw_network, cifmw_cephadm_rgw_vip,
and cifmw_cephadm_nfs_network and cifmw_cephadm_nfs_vip are now
used instead the shared cifmw_cephadm_vip variable.

This patch updates playbook manila_create_default_resources.yml
so that when CI for manila runs, a provider network is created.
Variables manila_provider_network_{name,vlan,start,end,range}
default to the storage network, but can be overridden to the
NFS network within a CI job definition.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417

Copy link
Contributor

openshift-ci bot commented Aug 27, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from fultonj. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

Thanks for the PR! ❤️
I'm marking it as a draft, once your happy with it merging and the PR is passing CI, click the "Ready for review" button below.

@github-actions github-actions bot marked this pull request as draft August 27, 2024 22:29
Copy link

This change depends on a change that failed to merge.

Change openstack-k8s-operators/architecture#376 is needed.

Copy link

This change depends on a change that failed to merge.

Change openstack-k8s-operators/architecture#376 is needed.

Copy link

This change depends on a change that failed to merge.

Change openstack-k8s-operators/architecture#376 is needed.

fultonj added a commit to fultonj/architecture that referenced this pull request Sep 5, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests.

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

TODO: update Manila CRDs to use this network.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
Copy link

This change depends on a change that failed to merge.

Change openstack-k8s-operators/architecture#376 is needed.

fultonj added a commit to fultonj/architecture that referenced this pull request Sep 5, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests.

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

TODO: update Manila CRDs to use this network.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
fultonj added a commit to fultonj/architecture that referenced this pull request Sep 5, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests.

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

TODO: update Manila CRDs to use this network.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
fultonj added a commit to fultonj/architecture that referenced this pull request Sep 5, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests.

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

TODO: update Manila CRDs to use this network.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
fultonj added a commit to fultonj/architecture that referenced this pull request Sep 5, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests.

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

TODO: update Manila CRDs to use this network.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
@fultonj fultonj marked this pull request as ready for review September 9, 2024 19:21
@fultonj fultonj requested a review from fmount September 9, 2024 19:21
fultonj added a commit to fultonj/architecture that referenced this pull request Sep 9, 2024
Manila Tempest tests need to connect to the share for
Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN 24 and
range 172.21.0.0/24 in uni04delta. The NFS network is
connected to Ceph and Compute EDPM nodes. A NNCP, NAD,
L2Advertisement and IPAddressPool are defined for the
NFS network so that a pod in k8s can connect to it; such
as the tempest pod which will perform the storage tests
and the manilaShares pod(s).

In order to make these changes, uni04delta now keeps its
own copy of the nncp and networking directories since
they differ (by the new network) from the generic ones
in the lib directory.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
@fmount
Copy link
Contributor

fmount commented Oct 3, 2024

  • podified-multinode-hci-deployment-crc is finally green

    • need to rebase on main

I rebased on main and removed the RGW related bits: they can be part of a different patch where we focus on the additional swift network.

@fmount
Copy link
Contributor

fmount commented Oct 3, 2024

@fultonj I'm not sure the plan is to add the swift network on the same patch, but my idea is to split those two changes if possible. For this reason I removed swift from the game and simply added the relevant NFS bits.

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/701c45f03a8e48bebd3a140217e6a249

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 37m 53s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 16m 43s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 26m 09s
podified-multinode-hci-deployment-crc FAILURE in 57m 09s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 40s
✔️ cifmw-pod-pre-commit SUCCESS in 6m 52s
✔️ build-push-container-cifmw-client SUCCESS in 20m 58s
✔️ cifmw-molecule-ci_gen_kustomize_values SUCCESS in 6m 09s
✔️ cifmw-molecule-cifmw_cephadm SUCCESS in 5m 21s

@fmount
Copy link
Contributor

fmount commented Oct 3, 2024

  • I see the storage network is created by the hook [1]
  • I see Ganesha and the ingress uses 172.18.0.0/24, the storage network, which is expected as we get a VIP from that network ("virtual_ip": "172.18.0.2") [2]
  • I do not see a nfs network being added to the hci job and I think this is expected (we need it in unidelta, not HCI)

@fultonj I'll wait for your input to discuss how to move forward with this. I think we need to testproject to check the NFS bits on a valid job/use case.

[1] https://logserver.rdoproject.org/73/2273/34ed0e444c64a477720095f94528d0c6ddd88cc3/github-check/podified-multinode-hci-deployment-crc/757c5a8/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/ctlplane/neutron/subnet_list
[2] https://logserver.rdoproject.org/73/2273/34ed0e444c64a477720095f94528d0c6ddd88cc3/github-check/podified-multinode-hci-deployment-crc/757c5a8/controller/ci-framework-data/logs/ceph/ceph_daemons.log

@fultonj
Copy link
Contributor Author

fultonj commented Oct 3, 2024

  • I see the storage network is created by the hook [1]
  • I see Ganesha and the ingress uses 172.18.0.0/24, the storage network, which is expected as we get a VIP from that network ("virtual_ip": "172.18.0.2") [2]
  • I do not see a nfs network being added to the hci job and I think this is expected (we need it in unidelta, not HCI)

That all makes sense to me.

@fultonj I'll wait for your input to discuss how to move forward with this. I think we need to testproject to check the NFS bits on a valid job/use case.

Yes, I agree a test project downstream is the step.

[1] https://logserver.rdoproject.org/73/2273/34ed0e444c64a477720095f94528d0c6ddd88cc3/github-check/podified-multinode-hci-deployment-crc/757c5a8/controller/ci-framework-data/logs/openstack-k8s-operators-openstack-must-gather/ctlplane/neutron/subnet_list [2] https://logserver.rdoproject.org/73/2273/34ed0e444c64a477720095f94528d0c6ddd88cc3/github-check/podified-multinode-hci-deployment-crc/757c5a8/controller/ci-framework-data/logs/ceph/ceph_daemons.log

@fultonj
Copy link
Contributor Author

fultonj commented Oct 3, 2024

the new file LGTM thanks

Manila Tempest tests need to connect to the NFS share
for Ganesha tests, and they use a special (openstack)
network for that [1].

This patch adds an NFS network with VLAN 24 and range
172.21.0.0/24 in reproducers networking-definition.yml.
It also adds a multus range for this network so that the
Tempest pod can access this network for testing. The NFS
network is added to the OCP nodes for the same reason.
The podified-multinode-hci-deployment-crc job is updated
to not deploy manila since it was never tested by tempest.

This patch updates playbook manila_create_default_resources.yml
so that when CI for manila runs, a provider network is created.
Variables manila_provider_network_{name,vlan,start,end,range}
default to the storage network, but can be overridden to the
NFS network within a CI job definition.

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417

Signed-off-by: John Fulton <[email protected]>
fmount pushed a commit to fultonj/architecture that referenced this pull request Oct 12, 2024
Manila Tempest tests need to connect to the share
for Ganesha via a special (openstack) network [1].

This patch adds the NFS storage network with VLAN
24 and range 172.21.0.0/24 in uni04delta. The NFS
network is connected to Ceph and Compute EDPM nodes.
A NNCP, NAD, L2Advertisement and IPAddressPool are
defined for the NFS network so that a pod in k8s can
connect to it; such as the tempest pod which will
perform the storage tests and the manilaShares pod(s).

[1] https://opendev.org/openstack/manila-tempest-plugin/src/branch/master/manila_tempest_tests/config.py#L99

Jira: https://issues.redhat.com/browse/OSPRH-7417
Depends-On: openstack-k8s-operators/ci-framework#2273

Signed-off-by: John Fulton <[email protected]>
Copy link

This PR is stale because it has been for over 15 days with no activity.
Remove stale label or comment or this will be closed in 7 days.

@fultonj
Copy link
Contributor Author

fultonj commented Oct 24, 2024

  • Putting this on hold for now
  • Planning to resume work on it during first week of November

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants