Add docs for enabling Istio native nftables feature #1122
Conversation
Signed-off-by: Yuanlin Xu <[email protected]>
Signed-off-by: Yuanlin Xu <[email protected]>
Signed-off-by: Yuanlin Xu <[email protected]>
yxun
changed the title
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1122 +/- ##
==========================================
+ Coverage 77.78% 78.75% +0.96%
==========================================
Files 44 44
Lines 2823 2255 -568
==========================================
- Hits 2196 1776 -420
+ Misses 521 369 -152
- Partials 106 110 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
We don't have a |
| @@ -0,0 +1,13 @@ | |||
| apiVersion: sailoperator.io/v1 | |||
There was a problem hiding this comment.
While this looks good. In the longer run, we should use templates and update it based on the args.
docs/common/istio-nftables.md
Outdated
|
|
||
| This feature configures Istio to use the `nftables` backend instead of `iptables` for traffic redirection. | ||
|
|
||
| #### Installation on OpenShift |
There was a problem hiding this comment.
Regarding the installation on OCP for Sail Operator, I feel we should link to the existing documentation rather than duplicating the steps here once again. If the steps require some explicit config that can be added as a note.
Signed-off-by: Yuanlin Xu <[email protected]>
yxun
force-pushed
the
nft-doc
branch
2 times, most recently
from
11eb9e9 to
79a1d1b
Compare
Signed-off-by: Yuanlin Xu <[email protected]>
docs/common/istio-nftables.md
Outdated
|
|
||
| ### Installation | ||
|
|
||
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio release-1.27 [release note](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, |
There was a problem hiding this comment.
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio release-1.27 [release note](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, | |
| The support for native nftables when using Istio sidecar mode was implemented in the upstream istio [release-1.27](https://github.com/istio/istio/blob/master/releasenotes/notes/nftables-sidecar.yaml). It is disabled by default. To enable it, you can set a feature flag as `values.global.nativeNftables=true`. For example, |
docs/common/istio-nftables.md
Outdated
| Installation with Helm | ||
|
|
||
| ```sh | ||
| helm install istiod-canary istio/istiod \ |
There was a problem hiding this comment.
Since we are talking about nftables, we can avoid using "istiod-canary"
| helm install istiod-canary istio/istiod \ | |
| helm install istiod istio/istiod \ |
docs/common/istio-nftables.md
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.28-alpha.24646157 |
There was a problem hiding this comment.
We can use 1.27 (or whatever is the latest supported version) here.
docs/common/istio-nftables.md
Outdated
| updateStrategy: | ||
| type: InPlace | ||
| inactiveRevisionDeletionGracePeriodSeconds: 30 |
There was a problem hiding this comment.
Since the updateStrategy section is anyways using the defaults, we can avoid specifying it in the yamls.
docs/common/istio-nftables.md
Outdated
| ```sh | ||
| kubectl apply -f - <<EOF | ||
| apiVersion: operators.coreos.com/v1alpha1 | ||
| kind: Subscription | ||
| metadata: | ||
| name: sailoperator | ||
| namespace: openshift-operators | ||
| spec: | ||
| channel: "1.27-nightly" | ||
| installPlanApproval: Automatic | ||
| name: sailoperator | ||
| source: community-operators | ||
| sourceNamespace: openshift-marketplace | ||
| EOF | ||
| ``` |
There was a problem hiding this comment.
As this document is for Sail Operator, lets avoid any OpenShift specific notes. Please use appropriate mechanism for Kind cluster.
docs/common/istio-nftables.md
Outdated
| inactiveRevisionDeletionGracePeriodSeconds: 30 | ||
| values: | ||
| global: | ||
| nativeNftables: true |
There was a problem hiding this comment.
Actually you can remove this whole section since we are capturing the same info below.
docs/common/istio-nftables.md
Outdated
| ``` | ||
|
|
||
| #### Install in Ambient Mode | ||
|
|
There was a problem hiding this comment.
yes, the Ambient mode part install and upgrade is in progress. When that part is clear, I will update the upgrade task JIRA and this PR draft.
docs/common/istio-nftables.md
Outdated
| kubectl apply -n test-ns -f https://raw.githubusercontent.com/istio/istio/refs/heads/master/samples/curl/curl.yaml | ||
| ``` | ||
|
|
||
| Attach a debug container and you will get the following rules in the `istio-proxy` container: |
There was a problem hiding this comment.
| Attach a debug container and you will get the following rules in the `istio-proxy` container: | |
| Attach a debug container and you can see the nftable rules from the `istio-proxy` container: |
docs/common/istio-nftables.md
Outdated
|
|
||
| ### Upgrade | ||
|
|
||
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. |
There was a problem hiding this comment.
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. | |
| The migration from iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application in test-ns namespace. |
docs/common/istio-nftables.md
Outdated
| The migration of using the existing Istio iptables backend to nftables backend can be done by upgrading Istio. The following example installs an Istio control plane with the iptables backend and a sample application curl in a data plane namespace test-ns. | ||
|
|
||
| ```sh | ||
| istioctl install -y |
There was a problem hiding this comment.
As this is a Sail Operator doc, lets use the installation mechanisms supported in Sail Operator.
Signed-off-by: Yuanlin Xu <[email protected]>
|
This PR is still WIP. I tried the Sail Operator 1.27.1 but still getting the following error IstioCNI default violates policy 299 - "unknown field "spec.values.global.nativeNftables"" The api field PR was merged in #1183 |
yxun
changed the title
|
The latest Sail Operator (1.27.1) log shows: Unfortunately , the api change PR was one commit behind that : We may need to bump Sail Operator to a newer 1.27 version and then I can verify the nftables feature using Sail Operator. |
Signed-off-by: Yuanlin Xu <[email protected]>
docs/common/istio-nftables.md
Outdated
| @@ -0,0 +1,277 @@ | |||
| [Return to Project Root](../README.md) | |||
There was a problem hiding this comment.
Hey, can you please convert these files to ASCII doc? We now have all the docs under the docs folder in ASCII format
There was a problem hiding this comment.
yes, I pushed another commit about adding a ascii one. I'm checking. If the conversion looks correct , I will remove the markdown file in another commit after review.
Signed-off-by: Yuanlin Xu <[email protected]>
yxun
changed the title
Signed-off-by: Yuanlin Xu <[email protected]>
Signed-off-by: Yuanlin Xu <[email protected]>
Signed-off-by: Yuanlin Xu <[email protected]>
|
Here are two findings : The doc PR is ready for review. Thanks. |
| updateStrategy: | ||
| type: InPlace | ||
| inactiveRevisionDeletionGracePeriodSeconds: 30 |
There was a problem hiding this comment.
Lets remove these fields as they are anyways default.
docs/common/istio-nftables.adoc
Outdated
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | ||
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. |
There was a problem hiding this comment.
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | |
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. | |
| When installing `Istio` and `IstioCNI` resources with the Sail Operator, you can enable nftables by setting `spec.values.global.nativeNftables=true` in the resource. This option configures Istio to use the nftables backend for traffic redirection instead of iptables. |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| To enable the native nftables feature, using the following steps: | ||
|
|
||
| [arabic] |
There was a problem hiding this comment.
Not sure what this is for. Can you eloborate?
There was a problem hiding this comment.
That was automatically generated when I run make gen-api-docs . I can remove that asciidoc format mark.
docs/common/istio-nftables.adoc
Outdated
| When you install IstioCNI and Istio resources with Sail Operator, you can create an instance of them with `+spec.values.global.nativeNftables=true+`. | ||
| This feature configures Istio to use the `+nftables+` backend instead of `+iptables+` for traffic redirection. | ||
|
|
||
| To enable the native nftables feature, using the following steps: |
There was a problem hiding this comment.
We can remove this line as we already mentioned above how to enable nftables.
docs/common/istio-nftables.adoc
Outdated
| metadata: | ||
| name: default | ||
| spec: | ||
| version: v1.27.2 |
There was a problem hiding this comment.
This should be updated to version 1.28
docs/common/istio-nftables.adoc
Outdated
| ==== Validation | ||
|
|
||
| When using the `+nftables+` backend, you can verify the traffic redirection rules using the `+nft list ruleset+` command in a data plane application or sidecar container. | ||
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a data plane namespace `+test-ns+`. |
There was a problem hiding this comment.
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a data plane namespace `+test-ns+`. | |
| You can find all rules are in the `+inet+` table. The following example installs a sample application `+curl+` in a namespace `+test-ns+` that is part of the mesh. |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| When using Sail Operator, the upgraded `IstioCNI` and `Istio` resources `spec.version` value need to be different from the prior instances. Otherwise, the appended `spec.values` will not be configured. | ||
|
|
||
| ===== Upgrade with Sail Operator on OpenShift |
There was a problem hiding this comment.
| ===== Upgrade with Sail Operator on OpenShift | |
| ===== Upgrade using Sail Operator |
docs/common/istio-nftables.adoc
Outdated
|
|
||
| ===== Upgrade with Sail Operator on OpenShift | ||
|
|
||
| To upgrade an iptable based Istio service mesh, using the following steps: |
There was a problem hiding this comment.
| To upgrade an iptable based Istio service mesh, using the following steps: | |
| To upgrade an iptable based Istio service mesh with nftables backend, use the following steps: |
docs/common/istio-nftables.adoc
Outdated
| . Upgrade the IstioCNI and Istio control plane with `+spec.values.global.nativeNftables=true+`. | ||
| More details about the Update Strategy are described in link:../update-strategy/update-strategy.adoc[update-strategy.adoc]. For example, | ||
|
|
||
| [source,sh] |
There was a problem hiding this comment.
The whole of the section is redundant, we are just repeating things again. Please re-work on the section below. We dont have to show the Yamls again, just include a note saying that spec.values.global.nativeNftables should be set to true in Istio and IstioCNI CRs.
|
|
||
| [source,sh] | ||
| ---- | ||
| kubectl rollout restart deployment -n test-ns |
There was a problem hiding this comment.
hmm... we should find a way to support re-programming the new rules without having to restart the pods. Did you take a look at the code to understand what it does when config changes and the rules do not match?
Are you saying that when we update any spec.values (without modifying the version) the updated values are not reflected in the Istio deployment? |
Signed-off-by: Yuanlin Xu <[email protected]>
|
Signed-off-by: Yuanlin Xu <[email protected]>
|
@yxun: The following test failed, say
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
When I run the latest Sail operator 1.27.3+ , I see Istio values overrides behavior works correctly in this upgrade case. My Quote finding was not correct. It could be caused by running an old Istio version which had not included nftables change there. |
Still work in progress about the re-programming rule changes without having to restart the application pods. |
5 participants
This PR adds new doc and samples about the Istio native nftables feature. That feature was merged in the upstream release-1.27. This doc outlines the configuration steps using Sail Operator.
Related issue: #1123