Add more detail to the EKS workaround (#15785)

* Add more detail to the EKS workaround

Signed-off-by: Benjamin Leggett <[email protected]>

* Fixup

Signed-off-by: Benjamin Leggett <[email protected]>

* Fixup more

Signed-off-by: Benjamin Leggett <[email protected]>

* Update content/en/docs/ambient/install/platform-prerequisites/index.md

Co-authored-by: Craig Box <[email protected]>

* Update content/en/docs/ambient/install/platform-prerequisites/index.md

Co-authored-by: Craig Box <[email protected]>

* okay look, strictly speaking based on how the alphabet works the linter is correct here

Signed-off-by: Benjamin Leggett <[email protected]>

* Update content/en/docs/ambient/install/platform-prerequisites/index.md

Co-authored-by: Daniel Hawton <[email protected]>

---------

Signed-off-by: Benjamin Leggett <[email protected]>
Co-authored-by: Craig Box <[email protected]>
Co-authored-by: Daniel Hawton <[email protected]>

Author: 3 people

.spelling

@@ -1055,6 +1055,8 @@ sds
 Searcy
 Secura
 SecurityCon
+SecurityGroupPolicy
+SecurityGroups
 security_bulletin
 selinux
 serverless

content/en/docs/ambient/install/platform-prerequisites/index.md

@@ -41,7 +41,31 @@ spec:
 
 ### Amazon Elastic Kubernetes Service (EKS)
 
-If you are using EKS with Amazon's VPC CNI **enabled**, [`POD_SECURITY_GROUP_ENFORCING_MODE` must be explicitly set to `standard`](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/README.md#pod_security_group_enforcing_mode-v1110), or pod health probes (which are by-default silently exempted from all policy enforcement by AWS VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which AWS VPC CNI is not aware of, and AWS VPC CNI does not have an option to exempt link-local addresses from policy enforcement.
+If you are using EKS:
+
+- with Amazon's VPC CNI
+- with Pod ENI trunking enabled
+- **and** you are using EKS pod-attached SecurityGroups via [SecurityGroupPolicy](https://aws.github.io/aws-eks-best-practices/networking/sgpp/#enforcing-mode-use-strict-mode-for-isolating-pod-and-node-traffic)
+
+[`POD_SECURITY_GROUP_ENFORCING_MODE` must be explicitly set to `standard`](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/README.md#pod_security_group_enforcing_mode-v1110), or pod health probes (which are by-default silently exempted from all policy enforcement by the VPC CNI) will fail. This is because Istio uses a link-local SNAT address for kubelet health probes, which Amazon's VPC CNI is not aware of, and the VPC CNI does not have an option to exempt link-local addresses from policy enforcement.
+
+You can check if you have pod ENI trunking enabled by running the following command:
+
+{{< text syntax=bash >}}
+$ kubectl set env daemonset aws-node -n kube-system --list | grep ENABLE_POD_ENI
+{{< /text >}}
+
+You can check if you have any pod-attached security groups in your cluster by running the following command:
+
+{{< text syntax=bash >}}
+$ kubectl get securitygrouppolicies.vpcresources.k8s.aws
+{{< /text >}}
+
+You can set `POD_SECURITY_GROUP_ENFORCING_MODE=standard` by running the following command, and recycling affected pods:
+
+{{< text syntax=bash >}}
+$ kubectl set env daemonset aws-node -n kube-system POD_SECURITY_GROUP_ENFORCING_MODE=standard
+{{< /text >}}
 
 ### k3d