Load balancing doesn't work correctly in BYO VNET scenarios

[ohorvath]

Version

Karpenter Version: v0.7.0

Kubernetes Version: v1.3.0

If you use BYO VNET for your AKS cluster with Standard Public LB, the ingress traffic only hits the nodes of the system nodepool and not the karpenter nodes. In this scenario AKS automatically creates a NSG and attaches it to the network interfaces of the system nodes and not to the karpenter nodes. This behavior ultimately blocks the ingress traffic to reach karpenter nodes. If the externalTrafficPolicy is set to Cluster, the traffic can reach the nodes through an extra hop from the system nodes but never directly. However if you set the externalTrafficPolicy to Local, there is no way to forward ingress traffic to the karpenter nodes.

Expected Behavior

An NSG should be associated with the karpenter nodes also, not just with the system nodes to enable ingress traffic from the public LB.

Actual Behavior

See above, due to the missing NSG, traffic can't reach the karpenter nodes directly.

Steps to Reproduce the Problem

Create a cluster with karpenter.
Install NGINX ingress controller with externalTrafficPolicy set to local.
Force the NGINX pods to move to the karpenter nodes.
Create an ingress and a backend app.
Try to reach the public LB externally.

Resource Specs and Logs

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment