Unrelated ingresses targeting the same service/port: annotations for session stickyness not respected

[mitchese]

We have two ingresses on separate hostnames which target the same service and port. When we annotated the second ingress with
nginx.ingress.kubernetes.io/upstream-hash-by: $remote_addr

this was not added to the nginx configuration and session stickyness did not work. We were able to trace it down to the list provided by

kubectl ingress-nginx backends

which would show that the upstream hash

         "upstreamHashByConfig": {
          "upstream-hash-by-subset-size": 3
     }, 

for both ingresses because it is in a section labeled by the namespace-service-port triplicate.

NGINX Ingress controller version 1.10.

Kubernetes version v1.29.9

Environment:

• Cloud provider or hardware configuration: onprem bare metal
• OS (e.g. from /etc/os-release): flatcar (various versions 3975.2.2)
• Kernel (e.g. uname -a): 6.6.54-flatcar

How to reproduce this issue:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/upstream-hash-by: $remote_addr
  name: a-public-ingress-sticky
spec:
  ingressClassName: nginx
  rules:
  - host: sticky.example.com
      http:
      paths:
      - backend:
          service:
            name: echo-server
            port:
              name: test
        path: /
        pathType: Prefix
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: private-ingress
spec:
  ingressClassName: nginx
  rules:
  - host: service.kubernetes.internal
      http:
      paths:
      - backend:
          service:
            name: echo-server
            port:
              name: test
        path: /
        pathType: Prefix

It is important that both ingresses in the above example are in the same namespace, and both point to the same service and same port, and that the first is alphabetically before the second.

The session stickyness annotation will not be applied and can be verified with
kubectl ingress-nginx backends

and looking for the presence of "upstream-hash-by": "$remote_addr", in the output.

Because these are two separate hostnames I would expect the ingress controller to handle them separately, and that annotations on the seemingly unrelated ingress

I can follow up later with a minikube example

[k8s-ci-robot]

This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[longwuyuan]

/assign

[longwuyuan]

/remove-kind bug

• Please look at the questions asked in the template of a new bug report by clicking the new bug report button
• Edit the description of this issue and answer the questions asked in the template, here in this issue
• Make sure to use markdown format as shown in the template
• Make sure to answer all questions so there is data for analysis

Right now, its not even known if you are using the controller released by this project. Or if so, then which version

An attempt to reproduce with 5 replicas of a pod using --image nginx:alpine and even one single ingress with that annotations fails to show the hash directive under the related server block.

/triage needs-information

[longwuyuan]

/kind support

[longwuyuan]

likely related to this https://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash