Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load Balancer Service Tests with MetalLB [It] Should ensure load balancer service works with 0 node ports when ETP=local #4394

Open
flavio-fernandes opened this issue May 24, 2024 · 0 comments
Open

Load Balancer Service Tests with MetalLB [It] Should ensure load balancer service works with 0 node ports when ETP=local #4394

flavio-fernandes opened this issue May 24, 2024 · 0 comments
Labels
area/e2e-testing kind/ci-flake Flakes seen in CI

Comments

@flavio-fernandes
Copy link
Contributor

Which jobs are flaking?

Flake: Load Balancer Service Tests with MetalLB [It] Should ensure load balancer service works with 0 node ports when ETP=local

e2e (control-plane, noHA, local, ipv4, noSnatGW, 1br, ic-single-node-zones)

https://github.com/ovn-org/ovn-kubernetes/actions/runs/9225716773/job/25384945148?pr=4393

Which tests are flaking?

Summarizing 1 Failure:
[FAIL] Load Balancer Service Tests with MetalLB [It] Should ensure load balancer service works with 0 node ports when ETP=local
/home/runner/work/ovn-kubernetes/ovn-kubernetes/test/e2e/service.go:1812

Ran 103 of 278 Specs in 3604.401 seconds
FAIL! -- 102 Passed | 1 Failed | 1 Flaked | 2 Pending | 173 Skipped
--- FAIL: TestE2E (3604.46s)
FAIL
FAIL github.com/ovn-org/ovn-kubernetes/test/e2e 3604.553s
FAIL
make: *** [Makefile:32: control-plane] Error 1

Since when has it been flaking?

¯\_(ツ)_/¯

Reason for failure (if possible)

No response

Anything else we need to know?

Load Balancer Service Tests with MetalLB Should ensure load balancer service works with 0 node ports when ETP=local
/home/runner/work/ovn-kubernetes/ovn-kubernetes/test/e2e/service.go:1786
  STEP: Creating a kubernetes client @ 05/24/24 16:10:41.819
  May 24 16:10:41.819: INFO: >>> kubeConfig: /home/runner/ovn.conf
  STEP: Building a namespace api object, basename lbservice-test @ 05/24/24 16:10:41.82
  May 24 16:10:41.825: INFO: Skipping waiting for service account
  May 24 16:10:41.834: INFO: Create the Load Balancer configuration
  May 24 16:10:41.834: INFO: Running '/usr/local/bin/kubectl --server=https://127.0.0.1:38665/ --kubeconfig=/home/runner/ovn.conf --namespace=default create -f loadbalancer.yaml'
  May 24 16:10:42.099: INFO: stderr: ""
  May 24 16:10:42.099: INFO: stdout: "persistentvolumeclaim/dynamic-claim created\ndeployment.apps/lb-backend-pod created\nservice/lbservice-test created\n"
  May 24 16:10:43.100: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:43.102: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:44.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:44.101: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:45.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:45.102: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:46.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:46.102: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:47.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:47.102: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:48.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:48.102: INFO: Unexpected number of Endpoints, got 0, expected 4
  May 24 16:10:49.099: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:49.102: INFO: Unexpected number of Endpoints, got 3, expected 4
  May 24 16:10:50.100: INFO: Waiting for amount of service:lbservice-test endpoints to be 4
  May 24 16:10:55.204: INFO: DEBUG: Dumping IPTRules # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *mangle
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-IPTABLES-HINT - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  :OVN-KUBE-ITP - [0:0]
  [104:10248] -A PREROUTING -m mark --mark 0x3f0 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
  [187002:873477635] -A PREROUTING -m mark --mark 0x0 -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
  [164715:345314151] -A OUTPUT -j OVN-KUBE-ITP
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *filter
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :KUBE-FIREWALL - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  [167:12019] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [9:518] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [37:2779] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [26:1895] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [8:466] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [9:518] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [21:1531] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [1:60] -A INPUT -p tcp -m tcp --dport 9107 -j DROP
  [53:3902] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [13:1837] -A INPUT -p tcp -m tcp --dport 9107 -j DROP
  [7:414] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [12:855] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [13:907] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [15:1114] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [18:1271] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [16:1167] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [4350:5416069] -A INPUT -i ovn-k8s-mp0 -m comment --comment "from OVN to localhost" -j ACCEPT
  [162749:652401363] -A INPUT -j KUBE-FIREWALL
  [3253:387157] -A FORWARD -i ovn-k8s-mp0 -j ACCEPT
  [2792:1660132] -A FORWARD -o ovn-k8s-mp0 -j ACCEPT
  [165380:345410366] -A OUTPUT -j KUBE-FIREWALL
  [0:0] -A KUBE-FIREWALL ! -s 127.0.0.0/8 -d 127.0.0.0/8 -m comment --comment "block incoming localnet connections" -m conntrack ! --ctstate RELATED,ESTABLISHED,DNAT -j DROP
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *nat
  :PREROUTING ACCEPT [1328:119598]
  :INPUT ACCEPT [881:87503]
  :OUTPUT ACCEPT [3120:208270]
  :POSTROUTING ACCEPT [2863:199461]
  :DOCKER_OUTPUT - [0:0]
  :DOCKER_POSTROUTING - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  :OVN-KUBE-EGRESS-IP-MULTI-NIC - [0:0]
  :OVN-KUBE-EGRESS-SVC - [0:0]
  :OVN-KUBE-ETP - [0:0]
  :OVN-KUBE-EXTERNALIP - [0:0]
  :OVN-KUBE-ITP - [0:0]
  :OVN-KUBE-NODEPORT - [0:0]
  :OVN-KUBE-SNAT-MGMTPORT - [0:0]
  [1393:120276] -A PREROUTING -j OVN-KUBE-ETP
  [1381:119700] -A PREROUTING -j OVN-KUBE-EXTERNALIP
  [1356:118344] -A PREROUTING -j OVN-KUBE-NODEPORT
  [93:5580] -A PREROUTING -d 172.18.0.1/32 -j DOCKER_OUTPUT
  [3347:225673] -A OUTPUT -j OVN-KUBE-EXTERNALIP
  [3347:225673] -A OUTPUT -j OVN-KUBE-NODEPORT
  [3347:225673] -A OUTPUT -j OVN-KUBE-ITP
  [304:22639] -A OUTPUT -d 172.18.0.1/32 -j DOCKER_OUTPUT
  [3793:260216] -A POSTROUTING -j OVN-KUBE-EGRESS-IP-MULTI-NIC
  [3777:259256] -A POSTROUTING -j OVN-KUBE-EGRESS-SVC
  [640:38256] -A POSTROUTING -o ovn-k8s-mp0 -j OVN-KUBE-SNAT-MGMTPORT
  [93:5580] -A POSTROUTING -d 172.18.0.1/32 -j DOCKER_POSTROUTING
  [18:2501] -A POSTROUTING -s 169.254.169.1/32 -j MASQUERADE
  [300:22150] -A POSTROUTING -s 10.244.2.0/24 -j MASQUERADE
  [0:0] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 127.0.0.11:42531
  [304:22639] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 127.0.0.11:48604
  [0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p tcp -m tcp --sport 42531 -j SNAT --to-source 172.18.0.1:53
  [0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 48604 -j SNAT --to-source 172.18.0.1:53
  [0:0] -A OVN-KUBE-EGRESS-SVC -m mark --mark 0x3f0 -m comment --comment DoNotSNAT -j RETURN
  [0:0] -A OVN-KUBE-EXTERNALIP -d 192.168.10.0/32 -p udp -m udp --dport 10001 -j DNAT --to-destination 10.96.26.208:10001
  [0:0] -A OVN-KUBE-EXTERNALIP -d 192.168.10.0/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.96.26.208:80
  [0:0] -A OVN-KUBE-NODEPORT -p udp -m addrtype --dst-type LOCAL -m udp --dport 30200 -j DNAT --to-destination 10.96.26.208:10001
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 31996 -j DNAT --to-destination 10.96.26.208:80
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 30686 -j DNAT --to-destination 10.96.110.45:443
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 30798 -j DNAT --to-destination 10.96.110.45:80
  [0:0] -A OVN-KUBE-SNAT-MGMTPORT -o ovn-k8s-mp0 -m comment --comment "OVN SNAT to Management Port" -j SNAT --to-source 10.244.2.2
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *mangle
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-IPTABLES-HINT - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *filter
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *nat
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:10:55 2024

  May 24 16:10:55.204: INFO: DEBUG: Matched rule :OVN-KUBE-ETP - [0:0] for pattern OVN-KUBE-ETP
  May 24 16:10:55.204: INFO: DEBUG: Matched rule [1393:120276] -A PREROUTING -j OVN-KUBE-ETP for pattern OVN-KUBE-ETP
  May 24 16:10:55.311: INFO: DEBUG: Dumping IPTRules # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *mangle
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-IPTABLES-HINT - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  :OVN-KUBE-ITP - [0:0]
  [104:10248] -A PREROUTING -m mark --mark 0x3f0 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
  [187002:873477635] -A PREROUTING -m mark --mark 0x0 -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
  [164715:345314151] -A OUTPUT -j OVN-KUBE-ITP
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *filter
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :KUBE-FIREWALL - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  [167:12019] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [9:518] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [37:2779] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [26:1895] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [8:466] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [9:518] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [21:1531] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [1:60] -A INPUT -p tcp -m tcp --dport 9107 -j DROP
  [53:3902] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [13:1837] -A INPUT -p tcp -m tcp --dport 9107 -j DROP
  [7:414] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [2:104] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [12:855] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [13:907] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [15:1114] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [18:1271] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [0:0] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [16:1167] -A INPUT -p tcp -m tcp --dport 9107 -j ACCEPT
  [4350:5416069] -A INPUT -i ovn-k8s-mp0 -m comment --comment "from OVN to localhost" -j ACCEPT
  [162749:652401363] -A INPUT -j KUBE-FIREWALL
  [3253:387157] -A FORWARD -i ovn-k8s-mp0 -j ACCEPT
  [2792:1660132] -A FORWARD -o ovn-k8s-mp0 -j ACCEPT
  [165380:345410366] -A OUTPUT -j KUBE-FIREWALL
  [0:0] -A KUBE-FIREWALL ! -s 127.0.0.0/8 -d 127.0.0.0/8 -m comment --comment "block incoming localnet connections" -m conntrack ! --ctstate RELATED,ESTABLISHED,DNAT -j DROP
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by iptables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *nat
  :PREROUTING ACCEPT [1328:119598]
  :INPUT ACCEPT [881:87503]
  :OUTPUT ACCEPT [3120:208270]
  :POSTROUTING ACCEPT [2863:199461]
  :DOCKER_OUTPUT - [0:0]
  :DOCKER_POSTROUTING - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  :OVN-KUBE-EGRESS-IP-MULTI-NIC - [0:0]
  :OVN-KUBE-EGRESS-SVC - [0:0]
  :OVN-KUBE-ETP - [0:0]
  :OVN-KUBE-EXTERNALIP - [0:0]
  :OVN-KUBE-ITP - [0:0]
  :OVN-KUBE-NODEPORT - [0:0]
  :OVN-KUBE-SNAT-MGMTPORT - [0:0]
  [1393:120276] -A PREROUTING -j OVN-KUBE-ETP
  [1381:119700] -A PREROUTING -j OVN-KUBE-EXTERNALIP
  [1356:118344] -A PREROUTING -j OVN-KUBE-NODEPORT
  [93:5580] -A PREROUTING -d 172.18.0.1/32 -j DOCKER_OUTPUT
  [3347:225673] -A OUTPUT -j OVN-KUBE-EXTERNALIP
  [3347:225673] -A OUTPUT -j OVN-KUBE-NODEPORT
  [3347:225673] -A OUTPUT -j OVN-KUBE-ITP
  [304:22639] -A OUTPUT -d 172.18.0.1/32 -j DOCKER_OUTPUT
  [3793:260216] -A POSTROUTING -j OVN-KUBE-EGRESS-IP-MULTI-NIC
  [3777:259256] -A POSTROUTING -j OVN-KUBE-EGRESS-SVC
  [640:38256] -A POSTROUTING -o ovn-k8s-mp0 -j OVN-KUBE-SNAT-MGMTPORT
  [93:5580] -A POSTROUTING -d 172.18.0.1/32 -j DOCKER_POSTROUTING
  [18:2501] -A POSTROUTING -s 169.254.169.1/32 -j MASQUERADE
  [300:22150] -A POSTROUTING -s 10.244.2.0/24 -j MASQUERADE
  [0:0] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 127.0.0.11:42531
  [304:22639] -A DOCKER_OUTPUT -d 172.18.0.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 127.0.0.11:48604
  [0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p tcp -m tcp --sport 42531 -j SNAT --to-source 172.18.0.1:53
  [0:0] -A DOCKER_POSTROUTING -s 127.0.0.11/32 -p udp -m udp --sport 48604 -j SNAT --to-source 172.18.0.1:53
  [0:0] -A OVN-KUBE-EGRESS-SVC -m mark --mark 0x3f0 -m comment --comment DoNotSNAT -j RETURN
  [0:0] -A OVN-KUBE-EXTERNALIP -d 192.168.10.0/32 -p udp -m udp --dport 10001 -j DNAT --to-destination 10.96.26.208:10001
  [0:0] -A OVN-KUBE-EXTERNALIP -d 192.168.10.0/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.96.26.208:80
  [0:0] -A OVN-KUBE-NODEPORT -p udp -m addrtype --dst-type LOCAL -m udp --dport 30200 -j DNAT --to-destination 10.96.26.208:10001
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 31996 -j DNAT --to-destination 10.96.26.208:80
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 30686 -j DNAT --to-destination 10.96.110.45:443
  [0:0] -A OVN-KUBE-NODEPORT -p tcp -m addrtype --dst-type LOCAL -m tcp --dport 30798 -j DNAT --to-destination 10.96.110.45:80
  [0:0] -A OVN-KUBE-SNAT-MGMTPORT -o ovn-k8s-mp0 -m comment --comment "OVN SNAT to Management Port" -j SNAT --to-source 10.244.2.2
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *mangle
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-IPTABLES-HINT - [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:10:55 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:10:55 2024
  *filter
  :INPUT ACCEPT [0:0]
  :FORWARD ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
...
  May 24 16:11:46.052: INFO: stderr: ""
  May 24 16:11:46.052: INFO: stdout: "bgpadvertisement.metallb.io/empty unchanged\n"
  STEP: dump namespace information after failure @ 05/24/24 16:11:46.052
  STEP: Collecting events from namespace "lbservice-test-7440". @ 05/24/24 16:11:46.052
  STEP: Found 0 events. @ 05/24/24 16:11:46.054
  May 24 16:11:46.056: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
  May 24 16:11:46.056: INFO: 
  May 24 16:11:46.059: INFO: 
  Logging node info for node ovn-control-plane
  May 24 16:11:46.062: INFO: Node Info: &Node{ObjectMeta:{ovn-control-plane    e124ba0c-2763-40e1-93cf-792ff497dc59 13346 0 2024-05-24 15:06:55 +0000 UTC <nil> <nil> map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/os:linux ingress-ready:true k8s.ovn.org/ovnkube-db:true k8s.ovn.org/zone-name:ovn-control-plane kubernetes.io/arch:amd64 kubernetes.io/hostname:ovn-control-plane kubernetes.io/os:linux node-role.kubernetes.io/control-plane:] map[k8s.ovn.org/host-cidrs:["172.18.0.2/16"] k8s.ovn.org/hybrid-overlay-distributed-router-gateway-ip:10.244.1.3 k8s.ovn.org/hybrid-overlay-distributed-router-gateway-mac:0a:58:0a:f4:01:03 k8s.ovn.org/l3-gateway-config:{"default":{"mode":"local","interface-id":"breth0_ovn-control-plane","mac-address":"02:42:ac:12:00:02","ip-addresses":["172.18.0.2/16"],"ip-address":"172.18.0.2/16","next-hops":["172.18.0.1"],"next-hop":"172.18.0.1","node-port-enable":"true","vlan-id":"0"}} k8s.ovn.org/network-ids:{"default":"0"} k8s.ovn.org/node-chassis-id:2bb33311-b3dd-461f-932e-bf807248b7fb k8s.ovn.org/node-gateway-router-lrp-ifaddr:{"ipv4":"100.64.0.2/16"} k8s.ovn.org/node-id:2 k8s.ovn.org/node-mgmt-port-mac-address:de:e2:a2:47:d6:50 k8s.ovn.org/node-primary-ifaddr:{"ipv4":"172.18.0.2/16"} k8s.ovn.org/node-subnets:{"default":["10.244.1.0/24"]} k8s.ovn.org/node-transit-switch-port-ifaddr:{"ipv4":"100.88.0.2/16"} k8s.ovn.org/remote-zone-migrated:ovn-control-plane k8s.ovn.org/secondary-host-egress-ips:[] k8s.ovn.org/zone-name:ovn-control-plane kubeadm.alpha.kubernetes.io/cri-socket:unix:///run/containerd/containerd.sock node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2024-05-24 15:06:55 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:ingress-ready":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{}}},"f:spec":{"f:providerID":{}}} } {kubeadm Update v1 2024-05-24 15:06:57 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:kubeadm.alpha.kubernetes.io/cri-socket":{}},"f:labels":{"f:node-role.kubernetes.io/control-plane":{}}}} } {kubectl-label Update v1 2024-05-24 15:07:54 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:k8s.ovn.org/ovnkube-db":{},"f:k8s.ovn.org/zone-name":{}}}} } {ovn-control-plane Update v1 2024-05-24 15:45:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:k8s.ovn.org/host-cidrs":{},"f:k8s.ovn.org/hybrid-overlay-distributed-router-gateway-ip":{},"f:k8s.ovn.org/hybrid-overlay-distributed-router-gateway-mac":{},"f:k8s.ovn.org/l3-gateway-config":{},"f:k8s.ovn.org/network-ids":{},"f:k8s.ovn.org/node-chassis-id":{},"f:k8s.ovn.org/node-gateway-router-lrp-ifaddr":{},"f:k8s.ovn.org/node-id":{},"f:k8s.ovn.org/node-mgmt-port-mac-address":{},"f:k8s.ovn.org/node-primary-ifaddr":{},"f:k8s.ovn.org/node-subnets":{},"f:k8s.ovn.org/node-transit-switch-port-ifaddr":{},"f:k8s.ovn.org/remote-zone-migrated":{},"f:k8s.ovn.org/secondary-host-egress-ips":{},"f:k8s.ovn.org/zone-name":{}}}} status} {kube-controller-manager Update v1 2024-05-24 15:57:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}},"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.0.0/24\"":{}}}} } {kubelet Update v1 2024-05-24 16:07:20 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.0.0/24,DoNotUseExternalID:,ProviderID:kind://docker/ovn/ovn-control-plane,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.0.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{4 0} {<nil>} 4 DecimalSI},ephemeral-storage: {{77851254784 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{16757354496 0} {<nil>} 16364604Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{4 0} {<nil>} 4 DecimalSI},ephemeral-storage: {{77851254784 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{16757354496 0} {<nil>} 16364604Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2024-05-24 16:07:20 +0000 UTC,LastTransitionTime:2024-05-24 15:06:52 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2024-05-24 16:07:20 +0000 UTC,LastTransitionTime:2024-05-24 15:06:52 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2024-05-24 16:07:20 +0000 UTC,LastTransitionTime:2024-05-24 15:06:52 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2024-05-24 16:07:20 +0000 UTC,LastTransitionTime:2024-05-24 15:57:06 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:172.18.0.2,},NodeAddress{Type:Hostname,Address:ovn-control-plane,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:f369fce65f8e44ddb0ef58e75304dc89,SystemUUID:14e38e09-95f1-4d9b-ba0b-e244e607b49e,BootID:ae3aae5c-2a79-428d-9801-15f6f1f7bcb1,KernelVersion:6.5.0-1021-azure,OSImage:Debian GNU/Linux 12 (bookworm),ContainerRuntimeVersion:containerd://1.7.13,KubeletVersion:v1.29.2,KubeProxyVersion:v1.29.2,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[docker.io/library/import-2024-05-24@sha256:a97c371426158bd2b94346d38c55c8a3631ca90ba06b26d89163c3203eb9a6bd docker.io/library/ovn-daemonset-f:pr],SizeBytes:862713915,},ContainerImage{Names:[docker.io/library/import-2024-05-24@sha256:4d06a01e319ab4a33b4bd67afeb70d71c68bca7a252517cd960791d5fcb0c8cb quay.io/metallb/speaker:dev-amd64],SizeBytes:119201985,},ContainerImage{Names:[docker.io/library/import-2024-02-14@sha256:9dc4f70aac00562ca2cf9e91f3738b24b9003a65632caa4594ead32a0c61acb5 registry.k8s.io/kube-apiserver-amd64:v1.29.2 registry.k8s.io/kube-apiserver:v1.29.2],SizeBytes:86153187,},ContainerImage{Names:[docker.io/library/import-2024-02-14@sha256:b58ead7037c74ab4985dd33c9cd341089c509dc2aa233da4724ecdef9bddcfb9 registry.k8s.io/kube-proxy-amd64:v1.29.2 registry.k8s.io/kube-proxy:v1.29.2],SizeBytes:83531977,},ContainerImage{Names:[docker.io/library/import-2024-02-14@sha256:de4f85102afe375b2b6b6a5decd3f3e8653639ebd3ed19003366a469cdec45a6 registry.k8s.io/kube-controller-manager-amd64:v1.29.2 registry.k8s.io/kube-controller-manager:v1.29.2],SizeBytes:80282953,},ContainerImage{Names:[quay.io/frrouting/frr@sha256:086acb1278fe86118345f456a1fbfafb80c34d03f7bca9137da0729a1aee5e9c quay.io/frrouting/frr:9.0.2],SizeBytes:79403819,},ContainerImage{Names:[docker.io/library/import-2024-05-24@sha256:a9b0eedc64c18e28134f446ed16c30c7474be4fde918b10b255321a10fed14db quay.io/metallb/controller:dev-amd64],SizeBytes:67194325,},ContainerImage{Names:[docker.io/library/import-2024-02-14@sha256:55f6e3c4013d0bf33a057922a72ab12120071da9fe3b356dabbfc2892a2b4325 registry.k8s.io/kube-scheduler-amd64:v1.29.2 registry.k8s.io/kube-scheduler:v1.29.2],SizeBytes:60614985,},ContainerImage{Names:[registry.k8s.io/etcd:3.5.10-0],SizeBytes:56649232,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:2c5b5b056076334e4cf431d964d102e44cbca8f1e6b16ac1e477a0ffbe6caac4 registry.k8s.io/e2e-test-images/agnhost:2.45],SizeBytes:52363605,},ContainerImage{Names:[registry.k8s.io/e2e-test-images/agnhost@sha256:a3f7549ea04c419276e8b84e90a515bbce5bc8a057be2ed974ec45492eca346e registry.k8s.io/e2e-test-images/agnhost:2.26],SizeBytes:49216572,},ContainerImage{Names:[docker.io/kindest/kindnetd:v20240202-8f1494ea],SizeBytes:27755257,},ContainerImage{Names:[docker.io/kindest/local-path-provisioner:v20240202-8f1494ea],SizeBytes:19365409,},ContainerImage{Names:[registry.k8s.io/coredns/coredns:v1.11.1],SizeBytes:18182961,},ContainerImage{Names:[docker.io/kindest/local-path-helper:v20230510-486859a6],SizeBytes:3052318,},ContainerImage{Names:[registry.k8s.io/pause:3.7],SizeBytes:311278,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
  May 24 16:11:46.062: INFO: 
  Logging kubelet events for node ovn-control-plane
  May 24 16:11:46.064: INFO: 
  Logging pods the kubelet thinks is on node ovn-control-plane
  May 24 16:11:46.074: INFO: ovs-node-fmpmd started at 2024-05-24 15:07:54 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container ovs-daemons ready: true, restart count 0
  May 24 16:11:46.074: INFO: ovnkube-control-plane-6744589fbf-nxh65 started at 2024-05-24 15:57:43 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container ovnkube-cluster-manager ready: true, restart count 1
  May 24 16:11:46.074: INFO: kube-controller-manager-ovn-control-plane started at 2024-05-24 15:06:58 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container kube-controller-manager ready: true, restart count 0
  May 24 16:11:46.074: INFO: kube-scheduler-ovn-control-plane started at 2024-05-24 15:06:58 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container kube-scheduler ready: true, restart count 0
  May 24 16:11:46.074: INFO: coredns-76f75df574-b6cmj started at 2024-05-24 15:08:02 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container coredns ready: true, restart count 0
  May 24 16:11:46.074: INFO: ovnkube-identity-85fcf584d5-5hhp4 started at 2024-05-24 15:56:39 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container ovnkube-identity ready: true, restart count 0
  May 24 16:11:46.074: INFO: kube-apiserver-ovn-control-plane started at 2024-05-24 15:06:58 +0000 UTC (0+1 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Container kube-apiserver ready: true, restart count 0
  May 24 16:11:46.074: INFO: speaker-4sj9s started at 2024-05-24 15:11:20 +0000 UTC (3+4 container statuses recorded)
  May 24 16:11:46.074: INFO: 	Init container cp-frr-files ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Init container cp-reloader ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Init container cp-metrics ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Container frr ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Container frr-metrics ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Container reloader ready: true, restart count 0
  May 24 16:11:46.074: INFO: 	Container speaker ready: true, restart count 0
  :OUTPUT ACCEPT [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:12:00 2024
  # Generated by ip6tables-save v1.8.9 (nf_tables) on Fri May 24 16:12:00 2024
  *nat
  :PREROUTING ACCEPT [0:0]
  :INPUT ACCEPT [0:0]
  :OUTPUT ACCEPT [0:0]
  :POSTROUTING ACCEPT [0:0]
  :KUBE-KUBELET-CANARY - [0:0]
  COMMIT
  # Completed on Fri May 24 16:12:00 2024

  May 24 16:12:00.881: INFO: DEBUG: Matched rule :OVN-KUBE-SNAT-MGMTPORT - [0:0] for pattern OVN-KUBE-SNAT-MGMTPORT
  May 24 16:12:00.881: INFO: DEBUG: Matched rule [640:38256] -A POSTROUTING -o ovn-k8s-mp0 -j OVN-KUBE-SNAT-MGMTPORT for pattern OVN-KUBE-SNAT-MGMTPORT
  May 24 16:12:00.881: INFO: DEBUG: Matched rule [0:0] -A OVN-KUBE-SNAT-MGMTPORT -o ovn-k8s-mp0 -m comment --comment "OVN SNAT to Management Port" -j SNAT --to-source 10.244.2.2 for pattern OVN-KUBE-SNAT-MGMTPORT
  STEP: by sending a TCP packet to service lbservice-test with type=LoadBalancer in namespace default with backend pod lb-backend-pod @ 05/24/24 16:12:00.881
  May 24 16:12:03.036: INFO: Unexpected error: failed to curl load balancer service: 
      <*errors.errorString | 0xc001679b60>: 
      failed to run "docker exec lbclient curl --max-time 120 [http://192.168.10.0:80/big.iso](http://192.168.10.0/big.iso) -o big.iso": exit status 56 (  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
  8  100M    8 8770k    0     0  4200k      0  0:00:24  0:00:02  0:00:22 4202k
      curl: (56) Recv failure: Connection reset by peer
      )
      {
          s: "failed to run \"docker exec lbclient curl --max-time 120 [http://192.168.10.0:80/big.iso](http://192.168.10.0/big.iso) -o big.iso\": exit status 56 (  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0\r  8  100M    8 8770k    0     0  4200k      0  0:00:24  0:00:02  0:00:22 4202k\ncurl: (56) Recv failure: Connection reset by peer\n)",
      }
  [FAILED] in [It] - /home/runner/work/ovn-kubernetes/ovn-kubernetes/test/e2e/service.go:1812 @ 05/24/24 16:12:03.037
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/e2e-testing kind/ci-flake Flakes seen in CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@flavio-fernandes