Skip to content
This repository has been archived by the owner on Oct 22, 2024. It is now read-only.

Commit

Permalink
e2e: test flake during metrics testing
Browse files Browse the repository at this point in the history
At least once the "direct-testing-metrics data full" test failed with:

Dec 15 22:36:03.221: Timed out after 10.005s.
Expected
    <string>: have pod IP
    Expected
        <string>:
    not to be empty
    GET failed
    Unexpected error:
        <*url.Error | 0xc000cb0e70>: {
            Op: "Get",
            URL: "http://default.format-node-hcz2c:10010/metrics",
            Err: <*errors.errorString | 0xc0015c9800>{
                s: "dialer failed: unable to upgrade connection: pod not found (\"format-node-hcz2c_default\")",
            },
        }
        Get "http://default.format-node-hcz2c:10010/metrics": dialer failed: unable to upgrade connection: pod not found ("format-node-hcz2c_default")
    occurred
to be empty

Better assertion annotations are necessary to understand which pod these
assertions are about.

If this was a pod that just began starting, then the 10s timeout may have been
too short.
  • Loading branch information
pohly committed Dec 16, 2021
1 parent 4e8e9dc commit 2c0baac
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions test/e2e/metrics/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ var _ = deploy.Describe("direct-testing", "direct-testing-metrics", "", func(d *

ip := pod.Status.PodIP
portNum := port.ContainerPort
Expect(ip).ToNot(BeEmpty(), "have pod IP")
Expect(portNum).ToNot(Equal(0), "have container port")
Expect(ip).ToNot(BeEmpty(), "have pod IP for pod %s", pod)
Expect(portNum).ToNot(Equal(0), "have container port for pod %s", pod)

url := fmt.Sprintf("http://%s.%s:%d/metrics",
pod.Namespace, pod.Name, port.ContainerPort)
Expand All @@ -91,7 +91,7 @@ var _ = deploy.Describe("direct-testing", "direct-testing-metrics", "", func(d *
return
}
data, err := ioutil.ReadAll(resp.Body)
framework.ExpectNoError(err, "read GET response")
framework.ExpectNoError(err, "read GET response for pod %s and url %s", pod, url)
name := pod.Name + "/" + container.Name
if isPmemCSI {
Expect(data).To(ContainSubstring("build_info"), name)
Expand All @@ -116,7 +116,7 @@ var _ = deploy.Describe("direct-testing", "direct-testing-metrics", "", func(d *
}
}
}
Expect(numPorts).NotTo(Equal(0), "at least one container should have a 'metrics' port")
Expect(numPorts).NotTo(Equal(0), "at least one container should have a 'metrics' port in pod %s", pod)
}
Expect(numPods).NotTo(Equal(0), "at least one container should have a 'metrics' port")
}
Expand All @@ -133,7 +133,7 @@ var _ = deploy.Describe("direct-testing", "direct-testing-metrics", "", func(d *
return strings.Join(InterceptGomegaFailures(func() {
testData(simple, pods)
}), "\n")
}, "10s", "1s").Should(BeEmpty())
}, "60s", "1s").Should(BeEmpty())
}

It("full", func() {
Expand Down

0 comments on commit 2c0baac

Please sign in to comment.