OCPBUGS-100388: Fix project name collision in test framework by redhat-chai-bot · Pull Request #31464 · openshift/origin

redhat-chai-bot · 2026-07-31T14:01:22Z

Summary

Add retry-on-AlreadyExists logic to setupProject() and setupNamespace() in test/extended/util/client.go.

Problem

When Ginkgo runs tests in parallel, each worker is a separate process that initializes k8s.io/apimachinery/pkg/util/rand with time.Now().UnixNano(). Workers starting at the same nanosecond get the same PRNG seed and generate identical project/namespace names, causing 409 AlreadyExists errors that immediately fail the test.

This affects multiple teams across all platforms. In a recent 48-hour window, 7 distinct jobs across Metal, GCP, Azure, and AWS hit this collision with different BaseNames (oc-adm-must-gather, server-side-apply, cloud-controller-manager, job-names, scheduling-pod-check).

Fix

On AlreadyExists error during project/namespace creation:

Log the collision
Generate a new name via names.SimpleNameGenerator.GenerateName()
Update the namespace and user references
Retry (up to 3 attempts)

This pattern already exists in the same file for CreateUser() (line ~1200) which handles apierrors.IsAlreadyExists.

Non-AlreadyExists errors still fail immediately through the existing Gomega assertion.

Testing

go build ./test/extended/util/... passes
go vet ./test/extended/util/... passes

@stbenjam requested in Slack thread

Summary by CodeRabbit

Bug Fixes
- Project and namespace creation now automatically retries when a generated name already exists.
- New names and associated user identities are generated for each retry.
- Other creation errors stop immediately, with the final error still reported.

Add retry-on-AlreadyExists logic to setupProject() and setupNamespace() in the test framework. When Ginkgo parallel workers start at the same nanosecond, the PRNG produces identical names causing AlreadyExists errors. On collision, generate a new name and retry up to 3 attempts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

openshift-ci-robot · 2026-07-31T14:01:28Z

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-98468, which is invalid:

expected the bug to target only the "5.0.0" version, but multiple target versions were set
expected the bug to be in one of the following states: NEW, ASSIGNED, POST, but it is Verified instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

Add retry-on-AlreadyExists logic to setupProject() and setupNamespace() in test/extended/util/client.go.

Problem

When Ginkgo runs tests in parallel, each worker is a separate process that initializes k8s.io/apimachinery/pkg/util/rand with time.Now().UnixNano(). Workers starting at the same nanosecond get the same PRNG seed and generate identical project/namespace names, causing 409 AlreadyExists errors that immediately fail the test.

This affects multiple teams across all platforms. In a recent 48-hour window, 7 distinct jobs across Metal, GCP, Azure, and AWS hit this collision with different BaseNames (oc-adm-must-gather, server-side-apply, cloud-controller-manager, job-names, scheduling-pod-check).

Fix

On AlreadyExists error during project/namespace creation:

Log the collision

Generate a new name via names.SimpleNameGenerator.GenerateName()

Update the namespace and user references

Retry (up to 3 attempts)

This pattern already exists in the same file for CreateUser() (line ~1200) which handles apierrors.IsAlreadyExists.

Non-AlreadyExists errors still fail immediately through the existing Gomega assertion.

Testing

go build ./test/extended/util/... passes

go vet ./test/extended/util/... passes

@stbenjam requested in Slack thread

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-merge-bot · 2026-07-31T14:01:32Z

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

coderabbitai · 2026-07-31T14:04:00Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 5b5c55fb-1292-4d04-a18b-fab567a5d9f1

📥 Commits

Reviewing files that changed from the base of the PR and between 2e785fb and e41e44e.

📒 Files selected for processing (1)

test/extended/util/client.go

🚧 Files skipped from review as they are similar to previous changes (1)

test/extended/util/client.go

Walkthrough

Project and namespace creation retry up to three times when generated names already exist. Each retry generates updated names and related metadata. Other errors stop immediately.

Changes

Creation Retry Handling

Layer / File(s)	Summary
Retry project and namespace creation `test/extended/util/client.go`	Project and namespace creation retry `AlreadyExists` errors up to three times. Each retry regenerates names and related identities or annotations. Other errors stop immediately, and the final error remains asserted.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: sjenning, p0lyn0mial

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Structure And Quality	⚠️ Warning	setupProject() and setupNamespace() retain many bare o.Expect(err).NotTo(o.HaveOccurred()) assertions for cluster creation and waits, which provide no operation-specific failure context.	Add meaningful messages to each relevant assertion, such as "failed to create project", "failed to wait for service account", and "failed to configure namespace security".

✅ Passed checks (14 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the project name collision fix, which matches the primary change in the pull request.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	The PR changes only setupProject/setupNamespace in client.go; no It, Describe, Context, or When titles are added or changed, and generated names stay in setup and logs.
Microshift Test Compatibility	✅ Passed	The PR changes only test utility retry logic; it adds no Ginkgo It/Describe/Context/When tests, so the MicroShift test-compatibility check is not applicable.
Single Node Openshift (Sno) Test Compatibility	✅ Passed	The PR changes only test/extended/util/client.go retry logic; the full candidate diff adds no Ginkgo tests or multi-node/HA assumptions.
Topology-Aware Scheduling Compatibility	✅ Passed	The commit only changes retry handling for project and namespace creation in test/extended/util/client.go; it adds no deployment, controller, manifest, or scheduling constraints.
Ote Binary Stdout Contract	✅ Passed	The PR adds only framework.Logf calls; vendored framework.Logf writes to GinkgoWriter, and setupProject/setupNamespace run from BeforeEach or test code. No process-level stdout write was added.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The PR changes only existing setup utility methods; the feature diff adds no Ginkgo tests, IPv4 assumptions, or external connectivity requirements.
No-Weak-Crypto	✅ Passed	The PR modifies retry logic in setupProject() and setupNamespace() to handle AlreadyExists errors. No weak cryptographic algorithms (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB) or insecure comparison...
Container-Privileges	✅ Passed	PR modifies Go test utility code (client.go) for retry logic on name collisions; no Kubernetes manifests, Dockerfiles, or container security configurations with privilege settings (privileged: true...
No-Sensitive-Data-In-Logs	✅ Passed	The PR adds retry-on-AlreadyExists logic with logging statements that expose only auto-generated test namespace/project names and usernames derived from them. No passwords, tokens, API keys, PII, s...

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

openshift-ci · 2026-07-31T14:04:08Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: redhat-chai-bot
Once this PR has been reviewed and has the lgtm label, please assign mkowalski for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/extended/util/client.go`:
- Around line 370-372: Update the retry bounds in test/extended/util/client.go
at lines 370-372 and 503-506 so both create flows allow the initial attempt plus
three retries. In the associated name-regeneration logic, stop generating a new
name after the final permitted retry fails.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ab06fee-a926-4810-896b-70d72f01ae29

📥 Commits

Reviewing files that changed from the base of the PR and between 2afc4ee and 2e785fb.

📒 Files selected for processing (1)

test/extended/util/client.go

openshift-ci-robot · 2026-07-31T14:15:10Z

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-100388, which is invalid:

expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

Add retry-on-AlreadyExists logic to setupProject() and setupNamespace() in test/extended/util/client.go.

Problem

When Ginkgo runs tests in parallel, each worker is a separate process that initializes k8s.io/apimachinery/pkg/util/rand with time.Now().UnixNano(). Workers starting at the same nanosecond get the same PRNG seed and generate identical project/namespace names, causing 409 AlreadyExists errors that immediately fail the test.

This affects multiple teams across all platforms. In a recent 48-hour window, 7 distinct jobs across Metal, GCP, Azure, and AWS hit this collision with different BaseNames (oc-adm-must-gather, server-side-apply, cloud-controller-manager, job-names, scheduling-pod-check).

Fix

On AlreadyExists error during project/namespace creation:

Log the collision

Generate a new name via names.SimpleNameGenerator.GenerateName()

Update the namespace and user references

Retry (up to 3 attempts)

This pattern already exists in the same file for CreateUser() (line ~1200) which handles apierrors.IsAlreadyExists.

Non-AlreadyExists errors still fail immediately through the existing Gomega assertion.

Testing

go build ./test/extended/util/... passes

go vet ./test/extended/util/... passes

@stbenjam requested in Slack thread

Summary by CodeRabbit

Bug Fixes

Project and namespace creation now automatically retries when a generated name already exists.

New names and associated user identities are generated for each retry.

Other creation errors stop immediately, with the final error still reported.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

redhat-chai-bot · 2026-07-31T14:15:16Z

/jira refresh

openshift-ci-robot · 2026-07-31T14:15:20Z

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-100388, which is invalid:

expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-merge-bot · 2026-07-31T15:02:55Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

redhat-chai-bot · 2026-07-31T15:14:06Z

/jira refresh

openshift-ci-robot · 2026-07-31T15:14:10Z

@redhat-chai-bot: This pull request references Jira Issue OCPBUGS-100388, which is invalid:

expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-merge-bot · 2026-07-31T16:31:20Z

Scheduling required tests:
/test e2e-aws-csi
/test e2e-aws-ovn-fips
/test e2e-aws-ovn-microshift
/test e2e-aws-ovn-microshift-serial
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-gcp-csi
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-ipv6
/test e2e-vsphere-ovn
/test e2e-vsphere-ovn-upi

redhat-chai-bot · 2026-07-31T20:40:52Z

/retest-required

redhat-chai-bot · 2026-08-01T00:19:37Z

/retest-required

openshift-ci · 2026-08-01T03:46:57Z

@redhat-chai-bot: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-serial-2of2	`e41e44e`	link	true	`/test e2e-aws-ovn-serial-2of2`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 31, 2026

openshift-ci Bot added the ready-for-human-review Indicates a PR has been reviewed by automated tools and is ready for human review label Jul 31, 2026

openshift-ci Bot requested review from p0lyn0mial and sjenning July 31, 2026 14:04

coderabbitai Bot requested changes Jul 31, 2026

View reviewed changes

Comment thread test/extended/util/client.go Outdated

redhat-chai-bot changed the title ~~OCPBUGS-98468: Fix project name collision in test framework~~ OCPBUGS-100388: Fix project name collision in test framework Jul 31, 2026

fixup: use <= for maxRetries to allow initial attempt + 3 retries

e41e44e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

coderabbitai Bot approved these changes Jul 31, 2026

View reviewed changes

Uh oh!

Conversation

redhat-chai-bot commented Jul 31, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Fix

Testing

Summary by CodeRabbit

Uh oh!

openshift-ci-robot commented Jul 31, 2026

Summary

Problem

Fix

Testing

Uh oh!

openshift-merge-bot Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

❌ Failed checks (1 warning)

Uh oh!

openshift-ci Bot commented Jul 31, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

openshift-ci-robot commented Jul 31, 2026

Summary

Problem

Fix

Testing

Summary by CodeRabbit

Uh oh!

redhat-chai-bot commented Jul 31, 2026

Uh oh!

openshift-ci-robot commented Jul 31, 2026

Uh oh!

openshift-merge-bot Bot commented Jul 31, 2026

Uh oh!

redhat-chai-bot commented Jul 31, 2026

Uh oh!

openshift-ci-robot commented Jul 31, 2026

Uh oh!

openshift-merge-bot Bot commented Jul 31, 2026

Uh oh!

redhat-chai-bot commented Jul 31, 2026

Uh oh!

redhat-chai-bot commented Aug 1, 2026

Uh oh!

openshift-ci Bot commented Aug 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

redhat-chai-bot commented Jul 31, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading