Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In tests, background tasks start executing before all sled agents register with Nexus, which can cause test flakes #7417

Open
iliana opened this issue Jan 28, 2025 · 0 comments
Labels
Test Flake Tests that work. Wait, no. Actually yes. Hang on. Something is broken.

Comments

@iliana
Copy link
Contributor

iliana commented Jan 28, 2025

After adding a lot of extra output to the tuf_artifact_replication background task and its omdb output handler in #7129, I ran into a problem where the output was different based on the number of sleds that had registered with Nexus before the background task starts.

Based on logs from a run, I see the contacting server nexus log message from the first simulated sled agent at 21:48:42.793Z, tuf_artifact_replication background task activation at 21:48:42.800Z, followed by the second sled agent contacting nexus at 21:48:43.179Z. The first background task fires at 21:48:42.784Z.

Getting rid of extra_sled_agents = 1 in the test_omdb_success_cases test might lessen the risk of flakes here but it won't completely eliminate it.

For now I'm planning to work around this by writing a redactor and referencing this issue in the comment.

@iliana iliana added the Test Flake Tests that work. Wait, no. Actually yes. Hang on. Something is broken. label Jan 28, 2025
iliana added a commit that referenced this issue Jan 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Test Flake Tests that work. Wait, no. Actually yes. Hang on. Something is broken.
Projects
None yet
Development

No branches or pull requests

1 participant