Feature request: Allow tests from different groups to not be interspersed

[cyphar]

Motivation

As part of my safe path library it is necessary to run some race tests that ensure that attackers cannot trick the library into walking outside of the rootfs.

Unfortunately, having the racing threads running as the same time as the other tests can cause spurious test failures (the short explanation is that openat2 will return -EAGAIN if it detected a race, and there is a hard limit on the number of retries the library does, so if you have a program doing a rename-exchange attack some tests in the suite will randomly fail).

Ideally it would be possible to tell nextest to only schedule the race tests in parallel with one another, and the other tests in parallel with one another, but not to intersperse the tests in the two groups. I was hoping that test groups let you do this but it appears they don't.

Proposal

Allow a test group to be marked as "exclusive" or something, which means that other test groups cannot be run at the same time as it (but running tests in the same group in parallel is still allowed). The configuration could be as simple as

[test-groups]
exclusive = { exclusive = true, max-threads = 16 }

[[profile.default.overrides]]
filter = "test(tests::test_race_resolve_partial::)"
test-group = "exclusive"

I'm not familiar with nextest's implementation, so I don't know how hard it would be to make that work. The ideal solution would be for this to be done through something like #[nextest(...)] which would let this kind of important information live with the tests themselves, but it seems (from #101) that this is not an acceptable solution.

Alternatives

The only solution I've found so far that can ensure that these tests are not run at the same time as other tests is to make them be run serially. max-threads doesn't work, but if you use threads-required it will make these tests hog all of the thread slots and thus ensure that they are the only thing running:

[[profile.default.overrides]]
filter = "test(tests::test_race_resolve_partial::)"
threads-required = "num-test-threads"

However, this significantly increases the test run time because all of the tests in the group need to be run in serial now, when running them in parallel is totally fine (it's just that they can't run in parallel with tests from outside their group).

The other option is to do two separate test runs, but this runs into issues where a naive cargo nextest run will fail while the Makefile-configured one won't. I already have workarounds in my Makefile (such as CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER='sudo -E') but adding more is quite ugly.

Additional context

No response