Skip to content

The [Collection] attribute is ignored when running unit tests from contextual menu in the "code view". #421

Open
@GillesTourreau

Description

@GillesTourreau

To reproduce the bug, I have created the following unit tests class which contains 4 unit tests that must not be executed in parallel, for that I added the [Collection("Foobar")] attribute on the unit test class.
I added in this unit test class some assertions to check that unit tests are not executed in parallel.

namespace XunitParallelExecutionVsRunnerBug
{
    [Collection("Foobar")]
    public class UnitTest1
    {
        private static readonly bool[] TestRun = new bool[typeof(UnitTest1).GetMethods().Length];

        [Fact]
        public async Task Test1()
        {
            StartTestRun(1);

            await Task.Delay(3000);

            EndTestRun(1);
        }

        [Fact]
        public async Task Test2()
        {
            StartTestRun(2);

            await Task.Delay(3000);

            EndTestRun(2);
        }
        
        [Fact]
        public async Task Test3()
        {
            StartTestRun(3);

            await Task.Delay(3000);

            EndTestRun(3);
        }

        [Fact]
        public async Task Test4()
        {
            StartTestRun(4);

            await Task.Delay(3000);

            EndTestRun(4);
        }

        private static void StartTestRun(int number)
        {
            lock (TestRun)
            {
                // Assert no other tests is running.
                for (var i = 0; i < TestRun.Length; i++)
                {
                    Assert.False(TestRun[i], $"The Test{i + 1} is running in parallel with the Test{number}...");
                }

                TestRun[number - 1] = true;
            }
        }

        private static void EndTestRun(int number)
        {
            lock (TestRun)
            {
                TestRun[number - 1] = false;
            }
        }
    }
}

When I run the unit tests from the Test Explorer, we can see that all the unit tests are executed sequentially.
Test explorer run

But when I run the unit tests from the code with the contextual menu (when right click on empty code line between two unit test methods), the unit tests are executed in parallel:
Test from view of the code

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions