Only the last finished thread (test) is showing in the Cucumber report.

[tausif29]

Current behavior

After upgrading preprocessor to the latest (>=20) , When Cypress tests are running in parallel, It overwrites previously executed groups/threads from the Cucumber's log.json file, Only the last executed/finished thread/group remains in the json, Hence When we execute command to generate the report, It shows up result of last finished group/thread.

Desired behavior

It should work like before, log.json should not be override, all the test result should be stored in the log.json.

Test code to reproduce

When you run the test in parallel using any plugin it will be reproducible. I was able to reproduce the same using below plugins:

1. cypress-split
2. cypress-parallel

Versions

• Cypress version: 13
• Preprocessor version: >=20
• Node version: 20.9.0

Checklist

• I've read the FAQ.
• I've read instructions for logging issues.
• I'm not using [email protected] (package name has changed and it is no longer the most recent version, see #689).

[badeball]

• Preprocessor version: >=20

Are you implying that this behavior changed from v19.2.0 to v20.0.0?

In any case, cypress-split is for splitting tests across multiple machines, in which case there is no overwriting of reports. Rather, they can be combined.

cypress-parallel parallel on the other hand, essentially just invokes $ cypress run N times. Conceptually, these runs may be considered related to each other, but this plugin is unaware of this relationship.

Simply put, upon execution, any existing report is deleted to make room for a new one. The alternative of always merging reports is unintuitive and not appropriate in most other scenarios. Thus closing.

[tausif29]

@badeball I am not sure from which version its broken, I was on version ^16.0.3 and directly updated to the latest and found this issue.

1. It was working fine with cypress-split - running all test cases parallelly on a single machine by executing a .js file.

const { exec } = require('child_process');

const SPLITS = process.env.SPLITS || 10;
const ENV = process.env.ENV || 'SB';

console.log(`Running tests with ${SPLITS} splits for environment ${ENV}`);

for (let i = 0; i < SPLITS; i++) {
  console.log(`Running Cypress with split group ${i} of ${SPLITS}`);
  
  const command = `npx cypress run --browser chrome  --env ENV=${ENV},split=${SPLITS},splitIndex=${i}`;
  
  const subprocess = exec(command);

  subprocess.stdout.on('data', (data) => {
    console.log(`Stdout for group ${i}: ${data}`);
  });

  subprocess.stderr.on('data', (data) => {
    console.error(`Stderr for group ${i}: ${data}`);
  });

  subprocess.on('close', (code) => {
    if (code === 0) {
      console.log(`Group ${i} completed successfully.`);
    } else {
      console.error(`Group ${i} failed with exit code ${code}.`);
    }
  });

  subprocess.on('error', (error) => {
    console.error(`Failed to start subprocess for group ${i}: ${error.message}`);
  });
}

console.log('All Cypress tests have been initiated.');

2. It was working well with cypress-parallel as well like below:

Upgrading to the latest is backfired for us, We also encountered another issue with attach not working under afterEach (not raising the issue as I have already found one in the listing)