Missing generating some json files after test

[szory]

Hi,
I am using:
wdio-cucumberjs-json-reporter (2.0.2),
webdriverio 6.3.4

and generating a html report with:
multiple-cucumber-html-reporter 1.18.0 using the json created.

Problem is with missing generating some json files (not every one), which should be created after finish every test. We have a over 150 tests which are divided into 7 sets. These sets have from 13 - 27 tests. This missing of generating tests is very random, sometimes is missing some tests in set number 2 and some times in set numer 6.
Does any one encountered similar problem?

[wswebcreation]

Hi @szory

Thanks for filing an issue, what could happen in this case is that the feature has the same name and that it's overwriting / concatenating the data. Can you first check in all your feature files if the feature names are all unique?

[szory]

Hi, Thx for quick response. I've just checked feature files and all the names are unique....

[wswebcreation]

can you then provide a reproducible example so I can debug it?

[szory]

we run these tests on Jenkins, that would be ok for you if I send Console Output from Jenkins

[wswebcreation]

Sorry @szory

Yes, can you send me the Console Output and also a log of the files that are generated?

[szory]

Hi @wswebcreation
Sorry for that huge delay....

this is the piece of Jenkins Console Output. In this sample of log is missing report called: IOOS Online Portal Password Reset:

06:29:45  2021-02-04T05:29:44.843Z INFO @wdio/local-runner: Start worker 0-70 with arg: 
06:29:45  [0-70] 2021-02-04T05:29:45.231Z INFO @wdio/local-runner: Run worker command: run
06:29:45  [0-70] 2021-02-04T05:29:45.377Z INFO webdriverio: Initiate new session using the ./protocol-stub protocol
06:29:45  [0-70] RUNNING in chrome - /src/ui/features/iqos_portal/password-reset.feature
06:29:45  [0-70] 2021-02-04T05:29:45.656Z INFO webdriverio: Initiate new session using the webdriver protocol
06:29:50  [0-70] 2021-02-04T05:29:50.915Z INFO @wdio/browserstack-service: Update job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357
06:29:51  [0-70] 2021-02-04T05:29:51.333Z INFO @wdio/browserstack-service: Windows 10 chrome session: https://automate.browserstack.com/builds/7cad3678d9bd81826e6c821092f0edd559aae365/sessions/a8b233f08c64ee57926fe8b6708d8f95ef152357
06:30:01  [0-70] 2021-02-04T05:30:01.040Z INFO @wdio/browserstack-service: Update job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357
06:30:01  [0-70] �[32m[INFO ]: Test email created
06:30:02  [0-70] �[32m[INFO ]: User created
06:32:44  [0-70] Error in "IOOS Online Portal Password Reset: Password Reset: And I extract "Reset Password Url" within latest email titled as "Reset Password""
06:32:44  Error: function timed out, ensure the promise resolves within 60000 milliseconds
06:32:44      at Timeout._onTimeout (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/user_code_runner.js:76:18)
06:32:44      at listOnTimeout (internal/timers.js:549:17)
06:32:44      at processTimers (internal/timers.js:492:7)
06:32:44  [0-70] 2021-02-04T05:32:39.365Z INFO @wdio/browserstack-service: Update (reloaded) job with sessionId a8b233f08c64ee57926fe8b6708d8f95ef152357, failed
06:32:44  [0-70] 2021-02-04T05:32:39.869Z INFO @wdio/browserstack-service: Update job with sessionId 31f059237d3eb781f80a69ae58726101532651d3
06:32:44  [0-70] 2021-02-04T05:32:40.270Z INFO @wdio/browserstack-service: Windows 10 chrome session: https://automate.browserstack.com/builds/7cad3678d9bd81826e6c821092f0edd559aae365/sessions/31f059237d3eb781f80a69ae58726101532651d3
06:32:44  [0-70] �[32m[INFO ]: Test email created
06:32:44  [0-70] �[32m[INFO ]: User created
06:34:08  [0-70] 2021-02-04T05:34:02.626Z WARN webdriver: Request encountered a stale element - terminating request
06:34:08  [0-70] 2021-02-04T05:34:03.837Z WARN webdriver: Request encountered a stale element - terminating request
06:35:04  [0-70] Error in "IQOS Online Portal Password Reset: Password Reset: And I extract "Reset Password Url" within latest email titled as "Reset Password""
06:35:04  Error: function timed out, ensure the promise resolves within 60000 milliseconds
06:35:04      at Timeout._onTimeout (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/user_code_runner.js:76:18)
06:35:04      at listOnTimeout (internal/timers.js:549:17)
06:35:04      at processTimers (internal/timers.js:492:7)
06:35:51  [0-70] 2021-02-04T05:35:47.312Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.339Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.364Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.391Z ERROR webdriver: Request failed with status 404 due to Error: Session not started or terminated
06:35:51  [0-70] 2021-02-04T05:35:47.394Z ERROR @wdio/sync: Error: Session not started or terminated
06:35:51      at processTicksAndRejections (internal/process/task_queues.js:97:5)
06:35:51      at Object.afterScenario (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/src/core/conf/wdio.template.conf.js:368:13)
06:35:51      at TestCaseRunner.aroundTestStep (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/runtime/test_case_runner.js:185:28)
06:35:51      at /home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/cucumber/lib/runtime/test_case_runner.js:253:7
06:35:51  [0-70] 2021-02-04T05:35:47.396Z INFO @wdio/browserstack-service: Update job with sessionId 31f059237d3eb781f80a69ae58726101532651d3
06:36:01  [0-70] 2021-02-04T05:36:01.593Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:03  [0-70] 2021-02-04T05:36:02.984Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:04  [0-70] 2021-02-04T05:36:04.356Z WARN webdriver: Request failed with status 404 due to Session not started or terminated
06:36:05  [0-70] 2021-02-04T05:36:05.735Z ERROR webdriver: Request failed with status 404 due to Error: Session not started or terminated
06:36:05  [0-70] 2021-02-04T05:36:05.736Z ERROR @wdio/local-runner: Failed launching test session: Error: Session not started or terminated
06:36:05      at processTicksAndRejections (internal/process/task_queues.js:97:5)
06:36:05      at Runner.endSession (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/@wdio/local-runner/node_modules/@wdio/runner/build/index.js:277:26)
06:36:05      at Runner.run (/home/jenkins/agent/workspace/DCE 2.0 Test Automation Framework/UI/UVT - testing/node_modules/@wdio/local-runner/node_modules/@wdio/runner/build/index.js:163:18)
06:36:05  [0-70] FAILED in chrome - /src/ui/features/iqos_portal/password-reset.feature

we have also this code in WDIO.config.js:

cucumberOpts: {
    retry: 2
}

retry mechanism failed at some reason that we've got 404 and json report was not generated

Thx in advance for your any help/advise

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ianhomer]

I encountered a similar situation (we're wdio / BrowserStack), but the signal looks similar. If BrowserStack is unavailable then we don't get a report.

I created a custom reporter to troubleshoot, e.g.

import WDIOReporter from '@wdio/reporter';
class MyCustomReporter extends WDIOReporter {
    public onRunnerStart(runner) {
        console.log('Runner start');
    }
}

and then register with reporters: ['cucumberjs-json', MyCustomReporter].

With this I see that @wdio/reporter events don't seem to get fired in this situation. For my situation it's not a wdio-cucumberjs-json-reporter issue here, more that the wdio worker is not even calling the reporting if a runner fails to start.