Skip to content

Commit

Permalink
Tests: use the default mocha reporter for better display for errors (#…
Browse files Browse the repository at this point in the history 
…1375)

## Context

When running `yarn test:` commands (like `yarn test:server`), the displayed errors are difficult to read. Especially when comparing two objects deeply (using `assert.deepEqual`):

I heavily use `yarn test:server` or `yarn test:gen-server` instead of `yarn test` in order to avoid launching a browser for the tests. 

An example of the display:

```
(2) Scim when enabled using GRIST_ENABLE_SCIM=1 DELETE /Users/{id} should return 403 for system users
AssertionError: expected { schemas: [ 'urn:ietf:params:scim:api:messages:2.0:Error' ],
  status: '403',
  detail: 'System user deletion not permitted.' } to deeply equal { schemas: [ 'urn:ietf:params:scim:api:messages:2.0:Error' ],
  detail: 'System user deletion not permitted.' }
    at Proxy.assertEql (node_modules/chai/lib/chai/core/assertions.js:1084:10)
    at Proxy.methodWrapper (node_modules/chai/lib/chai/utils/addMethod.js:57:25)
    at doAsserterAsyncAndAddThen (node_modules/chai-as-promised/lib/chai-as-promised.js:289:22)
    at Proxy.<anonymous> (node_modules/chai-as-promised/lib/chai-as-promised.js:255:20)
    at Proxy.overwritingMethodWrapper (node_modules/chai/lib/chai/utils/overwriteMethod.js:78:33)
    at Function.assert.deepEqual.assert.deepStrictEqual (node_modules/chai/lib/chai/interface/assert.js:232:56)
    at checkOperationOnTechUserDisallowed (test/server/lib/Scim.ts:146:16)
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
    at Context.<anonymous> (test/server/lib/Scim.ts:613:9)
```

## Proposed solution

We remove the `test/xunit-file` reporter and use the mocha default reporter. The above error is displayed like below, which is handier:
```
  2) Scim
       when enabled using GRIST_ENABLE_SCIM=1
         DELETE /Users/{id}
           should return 403 for system users:

      AssertionError: expected { schemas: [ 'urn:ietf:params:scim:api:messages:2.0:Error' ],
  status: '403',
  detail: 'System user deletion not permitted.' } to deeply equal { schemas: [ 'urn:ietf:params:scim:api:messages:2.0:Error' ],
  detail: 'System user deletion not permitted.' }
      + expected - actual

         "detail": "System user deletion not permitted."
         "schemas": [
           "urn:ietf:params:scim:api:messages:2.0:Error"
         ]
      -  "status": "403"
       }
      
      at Proxy.assertEql (node_modules/chai/lib/chai/core/assertions.js:1084:10)
      at Proxy.methodWrapper (node_modules/chai/lib/chai/utils/addMethod.js:57:25)
      at doAsserterAsyncAndAddThen (node_modules/chai-as-promised/lib/chai-as-promised.js:289:22)
      at Proxy.<anonymous> (node_modules/chai-as-promised/lib/chai-as-promised.js:255:20)
      at Proxy.overwritingMethodWrapper (node_modules/chai/lib/chai/utils/overwriteMethod.js:78:33)
      at Function.assert.deepEqual.assert.deepStrictEqual (node_modules/chai/lib/chai/interface/assert.js:232:56)
      at checkOperationOnTechUserDisallowed (test/server/lib/Scim.ts:146:16)
      at processTicksAndRejections (node:internal/process/task_queues:105:5)
      at Context.<anonymous> (test/server/lib/Scim.ts:613:9)
```

Alternatively, we may only use the `test/xunit-file` reporter when running the tests in Jenkins

## Has this been tested?

<!-- Put an `x` in the box that applies: -->

- [ ] 👍 yes, I added tests to the test suite
- [ ] 💭 no, because this PR is a draft and still needs work
- [x] 🙅 no, because this is not relevant here
- [ ] 🙋 no, because I need help <!-- Detail how we can help you -->
  • Loading branch information
@fflorent
fflorent authored Jan 14, 2025
1 parent e5bba90 commit e0772fe
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@
"build:prod": "buildtools/build.sh",
"start:prod": "sandbox/run.sh",
"test": "GRIST_TEST_LOGIN=1 ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} --slow 8000 $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" '_build/test/common/*.js' '_build/test/client/*.js' '_build/test/nbrowser/*.js' '_build/test/nbrowser_with_stubs/**/*.js' '_build/test/server/**/*.js' '_build/test/gen-server/**/*.js'",
"test:nbrowser": "GRIST_TEST_LOGIN=1 TEST_SUITE=nbrowser TEST_SUITE_FOR_TIMINGS=nbrowser TIMINGS_FILE=test/timings/nbrowser.txt ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" --slow 8000 -R test/xunit-file '_build/test/nbrowser/**/*.js'",
"test:stubs": "GRIST_TEST_LOGIN=1 ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" --slow 8000 -R test/xunit-file '_build/test/nbrowser_with_stubs/**/*.js'",
"test:nbrowser": "GRIST_TEST_LOGIN=1 TEST_SUITE=nbrowser TEST_SUITE_FOR_TIMINGS=nbrowser TIMINGS_FILE=test/timings/nbrowser.txt ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" --slow 8000 '_build/test/nbrowser/**/*.js'",
"test:stubs": "GRIST_TEST_LOGIN=1 ./test/test_env.sh mocha ${DEBUG:+-b --no-exit} $([ -z $DEBUG ] && echo --forbid-only) -g \"${GREP_TESTS}\" --slow 8000 '_build/test/nbrowser_with_stubs/**/*.js'",
"test:client": "./test/test_env.sh mocha ${DEBUG:+'-b'} '_build/test/client/**/*.js'",
"test:common": "./test/test_env.sh mocha ${DEBUG:+'-b'} '_build/test/common/**/*.js'",
"test:server": "TEST_SUITE=server TEST_SUITE_FOR_TIMINGS=server TIMINGS_FILE=test/timings/server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" -R test/xunit-file '_build/test/server/**/*.js'",
"test:gen-server": "TEST_SUITE=gen-server TEST_SUITE_FOR_TIMINGS=gen-server TIMINGS_FILE=test/timings/gen-server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" -R test/xunit-file '_build/test/gen-server/**/*.js'",
"test:server": "TEST_SUITE=server TEST_SUITE_FOR_TIMINGS=server TIMINGS_FILE=test/timings/server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" '_build/test/server/**/*.js'",
"test:gen-server": "TEST_SUITE=gen-server TEST_SUITE_FOR_TIMINGS=gen-server TIMINGS_FILE=test/timings/gen-server.txt ./test/test_env.sh mocha ${DEBUG:+'-b'} -g \"${GREP_TESTS}\" '_build/test/gen-server/**/*.js'",
"test:smoke": "./test/test_env.sh mocha _build/test/nbrowser/Smoke.js",
"test:docker": "./test/test_under_docker.sh",
"test:python": "sandbox_venv3/bin/python sandbox/grist/runtests.py ${GREP_TESTS:+discover -p \"test*${GREP_TESTS}*.py\"}",
Expand Down

0 comments on commit e0772fe

Please sign in to comment.