Docs: Improve callback docs and integrate js-reporters events spec

== Integrate js-reporters spec ==

Keep the link to the js-reporters project for now, but no longer rely on it for
discovery. The spec was written for implementors, which makes it a suboptimal
reading experience for end-users. It also suffered from numerous nullable
properties that in practice with QUnit are either never null or always null.

In our docs, I've left these off or explicitly described them in their
reliable non-null form to avoid any doubt or confusion over how or why
they behave a certain way.

Note that as of js-reporters 2.0, much of the spec has been removed which
QUnit currently still implements for compatibility (such as upfront recursively
declared child suites). These have not been documented and will be removed
in QUnit 3.0.

Ref https://github.com/js-reporters/js-reporters/releases/tag/v2.0.0.

The integration of the spec, rather than improving or creating a page in the
js-reporters repo for end-users, is motivated by
qunitjs/js-reporters#133, in which I conclude
that for now it is a better use of our effort for any re-usable cross-framework
reporters to consume TAP rather than tight runtime coupling. This also has the
benefit of not pushing down the "serialize actual value" and "format a diff"
problems down to end-consumers, which in practice are often done poorly or not
at all.

== Document unofficial `done` details ==

For `done()`, the `details.modules` property was added by commit 168b048 in
QUnit 1.16 (2014) for internal use by HTML Reporter.

It was never documented, however. It originally came with a `test` property as
well, but that hasn't been in use since commit 43a3d87 in QUnit 2.7 (2018).
Keep this for now since it's not adding delay or complexity, but I've left a
note to remove this in QUnit 3.0.

Document the rest as now-officially supported, with retroactive changelog.

Also document the oft-overlooked caveat with the legacy assertion counts exposed
from `QUnit.done()`. I believe the community has large moved away from using
the data provided by this callback.
For example, gruntjs/grunt-contrib-qunit#137.

Recommend `on('runEnd')` instead.

Author: Krinkle

docs/assert/expect.md

@@ -1,7 +1,7 @@
 ---
 layout: page-api
 title: assert.expect()
-excerpt: Specify how many assertions are expected to run within a test.
+excerpt: Specify how many assertions are expected in a test.
 groups:
   - assert
 redirect_from:
@@ -11,20 +11,61 @@ version_added: "1.0.0"
 
 `expect( amount )`
 
-Specify how many assertions are expected to run within a test.
+Specify how many assertions are expected in a test.
 
 | name | description |
 |------|-------------|
-| `amount` | Number of assertions in this test. |
+| `amount` | Number of expected assertions in this test. |
 
-To ensure that an explicit number of assertions are run within any test, use `assert.expect()` to register an expected count. If the number of assertions run does not match the expected count, the test will fail.
+This is most commonly used as `assert.expect(0)`, which indicates that a test may pass making any assertions. This means the test is only used to verify that the code to completion without any uncaught errors. This is is essentially the inverse of [`assert.throws()`](./throws.md).
+
+It can also be used to explicitly require a certain number of assertions to be recorded in a given test. If afterwards the number of assertions does not match the expected count, the test will fail.
+
+It is recommended to test asynchronous code with [`assert.step()`](./step.md) or [`assert.async()`](./async.md) instead.
 
 ## Examples
 
-Establish an expected assertion count
+### Example: No assertions
+
+A test without any assertions:
+
+```js
+QUnit.test('example', function (assert) {
+  assert.expect(0);
+
+  var android = new Robot();
+  android.up(2);
+  android.down(2);
+  android.left();
+  android.right();
+  android.left();
+  android.right();
+  android.attack();
+  android.jump();
+});
+```
+
+### Example: Custom assert
+
+If you use a generic assertion library that throws when an expectation is not met, you can use `assert.expect(0)` if there are no other assertions needed in the test.
+
+```js
+QUnit.test('example', function (assert) {
+  assert.expect(0);
+
+  var android = new Robot(database);
+  android.run();
+
+  database.assertNoOpenConnections();
+});
+```
+
+### Example: Explicit count
+
+Require an explicit assertion count.
 
 ```js
-QUnit.test('a test', function (assert) {
+QUnit.test('example', function (assert) {
   assert.expect(2);
 
   function calc (x, operation) {

docs/callbacks/QUnit.begin.md

@@ -1,7 +1,7 @@
 ---
 layout: page-api
 title: QUnit.begin()
-excerpt: Register a callback to fire whenever the test suite begins.
+excerpt: Register a callback to fire when the test run begins.
 groups:
   - callbacks
 redirect_from:
@@ -11,21 +11,25 @@ version_added: "1.0.0"
 
 `QUnit.begin( callback )`
 
-Register a callback to fire whenever the test suite begins. The callback may be an async function, or a function that returns a promise, which will be waited for before the next callback is handled.
+Register a callback to fire when the test run begins. The callback may be an async function, or a function that returns a Promise, which will be waited for before the next callback is handled.
 
 The callback will be called once, before QUnit runs any tests.
 
 | parameter | description |
 |-----------|-------------|
-| callback (function) | Callback to execute. Provides a single argument with the callback Details object |
+| `callback` (function) | Callback to execute, called with a `details` object. |
 
 ### Details object
 
-Passed to the callback:
-
 | property | description |
 |-----------|-------------|
-| `totalTests` | The number of total tests in the test suite |
+| `totalTests` (number) | Number of registered tests |
+| `modules` (array) | List of registered modules,<br>as  `{ name: string }` objects. |
+
+## Changelog
+
+| [QUnit 1.16](https://github.com/qunitjs/qunit/releases/tag/1.16.0) | Added `details.modules` property, containing `{ name: string }` objects.
+| [QUnit 1.15](https://github.com/qunitjs/qunit/releases/tag/1.15.0) | Added `details.totalTests` property.
 
 ## Examples
 

docs/callbacks/QUnit.done.md

@@ -1,7 +1,7 @@
 ---
 layout: page-api
 title: QUnit.done()
-excerpt: Register a callback to fire whenever the test suite ends.
+excerpt: Register a callback to fire when the test run has ended.
 groups:
   - callbacks
 redirect_from:
@@ -11,37 +11,42 @@ version_added: "1.0.0"
 
 `QUnit.done( callback )`
 
-Register a callback to fire whenever the test suite ends. The callback may be an async function, or a function that return a promise which will be waited for before the next callback is handled.
+Register a callback to fire when the test run has ended. The callback may be an async function, or a function that return a Promise which will be waited for before the next callback is handled.
 
 | parameter | description |
 |-----------|-------------|
-| callback (function) | Callback to execute. Provides a single argument with the callback Details object |
+| `callback` (function) | Callback to execute, called with a `details` object:
 
 ### Details object
 
-Passed to the callback:
-
 | property | description |
 |-----------|-------------|
-| `failed` (number) | The number of failed assertions |
-| `passed` (number) | The number of passed assertions |
-| `total` (number) | The total number of assertions |
-| `runtime` (number) | The time in milliseconds it took tests to run from start to finish. |
+| `failed` (number) | Number of failed assertions |
+| `passed` (number) | Number of passed assertions |
+| `total` (number) | Total number of assertions |
+| `runtime` (number) | Duration of the test run in milliseconds |
 
-## Examples
+<div class="note note--warning" markdown="1">
 
-Register a callback that logs test results to the console.
+Use of `details` is __deprecated__ and it's recommended to use [`QUnit.on('runEnd')`](./QUnit.on.md) instead.
 
-```js
-QUnit.done(details => {
-  console.log(
-    `Total: ${details.total} Failed: ${details.failed} ` +
-      `Passed: ${details.passed} Runtime: ${details.runtime}`
-  );
-});
-```
+Caveats:
+
+* This callback reports the **internal assertion count**.
+
+* The default browser and CLI interfaces for QUnit and other popular test frameworks, and most CI integrations, report the number of tests. Reporting the number _assertions_ may be confusing to developers.
+
+* Failed assertions of a [`test.todo()`](../QUnit/test.todo.md) test are reported exactly as such. While rare, this means that a test run and all tests within it may be reported as passing, while internally there were some failed assertions. Unfortunately, this internal detail is exposed for compatibility reasons.
+
+</div>
+
+## Changelog
+
+| [QUnit 2.2](https://github.com/qunitjs/qunit/releases/tag/2.2.0) | Deprecate `details` parameter in favour of `QUnit.on('runEnd')`.
+
+## Examples
 
-Using classic ES5 syntax:
+Register a callback that logs internal assertion counts.
 
 ```js
 QUnit.done(function (details) {

docs/callbacks/QUnit.log.md

@@ -13,8 +13,6 @@ version_added: "1.0.0"
 
 Register a callback to fire whenever an assertion completes.
 
-This is one of several callbacks QUnit provides. It's intended for continuous integration scenarios.
-
 **NOTE: The QUnit.log() callback does not handle promises and MUST be synchronous.**
 
 | parameter | description |

docs/callbacks/QUnit.on.md

@@ -9,20 +9,127 @@ version_added: "2.2.0"
 
 `QUnit.on( eventName, callback )`
 
-Register a callback to fire whenever the specified event is emitted. Conforms to the [js-reporters standard](https://github.com/js-reporters/js-reporters).
+Register a callback to fire whenever a specified event is emitted.
 
-Use this to listen for events related to the test suite's execution. Available event names and corresponding data payloads are defined in the [js-reporters specification](https://github.com/js-reporters/js-reporters).
+This API implements the [js-reporters CRI standard](https://github.com/js-reporters/js-reporters/blob/v2.1.0/spec/cri-draft.adoc), and is the primary interface for use by continuous integration plugins and other reporting software.
 
-**NOTE: The QUnit.on() callback does not handle promises and MUST be synchronous.**
+| type | parameter | description
+|--|--|--
+| `string` | `eventName` | Name of an event.
+| `Function` | `callback`| A callback function.
 
-| parameter | description |
-|-----------|-------------|
-| eventName (string) | The name of the event for which to execute the provided callback. |
-| callback (function) | Callback to execute. Receives a single argument representing the data for the event. |
+## The `runStart` event
 
-## Examples
+The `runStart` event indicates the beginning of a test run. It is emitted exactly once, and before any other events.
 
-Printing results of a test suite.
+| `Object` | `testCounts` | Aggregate counts about tests.
+| `number` | `testCounts.total` | Total number of registered tests.
+
+```js
+QUnit.on('runStart', runStart => {
+  console.log(`Test plan: ${runStart.testCounts.total}`);
+});
+```
+## The `suiteStart` event
+
+The `suiteStart` event indicates the beginning of a module. It is eventually be followed by a corresponding `suiteEnd` event.
+
+| `string` | `name` | Name of the module.
+| `Array<string>` | `fullName`| List of one or more strings, containing (in order) the names of any ancestor modules and the name of the current module.
+
+```js
+QUnit.on('suiteStart', suiteStart => {
+  console.log('suiteStart', suiteStart);
+  // name: 'my module'
+  // fullName: ['grandparent', 'parent', 'my module']
+});
+```
+
+## The `suiteEnd` event
+
+The `suiteEnd` event indicates the end of a module. It is emitted after its corresponding `suiteStart` event.
+
+| `string` | `name` | Name of the module.
+| `Array<string>` | `fullName`| List of one or more strings, containing (in order) the names of any ancestor modules and the name of the current module.
+| `string` | `status` | Aggregate result of tests in this module, one of:<br>`failed`: at least one test has failed; <br>`passed`: there were no failing tests, which means there were only tests with a passed, skipped, or todo status.
+| `number` | `runtime` | Duration of the module in milliseconds.
+
+```js
+QUnit.on('suiteEnd', suiteEnd => {
+  console.log(suiteEnd);
+  // …
+});
+```
+
+## The `testStart` event
+
+The `testStart` event indicates the beginning of a test. It is eventually followed by a corresponding `testEnd` event.
+
+| `string` | `name` | Name of the test.
+| `string|null` | `moduleName` | The module the test belongs to, or null for a global test.
+| `Array<string>` | `fullName` | List (in order) of the names of any ancestor modules and the name of the test itself.
+
+```js
+QUnit.on('testStart', testStart => {
+  console.log(testStart);
+  // name: 'my test'
+  // moduleName: 'my module'
+  // fullName: ['parent', 'my module', 'my test']
+
+  // name: 'global test'
+  // moduleName: null
+  // fullName: ['global test']
+});
+```
+
+## The `testEnd` event
+
+The `testEnd` event indicates the end of a test. It is emitted after its corresponding `testStart` event.
+
+Properties of a testEnd object:
+
+| `string` | `name` | Name of the test.
+| `string|null` | `moduleName` | The module the test belongs to, or null for a global test.
+| `Array<string>` | `fullName` | List (in order) of the names of any ancestor modules and the name of the test itself.
+| `string` | `status` | Result of the test, one of:<br>`passed`: all assertions passed or no assertions found;<br>`failed`: at least one assertion failed or it is a [todo test](../QUnit/test.todo.md) that no longer has any failing assertions;<br>`skipped`: the test was intentionally not run; or<br>`todo`: the test is "todo" and still has a failing assertion.
+| `number` | `runtime` | Duration of the test in milliseconds.
+| `Array<FailedAssertion>` | `errors` | For tests with status `failed` or `todo`, there will be at least one failed assertion. However, the list may be empty if the status is `failed` due to a "todo" test having no failed assertions.<br><br>Note that all negative test outcome communicate their details in this manner. For example, timeouts, uncaught errors, and [global pollution](../config/noglobals.md) also synthesize a failed assertion.
+
+Properties of a FailedAssertion object:
+
+| `boolean` | `passed` | False for a failed assertion.
+| `string|undefined` | `message` | Description of what the assertion checked.
+| `any` | `actual` | The actual value passed to the assertion.
+| `any` | `expected` | The expected value passed to the assertion.
+| `string|undefined` | `stack` | Stack trace, may be undefined if the result came from an old web browsers.
+
+```js
+QUnit.on('testEnd', testEnd => {
+  if (testEnd.status === 'failed') {
+    console.error('Failed! ' + testEnd.fullName.join(' > '));
+    testEnd.errors.forEach(assertion => {
+      console.error(assertion);
+      // message: speedometer
+      // actual: 75
+      // expected: 88
+      // stack: at dmc.test.js:12
+    });
+  }
+});
+```
+
+## The `runEnd` event
+
+The `runEnd` event indicates the end of a test run. It is emitted exactly once.
+
+| `string` | `status` | Aggregate result of all tests, one of:<br>`failed`: at least one test failed or a global error ocurred;<br>`passed`: there were no failed tests, which means there were only tests with a passed, skipped, or todo status. If [`QUnit.config.failOnZeroTests`](../config/failOnZeroTests.md) is disabled, then the run may also pass if there were no tests.
+| `Object` | `testCounts` | Aggregate counts about tests:
+| `number` | `testCounts.passed` | Number of passed tests.
+| `number` | `testCounts.failed` | Number of failed tests.
+| `number` | `testCounts.skipped` | Number of skipped tests.
+| `number` | `testCounts.todo` | Number of todo tests.
+| `number` | `testCounts.total` | Total number of tests, equal to the sum of the above properties.
+| `number` | `runtime` | Total duration of the run in milliseconds.
 
 ```js
 QUnit.on('runEnd', runEnd => {

docs/extension/QUnit.dump.parse.md

@@ -3,7 +3,7 @@ layout: page-api
 title: QUnit.dump.parse()
 excerpt: Extensible data dumping and string serialization.
 groups:
- - extension
+  - extension
 redirect_from:
   - "/QUnit.dump.parse/"
   - "/QUnit.jsDump.parse/"

docs/extension/QUnit.extend.md

@@ -3,8 +3,8 @@ layout: page-api
 title: QUnit.extend()
 excerpt: Copy the properties from one object into a target object.
 groups:
-- extension
-- deprecated
+  - extension
+  - deprecated
 redirect_from:
   - "/config/QUnit.extend/"
 version_added: "1.0.0"
@@ -15,13 +15,13 @@ version_deprecated: "2.12.0"
 
 Copy the properties defined by a mixin object into a target object.
 
+<p class="note note--warning" markdown="1">This method is __deprecated__ and it's recommended to use [`Object.assign()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign) instead.</p>
+
 | name | description |
 |------|-------------|
 | `target` | An object whose properties are to be modified |
 | `mixin` | An object describing which properties should be modified |
 
-<p class="note note--warning" markdown="1">This method is __deprecated__ and it's recommended to use [`Object.assign()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/assign) instead.</p>
-
 This method will modify the `target` object to contain the "own" properties defined by the `mixin`. If the `mixin` object specifies the value of any attribute as `undefined`, this property will instead be removed from the `target` object.
 
 ## Examples

docs/extension/QUnit.push.md

@@ -3,8 +3,8 @@ layout: page-api
 title: QUnit.push()
 excerpt: Report the result of a custom assertion.
 groups:
-- extension
-- deprecated
+  - extension
+  - deprecated
 redirect_from:
   - "/config/QUnit.push/"
 version_added: "1.0.0"
@@ -15,15 +15,15 @@ version_deprecated: "2.1.0"
 
 Report the result of a custom assertion.
 
+<p class="note note--warning" markdown="1">This method is __deprecated__ and it's recommended to use [`pushResult`](../assert/pushResult.md) in the assertion context instead.</p>
+
 | name | description |
 |------|-------------|
 | `result` (boolean) | Result of the assertion |
 | `actual` | Expression being tested |
 | `expected` | Known comparison value |
 | `message` (string) | A short description of the assertion |
 
-<p class="note note--warning" markdown="1">This method is __deprecated__ and it's recommended to use [`pushResult`](../assert/pushResult.md) in the assertion context instead.</p>
-
 `QUnit.push` reflects to the current running test, and it may leak assertions in asynchronous mode. Checkout [`assert.pushResult()`](../assert/pushResult.md) to set a proper custom assertion.
 
 Invoking `QUnit.push` allows to create a readable expectation that is not defined by any of QUnit's built-in assertions.