It should be possible to error and exit the documentation build process if the execution of the notebooks raises an Exception

[rdbisme]

Describe the problem/need and solution

Context
We have lots of code cells that execute code that normally should run correctly but when API changes and we forgot to update the docs, the docs build process should failt to force us to fix it.

Problem / Idea
Right now in the execution.py errors are just logged, but build continues. Therefore the sphinx build process exit correctly even if the execution of code cells failed.

Solution
If execution_allow_errors = False, exceptions should stop the build

Benefit
This will avoid to push unreliable code in docs

Guide for implementation

• I have a PR following.

Tasks and updates

No response

[welcome]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! 🎉

[chrisjsewell]

Hey @rdbisme, I can't remember if this was possible before, but after #380, an execution error will create a mystnb.exec type warning, e.g.

docs/examples/interactive.md: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
docs/examples/interactive.md: WARNING: Notebook exception traceback saved in: docs/_build/html/reports/examples/interactive.err.log [mystnb.exec]

In sphinx, to fail on a warning, you can either use sphinx-build -W to halt on the first warning (with a non-zero exit code), or sphinx-build -W --keep-going to run until the end, but then return a non-zero exit code if any warnings were encountered.
This is exactly how we do the CI in this repo (see also https://docs.readthedocs.io/en/stable/config-file/v2.html#sphinx-fail-on-warning)

You can also use https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-suppress_warnings, if you have known warnings that you do not wish to fail on.

I feel that this should address your issue, rather than the non-standard (for sphinx) method of raising an exception proposed in #393?

[chrisjsewell]

Actually, sorted it in #404 😉

[rdbisme]

Thanks. So you added an option to trigger the Exception? Nice!