Jupyter widget docs (#1786)

* Update scipy intersphinx path

* Save widget state in molecule cookbook

* Add notebooks section to developers guide

* Update changelog for #1786

* Move h5py warning to hidden notebook

* Skip testing warnings in hidden cookbook cells

* Re-run notebook

* Update MDTraj section to use MDTraj

* Fix bad merges

* More gracefully fail QCSchema test when OpenEye conflict encountered

https://docs.pytest.org/en/7.1.x/how-to/doctest.html#skipping-tests

* Go back to older MDTraj link even though it points to garbage

---------

Co-authored-by: Matthew W. Thompson <[email protected]>

.github/workflows/CI.yml

@@ -169,7 +169,11 @@ jobs:
       - name: Run examples in docstrings
         if: ${{ matrix.rdkit == true && matrix.openeye == true }}
         run: |
-          pytest -v --no-cov --doctest-modules --ignore-glob='openff/toolkit/_tests/test_*' --ignore=openff/toolkit/data/ --ignore=openff/toolkit/utils/utils.py openff/
+          pytest openff \
+            -v -x -n logical --no-cov --doctest-modules \
+            --ignore-glob='openff/toolkit/_tests*' \
+            --ignore=openff/toolkit/data/ \
+            --ignore=openff/toolkit/utils/utils.py
 
       - name: Codecov
         uses: codecov/codecov-action@v4

docs/conf.py

@@ -74,7 +74,7 @@
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3.7", None),
     "numpy": ("https://numpy.org/doc/stable", None),
-    "scipy": ("https://docs.scipy.org/doc/scipy/reference", None),
+    "scipy": ("https://docs.scipy.org/doc/scipy", None),
     "scikit.learn": ("https://scikit-learn.org/stable", None),
     "openmm": ("http://docs.openmm.org/latest/api-python/", None),
     "rdkit": ("https://www.rdkit.org/docs", None),
@@ -117,7 +117,7 @@
     "replacements",
     "deflist",
 ]
-myst_heading_anchors = 3
+myst_heading_anchors = 4
 
 # Myst NB settings
 # Never execute notebooks - this should be done by CI

docs/releasehistory.md

@@ -20,6 +20,7 @@ Releases follow the `major.minor.micro` scheme recommended by [PEP440](https://w
 
 - [PR #1795](https://github.com/openforcefield/openff-toolkit/pull/1795): Add `NAGLToolkitWrapper` to API reference
 - [PR #1796](https://github.com/openforcefield/openff-toolkit/pull/1796): Update docs tooling and fix warnings
+- [PR #1786](https://github.com/openforcefield/openff-toolkit/pull/1786): Describe contributing to documentation notebooks in developers guide.
 
 ## 0.15.2
 
@@ -61,7 +62,6 @@ This release adds compatibility with QCFractal >=0.50.0, but removes compatibilt
 
 
 ### Improved documentation and warnings
-
 - [PR #1732](https://github.com/openforcefield/openff-toolkit/pull/1732): Add documentation describing the use of PDB files with the toolkit.
 - [PR #1804](https://github.com/openforcefield/openff-toolkit/pull/1804): Makes `ForceField.parse_sources` docstring consistent with implementation.
 

docs/users/developing.md

@@ -377,7 +377,7 @@ If `pre-commit` is not used by the developer and style issues are found, a `pre-
 
 Note that tests (too slow) and type-checking (weird reasons) are not run by `pre-commit`. You should still manually run tests before commiting code.
 
-## Checking code coverage locally
+### Checking code coverage locally
 
 Run something like
 
@@ -388,6 +388,46 @@ $ open htmlcov/index.html
 
 to see a code coverage report. This uses [Coverage.py](https://coverage.readthedocs.io/) and also requires a pytest plugin (`pytest-cov`), both of which should be installed if you are using the provided conda environments. CodeCov provides this as an automated service with their own web frontend using a similar file generated by our CI. However, it can be useful to run this locally to check coverage changes (i.e. "is this function I added sufficiently coverged by tests?") without waiting for CI to complete.
 
+### Jupyter Notebooks in Rendered Docs
+
+We sometimes include [Jupyter Notebooks] in our book-style documentation. These are files with the `.ipynb` extension. Notebooks allow sections of the docs that include a large number of code examples to be tested easily, and support automatically rendering example output, including interactive [Jupyter widgets] such as the 3D molecule representations in the [molecule cookbook]. Note that the following information only applies to notebooks in the `docs` directory; notebooks in the `examples` directory are not included in the Toolkit docs (but may be included in [OpenFF Examples] via a different process).
+
+:::{admonition} Information
+For interactive widgets to appear in rendered documentation, they must be stored in the notebook each time it is re-executed. For information on how to do this, see [](#widgets).
+:::
+
+[Jupyter Notebooks]: https://jupyter.org/
+[Jupyter widgets]: https://ipywidgets.readthedocs.io/
+[molecule cookbook]: molecule_cookbook.ipynb#from-pdb-file
+[OpenFF Examples]: https://docs.openforcefield.org/examples
+
+#### MyST Markdown
+
+Documentation notebooks are rendered with the [MyST-NB] Sphinx extension, which provides the expanded Markdown syntax used in other Markdown files in this documentation. Admonitions, maths, and links to API references and other documentation pages can be included in the usual ways. This means that some correct Markdown will appear incorrect in Jupyter by default, as it uses a simpler Markdown dialect. To render MyST syntax, you can install the [`jupyterlab-myst`] Jupyter extension:
+
+```shell
+mamba install -c conda-forge jupyterlab-myst
+```
+
+Note that some features that will render correctly in Sphinx rely on extensions that are not available to `jupyterlab-myst`, and so may still appear incorrect even with the extension.
+
+[MyST-NB]: https://myst-nb.readthedocs.io/
+[`jupyterlab-myst`]: https://mystmd.org/guide/quickstart-jupyter-lab-myst
+
+#### Widgets
+
+Jupyter Notebooks can produce interactive [widgets] as output, and MyST-NB supports rendering them in HTML output. Any given widget includes both Python code and JavaScript code, and Jupyter defines ways for them to speak to each other which allows Python code to be run interactively via the widget in the notebook. 
+
+While the JavaScript components are embedded in the page by Sphinx and can run in the browser's JavaScript engine, Sphinx web pages do not include a Python runtime. This has two effects. Firstly, any interactivity that relies on the Python runtime will not work on the web. This is usually little more than an inconvenience for purpose-written documentation notebooks, but does explain why [NGLView trajectories in OpenFF Examples] cannot be played on the web. 
+
+Secondly, widget initialization happens in Python, so any state shared between Python and JavaScript must be included in the notebook itself before Sphinx can render it. Any time these notebooks are re-executed, the internal IDs of the widgets may change, causing the widget to forget its original state and be missing from the rendered documentation. To fix this, it's essential to ask Jupyter to **save widget state**. In Jupyter Notebook 7+ and Jupyter Lab, this option appears under `Settings -> Save Widget State Automatically`, which if ticked will save widget state when the notebook is saved. Note that this menu item will not appear if the open notebook has no widgets. In earlier versions of Notebook, the `Widgets -> Save Widget State` menu item must be selected any time the notebook is re-executed and saved.
+
+Widget state can also be saved programmatically by executing the notebook with [NBConvert], though since NBConvert does not include a JavaScript runtime the widget must be able to initialize itself from stored state. This is not possible, for example, in NGLView before version 3.0.6.
+
+[widgets]: https://ipywidgets.readthedocs.io/
+[NGLView trajectories in OpenFF Examples]:https://docs.openforcefield.org/en/latest/examples/openforcefield/openff-toolkit/toolkit_showcase/toolkit_showcase.html#visualize-the-simulation-nglview
+[NBConvert]: https://nbconvert.readthedocs.io/en/latest/execute_api.html#widget-state
+
 ## Supported Python versions
 
 The OpenFF Toolkit roughly follows [NEP 29](https://numpy.org/neps/nep-0029-deprecation_policy.html).