Unify the type preprocessing logic in Napoleon

[cbarrick]

Subject: Unify the type preprocessing logic in Napoleon

Feature or Bugfix

• Feature

Purpose

Allow Google-style docstrings to use the optional and default keywords described at https://numpydoc.readthedocs.io/en/latest/format.html#parameters

Detail

Previously, there were two separate type preprocessing functions: _convert_type_spec (used in Google-style docstrings) and _convert_numpy_type_spec (used in Numpy-style docstrings).

The Google version simply applied type-alias translations or wrapped the text in a :py:class: role.

The Numpy version does the same, plus adds special handling for keywords optional and default and delimiter words or, of, and and. This allows one to write in natural language, like Array of int instead of Array[int] or Widget, optional instead of Optional[Widget] or Widget | None. Numpy style is described in full at: https://numpydoc.readthedocs.io/en/latest/format.html#parameters

This commit eliminates the distinction and allows Google-style docstrings to use these preprocessing rules.

More details

The Numpy-style preprocessing code lived between the GoogleDocstring and NumpyDocstring classes. This was kinda an awkward location when the code would become shared by both classes.

This PR is broken into two commits. The first moves the code to a more palatable location. The second migrates GoogleDocstring to use the previously-named _convert_numpy_type_spec.

I think it will be much easier to review the diffs individually rather than the diff for the whole PR.

[cbarrick]

Friendly ping!

Any concerns with this change?

[cbarrick]

@AA-Turner Can you take a look at this PR, or route it to the most appropriate reviewer?

Am I missing anything from this change? Are there any concerns from the maintainers?

I am periodically rebasing this onto master to ensure it remains mergable.

I may be a tad unresponsive over the winter holidays. Happy to pick this back up in the new year.

[AA-Turner]

Hi Chris (@cbarrick),

Thank you for keeping this up-to-date -- we squash-merge PRs in general so merging master might be easier than rebasing, to save some time.

Likewise, thank you for the PR -- Napoleon is sadly a little neglected, so improvements here are greatly appreciated.

You note that the change here would serve to expand the X of Y etc syntax to Google style docstrings. The current Google style guide makes little mention of types-in-docstrings. Do you have particular thoughts here?

A

[cbarrick]

FWIW, I have no affiliation with the Python team at Google nor with any of Google's open source Python projects.

You are correct that the Google Python style guide does not mention a type-annotation syntax for docstrings. I don't really know the history here, but I wouldn't be surprised if this syntax was an addition by Napoleon.

I briefly took a look at both TensorFlow and Jax, two major open source projects that I know use Google-style docstrings. From my quick review of both projects, neither seems to be using this type annotation syntax.

The only major Python project I have found using this syntax is PyTorch, but (A) I haven't found any existing docs that would be broken by this change, and (B) their Sphinx conf.py doesn't set napoleon_preprocess_types anyway.

I did a search on GitHub for "napoleon_preprocess_types = True", and the largest project I could find using Google-style docstrings was Cleanlab. They don't seem to be using keywords like or in their docstrings, but they are using generic types like npt.NDArray[np.int_], and the processing of this is improved by this PR, because npt.NDArray and np.int_ will be identified as separate types instead of putting the whole thing into a single :py:class: that would fail to resolve to anything.

[cbarrick]

we squash-merge PRs in general so merging master might be easier than rebasing, to save some time.

Rebasing isn't a problem, because it's not like anyone else is working on Napoleon these days 😅. I've never had to resolve a conflict on this.

Regarding squash merges, I intentionally split this PR into two commits for cleaner diffs and history. Just something to consider if/when this gets merged.

[AA-Turner]

FWIW, I have no affiliation with the Python team at Google nor with any of Google's open source Python projects.

Fair enough, my fault for the assumption!

Thank you once again for the PR, it was a pleasure to review!

A