Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate the Python stubs automatically #1661

Draft
wants to merge 18 commits into
base: master
Choose a base branch
from

Conversation

dgelessus
Copy link
Contributor

@dgelessus dgelessus commented Feb 1, 2025

Closes #1660. Adds a script to automatically generate the Python "stub" files under Scripts/Python/plasma. The script must be run from the in-game Python console:

>>> __import__("generate_stubs").run()

This will generate stubs for all Plasma built-in modules into a subfolder "plasma_stubs_generated" under the game folder.

Everything happens fully automatically using runtime introspection. Only one part is "semi-manual": because Python cannot introspect the signatures of C-defined functions, all function parameters must be specified manually in the C++-defined function docstrings, on a line starting with Params: . This convention was apparently used by Cyan originally, so most function docstrings already follow this format.

The generator itself is almost completely finished, but this PR still needs a bit of work:

  • The generator currently removes all type annotations.
    • This will be mostly fixed by copying the parameter types from the existing stubs into the Params: lines in the C++-defined docstrings.
    • There's no way to specify type annotations for function return types and attribute/property types. Most likely, I'll implement this using another docstring line prefix, such as Returns: .
  • Some C++-defined docstrings are outdated, incorrect, slightly misformatted, or don't match the stubs, leading to spurious and incorrect changes.
  • The existing stubs are sometimes not sorted alphabetically. I'll fix this in a separate PR (Sort Python stubs consistently #1662), so that the diff of this PR will become more readable/meaningful.

With those things in mind, early feedback is welcome. I'm particularly interested in the opinion of @Hoikas, who has recently hand-written some stubs that will be overwritten by this script.

Except where class inheritance requires a different order.
And then in alphabetical order, if two enum constants have the same
numeric value.
@dgelessus dgelessus force-pushed the python_interface_stub_generator branch 2 times, most recently from 44284b9 to c4766ec Compare February 2, 2025 02:52
@dpogue
Copy link
Member

dpogue commented Feb 2, 2025

The generated signature for PtLocalizedYesNoDialog is resulting in Python syntax errors (it has a / as an argument, which I think is supposed to have special meaning and not show up as an argument itself?)

@dgelessus
Copy link
Contributor Author

I think / in parameter lists is allowed in real Python code since... some recent-ish version of Python 3. The actual syntax error is that the / appears after a *args (/ must come before *). This should get fixed along the way once I get type annotations working, because the manually type-annotated version of PtLocalizedYesNoDialog in the stub doesn't have this syntax error.

@dgelessus dgelessus force-pushed the python_interface_stub_generator branch from c4766ec to e76a5a8 Compare February 2, 2025 15:13
Fixes CI errors because of the stub incorrectly indicating that these
keyword arguments are required.
@dgelessus
Copy link
Contributor Author

Type annotations are now supported by the stub generator. I've introduced a new Type: docstring prefix, which works similarly to the existing Params: prefix, but allows specifying a full function signature including a return type annotation (example: Type: (x: int, y: int) -> int) or the type of a property getter (example: Type: Tuple[str, int]).

I've also brought all of the C++-defined docstrings up-to-date with the manual changes and added type annotations in the stubs. This should resolve all of the spurious changes.

This PR is basically finished now. I'll un-draft it once #1662 is merged and I've rebased this PR onto that (so that this PR's diff is more readable for reviewing).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants