Read project metadata from pyproject.toml

[stratofax]

Single source of truth!

One possible solution: create an __about__.py file that uses importlib.metadata to get the version and package_name from the poetry-generated environment. Then update constants.py to read the values from __about__.py instead of updating them by hand. This way, we don't have to rewrite code that reads the values from constants.py but we can effectively read the values from pyproject.toml.

[stratofax]

More here, from an issue on the poetry repo: How to get version from pyproject.toml from python app? · Issue #273 · python-poetry/poetry

[ion-oset]

@stratofax said:

I think I may have to use poetry build to create the necessary files to read version and package name from pyproject.toml -- in fact, it seems like poetry uses pyproject.toml to generate the files (like, say, setup.py) that importlib.metadata reads.

It does generate the files but we do not need poetry build. If you run poetry install in a virtualenv it does an an editable install (aka development mode) by default. This treats the package as if itself was installed externally. This is what allows from electos.ballotmaker to work when running in the virtualenv, and prevents subtle bugs between the installed and development versions. It is how we are supposed to use (and are using) Poetry, no special steps required.

The example in the post below shows how we can mostly just use importlib.metadata, and I think is this mostly preferable being standard and likely to stay supported over the long haul. But if we needed to pull data directly out of pyproject.toml we can do it by using one of the TOML libraries (though that imposes a dependency on a library not in stdlib prior to Python 3.11 - see PEP-680). Here's an example.

import tomllib

def _load_project_metadata(project_root):
    project_file = project_root / "pyproject.toml"
    with open(project_file) as file:
        data = tomllib.load(file)
        metadata = data['tool']['poetry']
    return metadata

[ion-oset]

In a Python 3.8 virtualenv for BallotLab where poetry install has been run this works for me, with some caveats noted below.

import importlib.metadata

package = "ballotlab"
dist = importlib.metadata.distribution(package)
meta = dist.metadata

properties = {
    "__package_name__":  "name",
    "__version__":       "version",
    "__author__":        "author",
    "__email__":         "author-email",
    "__description__":   "description",
    "__license__":       "license",
}

for key, value in properties.items():
    print(f"{key:<16} = {meta[value]}")

Caveats:

• We can't use __name__ or __package__ for the package name (the "name" metadata). Both are defined as part of the Python import machinery.
• __description__ is None because the fields allowed in project metadata are restricted to what's described in the Python Packaging Core Metadata Specification and there's no property called description.
  • Poetry requires it, but that choice makes it inconsistent with the core metadata.
• There's no official list of __dunder__ variables names. Avoiding collisions with ones defined elsewhere is left to us.
  • The only one that is effectively a (de facto) standard is __version__.

[stratofax]

I'm not sure we even need to use __dunder__ names if the goal is to assign these values to ALL_CAPS_CONSTANTS in constants.py, although assigning the version number to version may make some sense if it's a standard of sorts.

It looks like reading the version number is both possible and useful.

Do you want to implement this in constants.py?