Skip to content

PEP 761: Deprecating PGP signatures #977

Open
@sethmlarson

Description

@sethmlarson

Hello, I noticed this project uses PGP signatures and wanted to make you all aware of my proposed PEP for deprecating PGP signatures in future Python versions. Please take a look and leave questions in the discussion thread.

Activity

tianon

tianon commented on Oct 9, 2024

@tianon
Member

Oof, not ideal, but something we can/will adapt to if it's adopted.

Thank you so much for the proactive heads up! It's really, really appreciated. ❤️

tianon

tianon commented on Oct 9, 2024

@tianon
Member

The more I look at the details, the more concerned I am about this -- sigstore clients are not super generally available (doesn't appear to have pre-existing Debian or Alpine packages, for example, and definitely not in bookworm/stable), especially before we've built/installed Python (which we need to verify before we do 😬), but even if it weren't, installing new packages from PyPI during our build is not something we're comfortable with (we just got away from installing pip from PyPI 😅). 🤔

To put that another way, this creates a new verification problem for us in that we need to somehow fetch/verify/trust a sigstore client as well. 😞

We'll also need to maintain a mapping of release to release manager + OIDC Issuer (https://www.python.org/downloads/metadata/sigstore/), which is similar to what we already maintain for PGP, so not a huge burden but definitely not easier for us. 😅
(Already addressed explicitly in https://discuss.python.org/t/pep-761-deprecating-pgp-signatures-for-cpython-artifacts/67180/6 though, which is nice ❤️)

(Some variant of this might belong in the upstream discussion thread, but I'm mostly trying to write down ideation about how we would implement this, so for now I'm keeping it local. I do not mind if I get quoted in the upstream thread if something here is useful there, though -- I'm "tianon" there too if that happens and the quoter wants to @ me.)

tianon

tianon commented on Oct 9, 2024

@tianon
Member

We could use sigstore-go, as suggested by the upstream documentation, but we'd have to compile it ourselves because (somewhat ironically) that project doesn't publish precompiled binaries (and solving the "verification" problem there is an interesting/fun case of chicken-and-egg).

tianon

tianon commented on Oct 9, 2024

@tianon
Member

As an affected downstream, it would honestly be more useful to us to have a published checksum that's stronger than MD5 and more machine-readable than what's listed in the HTML table on https://www.python.org/downloads/release/python-3130/ (I don't think that value lives anywhere under https://www.python.org/ftp/python/3.13.0/, for example, that we could easily scrape it from).

tianon

tianon commented on Oct 9, 2024

@tianon
Member

(Yes, checksum and signature verify different things -- however, the different things they verify are valuable separately, and the checksum is a stronger verification for our purposes than the signatures are.)

sethmlarson

sethmlarson commented on Oct 9, 2024

@sethmlarson
Author

@tianon That's good to know that stronger checksum value would be useful, there's an open issue for it and I plan to tackle it myself (upgrading to SHA256). Also: a feature that's different than PGP is that Sigstore can verify a checksum value and signature against an identity (rather than requiring the original binary blob).

tianon

tianon commented on Oct 9, 2024

@tianon
Member

Very nice, that would be awesome! I dug a little more and found that the SPDX files published do include a checksum of the published artifacts too, but it's really buried so it'll require some finagling to extract. 😅

$ wget -qO- 'https://www.python.org/ftp/python/3.13.0/python-3.13.0.exe.spdx.json' | jq '.packages[] | select(.packageFileName == "python-3.13.0.exe")'
{
  "SPDXID": "SPDXRef-PACKAGE-cpython",
  "checksums": [
    {
      "algorithm": "SHA256",
      "checksumValue": "a9be7082ccd3d0b947d14a87bceadb1a3551382a68fcb64d245a2ebcc779b272"
    }
  ],
  "downloadLocation": "https://www.python.org/ftp/python/3.13.0/python-3.13.0.exe",
  "externalRefs": [
    {
      "referenceCategory": "SECURITY",
      "referenceLocator": "cpe:2.3:a:python:python:3.13.0:*:*:*:*:*:*:*",
      "referenceType": "cpe23Type"
    }
  ],
  "licenseConcluded": "PSF-2.0",
  "name": "CPython",
  "originator": "Organization: Python Software Foundation",
  "packageFileName": "python-3.13.0.exe",
  "primaryPackagePurpose": "LIBRARY",
  "supplier": "Organization: Python Software Foundation",
  "versionInfo": "3.13.0"
}
tianon

tianon commented on Oct 9, 2024

@tianon
Member

Also: a feature that's different than PGP is that Sigstore can verify a checksum value and signature against an identity (rather than requiring the original binary blob).

Exploration of this thought gave me a simpler solution:

$ wget -qO- 'https://www.python.org/ftp/python/3.10.15/Python-3.10.15.tar.xz.sigstore' | jq -r '.messageSignature.messageDigest.digest' | base64 -d | hexdump -ve '/1 "%02x"'; echo
aab0950817735172601879872d937c1e4928a57c409ae02369ec3d91dccebe79

(extracting the SHA256 straight from the sigstore bundles, which lets me get values further back than the 3.12 that added SPDX files 👀)

tianon

tianon commented on Oct 9, 2024

@tianon
Member

... SHA256 verification based on that is now implemented in #978 😄

tianon

tianon commented on Oct 9, 2024

@tianon
Member

To reiterate for the avoidance of doubt, I don't think this solves the same problem as actually verifying the provenance of the builds too, but it does make it feel less urgent for us to find a good solution to the chicken-and-egg problem that getting a suitable sigstore client during our builds is (and ultimately we'd like to be verifying both).

sspans-sbp

sspans-sbp commented on Oct 18, 2024

@sspans-sbp
Contributor

Gave sigstore verification a first shot in #983

  • a suitable cosign version is only available in Alpine Edge
  • cosign is not packaged for Debian at all
  • verification using the python sigstore client doesn't seem like a great option either

Only other option would be to do the download/check in a separate stage first (not pretty).

tianon

tianon commented on Oct 18, 2024

@tianon
Member

Indeed, getting an appropriate sigstore client here (either one of the direct sigstore options or cosign) in a secure and verifiable way is going to be our biggest hurdle, especially since none of them are packaged in Debian yet (and it'd be years before we'd see them in a stable release even if we start that work today).

We could download and/or build one of them from source in our Dockerfile, but then we have the same chicken-and-egg problem with verifying those, and if we're still trusting them with just a checksum, then we might as well only use the checksums from Python (now that we've got them, thanks to #978).

As noted in #983 (comment), it's great that Alpine Edge has cosign, but that unfortunately won't help us until Alpine 3.21 (and that's assuming it does successfully land in 3.21 - packages in Edge often don't). Additionally, as discussed in #980 (comment), I've been looking at python/cpython#90548 again recently and the fact that Alpine (slash musl) isn't actually a supported upstream platform, and contemplating what we should do about our Alpine builds in the future, with not maintaining them anymore being one option that's on the table.

tianon

tianon commented on Oct 18, 2024

@tianon
Member

To put that more succinctly, I'm currently questioning whether we get enough value out of verifying the signatures to make it worth doing something here, especially since there's no straightforward short-term answer to the supply chain / "root of trust" problems. 😞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @tianon@sethmlarson@sspans-sbp

        Issue actions

          PEP 761: Deprecating PGP signatures · Issue #977 · docker-library/python