feat(cargo-pgrx): add support for downloading specific pg versions

[t3hmrman]

This commit adds support for downloading a specific version of Postgres when running cargo pgrx init via tarball.

Some example invocations would look like:

• cargo pgx init --pg12=12.6

Where as previously you could only provide a path to an existing pg_config from a Postgres installation (which you needed to place yourself), with this PR we can now specify either the version or a tarball.

This PR does not solve the problem of allowing any URL to be used as a download URL, but instead only uses https://ftp.postgresql.org derived URLs.

Resolves #149

[eeeebbbbrrrr]

Can you explain a bit about why it's ever necessary to use a specific point release?

I'm not saying that it's wrong to want to, only that my experience is such that I only ever want to move forward to the "latest" point release. I don't recall ever needing to install a specific point release.

[t3hmrman]

So for earlier logic I'd defer to the earlier issue #149 -- I believe they wanted it for nix support.

Personally for the little extension I manage (https://github.com/VADOSWARE/pg_idkit), I like being specific about point releases because I can limit my support to let's say "the last 3 Postgres minor releases" or something like that.

Technically, I can achieve this by installing the specific version, and not running pgrx init... But I've found that running pgrx init was a LOT less error prone for me.

Right now, every time I try to run my automated release generation pipelines, if the latest version that is automatically pulled down by pgrx has changed, I need to jump in and update some matrix variables in my CI setup. This is necessary mostly because I find the files on disk and package them/do other things with them.

[workingjubilee]

@t3hmrman I think we basically have a similar concern in that we don't want to extend support for using pgrx with random point releases.

[workingjubilee]

@t3hmrman I would prefer it if specifying a point release came as a separate argument, to

• discourage doing it needlessly
• to make it possible to document that the minor version used does not affect the rest of pgrx

This will mean people will have to correctly split the major and minor version on their own and that's fine, I think. It might actually be useful to specify this argument in such a way that it allows e.g. easily building against random patch versions of Postgres, as well, but that's pure extra.

[eeeebbbbrrrr]

My main concern is that pgrx is feature-flagged by major version and I don’t see moving to feature-flags-per-point-release being workable.

I realize that’s NOT what this PR does and is not the intent here. But I’ve been around long enough to know what can happen when someone leaves a door cracked open.

I don’t want to block this from being merged, per se. But in the future my response to some complaint about point release source code incompatibilities is going to be “just use the latest point release”.

[workingjubilee]

My biggest concern is probably inevitably handling Postgres 17 beta 1.

[eeeebbbbrrrr]

We’d handle that like we did for 16 and explicitly not let users change the beta number themselves during beta

[workingjubilee]

Honestly, I'm not sure why we should be in the business of handling beta versions explicitly?

[eeeebbbbrrrr]

We set that precedent with the 16 betas. Certain cloud providers offer the betas and want/need/desire their pgrx extensions to work. And there can be internal api changes between betas.

[eeeebbbbrrrr]

During the beta/rc phase of a new pg release we just need to present whatever the latest of that is as the major version number and not allow users to switch around.

[t3hmrman]

hey @eeeebbbbrrrr and @workingjubilee I've got a sketch up of what the code would look like with the minor version as a separate argument -- if the changes look reasonable and it's what we want then I'll add a test!

[t3hmrman]

Well that was a bit of a wild ride -- the code looks quite a bit different now, had to change a few things and do some refactoring.

Here's What the output of the different combinations looks like:

Dowloading the latest PG15:

cargo pgrx init --pg15=download
   Discovered Postgres v12.18, v13.14, v14.11, v15.6, v16.2
  Downloading Postgres v15.6 from https://ftp.postgresql.org/pub/source/v15.6/postgresql-15.6.tar.bz2
    Untarring Postgres v15.6 to /tmp/pgrx-init-testing/15.6_unpack
     Renaming /tmp/pgrx-init-testing/15.6_unpack/postgresql-15.6 -> /tmp/pgrx-init-testing/15.6
  Configuring Postgres v15.6
    Compiling Postgres v15.6
   Installing Postgres v15.6 to /tmp/pgrx-init-testing/15.6/pgrx-install
   Validating /tmp/pgrx-init-testing/15.6/pgrx-install/bin/pg_config
 Initializing data directory at /tmp/pgrx-init-testing/data-15

Downloading a point release for 15:

cargo pgrx init --pg15=download --pg15-minor-version=5
  Downloading Postgres v15.5 from https://ftp.postgresql.org/pub/source/v15.5/postgresql-15.5.tar.bz2
    Untarring Postgres v15.5 to /tmp/pgrx-init-testing/15.5_unpack
     Renaming /tmp/pgrx-init-testing/15.5_unpack/postgresql-15.5 -> /tmp/pgrx-init-testing/15.5
  Configuring Postgres v15.5
    Compiling Postgres v15.5
   Installing Postgres v15.5 to /tmp/pgrx-init-testing/15.5/pgrx-install
   Validating /tmp/pgrx-init-testing/15.5/pgrx-install/bin/pg_config

Using a pre-existing on-disk pg_config:

cargo pgrx init --pg15=~/.pgrx/15.4/pgrx-install/bin/pg_config
   Validating /tmp/pgrx-init-testing/15.4/pgrx-install/bin/pg_config

Downloading a beta release for 16:

cargo pgrx init --pg16=download --pg16-minor-version=beta3
Error:
   0: managed download not supported for Rc and Beta versions

Location:
   cargo-pgrx/src/command/init.rs:128

  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ SPANTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

   0: cargo_pgrx::command::init::execute
      at cargo-pgrx/src/command/init.rs:246

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

Using regular init still discovers and sets up everything:

cargo pgrx init
   Discovered Postgres v12.18, v13.14, v14.11, v15.6, v16.2
  Downloading Postgres v15.6 from https://ftp.postgresql.org/pub/source/v15.6/postgresql-15.6.tar.bz2
  Downloading Postgres v14.11 from https://ftp.postgresql.org/pub/source/v14.11/postgresql-14.11.tar.bz2
  Downloading Postgres v16.2 from https://ftp.postgresql.org/pub/source/v16.2/postgresql-16.2.tar.bz2
  Downloading Postgres v13.14 from https://ftp.postgresql.org/pub/source/v13.14/postgresql-13.14.tar.bz2
  Downloading Postgres v12.18 from https://ftp.postgresql.org/pub/source/v12.18/postgresql-12.18.tar.bz2
    Untarring Postgres v13.14 to /tmp/pgrx-init-testing/13.14_unpack
     Renaming /tmp/pgrx-init-testing/13.14_unpack/postgresql-13.14 -> /tmp/pgrx-init-testing/13.14
  Configuring Postgres v13.14
    Untarring Postgres v12.18 to /tmp/pgrx-init-testing/12.18_unpack
     Removing /tmp/pgrx-init-testing/15.6_unpack
    Untarring Postgres v15.6 to /tmp/pgrx-init-testing/15.6_unpack
    Untarring Postgres v14.11 to /tmp/pgrx-init-testing/14.11_unpack
     Renaming /tmp/pgrx-init-testing/12.18_unpack/postgresql-12.18 -> /tmp/pgrx-init-testing/12.18
  Configuring Postgres v12.18
     Removing /tmp/pgrx-init-testing/15.6
    Untarring Postgres v16.2 to /tmp/pgrx-init-testing/16.2_unpack
     Renaming /tmp/pgrx-init-testing/15.6_unpack/postgresql-15.6 -> /tmp/pgrx-init-testing/15.6
  Configuring Postgres v15.6
     Renaming /tmp/pgrx-init-testing/14.11_unpack/postgresql-14.11 -> /tmp/pgrx-init-testing/14.11
  Configuring Postgres v14.11
     Renaming /tmp/pgrx-init-testing/16.2_unpack/postgresql-16.2 -> /tmp/pgrx-init-testing/16.2
  Configuring Postgres v16.2
    Compiling Postgres v13.14
    Compiling Postgres v12.18
    Compiling Postgres v16.2
    Compiling Postgres v15.6
    Compiling Postgres v14.11
   Installing Postgres v13.14 to /tmp/pgrx-init-testing/13.14/pgrx-install
   Installing Postgres v12.18 to /tmp/pgrx-init-testing/12.18/pgrx-install
   Installing Postgres v15.6 to /tmp/pgrx-init-testing/15.6/pgrx-install
   Installing Postgres v14.11 to /tmp/pgrx-init-testing/14.11/pgrx-install
   Installing Postgres v16.2 to /tmp/pgrx-init-testing/16.2/pgrx-install
   Validating /tmp/pgrx-init-testing/12.18/pgrx-install/bin/pg_config
 Initializing data directory at /tmp/pgrx-init-testing/data-12
   Validating /tmp/pgrx-init-testing/13.14/pgrx-install/bin/pg_config
 Initializing data directory at /tmp/pgrx-init-testing/data-13
   Validating /tmp/pgrx-init-testing/14.11/pgrx-install/bin/pg_config
 Initializing data directory at /tmp/pgrx-init-testing/data-14
   Validating /tmp/pgrx-init-testing/15.6/pgrx-install/bin/pg_config
   Validating /tmp/pgrx-init-testing/16.2/pgrx-install/bin/pg_config
 Initializing data directory at /tmp/pgrx-init-testing/data-16

[t3hmrman]

Sorry ya'll bad merge (I touched the toast code somehow) -- would appreciate a workflow re-run (@eeeebbbbrrrr/@workingjubilee)

[t3hmrman]

Hey is it reasonable to take this feature? Been working on updating pg_idkit for PG 17 and would be nice to be able to lock in the specific version!

I'd like to get some confirmation before I try and address the conflicts.

[workingjubilee]

hello.

[workingjubilee]

I believe Postgres has suddenly motivated this fully.

[eeeebbbbrrrr]

I believe Postgres has suddenly motivated this fully.

Maybe. Maybe the status quo is okay considering this is the first time in decade something like this has happened.

“Use the latest point release” is still good advice.