Skip to content

feat: multi-ext-versions-pgrcron #1549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 42 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
dc7b8cf
feat: multi-ext-versios-pgrcron
samrose Apr 15, 2025
278f67b
feat: add version to drv and patch instead of postPatch rewrite
samrose Apr 15, 2025
b53ab41
chore: newline
samrose Apr 15, 2025
3e565a7
feat: auto create multi version
samrose Apr 15, 2025
d48f7b7
chore: do not re-intro maintainers here not needed
samrose Apr 15, 2025
dbac632
feat: pg_cron version switcher in pkg and prestart
samrose Apr 24, 2025
adcc2e7
test: a tmp test for this branch to test older versions
samrose Apr 24, 2025
7f41631
test: temp test for ext handling
samrose Apr 24, 2025
75b6e53
test: instead of patch, add prior to start of machine in testinfra
samrose Apr 24, 2025
2d1aa55
test: only run on pg 15 for this test, as 1.3.1 limited to 15
samrose Apr 24, 2025
7b63c9f
feat: use jq from nixpkgs
samrose Apr 24, 2025
dd924e8
test: handle braces properly
samrose Apr 24, 2025
d136e4f
test: propagate errors from any failure
samrose Apr 25, 2025
e7c403c
test: more logging for healthcheck
samrose Apr 25, 2025
5e8c008
test: adding even more logging
samrose Apr 25, 2025
be713ac
test: extend logging more to see what happens when pg starts
samrose Apr 25, 2025
3e01f39
test: handle lib extension names per system
samrose Apr 28, 2025
6aa3376
test: more debugging
samrose Apr 28, 2025
b6322fd
test: move logging here
samrose Apr 28, 2025
138ab07
test: try direct
samrose Apr 28, 2025
141c3a0
test: use the right alias on machine
samrose Apr 28, 2025
ac4983a
test: do not unpack result
samrose Apr 29, 2025
de37a76
test: reorg and print logs while waiting continue on other checks whe…
samrose Apr 29, 2025
e85e36c
test: restructure checks to avoid race
samrose Apr 29, 2025
7f1c020
Update nix/ext/pg_cron.nix
samrose May 1, 2025
5cc0b2c
chore: refactor based on review
samrose May 2, 2025
ab1f694
fix: fixing broken elements of script
samrose May 5, 2025
ad6bffb
test: new flag and other fixes to testing
samrose May 13, 2025
8f1b895
test: pushing commit to register for nix build and testing
samrose May 19, 2025
9552777
feat: prestart detect version
samrose May 19, 2025
9c370c3
feat: provide jq correct
samrose May 19, 2025
2e9bd6f
test: rollback logging on test
samrose May 20, 2025
2e9c94a
test: trying with systemd restart
samrose May 20, 2025
475ca99
feat: fixing symlink issues, prestart bugs
samrose May 20, 2025
bb39e7f
fix: drill down in aliases to change core alias
samrose May 21, 2025
41e3fc5
chore: bump test vers.
samrose May 21, 2025
408b6ec
feat: handling symlinks
samrose May 21, 2025
064fbf0
chore: bump vers.
samrose May 21, 2025
2b09ce9
test: restore test to version from develop branch
samrose May 21, 2025
14d68b1
fix: dash instead of underscore
samrose May 21, 2025
4ee63ef
chore: bump dev version
samrose May 21, 2025
1c51a28
fix: use correct path for pg-extensions file
samrose May 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/testinfra-ami-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,13 +108,13 @@ jobs:
df -h / # Display available space

- name: Run tests
timeout-minutes: 10
timeout-minutes: 30
env:
AMI_NAME: "supabase-postgres-${{ steps.random.outputs.random_string }}"
run: |
# TODO: use poetry for pkg mgmt
pip3 install boto3 boto3-stubs[essential] docker ec2instanceconnectcli pytest pytest-testinfra[paramiko,docker] requests
pytest -vv -s testinfra/test_ami_nix.py
pytest -vvvv -s testinfra/test_ami_nix.py

- name: Cleanup resources on build cancellation
if: ${{ cancelled() }}
Expand Down
79 changes: 79 additions & 0 deletions ansible/files/postgres_prestart.sh.j2
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
#!/bin/bash

set -x # Print commands

log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
}

check_orioledb_enabled() {
local pg_conf="/etc/postgresql/postgresql.conf"
if [ ! -f "$pg_conf" ]; then
Expand All @@ -26,7 +32,78 @@ update_orioledb_buffers() {
fi
}

check_extensions_file() {
local extensions_file="/etc/adminapi/pg-extensions.json"
if [ ! -f "$extensions_file" ]; then
log "extensions: No extensions file found, skipping extensions versions check"
return 1
fi
if [ ! -r "$extensions_file" ]; then
log "extensions: Cannot read extensions file"
return 1
fi
return 0
}

get_pg_cron_version() {
if ! check_extensions_file; then
return
fi

local version
version=$(jq -r '.pg_cron // empty' "/etc/adminapi/pg-extensions.json")
if [ -z "$version" ]; then
log "pg_cron: Not specified in extensions file"
return
fi

if ! [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
log "pg_cron: Invalid version format: $version"
return
fi

# Log the version but don't include it in the output
log "pg_cron: Found version $version in extensions file" >&2
printf "%s" "$version"
}

switch_pg_cron_version() {
local version="$1"
local switch_script="/var/lib/postgresql/.nix-profile/bin/switch_pg_cron_version"

if [ ! -x "$switch_script" ]; then
log "pg_cron: No version switch script available at $switch_script"
return 1
fi

log "pg_cron: Switching to version $version"
# Run directly as root since we're already running as root
"$switch_script" "$version"
local exit_code=$?
if [ $exit_code -eq 0 ]; then
log "pg_cron: Version switch completed successfully"
else
log "pg_cron: Version switch failed with exit code $exit_code"
fi
return $exit_code
}

handle_pg_cron_version() {
local version
version=$(get_pg_cron_version)
if [ -n "$version" ]; then
# Don't fail if version switch fails
switch_pg_cron_version "$version" || log "pg_cron: Version switch failed but continuing"
fi
}

main() {
log "Starting prestart script"

# 1. pg_cron version handling
handle_pg_cron_version

# 2. orioledb handling
local has_orioledb=$(check_orioledb_enabled)
if [ "$has_orioledb" -lt 1 ]; then
return 0
Expand All @@ -35,6 +112,8 @@ main() {
if [ ! -z "$shared_buffers_value" ]; then
update_orioledb_buffers "$shared_buffers_value"
fi

log "Prestart script completed"
}

# Initial locale setup
Expand Down
2 changes: 2 additions & 0 deletions ansible/files/postgresql_config/postgresql.service.j2
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ TimeoutStopSec=90
TimeoutStartSec=86400
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
OOMScoreAdjust=-1000
EnvironmentFile=-/etc/environment.d/postgresql.env
LimitNOFILE=16384
Expand Down
6 changes: 6 additions & 0 deletions ansible/tasks/stage2-setup-postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@
shell: |
sudo -u postgres bash -c ". /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh && nix profile install github:supabase/postgres/{{ git_commit_sha }}#{{postgresql_version}}_src"
when: stage2_nix

- name: Install jq from nix binary cache
become: yes
shell: |
. /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh && nix profile install nixpkgs#jq --profile /nix/var/nix/profiles/default
when: stage2_nix

- name: Set ownership and permissions for /etc/ssl/private
become: yes
Expand Down
6 changes: 3 additions & 3 deletions ansible/vars.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ postgres_major:

# Full version strings for each major version
postgres_release:
postgresorioledb-17: "17.0.1.086-orioledb"
postgres17: "17.4.1.036"
postgres15: "15.8.1.093"
postgresorioledb-17: "17.0.1.086-orioledb-cron-1"
postgres17: "17.4.1.036-cron-1"
postgres15: "15.8.1.093-cron-1"

# Non Postgres Extensions
pgbouncer_release: "1.19.0"
Expand Down
Loading
Loading