Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

♻️ ⬆️ ⚗️ ⚠️ upgrade to postgres 17.2 (DONT MERGE) #7023

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

mrnicegyu11
Copy link
Member

@mrnicegyu11 mrnicegyu11 commented Jan 10, 2025

What do these changes do?

  • Upgrades postgres from 14 to 17.2 (latest)
  • Works on my local deployment of osparc-simcore, but only AFTER deleting the postgres named docker volume
  • Deadline is March31 2025 on AWS
  • Requires change in aws/terraform as well

Related issue/s

ITISFoundation/osparc-ops-environments#907

Dev-ops checklist

@mrnicegyu11 mrnicegyu11 self-assigned this Jan 10, 2025
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.65%. Comparing base (adb1867) to head (006bb7d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7023      +/-   ##
==========================================
- Coverage   86.94%   85.65%   -1.29%     
==========================================
  Files        1651     1641      -10     
  Lines       64811    64485     -326     
  Branches     2041     2041              
==========================================
- Hits        56351    55236    -1115     
- Misses       8122     8911     +789     
  Partials      338      338              
Flag Coverage Δ
integrationtests 64.88% <ø> (+1.44%) ⬆️
unittests 83.92% <ø> (-1.40%) ⬇️
Components Coverage Δ
api ∅ <ø> (∅)
pkg_aws_library 93.49% <ø> (ø)
pkg_dask_task_models_library 97.09% <ø> (ø)
pkg_models_library 91.58% <ø> (ø)
pkg_notifications_library 84.57% <ø> (ø)
pkg_postgres_database 88.27% <ø> (ø)
pkg_service_integration 70.02% <ø> (ø)
pkg_service_library 73.93% <ø> (+0.05%) ⬆️
pkg_settings_library 90.60% <ø> (ø)
pkg_simcore_sdk 85.38% <ø> (ø)
agent 96.82% <ø> (ø)
api_server 90.57% <ø> (ø)
autoscaling 96.09% <ø> (ø)
catalog 90.66% <ø> (ø)
clusters_keeper 99.48% <ø> (ø)
dask_sidecar 91.26% <ø> (ø)
datcore_adapter 93.18% <ø> (ø)
director 76.49% <ø> (ø)
director_v2 91.36% <ø> (-0.02%) ⬇️
dynamic_scheduler 97.19% <ø> (ø)
dynamic_sidecar 89.74% <ø> (ø)
efs_guardian 90.12% <ø> (ø)
invitations 93.44% <ø> (ø)
osparc_gateway_server ∅ <ø> (∅)
payments 92.66% <ø> (ø)
resource_usage_tracker 89.33% <ø> (+0.05%) ⬆️
storage 89.59% <ø> (ø)
webclient ∅ <ø> (∅)
webserver 79.82% <ø> (-4.25%) ⬇️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update adb1867...006bb7d. Read the comment docs.

@mrnicegyu11
Copy link
Member Author

mrnicegyu11 commented Jan 10, 2025

CC @sanderegg @YuryHrytsuk @matusdrobuliak66

didnt check the failing tests yet (I guess it's only codecov so this should be fine), anyone willing to help next sprint: would be very appreciated :--)

@mrnicegyu11 mrnicegyu11 changed the title ♻️⬆️ ⚗️ upgrade to postgres 17.2 ♻️ ⬆️ ⚗️ ⚠️ upgrade to postgres 17.2 Jan 10, 2025
@mrnicegyu11 mrnicegyu11 changed the title ♻️ ⬆️ ⚗️ ⚠️ upgrade to postgres 17.2 ♻️ ⬆️ ⚗️ ⚠️ upgrade to postgres 17.2 (DONT MERGE) Jan 10, 2025
@mrnicegyu11 mrnicegyu11 marked this pull request as ready for review January 10, 2025 09:14
@mrnicegyu11
Copy link
Member Author

This needs to be manually rolled-out, re-importing the DB from a dump, with maintenance downtime, and requires changes in aws/terraform for the RDS.

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a # devops section with the required actions from devops at the top of the ticket

Copy link
Member

@sanderegg sanderegg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what kind of help do you need? it looks all green.
I guess the only thing needed is the data migration (pg_upgrade or dump/reload)

@mrnicegyu11
Copy link
Member Author

what kind of help do you need? it looks all green. I guess the only thing needed is the data migration (pg_upgrade or dump/reload)

True thanks, I wrote the text assuming there would be issues, before the result of the tests was there. All good I think :--)

@pcrespov
Copy link
Member

@matusdrobuliak66 you mentioned once that we could do some sort of pruning /reindexing of the db. Perhaps we can take advantage ot hits to do so?

@matusdrobuliak66
Copy link
Contributor

matusdrobuliak66 commented Jan 13, 2025

@matusdrobuliak66 you mentioned once that we could do some sort of pruning /reindexing of the db. Perhaps we can take advantage ot hits to do so?

We can do this (mainly the second step):

  • Manually reindex table (can reduce the size), ex.:
reindex table resource_tracker_service_runs;
reindex table resource_tracker_credit_transactions;
  • Manually clean dead rows (Careful it is blocking operation!), ex.:
vacuum full resource_tracker_credit_transactions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants