feat(postgres): allow password to be a sync/async callable - #2261
Open
get-bowen wants to merge 1 commit into
Open
feat(postgres): allow password to be a sync/async callable#2261get-bowen wants to merge 1 commit into
get-bowen wants to merge 1 commit into
Conversation
Short-lived database credentials (AWS RDS/Aurora IAM tokens, Azure Entra ID tokens, Vault leases) expire long before a connection pool does. Allow the postgres 'password' credential to be a callable, resolved once per new connection, so tokens are refreshed transparently. - asyncpg resolves the callable natively; it is now forwarded with the right type - psycopg bakes credentials into an immutable conninfo string, so the password is injected at connect time via a generated connection class instead - Tortoise.star_password no longer crashes on a non-string password Refs tortoise#2034
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Allow the PostgreSQL
passwordcredential to be a sync or async callable, resolved once per new connection.Short-lived database credentials — AWS RDS/Aurora IAM tokens, Azure Entra ID tokens, Vault leases — expire long before a connection pool does. Today the only way to use them with Tortoise is to tear down and re-init the client on a timer, which kills in-flight queries when the pool is closed underneath them.
Refs #2034.
Scope
This deliberately adds the primitive rather than native IAM support. #2034 asks for an
iam_authflag with boto3 token generation built in; that would put a cloud-vendor SDK inside Tortoise and only solve one provider. A callable password is whatasyncpgand SQLAlchemy both expose, and it covers AWS IAM, Azure Entra ID, GCP, Vault and anything else in a few lines of user code.PostgreSQL only. MySQL has the same
create_pool(password=...)shape, but neitheraiomysqlnorasyncmyresolves callables, so supporting it there needs its own design — happy to follow up if you'd like it.Motivation and Context
We run Tortoise against Azure Postgres with Entra ID (passwordless) auth, where tokens live ~60 minutes. Our workaround was a background task calling
Tortoise.close_connections()+ re-init every 5 minutes. On the asyncpg backend_close()gives in-flight work a 10-second grace period beforepool.terminate(), so a long-running batch job gets its connections pulled mid-statement and fails withInterfaceError: pool is closing. Resolving the password per connection removes the need to recycle the pool at all.How Has This Been Tested?
tests/backends/test_password_factory.py:resolve_passwordover string/None/sync/async callables, asyncpg forwarding the callable untouched, psycopg keeping the password out of the conninfo, the generated psycopg connection class minting a fresh password per connect, andstar_passwordnot choking on a callable.make check— ruff format + ruff check + mypy (134 files) + bandit all clean.Implementation notes
connect_utils.py, awaits the result if awaitable), andBasePostgresClientforwardedpassworduntouched — so this worked by accident but was typedstr | Noneand undocumented. It is now typed, tested and documented so it doesn't get "fixed" away.psycopg_pooldoes support a callableconninfo, but only from 3.3, whilepyproject.tomlallowspsycopg[pool] >=3.0.12— this keeps that floor intact.Tortoise.star_password()was a latent crash: it doespassword[0 : len(password) // 3]andstr.replace(password, ...), bothTypeErroron a non-string. Now skips non-strings.Checklist: