-
Notifications
You must be signed in to change notification settings - Fork 116
CrateDB: Accept protocol scheme alias cratedb:// for source adapter
#541
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3 files reviewed, 1 comment
ingestr/src/sources.py
Outdated
| ssl = "false" | ||
| if sslmode.lower() in ["require", "verify-ca", "verify-full"]: | ||
| ssl = "true" | ||
| uri = uri.replace(f"sslmode={sslmode}", f"ssl={ssl}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
String replacement is unsafe - will replace any occurrence of sslmode={value} in the entire URI, including passwords or hostnames containing this substring.
Use parsed_uri._replace() and urlencode() like other URI transformations in this file (lines 156-158, 172-175):
| uri = uri.replace(f"sslmode={sslmode}", f"ssl={ssl}") | |
| query_params["ssl"] = [ssl] | |
| del query_params["sslmode"] | |
| uri = parsed_uri._replace(query=urlencode(query_params, doseq=True)).geturl() |
Prompt To Fix With AI
This is a comment left during a code review.
Path: ingestr/src/sources.py
Line: 240:240
Comment:
String replacement is unsafe - will replace any occurrence of `sslmode={value}` in the entire URI, including passwords or hostnames containing this substring.
Use `parsed_uri._replace()` and `urlencode()` like other URI transformations in this file (lines 156-158, 172-175):
```suggestion
query_params["ssl"] = [ssl]
del query_params["sslmode"]
uri = parsed_uri._replace(query=urlencode(query_params, doseq=True)).geturl()
```
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. The patch was improved accordingly.
4855029 to
8e0537f
Compare
While the source adapter is using `crate://`, based on CrateDB's SQLAlchemy dialect, the destination adapter is using `cratedb://`, based on `dlt-postgres`. In this spirit, CrateDB's source vs. destination adapters are using totally different implementations and protocols. The idea is to at least harmonize the "naming things" details on database addressing concerns, so it becomes less confusing for users, and less of an anomaly for the [new] product landing page generator. -- https://getbruin.com/product/ingestion/
8e0537f to
4f41067
Compare
cratedb:// for source adaptercratedb:// for source adapter
About
While the CrateDB source adapter is using
crate://, based on CrateDB's SQLAlchemy dialect, the destination adapter is usingcratedb://, based ondlt-postgres. In this spirit, CrateDB's source vs. destination adapters are using totally different implementations and protocols.The idea is to at least harmonize the "naming things" details on database addressing concerns, so it becomes less confusing for users, and less of an anomaly for the [new] product landing page generator.
-- https://getbruin.com/product/ingestion/
References
Review
Also FYI, @seut, @matriv, @surister, @hammerhead, and @zolbatar.