Skip to content

Commit

Permalink
Adds support for COPY TO/FROM Azure Blob Storage
Browse files Browse the repository at this point in the history
Supports following Azure Blob uri forms:
- `az://{container}/key`
- `azure://{container}/key`
- `https://{account}.blob.core.windows.net/{container}/key`

**Configuration**

The simplest way to configure object storage is by creating the standard [`~/.azure/config`](https://learn.microsoft.com/en-us/cli/azure/azure-cli-configuration?view=azure-cli-latest) file:

```bash
$ cat ~/.azure/config
[storage]
account = devstoreaccount1
key = Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==
```

Alternatively, you can use the following environment variables when starting postgres to configure the Azure Blob Storage client:
- `AZURE_STORAGE_ACCOUNT`: the storage account name of the Azure Blob
- `AZURE_STORAGE_KEY`: the storage key of the Azure Blob
- `AZURE_STORAGE_SAS_TOKEN`: the storage SAS token for the Azure Blob
- `AZURE_CONFIG_FILE`: an alternative location for the config file

**Bonus**
Additionally, PR supports following S3 uri forms:
- `s3://{bucket}/key`
- `s3a://{bucket}/key`
- `https://s3.amazonaws.com/{bucket}/key`
- `https://{bucket}.s3.amazonaws.com/key`

Closes #50
  • Loading branch information
aykut-bozkurt committed Nov 28, 2024
1 parent 80e449f commit 4268795
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/pgrx_tests/object_store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ mod tests {
);

let copy_to_command = format!(
"COPY (SELECT i FROM generate_series(1,10) i) TO '{}' WITH (format parquet);;",
"COPY (SELECT i FROM generate_series(1,10) i) TO '{}' WITH (format parquet);",
azure_blob_uri
);
Spi::run(copy_to_command.as_str()).unwrap();
Expand All @@ -365,7 +365,7 @@ mod tests {
);

let copy_to_command = format!(
"COPY (SELECT i FROM generate_series(1,10) i) TO '{}' WITH (format parquet);;",
"COPY (SELECT i FROM generate_series(1,10) i) TO '{}' WITH (format parquet);",
azure_blob_uri
);
Spi::run(copy_to_command.as_str()).unwrap();
Expand Down

0 comments on commit 4268795

Please sign in to comment.