You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/sql/copy.md
+34Lines changed: 34 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,6 +31,19 @@ COPY tbl TO '/path/to/file.csv' WITH (
31
31
);
32
32
```
33
33
34
+
You can also export data to a compressed CSV or JSON file:
35
+
36
+
```sql
37
+
COPY tbl TO '/path/to/file.csv.gz' WITH (
38
+
FORMAT ='csv',
39
+
compression_type ='gzip'
40
+
);
41
+
```
42
+
43
+
:::tip NOTE
44
+
When using compression, ensure the file extension matches the compression type: `.gz` for gzip, `.zst` for zstd, `.bz2` for bzip2, and `.xz` for xz.
45
+
:::
46
+
34
47
#### `WITH` Option
35
48
36
49
`WITH` adds options such as the file `FORMAT` which specifies the format of the exported file. In this example, the format is Parquet; it is a columnar storage format used for big data processing. Parquet efficiently compresses and encodes columnar data for big data analytics.
@@ -39,6 +52,7 @@ COPY tbl TO '/path/to/file.csv' WITH (
39
52
|---|---|---|
40
53
|`FORMAT`| Target file(s) format, e.g., JSON, CSV, Parquet |**Required**|
41
54
|`START_TIME`/`END_TIME`| The time range within which data should be exported. `START_TIME` is inclusive and `END_TIME` is exclusive. | Optional |
55
+
|`compression_type`| Compression algorithm for the exported file. Supported values: `gzip`, `zstd`, `bzip2`, `xz`. Only supported for CSV and JSON formats. | Optional |
42
56
|`TIMESTAMP_FORMAT`| Custom format for timestamp columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers (e.g., `'%Y-%m-%d %H:%M:%S'`). Only supported for CSV format. | Optional |
43
57
|`DATE_FORMAT`| Custom format for date columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers (e.g., `'%Y-%m-%d'`). Only supported for CSV format. | Optional |
44
58
|`TIME_FORMAT`| Custom format for time columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers (e.g., `'%H:%M:%S'`). Only supported for CSV format. | Optional |
@@ -85,10 +99,20 @@ Specifically, if you only have one file to import, you can use the following syn
85
99
COPY tbl FROM'/path/to/folder/xxx.parquet' WITH (FORMAT ='parquet');
86
100
```
87
101
102
+
You can also import data from a compressed CSV or JSON file:
103
+
104
+
```sql
105
+
COPY tbl FROM'/path/to/file.csv.gz' WITH (
106
+
FORMAT ='csv',
107
+
compression_type ='gzip'
108
+
);
109
+
```
110
+
88
111
| Option | Description | Required |
89
112
|---|---|---|
90
113
|`FORMAT`| Target file(s) format, e.g., JSON, CSV, Parquet, ORC |**Required**|
91
114
|`PATTERN`| Use regex to match files. e.g., `*_today.parquet`| Optional |
115
+
|`compression_type`| Compression algorithm for the imported file. Supported values: `gzip`, `zstd`, `bzip2`, `xz`. Only supported for CSV and JSON formats. | Optional |
92
116
93
117
:::tip NOTE
94
118
The CSV file must have a header row to be imported correctly. The header row should contain the column names of the table.
@@ -158,6 +182,7 @@ COPY (<QUERY>) TO '<PATH>' WITH (FORMAT = { 'CSV' | 'JSON' | 'PARQUET' });
158
182
|`QUERY`| The SQL SELECT statement to execute |**Required**|
159
183
|`PATH`| The file path where the output will be written |**Required**|
160
184
|`FORMAT`| The output file format: 'CSV', 'JSON', or 'PARQUET' |**Required**|
185
+
|`compression_type`| Compression algorithm for the exported file. Supported values: `gzip`, `zstd`, `bzip2`, `xz`. Only supported for CSV and JSON formats. | Optional |
161
186
|`TIMESTAMP_FORMAT`| Custom format for timestamp columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers. Only supported for CSV format. | Optional |
162
187
|`DATE_FORMAT`| Custom format for date columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers. Only supported for CSV format. | Optional |
163
188
|`TIME_FORMAT`| Custom format for time columns when exporting to CSV format. Uses [strftime](https://docs.rs/chrono/latest/chrono/format/strftime/index.html) format specifiers. Only supported for CSV format. | Optional |
@@ -168,6 +193,15 @@ For example, the following statement exports query results to a CSV file:
168
193
COPY (SELECT*FROM tbl WHERE host ='host1') TO '/path/to/file.csv' WITH (FORMAT ='csv');
169
194
```
170
195
196
+
You can also export query results to a compressed file:
197
+
198
+
```sql
199
+
COPY (SELECT*FROM tbl WHERE host ='host1') TO '/path/to/file.json.gz' WITH (
200
+
FORMAT ='json',
201
+
compression_type ='gzip'
202
+
);
203
+
```
204
+
171
205
You can also specify custom date and time formats when exporting to CSV:
0 commit comments