Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(prqlc): allow prqlc compile | duckdb #2594

Merged
merged 8 commits into from
May 22, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 54 additions & 3 deletions prql-compiler/prqlc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,19 @@

## Installation

`prqlc` can be installed with `cargo`:
`prqlc` is a single, dependency-free binary that compiles PRQL into SQL.
precompiled binaries are available for Linux, macOS, and Windows on the
[PRQL release page](https://github.com/PRQL/prql/releases).

If you have Rust toolchain installed, you can install `prqlc` via `cargo`:
eitsupi marked this conversation as resolved.
Show resolved Hide resolved

```sh
# From crates.io
cargo install prqlc
```

...or built from source:

```sh
# From GitHub
eitsupi marked this conversation as resolved.
Show resolved Hide resolved
cargo install --path prql-compiler/prqlc
```

Expand All @@ -22,6 +26,10 @@ brew install prql/prql/prql-compiler

## Usage

### prqlc compile

This command is working as a filter to compile PRQL string into SQL string.
max-sixty marked this conversation as resolved.
Show resolved Hide resolved

```sh
$ echo "from employees | filter has_dog | select salary" | prqlc compile

Expand All @@ -32,3 +40,46 @@ FROM
WHERE
has_dog
```

PRQL query can be executed with CLI tools working with SQL, such as
[DuckDB CLI](https://duckdb.org/docs/api/cli.html).

```sh
$ curl -sL https://raw.githubusercontent.com/mwaskom/seaborn-data/master/penguins.csv -o penguins.csv
eitsupi marked this conversation as resolved.
Show resolved Hide resolved
$ echo "from `penguins.csv` | take 3" | prqlc compile | duckdb
┌─────────┬───────────┬────────────────┬───────────────┬───────────────────┬─────────────┬─────────┐
│ species │ island │ bill_length_mm │ bill_depth_mm │ flipper_length_mm │ body_mass_g │ sex │
│ varchar │ varchar │ double │ double │ int64 │ int64 │ varchar │
├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼─────────┤
│ Adelie │ Torgersen │ 39.1 │ 18.7 │ 181 │ 3750 │ MALE │
│ Adelie │ Torgersen │ 39.5 │ 17.4 │ 186 │ 3800 │ FEMALE │
│ Adelie │ Torgersen │ 40.3 │ 18.0 │ 195 │ 3250 │ FEMALE │
└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴─────────┘
```

Executing this command without any argument will start interactive mode,
allowing you to write PRQL query interactively. In this mode, after you write
PRQL and press `Ctrl-D` (Linux, macOS) or `Ctrl-Z` (Windows) to display the
compiled SQL.

```sh
$ prqlc compile
```

As with using it as a filter, you can pass the SQL string output to the DuckDB
CLI, etc.

```sh
$ prqlc compile | duckdb
from `penguins.csv`
take 3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, this doesn't work so well on mine — does it for you?

I think maybe we write the Ctrl-d to stdout?

prqlc compile | duckdb
from `penguins.csv`
take 3
Error: near line 1: Parser Error: syntax error at or near "Enter"
LINE 1: Enter PRQL, then ctrl-d:
        ^

(Is there a way of retaining the query? I worry this approach is OK but loses the query on each run. But it's still reasonable to show the example)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the prqlc you are using was not built from this PR. (the prompt message is different)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way of retaining the query?

I believe the query will remain visible in the terminal as shown in this README, and I have verified that it works with bash and fish on Linux and PowerShell on Windows.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that the prqlc you are using was not built from this PR. (the prompt message is different)

Sorry, you're correct, apologies

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the query will remain visible in the terminal as shown in this README, and I have verified that it works with bash and fish on Linux and PowerShell on Windows.

Yes!

A total aside — something @snth and I discussed on the dev call was building a TUI, similar to the playground, where typing a query would pipe through to duckdb on each keystroke, could be a good way of exploring a dataset from the terminal


┌─────────┬───────────┬────────────────┬───────────────┬───────────────────┬─────────────┬─────────┐
│ species │ island │ bill_length_mm │ bill_depth_mm │ flipper_length_mm │ body_mass_g │ sex │
│ varchar │ varchar │ double │ double │ int64 │ int64 │ varchar │
├─────────┼───────────┼────────────────┼───────────────┼───────────────────┼─────────────┼─────────┤
│ Adelie │ Torgersen │ 39.1 │ 18.7 │ 181 │ 3750 │ MALE │
│ Adelie │ Torgersen │ 39.5 │ 17.4 │ 186 │ 3800 │ FEMALE │
│ Adelie │ Torgersen │ 40.3 │ 18.0 │ 195 │ 3250 │ FEMALE │
└─────────┴───────────┴────────────────┴───────────────┴───────────────────┴─────────────┴─────────┘
```
2 changes: 1 addition & 1 deletion prql-compiler/prqlc/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ impl Command {
// it's confusing whether it's waiting for input or not. This
// offers the prompt.
if input.is_stdin() && atty::is(atty::Stream::Stdin) {
println!("Enter PRQL, then ctrl-d:\n");
println!("# Enter PRQL, then press ctrl-d or ctrl-z (windows) to compile:\n");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
println!("# Enter PRQL, then press ctrl-d or ctrl-z (windows) to compile:\n");
eprintln!("# Enter PRQL, then press ctrl-d or ctrl-z (windows) to compile:\n");

...I think this will solve the issue above

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI we could use this approach to print the correct string:

#[cfg(not(unix))]

}

let file_tree = input.read_to_tree()?;
Expand Down
1 change: 1 addition & 0 deletions web/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@

- [Bindings](./bindings/README.md)

- [CLI](./bindings/cli.md)
eitsupi marked this conversation as resolved.
Show resolved Hide resolved
- [.NET](./bindings/dotnet.md)
- [Elixir](./bindings/elixir.md)
- [Java](./bindings/java.md)
Expand Down
1 change: 1 addition & 0 deletions web/book/src/bindings/cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{{#include ../../../../prql-compiler/prqlc/README.md}}