Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
9ea2d05
added query tracing and links
javier Dec 10, 2025
559d686
improved architecture guide. Adding multi URL to ILP docs
javier Dec 10, 2025
484a248
arrays and nanos
javier Dec 10, 2025
25e2a6e
byoc link
javier Dec 10, 2025
1f6f8ce
added reference to flyway
javier Dec 10, 2025
5668985
smaller images
javier Dec 10, 2025
ffcee5a
playbook initial draft
javier Dec 18, 2025
db4ace8
more drafts. Needs review
javier Dec 18, 2025
e4f327b
Merge branch 'main' into jv/playbook
javier Dec 18, 2025
cd5ad32
fixing broken links
javier Dec 18, 2025
c52ac49
draft pages. Might contain broken links and queries
javier Dec 18, 2025
99219b6
added trades table to schema info
javier Dec 18, 2025
6a13508
fixing broken links
javier Dec 18, 2025
0a58315
improved playbook content. Need to review queries mostly
javier Dec 18, 2025
bb1fbbc
fixing broken links
javier Dec 18, 2025
d0cb029
fixing links
javier Dec 18, 2025
2495dba
improved recipe
javier Dec 18, 2025
cf18db0
fixing link
javier Dec 18, 2025
a86a252
renamed files to match sidebar titles
javier Dec 19, 2025
fd2eae8
new playbook entries
javier Dec 19, 2025
c20e851
moved playbook inside guides and tutorials
javier Dec 19, 2025
4656c20
new label for playbook
javier Dec 19, 2025
7692abb
Merge main into jv/playbook, add Cookbook section
javier Jan 12, 2026
fcaad69
solved conflicts and links after docs revamp
javier Jan 12, 2026
80ddec8
renaming playbook to cookbook
javier Jan 12, 2026
4a13be6
Merge remote-tracking branch 'origin/main' into jv/playbook
javier Jan 12, 2026
30b4de5
changing the pivot cookbook recipe
javier Jan 12, 2026
52178b6
updated datasets page
javier Jan 12, 2026
3329140
changed link to cookbook overview
javier Jan 12, 2026
52a2248
improved query for compound interest
javier Jan 12, 2026
5eff2ab
improved queries for demo for financial recipes in cookbook
javier Jan 12, 2026
88375b7
improved queries
javier Jan 12, 2026
86f7cfb
improved recipe by using lag
javier Jan 12, 2026
09feb8b
improved queries
javier Jan 12, 2026
9d6fa5f
improved examples and revamped sidebar
javier Jan 13, 2026
f5cfe90
improved queries. Removed outdated section
javier Jan 13, 2026
0c8c5af
improved sql
javier Jan 13, 2026
a22f8f4
note formatting
javier Jan 13, 2026
9c07df2
improved formatting and examples
javier Jan 13, 2026
a8d8ac8
full revamp of this recipe
javier Jan 13, 2026
3fd2d4c
adding new recipe for local min and max
javier Jan 13, 2026
2b4af11
trimming recipe
javier Jan 13, 2026
d79c341
remove extra section
javier Jan 13, 2026
8eded5e
simplified queries and trimmed content
javier Jan 13, 2026
61db4e2
fixing histograms recipe
javier Jan 13, 2026
f6b92c0
removing extra fluff
javier Jan 13, 2026
1b3dd40
improved recipes
javier Jan 13, 2026
5dc103b
moved notes/info messages to the top
javier Jan 13, 2026
58932e8
making first level of cookbook not collapsed when opening the tutoria…
javier Jan 13, 2026
f245d9d
fixed markdown headers
javier Jan 13, 2026
3756e78
rephrasing
javier Jan 13, 2026
975c223
showing demo as .com
javier Jan 13, 2026
6579f82
Merge branch 'main' into jv/playbook
javier Jan 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
315 changes: 315 additions & 0 deletions documentation/cookbook/demo-data-schema.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,315 @@
---
title: Demo Data Schema
sidebar_label: Demo data schema
description: Schema and structure of the FX market data and cryptocurrency trades available on demo.questdb.io
---

The [QuestDB demo instance at demo.questdb.com](https://demo.questdb.io) contains two datasets that you can query directly: simulated FX market data and real cryptocurrency trades. This page describes the available tables and their structure.

:::tip
The demo instance is read-only. For testing write operations (INSERT, UPDATE, DELETE), you'll need to run QuestDB locally. See the [Quick Start guide](/docs/getting-started/quick-start/) for installation instructions.
:::

## Overview

The demo instance provides two independent datasets:

1. **FX Market Data (Simulated)** - Foreign exchange prices and order books
2. **Cryptocurrency Trades (Real)** - Live cryptocurrency trades from OKX exchange

---

## FX Market Data (Simulated)

The FX dataset contains simulated foreign exchange market data for 30 currency pairs. We fetch real reference prices from Yahoo Finance every few seconds, but all order book levels and price updates are generated algorithmically based on these reference prices.

### core_price Table

The `core_price` table contains individual FX price updates from various liquidity providers. Each row represents a bid/ask quote update for a specific currency pair from a specific ECN.

#### Schema

```sql title="core_price table structure"
CREATE TABLE 'core_price' (
timestamp TIMESTAMP,
symbol SYMBOL CAPACITY 16384 CACHE,
ecn SYMBOL CAPACITY 256 CACHE,
bid_price DOUBLE,
bid_volume LONG,
ask_price DOUBLE,
ask_volume LONG,
reason SYMBOL CAPACITY 256 CACHE,
indicator1 DOUBLE,
indicator2 DOUBLE
) timestamp(timestamp) PARTITION BY HOUR TTL 3 DAYS WAL;
```

#### Columns

- **`timestamp`** - Time of the price update (designated timestamp)
- **`symbol`** - Currency pair from the 30 tracked symbols (see list below)
- **`ecn`** - Electronic Communication Network providing the quote: **LMAX**, **EBS**, **Currenex**, or **Hotspot**
- **`bid_price`** - Bid price (price at which market makers are willing to buy)
- **`bid_volume`** - Volume available at the bid price
- **`ask_price`** - Ask price (price at which market makers are willing to sell)
- **`ask_volume`** - Volume available at the ask price
- **`reason`** - Reason for the price update: "normal", "liquidity_event", or "news_event"
- **`indicator1`**, **`indicator2`** - Additional market indicators

The table tracks **30 currency pairs**: EURUSD, GBPUSD, USDJPY, USDCHF, AUDUSD, USDCAD, NZDUSD, EURJPY, GBPJPY, EURGBP, AUDJPY, CADJPY, NZDJPY, EURAUD, EURNZD, AUDNZD, GBPAUD, GBPNZD, AUDCAD, NZDCAD, EURCAD, EURCHF, GBPCHF, USDNOK, USDSEK, USDZAR, USDMXN, USDSGD, USDHKD, USDTRY.

#### Sample Data

```questdb-sql demo title="Recent core_price updates"
SELECT * FROM core_price
WHERE timestamp IN today()
LIMIT -10;
```

**Results:**

| timestamp | symbol | ecn | bid_price | bid_volume | ask_price | ask_volume | reason | indicator1 | indicator2 |
| --------------------------- | ------ | -------- | --------- | ---------- | --------- | ---------- | --------------- | ---------- | ---------- |
| 2025-12-18T11:46:13.059566Z | USDCHF | LMAX | 0.7959 | 219884 | 0.7971 | 223174 | liquidity_event | 0.641 | |
| 2025-12-18T11:46:13.060542Z | USDSGD | Currenex | 1.291 | 295757049 | 1.2982 | 301215620 | normal | 0.034 | |
| 2025-12-18T11:46:13.061853Z | EURAUD | LMAX | 1.7651 | 6207630 | 1.7691 | 5631029 | liquidity_event | 0.027 | |
| 2025-12-18T11:46:13.064138Z | AUDNZD | LMAX | 1.1344 | 227668 | 1.1356 | 212604 | liquidity_event | 0.881 | |
| 2025-12-18T11:46:13.065041Z | GBPNZD | LMAX | 2.3307 | 2021166 | 2.3337 | 1712096 | normal | 0.308 | |
| 2025-12-18T11:46:13.065187Z | USDCAD | EBS | 1.3837 | 2394978 | 1.3869 | 2300556 | normal | 0.084 | |
| 2025-12-18T11:46:13.065722Z | USDZAR | EBS | 16.7211 | 28107021 | 16.7263 | 23536519 | liquidity_event | 0.151 | |
| 2025-12-18T11:46:13.066128Z | EURAUD | EBS | 1.763 | 810471822 | 1.7712 | 883424752 | news_event | 0.027 | |
| 2025-12-18T11:46:13.066700Z | CADJPY | Currenex | 113.63 | 20300827 | 114.11 | 19720915 | normal | 0.55 | |
| 2025-12-18T11:46:13.071607Z | NZDJPY | Currenex | 89.95 | 35284228 | 90.46 | 30552528 | liquidity_event | 0.69 | |

### market_data Table

The `market_data` table contains order book snapshots for currency pairs. Each row represents a complete view of the order book at a specific timestamp, with bid and ask prices and volumes stored as 2D arrays.

#### Schema

```sql title="market_data table structure"
CREATE TABLE 'market_data' (
timestamp TIMESTAMP,
symbol SYMBOL CAPACITY 16384 CACHE,
bids DOUBLE[][],
asks DOUBLE[][]
) timestamp(timestamp) PARTITION BY HOUR TTL 3 DAYS;
```

#### Columns

- **`timestamp`** - Time of the order book snapshot (designated timestamp)
- **`symbol`** - Currency pair (e.g., EURUSD, GBPJPY)
- **`bids`** - 2D array containing bid prices and volumes: `[[price1, price2, ...], [volume1, volume2, ...]]`
- **`asks`** - 2D array containing ask prices and volumes: `[[price1, price2, ...], [volume1, volume2, ...]]`

The arrays are structured so that:
- `bids[1]` contains bid prices (descending order - highest first)
- `bids[2]` contains corresponding bid volumes
- `asks[1]` contains ask prices (ascending order - lowest first)
- `asks[2]` contains corresponding ask volumes

#### Sample Query

```questdb-sql demo title="Recent order book snapshots"
SELECT timestamp, symbol,
array_count(bids[1]) as bid_levels,
array_count(asks[1]) as ask_levels
FROM market_data
WHERE timestamp IN today()
LIMIT -5;
```

**Results:**

| timestamp | symbol | bid_levels | ask_levels |
| --------------------------- | ------ | ---------- | ---------- |
| 2025-12-18T12:04:07.071512Z | EURAUD | 40 | 40 |
| 2025-12-18T12:04:07.072060Z | USDJPY | 40 | 40 |
| 2025-12-18T12:04:07.072554Z | USDMXN | 40 | 40 |
| 2025-12-18T12:04:07.072949Z | USDCAD | 40 | 40 |
| 2025-12-18T12:04:07.073002Z | USDSEK | 40 | 40 |

Each order book snapshot contains 40 bid levels and 40 ask levels.

### fx_trades Table

The `fx_trades` table contains simulated FX trade executions. Each row represents a trade that executed against the order book, with realistic partial fills and level walking.

#### Schema

```sql title="fx_trades table structure"
CREATE TABLE 'fx_trades' (
timestamp TIMESTAMP_NS,
symbol SYMBOL,
ecn SYMBOL,
trade_id UUID,
side SYMBOL,
passive BOOLEAN,
price DOUBLE,
quantity DOUBLE,
counterparty SYMBOL,
order_id UUID
) timestamp(timestamp) PARTITION BY HOUR TTL 1 MONTH WAL;
```

#### Columns

- **`timestamp`** - Time of trade execution with nanosecond precision (designated timestamp)
- **`symbol`** - Currency pair (same 30 pairs as `core_price`)
- **`ecn`** - ECN where trade executed: **LMAX**, **EBS**, **Currenex**, or **Hotspot**
- **`trade_id`** - Unique identifier for this specific trade
- **`side`** - Trade direction: **buy** or **sell**
- **`passive`** - Whether this was a passive (limit) or aggressive (market) order
- **`price`** - Execution price
- **`quantity`** - Trade size
- **`counterparty`** - 20-character LEI (Legal Entity Identifier) of the counterparty
- **`order_id`** - Parent order identifier (multiple trades can share the same `order_id` for partial fills)

#### Sample Data

```questdb-sql demo title="Recent FX trades"
SELECT * FROM fx_trades
WHERE timestamp IN today()
LIMIT -10;
```

**Results:**

| timestamp | symbol | ecn | trade_id | side | passive | price | quantity | counterparty | order_id |
| ------------------------------ | ------ | -------- | ------------------------------------ | ---- | ------- | ------- | -------- | -------------------- | ------------------------------------ |
| 2026-01-12T12:18:57.138282586Z | EURUSD | LMAX | d14e6e54-6c6b-495d-865d-47311a36519b | sell | false | 1.1705 | 193615.0 | 004409EA0ED5B9FF954B | db3cd1e6-c3e7-4909-8a64-31a2b6f0f9c0 |
| 2026-01-12T12:18:57.138912209Z | EURUSD | LMAX | be857ed7-848f-4d23-83ff-3e5636cbc9de | sell | false | 1.1707 | 107749.0 | 000A4FB276D1BE98F143 | db3cd1e6-c3e7-4909-8a64-31a2b6f0f9c0 |
| 2026-01-12T12:18:57.259555330Z | GBPUSD | EBS | 446cac16-9b25-4205-b1e1-3eda4a3bb539 | sell | false | 1.3401 | 192701.0 | 00119FEF98D9EC079D15 | d0d74987-8929-4c48-bc18-7164b1a956e3 |
| 2026-01-12T12:18:57.303333947Z | GBPUSD | EBS | 27515a12-9ab6-4175-8fa3-422d4529f365 | sell | true | 1.3404 | 66295.0 | 00363EC8480C058FD36C | 239eae98-fc45-4e1c-bd45-8933909a67fc |
| 2026-01-12T12:18:57.334406432Z | USDTRY | EBS | c82453b3-9961-40ea-a6ac-43c33fe0f235 | sell | true | 43.1001 | 65849.0 | 002A80CCE4AFD37D0642 | 2ce77a03-0f21-4241-8ca7-903080848dc0 |
| 2026-01-12T12:18:57.365445776Z | USDJPY | LMAX | bf918a88-60c2-4a20-8f53-65298b5a10fe | buy | false | 156.82 | 55548.0 | 00EB428CCC1C1C240F71 | 7458b51d-65fa-4ffb-8fa8-840e88d2c316 |
| 2026-01-12T12:18:57.479674129Z | USDJPY | EBS | c7c902bd-7075-4952-88d1-76d39ba4c706 | buy | false | 156.82 | 98591.0 | 00A10D27678CC03A0161 | 5992296a-684f-4783-9e8c-7206519a85f8 |
| 2026-01-12T12:18:57.480051522Z | USDJPY | EBS | a20b6f91-7148-4b64-8a36-85da5bec66f9 | buy | false | 156.85 | 178152.0 | 00CBD8490AE2844C8554 | 5992296a-684f-4783-9e8c-7206519a85f8 |
| 2026-01-12T12:18:57.509773474Z | GBPUSD | Currenex | ae6b771b-5abd-44c7-9e0e-3527ce6fb5b4 | sell | false | 1.3404 | 62305.0 | 006728CF215E44412D18 | 54ff8191-1891-4a5c-8b67-d5cd961ec5e8 |
| 2026-01-12T12:18:57.334732460Z | USDTRY | EBS | 469637a5-6553-4aad-aad9-f7114c8a442d | sell | true | 43.1 | 101177.0 | 002CAC92E93AB4B3D30C | 2ce77a03-0f21-4241-8ca7-903080848dc0 |

### FX Materialized Views

The FX dataset includes several materialized views providing pre-aggregated data at different time intervals:

#### Best Bid/Offer (BBO) Views

- **`bbo_1s`** - Best bid and offer aggregated every 1 second
- **`bbo_1m`** - Best bid and offer aggregated every 1 minute
- **`bbo_1h`** - Best bid and offer aggregated every 1 hour
- **`bbo_1d`** - Best bid and offer aggregated every 1 day

#### Core Price Aggregations

- **`core_price_1s`** - Core prices aggregated every 1 second
- **`core_price_1d`** - Core prices aggregated every 1 day

#### Market Data OHLC

- **`market_data_ohlc_1m`** - Open, High, Low, Close candlesticks at 1-minute intervals
- **`market_data_ohlc_15m`** - OHLC candlesticks at 15-minute intervals
- **`market_data_ohlc_1d`** - OHLC candlesticks at 1-day intervals

#### FX Trades OHLC

- **`fx_trades_ohlc_1m`** - OHLC candlesticks from trade executions at 1-minute intervals
- **`fx_trades_ohlc_1h`** - OHLC candlesticks from trade executions at 1-hour intervals

These views are continuously updated and optimized for dashboard and analytics queries on FX data.

### FX Data Volume

- **`market_data`**: Approximately **160 million rows** per day (order book snapshots)
- **`core_price`**: Approximately **73 million rows** per day (price updates across all ECNs and symbols)
- **`fx_trades`**: Approximately **5.1 million rows** per day (trade executions)

---

## Cryptocurrency Trades (Real)

The cryptocurrency dataset contains **real market data** streamed live from the OKX exchange using FeedHandler. These are actual executed trades, not simulated data.

### trades Table

The `trades` table contains real cryptocurrency trade data. Each row represents an actual executed trade for a cryptocurrency pair.

#### Schema

```sql title="trades table structure"
CREATE TABLE 'trades' (
symbol SYMBOL CAPACITY 256 CACHE,
side SYMBOL CAPACITY 256 CACHE,
price DOUBLE,
amount DOUBLE,
timestamp TIMESTAMP
) timestamp(timestamp) PARTITION BY DAY WAL;
```

#### Columns

- **`timestamp`** - Time when the trade was executed (designated timestamp)
- **`symbol`** - Cryptocurrency trading pair from the 12 tracked symbols (see list below)
- **`side`** - Trade side: **buy** or **sell**
- **`price`** - Execution price of the trade
- **`amount`** - Trade size (volume in base currency)

The table tracks **12 cryptocurrency pairs**: ADA-USDT, AVAX-USD, BTC-USDT, DAI-USD, DOT-USD, ETH-BTC, ETH-USDT, LTC-USD, SOL-BTC, SOL-USD, UNI-USD, XLM-USD.

#### Sample Data

```questdb-sql demo title="Recent cryptocurrency trades"
SELECT * FROM trades
LIMIT -10;
```

**Results:**

| symbol | side | price | amount | timestamp |
| -------- | ---- | ------- | ---------- | --------------------------- |
| BTC-USDT | buy | 85721.6 | 0.00045714 | 2025-12-18T19:31:11.203000Z |
| BTC-USD | buy | 85721.6 | 0.00045714 | 2025-12-18T19:31:11.203000Z |
| BTC-USDT | buy | 85726.6 | 0.00001501 | 2025-12-18T19:31:11.206000Z |
| BTC-USD | buy | 85726.6 | 0.00001501 | 2025-12-18T19:31:11.206000Z |
| BTC-USDT | buy | 85726.9 | 0.000887 | 2025-12-18T19:31:11.206000Z |
| BTC-USD | buy | 85726.9 | 0.000887 | 2025-12-18T19:31:11.206000Z |
| BTC-USDT | buy | 85731.3 | 0.00004393 | 2025-12-18T19:31:11.206000Z |
| BTC-USD | buy | 85731.3 | 0.00004393 | 2025-12-18T19:31:11.206000Z |
| ETH-USDT | sell | 2827.54 | 0.006929 | 2025-12-18T19:31:11.595000Z |
| ETH-USD | sell | 2827.54 | 0.006929 | 2025-12-18T19:31:11.595000Z |

### Cryptocurrency Materialized Views

The cryptocurrency dataset includes materialized views for aggregated trade data:

#### Trades Aggregations

- **`trades_latest_1d`** - Latest trade data aggregated daily
- **`trades_OHLC_15m`** - OHLC candlesticks for cryptocurrency trades at 15-minute intervals

These views are continuously updated and provide faster query performance for cryptocurrency trade analysis.

### Cryptocurrency Data Volume

- **`trades`**: Approximately **3.7 million rows** per day (real cryptocurrency trades)

---

## Data Retention

**FX tables** (`core_price` and `market_data`) use a **3-day TTL (Time To Live)**, meaning data older than 3 days is automatically removed. This keeps the demo instance responsive while providing sufficient recent data.

**Cryptocurrency trades table** has **no retention policy** and contains historical data dating back to **March 8, 2022**. This provides over 3 years of real cryptocurrency trade history for long-term analysis and backtesting.

## Using the Demo Data

You can run queries against both datasets directly on [demo.questdb.com](https://demo.questdb.io). Throughout the Cookbook, recipes using demo data will include a direct link to execute the query.



:::info Related Documentation
- [SYMBOL type](/docs/concepts/symbol/)
- [Arrays in QuestDB](/docs/query/datatypes/array/)
- [Designated timestamp](/docs/concepts/designated-timestamp/)
- [Time-series aggregations](/docs/query/functions/aggregation/)
:::
44 changes: 44 additions & 0 deletions documentation/cookbook/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: Cookbook Overview
sidebar_label: Overview
description: Quick recipes and practical examples for common QuestDB tasks and queries
---

The Cookbook is a collection of **short, actionable recipes** that demonstrate how to accomplish specific tasks with QuestDB. Each recipe follows a problem-solution-result format, making it easy to find and apply solutions quickly.

## What is the Cookbook?

Unlike comprehensive reference documentation, the Cookbook focuses on practical examples for:

- **Common SQL patterns** - Window functions, pivoting, time-series aggregations
- **Programmatic integration** - Language-specific client examples
- **Operations** - Deployment and configuration tasks

Each recipe provides a focused solution to a specific problem, with working code examples and expected results.

## Structure

The Cookbook is organized into three main sections:

- **SQL Recipes** - Common SQL patterns, window functions, and time-series queries
- **Programmatic** - Language-specific client examples and integration patterns
- **Operations** - Deployment, configuration, and operational tasks

## Running the Examples

**Most recipes run directly on our [live demo instance at demo.questdb.com](https://demo.questdb.com)** without any local setup. Queries that can be executed on the demo site are marked with a direct link to run them.

For recipes that require write operations or specific configuration, the recipe will indicate what setup is needed.

The demo instance contains live FX market data with tables for core prices and order book snapshots. See the [Demo Data Schema](/docs/cookbook/demo-data-schema/) page for details about available tables and their structure.

## Using the Cookbook

Each recipe follows a consistent format:

1. **Problem statement** - What you're trying to accomplish
2. **Solution** - Code example with explanation
3. **Results** - Expected output or verification
4. **Additional context** - Tips, variations, or related documentation links

Start by browsing the SQL Recipes section for common patterns, or jump directly to the recipe that matches your needs.
Loading