Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] redesign advanced measures/dimension per page with more examples #6931

Merged
merged 9 commits into from
Mar 26, 2025
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
position: 00
label: Advanced Expressions
collapsible: true
collapsed: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
title: "Advanced Expressions"
description: Tips & Tricks for Defining Metrics & Dimensions
sidebar_label: "Advanced Expressions"
sidebar_position: 00
---

## Overview

Within the metrcs view yaml, you can apply aggregate sql expressions to create derived metrics or non-aggregate expressions to adjust dimension settings. SQL expressions are specific to the underlying OLAP engine so keep that in mind when editing directly in the yaml.

:::note Examples in Docs
For most of the examples here, DuckDB is being used. However most if not all the functionality is possible on different OLAP engines. You will need to refer to that specific OLAP's reference documentation. Don't hestitate to reach out to us if you have any questions!
:::


:::tip

Rill's modeling layer provides open-ended SQL compatibility for complex SQL queries. More details can be found in our [modeling section](/build/models/models.md).

:::

## Measure Expressions

Measure expressions can take any SQL numeric function, a set of aggregates and apply filters to create derived metrics. Reminder on basic expressions are available in the [create metrics view definition](../metrics-view.md#measures).

See our dedicated examples and pages for the following advanced measures!
- **[Metric Formatting](./metric-formatting)**
- **[Case Statements and Filters](./case-statements)**
- **[Referencing Measures](./referencing)**
- **[Quantiles](./quantiles)**
- **[Fixed Metrics](./fixed-metrics)**
- **[Window Functions](./windows)**





## Dimension Expressions

To utilize an expression, replace the `column` property with `expression` and apply a non-aggregate sql expression. Common use cases would be editing fields such as `string_split(email, '@')[2]` to extract the domain from an email or combining values `concat(domain, child_url)` to get the full URL.

```yaml
- name: domain
display_name: Domain Name
expression: string_split(email, '@')[2]
description: "Extract the domain from an email"
```
See our dedicated examples and pages for the following advanced dimensions!

- **[Unnest Dimensions](./unnesting)**

## Druid Lookups

For those looking to add id to name mappings with Druid (as an OLAP engine), you can utilize expressions in your **Dimension** settings. Simply use the lookup function and provide the name of the lookup and id, i.e. `lookup(city_id, 'cities')`. Be sure to include the lookup table name in single quotes.

```yaml
- label: "Cities"
expression: lookup(city_id, 'cities')
description: "Cities"
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
---
title: "Case Statements and Filters"
description: Tips & Tricks for Metric Formatting
sidebar_label: "Case Statements and Filters"
sidebar_position: 02
---

One of the most common advanced measure expressions are [`case`](https://duckdb.org/docs/stable/sql/expressions/case.html) statements and [`filters`](https://duckdb.org/docs/stable/sql/query_syntax/filter.html) used to filter or apply logic to part of the result set. Use cases for case statements include filtered sums (e.g. only sum if a flag is true) and bucketing data (e.g. if between threshold x and y the apply an aggregate). While similar, case statements give you a bit more flexibilty as it allows you to set a custom value depending on what the case is. See below for some examples!

<img src = '/img/build/metrics-view/examples/case-example.png' class='rounded-gif' />
<br />

Please review the reference documentation, [here.](/reference/project-files/metrics-view)

## Examples

### Case Statements
The following expression sums of the values of Global_active_power only when considered to be a lower value.

```yaml
- name: total_low_active_power_measure
display_name: Total Low Global Active Power
description: Total sum of Global Active Power where considered Low
expression: SUM(CASE WHEN GAP_category = 'Low' THEN Global_active_power ELSE 0 END)
format_preset: humanize
valid_percent_of_total: true
```

The following expression only considers the total value of users who are identified.

```yaml
- name: total_value_for_identified_users
display_name: Total Value for Identified Users
description: Total Sum of Value for Identifed Users
expression: SUM(CASE WHEN user_id != '' OR user_id IS NOT NULL THEN value ELSE 0 END)
format_preset: humanize
valid_percent_of_total: true
```

The following expression modifies the value of the column based on the value of column XX

```yaml
- name: modify_value
display_name: Arithmetic on Value
description: Arithmetic on Value based on XX
expression: |
SUM(
CASE
WHEN XX = 'multiply_10' THEN Value * 10
WHEN XX = 'multiply_2' THEN Value * 2
WHEN XX = 'divide_5' THEN Value / 5
END
)
format_preset: humanize
valid_percent_of_total: true
```


### Filters
Similar to the above case statements, you can use the filter expression to filter the data on a specific column's value. However, in the example where we are explicilty changing the value in the CASE statement, this is not possible using only a filter.

```yaml
- name: total_low_active_power_measure
display_name: Total Low Global Active Power
description: Total sum of Global Active Power where considered Low
expression: sum(Global_active_power) FILTER (WHERE GAP_category = 'Low')
format_preset: humanize
valid_percent_of_total: true
```

```yaml
- name: total_value_for_identified_users
display_name: Total Value for Identified Users
description: Total Sum of Value for Identifed Users
expression: SUM(value) FILTER (WHERE user_id != '' OR used_id IS NOT NULL)
format_preset: humanize
valid_percent_of_total: true
```
## Demo
[See this project live in our demo!](https://ui.rilldata.com/demo/rill-kaggle-elec-consumption/explore/household_power_consumption_metrics_explore)
69 changes: 69 additions & 0 deletions docs/docs/build/metrics-view/advanced-expressions/fixed-metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: "Fixed Metrics"
description: Tips & Tricks for Metric Formatting
sidebar_label: "Fixed Metrics"
sidebar_position: 04
---

Some metrics may be at a different level of granularity where a sum across the metric is no longer accurate. As an example, perhaps you have have a campaign with a daily budget of $5000 across five line items. Summing `daily_budget` column would give an inaccurate total of $25,000 budget per day. For those familiar Tableau, this is referred to as a `FIXED metric`.

<img src = '/img/build/metrics-view/examples/incorrect-sum.png' class='rounded-gif' />
<br />

To create the correct value, you can utilize DuckDB's unnest functionality. In the example below, you would be pulling a single value of `daily_budget` based on `campaign_id` to get the sum of budget for the day by campaign ids. Note that you can use mutliple keys if your granularity is defined by mulitple dimensions.

```yaml
expression: |
select
sum(a.val) as value
from
( select unnest(
list(distinct {
key: campaign_id,
val: daily_budget
})
) a
)
```

:::note

The syntax for fixed metrics is specific to DuckDB as an OLAP engine as it required DuckDB specific commands. However, you can create a similar SQL expression using a different OLAP engine, too!

:::
Please review the reference documentation, [here.](/reference/project-files/metrics-view)


## Example

In the following example, each publishing company has a monthly minimum guarantee. As you'll see in the measure, `incorrect_sum_of_guarantee`, you'll get an incorrect value as this will sum multiple values as there are multiple shows and days per publisher. Another workaround would be to use MIN, MAX or AVG but when selecting multiple publishers, the values will not be accurate.

<img src = '/img/build/metrics-view/examples/selecting-publishers.png' class='rounded-gif' />
<br />

```yaml
- name: incorrect_sum_of_guarantee
expression: sum(min_guarantee_usd)
format_preset: currency_usd
valid_percent_of_total: false

- name: guarantee_usd_measure_monthly
display_name: Monthly Minimum Guarantee USD
description: Total minimum guarantee in USD recorded in the dataset.
expression: >
SELECT SUM(a.val) AS value
FROM (
SELECT unnest(
list(distinct {
key: publisher_id,
month: date_trunc('month', date),
val: min_guarantee_usd
})
) a
)
format_preset: currency_usd
valid_percent_of_total: false
```

## Demo
[See this project live in our demo!](https://ui.rilldata.com/demo/sample-podcast-project/explore/podcast_explore)
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
title: "Formatting your Measures!"
description: Tips & Tricks for Metric Formatting
sidebar_label: "Metric Formatting"
sidebar_position: 01
---

When creating your measures in Rill, you have the option to pick from a preset of formats that we provide to you or use the [d3-format](https://d3js.org/d3-format) parameter to format your data in any way you like. While the big number in the explore dashboard won't apply all the decimals changes (it will add currency or percentage if that is the type), you will be able to see the changes in the dimension leaderboard and pivot tables.

<img src = '/img/build/metrics-view/metrics-editor.png' class='rounded-gif' />
<br />



Using `format_d3` to control the format of a measure in the metrics view allows for further customization.

:::tip Invalid format Strings
If an invalid format string is supplied, measures will be formatted with `format_preset: humanize`. If neither `format_preset` nor `format_d3` is supplied, measures will be formatted with the `humanize` preset).

:::

:::warning Cannot have both
Measures cannot have both `format_preset` and `format_d3` entries.
:::
Please review the reference documentation, [here.](/reference/project-files/metrics-view)

## Customization

For further customization of your measures, you can swtich to the YAML view and with our [metrics view reference documentation](/reference/project-files/metrics-view) use the [format_d3_locale](https://d3js.org/d3-format#formatLocale) parameter to create specific formatting.

```yaml
format_d3:
format_d3_locale:
grouping:
currency:
```


## Examples

As explained in the introduction, you'll notice that in each of the screenshot the Big Number doesn't always follow the exact formatting, but will change based on percentage / currency formatting. This is as designed as there is a fixed width that the number has to be displayed. Instead you'll see these values in the dimension leaderboard, TDD and pivot tables.

If you have any quesitons, please review our [reference documentation.](/reference/project-files/metrics-view)

### Format a measure to include specific amount of decimals
<img src = '/img/build/metrics-view/examples/decimal-example.png' class='rounded-gif' />
<br />

In the case that you need to view more granular values of your data, you can set the decimal places to whatever value you need. In the above example, we are setting the average voltage measure to 4 decimals spots to get a more accurate representation for each dimension.

```yaml
format_d3: ".4f"
```


### Format currency with different ',' locations. IE: Indian Rupee
<img src = '/img/build/metrics-view/examples/currency-example.png' class='rounded-gif' />
<br />


```yaml
format_d3: "$,"
format_d3_locale:
grouping: [3, 2, 2]
currency: ["₹", ""]
```
As Indian Rupees are formatted in a different way than USD and EUR, you'll need to use the `format_d3_locale` parameter to set the exact grouping and currency. Likewise if the currency symbol is written after the numeric value, you can set the currency to `["". "$"]`.

### Percentages
<img src = '/img/build/metrics-view/examples/percent-example.png' class='rounded-gif' />
<br />

```yaml
format_d3: '.4%'
```
While our `format_preset: percentage` will automatically apply `.2%`, you can manually set the value in format_d3 if you are looking for a more specific measure format.



## Demo
[See this project live in our demo!](https://ui.rilldata.com/demo/rill-kaggle-elec-consumption/explore/household_power_consumption_metrics_explore)
44 changes: 44 additions & 0 deletions docs/docs/build/metrics-view/advanced-expressions/quantiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: "Quantiles"
description: Tips & Tricks for Metric Formatting
sidebar_label: "Quantiles"
sidebar_position: 03
---

### Quantiles

In addition to common aggregates, you may wish to look at the value of a metric within a certain band or quantile. In the example below, we can measure the P95 of a given measure using `QUANTILE_CONT`.

<img src = '/img/build/metrics-view/examples/percentile-visual.png' class='rounded-gif' />
<br />


Using [DuckDB aggregate function](https://duckdb.org/docs/stable/sql/functions/aggregates.html#quantile_contx-pos), you can easily calculate various quantiles.

:::tip Not on DuckDB?
If you are using a different OLAP engine to power your dashboard, simply use the correct function for quantile.

IE: [Clickhouse quantile](https://clickhouse.com/docs/sql-reference/aggregate-functions/reference/quantile), [Pinot percentile](https://docs.pinot.apache.org/configuration-reference/functions/percentile)
:::
Please review the reference documentation, [here.](/reference/project-files/metrics-view)

## Examples

<img src = '/img/build/metrics-view/examples/percentile-example.png' class='rounded-gif' />
<br />

In this example we see the values of P95 and P99 are calculated using the following expressions:

```yaml
- name: p95_quantile_global_intensity
expression: QUANTILE_CONT(Global_intensity, 0.95)
format_d3: ".3f"
description: P95 of Global Intensity
- name: p99_quantile_global_intensity
expression: QUANTILE_CONT(Global_intensity, 0.99)
format_d3: ".4f"
description: P95 of Global Intensity
```

## Demo
[See this project live in our demo!](https://ui.rilldata.com/demo/rill-kaggle-elec-consumption/explore/household_power_consumption_metrics_explore)
Loading