Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add credit consumption to cost per query #127

Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Features-20230815-013055.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Features
body: Add credit consumption to cost per query
time: 2023-08-15T01:30:55.1004512Z
custom:
Author: yingyingqiqi
PR: "127"
9 changes: 8 additions & 1 deletion models/cost_per_query.sql
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ query_cost as (
select
query_seconds_per_hour.*,
credits_billed_hourly.credits_used_compute * daily_rates.effective_rate as actual_warehouse_cost,
credits_billed_hourly.credits_used_compute * query_seconds_per_hour.fraction_of_total_query_time_in_hour * daily_rates.effective_rate as allocated_compute_cost_in_hour
credits_billed_hourly.credits_used_compute * query_seconds_per_hour.fraction_of_total_query_time_in_hour * daily_rates.effective_rate as allocated_compute_cost_in_hour,
credits_billed_hourly.credits_used_compute * query_seconds_per_hour.fraction_of_total_query_time_in_hour as allocated_compute_credits_in_hour
from query_seconds_per_hour
inner join credits_billed_hourly
on query_seconds_per_hour.warehouse_id = credits_billed_hourly.warehouse_id
Expand All @@ -91,6 +92,7 @@ cost_per_query as (
any_value(end_time) as end_time,
any_value(execution_start_time) as execution_start_time,
sum(allocated_compute_cost_in_hour) as compute_cost,
sum(allocated_compute_credits_in_hour) as compute_credits,
any_value(credits_used_cloud_services) as credits_used_cloud_services,
any_value(ran_on_warehouse) as ran_on_warehouse
from query_cost
Expand All @@ -114,6 +116,7 @@ all_queries as (
end_time,
execution_start_time,
compute_cost,
compute_credits,
credits_used_cloud_services,
ran_on_warehouse
from cost_per_query
Expand All @@ -126,6 +129,7 @@ all_queries as (
end_time,
execution_start_time,
0 as compute_cost,
0 as compute_credits,
credits_used_cloud_services,
ran_on_warehouse
from filtered_queries
Expand All @@ -139,12 +143,15 @@ select
all_queries.end_time,
all_queries.execution_start_time,
all_queries.compute_cost,
all_queries.compute_credits,
-- For the most recent day, which is not yet complete, this calculation won't be perfect.
-- For example, at 12PM on the latest day, it's possible that cloud credits make up <10% of compute cost, so the queries
-- from that day are not allocated any cloud_services_cost. The next time the model runs, after we have the full day of data,
-- this may change if cloud credits make up >10% of compute cost.
(div0(all_queries.credits_used_cloud_services, credits_billed_daily.daily_credits_used_cloud_services) * credits_billed_daily.daily_billable_cloud_services) * coalesce(daily_rates.effective_rate, current_rates.effective_rate) as cloud_services_cost,
div0(all_queries.credits_used_cloud_services, credits_billed_daily.daily_credits_used_cloud_services) * credits_billed_daily.daily_billable_cloud_services as cloud_services_credits,
all_queries.compute_cost + cloud_services_cost as query_cost,
all_queries.compute_credits + cloud_services_credits as query_credits,
all_queries.ran_on_warehouse,
coalesce(daily_rates.currency, current_rates.currency) as currency
from all_queries
Expand Down
6 changes: 6 additions & 0 deletions models/cost_per_query.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,16 @@ models:
description: When the query began executing on the warehouse (in the UTC time zone). This will always be after the start_time.
- name: compute_cost
description: Compute costs associated with the query, in the primary currency of your account. Can be 0 if the query did not run on a warehouse.
- name: compute_credits
description: Compute credits associated with the query, in the primary currency of your account. Can be 0 if the query did not run on a warehouse.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: cloud_services_cost
description: Cloud service costs associated with the query, in the primary currency of your account. Can be 0 if total cloud services credits consumption was less than 10% of total compute credits consumption on that day.
- name: cloud_services_credits
description: Cloud service credits associated with the query, in the primary currency of your account. Can be 0 if total cloud services credits consumption was less than 10% of total compute credits consumption on that day.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: query_cost
description: Total cost associated with the query, calculated as sum of compute_cost and cloud_services_cost, in the primary currency of your account.
- name: query_credits
description: Total cost associated with the query, calculated as sum of compute_credits and cloud_services_credits, in the primary currency of your account.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: ran_on_warehouse
description: Indicator for whether the query ran on a warehouse. Certain queries, such as metadata queries, can be entirely processed in cloud services.
- name: currency
Expand Down
3 changes: 3 additions & 0 deletions models/query_history_enriched.sql
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,11 @@ cost_per_query as (
select
cost_per_query.query_id,
cost_per_query.compute_cost,
cost_per_query.compute_credits,
cost_per_query.cloud_services_cost,
cost_per_query.cloud_services_credits,
cost_per_query.query_cost,
cost_per_query.query_credits,
cost_per_query.execution_start_time,

-- Grab all columns from query_history (except the query time columns which we rename below)
Expand Down
6 changes: 6 additions & 0 deletions models/query_history_enriched.yml
Original file line number Diff line number Diff line change
Expand Up @@ -156,10 +156,16 @@ models:
description: When the query began executing on the warehouse (in the UTC time zone). This will always be after the start_time.
- name: compute_cost
description: Compute costs associated with the query, in the primary currency of your account. Can be 0 if the query did not run on a warehouse.
- name: compute_credits
description: Compute credits associated with the query, in the primary currency of your account. Can be 0 if the query did not run on a warehouse.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: cloud_services_cost
description: Cloud service costs associated with the query, in the primary currency of your account. Can be 0 if total cloud services credits consumption was less than 10% of total compute credits consumption on that day.
- name: cloud_services_credits
description: Cloud service credits associated with the query, in the primary currency of your account. Can be 0 if total cloud services credits consumption was less than 10% of total compute credits consumption on that day.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: query_cost
description: Total cost associated with the query, calculated as sum of compute_cost and cloud_services_cost, in the primary currency of your account.
- name: query_credits
description: Total credits associated with the query, calculated as sum of compute_credits and cloud_services_credits, in the primary currency of your account.
NiallRees marked this conversation as resolved.
Show resolved Hide resolved
- name: currency
description: Spend currency, retrieved from Snowflake's daily rate sheet

Expand Down
Loading