Releases: fivetran/dbt_shopify_source
v0.20.0 dbt_shopify_source
PR #111 includes the following updates:
dbt Fusion Compatibility Updates
- Updated package to maintain compatibility with dbt-core versions both before and after v1.10.6, which introduced a breaking change to multi-argument test syntax (e.g.,
unique_combination_of_columns
). - Temporarily removed unsupported tests to avoid errors and ensure smoother upgrades across different dbt-core versions. These tests will be reintroduced once a safe migration path is available.
- Removed all
dbt_utils.unique_combination_of_columns
tests. - Removed all
accepted_values
tests.
- Removed all
Full Changelog: v0.19.0...v0.20.0
v0.20.0-a1 dbt_shopify_source
PR #109 introduces the following changes:
Feature Update
This pre-release adjusts v0.19.0 so that the GraphQL API-based models are run by default, rather than the REST API ones. It therefore updates the default value of the shopify_api
variable to be graphql
instead of its original rest
value.
This version is intended to be used by Fivetran Quickstart users who cannot manually adjust the shopify_api
variable value and have GraphQL Shopify connections.
v0.19.0 dbt_shopify_source
PR #108 includes the following updates:
Feature Update: GraphQL API Support
Fivetran very recently released a new version of the Shopify connector that leverages Shopify's newer GraphQL API instead of the REST API, as Shopify deprecated the REST API in October 2024. The GraphQL and REST API-based schemas are slightly different, but this package is designed to run for either or, not both. It will do so based on the value of the shopify_api
variable.
By default, shopify_api
is set to rest
and will run the stg_shopify__*
models in the rest
folder. If you would like to run the package on a GraphQL-based schema, adjust shopify_api
accordingly. This will run the stg_shopify_gql__*
models in the graphql
folder.
This variable is dynamically configured for you in Fivetran Quickstart based on your Shopify connection details.
vars:
shopify_api: graphql # By default = rest. Must be lowercase
Overall, the package aims for parity across the different API versions and aligns column names with their REST names, if the fields are supported in GraphQL. There will be a largely 1:1 relationship between REST API and GraphQL based models, except for models based on the following source tables:
ABANDONED_CHECKOUT_SHIPPING_LINE
: This is not available in the GraphQL schema, so there will be nostg_shopify_gql__abandoned_checkout_shipping_line
model or downstream transformations.ORDER_URL_TAG
: This is not available in the GraphQL schema, so there will be nostg_shopify_gql__order_url_tag
model or downstream transformations.
Full Changelog: v0.18.2...v0.19.0
v0.18.2 dbt_shopify_source
PR #104 includes the following updates:
Bug Fixes
- Removed the
not_null
test for theimage_id
field from thestg_shopify__media_image
model as this field may be empty.
PR #105 includes the following updates:
Under the Hood - July 2025 Updates
- Updated conditions in
.github/workflows/auto-release.yml
. - Added
.github/workflows/generate-docs.yml
. - Migrated
flags
(e.g.,send_anonymous_usage_stats
,use_colors
) fromsample.profiles.yml
tointegration_tests/dbt_project.yml
. - Updated
maintainer_pull_request_template.md
with improved checklist. - Refreshed README tag block:
- Updated Python image version to
3.10.13
inpipeline.yml
. - Updated
.gitignore
to exclude additional DBT, Python, and system artifacts.
Full Changelog: v0.18.1...v0.18.2
v0.18.1 dbt_shopify_source
PR #103 includes the following updates:
Under the Hood
- Added BigQuery JSON field support for the following model and columns:
stg_shopify__order
:total_shipping_price_set
,total_discounts_set
,total_line_items_price_set
,total_price_set
, andtotal_tax_set
columnsstg_shopify__transaction
:receipt
columnstg_shopify__collection
:rules
columnstg_shopify_shop
:enabled_presentment_currencies
column
- Added the
json_to_string()
macro for BigQuery to convert JSON fields to strings for reliable downstream parsing. - Included json versions to the integration tests to ensure json data type compatibility.
- Updated the Maintainer PR template.
Full Changelog: v0.18.0...v0.18.1
v0.18.0 dbt_shopify_source
PR #100 includes the following updates:
Schema And Data Updates
24 new models -- 6 deprecated models -- 1 deprecated field
Data Model | Change Type | Old Name | New Name | Notes |
---|---|---|---|---|
stg_shopify__discount_code |
Deprecated Staging Model | Deprecated the discount_code source table. |
||
stg_shopify__discount_code_tmp |
Deprecated Temp Model | Deprecated the discount_code_app source table. |
||
stg_shopify__price_rule |
Deprecated Staging Model | Deprecated the price_rule source table. |
||
stg_shopify__price_rule_tmp |
Deprecated Temp Model | Deprecated the price_rule source table. |
||
stg_shopify__product_image |
Deprecated Staging Model | Deprecated the product_image source table. |
||
stg_shopify__product_image_tmp |
Deprecated Temp Model | Deprecated the product_image source table. |
||
stg_shopify__product_variant | Deprecated Columns | image_id |
None | No longer supported in product_variant . |
stg_shopify__discount_code_app | New Staging Model | Source: discount_code_app table. |
||
stg_shopify__discount_code_basic | New Staging Model | Source: discount_code_basic table. |
||
stg_shopify__discount_code_bxgy | New Staging Model | Source: discount_code_bxgy table. |
||
stg_shopify__discount_code_free_shipping | New Staging Model | Source: discount_code_free_shipping table. |
||
stg_shopify__discount_application | New Staging Model | Source: discount_application table. |
||
stg_shopify__discount_allocation | New Staging Model | Source: discount_allocation table. |
||
stg_shopify__discount_basic_code | New Staging Model | Source: discount_basic_code table. |
||
stg_shopify__discount_redeem_code | New Staging Model | Source: discount_redeem_code table. |
||
stg_shopify__media | New Staging Model | Source: media table. |
||
stg_shopify__media_image | New Staging Model | Source: media_image table. |
||
stg_shopify__product_media | New Staging Model | Source: product_media table. |
||
stg_shopify__product_variant_media | New Staging Model | Source: product_variant_media table. |
||
stg_shopify__discount_code_app_tmp | New Temp Model | Source: discount_code_app table. |
||
stg_shopify__discount_code_basic_tmp | New Temp Model | Source: discount_code_basic table. |
||
stg_shopify__discount_code_bxgy_tmp | New Temp Model | Source: discount_code_bxgy table. |
||
stg_shopify__discount_code_free_shipping_tmp | New Temp Model | Source: discount_code_free_shipping table. |
||
stg_shopify__discount_application_tmp | New Temp Model | Source: discount_application table. |
||
stg_shopify__discount_allocation_tmp | New Temp Model | Source: discount_allocation table. |
||
stg_shopify__discount_basic_code_tmp | New Temp Model | Source: discount_basic_code table. |
||
stg_shopify__discount_redeem_code_tmp | New Temp Model | Source: discount_redeem_code table. |
||
stg_shopify__media_tmp | New Temp Model | Source: media table. |
||
stg_shopify__media_image_tmp | New Temp Model | Source: media_image table. |
||
stg_shopify__product_media_tmp | New Temp Model | Source: product_media table. |
||
stg_shopify__product_variant_media_tmp | New Temp Model | Source: product_variant_media table. |
Release Notes
- Applied the above schema changes in accordance with the April 2025 Fivetran connector update to accommodate new changes in the Shopify API. See the release notes for more details.
- Major updates included:
- Deprecated staging models and field that stem from the
discount_code
,price_rule
,product_image
sources, andimage_id
in theproduct_variant
table. - New models created that originate from the
discount_code_app
,discount_code_basic
,discount_code_bxgy
.discount_code_free_shipping
,discount_application
,discount_allocation
,product_image
,product_variant_media
,media
,media_image
source tables.
- Deprecated staging models and field that stem from the
Feature Updates
- If users are utilizing the
discount_code_app
and/orproduct_variant_media
sources, these models can be enabled by setting the variablesshopify_using_discount_code_app
and/orshopify_using_product_variant_media
respectively totrue
. More instructions are available in the README.
Under the Hood
- Created and removed seed files to ensure end models in the concurrent
dbt_shopify
release run successfully.
Documentation Notes
- Added/removed yml documentation for new/deprecated models and fields.
Full Changelog: v0.17.0...v0.18.0
v0.17.0 dbt_shopify_source
PR #98 includes the following updates:
Dependency Changes
- Removed the dependency on calogica/dbt_date as it is no longer actively maintained.
- To maintain functionality, the highly leveraged
dbt_date.convert_timezone
macro (see README for how to use) has been replicated within themacros/fivetran_date_macros
folder with minimal modifications. It has been prefixed withfivetran_
to avoid potential naming conflicts:dbt_date.convert_timezone
->shopify_source.fivetran_convert_timezone
Full Changelog: v0.16.0...v0.17.0
v0.16.0 dbt_shopify_source
PR #97 includes the following updates:
Breaking Changes
- Introduced the
inventory_quantity
source and thestg_shopify__inventory_quantity
model to support downstream inventory quantity tracking. See the documentation for details on the newly added columns.- This replaces the deprecated
available_quantity
column instg_shopify__inventory_level
.
- This replaces the deprecated
Documentation
- Added definitions for
inventory_quantity
andstg_shopify__inventory_quantity
.
Under the Hood
- Added seed
inventory_quantity_data
and macroget_inventory_quantity_columns
to support the newinventory_quantity
source.
Full Changelog: v0.15.0...v0.16.0
v0.15.0 dbt_shopify_source
This release includes the following updates:
Under the Hood
- (Affects Redshift only) Creates new
shopify_union_data
macro to accommodate Redshift's treatment of empty tables. (PR #95)- For each staging model, if the source table is not found in any of your schemas, the package will create a table with one row with null values for Redshift destinations. There will be no change in behavior in non-Redshift warehouses.
- This is necessary as Redshift will ignore explicit data casts when a table is completely empty and materialize every column as a
varchar
. This throws errors in downstream transformations in theshopify
package. The 1 row will ensure that Redshift will respect the package's datatype casts.
Documentation
- Corrected references to connectors and connections in the README. (#94)
Full Changelog: v0.14.0...v0.15.0
v0.14.0 dbt_shopify_source
PR #93 includes the following changes:
Breaking Changes
-
Updated for connector changes released on January 6, 2025. See the release notes for more details. Added the following columns to the
stg_shopify__*
staging tables:inventory_item
:duplicate_sku_count
harmonized_system_code
inventory_history_url
legacy_resource_id
measurement_id
measurement_weight_value
measurement_weight_unit
is_tracked_editable_locked
tracked_editable_reason
unit_cost_amount
unit_cost_currency_code
inventory_level
:inventory_level_id
can_deactivate
created_at
deactivation_alert
product_variant
:is_available_for_sale
display_name
legacy_resource_id
has_components_required
sellable_online_quantity
product_image
:media_id
status
url
-
For backward compatibility, the following columns were coalesced to combine values from the old column name with the new column name, with the resulting column retaining the new name:
inventory_item
:- The deprecated
cost
column is coalesced with the new columnunit_cost_amount
asunit_cost_amount
.
- The deprecated
product_image
:- The deprecated
src
column is coalesced with the new columnurl
asurl
.
- The deprecated
-
Marked the following columns as deprecated in the documentation. These columns will return
null
values following the connector update, and customers should expect this behavior until the columns are fully removed in a future release.inventory_level
:available_quantity
product_variant
:fulfillment_service
grams
inventory_management
requires_shipping
weight
weight_unit
option_*
product_image
:position
created_at
updated_at
variant_ids
Under the Hood
- Updated
shopify_*_data
seed data andget_*_columns
macros to include new columns for the following tables:inventory_item
inventory_level
product_image
product_variant
- Standardized boolean casting by updating
get_*_columns
to usedbt.type_boolean()
for consistency.
Full Changelog: v0.13.0...v0.14.0