-
Notifications
You must be signed in to change notification settings - Fork 943
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dbt clone command #3742
Merged
Merged
dbt clone command #3742
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
f2c0b62
add initial clone updates
graciegoheen eb71a3b
added initial clone command page
graciegoheen db5b2cc
update project state page
graciegoheen e575d98
Merge branch 'current' into dbt-clone
graciegoheen 98ff0b0
Fix link
dbeatty10 184a085
Update link
dbeatty10 14d9c4c
Update explanations for zero-copy cloning (or not)
dbeatty10 b1f734f
Link to docs for specifying / establishing state
dbeatty10 3f46631
Remove note in favor of link to docs for "establishing state"
dbeatty10 d9bcdb7
Incorporate feedback on phrasing for blue/green continuous deployment…
dbeatty10 a96502e
Incorporate feedback for delineating source and target environments
dbeatty10 6def43d
Merge branch 'current' into dbt-clone
graciegoheen 30a856a
add initial defer docs
graciegoheen e1b9972
add caveats for useful bullets
graciegoheen 1a087c7
error note
graciegoheen 4a844c1
Merge branch 'current' into dbt-clone
graciegoheen c42a530
error message begone
graciegoheen b3002bf
Merge branch 'dbt-clone' of https://github.com/dbt-labs/docs.getdbt.c…
graciegoheen bd77486
Merge branch 'current' into dbt-clone
graciegoheen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
--- | ||
title: "About dbt clone command" | ||
sidebar_label: "clone" | ||
id: "clone" | ||
--- | ||
|
||
The `dbt clone` command clones selected nodes from the [specified state](/reference/node-selection/syntax#establishing-state) to the target schema(s). This command makes use of the `clone` materialization: | ||
- If your data platform supports zero-copy cloning of tables, and this model exists as a table in the source environment, dbt will create it in your target environment as a clone | ||
- Otherwise, dbt will create a simple pointer view (`select * from` the source object) | ||
- By default, `dbt clone` will not recreate pre-existing relations in the current target. To override this, use the `--full-refresh` flag. | ||
- You may want to specify a higher number of [threads](/docs/running-a-dbt-project/using-threads) to decrease execution time since individual clone statements are independent of one another. | ||
|
||
The `clone` command is useful for: | ||
- blue/green continuous deployment (on data warehouses that support zero-copy cloning tables) | ||
- cloning current production state into development schema(s) | ||
- handling incremental models in Slim CI dbt Cloud jobs (on data warehouses that support zero-copy cloning tables) | ||
- testing code changes on downstream dependencies in your BI tool | ||
|
||
```bash | ||
# clone all of my models from specified state to my target schema(s) | ||
dbt clone --state path/to/artifacts | ||
|
||
# clone one_specific_model of my models from specified state to my target schema(s) | ||
dbt clone --select one_specific_model --state path/to/artifacts | ||
|
||
# clone all of my models from specified state to my target schema(s) and recreate all pre-exisiting relations in the current target | ||
dbt clone --state path/to/artifacts --full-refresh | ||
|
||
# clone all of my models from specified state to my target schema(s), running up to 50 clone statements in parallel | ||
dbt clone --state path/to/artifacts --threads 50 | ||
``` | ||
|
||
### When to use `dbt clone` instead of [deferral](/reference/node-selection/defer)? | ||
|
||
Unlike deferral, `dbt clone` requires some compute and creation of additional objects in your data warehouse. In many cases, deferral is a cheaper and simpler alternative to `dbt clone`. However, `dbt clone` covers additional use cases where deferral may not be possible. | ||
|
||
For example, by creating actual data warehouse objects, `dbt clone` allows you to test out your code changes on downstream dependencies _outside of dbt_ (such as a BI tool). | ||
|
||
As another example, you could `clone` your modified incremental models as the first step of your dbt Cloud CI job to prevent costly `full-refresh` builds for warehouses that support zero-copy cloning. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice!