Skip to content

Commit

Permalink
[docs] Delete embedded-elt docs
Browse files Browse the repository at this point in the history
  • Loading branch information
maximearmstrong committed Jan 14, 2025
1 parent 7cc0074 commit df9d2c1
Show file tree
Hide file tree
Showing 13 changed files with 9 additions and 70 deletions.
7 changes: 0 additions & 7 deletions docs/content/_apidocs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -438,13 +438,6 @@ Dagster also provides a growing set of optional add-on libraries to integrate wi
</td>
<td>Provides support for storing PySpark DataFrames in DuckDB.</td>
</tr>
<tr>
<td>
<a href="/_apidocs/libraries/dagster-embedded-elt">Embedded ELT</a> (
<code>dagster-embedded-elt</code>)
</td>
<td>Provides support for running embedded ELT within Dagster</td>
</tr>
<tr>
<td>
<a href="/_apidocs/libraries/dagster-fivetran">Fivetran</a> (
Expand Down
4 changes: 0 additions & 4 deletions docs/content/_navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -1594,10 +1594,6 @@
"title": "DuckDB & Polars (dagster-duckdb-polars)",
"path": "/_apidocs/libraries/dagster-duckdb-polars"
},
{
"title": "Embedded ELT (dagster-embedded-elt)",
"path": "/_apidocs/libraries/dagster-embedded-elt"
},
{
"title": "Fivetran (dagster-fivetran)",
"path": "/_apidocs/libraries/dagster-fivetran"
Expand Down
Binary file modified docs/content/api/modules.json.gz
Binary file not shown.
Binary file modified docs/content/api/searchindex.json.gz
Binary file not shown.
Binary file modified docs/content/api/sections.json.gz
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ defs = Definitions(assets=assets)

## Multi-asset decorators

In the scenario where a single API call or configuration can result in multiple assets, with a shared runtime or dependencies, one may consider creating a multi-asset decorator. Example implementations of this approach include [dbt](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-dbt), [dlt](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-embedded-elt/dagster_embedded_elt/dlt), and [Sling](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-embedded-elt/dagster_embedded_elt/sling).
In the scenario where a single API call or configuration can result in multiple assets, with a shared runtime or dependencies, one may consider creating a multi-asset decorator. Example implementations of this approach include [dbt](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-dbt), [dlt](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-dlt/dagster_dlt/dlt), and [Sling](https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-sling/dagster_sling/sling).

### Pros

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -386,10 +386,10 @@ def custom_replication_assets(
></ArticleListItem>
<ArticleListItem
title="dlt multi-asset integration example"
href="https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-embedded-elt/dagster_embedded_elt/dlt/asset_decorator.py"
href="https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-dlt/dagster_dlt/asset_decorator.py"
></ArticleListItem>
<ArticleListItem
title="Sling multi-asset integration example"
href="https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-embedded-elt/dagster_embedded_elt/sling/asset_decorator.py"
href="https://github.com/dagster-io/dagster/blob/master/python_modules/libraries/dagster-sling/dagster_sling/asset_decorator.py"
></ArticleListItem>
</ArticleList>
13 changes: 1 addition & 12 deletions docs/content/integrations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -71,10 +71,6 @@ Explore guides for integrations with external services.
></ArticleListItem>
<ArticleListItem title="DuckDB" href="/integrations/duckdb"></ArticleListItem>
<ArticleListItem title="dlt" href="/integrations/dlt"></ArticleListItem>
<ArticleListItem
title="Embedded ELT"
href="/integrations/embedded-elt"
></ArticleListItem>
<ArticleListItem
title="Fivetran"
href="/integrations/fivetran"
Expand All @@ -89,10 +85,7 @@ Explore guides for integrations with external services.
title="Pandera"
href="/integrations/pandera"
></ArticleListItem>
<ArticleListItem
title="Sling"
href="/integrations/sling"
></ArticleListItem>
<ArticleListItem title="Sling" href="/integrations/sling"></ArticleListItem>
<ArticleListItem title="Spark" href="/integrations/spark"></ArticleListItem>
<ArticleListItem
title="Snowflake"
Expand Down Expand Up @@ -147,10 +140,6 @@ Explore libraries that are maintained by the Dagster core team.
title="dlt"
href="/_apidocs/libraries/dagster-dlt"
></ArticleListItem>
<ArticleListItem
title="Embedded ELT"
href="/_apidocs/libraries/dagster-embedded-elt"
></ArticleListItem>
<ArticleListItem
title="Fivetran"
href="/_apidocs/libraries/dagster-fivetran"
Expand Down
6 changes: 2 additions & 4 deletions docs/content/integrations/dlt.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: Ingest data with ease using Dagster and dlt

# dlt & Dagster

<ExperimentalCallout />

The [data load tool (dlt)](https://dlthub.com/) open-source library defines a standardized approach for creating data pipelines that load often messy data sources into well-structured data sets. It offers many advanced features, such as:

- Handling connection secrets
Expand Down Expand Up @@ -392,10 +394,6 @@ Want to see real-world examples of dlt in production? Check out how we use it in
## Related
<ArticleList>
<ArticleListItem
title="Embedded ELT"
href="/integrations/embedded-elt"
></ArticleListItem>
<ArticleListItem
title="Sling & Dagster"
href="/integrations/sling"
Expand Down
33 changes: 0 additions & 33 deletions docs/content/integrations/embedded-elt.mdx

This file was deleted.

6 changes: 2 additions & 4 deletions docs/content/integrations/sling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: Replicate data across systems using the Sling integration

# Sling & Dagster

<ExperimentalCallout />

Sling provides an easy-to-use YAML configuration layer for loading data from files, replicating data between databases, exporting custom SQL queries to cloud storage, and much more.

---
Expand Down Expand Up @@ -294,10 +296,6 @@ def my_assets(context, sling: SlingResource):
## Related

<ArticleList>
<ArticleListItem
title="Embedded ELT"
href="/integrations/embedded-elt"
></ArticleListItem>
<ArticleListItem
title="dlt & Dagster"
href="/integrations/dlt"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ dagster_components.pipes_subprocess_script_collection
This is because the basic `dagster-components` package is lightweight and doesn't include components for specific tools. We can get access to a `sling` component by installing the `sling` extra:

```bash
uv add 'dagster-components[sling]' dagster-embedded-elt
uv add 'dagster-components[sling]' dagster-sling
```

Now let's see what's available:
Expand Down
2 changes: 0 additions & 2 deletions docs/tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ deps =
-e ../python_modules/libraries/dagster-wandb
-e ../python_modules/libraries/dagster-dlt
-e ../python_modules/libraries/dagster-sling
-e ../python_modules/libraries/dagster-embedded-elt
-e ../python_modules/libraries/dagster-deltalake
-e ../python_modules/libraries/dagster-deltalake-pandas
-e ../python_modules/libraries/dagster-deltalake-polars
Expand Down Expand Up @@ -84,7 +83,6 @@ deps =
-e ../python_modules/libraries/dagster-wandb
-e ../python_modules/libraries/dagster-dlt
-e ../python_modules/libraries/dagster-sling
-e ../python_modules/libraries/dagster-embedded-elt
-e ../python_modules/libraries/dagster-deltalake
-e ../python_modules/libraries/dagster-deltalake-pandas
-e ../python_modules/libraries/dagster-deltalake-polars
Expand Down

0 comments on commit df9d2c1

Please sign in to comment.