-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[docs] Delete embedded-elt references in docs #27074
Conversation
Deploy preview for dagster-docs-beta ready! Preview available at https://dagster-docs-beta-gg9ce4xg4-elementl.vercel.app Direct link to changed pages: |
080add6
to
2ea45f0
Compare
2d55b00
to
40d0d84
Compare
2ea45f0
to
f837991
Compare
40d0d84
to
4a98ea9
Compare
f837991
to
b08529e
Compare
4a98ea9
to
5c0bfd8
Compare
b08529e
to
936ade3
Compare
5c0bfd8
to
dcecfc7
Compare
936ade3
to
7773bc0
Compare
dcecfc7
to
8abfbce
Compare
7773bc0
to
5293d9c
Compare
8abfbce
to
46383a5
Compare
5293d9c
to
7cc0074
Compare
46383a5
to
924837d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@maximearmstrong is there a top level page that we can redirect for external references, like these coming from the website?
master ~/src/dagster-website
$ grep embedded-elt
pages/lp/dagster-plus-trial.tsx
195: button_link="/blog/dagster-embedded-elt"
pages/events/embedded-elt-dec-2023.tsx
18: url: `https://dagster.io/images/events/embedded-elt-og-image-min.jpg?v=7`,
33: content="https://dagster.io/images/events/embedded-elt-og-image-min.jpg?v=7"
48: 'linear-gradient(rgba(000, 000, 000, 0.1), rgba(000, 000, 000, 0.1)), url("/images/events/embedded-elt-event-page-header-min.jpg")',
107: <a href="/blog/dagster-embedded-elt" className="underline">
pages/guides/etl/etl-pipelines-5-key-components-and-5-critical-best-practices.mdx
161:With a built-in toolkit for data ingestion thanks to our [Embedded ELT](https://docs.dagster.io/integrations/embedded-elt) feature and [integrations with leading providers like Fivetran](https://docs.dagster.io/integrations/fivetran), Dagster allows you to orchestrate your ETL workloads on your platform of choice.
pages/guides/etl/etl-tools-key-features-and-10-tools-to-know-in-2025.mdx
474:It also features integrations with leading providers like [Fivetran](https://docs.dagster.io/integrations/fivetran) and [Airbyte,](https://docs.dagster.io/integrations/airbyte) and a [built-in toolkit for data ingestion using the most popular open-source data integration libraries.](https://docs.dagster.io/integrations/embedded-elt)
pages/events/dagster-launch-week-oct-2023.tsx
215: <img className="h-10" src="/images/icons/embedded-elt.svg" />
268: link="/blog/dagster-embedded-elt"
pages/guides/etl.mdx
194:With a built-in toolkit for data ingestion thanks to our [Embedded ELT](https://docs.dagster.io/integrations/embedded-elt) feature and [integrations with leading providers like Fivetran](https://docs.dagster.io/integrations/fivetran), Dagster allows you to orchestrate your ETL workloads on your platform of choice.
pages/blog/elt-options-in-dagster.mdx
43:By selectively replacing your expensive data ingestion steps with Dagster embedded ELT [you can achieve both significant cost savings, and improve the performance of your data processes](https://dagster.io/blog/sling-out-your-etl-provider-with-embedded-elt).
77:Dagster's embedded ELT feature is a cost-effective and flexible solution for implementing ELT processes. It includes [components like Sling and dlt](https://dagster.io/blog/expanding-dagsters-embedded-elt-ecosystem-with-dlthub-for-data-ingestion) within the dagster-embedded-elt package, which facilitate data ingestion and transformation within Dagster pipelines. This feature provides better control over ingestion steps and integrates with Dagster's orchestration capabilities, making it a cost-effective choice.
pages/blog/see-the-forest-and-trees-dagster-plus.mdx
87:Once you’ve pinpointed the areas where an alternative to Fivetran would save you a lot of money, check out our blog on [using embedded elt](https://dagster.io/blog/sling-out-your-etl-provider-with-embedded-elt) and see how our integrations with [Sling](https://docs.dagster.io/integrations/embedded-elt/sling#sling--dagster) or [dlt](https://docs.dagster.io/integrations/embedded-elt/dlt) can help you reduce your spend.
pages/events/index.tsx
99: link="/blog/dagster-embedded-elt"
components/TrialFeatureCard.tsx
146: href="/blog/dagster-embedded-elt"
pages/blog/sling-out-your-etl-provider-with-embedded-elt.mdx
9:coverImage: '/posts/sling-out-your-etl-provider-with-embedded-elt/sling-out-your-etl-provider-min.jpg'
23:The [Dagster Embedded ELT](https://docs.dagster.io/integrations/embedded-elt) library provides a lightweight toolkit for ingesting and loading data with Dagster. In its current form, Embedded ELT provides a series of assets and resources for syncing data using Sling. Sling is a powerful and lightweight ingestion engine that allows us to define how data is synchronized from source to target data system. To learn more about Embedded ELT you can check out [Pedram’s blog post](https://dagster.io/blog/dagster-embedded-elt) introducing the library.
256: src="/posts/sling-out-your-etl-provider-with-embedded-elt/dagster-data-chart.png"
277: src="/posts/sling-out-your-etl-provider-with-embedded-elt/dagster-assets.png"
pages/blog/expanding-dagsters-embedded-elt-ecosystem-with-dlthub-for-data-ingestion.mdx
6:coverImage: '/posts/expanding-dagster-embedded-elt-ecosystem-with-dltHub/dagster-dlt-cover-min.jpg'
13:Today, we’re excited to introduce our officially supported dlt integration. Last October we introduced the [first integration in the Embedded ELT family](https://dagster.io/blog/dagster-embedded-elt) of integrations; leveraging Sling to easily ingest and replicate data between systems.
20: src="/posts/expanding-dagster-embedded-elt-ecosystem-with-dltHub/dagster-dlt-image-one.png"
25: https://pypistats.org/packages/dagster-embedded-elt
78: src="/posts/expanding-dagster-embedded-elt-ecosystem-with-dltHub/dagster-dlt-image-two.png"
85:For more advanced scenarios the dlt integration has followed the implementation architecture of the dbt and Sling integrations, in which a translator can be defined to adjust the parameters of the generated assets. For more information, reference [the API documentation](https://docs.dagster.io/_apidocs/libraries/dagster-embedded-elt#dlt-dagster-embedded-elt-dlt).
103:- [Dagster Embedded ELT Documentation](https://docs.dagster.io/integrations/embedded-elt)
pages/blog/data-platform-week-2024.mdx
71:Additionally, we are continuously investing in purpose-built open-source tooling to make it easier than ever for data teams to use the right tool for the job. With our embedded ELT package, we incorporated ingestion tools Sling and dlt so data teams can get data into their platforms cost-effectively. When we switched to using embedded ELT internally, we saved [$40K annually](https://dagster.io/blog/sling-out-your-etl-provider-with-embedded-elt) on managed solution spending.
pages/integrations/dagster-sling.mdx
8:apireflink: 'https://docs.dagster.io/_apidocs/libraries/dagster-embedded-elt'
9:docslink: 'https://docs.dagster.io/integrations/embedded-elt/sling'
27:pip install dagster-embedded-elt
pages/integrations/dagster-dlt.mdx
8:apireflink: 'https://docs.dagster.io/_apidocs/libraries/dagster-embedded-elt'
9:docslink: 'https://docs.dagster.io/integrations/embedded-elt/dlt'
27:pip install dagster-embedded-elt
pages/blog/dagster-embedded-elt.mdx
9:coverImage: '/posts/dagster-embedded-elt/dagster-embedded-elt-cover-min.jpg'
19: src="/posts/dagster-embedded-elt/dagster-embedded-etl-ingestion-tools-min.jpeg"
41: src="/posts/dagster-embedded-elt/embedded-elt-embeddable-libraries-min.png"
82:In that spirit, we’re introducing _dagster-embedded-elt_: a framework for building ELT pipelines with Dagster through helpful pre-built assets and resources.
89: src="/posts/dagster-embedded-elt/embedded-elt-sling-connectors-min.png"
104:To use our `dagster-embedded-elt` to sync data between systems, it's as simple creating a replication configuration, defining your sources and targets, and calling the replicate method.
178: Learn more about Embedded ELT in <a href="https://docs.dagster.io/integrations/embedded-elt">the Dagster Docs</a>.
182:* [Sling Out Your ETL Provider with Embedded ELT](sling-out-your-etl-provider-with-embedded-elt)
183:* [Expanding the Dagster Embedded ELT Ecosystem with dltHub for Data Ingestion](expanding-dagsters-embedded-elt-ecosystem-with-dlthub-for-data-ingestion)
9917778
to
bb05b7c
Compare
a2b689c
to
e97d5c6
Compare
bb05b7c
to
ca44917
Compare
e97d5c6
to
fdfa196
Compare
ca44917
to
8ff6725
Compare
fdfa196
to
5cc6e0c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like the mdx needs to be synced:
[2025-01-14T17:32:45Z] python file=/integrations/sling/sling_connection_resources.py
[2025-01-14T17:32:45Z] -from dagster_sling.resources import SlingConnectionResource, SlingResource
[2025-01-14T17:32:45Z] +from dagster_sling import SlingConnectionResource, SlingResource
[2025-01-14T17:32:45Z]
[2025-01-14T17:32:45Z] from dagster import EnvVar
[2025-01-14T17:32:45Z]
[2025-01-14T17:32:46Z]
[2025-01-14T17:32:46Z] real 0m0.934s
[2025-01-14T17:32:46Z] user 0m0.811s
[2025-01-14T17:32:46Z] sys 0m0.139s
[2025-01-14T17:32:49Z] 🚨 Error: The command exited with status 1
8ff6725
to
87139b3
Compare
5cc6e0c
to
0220534
Compare
87139b3
to
d4595cf
Compare
0220534
to
f2b0182
Compare
d4595cf
to
e8e90d5
Compare
f2b0182
to
d0a8534
Compare
d0a8534
to
607b193
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Language on /integrations/embedded-elt
will need to be updated to reflect that it's two packages.
content/integrations/embedded-elt.mdx
10:The `dagster-embedded-elt` package provides a framework for building ELT pipelines with Dagster through helpful asset decorators and resources.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved pending small change to /integrations/embedded-elt
, can also be done in follow-up.
## Summary & Motivation Fixes this [comment](#27074 (review)) in #27074 ## How I Tested These Changes Docs preview
## Summary & Motivation We keep the top page at `/integrations/embedded-elt` but references across docs and examples are deleted in favor of references to the dlt and Sling pages.
## Summary & Motivation Fixes this [comment](dagster-io#27074 (review)) in dagster-io#27074 ## How I Tested These Changes Docs preview
Summary & Motivation
We keep the top page at
/integrations/embedded-elt
but references across docs and examples are deleted in favor of references to the dlt and Sling pages.