Skip to content

Commit 9de40ee

Browse files
authored
Add structure for remaining guides content (#26423)
## Summary & Motivation Reorganize 'guides' and 'getting started' content (['Docs' section](https://docs-preview.dagster.io/) of docs) to prepare for remaining content. No need for a line-level review on this one; we just need to make sure the tests are green (except Vale—that's a bigger problem), and that staging loads and looks basically fine. ## How I Tested These Changes Local build ## Changelog > Insert changelog entry or delete this section. --------- Signed-off-by: nikki everett <[email protected]>
1 parent 24b60b3 commit 9de40ee

File tree

116 files changed

+416
-240
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

116 files changed

+416
-240
lines changed

docs/docs-beta/docs/dagster-plus/deployment/deployment-types/serverless/security.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,11 +22,11 @@ The default I/O manager cannot be used if you are a Serverless user who:
2222
- Are otherwise working with data subject to GDPR or other such regulations
2323
:::
2424

25-
In Serverless, code that uses the default [I/O manager](/guides/build/configure/io-managers) is automatically adjusted to save data in Dagster+ managed storage. This automatic change is useful because the Serverless filesystem is ephemeral, which means the default I/O manager wouldn't work as expected.
25+
In Serverless, code that uses the default [I/O manager](/guides/operate/io-managers) is automatically adjusted to save data in Dagster+ managed storage. This automatic change is useful because the Serverless filesystem is ephemeral, which means the default I/O manager wouldn't work as expected.
2626

2727
However, this automatic change also means potentially sensitive data could be **stored** and not just processed or orchestrated by Dagster+.
2828

29-
To prevent this, you can use [another I/O manager](/guides/build/configure/io-managers#built-in) that stores data in your infrastructure or [adapt your code to avoid using an I/O manager](/guides/build/configure/io-managers#before-you-begin).
29+
To prevent this, you can use [another I/O manager](/guides/operate/io-managers#built-in) that stores data in your infrastructure or [adapt your code to avoid using an I/O manager](/guides/operate/io-managers#before-you-begin).
3030

3131
:::note
3232
You must have [boto3](https://pypi.org/project/boto3/) or `dagster-cloud[serverless]` installed as a project dependency otherwise the Dagster+ managed storage can fail and silently fall back to using the default I/O manager.

docs/docs-beta/docs/dagster-plus/deployment/management/settings/customizing-agent-settings.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -132,4 +132,4 @@ compute_logs:
132132
ServerSideEncryption: "AES256"
133133
show_url_only: true
134134
region: "us-west-1"
135-
```
135+
```

docs/docs-beta/docs/dagster-plus/features/authentication-and-access-control/rbac/user-roles-permissions.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ TODO: add picture previously at "/images/dagster-cloud/user-token-management/cod
115115
| Start and stop [schedules](/guides/automate/schedules) ||||||
116116
| Start and stop [schedules](/guides/automate/sensors) ||||||
117117
| Wipe assets ||||||
118-
| Launch and cancel [schedules](/guides/build/backfill) ||||||
118+
| Launch and cancel [schedules](/guides/automate/schedules) ||||||
119119
| Add dynamic partitions ||||||
120120

121121
### Deployments

docs/docs-beta/docs/dagster-plus/features/authentication-and-access-control/scim/okta-scim.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ In this guide, we'll walk you through configuring [Okta SCIM provisioning](https
1818
With Dagster+'s Okta SCIM provisioning feature, you can:
1919

2020
- **Create users**. Users that are assigned to the Dagster+ application in the IdP will be automatically added to your Dagster+ organization.
21-
- **Update user attributes.** Updating a users name or email address in the IdP will automatically sync the change to your user list in Dagster+.
21+
- **Update user attributes.** Updating a user's name or email address in the IdP will automatically sync the change to your user list in Dagster+.
2222
- **Remove users.** Deactivating or unassigning a user from the Dagster+ application in the IdP will remove them from the Dagster+ organization
2323
{/* - **Push user groups.** Groups and their members in the IdP can be pushed to Dagster+ as [Teams](/dagster-plus/account/managing-users/managing-teams). */}
2424
- **Push user groups.** Groups and their members in the IdP can be pushed to Dagster+ as

docs/docs-beta/docs/dagster-plus/features/catalog-views.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ In this guide, you'll learn how to create, access, and share catalog views with
1717
<summary>Prerequisites</summary>
1818

1919
- **Organization Admin**, **Admin**, or **Editor** permissions on Dagster+
20-
- Familiarity with [Assets](/guides/build/assets-concepts/index.mdx and [Asset metadata](/guides/build/create-a-pipeline/metadata)
20+
- Familiarity with [Assets](/guides/build/create-asset-pipelines/assets-concepts/index.mdx and [Asset metadata](/guides/build/create-asset-pipelines/metadata)
2121

2222
</details>
2323

docs/docs-beta/docs/dagster-plus/features/ci-cd/branch-deployments/change-tracking.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ unlisted: true
88
This guide is applicable to Dagster+.
99
:::
1010

11-
Branch Deployments Change Tracking makes it eaiser for you and your team to identify how changes in a pull request will impact data assets. By the end of this guide, you'll understand how Change Tracking works and what types of asset changes can be detected.
11+
Branch Deployments Change Tracking makes it easier for you and your team to identify how changes in a pull request will impact data assets. By the end of this guide, you'll understand how Change Tracking works and what types of asset changes can be detected.
1212

1313
## How it works
1414

docs/docs-beta/docs/dagster-plus/features/ci-cd/branch-deployments/testing.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,14 @@ unlisted: true
88
This guide is applicable to Dagster+.
99
:::
1010

11-
This guide details a workflow to test Dagster code in your cloud environment without impacting your production data. To highlight this functionality, well leverage Dagster+ branch deployments and a Snowflake database to:
11+
This guide details a workflow to test Dagster code in your cloud environment without impacting your production data. To highlight this functionality, we'll leverage Dagster+ branch deployments and a Snowflake database to:
1212

1313
- Execute code on a feature branch directly on Dagster+
1414
- Read and write to a unique per-branch clone of our Snowflake data
1515

1616
With these tools, we can merge changes with confidence in the impact on our data platform and with the assurance that our code will execute as intended.
1717

18-
Here’s an overview of the main concepts well be using:
18+
Here’s an overview of the main concepts we'll be using:
1919

2020
{/* - [Assets](/concepts/assets/software-defined-assets) - We'll define three assets that each persist a table to Snowflake. */}
2121
- [Assets](/todo) - We'll define three assets that each persist a table to Snowflake.
@@ -35,7 +35,7 @@ Here’s an overview of the main concepts we’ll be using:
3535
## Prerequisites
3636

3737
:::note
38-
This guide is an extension of the <a href="/guides/dagster/transitioning-data-pipelines-from-development-to-production"> Transitioning data pipelines from development to production </a> guide, illustrating a workflow for staging deployments. Well use the examples from this guide to build a workflow atop Dagster+’s branch deployment feature.
38+
This guide is an extension of the <a href="/guides/dagster/transitioning-data-pipelines-from-development-to-production"> Transitioning data pipelines from development to production </a> guide, illustrating a workflow for staging deployments. We'll use the examples from this guide to build a workflow atop Dagster+’s branch deployment feature.
3939
:::
4040

4141
To complete the steps in this guide, you'll need:
@@ -52,7 +52,7 @@ To complete the steps in this guide, you'll need:
5252

5353
## Overview
5454

55-
We have a `PRODUCTION` Snowflake database with a schema named `HACKER_NEWS`. In our production cloud environment, wed like to write tables to Snowflake containing subsets of Hacker News data. These tables will be:
55+
We have a `PRODUCTION` Snowflake database with a schema named `HACKER_NEWS`. In our production cloud environment, we'd like to write tables to Snowflake containing subsets of Hacker News data. These tables will be:
5656

5757
- `ITEMS` - A table containing the entire dataset
5858
- `COMMENTS` - A table containing data about comments
@@ -128,14 +128,14 @@ As you can see, our assets use an [I/O manager](/todo) named `snowflake_io_manag
128128

129129
## Step 2: Configure our assets for each environment
130130

131-
At runtime, wed like to determine which environment our code is running in: branch deployment, or production. This information dictates how our code should execute, specifically with which credentials and with which database.
131+
At runtime, we'd like to determine which environment our code is running in: branch deployment, or production. This information dictates how our code should execute, specifically with which credentials and with which database.
132132

133-
To ensure we can't accidentally write to production from within our branch deployment, well use a different set of credentials from production and write to our database clone.
133+
To ensure we can't accidentally write to production from within our branch deployment, we'll use a different set of credentials from production and write to our database clone.
134134

135135
{/* Dagster automatically sets certain [environment variables](/dagster-plus/managing-deployments/reserved-environment-variables) containing deployment metadata, allowing us to read these environment variables to discern between deployments. We can access the `DAGSTER_CLOUD_IS_BRANCH_DEPLOYMENT` environment variable to determine the currently executing environment. */}
136136
Dagster automatically sets certain [environment variables](/todo) containing deployment metadata, allowing us to read these environment variables to discern between deployments. We can access the `DAGSTER_CLOUD_IS_BRANCH_DEPLOYMENT` environment variable to determine the currently executing environment.
137137

138-
Because we want to configure our assets to write to Snowflake using a different set of credentials and database in each environment, well configure a separate I/O manager for each environment:
138+
Because we want to configure our assets to write to Snowflake using a different set of credentials and database in each environment, we'll configure a separate I/O manager for each environment:
139139

140140
```python file=/guides/dagster/development_to_production/branch_deployments/repository_v1.py startafter=start_repository endbefore=end_repository
141141
# definitions.py
@@ -232,7 +232,7 @@ def drop_prod_clone():
232232
drop_database_clone()
233233
```
234234

235-
Weve defined `drop_database_clone` and `clone_production_database` to utilize the <PyObject object="SnowflakeResource" module="dagster_snowflake" />. The Snowflake resource will use the same configuration as the Snowflake I/O manager to generate a connection to Snowflake. However, while our I/O manager writes outputs to Snowflake, the Snowflake resource executes queries against Snowflake.
235+
We've defined `drop_database_clone` and `clone_production_database` to utilize the <PyObject object="SnowflakeResource" module="dagster_snowflake" />. The Snowflake resource will use the same configuration as the Snowflake I/O manager to generate a connection to Snowflake. However, while our I/O manager writes outputs to Snowflake, the Snowflake resource executes queries against Snowflake.
236236

237237
We now need to define resources that configure our jobs to the current environment. We can modify the resource mapping by environment as follows:
238238

@@ -322,7 +322,7 @@ Opening a pull request for our current branch will automatically kick off a bran
322322

323323
Alternatively, the logs for the branch deployment workflow can be found in the **Actions** tab on the GitHub pull request.
324324

325-
We can also view our database in Snowflake to confirm that a clone exists for each branch deployment. When we materialize our assets within our branch deployment, well now be writing to our clone of `PRODUCTION`. Within Snowflake, we can run queries against this clone to confirm the validity of our data:
325+
We can also view our database in Snowflake to confirm that a clone exists for each branch deployment. When we materialize our assets within our branch deployment, we'll now be writing to our clone of `PRODUCTION`. Within Snowflake, we can run queries against this clone to confirm the validity of our data:
326326

327327
![Instance overview](/images/guides/development_to_production/branch_deployments/snowflake.png)
328328

@@ -383,7 +383,7 @@ Opening a merge request for our current branch will automatically kick off a bra
383383

384384
![Instance overview](/images/guides/development_to_production/branch_deployments/instance_overview.png)
385385

386-
We can also view our database in Snowflake to confirm that a clone exists for each branch deployment. When we materialize our assets within our branch deployment, well now be writing to our clone of `PRODUCTION`. Within Snowflake, we can run queries against this clone to confirm the validity of our data:
386+
We can also view our database in Snowflake to confirm that a clone exists for each branch deployment. When we materialize our assets within our branch deployment, we'll now be writing to our clone of `PRODUCTION`. Within Snowflake, we can run queries against this clone to confirm the validity of our data:
387387

388388
![Instance overview](/images/guides/development_to_production/branch_deployments/snowflake.png)
389389

@@ -489,4 +489,4 @@ close_branch:
489489

490490
After merging our branch, viewing our Snowflake database will confirm that our branch deployment step has successfully deleted our database clone.
491491

492-
Weve now built an elegant workflow that enables future branch deployments to automatically have access to their own clones of our production database that are cleaned up upon merge!
492+
We've now built an elegant workflow that enables future branch deployments to automatically have access to their own clones of our production database that are cleaned up upon merge!

docs/docs-beta/docs/dagster-plus/features/insights/asset-metadata.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ You'll need one or more assets that emit the same metadata key at run time. Insi
2121
are most valuable when you have multiple assets that emit the same kind of metadata, such as
2222
such as the number of rows processed or the size of a file uploaded to object storage.
2323

24-
Follow [the metadata guide](/guides/build/create-a-pipeline/metadata#runtime-metadata) to add numeric metadata
24+
Follow [the metadata guide](/guides/build/create-asset-pipelines/metadata#runtime-metadata) to add numeric metadata
2525
to your asset materializations.
2626

2727
## Step 2: Enable viewing your metadata in Dagster+ Insights

docs/docs-beta/docs/dagster-plus/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ Dagster+ is a managed orchestration platform built on top of Dagster's open sour
77

88
Dagster+ is built to be the most performant, reliable, and cost effective way for data engineering teams to run Dagster in production. Dagster+ is also great for students, researchers, or individuals who want to explore Dagster with minimal overhead.
99

10-
Dagster+ comes in two flavors: a fully [Serverless](/dagster-plus/deployment/deployment-types/serverless) offering and a [Hybrid](/dagster-plus/deployment/deployment-types/hybrid) offering. In both cases, Dagster+ does the hard work of managing your data orchestration control plane. Compared to a [Dagster open source deployment](/guides/), Dagster+ manages:
10+
Dagster+ comes in two flavors: a fully [Serverless](/dagster-plus/deployment/deployment-types/serverless) offering and a [Hybrid](/dagster-plus/deployment/deployment-types/hybrid) offering. In both cases, Dagster+ does the hard work of managing your data orchestration control plane. Compared to a [Dagster open source deployment](guides/deploy/index.md), Dagster+ manages:
1111

1212
- Dagster's web UI at https://dagster.plus
1313
- Metadata stores for data cataloging and cost insights

docs/docs-beta/docs/getting-started/glossary.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
---
22
title: Glossary
33
sidebar_position: 30
4-
sidebar_label: Glossary
54
unlisted: true
65
---
76

0 commit comments

Comments
 (0)