diff --git a/docs/connector-development/writing-connector-docs.md b/docs/connector-development/writing-connector-docs.md
index 348ca5e9db79..dd5bb4af8003 100644
--- a/docs/connector-development/writing-connector-docs.md
+++ b/docs/connector-development/writing-connector-docs.md
@@ -1,29 +1,105 @@
# Writing Connector Documentation
-This document provides guidance on tools and techniques specifically for writing documentation for Airbyte connectors.
+This topic guides you through writing documentation for Airbyte connectors. The systems and practices described in [Updating Documentation](../contributing-to-airbyte/writing-docs.md) apply here as well. However, there are several features and restrictions that only apply to connectors.
-For more information about writing documentation at Airbyte generally, please also see the [Contributing Guide for Airbyte Documentation](../contributing-to-airbyte/writing-docs.md).
+## QA checks
-## Custom markdown extensions for connector docs
+If you're writing docs for a new connector, your docs must pass our [QA checks](../contributing-to-airbyte/resources/qa-checks).
-Airbyte's markdown documentation—particularly connector-specific documentation—needs to gracefully support multiple different contexts: key details may differ between open-source builds and Airbyte Cloud, and the more exhaustive explanations appropriate for https://docs.airbyte.com may bury key details when rendered as inline documentation within the Airbyte application. In order to support all these different contexts without resorting to multiple overlapping files that must be maintained in parallel, Airbyte's documentation tooling supports multiple nonstandard features.
+## Custom Markdown extensions for connector docs
-Please familiarize yourself with all the tools available to you when writing documentation for a connector, so that you can provide appropriately tailored information to your readers in whichever context they see it.
+Airbyte's connector documentation must gracefully support different contexts in a way platform documentation doesn't.
-:::note
-As a general rule, features that introduce new behavior or prevent certain content from rendering will affect how the Airbyte UI displays markdown content, but have no impact on https://docs.airbyte.com. If you want to test out these in-app features in [a local Airbyte build](https://docs.airbyte.com/contributing-to-airbyte/developing-locally/#develop-on-airbyte-webapp), ensure that you have the `airbyte` git repository checked out to the same parent directory as the airbyte platform repository: if so, development builds will by default fetch connector documentation from your local filesystem, allowing you to freely edit their content and view the rendered output.
-:::
+- https://docs.airbyte.com
+- In-app documentation in self-managed versions of Airbyte
+- In-app documentation in the Cloud version of Airbyte
+
+Key details about setting up a connection may differ between Cloud and Self-Managed. We created custom Markdown extensions that can show or hide different pieces of content based on the reader's environment. This is a rudimentary form of a concept called single-sourcing: writing content once, but using it in multiple contexts. This prevents us from having to maintain multiple highly similar pools of content.
+
+The following features for single-sourcing are available. You can combine them to produce a more meaningful result.
+
+### Hide content from the Airbyte UI
+
+Some content is important to document, but unhelpful in Airbyte's UI. This could be:
+
+- Background information that helps people understand a connector but doesn't affect the configuration process
+- Edge cases with complex solutions
+- Context about each environment, which doesn't need to be seen if you're in that environment
-## Mapping UI to Associated Docs
+Wrapping content in `...` tags prevents Airbyte from rendering that content in-app, but https://docs.airbyte.com renders it normally.
-Sometimes a connector's configuration UI may have a field that requires more explanation than can be provided in the UI itself. In these cases, the connector developer can use the `FieldAnchor` syntax below to link documentation to a specific UI component.
+### Hide content from Cloud
-When a user selects the associated field in the UI, the documentation will automatically scroll to the related documentation section the right-hand panel.
+You can hide content from Airbyte Cloud, while still rendering it in Self-Managed and https://docs.airbyte.com.
-The `FieldAnchor` syntax accepts a modified `jsonpath` expression, as follows:
+```md
+
+
+Only Self-Managed builds of the Airbyte UI will render this content.
+
+
+```
+
+### Hide content from Self-Managed
+
+You can hide content from Airbyte Self-Managed, while still rendering it in Cloud and https://docs.airbyte.com.
-`example-a.md`:
```md
+
+
+Only Cloud builds of the Airbyte UI will render this content.
+
+
+```
+
+### Example
+
+Here's an example where the configuration steps are different in Cloud and Self-Managed. In this case, you want to render everything on https://docs.airbyte.com, but you want in-app content reflect only the environment the user is running.
+
+```markdown title="connector-doc.md"
+# My connector
+
+This content is rendered everywhere.
+
+
+
+
+
+## For open source:
+
+
+
+Only self-managed builds of the Airbyte UI will render this content.
+
+
+
+
+
+
+## For Airbyte Cloud:
+
+
+
+Only Cloud builds of the Airbyte UI will render this content.
+
+
+```
+
+### Testing your content
+
+To test in-app content in [a local Airbyte build](https://docs.airbyte.com/contributing-to-airbyte/developing-locally/#develop-on-airbyte-webapp), check out the `airbyte` git repository to the same branch and directory as the airbyte platform repository. Development builds fetch connector documentation from your local filesystem, so you can edit their content and view the rendered output in Airbyte.
+
+To test https://docs.airbyte.com content, [build Docusaurus locally](../contributing-to-airbyte/writing-docs.md#set-up-your-environment).
+
+## Map the UI to associated content
+
+Sometimes a field requires more explanation than can be provided in a tooltip. In these cases, use the `` tag to link documentation to a specific UI component.
+
+When a user selects that field in the UI, the in-app documentation panel automatically scrolls to the related documentation, highlighting all content contained inside the `` tag.
+
+The `FieldAnchor` syntax accepts a modified version of `jsonpath`, without the conventional `$.` prefix. It looks like this:
+
+```md title="example-a.md"
## Configuring Widgets
@@ -35,9 +111,8 @@ The `FieldAnchor` syntax accepts a modified `jsonpath` expression, as follows:
Taking a more complex example, you can access deeper-nested fields using `jsonpath` expressions syntax:
-`example-b.md`:
-```md
-## Configuring Unstructures Streams
+```md title="example-b.md"
+## Configuring Unstructured Streams
@@ -46,66 +121,81 @@ Taking a more complex example, you can access deeper-nested fields using `jsonpa
```
-Note:
-- The syntax expected is a modified version of jsonpath, without the conventional `$.` prefix.
-- The `FieldAnchor` syntax is only supported in the context of connector documentation, and will not render on the documentation site.
+:::note
+The `FieldAnchor` tag only affects in-app content for sources and destinations. It has no effect on https://docs.airbyte.com or any platform content.
+:::
How it works:
-- When a user focuses the field identified by the `field` attribute in the connector setup UI, the documentation pane will automatically scroll to the associated section of the documentation, highlighting all content contained inside the `` tag.
-- These are rendered as regular divs in the documentation site, so they have no effect in places other than the in-app documentation panel.
-- There must be blank lines between a custom tag like `FieldAnchor` the content it wraps for the documentation site to render markdown syntax inside the custom tag to html.
+- There must be blank lines between a custom tag like `FieldAnchor` the content it wraps.
- The `field` attribute must be a valid `jsonpath` expression to one of the properties nested under `connectionSpecification.properties` in that connector's `spec.json` or `spec.yaml` file. For example, if the connector spec contains a `connectionSpecification.properties.replication_method.replication_slot`, you would mark the start of the related documentation section with `` and its end with ``.
-- It's also possible to highlight the same section for multiple fields by separating them with commas, like ``.
-- To mark a section as highlighted after the user picks an option from a `oneOf`: use a `field` prop like `path.to.field[value-of-selection-key]`, where the `value-of-selection-key` is the value of a `const` field nested inside that `oneOf`. For example, if the specification of the `oneOf` field is:
-
-```json
-"replication_method": {
- "type": "object",
- "title": "Update Method",
- "oneOf": [
- {
- "title": "Read Changes using Binary Log (CDC)",
- "required": ["method"],
- "properties": {
- "method": {
- "type": "string",
- "const": "CDC",
- "order": 0
- },
- "initial_waiting_seconds": {
- "type": "integer",
- "title": "Initial Waiting Time in Seconds (Advanced)",
+- Highlight the same section for multiple fields by separating them with commas, like this: ``.
+- To highlight a section after the user picks an option from a `oneOf`: use a `field` prop like `path.to.field[value-of-selection-key]`, where the `value-of-selection-key` is the value of a `const` field nested inside that `oneOf`.
+
+ For example, if the specification of the `oneOf` field is:
+
+ ```json
+ "replication_method": {
+ "type": "object",
+ "title": "Update Method",
+ "oneOf": [
+ {
+ "title": "Read Changes using Binary Log (CDC)",
+ "required": ["method"],
+ "properties": {
+ "method": {
+ "type": "string",
+
+ "const": "CDC",
+ "order": 0
+ },
+ "initial_waiting_seconds": {
+ "type": "integer",
+ "title": "Initial Waiting Time in Seconds (Advanced)",
+ },
+ }
},
- }
- },
- {
- "title": "Scan Changes with User Defined Cursor",
- "required": ["method"],
- "properties": {
- "method": {
- "type": "string",
- "const": "STANDARD",
- "order": 0
+ {
+ "title": "Scan Changes with User Defined Cursor",
+ "required": ["method"],
+ "properties": {
+ "method": {
+ "type": "string",
+
+ "const": "STANDARD",
+ "order": 0
+ }
+ }
}
- }
+ ]
}
- ]
-}
-```
+ ```
-The selection keys are `CDC` and `STANDARD`, so you can wrap a specific replication method's documentation section with a `...` tag, and it will be highlighted if the user selects CDC replication in the UI.
+ The selection keys are `CDC` and `STANDARD`. Wrap a specific replication method's documentation section with a `...` tag to highlight it if the user selects CDC replication in the UI.
-:::tip
-Because of their close connection with the connector setup form fields, `` tags are only enabled for the source and destination setup pages.
-:::
+### Documenting PyAirbyte usage
+
+PyAirbyte is a Python library that allows you to run syncs within a Python script for a subset of Airbyte's connectors. Documentation around PyAirbyte connectors is automatically generated from the connector's JSON schema spec. There are a few approaches to combine full control over the documentation with automatic generation for common cases:
-## Prevent specific docs from rendering in the Config UI with ``
+- If a connector:
+
+ 1. Is PyAirbyte-enabled (`remoteRegistries.pypi.enabled` is set in the `metadata.yaml` file of the connector), and
+ 2. Has no second-level heading `Usage with PyAirbyte` in the documentation
+
+ The documentation will be automatically generated and placed above the `Changelog` section.
-Certain content is important to document, but unhelpful in the context of the Airbyte UI's inline documentation views:
+- By manually specifying a `Usage with PyAirbyte` section, this is disabled. The following is a good starting point for this section:
-- background information that helps users understand a connector but doesn't affect configuration
-- edge cases that are unusual but time-consuming to solve
-- context for readers on the documentation site about environment-specific content (see [below](#environment-specific-in-app-content-with-magic-html-comments))
+```md
+
+
+## Usage with PyAirbyte
+
+
+
+
+
+
+```
-Wrapping such content in a pair of `...` tags will prevent it from being rendered within the Airbyte UI without affecting its presentation on https://docs.airbyte.com. This allows a single markdown file to be the source of truth for both a streamlined in-app reference and a more thorough treatment on the documentation website.
+The `PyAirbyteExample` component will generate a code example that can be run with PyAirbyte, excluding an auto-generated sample configuration based on the configuration schema. The `SpecSchema` component will generate a reference table with the connector's JSON schema spec, like a non-interactive version of the connector form in the UI. It can be used on any docs page.
diff --git a/docs/contributing-to-airbyte/resources/qa-checks.md b/docs/contributing-to-airbyte/resources/qa-checks.md
index 6d0ceaeb3a82..f52e7524a411 100644
--- a/docs/contributing-to-airbyte/resources/qa-checks.md
+++ b/docs/contributing-to-airbyte/resources/qa-checks.md
@@ -4,27 +4,30 @@ This document is listing all the static-analysis checks that are performed on th
These checks are running in our CI/CD pipeline and are used to ensure a connector is following the best practices and is respecting the Airbyte standards.
Meeting these standards means that the connector will be able to be safely integrated into the Airbyte platform and released to registries (DockerHub, Pypi etc.).
You can consider these checks as a set of guidelines to follow when developing a connector.
-They are by no mean replacing the need for a manual review of the connector codebase and the implementation of good test suites.
+They do not replace the need for a manual review of the connector codebase and the implementation of good test suites.
## đź“„ Documentation
-### Breaking changes must be accompanied by a migration guide
+### Major version upgrades must be accompanied by a migration guide
_Applies to the following connector types: source, destination_
+
_Applies to the following connector languages: java, low-code, python, manifest-only_
+
_Applies to connector with any support level_
-_Applies to connector with 100 internal support level_
+
_Applies to connector with any Airbyte usage level_
-When a breaking change is introduced, we check that a migration guide is available. It should be stored under `./docs/integrations/s/-migrations.md`.
-This document should contain a section for each breaking change, in order of the version descending. It must explain users which action to take to migrate to the new version.
+When a connector experiences a major version upgrade, we check that a migration guide is available. It should be stored under `./docs/integrations/s/-migrations.md`. This document should contain a section for each major change, ordered by descending version. It must explain which action to take to migrate to the new version.
### Connectors must have user facing documentation
_Applies to the following connector types: source, destination_
+
_Applies to the following connector languages: java, low-code, python, manifest-only_
+
_Applies to connector with any support level_
-_Applies to connector with 100 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should be stored under `./docs/integrations/s/.md`.
@@ -32,9 +35,11 @@ The user facing connector documentation should be stored under `./docs/integrati
### Links used in connector documentation are valid
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should update invalid links in connector documentation. For links that are used as example and return 404 status code, use `example: ` before link to skip it.
@@ -42,61 +47,58 @@ The user facing connector documentation should update invalid links in connector
### Connectors documentation headers structure, naming and order follow our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
This check expects the following order of headers in the documentation:
-````
-
-
- # CONNECTOR_NAME_FROM_METADATA
+```md
+# CONNECTOR_NAME_FROM_METADATA
- ## Prerequisites
+## Prerequisites
- ## Setup guide
+## Setup guide
- ## Set up CONNECTOR_NAME_FROM_METADATA
+## Set up CONNECTOR_NAME_FROM_METADATA
- ### For Airbyte Cloud:
+### For Airbyte Cloud:
- ### For Airbyte Open Source:
+### For Airbyte Open Source:
- ### CONNECTOR_SPECIFIC_FEATURES
+### CONNECTOR_SPECIFIC_FEATURES
- ## Set up the CONNECTOR_NAME_FROM_METADATA connector in Airbyte
+## Set up the CONNECTOR_NAME_FROM_METADATA connector in Airbyte
- ### For Airbyte Cloud:
+### For Airbyte Cloud:
- ### For Airbyte Open Source:
+### For Airbyte Open Source:
- ## CONNECTOR_SPECIFIC_FEATURES
+## CONNECTOR_SPECIFIC_FEATURES
- ## Supported sync modes
+## Supported sync modes
- ## Supported Streams
+## Supported Streams
- ## CONNECTOR_SPECIFIC_FEATURES
+## CONNECTOR_SPECIFIC_FEATURES
- ### Performance considerations
+### Performance considerations
- ## Data type map
+## Data type map
- ## Limitations & Troubleshooting
+## Limitations & Troubleshooting
- ### CONNECTOR_SPECIFIC_FEATURES
+### CONNECTOR_SPECIFIC_FEATURES
- ### Tutorials
-
- ## Changelog
-
-
-````
+### Tutorials
+## Changelog
+```
List of not required headers, which can be not exist in the documentation and their strict check will be skipped:
@@ -120,138 +122,141 @@ List of not required headers, which can be not exist in the documentation and th
### Prerequisites section of the documentation describes all required fields from specification
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should update `Prerequisites` section with description for all required fields from source specification. Having described all required fields in a one place helps Airbyte users easily set up the source connector.
-If spec has required credentials/access_token/refresh_token etc, check searches for one of ["account", "auth", "credentials", "access", "client"] words. No need to add credentials/access_token/refresh_token etc to the section
+If spec has required credentials/access_token/refresh_token etc, check searches for one of `["account", "auth", "credentials", "access", "client"]` words. No need to add credentials/access_token/refresh_token etc to the section
### Main Source Section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that CONNECTOR_NAME_FROM_METADATA header section content follows standard template:
-````
-
+```md
This page contains the setup guide and reference information for the [CONNECTOR_NAME_FROM_METADATA]({docs_link}) source connector.
-
-
-````
+```
### 'For Airbyte Cloud:' section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that For Airbyte Cloud: header section content follows standard template:
-````
-
+```md
1. [Log into your Airbyte Cloud](https://cloud.airbyte.com/workspaces) account.
2. Click Sources and then click + New source.
3. On the Set up the source page, select CONNECTOR_NAME_FROM_METADATA from the Source type dropdown.
4. Enter a name for the CONNECTOR_NAME_FROM_METADATA connector.
-
````
### 'For Airbyte Open Source:' section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that For Airbyte Open Source: header section content follows standard template:
-````
-
+```md
1. Navigate to the Airbyte Open Source dashboard.
-
````
### 'Supported sync modes' section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that Supported sync modes header section content follows standard template:
-````
-
+```md
The CONNECTOR_NAME_FROM_METADATA source connector supports the following [sync modes](https://docs.airbyte.com/cloud/core-concepts/#connection-sync-modes):
-
-
-````
+```
### 'Tutorials' section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that Tutorials header section content follows standard template:
-````
-
+```md
Now that you have set up the CONNECTOR_NAME_FROM_METADATA source connector, check out the following CONNECTOR_NAME_FROM_METADATA tutorials:
-
-
-````
+```
### 'Changelog' section of the documentation follows our guidelines
_Applies to the following connector types: source_
+
_Applies to the following connector languages: python, low-code_
+
_Applies to connector with any support level_
-_Applies to connector with 300 internal support level_
+
_Applies to connector with any Airbyte usage level_
The user facing connector documentation should follow the guidelines defined in the [standard template](../../../airbyte-ci/connectors/connectors_qa/src/connectors_qa/checks/documentation/templates/template.md.j2).
Check verifies that Changelog header section content follows standard template:
-````
+```md
Expand to review
-````
+```
### Connectors must have a changelog entry for each version
_Applies to the following connector types: source, destination_
+
_Applies to the following connector languages: java, low-code, python, manifest-only_
+
_Applies to connector with any support level_
-_Applies to connector with 100 internal support level_
+
_Applies to connector with any Airbyte usage level_
Each new version of a connector must have a changelog entry defined in the user facing documentation in `./docs/integrations/s/.md`.
@@ -288,7 +293,7 @@ _Applies to connector with any Airbyte usage level_
Python connectors must have a CDK tag in their metadata. It must be set in the `tags` field in metadata.yaml. The values can be `cdk:low-code`, `cdk:python`, or `cdk:file`.
-### Breaking change deadline should be a week in the future
+### Major version upgrade deadline should be a week in the future
_Applies to the following connector types: source, destination_
_Applies to the following connector languages: java, low-code, python, manifest-only_
@@ -296,7 +301,7 @@ _Applies to connector with any support level_
_Applies to connector with any internal support level_
_Applies to connector with any Airbyte usage level_
-If the connector version has a breaking change, the deadline field must be set to at least a week in the future.
+If this is a major version upgrade, the deadline field must be set to at least a week in the future.
### Certified source connector must have a value filled out for maxSecondsBetweenMessages in metadata
diff --git a/docs/contributing-to-airbyte/writing-docs.md b/docs/contributing-to-airbyte/writing-docs.md
index 55455456446c..5ebb35c4a3c5 100644
--- a/docs/contributing-to-airbyte/writing-docs.md
+++ b/docs/contributing-to-airbyte/writing-docs.md
@@ -1,48 +1,66 @@
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
-# Updating Documentation
+# Updating documentation
-We welcome contributions to the Airbyte documentation!
+Everyone is welcome to contribute to Airbyte's documentation!
-Our docs are written in [Markdown](https://guides.github.com/features/mastering-markdown/) following the [Google developer documentation style guide](https://developers.google.com/style/highlights) and the files are stored in our [Github repository](https://github.com/airbytehq/airbyte/tree/master/docs). The docs are published at [docs.airbyte.com](https://docs.airbyte.com/) using [Docusaurus](https://docusaurus.io/) and [GitHub Pages](https://pages.github.com/).
+Our documentation is stored in the [Airbyte repository](https://github.com/airbytehq/airbyte/tree/master/docs) on GitHub. It's published at [docs.airbyte.com](https://docs.airbyte.com/) using [GitHub Pages](https://pages.github.com/). Connector docs are also rendered within Airbyte itself when setting up new connectors. The docs are built on [Docusaurus](https://docusaurus.io/). Content is written in [Markdown](https://guides.github.com/features/mastering-markdown/) and all topics are in the `/docs` folder. Configuration files are in the `/docusaurus` folder.
-## Finding good first issues
+## Open source contributions welcome
-Before contributing to Airbyte docs, read the Airbyte Community [Code of Conduct](../community/code-of-conduct.md).
+Open source contributors are a vital part of Airbyte. Technical writers are welcome to use any content you author in your portfolio.
-The Docs team maintains a list of [good first issues](https://github.com/airbytehq/airbyte/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%2Fdocumentation+label%3A%22good+first+issue%22) for new contributors.
+If you're interested in becoming a regular contributor, join Airbyte's [Contributor program](https://airbyte.com/community/contributor-program). We pay for high-quality work on select issues. AI-generated content isn't eligible for the contributor program. You're welcome to use AI as a research tool and editing partner, but paid submissions must be principally human-generated.
-- If you're new to technical writing, start with the smaller issues (fixing typos, broken links, spelling and grammar, and so on). You can [edit the files directly on GitHub](#editing-directly-on-github).
-- If you're an experienced technical writer or a developer interested in technical writing, comment on an issue that interests you to discuss it with the Docs team. Once we decide on the approach and the tasks involved, [edit the files and open a Pull Request](#editing-on-your-local-machine) for the Docs team to review.
+## Before you start
-:::tip
-If you're new to GitHub and Markdown, complete [the First Contributions tutorial](https://github.com/firstcontributions/first-contributions) and learn [Markdown basics](https://guides.github.com/features/mastering-markdown/) before contributing to Airbyte documentation. Even if you're familiar with the basics, you may be interested in Airbyte's [custom markdown extensions for connector docs](#custom-markdown-extensions-for-connector-docs).
-:::
+Before you contribute, familiarize yourself with these concepts.
+
+### Read our code of conduct
+
+Read the Airbyte Community [code of conduct](../community/code-of-conduct.md).
+
+### Learn Docusaurus basics
+
+Even if you've never used it before, most developers and technical writers find Docusaurus intuitive.
+
+- [Markdown basics](https://guides.github.com/features/mastering-markdown/)
+- [Docusaurus 5-minute tutorial](https://tutorial.docusaurus.io/)
+
+### Style guide
+
+Follow the [Google developer documentation style guide](https://developers.google.com/style/highlights). It's a widely adopted style guide with good examples and easy-to-follow rules. We don't enforce these rules with automation, but might in the future. Fow now, just try to follow it to the best of your ability.
-## Editing directly on GitHub
+### Find a good first issue
-To make minor changes (example: fixing typos) or edit a single file, you can edit the file directly on GitHub:
+The Docs team maintains a list of [good first issues](https://github.com/airbytehq/airbyte/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%2Fdocumentation+label%3A%22good+first+issue%22) for new contributors. If you'd like to start a complex documentation project, create or comment on a [GitHub issue](https://github.com/airbytehq/airbyte/issues) and tag [@ian-at-airbyte](https://github.com/ian-at-airbyte) so we can decide on an approach together.
-1. Click **Edit this page** at the bottom of any published document on [docs.airbyte.com](https://docs.airbyte.com/). You'll be taken to the GitHub editor.
-2. [Edit the file directly on GitHub and open a Pull Request](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files).
+## Edit files directly on GitHub
-## Editing on your local machine
+To make minor changes like fixing typos or editing a single file, you can edit the file directly in your browser.
+
+1. Click **Edit this page** at the bottom of any page on [docs.airbyte.com](https://docs.airbyte.com/). You'll be taken to the GitHub editor.
+2. Edit the file directly on GitHub and open a Pull Request ([help](https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files)).
+
+## Edit files on your local machine
+
+If you're making substantial documentation changes, it's best to clone the repository and work locally so you can test as you work.
### Prerequisites
-To contribute to our documentation, please ensure following required technologies are installed on your local machine:
+Install these tools on your local machine, first.
-1. [`Node.js`](https://nodejs.org/en/learn/getting-started/how-to-install-nodejs)
+1. [Node.js](https://nodejs.org/en/learn/getting-started/how-to-install-nodejs)
2. [`pnpm`](https://pnpm.io/installation)
+3. A tool to work with GitHub, like [Git](https://git-scm.com/) or [GitHub Desktop](https://github.com/apps/desktop)
+4. A code editor, like [Visual Studio Code](https://code.visualstudio.com/).
-### Setup and Making Changes
-
-To make complex changes or edit multiple files, edit the files on your local machine:
+### Fork and clone the repo
-1. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the Airbyte [repository](https://github.com/airbytehq/airbyte).
+1. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the [Airbyte repo](https://github.com/airbytehq/airbyte).
-2. Clone the fork on your local machine:
+2. Clone the fork on your local machine.
```bash
git clone git@github.com:{YOUR_USERNAME}/airbyte.git
@@ -56,346 +74,230 @@ To make complex changes or edit multiple files, edit the files on your local mac
cd airbyte
```
-3. Create a feature branch from which to make changes:
+3. Create a feature branch.
```bash
git checkout -b {YOUR_USERNAME}/{FEATURE/BUG}
```
- (e.g. `jdoe/source-stock-api-stream-fix`)
-
-4. Test changes locally:
-
- To install the docs locally, run the following commands in your terminal:
+ For example:
```bash
- cd docusaurus
- pnpm install
+ git checkout -b jdoe/source-stock-api-stream-fix
```
- To see changes as you make them, run:
+### Set up your environment
+
+Open a terminal and install the docs locally.
+
+```bash
+cd docusaurus
+pnpm install
+```
+
+To see changes as you make them in a dev build:
+
+1. Run:
```bash
pnpm start
```
- Then navigate to [http://localhost:3005/](http://localhost:3005/). Whenever you make and save changes, you will see them reflected in the server. You can stop the running server in OSX/Linux by pressing `Ctrl-C` in the terminal.
+2. Navigate to [http://localhost:3005/](http://localhost:3005/). Whenever you make and save changes, you will see them reflected in the server. To stop the running server, press Ctrl+C in the terminal.
- You can also build the docs locally and see the resulting changes. This is useful if you introduce changes that need to be run at build-time (e.g. adding a docs plug-in). To do so, run:
+To create an optimized production build that does not update automatically:
+
+1. Run:
```bash
pnpm build
pnpm serve
```
- Then navigate to [http://localhost:3000/](http://localhost:3000/) to see your changes. You can stop the running server in OSX/Linux by pressing `Ctrl-C` in the terminal.
-
-5. [Follow the GitHub workflow](https://docs.github.com/en/get-started/quickstart/contributing-to-projects/) to edit the files and create a pull request.
+2. Navigate to [http://localhost:3000/](http://localhost:3000/) to see your changes. To stop the running server, press Ctrl+C in the terminal.
- :::note
- Before we accept any contributions, you'll need to sign the Contributor License Agreement (CLA). By signing a CLA, we can ensure that the community is free and confident in its ability to use your contributions. You will be prompted to sign the CLA while opening a pull request.
- :::
+### Author content
-6. Assign `airbytehq/docs` as a Reviewer for your pull request.
+[Follow the basic GitHub workflow](https://docs.github.com/en/get-started/quickstart/contributing-to-projects/) to update content: branch, write, commit, pull request, merge.
-## Guide To Writing Connector Docs
+## Content templates
-For instructions specific to connector configuration docs, please see the [Connector Documentation Guide](../connector-development/writing-connector-docs.md).
+Every page must have a purpose. Bad documentation often has origins in:
-## Common Tools and Patterns
+- Poorly-defined goals, or no goal
+- Failed execution of otherwise good goals
+- The intimidating effect of a blank page
-### Select between mutually-exclusive content options with ``
+The [Good Docs Project](https://www.thegooddocsproject.dev/) maintains a collection of open-source docs templates you can use to help you write and update articles. Here are common patterns we see at Airbyte:
-Tabs are a built-in feature of Docusaurus, the tool we use to build `https://docs.airbyte.com`; please refer to [their documentation](https://docusaurus.io/docs/markdown-features/tabs) for their options and behavior in this context. For better site-agnostic documentation, and because we like the feature, we maintain a separate `Tabs` implementation with limited, one-way API compatibility: all usage options we document should behave the same in-app and on `https://docs.airbyte.com`. If you find a discrepancy or breakage, we would appreciate if you [report it as a bug](https://github.com/airbytehq/airbyte/issues/new?assignees=&labels=type%2Fenhancement%2Carea%2Fdocumentation+needs-triage&projects=&template=8-documentation.yaml)! The reverse is not necessarily true, however: Docusaurus supports many use cases besides ours, so supporting its every usage pattern is a deliberate non-goal.
+| Purpose | Overview | Template |
+| --------------- | --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
+| Concept | Explain a concept, context, or background information about a product or its features. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/concept) |
+| How-to | A concise set of numbered steps to do one task with the product. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/how-to) |
+| Tutorial | Instructions to set up an example project intended for hands-on learning. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/tutorial) |
+| Troubleshooting | Common problems experienced by users, an explanation of the causes, and steps to resolve the issue. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/troubleshooting) |
+| Reference | Specific, in-depth details about a particular topic. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/reference) |
+| Release note | Communicate new features, improvements, bug fixes, and known issues about a product. | [Template](https://gitlab.com/tgdp/templates/-/tree/main/release-notes) |
-:::info
-Because Docusaurus uses an mdx component, you must include the following import lines in any markdown file which uses tabs:
-
-```js
-import Tabs from "@theme/Tabs";
-import TabItem from "@theme/TabItem";
-```
+[View all templates](https://www.thegooddocsproject.dev/template).
-This is not optional: if these lines are missing, the documentation site will have errors. They won't show up in the rendered document, however.
+:::important
+These templates can only be used for platform docs. Docs for connectors have their own template. See [Write connector docs](#connector-docs) for more details.
:::
-Here's an example set of tabs; note that you can put any number of `...` tags inside a ``.
-
-```md
-
-
-
-When configuring this hypothetical connector using basic HTTP auth, you should mind some tricky security considerations! This just a hypothetical, though, so I never bothered to come up with any.
-
-As the first tab, this would be shown by default if no `TabItem` were marked as `default`.
-
-
-
-
-When configuring this hypothetical connector using OAuth authentication, you should do a dance. Good for you! Since it's not the first `TabItem` in its set, we had to explicitly mark this tab as `default` for it to get top billing.
-
-
-
-```
-
-That renders as the following:
-
-
-
-
-When configuring this hypothetical connector using basic HTTP auth, you should mind some tricky security considerations! This just a hypothetical, though, so I never bothered to come up with any.
-
-As the first tab, this would be shown by default if no `TabItem` were marked as `default`.
-
-
-
-
-When configuring this hypothetical connector using OAuth authentication, you should do a dance. Good for you! Since it's not the first `TabItem` in its set, we had to explicitly mark this tab as `default` for it to get top billing.
-
-
-
-
-- You don't need to mark any tab as `default`
-- If you don't, the first tab (here, Basic HTTP) will be the initial selection instead
-- You can use ordinary markdown syntax inside a `TabItem`
-- **however**, due to bugs in our in-app markdown rendering library, you should be dilligent about using empty lines to separate different formatting-related things (surrounding tags and their contents, paragraphs vs lists, etc)
-- You should also avoid indenting `TabItem` tags and their content according to html conventions, since text indented by four spaces (common for html nested inside two levels of tags) can be interpreted as a code block; different markdown rendering tools can handle this inconsistently.
+## Write connector docs {#connector-docs}
-### Environment-specific in-app content with magic html comments
+If you're writing docs for a data source or destination, there are special rules you must follow. See the [Connector Documentation Guide](../connector-development/writing-connector-docs.md). Platform documentation is less formulaic.
-Sometimes, there are connector setup instructions which differ between open-source Airbyte builds and Airbyte Cloud. Document both cases, but wrap each in a pair of special HTML comments:
+## Common patterns and components
-```md
-
-
+Since the docs site is based on Docusaurus, it inherits all of Docusaurus' capabilities. There are also some Airbyte-specific elements to be aware of. Most of these customizations can be combined together.
-## For open source:
+### Tabs
-
-
-Only open-source builds of the Airbyte UI will render this content.
-
-
-
-
-
-
-## For Airbyte Cloud:
-
-
-
-Only cloud builds of the Airbyte UI will render this content.
-
-
-
-Content outside of the magic-comment-delimited blocks will be rendered everywhere.
-```
-
-Note that the documentation site will render _all_ environment-specific content, so please introduce environment-specific variants with some documentation-site-only context (like the hidden subheadings in the example above) to disambiguate.
-
-### Contextually-styled callouts with admonition blocks
-
-We have added support for [Docusaurus' admonition syntax](https://docusaurus.io/docs/markdown-features/admonitions) to Airbyte's in-app markdown renderer.
-
-To make an admonition, wrap text with lines of three colons, with the first colons immediately followed (no space) by a tag specifying the callout's semantic styling, which will be one of `tip`, `warning`, `caution`, `danger`, `note`, or `info`. The syntax parallells a code block's, but with colons instead of backticks.
-
-Examples of the different admonition types:
-
-```md
-:::note
-
-A **note** with _Markdown_ `syntax`.
-
-:::
-```
+Use tabs to display mutually-exclusive concepts in a concise way. See [Tabs](https://docusaurus.io/docs/markdown-features/tabs).
:::note
+We maintain a separate `Tabs` implementation to support rendering tabs in Airbyte's in-app documentation. Our in-app renderer creates some additional rules that aren't necessarily true in other Docusaurus implementations:
-A **note** with _Markdown_ `syntax`.
-
+- Always use empty lines to separate different Markup elements (tags, paragraphs, lists, etc.)
+- Do not indent `TabItem` tags and their content according to normal HTML conventions. Different Markdown rendering tools handle indented tags inconsistently.
:::
-```md
-:::tip
+### Code blocks
-A **tip** with _Markdown_ `syntax`.
+Code blocks are used to represent sample code and command line input and output with easy-to-read syntax highlighting. See [Code blocks](https://docusaurus.io/docs/markdown-features/code-blocks).
-:::
-```
+Not all languages are turned on. If syntax highlighting doesn't work for your language, add it to `docusaurus/docusaurus.config.js`'s `module.exports.themeConfig.prism.additionalLanguages` property. Then, restart your local server to test the new language.
-:::tip
+### Admonitions (notes, warnings, tips, etc.)
-A **tip** with _Markdown_ `syntax`.
+Docusaurus has custom markup to create a note, warning, tip, danger, or info block. See [Admonitions](https://docusaurus.io/docs/markdown-features/admonitions).
-:::
+### Expandable details panels
-```md
-:::info
+Details panels are a great way to render content that's not relevant to everyone, or to condense long pages. See [Details](https://docusaurus.io/docs/markdown-features#details).
-Some **info** with _Markdown_ `syntax`.
+### Product badges
-:::
-```
+Some platform features are reserved for certain Airbyte products. To avoid confusion and indicate which products a topic applies to, pages can display badges in a prominent location under the title. [This page is an example](../using-airbyte/getting-started/oss-quickstart).
-:::info
+To enable badges, include `products` in the Markdown metadata. The following values are possible, and you can combine them as needed.
-Some **info** with _Markdown_ `syntax`.
+- `all`: Self-Managed Community, Self-Managed Enterprise, and Cloud
+- `oss-community`: Self-Managed Community only
+- `oss-enterprise`: Self-Managed Enterprise only
+- `cloud`: Cloud only
+- `cloud-teams`: Cloud Teams only
-:::
+In this example, the Self-Managed Community badge is highlighted, and Cloud and Self-Managed Enterprise badges are grayed out.
-```md
-:::caution
+```markdown
+---
+products: oss-community
+---
-A **caution** with _Markdown_ `syntax`.
+# This topic is only for Self-Managed Community
-:::
+Some text.
```
-:::caution
-
-A **caution** with _Markdown_ `syntax`.
+In this example, the Self-Managed Community badge is grayed out, but the Cloud Teams and Self-Managed Enterprise badges are highlighted.
-:::
-
-```md
-:::danger
+```markdown
+---
+products: cloud-teams, oss-enterprise
+---
-Some **dangerous** content with _Markdown_ `syntax`.
+# This topic is only for Cloud Teams and Self-Managed Enterprise
-:::
+Some text.
```
-:::danger
+### Diagrams
-Some **dangerous** content with _Markdown_ `syntax`.
+The [Mermaid](https://mermaid.js.org/) plugin runs on our Docusaurus instance. If possible, you should create diagrams with Mermaid, because they're easier to build, maintain, and translate. They're also more suitable for assistive technologies like screen readers.
+:::warning
+Connector docs do **not** support Mermaid. Mermaid can also be unsuitable for complex diagrams, where you need more visual control. If Mermaid can't meet your needs, create an SVG diagram or use Whimsical. Airbyte employees have access to a paid Whimsical account with Airbyte's color palette built in.
:::
-### Collapsible content with `` and ``
-
-```md
-## Ordinary markdown content
+Here is an example of how to create a Mermaid diagram:
-
- Here is an expandible section! Everything but this title is hidden by default.
- Here is the dropdown content; if users expand this section, they will be able to read your valuable but perhaps nonessential content.
-
+Add a code block to your Markdown and specify `mermaid` as the language.
-Back to ordinary markdown content.
+````md
+```mermaid
+---
+title: Order example
+---
+erDiagram
+ CUSTOMER ||--o{ ORDER : places
+ ORDER ||--|{ LINE-ITEM : contains
+ CUSTOMER }|..|{ DELIVERY-ADDRESS : uses
```
+````
-Eagle-eyed readers may note that _all_ markdown should support this feature since it's part of the html spec. However, it's worth special mention since these dropdowns have been styled to be a graceful visual fit within our rendered documentation in all environments.
-
-### Documenting PyAirbyte usage
-
-PyAirbyte is a Python library that allows to run syncs within a Python script for a subset of connectors. Documentation around PyAirbyte connectors is automatically generated from the connector's JSON schema spec. There are a few approaches to combine full control over the documentation with automatic generation for common cases:
-
-- If a connector is PyAirbyte enabled (`remoteRegistries.pypi.enabled` set in the `metadata.yaml` file of the connector) and there is no second-level heading `Usage with PyAirbyte` in the documentation, the documentation will be automatically generated and placed above the `Changelog` section.
-- By manually specifying a `Usage with PyAirbyte` section, this automatism is disabled. The following is a good starting point for this section:
-
-```md
-
-
-## Usage with PyAirbyte
+This produces the following diagram.
-
-
-
-
-
+```mermaid
+---
+title: Order example
+---
+erDiagram
+ CUSTOMER ||--o{ ORDER : places
+ ORDER ||--|{ LINE-ITEM : contains
+ CUSTOMER }|..|{ DELIVERY-ADDRESS : uses
```
-The `PyAirbyteExample` component will generate a code example that can be run with PyAirbyte, excluding an auto-generated sample configuration based on the configuration schema. The `SpecSchema` component will generate a reference table with the connector's JSON schema spec, like a non-interactive version of the connector form in the UI. It can be used on any docs page.
+The [Mermaid documentation](https://mermaid.js.org/intro) goes into more depth.
-## Additional guidelines
+### Update the sidebar
-- If you're adding a new file, update the [sidebars.js file](https://github.com/airbytehq/airbyte/blob/master/docusaurus/sidebars.js)
-- If you're adding a README to a code module, make sure the README has the following components:
- - A brief description of the module
- - Development pre-requisites (like which language or binaries are required for development)
- - How to install dependencies
- - How to build and run the code locally & via Docker
- - Any other information needed for local iteration
+If you're adding a new file, removing a file, or moving things around, update [`docusaurus/sidebars.js`](https://github.com/airbytehq/airbyte/blob/master/docusaurus/sidebars.js) to reflect the new structure.
-## Advanced tasks
+### Add a redirect
-### Adding a redirect
+If you're moving or renaming a page, you should add a redirect to its new location. If you're deleting a page, you should add a redirect to the most relevant new file, like a replacement topic or a parent page.
To add a redirect, open the [`docusaurus/redirects.yml`](https://github.com/airbytehq/airbyte/blob/master/docusaurus/redirects.yml) file and add an entry from which old path to which new path a redirect should happen.
:::note
-Your path **needs** a leading slash `/` to work
-:::
-
-### Deploying and reverting the documentation site
-
-:::note
-Only the Airbyte team and maintainers have permissions to deploy the documentation site.
+Your path needs a leading slash `/` to work
:::
-#### Automated documentation site deployment
+### Document a code module
-When `docs/` folder gets changed in `master` branch of the repository, [`Deploy docs.airbyte.com` Github workflow](https://github.com/airbytehq/airbyte/actions/workflows/deploy-docs-site.yml) steps in, builds and deploys the documentation site. This process is automatic, takes five to ten minutes, and needs no human intervention.
+If you're adding a README to a code module, make sure the README has the following components:
-#### Manual documentation site deployment
+ - A brief description of the module
+ - Development prerequisites (like which language or binaries are required for development)
+ - How to install dependencies
+ - How to build and run the code locally and via Docker
+ - Any other information needed for local iteration
-:::note
-Manual deployment is reserved for emergency cases. Please, bear in mind that automatic deployment is triggered by changes to `docs/` folder, so it needs to be disabled to avoid interference with manual deployment.
+:::tip
+AI tools like ChatGPT and GitHub Copilot are good at describing code. For open-source repositories, you might find it helpful to point one to your code module and ask it to generate a first draft for you. Airbyte employees working on a proprietary repository should follow Airbyte's AI usage policies.
:::
-You'll need a GitHub SSH key to deploy the documentation site using the [deployment tool](https://github.com/airbytehq/airbyte/blob/master/tools/bin/deploy_docusaurus).
+## Create a pull request
-To deploy the documentation site, run:
+When you're ready to submit your work, create a pull request into `master`.
-```bash
-cd airbyte
-# or cd airbyte-cloud
-git checkout master
-git pull
-./tools/bin/deploy_docusaurus
-```
+### Review and approval
-To revert/rollback doc changes, run:
+New pull requests with docs changes are automatically submitted to [#docs-prs](https://airbytehq-team.slack.com/archives/C075JMUK2FJ) in Airbyte's internal Slack. [@ian-at-airbyte](https://github.com/ian-at-airbyte) monitors this feed. If you want a specific person to review your work, add them as a reviewer. If not, you don't need to tag anyone, but you must wait for at least one Airbyte employee to explicitly approve your pull request before anyone can merge it.
-```
-cd airbyte
-git checkout
-./tools/bin/deploy_docusaurus
-```
-
-### Adding a diagram
+Vercel builds a preview of the updated docs site automatically. If you're a community contributor, someone from Airbyte must authorize that build before it proceeds.
-We have the docusaurus [Mermaid](https://mermaid.js.org/) plugin which has a variety of diagram
-types and syntaxes available.
-
-:::danger
-The connector specific docs do **not** currently support this, only use this for general docs.
+:::tip Collaboration tip
+If you're not finished but want to collaborate with others, create a draft pull request. Vercel will still build your docs site, creating a preview others can see, and you can continue to work on your branch to incorporate feedback.
:::
-Here is an example from the [Mermaid docs](https://mermaid.js.org/syntax/entityRelationshipDiagram.html)
-you would add the following to your markdown wrapped in a code block.
-
-```md
- ---
- title: Order example
- ---
- erDiagram
- CUSTOMER ||--o{ ORDER : places
- ORDER ||--|{ LINE-ITEM : contains
- CUSTOMER }|..|{ DELIVERY-ADDRESS : uses
-```
-
-which produces the following diagram
+:::note
+Before we accept your contribution, you need to sign the Contributor License Agreement (CLA). By signing a CLA, we can ensure that the community is free and confident in its ability to use your contributions. You will be prompted to sign the CLA while opening a pull request.
+:::
-```mermaid
----
-title: Order example
----
-erDiagram
- CUSTOMER ||--o{ ORDER : places
- ORDER ||--|{ LINE-ITEM : contains
- CUSTOMER }|..|{ DELIVERY-ADDRESS : uses
-```
+## Deploy the documentation site
-check out the rest of the Mermaid documentation for its capabilities just be aware that not all
-the features are available to the docusaurus plugin.
+When someone merges documentation changes into the `master` branch, updated docs are deployed automatically. This takes 5-10 minutes and needs no human intervention.
\ No newline at end of file