-
Notifications
You must be signed in to change notification settings - Fork 457
chore(byoc): Refactor BYOC private networking setup docs #5780
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
f8042ea
chore(byoc): Refactor BYOC private networking setup docs
duanshiqiang b8cdb7a
Update docs/cloud/guides/infrastructure/01_deployment_options/byoc/03…
duanshiqiang 9eaa6af
Update docs/cloud/guides/infrastructure/01_deployment_options/byoc/03…
duanshiqiang 60f17db
Update docs/cloud/guides/infrastructure/01_deployment_options/byoc/03…
duanshiqiang 2c7be8f
Update regarding PR comments
duanshiqiang 27804cc
Apply suggestions from code review
dhtclk 428595a
Merge branch 'main' into feature-gcp-private-connect
duanshiqiang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
141 changes: 141 additions & 0 deletions
141
...rastructure/01_deployment_options/byoc/03_onboarding/03_network_setup/02_gcp.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,141 @@ | ||
| --- | ||
| title: 'BYOC GCP private networking setup' | ||
| slug: /cloud/reference/byoc/onboarding/network-gcp | ||
| sidebar_label: 'GCP private networking setup' | ||
| keywords: ['BYOC', 'cloud', 'bring your own cloud', 'vpc peering', 'gcp', 'private service connect'] | ||
| description: 'Set up VPC Peering or Private Service Connect for BYOC on GCP' | ||
| doc_type: 'reference' | ||
| --- | ||
|
|
||
| import Image from '@theme/IdealImage'; | ||
| import byoc_vpcpeering from '@site/static/images/cloud/reference/byoc-vpcpeering-gcp-1.png'; | ||
| import byoc_vpcpeering2 from '@site/static/images/cloud/reference/byoc-vpcpeering-gcp-2.png'; | ||
| import byoc_vpcpeering3 from '@site/static/images/cloud/reference/byoc-vpcpeering-gcp-3.png'; | ||
| import byoc_privatelink_1 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-1.png'; | ||
| import byoc_privatelink_2 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-2.png'; | ||
| import byoc_privatelink_3 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-3.png'; | ||
| import byoc_privatelink_4 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-4.png'; | ||
| import byoc_privatelink_5 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-5.png'; | ||
| import byoc_privatelink_6 from '@site/static/images/cloud/reference/byoc-privatelink-gcp-6.png'; | ||
|
|
||
| ClickHouse BYOC on GCP supports two private connection options including VPC Peering and Private Service Connect. Traffic flows entirely within the GCP network, never traversing the public internet. | ||
|
|
||
| ## Prerequisites {#common-prerequisites} | ||
|
|
||
| Common steps required by both VPC peering and Private Service Connect. | ||
|
|
||
| ### Enable private load balancer for ClickHouse BYOC {#step-enable-private-load-balancer-for-clickhouse-byoc} | ||
| Contact ClickHouse Support to enable Private Load Balancer. | ||
|
|
||
| ## Set up VPC peering {#gcp-vpc-peering} | ||
|
|
||
| Please familiarize yourself with [GCP VPC peering feature](https://docs.cloud.google.com/vpc/docs/vpc-peering) and note the limitations of VPC peering (for example subnet IP ranges can't overlap across peered VPC networks). ClickHouse BYOC utilizes a private load balancer to allow network connectivity through the peering to ClickHouse services. | ||
|
|
||
| To create or delete VPC peering for ClickHouse BYOC, follow the steps: | ||
|
|
||
| :::note | ||
| The example steps are for a simple scenario, for advanced scenarios such as peering with on-premises connectivity, some adjustments may be required. | ||
| ::: | ||
|
|
||
| <VerticalStepper headerLevel="h3"> | ||
|
|
||
| ### Create a peering connection {#step-1-create-a-peering-connection} | ||
|
|
||
| In this example, we are setting up peering between the BYOC VPC network and another existing VPC network. | ||
|
|
||
| 1. Navigate to the "VPC Network" in ClickHouse BYOC Google Cloud Project. | ||
| 2. Select "VPC network peering". | ||
| 3. Click "Create connection". | ||
| 4. Input the necessary fields as per your requirements. Below is a screenshot for creating a peering within same GCP project. | ||
|
|
||
| <Image img={byoc_vpcpeering} size="md" alt="BYOC Create Peering Connection" border /> | ||
|
|
||
| GCP VPC peering requires 2 connections between the 2 networks to work (i.e. a connection from BYOC network to the existing VPC network and a connection from the existing VPC network to the BYOC network). So you need to similarly create 1 more connection in reverse direction, below is a screenshot for the second peering connection creation: | ||
|
|
||
| <Image img={byoc_vpcpeering2} size="md" alt="BYOC Accept Peering Connection" border /> | ||
|
|
||
| After both connections are created, the status of the 2 connections should become "Active" after refreshing the Google Cloud Console webpage: | ||
|
|
||
| <Image img={byoc_vpcpeering3} size="lg" alt="BYOC Accept Peering Connection" border /> | ||
|
|
||
| The ClickHouse service should now be accessible from the peered VPC. | ||
|
|
||
| ### Access ClickHouse service via peering connection {#step-2-access-ch-service-via-peering} | ||
|
|
||
| To access ClickHouse privately, a private load balancer and endpoint are provisioned for secure connectivity from the user's peered VPC. The private endpoint follows the public endpoint format with a `-private` suffix. For example: | ||
| - **Public endpoint**: `h5ju65kv87.mhp0y4dmph.us-east1.gcp.byoc.clickhouse.cloud` | ||
| - **Private endpoint**: `h5ju65kv87-private.mhp0y4dmph.us-east1.gcp.byoc.clickhouse.cloud` | ||
|
|
||
| </VerticalStepper> | ||
|
|
||
| ## Set up PSC (Private Service Connect) {#gcp-psc} | ||
|
|
||
| GCP PSC (Private Service Connect) provides secure, private connectivity to your ClickHouse BYOC services without requiring VPC peering or internet gateways. | ||
|
|
||
| <VerticalStepper headerLevel="h3"> | ||
|
|
||
| ### Request PSC service setup {#step-1-request-psc-setup} | ||
|
|
||
| Contact [ClickHouse Support](https://clickhouse.com/cloud/bring-your-own-cloud) to request PSC service setup for your BYOC deployment. No specific information is required at this stage—simply indicate that you want to set up PSC connectivity. | ||
|
|
||
| ClickHouse Support will enable the necessary infrastructure components, including **the private load balancer** and **PSC Service**. | ||
|
|
||
| ### Obtain GCP PSC service name and DNS name {#step-2-obtain-gcp-service-attachment-and-dns-name-for-private-service-connect} | ||
|
|
||
| ClickHouse Support will provide you with the PSC Service name. You can also obtain it in the ClickHouse Cloud console, under "Organization" -> "Infrastructure", click into the infra name to see the details. | ||
|
|
||
| <Image img={byoc_privatelink_1} size="lg" alt="BYOC PSC Endpoint" border /> | ||
| <Image img={byoc_privatelink_2} size="lg" alt="BYOC PSC Endpoint" border /> | ||
|
|
||
| You can also find the PSC service name in the GCP Private Service Connect console under "Published services" (filter by service name or look for ClickHouse services) | ||
|
|
||
| <Image img={byoc_privatelink_3} size="lg" alt="BYOC PSC Endpoint" border /> | ||
| <Image img={byoc_privatelink_4} size="lg" alt="BYOC PSC Endpoint" border /> | ||
|
|
||
| ### Create a PSC endpoint in your network {#step-3-create-endpoint} | ||
|
|
||
| After ClickHouse Support has enabled PSC service on their side, you need to create a PSC endpoint in your client application network to connect to the ClickHouse PSC service. | ||
|
|
||
| 1. **Create the PSC Endpoint**: | ||
| - Navigate to the GCP Console -> Network Services → Private Service Connect → Connect Endpoint | ||
| - Select "Published service" for "Target" and input the PSC service name obtained at last step to "Target details" | ||
| - Input a valid endpoint name | ||
| - Choose your network and select subnets (This is the network where your client application will be connecting from) | ||
| - Choose or create a new IP address for the endpoint, the IP address needs to be used by step [Set private DNS name for endpoint](#step-4-set-private-dns-name-for-endpoint) | ||
| - Click "Add Endpoint", wait a moment for the endpoint to be created. | ||
| - The endpoint status should become "Accepted", contact ClickHouse support if it's not auto-accepted. | ||
|
|
||
| <Image img={byoc_privatelink_5} size="lg" alt="BYOC PSC endpoint creation" border /> | ||
|
|
||
| 2. **Obtain PSC Connection ID**: | ||
| - Click into the endpoint detail and obtain the "PSC Connection ID" to be used by step [Add endpoint's PSC Connection ID to service allowlist](#step-5-add-endpoint-id-allowlist) | ||
|
|
||
| <Image img={byoc_privatelink_6} size="lg" alt="BYOC PSC endpoint detail" border /> | ||
|
|
||
| ### Set private DNS name for endpoint {#step-4-set-private-dns-name-for-endpoint} | ||
|
|
||
| :::note | ||
| There are various ways to configure DNS. Please set up DNS according to your specific use case. | ||
| ::: | ||
|
|
||
| You need to point all subdomains (wildcard) of the "DNS name", taken from [Obtain GCP PSC service name and DNS name](#step-2-obtain-gcp-service-attachment-and-dns-name-for-private-service-connect) step, to GCP PSC endpoint IP address. This ensures that services/components within your VPC/Network can resolve it properly. | ||
|
|
||
| ### Add endpoint's PSC Connection ID to service allowlist {#step-5-add-endpoint-id-allowlist} | ||
|
|
||
| Once your PSC endpoint is created and the status is "Accepted", you need to add the Endpoint's PSC Connection ID to the allowlist for **each ClickHouse service** you want to access via PSC. | ||
|
|
||
| **Contact ClickHouse Support**: | ||
| - Provide the Endpoint's PSC Connection IDs to ClickHouse Support | ||
| - Specify which ClickHouse services should allow access from this endpoint | ||
| - ClickHouse Support will add the Endpoint Connection IDs to the service allowlist | ||
|
|
||
| ### Connect to ClickHouse via PSC {#step-6-connect-via-psc-endpoint} | ||
|
|
||
| After the Endpoint Connection IDs is added to the allowlist, you can connect to your ClickHouse service using the PSC endpoint. | ||
|
|
||
| The PSC endpoint format is similar to the public endpoint, but includes a `p` subdomain. For example: | ||
|
|
||
| - **Public endpoint**: `h5ju65kv87.mhp0y4dmph.us-east1.gcp.clickhouse-byoc.com` | ||
| - **PSC endpoint**: `h5ju65kv87.p.mhp0y4dmph.us-east1.gcp.clickhouse-byoc.com` | ||
|
|
||
| </VerticalStepper> |
15 changes: 15 additions & 0 deletions
15
...frastructure/01_deployment_options/byoc/03_onboarding/03_network_setup/index.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| --- | ||
| title: 'Private networking setup' | ||
| slug: /cloud/reference/byoc/onboarding/network | ||
| sidebar_label: 'Private networking setup' | ||
| hide_title: true | ||
| description: 'Table of contents page for the ClickHouse Cloud BYOC Private Networking Setup section' | ||
| doc_type: 'landing-page' | ||
| keywords: ['BYOC', 'cloud', 'bring your own cloud', 'vpc peering', 'privatelink', 'private service connect'] | ||
| --- | ||
|
|
||
| # Private networking setup | ||
|
|
||
| ClickHouse BYOC supports various private networking options to enhance security and enable direct connectivity for your services. This guide walks you through the recommended approaches for securely connecting ClickHouse Cloud deployments in your own AWS or GCP account to other networks or services, such as your internal applications or analytics tools. We cover options such as VPC Peering, AWS PrivateLink, and GCP Private Service Connect, and outline the main steps and considerations for each. | ||
|
|
||
| If you require a private network connection to your ClickHouse BYOC deployment, follow the steps in the guides or consult ClickHouse Support for assistance with more advanced scenarios. | ||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.