Skip to main content
Skip to main content
Edit this page

Azure Private Link

Scale plan feature

Azure Private Link is available in the Scale and Enterprise plans. To upgrade, visit the Plans page in the cloud console.

This guide shows how to use Azure Private Link to provide private connectivity via a virtual network between Azure (including customer-owned and Microsoft Partner services) and ClickHouse Cloud. Azure Private Link simplifies the network architecture and secures the connection between endpoints in Azure by eliminating data exposure to the public internet.

Unlike AWS and GCP, Azure supports cross-region connectivity via Private Link. This enables you to establish connections between VNets located in different regions where you have ClickHouse services deployed.

note

Additional charges may be applied to inter-region traffic. Please check latest Azure documentation.

Please complete the following steps to enable Azure Private Link:

  1. Obtain Azure connection alias for Private Link
  2. Create a Private Endpoint in Azure
  3. Add the Private Endpoint GUID to your ClickHouse Cloud organization
  4. Add the Private Endpoint GUID to your service(s) allow list
  5. Access your ClickHouse Cloud service using Private Link

Attention​

ClickHouse attempts to group your services to reuse the same published Private Link service within the Azure region. However, this grouping is not guaranteed, especially if you spread your services across multiple ClickHouse organizations. If you already have Private Link configured for other services in your ClickHouse organization, you can often skip most of the steps because of that grouping and proceed directly to the final step: Add the Private Endpoint GUID to your service(s) allow list.

Find Terraform examples at the ClickHouse Terraform Provider repository.

Option 1: ClickHouse Cloud console​

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink, then open the Settings menu. Click on the Set up private endpoint button. Make a note of the Service name and DNS name which will be used for setting up Private Link.

Make a note of the Service name and DNS name, it will be needed in the next steps.

Option 2: API​

Before you get started, you'll need a ClickHouse Cloud API key. You can create a new key or use an existing one.

Once you have your API key, set the following environment variables before running any commands:

Get your ClickHouse INSTANCE_ID by filtering by region, provider and service name:

Obtain your Azure connection alias and Private DNS hostname for Private Link:

Make a note of the endpointServiceId. You'll use it in the next step.

Create a Private Endpoint in Azure​

info

This section covers ClickHouse-specific details for configuring ClickHouse via Azure Private Link. Azure-specific steps are provided as a reference to guide you on where to look, but they may change over time without notice from the Azure cloud provider. Please consider Azure configuration based on your specific use case.

Please note that ClickHouse is not responsible for configuring the required Azure private endpoints, DNS records.

For any issues related to Azure configuration tasks, contact Azure Support directly.

In this section, we're going to create a Private Endpoint in Azure. You can use either the Azure Portal or Terraform.

Option 1: Using Azure Portal to create a Private Endpoint in Azure​

In the Azure Portal, open Private Link Center → Private Endpoints.

Open the Private Endpoint creation dialog by clicking on the Create button.

In the following screen, specify the following options:

  • Subscription / Resource Group: Please choose the Azure subscription and resource group for the Private Endpoint.
  • Name: Set a name for the Private Endpoint.
  • Region: Choose region where the deployed VNet that will be connected to ClickHouse Cloud via Private Link.

After you have completed the above steps, click the Next: Resource button.

Select the option Connect to an Azure resource by resource ID or alias.

For the Resource ID or alias, use the endpointServiceId you have obtained from the Obtain Azure connection alias for Private Link step.

Click Next: Virtual Network button.

  • Virtual network: Choose the VNet you want to connect to ClickHouse Cloud using Private Link
  • Subnet: Choose the subnet where Private Endpoint will be created

Optional:

  • Application security group: You can attach ASG to Private Endpoint and use it in Network Security Groups to filter network traffic to/from Private Endpoint.

Click Next: DNS button.

Click the Next: Tags button.

Optionally, you can attach tags to your Private Endpoint.

Click the Next: Review + create button.

Finally, click the Create button.

The Connection status of the created Private Endpoint will be in Pending state. It will change to Approved state once you add this Private Endpoint to the service allow list.

Open the network interface associated with Private Endpoint and copy the Private IPv4 address(10.0.0.4 in this example), you will need this information in the next steps.

Option 2: Using Terraform to create a Private Endpoint in Azure​

Use the template below to use Terraform to create a Private Endpoint:

Obtaining the Private Endpoint resourceGuid​

In order to use Private Link, you need to add the Private Endpoint connection GUID to your service allow list.

The Private Endpoint resource GUID is only exposed in the Azure Portal. Open the Private Endpoint created in previous step and click JSON View:

Under properties, find resourceGuid field and copy this value:

You need will need to create a Private DNS zone (${location_code}.privatelink.azure.clickhouse.cloud) and attach it to your VNet to access resources via Private Link.

Create Private DNS zone​

Option 1: Using Azure portal

Please follow the following guide to create an Azure private DNS zone using the Azure Portal.

Option 2: Using Terraform

Use the following Terraform template to create a Private DNS zone:

Create a wildcard DNS record​

Create a wildcard record and point to your Private Endpoint:

Option 1: Using Azure Portal

  1. Open the MyAzureResourceGroup resource group and select the ${region_code}.privatelink.azure.clickhouse.cloud private zone.
  2. Select + Record set.
  3. For Name, type *.
  4. For IP Address, type the IP address you see for Private Endpoint.
  5. Select OK.

Option 2: Using Terraform

Use the following Terraform template to create a wildcard DNS record:

To link the private DNS zone to a virtual network, you'll need create a virtual network link.

Option 1: Using Azure Portal

Please follow the following guide to link the virtual network to your private DNS zone.

Option 2: Using Terraform

note

There are various ways to configure DNS. Please set up DNS according to your specific use case.

You need to point "DNS name", taken from Obtain Azure connection alias for Private Link step, to Private Endpoint IP address. This ensures that services/components within your VPC/Network can resolve it properly.

Verify DNS setup​

xxxxxxxxxx.westus3.privatelink.azure.clickhouse.cloud domain should be pointed to the Private Endpoint IP. (10.0.0.4 in this example).

Add the Private Endpoint GUID to your ClickHouse Cloud organization​

Option 1: ClickHouse Cloud console​

To add an endpoint to organization, proceed to the Add the Private Endpoint GUID to your service(s) allow list step. Adding the Private Endpoint GUID using the ClickHouse Cloud console to the services allow list automatically adds it to organization.

To remove an endpoint, open Organization details -> Private Endpoints and click the delete button to remove the endpoint.

Option 2: API​

Set the following environment variables before running any commands:

Set the ENDPOINT_ID environment variable using data from the Obtaining the Private Endpoint resourceGuid step.

Run the following command to add the Private Endpoint:

You can also run the following command to remove a Private Endpoint:

After adding or removing a Private Endpoint, run the following command to apply it to your organization:

Add the Private Endpoint GUID to your service(s) allow list​

By default, a ClickHouse Cloud service is not available over a Private Link connection even if the Private Link connection is approved and established. You need to explicitly add the Private Endpoint GUID for each service that should be available using Private Link.

Option 1: ClickHouse Cloud console​

In the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink then navigate to Settings. Enter the Endpoint ID obtained from the previous step.

note

If you want to allow access from an existing PrivateLink connection, use the existing endpoint drop-down menu.

Option 2: API​

Set these environment variables before running any commands:

Execute it for each service that should be available using Private Link.

Run the following command to add the Private Endpoint to the services allow list:

You can also run the following command to remove a Private Endpoint from the services allow list:

After adding or removing a Private Endpoint to the services allow list, run the following command to apply it to your organization:

Each service with Private Link enabled has a public and private endpoint. In order to connect using Private Link, you need to use a private endpoint which will be privateDnsHostnameAPI or DNS nameconsole taken from Obtain Azure connection alias for Private Link.

Obtaining the Private DNS Hostname​

Option 1: ClickHouse Cloud console​

In the ClickHouse Cloud console, navigate to Settings. Click on the Set up private endpoint button. In the opened flyout, copy the DNS Name.

Option 2: API​

Set the following environment variables before running any commands:

Run the following command:

You should receive a response similar to the following:

In this example, connection to the xxxxxxx.region_code.privatelink.azure.clickhouse.cloud hostname will be routed to Private Link. Meanwhile, xxxxxxx.region_code.azure.clickhouse.cloud will be routed over the internet.

Use the privateDnsHostname to connect to your ClickHouse Cloud service using Private Link.

Troubleshooting​

Test DNS setup​

Run the following command:

where "dns name" privateDnsHostnameAPI or DNS nameconsole from Obtain Azure connection alias for Private Link

You should receive the following response:

Connection reset by peer​

Most likely, the Private Endpoint GUID was not added to the service allow-list. Revisit the Add Private Endpoint GUID to your services allow-list step.

Private Endpoint is in Pending state​

Most likely, the Private Endpoint GUID was not added to the service allow-list. Revisit the Add Private Endpoint GUID to your services allow-list step.

Test connectivity​

If you have problems with connecting using Private Link, check your connectivity using openssl. Make sure the Private Link endpoint status is Accepted.

OpenSSL should be able to connect (see CONNECTED in the output). errno=104 is expected.

Checking Private Endpoint filters​

Set the following environment variables before running any commands:

Run the following command to check Private Endpoint filters:

More information​

For more information about Azure Private Link, please visit azure.microsoft.com/en-us/products/private-link.