Skip to main content
Skip to main content
Edit this page

AWS PrivateLink

Scale plan feature

AWS PrivateLink is available in the Scale and Enterprise plans. To upgrade, visit the Plans page in the cloud console.

You can use AWS PrivateLink to establish secure connectivity between VPCs, AWS services, your on-premises systems, and ClickHouse Cloud without exposing traffic to the public Internet. This document outlines the steps to connect to ClickHouse Cloud using AWS PrivateLink.

To restrict access to your ClickHouse Cloud services exclusively through AWS PrivateLink addresses, follow the instructions provided by ClickHouse Cloud IP Access Lists.

note

ClickHouse Cloud currently supports cross-region PrivateLink in beta.

Please complete the following to enable AWS PrivateLink:

  1. Obtain Endpoint "Service name".
  2. Create AWS Endpoint.
  3. Add "Endpoint ID" to ClickHouse Cloud organization.
  4. Add "Endpoint ID" to ClickHouse service allow list.

Find Terraform examples here.

Attention​

ClickHouse attempts to group your services to reuse the same published service endpoint within the AWS region. However, this grouping is not guaranteed, especially if you spread your services across multiple ClickHouse organizations. If you already have PrivateLink configured for other services in your ClickHouse organization, you can often skip most of the steps because of that grouping and proceed directly to the final step: Add ClickHouse "Endpoint ID" to ClickHouse service allow list.

Prerequisites​

Before you get started you will need:

  1. Your AWS account.
  2. ClickHouse API key with the necessary permissions to create and manage private endpoints on ClickHouse side.

Steps​

Follow these steps to connect your ClickHouse Cloud services via AWS PrivateLink.

Obtain Endpoint "Service name"​

Option 1: ClickHouse Cloud console​

In the ClickHouse Cloud console, open the service you want to connect via PrivateLink, then navigate to the Settings menu.

Make a note of the Service name and DNS name, then move onto next step.

Option 2: API​

First, set the following environment variables before running any commands:

Get your ClickHouse INSTANCE_ID by filtering by region, provider and service name:

Obtain endpointServiceId and privateDnsHostname for your PrivateLink configuration:

This command should return something like:

Make a note of the endpointServiceId and privateDnsHostname move onto next step.

Create AWS Endpoint​

info

This section covers ClickHouse-specific details for configuring ClickHouse via AWS PrivateLink. AWS-specific steps are provided as a reference to guide you on where to look, but they may change over time without notice from the AWS cloud provider. Please consider AWS configuration based on your specific use case.

Please note that ClickHouse is not responsible for configuring the required AWS VPC endpoints, security group rules or DNS records.

If you previously enabled "private DNS names" while setting up PrivateLink and are experiencing difficulties configuring new services via PrivateLink, please contact ClickHouse support. For any other issues related to AWS configuration tasks, contact AWS Support directly.

Option 1: AWS console​

Open the AWS console and Go to VPC → Endpoints → Create endpoints.

Select Endpoint services that use NLBs and GWLBs and use Service nameconsole or endpointServiceIdAPI you got from Obtain Endpoint "Service name" step in Service Name field. Click Verify service:

If you want to establish a cross-regional connection via PrivateLink, enable the "Cross region endpoint" checkbox and specify the service region. The service region is where the ClickHouse instance is running.

If you get a "Service name could not be verified." error, please contact Customer Support to request adding new regions to the supported regions list.

Next, select your VPC and subnets:

As an optional step, assign Security groups/Tags:

note

Make sure that ports 443, 8443, 9440, 3306 are allowed in the security group.

After creating the VPC Endpoint, make a note of the Endpoint ID value; you'll need it for an upcoming step.

Option 2: AWS CloudFormation​

Next, you need to create a VPC Endpoint using Service nameconsole or endpointServiceIdAPI you got from Obtain Endpoint "Service name" step. Make sure to use correct subnet IDs, security groups, and VPC ID.

After creating the VPC Endpoint, make a note of the Endpoint ID value; you'll need it for an upcoming step.

Option 3: Terraform​

service_name below is Service nameconsole or endpointServiceIdAPI you got from Obtain Endpoint "Service name" step

After creating the VPC Endpoint, make a note of the Endpoint ID value; you'll need it for an upcoming step.

Set Private DNS Name for Endpoint​

note

There are various ways to configure DNS. Please set up DNS according to your specific use case.

You need to point "DNS name", taken from Obtain Endpoint "Service name" step, to AWS Endpoint network interfaces. This ensures that services/components within your VPC/Network can resolve it properly.

Add Endpoint ID to ClickHouse Cloud organization​

Option 1: ClickHouse Cloud console​

To add an endpoint to organization, proceed to the Add Endpoint ID to service(s) allow list step. Adding the Endpoint ID using the ClickHouse Cloud console to the services allow list automatically adds it to organization.

To remove an endpoint, open Organization details -> Private Endpoints and click the delete button to remove the endpoint.

Option 2: API​

Set the following environment variables before running any commands:

Set the ENDPOINT_ID environment variable using data from Create AWS Endpoint step.

To add an endpoint, run:

To remove an endpoint, run:

Add "Endpoint ID" to ClickHouse service allow list​

Option 1: ClickHouse Cloud console​

To add please navigate to the ClickHouse Cloud console, open the service that you would like to connect via PrivateLink then navigate to Settings. Enter the Endpoint ID obtained from the Create AWS Endpoint step. Click "Create endpoint".

note

If you want to allow access from an existing PrivateLink connection, use the existing endpoint drop-down menu.

To remove please navigate to the ClickHouse Cloud console, find the service, then navigate to Settings of the service, find endpoint you would like to remove.Remove it from the list of endpoints.

Option 2: API​

You need to add an Endpoint ID to the allow-list for each instance that should be available using PrivateLink.

Set the ENDPOINT_ID environment variable using data from Create AWS Endpoint step.

Set the following environment variables before running any commands:

To add an endpoint ID to an allow-list:

To remove an endpoint ID from an allow-list:

Each service with Private Link enabled has a public and private endpoint. In order to connect using Private Link, you need to use a private endpoint which will be privateDnsHostnameAPI or DNS Nameconsole taken from Obtain Endpoint "Service name".

Getting Private DNS Hostname​

Option 1: ClickHouse Cloud console​

In the ClickHouse Cloud console, navigate to Settings. Click on the Set up private endpoint button. In the opened flyout, copy the DNS Name.

Option 2: API​

Set the following environment variables before running any commands:

You can retrieve INSTANCE_ID from step.

This should output something like:

In this example connection via value of privateDnsHostname host name will be routed to PrivateLink, but connection via endpointServiceId hostname will be routed over the Internet.

Troubleshooting​

In most cases, you only need to create a single endpoint service for each VPC. This endpoint can route requests from the VPC to multiple ClickHouse Cloud services. Please refer here

Connection to private endpoint timed out​

  • Please attach security group to VPC Endpoint.
  • Please verify inbound rules on security group attached to Endpoint and allow ClickHouse ports.
  • Please verify outbound rules on security group attached to VM which is used to connectivity test and allow connections to ClickHouse ports.

Private Hostname: Not found address of host​

  • Please check your DNS configuration

Connection reset by peer​

  • Most likely Endpoint ID was not added to service allow list, please visit step

Checking Endpoint filters​

Set the following environment variables before running any commands:

You can retrieve INSTANCE_ID from step.

Connecting to a remote database​

Let's say you are trying to use MySQL or PostgreSQL table functions in ClickHouse Cloud and connect to your database hosted in an Amazon Web Services (AWS) VPC. AWS PrivateLink cannot be used to enable this connection securely. PrivateLink is a one-way, unidirectional connection. It allows your internal network or Amazon VPC to connect securely to ClickHouse Cloud, but it does not allow ClickHouse Cloud to connect to your internal network.

According to the AWS PrivateLink documentation:

Use AWS PrivateLink when you have a client/server set up where you want to allow one or more consumer VPCs unidirectional access to a specific service or set of instances in the service provider VPC. Only the clients in the consumer VPC can initiate a connection to the service in the service provider VPC.

To do this, configure your AWS Security Groups to allow connections from ClickHouse Cloud to your internal/private database service. Check the default egress IP addresses for ClickHouse Cloud regions, along with the available static IP addresses.