Configurable Backups in ClickHouse Cloud

We’re excited to announce configurable backup controls are now available in ClickHouse Cloud in private preview.

The ability to take backups of your data is table stakes for any database offering. Backups provide a safety net by ensuring that if data is lost for any unforeseen reason - whether it be accidental deletion, corruption, and so on - the service can be restored to the previous state of the last successful backup. This minimizes downtime and prevents business-critical data from being permanently lost. Backups are critical to any disaster recovery (DR) plan by ensuring that the business can quickly recover from disruptive events, maintain continuity, and resume normal operations, thus minimizing financial and reputational impact. Additionally, backups also help enterprises satisfy their compliance and legal requirements around data retention and are evidence that they have policies in place to prevent loss of data.

All ClickHouse Cloud services come with default backup policies. For Production services, backups are taken daily and retained for two days. This ensures that if data is lost for any unforeseen reason, the service can be restored to the last successful backup. Backups are a combination of full and incremental backups that constitute a sequence of backups utilized together to restore data to a new service if needed.

ClickHouse Cloud default backups (with a 24-hour backup frequency, and 48-hour retention) satisfy the business needs for some customers. However, certain customers need the additional ability to take backups more frequently and retain them for a longer duration to meet their Business Continuity, or in some cases, compliance requirements.

Additionally, some customers prefer that backups happen at a certain time during the day. This prevents the backup process from competing with compute resources allocated to the service during critical hours, and importantly, it gives customers control over the backups being taken at a time when most of their daily data changes have been committed.

To address these needs, we are making our backup process more flexible by giving customers the ability to configure the start time, retention, and frequency for backups of their Production and Dedicated tier services. We will continue to provide the two default backups at no cost. However, changes to the backups schedule that require retention for a longer duration, or more frequent backups that require additional copies of the data to be retained, may incur additional charges.

To configure backups for your service and set the schedule to be different from the default, go to the service settings page and navigate to the Backups section. Click the “Change backup configuration.”

The Backup configuration page lets you modify the retention, frequency, or start time of the backups for your service. You can choose any start time over a 24-hour window, and backups will start within an hour of the scheduled time. Backups can be set up to happen as frequently as every 6 hours, and as infrequently as every 24 hours, with several intermediate values supported within that range. Retention ranges from 1 day and goes up to 30 days, which refers to the ability to roll back to a certain point in time.

NOTE: Frequency and start time (Scheduled) are mutually exclusive settings for backups. For example, if you select 2 AM UTC as the start time for your backups, you won’t be able to simultaneously set a frequency of backing up the data every 6 hours.

All available backups for your service are displayed on the backups page in the Cloud Console. In the example below, backups have been configured to happen every 6 hours, with a 5-day retention. From here (under Actions) you can also select a particular backup and choose to restore it to a new service. Details of restoring a backup to a service are covered in our public docs.

To understand the cost impact of your backup configuration, you can look at the usage breakdown for your service under the “Organization” section on the Cloud Console. If there are costs associated with the backup configuration you’ve selected, they will be displayed on this page under the column “Backups.”

We plan to make ClickHouse Cloud backups even more seamless and flexible. We will soon enable on-demand backups, so you can kick off a backup at any point in time from the UI, or programmatically via APIs.

We will also soon support the ability to export backups cross-region. This gives you the ability to fulfill your DR (disaster recovery) requirements in situations where the primary region has an interruption in service.

Additionally, we are looking to enable the capability to export backups to your own cloud service account. This will allow for more control over the backup lifecycle, as well as data retention for data residency or other compliance purposes.

Finally, in the coming months, we also plan to improve our backup capability to support continuous backups and point-in-time restores (PITR). This will make it possible to have even more granular RPO for data stored in ClickHouse Cloud.