> ## Documentation Index
> Fetch the complete documentation index at: https://clickhouse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Separation of storage and compute

> This guide explores how you can use ClickHouse and S3 to implement an architecture with separated storage and compute.

export const Image = ({img, alt, size = "lg"}) => {
  const normalizedSize = ["sm", "md", "lg"].includes(size) ? size : "lg";
  return <div className={`ch-image-${normalizedSize}`}>
      <Frame>
        <img src={img} alt={alt} />
      </Frame>
    </div>;
};

<h2 id="overview">
  Overview
</h2>

This guide explores how you can use ClickHouse and S3 to implement an architecture with separated storage and compute.

Separation of storage and compute means that computing resources and storage resources are managed independently. In ClickHouse, this allows for better scalability, cost-efficiency, and flexibility. You can scale storage and compute resources separately as needed, optimizing performance and costs.

Using ClickHouse backed by S3 is especially useful for use cases where query performance on "cold" data is less critical. ClickHouse provides support for using S3 as the storage for the `MergeTree` engine using `S3BackedMergeTree`. This table engine enables you to exploit the scalability and cost benefits of S3 while maintaining the insert and query performance of the `MergeTree` engine.

Please note that implementing and managing a separation of storage and compute architecture is more complicated compared to standard ClickHouse deployments. While self-managed ClickHouse allows for separation of storage and compute as discussed in this guide, we recommend using [ClickHouse Cloud](https://clickhouse.com/cloud), which allows you to use ClickHouse in this architecture without configuration using the [`SharedMergeTree` table engine](/docs/products/cloud/features/infrastructure/shared-merge-tree).

*This guide assumes you're using  ClickHouse version 22.8 or higher.*

<Warning>
  Don't configure any AWS/GCS life cycle policy. This isn't supported and could lead to broken tables.
</Warning>

<Steps>
  <Step title="Use S3 as a ClickHouse disk" id="1-use-s3-as-a-clickhouse-disk">
    <h3 id="creating-a-disk">
      Creating a disk
    </h3>

    Create a new file in the ClickHouse `config.d` directory to store the storage configuration:

    ```bash theme={null}
    vim /etc/clickhouse-server/config.d/storage_config.xml
    ```

    Copy the following XML in to the newly created file, replacing `BUCKET`, `ACCESS_KEY_ID`, `SECRET_ACCESS_KEY` with the AWS bucket details where you'd like to store your data:

    ```xml theme={null}
    <clickhouse>
      <storage_configuration>
        <disks>
          <s3_disk>
            <type>s3</type>
            <endpoint>$BUCKET</endpoint>
            <access_key_id>$ACCESS_KEY_ID</access_key_id>
            <secret_access_key>$SECRET_ACCESS_KEY</secret_access_key>
            <metadata_path>/var/lib/clickhouse/disks/s3_disk/</metadata_path>
          </s3_disk>
          <s3_cache>
            <type>cache</type>
            <disk>s3_disk</disk>
            <path>/var/lib/clickhouse/disks/s3_cache/</path>
            <max_size>10Gi</max_size>
          </s3_cache>
        </disks>
        <policies>
          <s3_main>
            <volumes>
              <main>
                <disk>s3_disk</disk>
              </main>
            </volumes>
          </s3_main>
        </policies>
      </storage_configuration>
    </clickhouse>
    ```

    If you need to further specify settings for the S3 disk, for example to specify a `region` or send a custom HTTP `header`, you can find the list of relevant settings [here](/docs/reference/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-s3).

    You can also replace `access_key_id` and `secret_access_key` with the following, which will attempt to obtain credentials from environment variables and Amazon EC2 metadata:

    ```bash theme={null}
    <use_environment_credentials>true</use_environment_credentials>
    ```

    After you've created your configuration file, you need to update the owner of the file to the clickhouse user and group:

    ```bash theme={null}
    chown clickhouse:clickhouse /etc/clickhouse-server/config.d/storage_config.xml
    ```

    You can now restart the ClickHouse server to have the changes take effect:

    ```bash theme={null}
    service clickhouse-server restart
    ```
  </Step>

  <Step title="Create a table backed by S3" id="2-create-a-table-backed-by-s3">
    To test that we've configured the S3 disk properly, we can attempt to create and query a table.

    Create a table specifying the new S3 storage policy:

    ```sql theme={null}
    CREATE TABLE my_s3_table
      (
        `id` UInt64,
        `column1` String
      )
    ENGINE = MergeTree
    ORDER BY id
    SETTINGS storage_policy = 's3_main';
    ```

    Note that we didn't have to specify the engine as `S3BackedMergeTree`. ClickHouse automatically converts the engine type internally if it detects the table is using S3 for storage.

    Show that the table was created with the correct policy:

    ```sql theme={null}
    SHOW CREATE TABLE my_s3_table;
    ```

    You should see the following result:

    ```response theme={null}
    ┌─statement────────────────────────────────────────────────────
    │ CREATE TABLE default.my_s3_table
    (
      `id` UInt64,
      `column1` String
    )
    ENGINE = MergeTree
    ORDER BY id
    SETTINGS storage_policy = 's3_main', index_granularity = 8192
    └──────────────────────────────────────────────────────────────
    ```

    Let's now insert some rows into our new table:

    ```sql theme={null}
    INSERT INTO my_s3_table (id, column1)
      VALUES (1, 'abc'), (2, 'xyz');
    ```

    Let's verify that our rows were inserted:

    ```sql theme={null}
    SELECT * FROM my_s3_table;
    ```

    ```response theme={null}
    ┌─id─┬─column1─┐
    │  1 │ abc     │
    │  2 │ xyz     │
    └────┴─────────┘

    2 rows in set. Elapsed: 0.284 sec.
    ```

    In the AWS console, if your data was successfully inserted to S3, you should see that ClickHouse has created new files in your specified bucket.

    If everything worked successfully, you're now using ClickHouse with separated storage and compute!

    <Image img="https://mintcdn.com/private-7c7dfe99/k4wNHsd_gyvah7Fr/images/guides/s3_bucket_example.webp?fit=max&auto=format&n=k4wNHsd_gyvah7Fr&q=85&s=5125bc42e40bfb41790536f985463ff6" size="md" alt="S3 bucket example using separation of compute and storage" border width="1208" height="736" data-path="images/guides/s3_bucket_example.webp" />
  </Step>

  <Step title="Implementing replication for fault tolerance (optional)" id="3-implementing-replication-for-fault-tolerance-optional">
    <Warning>
      Don't configure any AWS/GCS life cycle policy. This isn't supported and could lead to broken tables.
    </Warning>

    For fault tolerance, you can use multiple ClickHouse server nodes distributed across multiple AWS regions, with an S3 bucket for each node.

    Replication with S3 disks can be accomplished by using the `ReplicatedMergeTree` table engine. See the following guide for details:

    * [Replicating a single shard across two AWS regions using S3 Object Storage](/docs/integrations/connectors/data-ingestion/AWS/integrating-s3-with-clickhouse#s3-multi-region).
  </Step>
</Steps>

<h2 id="further-reading">
  Further reading
</h2>

* [SharedMergeTree table engine](/docs/products/cloud/features/infrastructure/shared-merge-tree)
* [SharedMergeTree announcement blog](https://clickhouse.com/blog/clickhouse-cloud-boosts-performance-with-sharedmergetree-and-lightweight-updates)