Hudi table engine

This engine provides a read-only integration with existing Apache Hudi tables in Amazon S3.

Create table

Note that the Hudi table must already exist in S3, this command does not take DDL parameters to create a new table.

CREATE TABLE hudi_table
    ENGINE = Hudi(url, [aws_access_key_id, aws_secret_access_key,] [extra_credentials])

Engine parameters

url — Bucket url with the path to an existing Hudi table.
aws_access_key_id, aws_secret_access_key - Long-term credentials for the AWS account user. You can use these to authenticate your requests. Parameter is optional. If credentials are not specified, they are used from the configuration file.
extra_credentials - Optional. Used to pass a role_arn for role-based access in ClickHouse Cloud. See Secure S3 for configuration steps.

Engine parameters can be specified using Named Collections. Example

CREATE TABLE hudi_table ENGINE=Hudi('http://mars-doc-test.s3.amazonaws.com/clickhouse-bucket-3/test_table/', 'ABC123', 'Abc+123')

Using named collections:

<clickhouse>
    <named_collections>
        <hudi_conf>
            <url>http://mars-doc-test.s3.amazonaws.com/clickhouse-bucket-3/</url>
            <access_key_id>ABC123</access_key_id>
            <secret_access_key>Abc+123</secret_access_key>
        </hudi_conf>
    </named_collections>
</clickhouse>

CREATE TABLE hudi_table ENGINE=Hudi(hudi_conf, filename = 'test_table')

Limitations

Merge-on-Read is not implemented: the log files of a Merge-on-Read table are not read. Reading a table whose latest file slices have log files, that is changes written after the last compaction, therefore throws NOT_IMPLEMENTED instead of returning the values from before those changes. Schema inference throws for such a table too. Copy-on-Write tables and Merge-on-Read tables without such log files, for example right after a compaction in Hudi, are read as usual.

SQL Reference

Data Types

Engines

Functions

Formats

Settings

System Tables

Data Lakes

Create table

Limitations

See also

​Create table

​Limitations

​See also

Create table

Limitations

See also