Skip to main content
These settings are available in system.settings and are autogenerated from source.

iceberg_insert_max_bytes_in_data_file

Max bytes of iceberg parquet data file on insert operation. The default mirrors the documented default of the Iceberg table property write.target-file-size-bytes (512 MiB), see https://iceberg.apache.org/docs/1.5.2/configuration/. The limit is compared against the size of the file written so far, that is, the compressed size, so a file can exceed it by what the writer still buffers, such as incomplete Parquet row groups and the ones being encoded. iceberg_insert_max_rows_in_data_file caps the file independently of its size.

iceberg_insert_max_data_files_in_upload

The maximum number of finished data files of one insert stream whose upload is not waited for while the INSERT writes the next file, for object storages that upload in the background (S3, Azure). Such a file holds its upload buffers until its upload completes, so higher values hide more upload latency at the cost of memory. 0 waits for the upload of each data file before writing the next one.

iceberg_insert_max_partitions

Max allowed partitions count per one insert operation for Iceberg table engine.

iceberg_insert_max_rows_in_data_file

Max rows of iceberg parquet data file on insert operation.

iceberg_insert_max_sort_buffer_bytes

Max uncompressed bytes an INSERT into an Iceberg table with a sort order buffers across all partitions before writing them sorted. Past it, the largest buffer is written early as a smaller sorted data file. 0 means unlimited.

iceberg_insert_single_manifest_for_all_partitions

An INSERT into a partitioned Iceberg table writes one manifest file for all the partitions it touches, like Spark and PyIceberg, instead of one manifest file per partition. This holds per insert stream: every stream commits its own snapshot with its own manifest, so an INSERT writes exactly one manifest only with max_insert_threads = 1. OPTIMIZE TABLE ... MANIFEST then also merges manifests of different partitions into one. ALTER TABLE ... DROP PARTITION cannot drop a partition whose manifest is shared with other partitions.