hdfs_create_new_file_on_insert
Enables or disables creating a new file on each insert in HDFS engine tables. If enabled, on each insert a new HDFS file will be created with the name, similar to this pattern: initial:data.Parquet.gz -> data.1.Parquet.gz -> data.2.Parquet.gz, etc.
Possible values:
- 0 —
INSERTquery appends new data to the end of the file. - 1 —
INSERTquery creates a new file.
hdfs_ignore_file_doesnt_exist
Ignore absence of file if it does not exist when reading certain keys. Possible values:- 1 —
SELECTreturns empty result. - 0 —
SELECTthrows an exception.
hdfs_replication
The actual number of replications can be specified when the hdfs file is created.hdfs_skip_empty_files
Enables or disables skipping empty files in HDFS engine tables. Possible values:- 0 —
SELECTthrows an exception if empty file is not compatible with requested format. - 1 —
SELECTreturns empty result for empty file.
hdfs_skip_hidden_files
Enables or disables skipping hidden paths when listing files by glob in HDFS engine tables: files with a path segment starting with_ or . below the non-glob prefix of the path. A segment containing = is a partition directory (such as _p=1 for a partition column _p) and is not hidden. Hive-ecosystem writers (Hive, Spark, Trino) stage in-progress output and write marker files under such paths (_temporary/, .spark-staging-<id>/, _SUCCESS), and reading them races with the writer’s commit. Segments within the non-glob prefix are not checked. Tables with partition_strategy = 'hive' always skip hidden paths regardless of this setting.
Possible values:
- 0 — every file matched by the glob is read.
- 1 — files with a hidden path segment below the non-glob prefix are skipped.
hdfs_throw_on_zero_files_match
Throw an error if matched zero files according to glob expansion rules. Possible values:- 1 —
SELECTthrows an exception. - 0 —
SELECTreturns empty result.
hdfs_truncate_on_insert
Enables or disables truncation before an insert in hdfs engine tables. If disabled, an exception will be thrown on an attempt to insert if a file in HDFS already exists. Possible values:- 0 —
INSERTquery appends new data to the end of the file. - 1 —
INSERTquery replaces existing content of the file with the new data.