input_format_allow_errors_num
Sets the maximum number of acceptable errors when reading from text formats (CSV, TSV, etc.). The default value is 0. Always pair it withinput_format_allow_errors_ratio.
If an error occurred while reading rows but the error counter is still less than input_format_allow_errors_num, ClickHouse ignores the row and moves on to the next one.
If both input_format_allow_errors_num and input_format_allow_errors_ratio are exceeded, ClickHouse throws an exception.
input_format_allow_errors_ratio
Sets the maximum percentage of errors allowed when reading from text formats (CSV, TSV, etc.). The percentage of errors is set as a floating-point number between 0 and 1. The default value is 0. Always pair it withinput_format_allow_errors_num.
If an error occurred while reading rows but the error counter is still less than input_format_allow_errors_ratio, ClickHouse ignores the row and moves on to the next one.
If both input_format_allow_errors_num and input_format_allow_errors_ratio are exceeded, ClickHouse throws an exception.
input_format_allow_seeks
Allow seeks (or range reads) while reading ORC, Parquet, and Arrow input formats. When enabled and the source supports it (e.g. local file, S3, HTTP with range support and known size), ClickHouse can read only the needed byte ranges and use less memory. When disabled, or when the source does not support seeks (e.g. no file size, or stream not seekable), some readers may fall back to loading the full file into memory. Enabled by default.input_format_arrow_allow_missing_columns
Allow missing columns while reading Arrow input formatsinput_format_arrow_case_insensitive_column_matching
Ignore case when matching Arrow columns with CH columns.input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference
Skip columns with unsupported types while schema inference for format Arrowinput_format_arrow_use_native_reader
Use the native ClickHouse reader for the Arrow and ArrowStream formats instead of the one based on the Apache Arrow library.input_format_avro_allow_missing_fields
For Avro/AvroConfluent format: when field is not found in schema use default value instead of errorinput_format_avro_null_as_default
For Avro/AvroConfluent format: insert default in case of null and non Nullable columinput_format_binary_decode_types_in_binary_format
Read data types in binary format instead of type names in RowBinaryWithNamesAndTypes input formatinput_format_binary_max_type_complexity
Max type nodes when decoding binary types (not depth, but total count).Map(String, UInt32) = 3 nodes. Protects against malicious inputs. 0 = unlimited.
input_format_binary_read_json_as_string
Read values of JSON data type as JSON String values in RowBinary input format.input_format_bson_skip_fields_with_unsupported_types_in_schema_inference
Skip fields with unsupported types while schema inference for format BSON.input_format_capn_proto_skip_fields_with_unsupported_types_in_schema_inference
Skip columns with unsupported types while schema inference for format CapnProtoinput_format_column_name_matching_mode
Defines the column name matching mode when ingesting data through various formats (including but not limited to JSONEachRow, CSVWithNames, JSONColumns, BSONEachRow, RowBinaryWithNames). Supported modes:- match_case: match case-sensitively
- ignore_case: match case-insensitively
- auto: first tries to match case-sensitively, if fails, tries to match case-insensitively.
input_format_connection_handling
When this option is enabled, if the connection closes unexpectedly, any remaining data in the buffer will be parsed and processed instead of being treated as an errorEnabling this option disables parallel parsing and makes deduplication impossible
input_format_csv_allow_cr_end_of_line
If it is set true, \r will be allowed at end of line not followed byinput_format_csv_allow_variable_number_of_columns
Ignore extra columns in CSV input (if file has more columns than expected) and treat missing fields in CSV input as default valuesinput_format_csv_allow_whitespace_or_tab_as_delimiter
Allow to use spaces and tabs(\t) as field delimiter in the CSV stringsinput_format_csv_arrays_as_nested_csv
When reading Array from CSV, expect that its elements were serialized in nested CSV and then put into string. Example: ”[""Hello"", ""world"", ""42"""" TV""]”. Braces around array can be omitted.input_format_csv_deserialize_separate_columns_into_tuple
If it set to true, then separate columns written in CSV format can be deserialized to Tuple column.input_format_csv_detect_header
Automatically detect header with names and types in CSV formatinput_format_csv_empty_as_default
Treat empty fields in CSV input as default values.input_format_csv_enum_as_number
Treat inserted enum values in CSV formats as enum indicesinput_format_csv_missing_nullable_as_empty_string
Controls howNullable(String) is read from a missing value in CSV. A missing value is an empty space between/before/after commas, not surrounded by quotes. If this setting is enabled, regardless of the value of input_format_csv_empty_as_default, the missing value of Nullable(String) will be interpreted as an empty String, not as NULL.
input_format_csv_skip_first_lines
Skip specified number of lines at the beginning of data in CSV formatinput_format_csv_skip_trailing_empty_lines
Skip trailing empty lines in CSV formatinput_format_csv_trim_whitespaces
Trims spaces and tabs (\t) characters at the beginning and end in CSV stringsinput_format_csv_try_infer_numbers_from_strings
If enabled, during schema inference ClickHouse will try to infer numbers from string fields. It can be useful if CSV data contains quoted UInt64 numbers. Disabled by default.input_format_csv_try_infer_strings_from_quoted_tuples
Interpret quoted tuples in the input data as a value of type String.input_format_csv_use_best_effort_in_schema_inference
Use some tweaks and heuristics to infer schema in CSV formatinput_format_csv_use_default_on_bad_values
Allow to set default value to column when CSV field deserialization failed on bad valueinput_format_custom_allow_variable_number_of_columns
Ignore extra columns in CustomSeparated input (if file has more columns than expected) and treat missing fields in CustomSeparated input as default valuesinput_format_custom_detect_header
Automatically detect header with names and types in CustomSeparated formatinput_format_custom_skip_trailing_empty_lines
Skip trailing empty lines in CustomSeparated formatinput_format_defaults_for_omitted_fields
When performingINSERT queries, replace omitted input column values with default values of the respective columns. This option applies to JSONEachRow (and other JSON formats), CSV, TabSeparated, TSKV, Parquet, Arrow, Avro, ORC, Native formats and formats with WithNames/WithNamesAndTypes suffixes.
When this option is enabled, extended table metadata are sent from server to client. It consumes additional computing resources on the server and can reduce performance.
- 0 — Disabled.
- 1 — Enabled.
input_format_force_null_for_omitted_fields
Force initialize omitted fields with null valuesinput_format_geojson_unsupported_geometry_handling
Controls what happens when a validGeoJSON geometry type that cannot be represented in ClickHouse’s Geometry type (such as GeometryCollection or MultiPoint) must be stored in the geometry column while reading GeoJSON input.
Possible values:
'throw'(default) — throw an exception.'null'— insert aNULLvalue for thegeometrycolumn and continue parsing.
geometry column is materialized. When it is not a requested output column, such a geometry is validated for well-formedness but does not trigger the handling.
input_format_hive_text_allow_variable_number_of_columns
Ignore extra columns in Hive Text input (if file has more columns than expected) and treat missing fields in Hive Text input as default valuesinput_format_hive_text_collection_items_delimiter
Delimiter between collection(array or map) items in Hive Text Fileinput_format_hive_text_fields_delimiter
Delimiter between fields in Hive Text Fileinput_format_hive_text_map_keys_delimiter
Delimiter between a pair of map key/values in Hive Text Fileinput_format_import_nested_json
Enables or disables the insertion of JSON data with nested objects. Supported formats: Possible values:- 0 — Disabled.
- 1 — Enabled.
- Usage of Nested Structures with the
JSONEachRowformat.
input_format_ipv4_default_on_conversion_error
Deserialization of IPv4 will use default values instead of throwing exception on conversion error. Disabled by default.input_format_ipv6_default_on_conversion_error
Deserialization of IPV6 will use default values instead of throwing exception on conversion error. Disabled by default.input_format_json_compact_allow_variable_number_of_columns
Allow variable number of columns in rows in JSONCompact/JSONCompactEachRow input formats. Ignore extra columns in rows with more columns than expected and treat missing columns as default values. Disabled by default.input_format_json_defaults_for_missing_elements_in_named_tuple
Insert default values for missing elements in JSON object while parsing named tuple. This setting works only when settinginput_format_json_named_tuples_as_objects is enabled.
Enabled by default.
input_format_json_empty_as_default
When enabled, replace empty input fields in JSON with default values. For complex default expressionsinput_format_defaults_for_omitted_fields must be enabled too.
Possible values:
- 0 — Disable.
- 1 — Enable.
input_format_json_ignore_unknown_keys_in_named_tuple
Ignore unknown keys in json object for named tuples. Enabled by default.input_format_json_ignore_unnecessary_fields
Ignore unnecessary fields and not parse them. Enabling this may not throw exceptions on json strings of invalid format or with duplicated fieldsinput_format_json_infer_array_of_dynamic_from_array_of_different_types
If enabled, during schema inference ClickHouse will use Array(Dynamic) type for JSON arrays with values of different data types. Example:input_format_json_infer_incomplete_types_as_strings
Allow to use String type for JSON keys that contain onlyNull/{}/[] in data sample during schema inference.
In JSON formats any value can be read as String, and we can avoid errors like Cannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps during schema inference
by using String type for keys with unknown types.
Example:
input_format_json_map_as_array_of_tuples
Deserialize maps columns as JSON arrays of tuples. Disabled by default.input_format_json_max_depth
Maximum depth of a field in JSON. This is not a strict limit, it does not have to be applied precisely.input_format_json_named_tuples_as_objects
Parse named tuple columns as JSON objects. Enabled by default.input_format_json_read_arrays_as_strings
Allow parsing JSON arrays as strings in JSON input formats. Example:input_format_json_read_bools_as_numbers
Allow parsing bools as numbers in JSON input formats. Enabled by default.input_format_json_read_bools_as_strings
Allow parsing bools as strings in JSON input formats. Enabled by default.input_format_json_read_numbers_as_strings
Allow parsing numbers as strings in JSON input formats. Enabled by default.input_format_json_read_objects_as_strings
Allow parsing JSON objects as strings in JSON input formats. Example:input_format_json_throw_on_bad_escape_sequence
Throw an exception if JSON string contains bad escape sequence in JSON input formats. If disabled, bad escape sequences will remain as is in the data. Enabled by default.input_format_json_try_infer_named_tuples_from_objects
If enabled, during schema inference ClickHouse will try to infer named Tuple from JSON objects. The resulting named Tuple will contain all elements from all corresponding JSON objects from sample data. Example:input_format_json_try_infer_numbers_from_strings
If enabled, during schema inference ClickHouse will try to infer numbers from string fields. It can be useful if JSON data contains quoted UInt64 numbers. Disabled by default.input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects
Use String type instead of an exception in case of ambiguous paths in JSON objects during named tuples inferenceinput_format_json_validate_types_from_metadata
For JSON/JSONCompact/JSONColumnsWithMetadata input formats, if this setting is set to 1, the types from metadata in input data will be compared with the types of the corresponding columns from the table. Enabled by default.input_format_max_block_size_bytes
Limits the size of the blocks formed during data parsing in input formats in bytes. Used in row based input formats when block is formed on ClickHouse side. 0 means no limit in bytes.input_format_max_block_wait_ms
Limits the maximum time in milliseconds to wait before emitting a block during parsing in row-based input formats. 0 means no limit.This option only works if
input_format_connection_handling is enabled. Setting a value also disables parallel parsing and makes deduplication impossible.For streaming inserts, you must also set
min_insert_block_size_rows=0 and min_insert_block_size_bytes=0. Otherwise, parsed blocks may still be accumulated in memory by the block squashing stage until those thresholds are reached, preventing timely inserts.input_format_max_bytes_to_read_for_schema_inference
The maximum amount of data in bytes to read for automatic schema inference.input_format_max_rows_to_read_for_schema_inference
The maximum rows of data to read for automatic schema inference.input_format_msgpack_number_of_columns
The number of columns in inserted MsgPack data. Used for automatic schema inference from data.input_format_mysql_dump_map_column_names
Match columns from table in MySQL dump and columns from ClickHouse table by namesinput_format_mysql_dump_table_name
Name of the table in MySQL dump from which to read datainput_format_native_allow_types_conversion
Allow data types conversion in Native input formatinput_format_native_decode_types_in_binary_format
Read data types in binary format instead of type names in Native input formatinput_format_null_as_default
Enables or disables the initialization of NULL fields with default values, if data type of these fields is not nullable. If column type is not nullable and this setting is disabled, then insertingNULL causes an exception. If column type is nullable, then NULL values are inserted as is, regardless of this setting.
This setting is applicable for most input formats.
For complex default expressions input_format_defaults_for_omitted_fields must be enabled too.
Possible values:
- 0 — Inserting
NULLinto a not nullable column causes an exception. - 1 —
NULLfields are initialized with default column values.
input_format_orc_allow_missing_columns
Allow missing columns while reading ORC input formatsinput_format_orc_case_insensitive_column_matching
Ignore case when matching ORC columns with CH columns.input_format_orc_dictionary_as_low_cardinality
Treat ORC dictionary encoded columns as LowCardinality columns while reading ORC files.input_format_orc_filter_push_down
When reading ORC files, skip whole stripes or row groups based on the WHERE/PREWHERE expressions, min/max statistics or bloom filter in the ORC metadata.input_format_orc_reader_time_zone_name
The time zone name for ORC row reader, the default ORC row reader’s time zone is GMT.input_format_orc_row_batch_size
Batch size when reading ORC stripes.input_format_orc_skip_columns_with_unsupported_types_in_schema_inference
Skip columns with unsupported types while schema inference for format ORCinput_format_orc_use_fast_decoder
Use a faster ORC decoder implementation.input_format_parallel_parsing
Enables or disables order-preserving parallel parsing of data formats. Supported only for TabSeparated (TSV), TSKV, CSV and JSONEachRow formats. Possible values:- 1 — Enabled.
- 0 — Disabled.
input_format_parquet_allow_geoparquet_parser
Use geo column parser to convert Array(UInt8) into Point/Linestring/Polygon/MultiLineString/MultiPolygon typesinput_format_parquet_allow_missing_columns
Allow missing columns while reading Parquet input formatsinput_format_parquet_bloom_filter_push_down
When reading Parquet files, skip whole row groups based on the WHERE expressions and bloom filter in the Parquet metadata.input_format_parquet_case_insensitive_column_matching
Ignore case when matching Parquet columns with CH columns.input_format_parquet_enable_json_parsing
When reading Parquet files, parse JSON columns as ClickHouse JSON Column.input_format_parquet_enable_row_group_prefetch
Enable row group prefetching during parquet parsing. Currently, only single-threaded parsing can prefetch.input_format_parquet_filter_push_down
When reading Parquet files, skip whole row groups based on the WHERE/PREWHERE expressions and min/max statistics in the Parquet metadata.input_format_parquet_local_file_min_bytes_for_seek
Min bytes required for local read (file) to do seek, instead of read with ignore in Parquet input formatinput_format_parquet_local_time_as_utc
Determines the data type used by schema inference for Parquet timestamps with isAdjustedToUTC=false. If true: DateTime64(…, ‘UTC’), if false: DateTime64(…). Neither behavior is fully correct as ClickHouse doesn’t have a data type for local wall-clock time. Counterintuitively, ‘true’ is probably the less incorrect option, because formatting the ‘UTC’ timestamp as String will produce representation of the correct local time.input_format_parquet_max_block_size
Max block size for parquet reader.input_format_parquet_memory_high_watermark
Approximate memory limit for Parquet reader v3. Limits how many row groups or columns can be read in parallel. When reading multiple files in one query, the limit is on total memory usage across those files.input_format_parquet_memory_low_watermark
Schedule prefetches more aggressively if memory usage is below than threshold. Potentially useful e.g. if there are many small bloom filters to read over network.input_format_parquet_page_filter_push_down
Skip pages using min/max values from column index.input_format_parquet_prefer_block_bytes
Average block bytes output by parquet readerinput_format_parquet_preserve_order
Avoid reordering rows when reading from Parquet files. Not recommended as row ordering is generally not guaranteed, and other parts of query pipeline may break it. UseORDER BY _row_number instead.
input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference
Skip columns with unsupported types while schema inference for format Parquetinput_format_parquet_use_offset_index
Minor tweak to how pages are read from parquet file when no page filtering is used.input_format_parquet_verify_checksums
Verify page checksums when reading parquet files.input_format_protobuf_flatten_google_wrappers
Enable Google wrappers for regular non-nested columns, e.g. google.protobuf.StringValue ‘str’ for String column ‘str’. For Nullable columns empty wrappers are recognized as defaults, and missing as nullsinput_format_protobuf_oneof_presence
Indicate which field of protobuf oneof was found by means of setting enum value in a special columinput_format_protobuf_skip_fields_with_unsupported_types_in_schema_inference
Skip fields with unsupported types while schema inference for format Protobufinput_format_record_errors_file_path
Path of the file used to record errors while reading text formats (CSV, TSV).input_format_skip_unknown_fields
Enables or disables skipping insertion of extra data. When writing data, ClickHouse throws an exception if input data contain columns that do not exist in the target table. If skipping is enabled, ClickHouse does not insert extra data and does not throw an exception. Supported formats:- JSONEachRow (and other JSON formats)
- BSONEachRow (and other JSON formats)
- TSKV
- All formats with suffixes WithNames/WithNamesAndTypes
- MySQLDump
- Native
- 0 — Disabled.
- 1 — Enabled.
input_format_try_infer_dates
If enabled, ClickHouse will try to infer typeDate from string fields in schema inference for text formats. If all fields from a column in input data were successfully parsed as dates, the result type will be Date, if at least one field was not parsed as date, the result type will be String.
Enabled by default.
input_format_try_infer_datetimes
If enabled, ClickHouse will try to infer typeDateTime64 from string fields in schema inference for text formats. If all fields from a column in input data were successfully parsed as datetimes, the result type will be DateTime64, if at least one field was not parsed as datetime, the result type will be String.
Enabled by default.
input_format_try_infer_datetimes_only_datetime64
When input_format_try_infer_datetimes is enabled, infer only DateTime64 but not DateTime typesinput_format_try_infer_exponent_floats
Try to infer floats in exponential notation while schema inference in text formats (except JSON, where exponent numbers are always inferred)input_format_try_infer_integers
If enabled, ClickHouse will try to infer integers instead of floats in schema inference for text formats. If all numbers in the column from input data are integers, the result type will beInt64, if at least one number is float, the result type will be Float64.
Enabled by default.
input_format_try_infer_variants
If enabled, ClickHouse will try to infer typeVariant in schema inference for text formats when there is more than one possible type for column/array elements.
Possible values:
- 0 — Disabled.
- 1 — Enabled.
input_format_tsv_allow_variable_number_of_columns
Ignore extra columns in TSV input (if file has more columns than expected) and treat missing fields in TSV input as default valuesinput_format_tsv_crlf_end_of_line
If it is set true, file function will read TSV format with \r\n instead of \n.input_format_tsv_detect_header
Automatically detect header with names and types in TSV formatinput_format_tsv_empty_as_default
Treat empty fields in TSV input as default values.input_format_tsv_enum_as_number
Treat inserted enum values in TSV formats as enum indices.input_format_tsv_skip_first_lines
Skip specified number of lines at the beginning of data in TSV formatinput_format_tsv_skip_trailing_empty_lines
Skip trailing empty lines in TSV formatinput_format_tsv_use_best_effort_in_schema_inference
Use some tweaks and heuristics to infer schema in TSV formatinput_format_values_accurate_types_of_literals
For Values format: when parsing and interpreting expressions using template, check actual type of literal to avoid possible overflow and precision issues.input_format_values_deduce_templates_of_expressions
For Values format: if the field could not be parsed by streaming parser, run SQL parser, deduce template of the SQL expression, try to parse all rows using template and then interpret expression for all rows.input_format_values_interpret_expressions
For Values format: if the field could not be parsed by streaming parser, run SQL parser and try to interpret it as SQL expression.input_format_with_names_use_header
Enables or disables checking the column order when inserting data. To improve insert performance, we recommend disabling this check if you are sure that the column order of the input data is the same as in the target table. Supported formats:- CSVWithNames
- CSVWithNamesAndTypes
- TabSeparatedWithNames
- TabSeparatedWithNamesAndTypes
- JSONCompactEachRowWithNames
- JSONCompactEachRowWithNamesAndTypes
- JSONCompactStringsEachRowWithNames
- JSONCompactStringsEachRowWithNamesAndTypes
- RowBinaryWithNames
- RowBinaryWithNamesAndTypes
- CustomSeparatedWithNames
- CustomSeparatedWithNamesAndTypes
- 0 — Disabled.
- 1 — Enabled.
input_format_with_types_use_header
Controls whether format parser should check if data types from the input data match data types from the target table. Supported formats:- CSVWithNamesAndTypes
- TabSeparatedWithNamesAndTypes
- JSONCompactEachRowWithNamesAndTypes
- JSONCompactStringsEachRowWithNamesAndTypes
- RowBinaryWithNamesAndTypes
- CustomSeparatedWithNamesAndTypes
- 0 — Disabled.
- 1 — Enabled.