> ## Documentation Index
> Fetch the complete documentation index at: https://clickhouse.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

> Documentation for the Arrow format

# Arrow

| Input | Output | Alias |
| ----- | ------ | ----- |
| ✔     | ✔      |       |

<h2 id="description">
  Description
</h2>

[Apache Arrow](https://arrow.apache.org/) comes with two built-in columnar storage formats.
ClickHouse supports read and write operations for these formats.
`Arrow` is Apache Arrow's "file mode" format, designed for in-memory random access.

<h2 id="data-types-matching">
  Data types matching
</h2>

The table below shows the supported data types and how they correspond to ClickHouse [data types](/docs/reference/data-types/index) in `INSERT` and `SELECT` queries.

| Arrow data type (`INSERT`)              | ClickHouse data type                                                                                      | Arrow data type (`SELECT`) |
| --------------------------------------- | --------------------------------------------------------------------------------------------------------- | -------------------------- |
| `BOOL`                                  | [Bool](/docs/reference/data-types/boolean)                                                                     | `BOOL`                     |
| `UINT8`, `BOOL`                         | [UInt8](/docs/reference/data-types/int-uint)                                                                   | `UINT8`                    |
| `INT8`                                  | [Int8](/docs/reference/data-types/int-uint)/[Enum8](/docs/reference/data-types/enum)                                | `INT8`                     |
| `UINT16`                                | [UInt16](/docs/reference/data-types/int-uint)                                                                  | `UINT16`                   |
| `INT16`                                 | [Int16](/docs/reference/data-types/int-uint)/[Enum16](/docs/reference/data-types/enum)                              | `INT16`                    |
| `UINT32`                                | [UInt32](/docs/reference/data-types/int-uint)                                                                  | `UINT32`                   |
| `INT32`                                 | [Int32](/docs/reference/data-types/int-uint)                                                                   | `INT32`                    |
| `UINT64`                                | [UInt64](/docs/reference/data-types/int-uint)                                                                  | `UINT64`                   |
| `INT64`                                 | [Int64](/docs/reference/data-types/int-uint)                                                                   | `INT64`                    |
| `FLOAT`, `HALF_FLOAT`                   | [Float32](/docs/reference/data-types/float)                                                                    | `FLOAT32`                  |
| `DOUBLE`                                | [Float64](/docs/reference/data-types/float)                                                                    | `FLOAT64`                  |
| `DATE32`                                | [Date32](/docs/reference/data-types/date32)                                                                    | `UINT16`                   |
| `DATE64`                                | [DateTime](/docs/reference/data-types/datetime)                                                                | `UINT32`                   |
| `TIMESTAMP`                             | [DateTime64](/docs/reference/data-types/datetime64)                                                            | `TIMESTAMP`                |
| `TIME32`, `TIME64`                      | [Time64](/docs/reference/data-types/time64)                                                                    | `TIME32`, `TIME64`         |
| `STRING`, `BINARY`                      | [String](/docs/reference/data-types/string)                                                                    | `BINARY`                   |
| `STRING`, `BINARY`, `FIXED_SIZE_BINARY` | [FixedString](/docs/reference/data-types/fixedstring)                                                          | `FIXED_SIZE_BINARY`        |
| `DECIMAL`                               | [Decimal](/docs/reference/data-types/decimal)                                                                  | `DECIMAL`                  |
| `DECIMAL256`                            | [Decimal256](/docs/reference/data-types/decimal)                                                               | `DECIMAL256`               |
| `LIST`                                  | [Array](/docs/reference/data-types/array)                                                                      | `LIST`                     |
| `STRUCT`                                | [Tuple](/docs/reference/data-types/tuple)                                                                      | `STRUCT`                   |
| `MAP`                                   | [Map](/docs/reference/data-types/map)                                                                          | `MAP`                      |
| `UINT32`                                | [IPv4](/docs/reference/data-types/ipv4)                                                                        | `UINT32`                   |
| `FIXED_SIZE_BINARY`, `BINARY`           | [IPv6](/docs/reference/data-types/ipv6)                                                                        | `FIXED_SIZE_BINARY`        |
| `FIXED_SIZE_BINARY`, `BINARY`           | [Int128/UInt128/Int256/UInt256](/docs/reference/data-types/int-uint)                                           | `FIXED_SIZE_BINARY`        |
| `DURATION`                              | [Interval](/docs/reference/data-types/special-data-types/interval) (Nanosecond/Microsecond/Millisecond/Second) | `DURATION`                 |
| `INT64`                                 | [Interval](/docs/reference/data-types/special-data-types/interval) (Minute/Hour/Day/Week/Month/Quarter/Year)   | `INT64`                    |

Arrays can be nested and can have a value of the `Nullable` type as an argument. `Tuple` and `Map` types can also be nested.

The `DICTIONARY` type is supported for `INSERT` queries, and for `SELECT` queries there is an [`output_format_arrow_low_cardinality_as_dictionary`](/docs/reference/settings/formats/output-format#output_format_arrow_low_cardinality_as_dictionary) setting that allows to output [LowCardinality](/docs/reference/data-types/lowcardinality) type as a `DICTIONARY` type. Note that there might be unused values in `LowCardinality` dictionary, which can lead to unused values in Arrow `DICTIONARY` during output.

Unsupported Arrow data types:

* `JSON`
* `ENUM`.

The data types of ClickHouse table columns do not have to match the corresponding Arrow data fields. When inserting data, ClickHouse interprets data types according to the table above and then [casts](/docs/reference/functions/regular-functions/type-conversion-functions#CAST) the data to the data type set for the ClickHouse table column.

<h2 id="example-usage">
  Example usage
</h2>

In the example below we use the `forex` dataset available in the
[ClickHouse SQL playground](https://sql.clickhouse.com).

<h3 id="selecting-data">
  Selecting data
</h3>

We select one day of `EUR/USD` exchange rates from the playground and save it
into a local `forex_eurusd.arrow` file. We query the playground over the HTTP
interface, where the host is `sql-clickhouse.clickhouse.com` and the user is
`demo` (which has no password):

```bash theme={null}
curl "https://sql-clickhouse.clickhouse.com:8443/?user=demo&database=forex" \
    --data-binary "
        SELECT
            concat(base, '.', quote) AS base_quote,
            datetime AS last_update,
            CAST(bid, 'Float32') AS bid,
            CAST(ask, 'Float32') AS ask,
            ask - bid AS spread
        FROM forex
        WHERE base = 'EUR' AND quote = 'USD'
            AND datetime >= '2020-01-01' AND datetime < '2020-01-02'
        ORDER BY datetime ASC
        FORMAT Arrow
        SETTINGS output_format_arrow_compression_method='zstd'" > forex_eurusd.arrow
```

<h3 id="reading-data">
  Reading the file back
</h3>

We can now read the local Arrow file back with
[`clickhouse-local`](/docs/concepts/features/tools-and-utilities/clickhouse-local) using the
[`file`](/docs/reference/functions/table-functions/file) table function. The file is
self-describing, so the `Arrow` format infers the schema automatically:

```bash theme={null}
clickhouse-local --query "
    SELECT *
    FROM file('forex_eurusd.arrow', Arrow)
    ORDER BY last_update ASC
    LIMIT 5
    FORMAT PrettyCompact"
```

```response title="Response" theme={null}
   ┌─base_quote─┬─────────────last_update─┬─────bid─┬─────ask─┬────────────────spread─┐
1. │ EUR.USD    │ 2020-01-01 17:00:00.065 │  1.1212 │ 1.12172 │ 0.0005199909210205078 │
2. │ EUR.USD    │ 2020-01-01 17:00:10.447 │  1.1212 │ 1.12192 │ 0.0007200241088867188 │
3. │ EUR.USD    │ 2020-01-01 17:00:10.498 │ 1.12117 │ 1.12161 │ 0.0004400014877319336 │
4. │ EUR.USD    │ 2020-01-01 17:00:12.579 │  1.1212 │ 1.12161 │ 0.0004100799560546875 │
5. │ EUR.USD    │ 2020-01-01 17:00:12.630 │  1.1212 │ 1.12172 │ 0.0005199909210205078 │
   └────────────┴─────────────────────────┴─────────┴─────────┴───────────────────────┘
```

<h3 id="inserting-data">
  Inserting data
</h3>

To load an Arrow file into a ClickHouse table, pipe it into `clickhouse-client`
with `FORMAT Arrow`:

```bash theme={null}
cat forex_eurusd.arrow | clickhouse-client --query="INSERT INTO some_table FORMAT Arrow"
```

<h2 id="format-settings">
  Format settings
</h2>

| Setting                                                                      | Description                                                                                                                                      | Default     |
| ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- |
| `input_format_arrow_allow_missing_columns`                                   | Allow missing columns while reading Arrow input formats                                                                                          | `1`         |
| `input_format_arrow_case_insensitive_column_matching`                        | Ignore case when matching Arrow columns with CH columns.                                                                                         | `0`         |
| `input_format_arrow_import_nested`                                           | Obsolete setting, does nothing.                                                                                                                  | `0`         |
| `input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference` | Skip columns with unsupported types while schema inference for format Arrow                                                                      | `0`         |
| `output_format_arrow_compression_method`                                     | Compression method for Arrow output format. Supported codecs: lz4\_frame, zstd, none (uncompressed)                                              | `lz4_frame` |
| `output_format_arrow_fixed_string_as_fixed_byte_array`                       | Use Arrow FIXED\_SIZE\_BINARY type instead of Binary for FixedString columns.                                                                    | `1`         |
| `output_format_arrow_low_cardinality_as_dictionary`                          | Enable output LowCardinality type as Dictionary Arrow type                                                                                       | `0`         |
| `output_format_arrow_string_as_string`                                       | Use Arrow String type instead of Binary for String columns                                                                                       | `1`         |
| `output_format_arrow_unsupported_types_as_binary`                            | Output a type that has no Arrow equivalent (e.g. `BFloat16`, `AggregateFunction`) as raw binary data. If false, such a type raises an exception. | `1`         |
| `output_format_arrow_use_64_bit_indexes_for_dictionary`                      | Always use 64 bit integers for dictionary indexes in Arrow format                                                                                | `0`         |
| `output_format_arrow_use_signed_indexes_for_dictionary`                      | Use signed integers for dictionary indexes in Arrow format                                                                                       | `1`         |
