---
title: "How to convert CSV to Avro"
date: "2026-06-15"
author: "Al Brown"
category: "Engineering"
excerpt: "Convert a CSV file to Avro with one clickhouse-local command. The Avro schema is derived from the column types inferred from the CSV, with no server and no upload."
---

# How to convert CSV to Avro

To convert a CSV file to Avro, use [`clickhouse local`](/resources/engineering/what-is-clickhouse-local). It runs SQL directly on files from the command line, with no server to install. It's part of ClickHouse, so the same query scales to billions of rows when you outgrow your laptop.

Install it with [`clickhousectl`](https://clickhouse.com/docs/install):


```bash
curl https://clickhouse.com/cli | sh   # install clickhousectl
clickhousectl local use latest         # download ClickHouse and put it on your PATH
```

Then write the result with `FORMAT Avro`:


```bash
clickhouse local -q "SELECT * FROM file('events.csv') INTO OUTFILE 'events.avro' FORMAT Avro"
```

```response
2026-01-20	20	US	signup	24.19	5
2026-01-19	19	GB	purchase	23.18	4
2026-01-18	18	AU	view	22.17	3
2026-01-17	17	IN	click	21.16	2
2026-01-16	16	FR	signup	20.15	1
```

`clickhouse local` reads the CSV header for column names, infers each column's type from the data, and builds the Avro record schema from those types. The CSV is read in place with no import step, and the file streams row by row so conversions larger than RAM complete without issue.

## What the schema becomes

The interesting part of CSV to Avro is the schema. CSV has none; Avro requires one. `clickhouse-local` fills the gap by inferring types from the CSV, then translating them into Avro field types. Look at what it inferred from the CSV:


```bash
clickhouse local -q "DESCRIBE file('events.csv')"
```

```text
event_date	Nullable(Date)
event_id	Nullable(Int64)
country	Nullable(String)
action	Nullable(String)
amount	Nullable(Float64)
items	Nullable(Int64)
```

Avro carries its schema in the file header as JSON. The header from `events.avro` shows the direct mapping:


```response
{"type":"record","name":"row","fields":[{"name":"event_date","type":["null",{"type":"int","logicalType":"date"}]},{"name":"event_id","type":["null","long"]},{"name":"country","type":["null","string"]},{"name":"action","type":["null","string"]},{"name":"amount","type":["null","double"]},{"name":"items","type":["null","long"]}]}
```

`Int64` became Avro `long`, `Float64` became `double`, `String` became `string`, and `Date` became an `int` with `logicalType: date` (Avro stores dates as a day count). Because CSV inference marks every column `Nullable`, each Avro field is a union of `["null", T]`. That is correct but verbose, and it forces every consumer to handle nulls.

> Prefer Python? See [how to read an Avro file in Python with chDB](/resources/engineering/read-avro-file-python). The chDB conversion block is at the end of this page.

## Pin the types for a clean schema

If you know the data has no missing values, declare the types yourself. Pass the format and an explicit, non-`Nullable` schema as the second and third arguments to `file()`. The Avro fields then drop the null unions:


```bash
clickhouse local -q "
SELECT * FROM file('events.csv', 'CSVWithNames',
  'event_date Date, event_id UInt32, country String, action String, amount Float64, items UInt8')
INTO OUTFILE 'events_typed.avro' TRUNCATE FORMAT Avro"
```

The embedded schema is now plain scalars, no unions:


```response
{"type":"record","name":"row","fields":[{"name":"event_date","type":{"type":"int","logicalType":"date"}},{"name":"event_id","type":"int"},{"name":"country","type":"string"},{"name":"action","type":"string"},{"name":"amount","type":"double"},{"name":"items","type":"int"}]}
```

This is the lever an upload-based converter never gives you: you control the exact Avro types from the source CSV, including the integer width and whether a field is nullable. `UInt32` and `UInt8` collapse to Avro `int` (Avro has no unsigned integers), so pick a signed type that fits your range if the distinction matters downstream.

## Verify the round-trip

Read the Avro file straight back to confirm the data and types survived:


```bash
clickhouse local -q "SELECT * FROM file('events.avro') ORDER BY amount DESC LIMIT 5"
```

```text
2026-01-20	20	US	signup	24.19	5
2026-01-19	19	GB	purchase	23.18	4
2026-01-18	18	AU	view	22.17	3
2026-01-17	17	IN	click	21.16	2
2026-01-16	16	FR	signup	20.15	1
```

## Options worth knowing

These are the choices that matter when the Avro file leaves your machine and lands in another system.

- **Compression codec.** Avro compresses each data block. `clickhouse-local` defaults to `snappy` (fast, widely supported). Switch to `deflate` for smaller files when read speed matters less:

  ```bash
  clickhouse local -q "SELECT * FROM file('events.csv') INTO OUTFILE 'events.avro' TRUNCATE FORMAT Avro SETTINGS output_format_avro_codec='deflate'"
  ```

- **Nullability.** Decide it deliberately. Inferred `Nullable` columns produce `["null", T]` unions; an explicit non-`Nullable` schema produces plain types, as shown above.
- **Type lossiness.** A flat CSV maps cleanly to Avro. There is nothing nested to flatten here, unlike going the other direction from a deeply nested format. The only quiet cast is unsigned to signed integers.
- **Compressed input.** A `.csv.gz` or `.csv.zst` source is decompressed transparently. The same command works; the extension is detected automatically.

Online CSV-to-Avro converters own a lot of these searches, and they are fine for a one-off small file. The reasons to do it locally: nothing leaves your machine, you control the exact Avro types, it scripts into a pipeline, and it handles files bigger than memory.

## How fast is it?

On a 3,000,000-row, ~120 MB CSV (`events_large.csv`), the full conversion to Avro, parsing the CSV text and encoding the Avro blocks, completes in:


```bash
clickhouse local -q "SELECT * FROM file('events_large.csv') INTO OUTFILE 'events_large.avro' TRUNCATE FORMAT Avro"
```

**~0.73 seconds**, best of three with a warm OS page cache, on an Apple M4 Pro laptop (14 cores, 24 GB RAM). The number can move a little under concurrent load, but for a CSV this size the conversion is bound by parsing, not by Avro encoding. The 3,000,000 rows round-trip exactly when read back.

## chDB: the same conversion in Python

If you work in Python, [chDB](/resources/engineering/what-is-chdb) is the same ClickHouse engine in-process. Run the same `SELECT ... FORMAT Avro` and write the bytes to a file:


```python
import chdb

avro_bytes = chdb.query(
    "SELECT * FROM file('events.csv', 'CSVWithNames') FORMAT Avro"
).bytes()

with open("events.avro", "wb") as f:
    f.write(avro_bytes)
```

The schema is derived from the inferred types exactly as with the CLI. This is a drop-in alternative to writing Avro by hand with `pyarrow` or `fastavro` when your input is already a CSV.

## Reverse direction?

Going the other way is the mirror command, swap the formats. See [how to convert Avro to CSV](/resources/engineering/convert-avro-to-csv). To learn the format itself, see [what is an Avro file](/resources/engineering/what-is-an-avro-file), and to query Avro in place without converting, see [how to read an Avro file](/resources/engineering/read-avro-file).

## Run it yourself

The complete, runnable example lives in the ClickHouse examples repo: `generate.sh` to create the sample CSVs (including the ~120 MB file used for the timing above), `run.sh` with every command on this page, `run.py` / `run.ipynb` for the chDB version, and `expected_output.txt`.

[github.com/ClickHouse/examples/tree/main/local-analytics/convert-csv-to-avro](https://github.com/ClickHouse/examples/tree/main/local-analytics/convert-csv-to-avro)

The same SQL that converts a file on your laptop runs unchanged against a ClickHouse server or [ClickHouse Cloud](https://clickhouse.com/cloud) when the data outgrows it.