---
title: "How to read an NPY file in Python"
date: "2026-06-15"
author: "Al Brown"
category: "Engineering"
excerpt: "Read a NumPy .npy array into a DataFrame with chDB and work with it using the pandas API you already know, without loading the whole array into memory."
---

# How to read an NPY file in Python

To read an NPY file in Python, use [chDB](/resources/engineering/what-is-chdb) to load the array into a DataFrame and work with it using the pandas API you already know:


```python
from chdb.datastore import DataStore
df = DataStore.from_file("data/readings.npy", format="Npy")
```

`df` supports the pandas API you already use (indexing, filtering, `groupby`, `merge`), but the work runs on ClickHouse's engine instead of pandas, so it stays fast as files grow. There is no server to start and no separate load step.


```bash
pip install chdb
```

## Read a .npy file into a DataFrame

`DataStore.from_file` reads the array into a lazy, ClickHouse-backed object. Point it at the file with `format="Npy"` and print the result:


```python
from chdb.datastore import DataStore

df = DataStore.from_file("data/readings.npy", format="Npy")
print(df)
print(df.dtypes)
```

```text
   array
0  50.00
1  63.09
2  74.73
3  83.66
4  88.88
5  89.82
6  86.37
7  78.92
array    float64
dtype: object
```

One thing is worth knowing about the shape: a `.npy` file stores exactly one numeric array and carries no column names, so chDB exposes a single column called `array`. The dtype is inferred from the array's element type: `float64` here because the file holds 64-bit floats.

Like the CSV and JSON readers, `df` is not the whole array loaded into memory. It is a lazy object that runs on ClickHouse's engine when you materialize a result by printing, calling `len()`, or invoking `.to_pandas()`.

## Filter and aggregate the way you already do

Use the pandas API you already know to aggregate the column:


```python
from chdb.datastore import DataStore

df = DataStore.from_file("data/readings.npy", format="Npy")
mean_val = df["array"].mean()
print(f"mean: {float(mean_val):.2f}")

max_val = df["array"].max()
min_val = df["array"].min()
print(f"max: {float(max_val):.2f}  min: {float(min_val):.2f}")
```

```text
mean: 76.93
max: 89.82  min: 50.00
```

The same approach works for boolean indexing after you materialize to real pandas with `.to_pandas()`:


```python
pdf = df.to_pandas()
good = pdf[pdf["array"] > 75]
print(f"readings above 75: {len(good)}")
```

```text
readings above 75: 5
```

## A 2-D .npy reads as one row per outer element

A `.npy` can hold a matrix as well as a flat array. Save one with `np.save("matrix.npy", np.arange(12).reshape(4, 3))` and chDB reads each outer element as a row in the `array` column:


```python
mat = DataStore.from_file("data/matrix.npy", format="Npy")
print(mat)
```

```text
         array
0    [0, 1, 2]
1    [3, 4, 5]
2    [6, 7, 8]
3  [9, 10, 11]
```

Each element in the `array` column is a list. Call `.to_pandas()` to hand it off to pandas or NumPy for further processing.

## Join two arrays by position

NumPy workflows often keep parallel arrays in separate files: readings in one `.npy`, a quality flag in another, aligned by position. Load each file, hand both off to real pandas, and join on the index:


```python
readings_pdf = (
    DataStore.from_file("data/readings.npy", format="Npy")
    .to_pandas()
    .rename(columns={"array": "reading"})
)
flags_pdf = (
    DataStore.from_file("data/flags.npy", format="Npy")
    .to_pandas()
    .rename(columns={"array": "ok"})
)
combined = readings_pdf.join(flags_pdf)
print(combined)
```

```text
   reading  ok
0    50.00   0
1    63.09   1
2    74.73   1
3    83.66   1
4    88.88   0
5    89.82   1
6    86.37   1
7    78.92   1
```

From there, filter to only the rows where `ok == 1` and compute the mean in pandas:


```python
mean_ok = combined[combined["ok"] == 1]["reading"].mean()
print(f"mean of flagged readings: {mean_ok:.2f}")
```

## Hand off to real pandas when you need it

When a library downstream needs an actual pandas DataFrame (scikit-learn, a plotting call, anything that mutates in place), call `.to_pandas()` to materialize one:


```python
df = DataStore.from_file("data/readings.npy", format="Npy")
pdf = df.to_pandas()   # a real pandas.DataFrame, in memory
```

```text
<class 'pandas.DataFrame'>
```

The usual pattern is to do the heavy filtering and aggregation on the chDB object, where it is fast on large files, then `.to_pandas()` the small result and continue in ordinary pandas.

## Works in Jupyter

Printing a chDB DataStore object renders it as a table in a notebook, and `.to_pandas()` feeds straight into `.plot()`, joins, or any pandas code. The companion folder ships a `run.ipynb` you can open and run cell by cell.

## Run it yourself

The complete, runnable example is here, with `generate.sh` to create the sample `.npy` files plus `run.ipynb` and a `run.py` mirror containing the exact code above:

[github.com/ClickHouse/examples/tree/main/local-analytics/chdb-npy](https://github.com/ClickHouse/examples/tree/main/local-analytics/chdb-npy)

Working with other formats? [Read a JSON file in Python](/resources/engineering/read-json-file-python) and [read a Parquet file in Python](/resources/engineering/read-parquet-file-python) use the same chDB datastore pattern; only the reader changes. The same code scales from a file on your laptop to [ClickHouse Cloud](https://clickhouse.com/cloud) with no rewrite.