When using the Memory table engine on ClickHouse Cloud, data is not replicated across all nodes (by design). To guarantee that all queries are routed to the same node and that the Memory table engine works as expected, you can do one of the following:
- Execute all operations in the same session
- Use a client that uses TCP or the native interface (which enables support for sticky connections) such as clickhouse-client
PREWHERE, including the automatic move of WHERE conditions controlled by the optimize_move_to_prewhere setting: only the columns of the conditions are read at first, and the remaining columns are read only for the blocks where some rows pass, and only for the passing rows. This is especially beneficial together with compress = true, because for a selective condition, most columns are never decompressed. SELECT count() FROM table without a filter is served from metadata without reading the data.
Maximal productivity (over 10 GB/sec) is reached on simple queries, because there is no reading from the disk, decompressing, or deserializing data. (We should note that in many cases, the productivity of the MergeTree engine is almost as high.)
When restarting a server, data disappears from the table and the table becomes empty.
Normally, using this table engine is not justified. However, it can be used for tests, and for tasks where maximum speed is required on a relatively small number of rows (up to approximately 100,000,000).
The Memory engine is used by the system for temporary tables with external query data (see the section “External data for processing a query”), and for implementing GLOBAL IN (see the section “IN operators”).
Upper and lower bounds can be specified to limit Memory engine table size, effectively allowing it to act as a circular buffer (see Engine Parameters).
Engine parameters
min_bytes_to_keep— Minimum bytes to keep when memory table is size-capped.- Default value:
0 - Requires
max_bytes_to_keep
- Default value:
max_bytes_to_keep— Maximum bytes to keep within memory table where oldest rows are deleted on each insertion (i.e circular buffer). Max bytes can exceed the stated limit if the oldest batch of rows to remove falls under themin_bytes_to_keeplimit when adding a large block.- Default value:
0
- Default value:
min_rows_to_keep— Minimum rows to keep when memory table is size-capped.- Default value:
0 - Requires
max_rows_to_keep
- Default value:
max_rows_to_keep— Maximum rows to keep within memory table where oldest rows are deleted on each insertion (i.e circular buffer). Max rows can exceed the stated limit if the oldest batch of rows to remove falls under themin_rows_to_keeplimit when adding a large block.- Default value:
0
- Default value:
compress- Whether to compress data in memory.- Default value:
false
- Default value:
Usage
Initialize settingsbytes and rows capping parameters can be set at the same time, however, the lower bounds of max and min will be adhered to.