Note that column addition will be propagated at the end of a batch’s sync, which could occur after the sync interval or pull batch size is reached. More information on controlling syncs here
Default values for added columns
For an added column, ClickPipes propagates a default only when it can translate the value safely to ClickHouse. When a default is propagated, ClickHouse uses it when reading data parts that predate the new column. As a result, rows already present in ClickHouse show the same value that PostgreSQL uses for rows that predateADD COLUMN; a full table refresh is not required.
For PostgreSQL, ClickPipes reads the value PostgreSQL stored for pre-existing rows, rather than the column’s current default expression. This matters when the default is an expression: a non-volatile expression such as now() is evaluated by PostgreSQL when the column is added, and ClickPipes propagates that resulting value, not the expression. Later SET DEFAULT or DROP DEFAULT operations do not change the default used for existing rows and are not propagated to ClickHouse.
The following values are propagated:
- Boolean and finite numeric values.
- Text values, including
varchar,char, enums, UUIDs, JSON/JSONB,hstore, network addresses, and dates. - Timestamps. Timestamps with time zones are normalized to UTC.
NULL, non-finite numeric values, bytea, arrays, intervals, time-of-day values, and geometric types. The column is still added, but ClickHouse uses its normal type default for parts that predate the column. A full table refresh is required if those existing rows must contain the source default.