ingestr v1.1.69 - ClickHouse Type Fixes, Durable CDC Refresh and BigQuery Warnings


bruin-data published ingestr v1.1.69 on October 19, 2026. The release is a batch of correctness fixes for loaders, plus a CDC change that supports durable full refresh during streaming.

The full release notes and downloads are on the GitHub release page. Nothing here is marked breaking, and this is a stable release, not a prerelease. Operators running mixed source and destination fleets will find the changes grouped below.

Three pull requests in this release target ClickHouse table behavior:

  • PR #1311 adds scd2 support on replicated cluster tables. Teams running scd2 ingestion on Replicated merge tree tables previously had no supported path here.
  • PR #1313 stops ingestr from using ReplacingMergeTree for tables without primary keys. Without a primary key, a ReplacingMergeTree table has no meaningful dedup key, so the table choice is now explicit.
  • PR #1320 changes array creation from arrays of nullable elements to Array(Nullable(...)). That keeps the array itself non nullable, which is what most downstream queries expect.

On the DuckDB side, PR #1312 creates append tables without a primary key constraint. Append mode should not require uniqueness, and imposing the constraint at write time could block loads the source data cannot satisfy.

Two BigQuery fixes and one Iceberg fix land together.

PR #1317 fixes reads of arrays and unparameterized NUMERIC values from BigQuery sources. Columns defined without explicit precision were producing wrong values on read.

PR #1319 fixes replace and merge operations on Iceberg tables that contain array columns. PR #1321 adds a warning when someone applies partition_by or cluster_by against an existing BigQuery table that already has different partitioning. A warning is the right call there: the clustering change is a table wide rebuild and the operator should see it coming.

PR #1322 adds durable full refresh during streaming. When a full refresh is requested while a stream is running, the checkpoint survives instead of being dropped.

PR #1316 skips missing interval bounds in incremental reads on Databricks. Sources do not always emit a clean first and last interval marker, and the older read path treated that as an error.

PR #1226 changes the multitable router so terminal errors reach the consumer instead of being swallowed. If a route fails permanently, the job now surfaces that exact error rather than a generic wrapper. That matters for people wiring ingestr into alerting, since the alert body is now the real failure.

PR #1318 writes NaN and Inf floats as nulls for CleverTap and HubSpot destinations. Both sources produce non finite floats in practice, and several columnar destinations reject them outright.

The notes list no breaking changes and no migration steps. The ClickHouse array change in #1320 alters newly created tables only. Existing tables keep the old layout, so do not expect reads of older tables to change shape.

The full changelog diff from the previous tag is at the release page.