ingestr v1.0.71 - Iceberg CDC And Sized String Metadata

ingestr v1.0.71 was published on 2026 07 08 as a regular GitHub release for bruin-data/ingestr. The main operator visible change is support for Iceberg incremental strategies, backed by fixes around Postgres CDC, MySQL destination loading, and richer column metadata.

The full release notes and downloads are on the GitHub release page.

The headline change is Add Iceberg incremental strategies, recorded against pull request #912 in the release notes. For ELT work, this matters because Iceberg tables are usually interesting when they are not treated as flat append dumps. Incremental movement is where partition layout, snapshot state, and repeatable sync behavior start to matter.

The notes do not spell out every strategy detail, so this is not a claim about a specific watermark design. The useful signal is narrower. ingestr now has explicit release work for Iceberg incremental paths, which should reduce the need to treat Iceberg as a special destination outside the normal pipeline run.

That also changes how operators should test upgrades. A pipeline that writes simple full refresh data may not notice much. A pipeline that expects repeatable incremental behavior into Iceberg deserves a closer staging run, especially around retry behavior and table state after partial failure.

Postgres CDC gets several changes in this release. The notes call out hardening under sustained load, handling nil schema for late CDC tables, and support for new tables in Postgres CDC. That cluster of changes is more important than any single line item.

CDC systems fail in boring ways. A new table appears after replication has started. Schema information is not ready when an event arrives. Load increases and queue timing changes expose assumptions that were invisible in short tests. The 529cdf58fdd7519ba7295f5f4327174819a8a8f0 and a6b9528524f53747e161002e0a585777215bb7ed entries point at exactly that class of issue.

For platform teams, this release is worth checking if ingestr is used against Postgres sources with active schema growth. The practical test is simple: create a table after CDC is running, write rows into it, and verify that the destination receives data without a manual restart. Also test sustained write volume, because the release notes explicitly name that condition.

v1.0.71 spends a lot of work on sized string handling. The notes mention sized string types in --columns overrides, schema evolution that can widen string column length, and string length capture on reads for Trino, Databricks, and CrateDB. Snowflake read paths now preserve string column length too.

This is useful because generic string handling can hide real schema intent. A source column with a bounded VARCHAR is not the same operational object as an unbounded string field. Size limits affect BigQuery table schemas, Databricks and MaxCompute behavior, and later migrations when a field grows.

The BigQuery fixes are specific. MaxLength is now applied to STRING columns in table schema and when adding string columns through schema evolution. The column override parser also rejects nonpositive sized string lengths and rejects sized strings with extra parameters. That is better than accepting a shape that later fails inside a destination engine.

There is one caveat in the same batch. Athena does not claim sized string support because Iceberg has none there, according to the release note entry. Databricks and MaxCompute also had work around oversized VARCHAR length, ending with a fallback to unbounded string for oversized length. That is the kind of behavior to document in pipeline contracts if downstream consumers care about exact column size.

MySQL destination loading is optimized in cf471d04e8268f86310cf5376b8ce0ddfda350dc. The notes do not include benchmark data, so treat this as a release signal rather than a throughput promise. If MySQL is a target in a busy ingestion path, compare row counts, load duration, and lock behavior before and after the upgrade.

Snowflake decimal metadata is also preserved in this release. That is a small line item with a large blast radius when financial, billing, or metric data is involved. Losing decimal precision or scale often shows up late, after data has already been queried, exported, or used in reconciliation.

The release also includes documentation and integration test updates for sized string type names. That suggests the schema behavior was not only added in code paths but also covered as a user visible contract. It is still worth reading the GitHub release page before updating any job that depends on destination type mapping.