ingestr v1.0.65 - Arrow Offset Fixes And OneLake Paths

ingestr v1.0.65 was published on July 6, 2026. The most important item is the databuffer fix that honors Arrow array offsets during casts, because sliced arrays are a data correctness problem before they are a speed problem. The release also tightens OneLake path handling and adds CDC metadata markers for jobs that need reliable resume behavior.

The full release notes and downloads are on the GitHub release page. The release is not marked as a prerelease.

The clearest operator visible change is in databuffer: Arrow array offsets are now honored during casts. That reads like a small internal fix, but it sits on a sensitive boundary. Arrow arrays can represent a logical slice over a larger backing buffer. If casting code ignores that offset, the code can read the wrong logical window.

For pipeline owners, this is a correctness fix before it is anything else. It matters when a source or transform hands ingestr a sliced Arrow array and the destination path expects the cast output to preserve row position. The release notes do not claim a broad performance change, so the right reading is narrower: less risk around sliced Arrow data passing through the buffer layer.

This is the kind of patch that should get attention in jobs that move typed data between systems with different column rules. Numeric casts, timestamp casts, and nullable column handling tend to hide bad assumptions until a specific batch shape appears. The changelog does not list a migration step, but it does point at a data path bug worth upgrading for.

The OneLake changes are about path discipline. The release keeps area directory materialization and skips managed lakehouse path prefixes. That is storage layout work, not a user interface change.

For platform teams, the practical detail is that ingestr is drawing a clearer line between the directories it should materialize and path prefixes that are managed by the lakehouse side. That reduces ambiguity in jobs that write into OneLake backed layouts. It also makes failures easier to reason about, because a bad path decision is often visible only after a run has written to a location that another service treats as reserved.

The release notes do not describe a schema migration, a destination rename, or a new config key for OneLake. Treat this as a safer path handling patch. If you run ingestr against OneLake destinations, the area directory and managed prefix behavior is the part to validate in staging.

The changelog includes CDC metadata work for BigQuery and an annotation on CDC resume cursor lookup with ingestr_step. That is useful plumbing for operators who need to explain where a resumed load picked up work after an interruption.

Resume cursor behavior is one of the places where ETL tools either help during incident review or waste time. The release notes do not say that resume semantics changed. They do say the lookup is now tagged with ingestr_step, which should make the surrounding metadata easier to follow when a run is split across steps.

The BigQuery CDC metadata item sits in the same category. It is not a new connector announcement in these notes. It is a metadata improvement around a data movement path that already needs clear audit signals. For teams that treat CDC state as part of operational evidence, this is the relevant change.

Several entries are maintainer facing. The new add-source agent skill is for implementing new ingestr sources. There is also registration work around register.go, per source readParallel, and shared query parsing through pkg/tablespec.

That does not mean these changes are irrelevant to users. Source registration rules and table parameter parsing decide how new connectors become predictable parts of the tool. The useful signal is that ingestr is centralizing more of that behavior instead of letting each source invent its own parsing and parallel read conventions.

For operators, this is background work unless you maintain a fork or build custom sources. For maintainers, it is more direct: new source work should follow the shared table spec path and the registration pattern named in the changelog.