Apache Beam v2.75.0 - Dataflow And IO Release Notes

Apache Beam v2.75.0 was published on July 8, 2026, with the main operator change around Dataflow Streaming Engine state tag encoding v2 becoming the default for new Java jobs. The release also adds Delta Lake reads, expands YAML output targets, and changes Flink runner support in ways that matter for batch and streaming pipeline owners.

The full release notes and downloads are on the GitHub release page.

Dataflow Runner v2 is now named Dataflow Portable Runner in the Beam 2.75.0 notes. That is mostly a naming change, but it matters for teams that keep runbooks, pipeline templates, or internal platform docs tied to the old runner name. The release points readers to the Dataflow public documentation for when Portable Runner should be enabled.

The larger change is state tag encoding v2. Beam now enables it by default for new Java Dataflow Streaming Engine jobs. Operators can disable it with --experiments=disable_streaming_engine_state_tag_encoding_v2, or set --updateCompatibilityVersion=2.74.0 as a pipeline option.

The important constraint is update compatibility. The tag encoding version cannot change during a job update. A new job started on Beam 2.75.0 with tag encoding v2 cannot be downgraded below Beam 2.73.0, because older releases do not support the encoding. That should be treated as a rollout boundary for long running streaming jobs, not just as a normal dependency bump.

Beam also adds FnApi data stream protocol support that lets runners isolate bundles that process input slowly from other bundles. The notes do not claim a specific runner outcome, but the direction is clear: slow bundle handling is moving closer to the protocol layer, where runner implementations can make better isolation decisions.

Java users get Delta Lake read support in this release. That is the most direct data lake change in Beam 2.75.0, and it gives Beam another path into table data that often sits between Spark, Flink, and warehouse jobs.

ClickHouseIO now writes DateTime64(precision[, 'timezone']) columns with sub second precision. That is a narrow fix, but it matters when Beam is used as an ingestion path for event data where timestamp truncation changes ordering, deduplication, or later window analysis.

IcebergIO also gets a bug fix for manifest column bounds. Beam had been writing bounds padded with trailing 0x00 bytes, which could break equality predicate pushdown in some query engines. This is the kind of storage bug that does not always fail a pipeline, but can move cost and latency into downstream reads.

Python gets MqttIO through cross language support. That is useful for edge and telemetry ingestion patterns where MQTT is the source protocol but the rest of the pipeline remains in Beam.

Beam 2.75.0 adds Python SDK memory profiling with Memray and instrumentation for off the shelf profiling agents when launching the Python SDK Harness. For production Python pipelines, that is more useful than another generic logging knob. Memory growth in user code, coders, or batched transforms is easier to chase when profiling starts where the harness starts.

The release also adds a Qdrant VectorDatabaseWriteConfig implementation. That puts vector database writes into the supported Beam surface, which is relevant for embedding pipelines that need repeatable batch loads rather than one off scripts.

YAML users get several new pipeline building blocks. Beam switches the JavaScript execution library from js2py to Quickjs, adds HuggingFaceModelHandler, and adds WriteToMongoDB and WriteToDatadog transforms. These are not just syntax changes. They expand what can be described in Beam YAML without dropping into SDK code for every sink or model handler.

There is one explicit breaking change for Python. Type hints on dataclass fields are now honored during type inference. If a pipeline depended on fallback to any behavior, the compatibility option is --exclude_infer_dataclass_field_type. The release notes recommend fixing forward instead, which is the better long term answer if those dataclasses describe real records.

Flink support also changes. Beam adds support for Flink 2.1 and 2.2, while dropping support for Flink 1.17 and 1.18. That is a runtime matrix change, so teams using the Flink runner should check both Beam and cluster versions before the upgrade.

There is a known Gradle capability conflict when using the Flink runner with Flink 2.1 or later beside libraries that require org.lz4:lz4-java, such as Kafka clients. Flink 2.1 and later ship at.yawk.lz4:lz4-java, which declares the same capability. The documented fix is a capabilitiesResolution rule in build.gradle that selects at.yawk.lz4:lz4-java.

The release page is the source of record for this update: Apache Beam v2.75.0 on GitHub.