Apache Beam is the unified batch and streaming SDK that runs on Flink, Spark and Dataflow. The master branch saw 84 commits in the last week, and several of them change runtime behavior: Flink batch serialization, RunInference batching and a few bugs that lost data without an error.
The repository is apache/beam. The window touched 334 files, but much of that is dependency churn in sdks/go.mod and sdks/go.sum plus a busy CHANGES.md. The interesting part is smaller.
Flink batch jobs stop decoding records they only move ¶
The biggest operator visible change is the Flink batch copy work. Flink’s sorter copies records with TypeSerializer#copy when it spills. Beam coders do not delimit their output, so CoderTypeSerializer had to decode and re encode each record. The author profiled a Scio batch job and attributed about half of task CPU to that.
The commit adds two batch only options, both on by default:
--lengthPrefixBatchRecordswrites the encoded length before each record, socopy()becomes a byte copy.--lazyBatchGroupingInputskeeps the KV encoded in aLazyKvWindowedValueuntilgetValue()is called. The key selector reads the key from raw bytes.
A third change removes per byte locking. ByteArrayInputStream has synchronized reads, and Avro’s unbuffered decoder calls read() once per byte. The fix uses Flink’s ByteArrayInputStreamWithPos. On a job decoding about 12.7 GB of Avro per subtask, the median subtask went from 336 s to 199 s, and stragglers up to 4x the median disappeared.
Those numbers come from one profiled job, so treat them as a ceiling for your own gains. Streaming jobs and state and timer serializers are untouched, so savepoints are not affected. Defaults flipping on means a batch spill format change. If you hit a problem, both flags accept =false.
--runner=FlinkRunner --lengthPrefixBatchRecords=false --lazyBatchGroupingInputs=false
RunInference learns about model servers ¶
Three commits from the same contributor reshape how RunInference behaves with LLM servers. Prefix aware batching adds key_fn and num_shards to BatchElements. The problem is plain: batching per worker mixes unrelated prompt prefixes, which thrashes the KV cache of a server such as vLLM. With a key, elements that share a prefix route to the same worker.
Review feedback changed the design along the way. Keyed batching now reuses the stateful BatchElements DoFns instead of GroupIntoBatches, so adaptive batch sizing still works. Combining key_fn with length_fn is rejected, because both pick the state key. The default of 8 shards per prefix exists because one shard serves one worker at a time, so a hot prefix would cap at one worker’s throughput.
A second commit adds a capacity based autoscaling signal. Dataflow GPU autoscaling reads num_loaded_models_<tag>. An out of process server always has one loaded copy, so the handler needs another capacity estimate. The new module scrapes a Prometheus completed counter and an in flight gauge, then publishes per slot throughput times p90 concurrency over a trailing window. The commit history shows the API being cut down from five classes to three. Expect more movement before it settles.
A smaller change reports the model identifier to handlers, which the metric tag resolution uses.
Silent data loss fixes in state and batching ¶
Two fixes deserve a check against your pipelines.
The State iterable fix is in CachingStateIterable. When the cache was only partly loaded, remove() and appendHelper() evicted the cache entry but then fell through to block combination code. That stored a mutated block with a nextToken, and later iterations returned truncated data. The fix is missing return statements. It is a two line change for a bug with no error signal.
The GroupIntoBatches fixes correct the watermark hold update for out of order timestamps. The commit message shows the scope was trimmed in review: changes to timer state persistence were reverted, and a non positive batchSize stays allowed for compatibility. Streaming users who saw late data drops downstream of batching should read the test changes.
The Solace write fix is a related shape of bug. Writer DoFns on a worker share pooled sessions, and tearing down one DoFn after an aborted bundle closed sessions for the others mid bundle, losing in flight ACKs. Sessions now stay open until a JVM shutdown hook.
Beam YAML and pyarrow fixes ¶
Two Python fixes landed on the same day. Beam YAML generic filters only accepted < and >. The cause was a greedy regex in yaml_mapping.py, which split a >= 1 into a > and =. Making the left side lazy fixed it, with tests for every operator.
The pyarrow rebatching fix repairs batched DoFns that rebatch into a pyarrow.Table with more than one column. The symptom was ArrowInvalid: Column 1 ... expected length N but got length 0, for example ReadFromCsv straight into WriteToParquet.
What to watch ¶
- Benchmark Flink batch jobs before and after. The new defaults change spill format, and the gain depends on your coders. Avro heavy jobs should benefit most.
- The RunInference autoscaling and batching APIs are new and still being reshaped. Pin your Beam version and read
CHANGES.mdbefore building on them. - If you use stateful DoFns with large iterables or
GroupIntoBatches, add a count check on output after you pick up these fixes. The old behavior was wrong without being loud.