Dagster spent the last two days on operator visible GraphQL and snapshot work: bounded run selection queries for large asset jobs, cheaper asset check resolution from execution plan snapshots, and published Dagster+ code location size limits. Eight commits, 30 files, net +1169 lines.
Bounded GraphQL queries for large run selections ¶
Asset jobs that materialize hundreds or thousands of assets produce run records whose assetSelection and assetCheckSelection lists can reach tens of thousands of entries. The Dagster UI and any custom GraphQL client previously had to pull the full lists on every run detail fetch.
The limit args and count fields commit adds optional limit arguments to both selection fields on the Run type, plus non null assetSelectionCount and assetCheckSelectionCount fields that always return the full cardinality. A client can render a bounded preview with accurate totals without transferring the entire set.
Changes land in pipeline.py, schema.graphql, and builders.ts. The resolver sorts keys by path before truncating so preview order stays stable. Omitting limit returns the full list; count fields are additive and backward compatible.
assetSelection(limit: 50) {
path
}
assetSelectionCount
assetCheckSelection(limit: 20) {
assetKey { path }
name
}
assetCheckSelectionCount
test_asset_checks.py covers the new fields. Audit any custom query that pulls full Run selections on wide asset jobs.
Asset checks from execution plan snapshots ¶
Before the run header asset checks commit, the run detail header showed asset pills but often omitted check pills on __ASSET_JOB runs. RunAssetCheckTags skipped its query for hidden asset group jobs and fell back to run.assetCheckSelection, which is None when a run targets every check on selected assets. The Run.assetChecks resolver scanned the full event log for ASSET_CHECK_EVENTS, which is correct but slow on long runs.
The fix reads check keys from the execution plan snapshot instead. execution_plan_snapshot.py now exposes an asset_check_keys property that mirrors how asset_selection is derived from StepOutputProperties.asset_check_key on planned step outputs. fetch_asset_checks.py reads the plan snapshot first and falls back to the event log scan only for runs that predate execution plan snapshots.
On a run detail page, one indexed snapshot fetch replaces a full event log walk. The runs feed is unchanged and still undercounts check pills when assetCheckSelection is None; the new count fields are the likely fix path.
Documented snapshot size ceilings for Dagster+ ¶
The snapshot size limits docs commit documents Dagster+ constraints and ships estimate_snapshot_size.py for local measurement.
Each repository and job snapshot, including the implicit asset job, is capped at roughly 130 MB uncompressed. gRPC defaults at 100 MB often fail first with RESOURCE_EXHAUSTED. The docs mention DAGSTER_GRPC_MAX_RX_BYTES and DAGSTER_GRPC_MAX_SEND_BYTES on agent and code server, but recommend shrinking definitions: upgrade Dagster, trim duplicated metadata, use partitions instead of per entity assets, or split code locations.
GraphQL codegen consolidated behind one script ¶
The codegen standardization commit replaces per package TypeScript generators with scripts/generate_graphql.py, run from root just targets in FE then BE order. OSS codegen still reads the internal schema when present and skips cleanly when it is not. Most of the diff is regenerated JSON; runtime API behavior is unchanged.
Run UI fixes at the edges ¶
The lineage scope fix passes the loaded executionPlan into the run actions menu lineage link. The MiddleTruncate canvas reuse commit reuses one offscreen canvas instead of mounting transient canvases on document.body, cutting roughly 6 seconds of reflow self time on catalog loads.
What to watch ¶
Custom GraphQL clients should adopt limit plus count fields on run selection queries before the UI starts depending on them for wide asset jobs. The runs feed still needs a batched or stored count path for accurate check pills when assetCheckSelection is None; watch for a follow up that wires assetCheckSelectionCount into list fragments.
Dagster+ teams near snapshot limits should pin the estimation script in pre deploy checks. The 130 MB storage cap and 100 MB gRPC default are separate failure modes; measure both.
Codegen contributors should use the root just GraphQL target rather than per package yarn scripts that no longer exist in ui-core/package.json.