Airbyte Discover Skips Broken Snowflake Objects

Airbyte, the open source ELT project, put 300 commits on master in the last seven days. Most of them are docs, image bumps, and CI. The sync behavior that changed is narrower: Snowflake discover can skip a broken view, Shopify bulk jobs fail when Shopify returns nothing to read, and Amazon Seller Partner 6.0.0 stops using amazon-order-id as a primary key on two flat file order reports.

source-snowflake 2.0.1 changes discover only. Discover now skips a broken object instead of failing the catalog. CHECK and READ still raise. Tolerance is on only when the CDK operation name is discover.

SnowflakeSourceMetadataQuerier probes with LIMIT 0 when privilege checks are on. The probe is swallowed only for a known object failure:

private val TOLERABLE_OBJECT_ERROR_CODES = setOf(2003, 2043, 2037, 2057)
private val TOLERABLE_OBJECT_SQLSTATES = setOf("42501", "42601", "02000")

2003 and 2043 mean the object is missing or unauthorized. 2037 is a view that fails to expand, including an unset session variable. 2057 is a column count mismatch. 42501, 42601, and 02000 cover privileges, compilation, and an object the user cannot see.

A whole object failure (SQLSTATE 42601 or one of those four codes) drops the stream. Other tolerated errors still probe column by column. The skip is only a debug log.

One bad view no longer blocks the rest of the schema. It can also vanish from the catalog until debug logging is on. Selecting it later still fails on code 2037.

source-shopify 4.1.1 refuses a bulk export that finished without a file. Jobs with no result URL now fail. The bump is recorded in the Shopify source docs.

Checkpointed streams still resume a FAILED job from partialDataUrl when that file exists and has records. If the partial file is missing, or it has no cursor to advance, ShopifyBulkManager raises BulkJobFailed. A new EXPIRED status takes the same path when the result URL is gone. ACCESS_DENIED on a failed payload still goes to the scope handler. BulkJobFailed is a system_error, and the message (job id, state, errorCode, objectCount) is shown to the user.

The retry decorator in the same commit was wrong. except BULK_RETRY_ERRORS or more_exceptions never added more_exceptions, because a non empty tuple is true in Python. The clause now concatenates the tuples. A partial file that still has a cursor keeps resuming. A job with nothing to resume from fails instead of looking finished.

The flat file All Orders streams no longer declare a primary key. source-amazon-seller-partner goes from 5.10.5 to 6.0.0. The source docs and docs/integrations/sources/amazon-seller-partner-migrations.md are the upgrade notes.

GET_FLAT_FILE_ALL_ORDERS_DATA_BY_ORDER_DATE_GENERAL and GET_FLAT_FILE_ALL_ORDERS_DATA_BY_LAST_UPDATE_GENERAL emit one row per order item. The old key was amazon-order-id. Incremental | Append + Deduped collapsed every line of a multi item order into one row. No column set has been shown to be unique. Version 5.10.4 added order-item-id to the schema. 6.0.0 still sets primary_key to an empty list, so deduped incremental is no longer offered.

Refresh the schema. If either stream was deduped, the destination is already missing line items. Switch both to Incremental | Append or Full Refresh | Overwrite, clear them, and sync again. Clearing deletes destination data. Amazon only serves orders less than two years old, and the connector starts at the replication start date, so older rows cannot be rebuilt. Snapshot first if you need that history.

Append and overwrite streams do not need a clear. After a reload, a row is an item. Counts need COUNT(DISTINCT "amazon-order-id"). A lookback window must be deduped downstream on the full row or on _airbyte_extracted_at.

Destination Postgres 3.0.22 strips a leading plus before COPY. Date and timestamp values lose that prefix, so +10000-01-02 is written as 10000-01-02. A negative year such as -0001-01-02 keeps the minus. Rows already stored are unchanged.

Extract CDK 1.1.12 null guards JDBC getDate and getTimestamp. MySQL zero dates with zeroDateTimeBehavior=convertToNull can make getString return text while the typed getter returns null. The accessor used to throw. It now returns null, so a zero date and a SQL null look the same.

source-uptick 1.3.0 adds the knobs operators will actually set. Error classes, a request budget, and concurrency move the manifest from CDK 6.60.5 to 7.30.1. metadata.yaml limits allowedHosts to ${base_url}. A bad password (HTTP 400 invalid_grant) or a bad client secret (HTTP 401 invalid_client) fails the token call as config_error. A 401 on a stream refreshes once, so a rotated password does not loop logins. HTTP 403 is a permission error. Retry-After caps at 1800 seconds. The budget defaults to 60 requests per minute, and workers default to 3 with a cap of 10. The base URL is forced to https plus the host, and a path on it is dropped.

source-greenhouse 1.3.0 adds Client Credentials for a Harvest v3 custom integration. That path is what a self managed install can use. Partner OAuth stays on Airbyte Cloud. The Greenhouse docs limit the 24 hour refresh token warning to OAuth. Leave the Site Admin user id blank to use the integration service user.

source-gitlab 4.4.41 reads each project twice. Child pipelines and pipeline_trigger_jobs come from a second call with source=parent_pipeline. The GitLab docs say the old per project cursor is kept and children backfill from the start date. Above about 5,000 projects the connector keeps one shared cursor, and older child pipelines are skipped until you reset pipelines, pipelines_extended, and jobs.

source-quickbooks 4.2.0 was pinned off. Cloud and OSS no longer force image 4.0.4, and metadata.yaml turns on an autopilot rollout with strategy fast. A workspace can still be on 4.0.4 while that runs. 4.2.0 is the certified manifest. Read the token rotation and cursor notes before you assume the running image changed.

Snapshot the two Amazon order tables before a clear if they used deduped incremental. Rows older than the replication start date, or older than the two year window Amazon still serves, will not reload.

GitLab connections past about 5,000 projects need one reset of pipelines, pipelines_extended, and jobs after 4.4.41. Otherwise historical child pipelines stay behind the shared cursor.

On October 1, 2026, Outreach drops contactHistogram from the default prospects payload. New values in that column are null. The primary key and cursor stay, per the Outreach docs. On October 13, 2026, Microsoft retires general purpose v1 and legacy Blob Storage accounts. The docs warning says names and data survive a move to general purpose v2, but access can blip and billing can change.