goai v0.9.9 - Streaming Timeouts And Capability Detection

goai v0.9.9 was published on 30 August 2026. Anthropic DoGenerate now streams long running thinking requests instead of holding a silent HTTP POST open until the model finishes. Those calls were dying on idle timeouts at proxies and load balancers, not on API errors.

The full release notes and downloads are on the GitHub release page.

A non streaming Messages call holds the HTTP response open with no bytes on the wire until the model is done. A request that thinks for minutes hits idle timeout enforcement on the client, a proxy, or a load balancer. The POST dies.

The failure showed up with native structured output, tools, and a thinking model together. GenerateObject is the structured output entry point that runs a tool loop. StreamObject is single step by design. Every GenerateObject step went through DoGenerate, so that mix was reachable only on the non streaming transport. A thinking model ran for twenty minutes across twenty tool calls and never returned. The request died on a stalled POST.

#165 changes the transport. DoGenerate now sends stream:true and rebuilds the Message document from SSE events when the request is expected to think. That includes a thinking or effort provider option, or a model from the 5.x generation, which thinks with no thinking parameter on the wire. Callers can force either transport with ProviderOptions["streamingTransport"]. That key is SDK internal and never sent on the wire.

Reassembly goes through parseResponse rather than mapping events onto provider.GenerateResult. provider.StreamChunk has no fields for thinking block signatures, which must be replayed on the next turn. It also has no fields for redacted_thinking payloads, server tool result blocks, citations, or context_management metadata. Truncated streams now fail as protocol errors. accumulateStreamedMessage requires message_start and message_stop and rejects premature EOF. Bedrock event:error payloads are consumed via NextEvent(), so AWS exceptions are not reported as partial success. Hostile content block indexes are rejected instead of forcing unbounded allocation.

Auto streaming is provider aware. WithAutoStreaming is on by default for the direct Anthropic API and off for Bedrock. Bedrock’s InvokeModelWithResponseStream needs a different IAM permission than InvokeModel. doHTTP already derives streaming from body["stream"], so Bedrock and Vertex URL builders pick the streaming endpoint with no extra change.

supportsThinking and supportsNativeOutputFormat used to match hardcoded model id substrings. Those lists went stale. Current models were reported as lacking capabilities they have. supportsThinking feeds Capabilities().Reasoning, so callers that gate on reported capabilities skipped thinking. supportsNativeOutputFormat decides structuredOutputMode: "auto". When it returned false, auto fell back to the tool trick path, which forces a synthetic json_response tool.

#164 replaces those lists with an anthropicModelVersion helper that parses major and minor generation out of the id. Thinking is Claude 4 and later, plus legacy claude-3-7-sonnet. Native structured output uses an explicit documented compatibility list, not a 4.1 and later guess. claude-opus-4-1 is unsupported. The non versioned claude-mythos-preview is supported. The parser is unanchored, so Bedrock ids such as us.anthropic.claude-sonnet-4-6 still match, as do Vertex ids with an @date suffix. A trailing numeric run is a minor version only when it is one or two digits. claude-sonnet-4-20250514 is 4.0 with a release date, not minor 20250514.

Platform adapters select their own set through WithNativeOutputFormatSupport. Bedrock uses a narrower documented subset. MiniMax opts out. Vertex and Azure keep the direct set. Native output_config.format is not a superset of the tool trick, so the sanitizer is now a type aware transform. It converts oneOf to anyOf, adds additionalProperties:false on objects, and records unsupported constraints in the description instead of dropping them silently. An invalid response format schema now returns an error instead of falling back to the tool trick.

#161 is a memory fix, not a protocol change. Stream goroutines captured resp, which kept the whole *http.Response graph reachable for the life of the stream, including resp.Request and the serialized request body. The fix extracts resp.Body into a local variable before the goroutine starts. The rest of the graph can become GC eligible while the stream is still active.

The same pattern is used in the six providers that share this stream shape. Vertex’s DoStream also closed the response body twice, once from the cancellation goroutine and once from the stream defer, without sync.Once. It now uses the same closeBody and sync.Once pattern as the other five. Concurrent cancellation can no longer double close the body. Tests include TestChat_Stream_DoesNotRetainRequestGraph.

Long streams no longer pin the original request bytes in heap for the whole read.

No breaking API is listed in the v0.9.9 notes. Two defaults still matter for operators.

Bedrock deployments that only grant bedrock:InvokeModel should leave auto streaming off, which is already the Bedrock default. Turning it on requires InvokeModelWithResponseStream. Direct Anthropic API callers get auto streaming for thinking work without a config change. Override either way with ProviderOptions["streamingTransport"] or WithAutoStreaming. Non boolean values are rejected.

Callers on structuredOutputMode: "auto" who relied on the tool trick for models that now advertise native output will hit the native schema transform. Constraints the native path does not support move into the schema description. Callers who need the original validation should stay on the tool trick path explicitly.

The full changelog covers v0.9.8...v0.9.9.