Argo Workflows v4.1.3 - Daemon Status and Empty OTLP Endpoints

Argo Workflows v4.1.3 was published on 11 September 2026 as a patch on the 4.1 line. Controller, telemetry, and archive fixes ship with security bumps golang.org/x/crypto v0.56.0, google.golang.org/grpc v1.83.2, and qs v6.16.0. On a succeeded workflow, assessNodeStatus keeps NodeSucceeded after the controller tears a daemon pod down.

The full release notes and downloads are on the GitHub release page. That page is the standard install blurb. The selected commit list is in the changelog, and the notes also point at the project blog. Every functional fix in that list is a cherry pick onto the 4.1 line.

killDaemonedChildren sets the node phase to NodeSucceeded before it signals the pod. The later pod event confirms that teardown. On the v4 line, assessNodeStatus no longer forces a daemon PodFailed straight to NodeSucceeded. That v3 guard was removed when daemon retry support landed, and inferFailedReason treats the main container exit as a real failure. Daemon argoexec exits 143 on SIGTERM, so a succeeded workflow still showed NodeFailed. preStop hooks did not stop the rewrite.

v4.1.3 checks the stored phase in the PodFailed branch. If it is already NodeSucceeded, the controller keeps it and skips inferFailedReason. A later PodSucceeded on that daemon is preserved the same way. Any exit code is ignored once the stored phase is NodeSucceeded. A daemon that fails before teardown is still a failure.

inferFailedReason in workflow/controller/operator.go failed the node on a nonzero artifact plugin sidecar exit, other than 137 or 143. The wait container, or the supervisor container, saves outputs and then stops the sidecar with a file signal. terminateContainers can also run argoexec kill 15 1 in the same window. A kill exec that races PID 1 can record a phantom nonzero status for a clean exit. The kubelet then reports PodFailed.

common.IsArtifactPluginSidecar now skips that container. The log line is ignoring artifact plugin sidecar exit code, with fields exitCode and containerName. Main, wait, and supervisor still decide the phase. A wait exit of 1 stays NodeError (wait: Error (exit code 1)). A main exit of 1 stays NodeFailed. Other user sidecars still fail the node.

NewMetrics and NewTracing used os.LookupEnv on OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_METRICS_ENDPOINT, and OTEL_EXPORTER_OTLP_TRACES_ENDPOINT. An empty string still enabled the exporter, against docs/telemetry-configuration.md. resolveOTLPEndpoint in util/telemetry/otlp.go trims with strings.TrimSpace, prefers the signal specific variable, and falls back to the common endpoint. An empty or whitespace only result leaves the exporter down.

workflowsResource in util/telemetry/resource.go applied resource.WithAttributes after resource.WithFromEnv, so static attributes overwrote OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES. Static attributes now come first. A set OTEL_SERVICE_NAME shows up on controller and server resources. A defined but blank OTLP variable no longer dials an empty address.

In memory locks in workflow/sync/semaphore.go print available slots over the limit, so a held mutex is Lock status: 0/1. workflow/sync/database_semaphore.go printed holders over the limit, so the same mutex was 1/1. One workflow could show both strings while both locks were free. The database message now uses limit minus holders, so one holder is Lock status: 0/1 on Postgres and on MySQL.

SynchronizationStatus.GetStatus returned a nil *SemaphoreStatus or *MutexStatus inside an interface. That value is not equal to nil, so Manager.Release in workflow/sync/sync_manager.go called LockReleased and panicked when Status.Synchronization was set but never initialized. GetStatus now returns an untyped nil for an unset pointer.

newInternalSemaphore and newDatabaseSemaphore treated getLimit() of 0 as failed to initialize semaphore. A failed fetch also returns 0, so a closed gate (limit 0 until an operator raises it) could not be built. Both constructors call limitGetter.get and fail only on error. A later acquire still fails when no slot is free.

GetNodeTemplate calls GetStoredTemplate on wf.Status.StoredTemplates before the live ClusterWorkflowTemplate informer, matching resolveTemplateImpl. Without that lookup, a templateRef into another ClusterWorkflowTemplate became Error with template <name> not found in workflow template <cwt-name> after a live edit, even though the pod had run on the stored copy. templateName nodes already used storedWorkflowSpec.

ParseProgress panicked on a value with no slash, such as 5 (index out of range [1] with length 1). Call sites are the progress file in workflow/executor/executor.go and a pod annotation in workflow/controller/workflowpod.go. parts() pads a missing denominator, and ParseProgress("5") returns not ok.

postgresql.azureToken and postgresql.awsRDSToken built a DSN via ConnectionURL, which appended default_query_exec_mode=cache_describe. lib/pq forwards that unknown keyword, and the server returns SQLSTATE 42704 before authentication. The password path uses url.URL and was fine. Token DSNs now emit only lib/pq keywords and share sslmode with that path.

The GitHub release page points at the upgrading guide and at open issues for breaking changes and known issues. Blank OTLP endpoint variables leave the exporter down. Successful daemon teardown stays NodeSucceeded. Database Lock status: lines count available slots over the limit. A semaphore limit of 0 initializes, then refuses acquires.

Controller and server installs remain the manifest published with the tag:

kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v4.1.3/install.yaml

The Mac and Linux amd64 CLI archive is argo-$ARGO_OS-amd64.gz. The install notes end with chmod +x, a move to /usr/local/bin/argo, and argo version.