Apache NiFi Fixes Stale Prometheus Series And S3 Path Style

Apache NiFi is the JVM flow engine that sits between queues and talks to S3, SMB, email, and the rest of the warehouse. Main took 28 commits in the past week, 175 files, 5073 insertions and 406 deletions. The operator visible work is Prometheus series that no longer linger after an instance ID change, S3 processors that no longer force path style access, and Parameter Context updates that used to fail in a cluster.

The scrape path in StandardNiFiServiceFacade kept JVM, connection analytics, and cluster registries as fields on the facade. Those collectors are labeled with instance. When the instance ID changed, Prometheus kept the old series because the collector objects never went away.

A scrape path change now constructs the three registries inside the scrape method. The next scrape starts empty. The old label set is gone.

JvmMetricsRegistry jvmMetricsRegistry = new JvmMetricsRegistry();
ConnectionAnalyticsMetricsRegistry connectionAnalyticsMetricsRegistry = new ConnectionAnalyticsMetricsRegistry();
ClusterMetricsRegistry clusterMetricsRegistry = new ClusterMetricsRegistry();

The same week removed connected_nodes from cluster_connected_node_count. That label stored a string like 2 / 3. Every join or leave minted a new series and left the previous one sitting in the registry. The gauge now has only instance. cluster_total_node_count already carries the denominator. Dashboards that matched on connected_nodes need a rewrite. Recording rules that parsed the two numbers out of the label will go silent.

SiteToSiteStatusReportingTask also stopped throwing on a null controller status during startup. That is a debug log and an early return, not a metrics change, but it hits the same first scrape after boot window.

AbstractS3Processor used to enable path style access whenever ENDPOINT_OVERRIDE was set, even if USE_PATH_STYLE_ACCESS was false. That is the wrong default for AWS virtual hosted style. It also surprises S3 compatible endpoints that expect DNS style buckets and do not want path style forced on them.

The S3 client builder change keys path style on the property alone. CopyS3Object, DeleteS3Object, FetchS3Object, GetS3ObjectMetadata, GetS3ObjectTags, ListS3, and TagS3Object now expose USE_PATH_STYLE_ACCESS in their property lists so the flag is actually configurable. MinIO and Ceph flows that relied on “override implies path style” will start talking virtual hosted style after upgrade unless that property is set to true.

CopyS3Object also stopped inventing an s3://bucket/key transit URL. Provenance now calls S3Util.getResourceUrl, which uses the SDK getUrl helper. A custom endpoint shows the URL the client would actually hit, not a synthetic s3:// string. Provenance consumers that parse s3:// need to accept HTTPS URLs.

Cluster updates that added provider backed inheritance used to fail. StandardNiFiServiceFacade rebuilt the DTO with inherited parameters marked incorrectly, then the replica request disagreed with the local context. The cluster update path now computes effective updates once, skips locally owned parameters that were not in the request, and sets provided, inherited, and the source context on each entity. Coverage landed in ParameterContextIT and the facade tests.

A second footgun was startup itself. Parameter names that violate the current pattern (letters, numbers, spaces, and .-_) used to abort flow inherit. Nodes with a leftover PARAMETER_{{ ENVIRONMENT }} style name in flow.json.gz would not start. Flow inherit now warns and loads the name so REST can delete it. New names still fail validation on write.

ExtractEmailAttachments now copies MIME part headers onto child FlowFiles as email.attachment.header.<name>. That is a small processor change, but Record and RouteOnAttribute flows will start matching on those attributes as soon as they appear.

NIP-39 will replace per component access policies with controller scoped authorization. FlowController now logs a deprecation warning at flow init when a ManagedAuthorizer still has policies on processors, controller services, ports, funnels, labels, remote groups, Parameter Contexts, or Parameter Providers. Root group policies are ignored. The log line is a count, not a dump of every resource. Grep deprecation logs after upgrade if this cluster grew policies processor by processor.

A separate framework extension adds PropertyEncryptionProvider. The default NAR is PasswordBasedPropertyEncryptionProvider, which still derives the key from nifi.sensitive.props.key. Flow serialization does not switch algorithms. The new property is a hook for a later KMS provider. Leave it at the default unless you are writing that NAR. The administration guide documents the prefix nifi.security.property.encryption.provider.properties. for extra provider config.

nifi.security.property.encryption.provider.implementation=org.apache.nifi.security.encryption.password.PasswordBasedPropertyEncryptionProvider

OpenSAML moved from 5.1.6 to 5.2.3. Spring Security SAML 2 principal handling dropped deprecated APIs. Those are compile and runtime hygiene, not a behavior flip.

Connector verification and purge results in ConnectorResource used to expire in 1 millisecond. They now stay for one minute. UI clients that poll a completed request after kicking off work on a second connector should stop seeing 404s. Repository APIs also started carrying component and Connector context per NIP-40, and FlowController now exposes node connection state. That is plumbing for later isolation work, not a user facing toggle.

Root pom.xml bumped SLF4J to 2.0.19, Jetty to 12.1.13, Parquet to 1.18.1, plus the MySQL binlog connector, Elasticsearch client, Snowflake JDBC, and AWS SDK patch lines. Rebuild custom NARs against the new BOM before dropping them on a node.

SmbjClientProviderService now accepts a KerberosUserService. SMB flows that still put username and password on the processor should move credentials into the controller service while that path is still supported. Processor authors also got FieldValidator and RecordValidator on the Record API. Schema validation can now run custom field checks without forking StandardSchemaValidator.