NIFI-15911 Add proxy support to StandardSnowflakeIngestManagerProviderService#11314
NIFI-15911 Add proxy support to StandardSnowflakeIngestManagerProviderService#11314rakesh-rsky wants to merge 1 commit into
Conversation
| SCHEMA, | ||
| PIPE | ||
| PIPE, | ||
| ProxyConfigurationService.PROXY_CONFIGURATION_SERVICE |
There was a problem hiding this comment.
Should we use ProxyConfiguration.createProxyConfigPropertyDescriptor(...) here and add a customValidate that calls ProxyConfiguration.validateProxySpec(...), like other proxy aware components, so an unsupported proxy type is rejected at configuration time instead of failing silently at runtime?
| return null; | ||
| } | ||
| final Proxy proxy = proxyConfigurationService.getConfiguration().createProxy(); | ||
| // Use a custom ProxySelector so that the Proxy.Type (HTTP or SOCKS) is preserved, |
There was a problem hiding this comment.
The JDK java.net.http.HttpClient only applies HTTP proxies and ignores a SOCKS Proxy, so a configured SOCKS proxy would silently fall back to a direct connection. Should we drop the SOCKS handling claim or validate that only HTTP proxy types are configured?
| if (proxyConfigurationService == null) { | ||
| return null; | ||
| } | ||
| final Proxy proxy = proxyConfigurationService.getConfiguration().createProxy(); |
There was a problem hiding this comment.
Could we use ProxyConfiguration.getConfiguration(context), which returns DIRECT when the service is unset, and ProxyConfiguration.hasCredential() to simplify this code and remove the two null returning helper methods?
| }; | ||
| } | ||
|
|
||
| private Authenticator buildProxyAuthenticator(final ProxyConfigurationService proxyConfigurationService) { |
There was a problem hiding this comment.
The Snowpipe endpoint is HTTPS, and the JDK disables Basic proxy authentication over CONNECT tunneling by default via jdk.http.auth.tunneling.disabledSchemes. Was the username and password path tested against a real authenticated HTTPS proxy?
There was a problem hiding this comment.
You're right. The Authenticator is the correct Java API for proxy credentials, but there's a JDK limitation — Basic authentication over CONNECT tunneling is disabled by default via jdk.http.auth.tunneling.disabledSchemes=Basic. There's no way to override this per HttpClient instance since it's a JVM-wide setting. So authenticated proxy won't work out of the box for HTTPS targets like Snowflake. To handle this, I've added a note directly in the property description telling users to add -Djdk.http.auth.tunneling.disabledSchemes="" to their bootstrap.conf if they need proxy credentials with Snowflake.
| } | ||
|
|
||
| @Test | ||
| void testInsertFilesViaProxy() throws InterruptedException, NoSuchAlgorithmException { |
There was a problem hiding this comment.
This test uses an HTTP base URI and a null authenticator, so it does not cover HTTPS tunneling or proxy credentials, which are the parts most likely to break. Can we add coverage for the authenticated proxy and HTTPS cases?
There was a problem hiding this comment.
I've updated the proxy test, it now points the base URI to a different host (http://fake-snowflake.example.com) while the proxy selector points to MockWebServer. When Java's HttpClient routes through an HTTP proxy, it sends the request target in absolute form, so we assert the recorded request starts with http://fake-snowflake.example.com. This proves the proxy selector is actually being used. For the HTTPS + credentials case as mentioned in the comment above, this hits the JDK limitation with CONNECT tunneling and would need a full TLS proxy setup to test properly, which is more of an integration test scenario. The property description now documents what users need to configure to make it work.
…rService Without this change, SnowpipeIngestClient always connects to Snowflake directly, which fails in environments behind a corporate proxy. This adds proxy support by wiring ProxyConfigurationService into the service's HTTP client. Changes: - Added PROXY_CONFIGURATION_SERVICE property to StandardSnowflakeIngestManagerProviderService - Used a custom ProxySelector that correctly handles HTTP, SOCKS, and DIRECT proxy types - Added scoped Authenticator for proxy username/password credentials - Extended SnowpipeIngestClient with a constructor accepting ProxySelector and Authenticator
rakesh-rsky
force-pushed
the
fix/NIFI-15911-proxy-support-snowflake-ingest
branch
from
baeeac2 to
3baff7d
Compare
2 participants
NIFI-15911 Add proxy support to StandardSnowflakeIngestManagerProviderService
StandardSnowflakeIngestManagerProviderService uses SnowpipeIngestClient to send data to Snowflake via the Snowpipe REST API. Previously, the HTTP client had no proxy configuration, causing failures for NiFi deployments running behind a corporate or network proxy.
This change adds proxy support by wiring ProxyConfigurationService into the controller service.
Changes:
SnowpipeIngestClient
Summary
NIFI-15911
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking
Pull Request Tracking
NIFI-00000NIFI-00000VerifiedstatusPull Request Formatting
mainbranchVerification
Please indicate the verification steps performed prior to pull request creation.
Build
./mvnw clean install -P contrib-checkLicensing
LICENSEandNOTICEfilesDocumentation