You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
When issuing the Sharded Atlas Checks to spark there is a way to specify a URL to the atlas checks configuration file, but only a public URL is allowed. If an S3 link is provided then the spark jobs fail.
Describe the solution you'd like
This request is to allow to specify an AWS S3 link (s3://blah/configuration.json) in the "-configFiles" parameter so that private configuration files can be used when running the spark atlas checks jobs. The "-input" and "-output" parameter already allows for S3 links so I would think that this would be pretty easy.
Describe alternatives you've considered
The alternative is to post my config files somewhere that can be reached by public http URL. This is OK, but just could be better if it was an S3 link.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
When issuing the Sharded Atlas Checks to spark there is a way to specify a URL to the atlas checks configuration file, but only a public URL is allowed. If an S3 link is provided then the spark jobs fail.
Describe the solution you'd like
This request is to allow to specify an AWS S3 link (s3://blah/configuration.json) in the "-configFiles" parameter so that private configuration files can be used when running the spark atlas checks jobs. The "-input" and "-output" parameter already allows for S3 links so I would think that this would be pretty easy.
Describe alternatives you've considered
The alternative is to post my config files somewhere that can be reached by public http URL. This is OK, but just could be better if it was an S3 link.
The text was updated successfully, but these errors were encountered: