Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDAP-517 - Credential management for AWS collections #312

Draft
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

RKuttruff
Copy link
Contributor

This PR augments the configurations of Zarr (and eventually future) collections.

Initially, collections in non-public S3 buckets needed to be provided an Access Key ID and Secret Access Key pair:

config:
  aws:
    accessKeyID: <secret>
    secretAccessKey: <secret>
    public: false

Obviously, this is not ideal as long-term AWS keys are not only directly stored, but also need to be maintained by end users. This PR allows more options in defining AWS credentials.

Profiles

You can use credentials defined in AWS named profiles:

config:
  aws:
    profile: <profile name>

NASA DAAC Temporary S3 credentials

NASA DAACs make their data available for direct S3 access for running in the us-west-2 region. You can utilize this by specifying the credential endpoint or by picking from a list of predefined DAACs.

NOTE: This only works if running in the us-west-2 AWS region. A check will be made to see in SDAP is running in that region and these collections will be excluded if not

config:
  earthdata:
    endpoint: https://api.giovanni.earthdata.nasa.gov/s3credentials
config:
  earthdata:
    daac: podaac

An Earthdata login is required for this. Provide either EDL_USERNAME and EDL_PASSWORD environment variables or specify the login on a per-collection basis:

config:
  earthdata:
    daac: gesdisc
    edl_username: <user>
    edl_password: <pass>

Predefined DAACs:

  • podaac
  • podaac-swot
  • gesdisc
  • lpdaac
  • obdaac
  • nsidc
  • laads
  • asfdaac
  • asfdaac-sentinel1

@RKuttruff
Copy link
Contributor Author

Some changes of this PR were copied to #313

rileykk added 7 commits June 6, 2024 16:05
# Conflicts:
#	data-access/nexustiles/nexustiles.py
#	poetry.lock
# Conflicts:
#	data-access/nexustiles/nexustiles.py
# Conflicts:
#	data-access/nexustiles/AbstractTileService.py
#	poetry.lock
#	pyproject.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant