Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ASDI EC2 resources for data production pipeline #380

Open
MichaelTiemannOSC opened this issue Jan 27, 2025 · 0 comments
Open

Use ASDI EC2 resources for data production pipeline #380

MichaelTiemannOSC opened this issue Jan 27, 2025 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@MichaelTiemannOSC
Copy link
Contributor

Recently we migrated published ZARR data to ASDI for data hosting. We believe we can use ASDI EC2 resources to process raw data to produce ZARR files. This issue tracks the process of actually migrating the production of ZARR files that are published on ASDI to ASDI compute servers.

@ModeSevenIndustrialSolutions can help with any infra details to make this work. We just need a good way to identify and document how to run compliant pipelines on ASDI rather than OS-C hosted resources.
@joemoorhouse feel free to delegate this to anybody from the PhysRisk team who has the technical ability to assess, implement, and document the satisfaction of this issue.

Of course any PhysRisk analytics that go beyond publishing approved datasets remains the cost of either the analyst (working in their own environment with our data) or the cost of OS-Climate (as approved by our project plan).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants