Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design patch-to-point job definition #214

Open
kvantricht opened this issue Oct 30, 2024 · 1 comment
Open

Design patch-to-point job definition #214

kvantricht opened this issue Oct 30, 2024 · 1 comment
Assignees

Comments

@kvantricht
Copy link
Contributor

Once our extractions are migrated to STAC api, we need to think about how to launch actual patch-to-point jobs, most likely per ref_id, then further split into per EPSG zone.

@VincentVerelst
Copy link
Contributor

VincentVerelst commented Nov 29, 2024

A first draft, very prone to change:

  • Query STAC API with a property filter equal to a certain EPSG code
  • From this we can construct a new job_tracker where each job corresponds to a different EPSG code
  • Get the total bounds of all STAC items returned from the query. With these bounds query the RDM API to get all features intersecting with the bounds and with extract_flag==1 --> Here we need a sampling procedure (just take centroids?)
  • load_stac the S1 and S2 patch extraction from the API and load pre-composited Agera5 and DEM, apply standard preprocessing and merge
  • aggregate_spatial on the sampled points and execute the batch job --> To be seen if it's possible in one batch job
  • Download results as geoparquet and merge to existing geoparquet database of point extractions
  • If job succesful: mark all extracted features as extracted (using extract_flag?) in RDM, to avoid double extractions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants