Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement WorldCereal inference pipeline using GFMAP #38

Closed
GriffinBabe opened this issue Apr 12, 2024 · 2 comments · Fixed by #48
Closed

Implement WorldCereal inference pipeline using GFMAP #38

GriffinBabe opened this issue Apr 12, 2024 · 2 comments · Fixed by #48
Assignees
Milestone

Comments

@GriffinBabe
Copy link
Contributor

As discussed with @GriffinBabe, I worked on a minimal example showing the entire flow required to create a crop/no-crop map for WorldCereal in OpenEO.

The notebook is here: https://github.com/WorldCereal/wc-classification/blob/openeo/notebooks/OPENEO/inference_example_openeo.ipynb

Actual inference code is here: https://github.com/WorldCereal/wc-classification/blob/openeo/src/worldcereal/openeo/inference.py

It's so far not yet done with GFMAP. Example starts from pregenerated result from the worldcereal_preprocessed_inputs block, which in GFMAP terminology contains the data fetching and preprocessing steps.
That NetCDF is loaded and a UDF is applied to it locally. That UDF contains both feature extraction (using presto encoder) and model inference (using catboost).

Only works locally for numerous reasons:
No env compiled yet that contains the required libraries
Access needed to EPSG (hardcoded for the moment) to get the coordinates required as input for Presto
On CDSE wouldn't work yet anyway because Meteo collection loader is required
Probably this effort should only be done when we know how to do it in GFMAP directly instead of first as a standalone and then refactoring it.

@GriffinBabe GriffinBabe self-assigned this Apr 12, 2024
@HansVRP
Copy link
Contributor

HansVRP commented May 6, 2024

Here the repo where I have been working on the mvp example:

https://github.com/WorldCereal/worldcereal-classification/tree/hv_mvp_inferenceUDF/minimal_wc_presto

@HansVRP
Copy link
Contributor

HansVRP commented May 14, 2024

TODO; check how the developed architecture would work with the delivered UDF for inference

@kvantricht kvantricht linked a pull request Jun 7, 2024 that will close this issue
@kvantricht kvantricht added this to the System V1 milestone Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants