Finalize inference UDP and share with GISAT

[jdegerickx]

No description provided.

[jdegerickx]

@jdries , how should we proceed here?
Would be good to have an inference UDP both for GISAT and Maps4GPP project to allow people to work on user interfaces...

Basically the UDP should be a wrapper around the generate_map function --> https://github.com/WorldCereal/worldcereal-classification/blob/main/src/worldcereal/job.py#L69

Does the current UDP already allow to pass the different input parameters to generate_map?

Should we ask the data engineering team to update the UDP? Guess would be more efficient that way, instead of me trying to figure it out...

[jdries]

@jdegerickx yes indeed, next step is to get this improved/merged:
Open-EO/openeo-gfmap#144
I asked @HansVRP to have a look and take over if possible.

It's also possible to update the current udp definition as a kind of quick fix, but prefer to also get the code right.

[jdegerickx]

thanks @jdries !
@HansVRP , the inference pipeline should be stable for a while now, so best to get this UDP ready asap so our partners can start integration.
Would be happy to test once we have a first version ready!

[jdegerickx]

@HansVRP, @VincentVerelst , any update on this one?

[HansVRP]

WIll be planned in this sprint

[VincentVerelst]

@jdegerickx, so this issue now refers to the crop extent UDP, not the crop type UDP?

[jdegerickx]

@VincentVerelst, would be good to have two separate worldcereal inference UDP's:

• one which does default crop extent mapping --> see this notebook to get an idea of what it should support:
  https://github.com/WorldCereal/worldcereal-classification/blob/main/notebooks/worldcereal_v1_demo_default_cropland_extended.ipynb

• one which does custom model inference, supporting both crop extent and crop type. There are two notebooks related to this:

1. https://github.com/WorldCereal/worldcereal-classification/blob/main/notebooks/worldcereal_v1_demo_custom_croptype_extended.ipynb
2. https://github.com/WorldCereal/worldcereal-classification/blob/main/notebooks/worldcereal_v1_demo_custom_cropland.ipynb

I suggest we just start with the default crop extent mapping and later on create a separate issue for the custom model inference, if that's ok?

Additional notes:

• the notebooks show how to retrieve the worldcereal crop calendars for a given bounding box. This should not be part of the UDP.
• all notebooks also contain additional steps to properly visualize the resulting products after applying the "generate_map" function. These steps should obviously not be part of the UDP.

[VincentVerelst]

• Worldcereal crop extent UDP
• Worldcereal crop type UDP

[VincentVerelst]

@jdegerickx , ok to close this issue and later open a new one for custom crop type?

[jdegerickx]

I still would like to test the UDP myself first.
Then we would need a discussion on how to deal with postprocessing parameters in the UDP.
And only then move to crop type inference.

[jdegerickx]

@jdries , @VincentVerelst , cf. mail Eversis. Things to clarify:

• is the temporal extent correctly parsed in the UDP? Is it just a problem with the final filename?
• normally by default postprocessing is applied in the WorldCereal inference run. This means that final product would be called "cropland", not "cropland-raw", as is now the case. So it is not clear to me whether post-processing is applied but the filename is wrong, or whether post-processing has not been applied...

[VincentVerelst]

@jdries , @VincentVerelst , cf. mail Eversis. Things to clarify:

• is the temporal extent correctly parsed in the UDP? Is it just a problem with the final filename?
• normally by default postprocessing is applied in the WorldCereal inference run. This means that final product would be called "cropland", not "cropland-raw", as is now the case. So it is not clear to me whether post-processing is applied but the filename is wrong, or whether post-processing has not been applied...

@jdegerickx

• The filename is determined after the UDP call, so the UDP should have no influence on it
• Postprocessing is part of the UDP. The postprocessing parameters are the default parameters from this notebook: https://github.com/WorldCereal/worldcereal-classification/blob/main/notebooks/worldcereal_v1_demo_default_cropland_extended.ipynb

[jdegerickx]

@VincentVerelst, I'm not sure where this filename creation actually happens? Is it part of your UDP example script you sent to Eversis?

[jdegerickx]

ok, tested again after fix by @jdries on the filename.
UDP now returns two identical assets, one called "openEO.tif" and one called "worldcereal-cropland-extent.tif"
Not sure this was intentional?
Example run:
cdse-j-24112740bb56426b9b353e57e1c5ac09

I see the final product still has all the bands (4 in total). By default in the postprocessing parameters, the "keep_class_probs" flag is set to False, meaning that in the end only the classification label (band 1) and winning probability (band 2) should be stored. This makes me wonder whether post-processing is correctly applied? @VincentVerelst , could you check the latter?

[jdegerickx]

next step: build another UDP for the crop TYPE inference.
Based on this notebook --> https://github.com/WorldCereal/worldcereal-classification/blob/main/notebooks/worldcereal_v1_demo_custom_croptype.ipynb

Only major difference is that user needs to be able to provide a model_url.
In the notebook:
parameters = CropTypeParameters()
parameters.classifier_parameters.classifier_url = model_url

the save_mask parameter can be set to its default value (False), in contrast to what happens in the notebook. If done this way, the inference run will only generate one single asset, which should simplify things a bit...