-
Notifications
You must be signed in to change notification settings - Fork 8
add sen2like udp #277
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ValentinaHutter
wants to merge
6
commits into
ESA-APEx:main
Choose a base branch
from
ValentinaHutter:sen2like
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
add sen2like udp #277
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
4774ea9
update descriptions, use Sentinel2_L2A
ValentinaHutter 4fbb6c2
Merge branch 'main' into sen2like
jdries fc19bba
update benchmark spatial_extent, cost estimate
ValentinaHutter 8d1cd87
Merge branch 'sen2like' of github.com:ValentinaHutter/apex_algorithms…
ValentinaHutter ba3c918
Update algorithm_catalog/eodc/sen2like/records/sen2like.json
ValentinaHutter 59ca687
Merge branch 'main' into sen2like
ValentinaHutter File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
31 changes: 31 additions & 0 deletions
31
algorithm_catalog/eodc/sen2like/benchmark_scenarios/sen2like.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| [ | ||
| { | ||
| "id": "sen2like", | ||
| "type": "openeo", | ||
| "description": "Sen2like test", | ||
| "backend": "https://openeo.eodc.eu/openeo/1.2.0", | ||
| "process_graph": { | ||
| "ardsen2like": { | ||
| "process_id": "sen2like", | ||
| "namespace": "https://raw.githubusercontent.com/ESA-APEx/apex_algorithms/refs/heads/sen2like/algorithm_catalog/eodc/sen2like/openeo_udp/sen2like.json", | ||
| "arguments": { | ||
| "bbox": { | ||
| "east": 16.414, | ||
| "north": 48.008, | ||
| "south": 47.962, | ||
| "west": 16.342 | ||
| }, | ||
| "temporal_extent": [ | ||
| "2023-05-01", | ||
| "2023-05-30" | ||
| ] | ||
| }, | ||
| "result": true | ||
| } | ||
| }, | ||
| "reference_data": { | ||
| }, | ||
| "reference_options": { | ||
| } | ||
| } | ||
| ] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| # Sen2like | ||
|
|
||
| The Sen2Like processor was developed by ESA as part of the EU Copernicus program. It creates Sentinel-2 like harmonized (Level-2H) or fused (Level-2F) surface reflectances by harmonizing Sentinel-2 and Landsat 8/Landsat 9 to increase the temporal revisits. The fusion involves the upscaling of Landsat 8/Landsat 9 data to Sentinel-2 resolution. Furthermore, the resulting sen2like L2H/ L2F data can be processed using openEO to generate statistics, vegetation indices, do comparisons with other datasets, etc. | ||
|
|
||
| ### Methodology | ||
|
|
||
| The processing of the incoming Landsat and Sentinel-2 L1C data includes the following main processing steps: Geometric Processing, Stitching, Geometric Check, Inter-calibration, Atmospheric correction, BRDF Adjustment, SBAF, Topographic Correction, Data Fusion. If Sentinel-2 L2A data is provided, sen2like processing will not include Atmospheric and Topographic Correction. In the geometric processing step, the input images are co-registered to a Sentinel-2 reference image. The atmospheric correction step makes use of the sen2cor processor and additionally relies on Copernicus Atmosphere Monitoring Service (CAMS) Near Real Time and Reanalysis data as well as the Copernicus Digital Elevation Model. The Data Fusion step alignes Landsat 8 image pixel spacing fully with Sentinel2 image pixel spacing. Depending on the band, the resolution of the Landsat L2F product is 10 m, 20 m or 30 m. | ||
|
|
||
| ### Quality | ||
|
|
||
| The geometric check process is a Quality Control step of the product. For further information, see Sen2like User Manual. | ||
|
|
||
| ### Links | ||
|
|
||
| - [RD1] openEO platform Sen2like documentation https://docs.openeo.cloud/usecases/ard/sen2like | ||
|
|
||
| - [RD1] Saunier, S. (2025). Sen2like User Manual https://github.com/senbox-org/sen2like/blob/master/sen2like/docs/source/S2-SEN2LIKE-UM-V1.10.pdf" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| import json | ||
| from pathlib import Path | ||
|
|
||
| import openeo | ||
| from openeo.api.process import Parameter | ||
| from openeo.rest._datacube import THIS | ||
| from openeo.rest.udp import build_process_dict | ||
|
|
||
| def generate(): | ||
| connection = openeo.connect("https://openeo.eodc.eu/openeo/1.2.0/").authenticate_oidc() | ||
|
|
||
| spatial_extent = Parameter.spatial_extent( | ||
| name="spatial_extent", | ||
| description="Limits the data to process to the specified bounding box or polygons.\\n\\nFor raster data, the process loads the pixel into the data cube if the point at the pixel center intersects with the bounding box or any of the polygons (as defined in the Simple Features standard by the OGC).\\nFor vector data, the process loads the geometry into the data cube if the geometry is fully within the bounding box or any of the polygons (as defined in the Simple Features standard by the OGC). Empty geometries may only be in the data cube if no spatial extent has been provided.\\n\\nEmpty geometries are ignored.\\nSet this parameter to null to set no limit for the spatial extent." | ||
| ) | ||
|
|
||
| temporal_extent = Parameter.temporal_interval( | ||
| name="temporal_extent", | ||
| description="Temporal extent specified as two-element array with start and end date/date-time." | ||
| ) | ||
|
|
||
| schema = { | ||
| "type": "string", | ||
| "enum": ["B01", "B02", "B03", "B04", "B05", "B06", "B07", "B08", "B8A", "B11", "B12"], | ||
| } | ||
| bands = Parameter.array( | ||
| name="bands", | ||
| description="Sentinel-2 bands to include in the composite.", | ||
| item_schema=schema, | ||
| default=["B04", "B03", "B02"], | ||
| optional=True, | ||
| ) | ||
|
|
||
| collection = 'SENTINEL2_L2A' | ||
|
|
||
| s2_l1c = connection.load_collection( | ||
| collection, | ||
| spatial_extent=spatial_extent, | ||
| temporal_extent=temporal_extent, | ||
| bands=bands) | ||
|
|
||
| sen2like = s2_l1c.process('sen2like', { | ||
| 'data': THIS, | ||
| 'target_product': 'L2F', | ||
| 'export_original_files': True, | ||
| 'cloud_cover': 50}) | ||
|
|
||
| returns = { | ||
| "description": "A data cube with the newly computed values.\n\nThe result will combine Sentinel-2 and Landsat timesteps for the requested extent. Just like Sentinel 2 data, sen2like generates .SAFE output files, which are zipped for the purpose of openEO. For both Landsat and Sentinel acquisitions the .SAFE files include the requested bands. Note, that the sen2like Landsat outputs do not include equivalents of Sentinel-2 bands B05, B06, B07.", | ||
| "schema": { | ||
| "type": "object", | ||
| "subtype": "datacube" | ||
| } | ||
| } | ||
|
|
||
| return build_process_dict( | ||
| process_graph=sen2like, | ||
| process_id="sen2like", | ||
| summary="Computes a harmonzed Sentinel-2 and Landsat timeseries.", | ||
| description=(Path(__file__).parent / "README.md").read_text(), | ||
| parameters=[ | ||
| spatial_extent, | ||
| temporal_extent | ||
| ], | ||
| returns=returns, | ||
| categories=["sentinel-2", "ARD"] | ||
| ) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| with open("sen2like.json", "w") as f: | ||
| json.dump(generate(), f, indent=2) |
171 changes: 171 additions & 0 deletions
171
algorithm_catalog/eodc/sen2like/openeo_udp/sen2like.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,171 @@ | ||
| { | ||
| "process_graph": { | ||
| "loadcollection1": { | ||
| "process_id": "load_collection", | ||
| "arguments": { | ||
| "bands": { | ||
| "from_parameter": "bands" | ||
| }, | ||
| "id": "SENTINEL2_L2A", | ||
| "spatial_extent": { | ||
| "from_parameter": "spatial_extent" | ||
| }, | ||
| "temporal_extent": { | ||
| "from_parameter": "temporal_extent" | ||
| } | ||
| } | ||
| }, | ||
| "sen2like1": { | ||
| "process_id": "sen2like", | ||
| "arguments": { | ||
| "cloud_cover": 50, | ||
| "data": { | ||
| "from_node": "loadcollection1" | ||
| }, | ||
| "export_original_files": true, | ||
| "target_product": "L2F" | ||
| }, | ||
| "result": true | ||
| } | ||
| }, | ||
| "id": "sen2like", | ||
| "description": "# Sen2like\n\nThe Sen2Like processor was developed by ESA as part of the EU Copernicus program. It creates Sentinel-2 like harmonized (Level-2H) or fused (Level-2F) surface reflectances by harmonizing Sentinel-2 and Landsat 8/Landsat 9 to increase the temporal revisits. Based on the resulting L2F product, multiple indices can be computed, such as the NDVI and LAI. The fusion also involves the upscaling of Landsat 8/Landsat 9 data to Sentinel-2 resolution. With the new L2F data higher time-series resolution vegetation indices (such as NDVI, LAI, FAPAR, FCOVER) can be calculated.\n\n### Methodology\n\n\n\n### Quality\n\n\n\n### Links\n\n- [RD1] openEO platform Sen2like documentation https://docs.openeo.cloud/usecases/ard/sen2like", | ||
| "parameters": [ | ||
| { | ||
| "name": "bands", | ||
| "description": "Only adds the specified bands into the data cube so that bands that don't match the list of band names are not available. Applies to all dimensions of type `bands`.\n\nEither the unique band name (metadata field `name` in bands) or one of the common band names (metadata field `common_name` in bands) can be specified. If the unique band name and the common name conflict, the unique band name has a higher priority.\n\nThe order of the specified array defines the order of the bands in the data cube. If multiple bands match a common name, all matched bands are included in the original order.\n\nIt is recommended to use this parameter instead of using ``filter_bands()`` directly after loading unbounded data.", | ||
| "schema": [ | ||
| { | ||
| "type": "array", | ||
| "minItems": 1, | ||
| "items": { | ||
| "type": "string", | ||
| "subtype": "band-name" | ||
| } | ||
| }, | ||
| { | ||
| "title": "No filter", | ||
| "description": "Don't filter bands. All bands are included in the data cube.", | ||
| "type": "null" | ||
| } | ||
| ], | ||
| "default": null, | ||
| "optional": true | ||
| }, | ||
| { | ||
| "name": "spatial_extent", | ||
| "description": "Limits the data to process to the specified bounding box or polygons.\\n\\nFor raster data, the process loads the pixel into the data cube if the point at the pixel center intersects with the bounding box or any of the polygons (as defined in the Simple Features standard by the OGC).\\nFor vector data, the process loads the geometry into the data cube if the geometry is fully within the bounding box or any of the polygons (as defined in the Simple Features standard by the OGC). Empty geometries may only be in the data cube if no spatial extent has been provided.\\n\\nEmpty geometries are ignored.\\nSet this parameter to null to set no limit for the spatial extent.", | ||
| "schema": [ | ||
| { | ||
| "title": "Bounding Box", | ||
| "type": "object", | ||
| "subtype": "bounding-box", | ||
| "required": [ | ||
| "west", | ||
| "south", | ||
| "east", | ||
| "north" | ||
| ], | ||
| "properties": { | ||
| "west": { | ||
| "description": "West (lower left corner, coordinate axis 1).", | ||
| "type": "number" | ||
| }, | ||
| "south": { | ||
| "description": "South (lower left corner, coordinate axis 2).", | ||
| "type": "number" | ||
| }, | ||
| "east": { | ||
| "description": "East (upper right corner, coordinate axis 1).", | ||
| "type": "number" | ||
| }, | ||
| "north": { | ||
| "description": "North (upper right corner, coordinate axis 2).", | ||
| "type": "number" | ||
| }, | ||
| "base": { | ||
| "description": "Base (optional, lower left corner, coordinate axis 3).", | ||
| "type": [ | ||
| "number", | ||
| "null" | ||
| ], | ||
| "default": null | ||
| }, | ||
| "height": { | ||
| "description": "Height (optional, upper right corner, coordinate axis 3).", | ||
| "type": [ | ||
| "number", | ||
| "null" | ||
| ], | ||
| "default": null | ||
| }, | ||
| "crs": { | ||
| "description": "Coordinate reference system of the extent, specified as as [EPSG code](http://www.epsg-registry.org/) or [WKT2 CRS string](http://docs.opengeospatial.org/is/18-010r7/18-010r7.html). Defaults to `4326` (EPSG code 4326) unless the client explicitly requests a different coordinate reference system.", | ||
| "anyOf": [ | ||
| { | ||
| "title": "EPSG Code", | ||
| "type": "integer", | ||
| "subtype": "epsg-code", | ||
| "minimum": 1000, | ||
| "examples": [ | ||
| 3857 | ||
| ] | ||
| }, | ||
| { | ||
| "title": "WKT2", | ||
| "type": "string", | ||
| "subtype": "wkt2-definition" | ||
| } | ||
| ], | ||
| "default": 4326 | ||
| } | ||
| } | ||
| }, | ||
| { | ||
| "title": "Vector data cube", | ||
| "description": "Limits the data cube to the bounding box of the given geometries in the vector data cube. For raster data, all pixels inside the bounding box that do not intersect with any of the polygons will be set to no data (`null`). Empty geometries are ignored.", | ||
| "type": "object", | ||
| "subtype": "datacube", | ||
| "dimensions": [ | ||
| { | ||
| "type": "geometry" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "title": "No filter", | ||
| "description": "Don't filter spatially. All data is included in the data cube.", | ||
| "type": "null" | ||
| } | ||
| ] | ||
| }, | ||
| { | ||
| "name": "temporal_extent", | ||
| "description": "Temporal extent specified as two-element array with start and end date/date-time.", | ||
| "schema": { | ||
| "type": "array", | ||
| "subtype": "temporal-interval", | ||
| "uniqueItems": true, | ||
| "minItems": 2, | ||
| "maxItems": 2, | ||
| "items": { | ||
| "anyOf": [ | ||
| { | ||
| "type": "string", | ||
| "subtype": "date-time", | ||
| "format": "date-time" | ||
| }, | ||
| { | ||
| "type": "string", | ||
| "subtype": "date", | ||
| "format": "date" | ||
| }, | ||
| { | ||
| "type": "null" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| } | ||
| ] | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the 'bbox' parameter would be 'spatial_extent' according to current udp definition.
If I however try to run this scenario, it gives "No Sentinel2 tiles available for provided request!"
Test job id: d5c7336a-fe72-4867-9762-c4fad4c10025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I now updated the parameter name. Sorry, I initially tested it on the dev backend, where it ran through successfully, but I only deployed the latest changes to production today.