Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dateline #310

Open
wants to merge 20 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
bfbe212
Add testMissingS2DateLine https://github.com/Open-EO/openeo-geotrelli…
EmileSonneveld May 28, 2024
2882a53
Case that better shows error.
EmileSonneveld Jun 4, 2024
5eec0bf
Avoid north pole impricision when reprojecting between LatLng and UTM…
EmileSonneveld Jun 10, 2024
059eb58
Avoid RATE_LIMIT_EXCEEDED.
EmileSonneveld Jun 11, 2024
93c2b75
Fix to make tests run on Jenkins. https://github.com/Open-EO/openeo-g…
EmileSonneveld Jun 28, 2024
14cfc60
formatting
EmileSonneveld Jun 28, 2024
223b579
Assert sensible extends in UTM zones. https://github.com/Open-EO/open…
EmileSonneveld Jul 23, 2024
7b580ec
Better choose common CRS. Add test for UTM->LatLng and LatLng->UTM. A…
EmileSonneveld Jul 24, 2024
f5990c0
Clean up healthCheckExtent().
EmileSonneveld Jul 24, 2024
7b6760d
fix
EmileSonneveld Jul 25, 2024
0536397
Merge branch 'develop' into dateline
EmileSonneveld Jul 25, 2024
500fccf
comment
EmileSonneveld Jul 25, 2024
d8691bc
Add log to see if different UTM test case is already covered.
EmileSonneveld Aug 9, 2024
c99b6d2
Merge branch 'develop' into dateline
EmileSonneveld Aug 9, 2024
e98879c
Merge branch 'develop' into dateline
EmileSonneveld Aug 9, 2024
4896876
Revert "Add log to see if different UTM test case is already covered."
EmileSonneveld Aug 9, 2024
5ef4a96
Merge branch 'develop' into dateline
EmileSonneveld Sep 16, 2024
0309ef2
Test EPSG:3035 and EPSG:4326 too. https://github.com/Open-EO/openeo-g…
EmileSonneveld Sep 17, 2024
9c9fc1a
Merge branch 'develop' into dateline
EmileSonneveld Oct 28, 2024
ad69dd7
fix after merge
EmileSonneveld Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ import org.locationtech.jts.geom.Geometry
import org.openeo.geotrellis.OpenEOProcessScriptBuilder.AnyProcess
import org.openeo.geotrellis.file.{AbstractPyramidFactory, FixedFeaturesOpenSearchClient}
import org.openeo.geotrellis.tile_grid.TileGrid
import org.openeo.geotrellis.{OpenEOProcessScriptBuilder, sortableSourceName}
import org.openeo.geotrellis.{OpenEOProcessScriptBuilder, healthCheckExtent, sortableSourceName}
import org.openeo.geotrelliscommon.{BatchJobMetadataTracker, ByKeyPartitioner, CloudFilterStrategy, ConfigurableSpatialPartitioner, DataCubeParameters, DatacubeSupport, L1CCloudFilterStrategy, MaskTileLoader, NoCloudFilterStrategy, ResampledTile, SCLConvolutionFilterStrategy, SpaceTimeByMonthPartitioner, SparseSpaceTimePartitioner, autoUtmEpsg, retryForever}
import org.openeo.opensearch.OpenSearchClient
import org.openeo.opensearch.OpenSearchResponses.{Feature, Link}
import org.slf4j.LoggerFactory
import org.slf4j.{Logger, LoggerFactory}

import java.io.{IOException, Serializable}
import java.net.URI
Expand Down Expand Up @@ -236,7 +236,7 @@ class MultibandCompositeRasterSource(val sourcesListWithBandIds: NonEmptyList[(R

object FileLayerProvider {

private val logger = LoggerFactory.getLogger(classOf[FileLayerProvider])
private implicit val logger: Logger = LoggerFactory.getLogger(classOf[FileLayerProvider])
private val maxRetries = sys.env.getOrElse("GDALREAD_MAXRETRIES", "10").toInt

{
Expand Down Expand Up @@ -1261,13 +1261,27 @@ class FileLayerProvider private(openSearch: OpenSearchClient, openSearchCollecti
* Several edge cases to cover:
* - if feature extent is whole world, it may be invalid in target crs
* - if feature is in utm, target extent may be invalid in feature crs
* this is why we take intersection
* this is why we take intersection.
* We convert both extents to a common CRS before taking the intersection.
* If one of the CRSes can cover the whole world (non-UTM), this will be used as common CRS.
* We give priority to use the target CRS as common one, because the intersection will be converted to it anyway
*/
val targetExtentInLatLon = targetExtent.reproject(feature.crs.get)
val featureExtentInLatLon = feature.rasterExtent.get.reproject(feature.crs.get,LatLng)

val intersection = featureExtentInLatLon.intersection(targetExtentInLatLon).map(_.buffer(1.0)).getOrElse(featureExtentInLatLon)
val tmp = expandToCellSize(intersection.reproject(LatLng, targetExtent.crs), theResolution)
val featureIsUTM = feature.crs.get.proj4jCrs.getProjection.getName == "utm"
val targetIsUTM = targetExtent.crs.proj4jCrs.getProjection.getName == "utm"
val commonCrs = if (!targetIsUTM) targetExtent.crs
else if (!featureIsUTM) feature.crs.get
else targetExtent.crs // Avoid conversion imprecision by intersecting directly in the target CRS

val featureExtentInCommonCRS = feature.rasterExtent.get.reproject(feature.crs.get, commonCrs)
val targetExtentInCommonCRS = targetExtent.extent.reproject(targetExtent.crs, commonCrs)

val intersection = featureExtentInCommonCRS.intersection(targetExtentInCommonCRS).map(_.buffer(1.0))
val intersectionTargetCrs = intersection match {
case None => targetExtent.extent.reproject(targetExtent.crs, targetExtent.crs)
case Some(value) => value.reproject(commonCrs, targetExtent.crs)
}
val tmp = expandToCellSize(intersectionTargetCrs, theResolution)
healthCheckExtent(ProjectedExtent(tmp, targetExtent.crs))

val alignedToTargetExtent = re.createAlignedRasterExtent(tmp)
Some(alignedToTargetExtent.toGridType[Long])
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
package org.openeo

import _root_.geotrellis.proj4.LatLng
import _root_.geotrellis.raster._
import net.jodah.failsafe.event.{ExecutionAttemptedEvent, ExecutionCompletedEvent, ExecutionScheduledEvent}
import net.jodah.failsafe.{ExecutionContext, Failsafe, RetryPolicy => FailsafeRetryPolicy}
import _root_.geotrellis.vector._
import org.slf4j.Logger
import scalaj.http.{HttpResponse, HttpStatusException}
import software.amazon.awssdk.awscore.retry.conditions.RetryOnErrorCodeCondition
Expand Down Expand Up @@ -216,4 +218,28 @@ package object geotrellis {
httpResponseCallback
})
}

def healthCheckExtent(projectedExtent: ProjectedExtent)(implicit logger: Logger): Boolean = {
val horizontal_tolerance = 1.1
val polygonIsUTM = projectedExtent.crs.proj4jCrs.getProjection.getName == "utm"
if (polygonIsUTM) {
// This is an extend that has the highest sensible values for northern and/or southern hemisphere UTM zones
val utmProjectedBoundsOriginal = Extent(166021.44, 0000000.00, 833978.56, 10000000)
val utmProjectedBounds = utmProjectedBoundsOriginal.buffer(
utmProjectedBoundsOriginal.width * horizontal_tolerance, 0)
if (!projectedExtent.extent.intersects(utmProjectedBounds)) {
logger.warn("healthCheckExtent dangerous extent: " + projectedExtent)
return false
}
} else if (projectedExtent.crs == LatLng) {
if ((projectedExtent.extent.xmin < -180 * horizontal_tolerance)
|| (projectedExtent.extent.xmax > +180 * horizontal_tolerance)
|| (projectedExtent.extent.ymin < -90)
|| (projectedExtent.extent.ymax > +90)) {
logger.warn("healthCheckExtent dangerous extent: " + projectedExtent)
return false
}
}
true
}
}
Binary file not shown.
Loading