Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add union to merge methods #3482

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -63,4 +63,35 @@ trait MultibandTileMergeMethods extends TileMergeMethods[MultibandTile] {

ArrayMultibandTile(bands)
}

/**
* Union this [[MultibandTile]] with the other one. The output tile's
* extent will be the minimal extent which encompasses both input extents.
* A new MutlibandTile is returned.
*
* @param extent The extent of this MultiBandTile
* @param otherExtent The extent of the other MultiBandTile
* @param other The other MultiBandTile
* @param method The resampling method
* @param unionF The function which decides how values from rasters being combined will be transformed
* @return A new MultiBandTile, the result of the merge
*/
def union(
extent: Extent,
otherExtent: Extent,
other: MultibandTile,
method: ResampleMethod,
unionF: (Option[Double], Option[Double]) => Double
): MultibandTile = {
val bands: Seq[Tile] =
for {
bandIndex <- 0 until self.bandCount
} yield {
val thisBand = self.band(bandIndex)
val thatBand = other.band(bandIndex)
thisBand.union(extent, otherExtent, thatBand, method, unionF)
}

ArrayMultibandTile(bands)
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -50,4 +50,16 @@ abstract class RasterMergeMethods[
*/
def merge(other: Raster[T]): Raster[T] =
merge(other, NearestNeighbor)

/**
* Union this [[Raster]] with the other one. All places in the
* present raster that contain NODATA are filled-in with data from
* the other raster. A new Raster is returned.
*
* @param other The other Raster
* @param method The resampling method
* @return A new Raster, the result of the merge
*/
def union(other: Raster[T], method: ResampleMethod, unionF: (Option[Double], Option[Double]) => Double): Raster[T] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

untionF => unionFunc (it took me a while to understand what F stands for here) :D

Raster(self.tile.union(self.extent, other.extent, other.tile, method, unionF), self.extent.combine(other.extent))
}
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,7 @@ abstract class RasterTileFeatureMergeMethods[

def merge(other: TileFeature[Raster[T], D], method: ResampleMethod): TileFeature[Raster[T], D] =
TileFeature(self.tile.merge(other.tile, method), Semigroup[D].combine(self.data, other.data))

def union(other: TileFeature[Raster[T], D], method: ResampleMethod, unionF: (Option[Double], Option[Double]) => Double): TileFeature[Raster[T], D] =
TileFeature(self.tile.union(other.tile, method, unionF), Semigroup[D].combine(self.data, other.data))
}
Original file line number Diff line number Diff line change
Expand Up @@ -154,4 +154,69 @@ trait SinglebandTileMergeMethods extends TileMergeMethods[Tile] {
case _ =>
self
}

def union(extent: Extent, otherExtent: Extent, other: Tile, method: ResampleMethod, unionF: (Option[Double], Option[Double]) => Double): Tile = {
val unionInt = (l: Option[Double], r: Option[Double]) => unionF(l, r).toInt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, mb the signature (l: Option[Int], r: Option[Int]) => Int is a bit more clear, any way expensive casts occur. I'm actually wondering what's the cost of those.


val combinedExtent = otherExtent combine extent
val re = RasterExtent(extent, self.cols, self.rows)
val gridBounds = re.gridBoundsFor(combinedExtent, false)
val targetCS = CellSize(combinedExtent, gridBounds.width, gridBounds.height)
val targetRE = RasterExtent(combinedExtent, targetCS)
val mutableTile = ArrayTile.empty(self.cellType, targetRE.cols, targetRE.rows)

self.cellType match {
case BitCellType | ByteCellType | UByteCellType | ShortCellType | UShortCellType | IntCellType =>
val interpolateLeft: (Double, Double) => Int = Resample(method, self, extent, targetCS).resample _
val interpolateRight: (Double, Double) => Int = Resample(method, other, otherExtent, targetCS).resample _
// Assume 0 as the transparent value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really an assumption here? In the original merge logic we checked it to merge new pixels into the new tile in case it's no data.

cfor(0)(_ < targetRE.rows, _ + 1) { row =>
cfor(0)(_ < targetRE.cols, _ + 1) { col =>
val (x,y) = targetRE.gridToMap(col, row)
val (l,r) = (interpolateLeft(x, y), interpolateRight(x, y))
mutableTile.set(col, row, unionInt(Some(l.toDouble), Some(r.toDouble)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set only if isData?

val v = unionInt(l.some, r.some)
if(isData(v)) mutableTile.set(col, row, v)

It's actually a debatable question, but the NoData handling for Doubles and Ints is different and can be hard to capture for users with the (Double, Double) => Double merge function passed. Macro relies on a type, users should figure out themselfes the underlying nodata handling logic.

}
}
case FloatCellType | DoubleCellType =>
val interpolateLeft: (Double, Double) => Double = Resample(method, self, extent, targetCS).resampleDouble _
val interpolateRight: (Double, Double) => Double = Resample(method, other, otherExtent, targetCS).resampleDouble _

// Assume 0.0 as the transparent value
cfor(0)(_ < targetRE.rows, _ + 1) { row =>
cfor(0)(_ < targetRE.cols, _ + 1) { col =>
val (x,y) = targetRE.gridToMap(col, row)
val (l,r) = (interpolateLeft(x, y), interpolateRight(x, y))
mutableTile.setDouble(col, row, unionF(Some(l), Some(r)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isData handling as well? (in case it's not implied that it's incapuslated into the union function definition)

}
}
case x if x.isFloatingPoint =>
val interpolateLeft: (Double, Double) => Double = Resample(method, self, extent, targetCS).resampleDouble _
val interpolateRight: (Double, Double) => Double = Resample(method, other, otherExtent, targetCS).resampleDouble _
cfor(0)(_ < targetRE.rows, _ + 1) { row =>
cfor(0)(_ < targetRE.cols, _ + 1) { col =>
val (x,y) = targetRE.gridToMap(col, row)
val l = interpolateLeft(x, y)
val r = interpolateRight(x, y)
val maybeL = if (isNoData(l)) None else Some(l)
val maybeR = if (isNoData(r)) None else Some(r)
mutableTile.setDouble(col, row, unionF(maybeL, maybeR))
}
}
case _ =>
val interpolateLeft: (Double, Double) => Int = Resample(method, self, extent, targetCS).resample _
val interpolateRight: (Double, Double) => Int = Resample(method, other, otherExtent, targetCS).resample _
cfor(0)(_ < targetRE.rows, _ + 1) { row =>
cfor(0)(_ < targetRE.cols, _ + 1) { col =>
val (x,y) = targetRE.gridToMap(col, row)
val l = interpolateLeft(x, y)
val r = interpolateRight(x, y)
val maybeL = if (isNoData(l)) None else Some(l.toDouble)
val maybeR = if (isNoData(r)) None else Some(r.toDouble)
mutableTile.set(col, row, unionInt(maybeL, maybeR))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not there be extra isNoData(self.getDouble(col, row)) checks? (in case it's not implied that it's incapuslated into the union function definition)

//if (l!=r) println(s"x => ${x}, y => ${y}, col => ${col}, row => ${row} | l,r => ${l}, ${r}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A forgotten comment?

}
}
}
mutableTile
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -31,4 +31,13 @@ abstract class TileFeatureMergeMethods[

def merge(extent: Extent, otherExtent: Extent, other: TileFeature[T, D], method: ResampleMethod): TileFeature[T, D] =
TileFeature(self.tile.merge(extent, otherExtent, other.tile, method), Semigroup[D].combine(self.data, other.data))

def union(
extent: Extent,
otherExtent: Extent,
other: TileFeature[T, D],
method: ResampleMethod,
unionF: (Option[Double], Option[Double]) => Double
): TileFeature[T, D] =
TileFeature(self.tile.union(extent, otherExtent, other.tile, method, unionF), Semigroup[D].combine(self.data, other.data))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really realted to this PR, but it looks like it's the time to import cats.syntax.semigroup._ :D

}
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,9 @@ trait TileMergeMethods[T] extends MethodExtensions[T] {
def merge(extent: Extent, otherExtent: Extent, other: T): T =
merge(extent, otherExtent, other, NearestNeighbor)


def union(extent: Extent, otherExtent: Extent, other: T, method: ResampleMethod, unionF: (Option[Double], Option[Double]) => Double): T

def union(extent: Extent, otherExtent: Extent, other: T, unionF: (Option[Double], Option[Double]) => Double): T =
union(extent, otherExtent, other, NearestNeighbor, unionF)
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/*
* Copyright 2016 Azavea
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package geotrellis.raster.merge

import geotrellis.raster._
import geotrellis.raster.testkit._
import geotrellis.raster.resample.NearestNeighbor
import geotrellis.vector.Extent

import org.scalatest.matchers.should.Matchers
import org.scalatest.funspec.AnyFunSpec

class TileUnionMethodsSpec extends AnyFunSpec
with Matchers
with TileBuilders
with RasterMatchers {
describe("SinglebandTileMergeMethods") {

it("should union two tiles such that the extent of the output is equal to the minimum extent which covers both") {
val cellTypes: Seq[CellType] =
Seq(
BitCellType,
ByteCellType,
ByteConstantNoDataCellType,
ByteUserDefinedNoDataCellType(1.toByte),
UByteCellType,
UByteConstantNoDataCellType,
UByteUserDefinedNoDataCellType(1.toByte),
ShortCellType,
ShortConstantNoDataCellType,
ShortUserDefinedNoDataCellType(1.toShort),
UShortCellType,
UShortConstantNoDataCellType,
UShortUserDefinedNoDataCellType(1.toShort),
IntCellType,
IntConstantNoDataCellType,
IntUserDefinedNoDataCellType(1),
FloatCellType,
FloatConstantNoDataCellType,
FloatUserDefinedNoDataCellType(1.0f),
DoubleCellType,
DoubleConstantNoDataCellType,
DoubleUserDefinedNoDataCellType(1.0)
)

for(ct <- cellTypes) {
val arr = Array.ofDim[Double](100).fill(5.0)
arr(50) = 1.0
arr(55) = 0.0
arr(60) = Double.NaN

val tile1 =
DoubleArrayTile(arr, 10, 10).convert(ct)
val e1 = Extent(0, 0, 1, 1)
val tile2 =
tile1.prototype(ct, tile1.cols, tile1.rows)
val e2 = Extent(1, 1, 2, 2)
val unioned = tile1.union(e1, e2, tile2, NearestNeighbor, (d1, d2) => d1.getOrElse(4))
withClue(s"Failing on cell type $ct: ") {
unioned.rows shouldBe (20)
unioned.cols shouldBe (20)
}
}
}
}
}