Ranges?

[m-mohr]

It comes up over and over again, the range values. Recently in #31. A common example seems to be something like:

• "categorical" no-data values (e.g. -1, -3)
• One range with continuous data (e.g. >= 0)

Should we cater for this? I think the simplest solution would be to allow for value an array with two values that can on one side ne null (for open-ended range) as defined also by the STAC Collection extents.

Then you could have something like:

{
...
          "unit": "mm",
          "classification:classes": [
            {
              "value": -1,
              "name": "missing-value",
              "description": "Missing value (no-data)",
              "nodata": true
            },
            {
              "value": -3,
              "name": "no-coverage",
              "description": "No coverage (no-data)",
              "nodata": true
            },
            {
              "value": [0, null],
              "name": "data",
              "description": "Actual data values in mm"
            }
          ],
...
}

[m-mohr]

One issue that might occur is to define > 0, then you'd need to do something like [0.000000000000000000000000000000000000000001, null]

So an alternative would be to allow a minimal subset of JSON Schema (minimum, maximum, exclusiveMinimum, exclusiveMaximum) and allow an object instead of an array, e.g. for > 0:

{
  "exclusiveMinimum": 0
}

[drwelby]

would it be terrible to be super explicit like

"classification:classes": [
            {
              "value": -1,
              "name": "missing-value",
              "description": "Missing value (no-data)",
              "nodata": true
            },
            {
              "values": [-3, 1, 7],
              "name": "no-coverage",
              "description": "No coverage (no-data)",
              "nodata": true
            },
            {
              "range": [0, null], # or json schema object...
              "name": "data",
              "description": "Actual data values in mm"
            }
          ],

[m-mohr]

Yes, I think it is terrible ;-) How would you decide whether [1,2] is a range from 1 to 2 or the two categorical values 1 and 2?

[drwelby]

that's why the keys are explicit value, values, and range

[m-mohr]

Ooooh, I didn't catch that difference. Sorry. I don't think that is necessary, it is more complicated to describe and read but doesn't give any obvious benefit to me?

[drwelby]

I just don't like that [1, null] is a magic range while [1, 255] is ambiguous as a range or a list of values.

I do really like ranges that are json schema objects.

and of course I still don't like putting ranges into classes ;), but I want to at least get somewhere with the concept.

[m-mohr]

I did not consider lists of values in my proposal yet because you can emulate them just by having the classes multiple times while you can't reasonable express continuous ranges. But yeah, the full-fledged solution would be:

• integer: single categorical value
• array of integers: multiple categorical values
• json schema like object: continuous ranges

Not sure whether we should cater for all, the use cases I've heard about so far were only continuous ranges.

[m-mohr]

Alternatively, we could try something like the following although I'm not sure that would be valid in raster as we give "made-up" statistics / exclude no-data values from statistics. It also feels less intuitive.

      "raster:bands": [
        {
          "unit": "mm",
          "data_type": "float64",
          "statistics": {
            "minimum": 0
          },
          "classification:incomplete": true,
          "classification:classes": [
            {
              "value": -1.0,
              "name": "missing-value",
              "description": "Missing value (no-data)",
              "nodata": true
            },
            {
              "value": -3.0,
              "name": "no-coverage",
              "description": "No coverage (no-data)",
              "nodata": true
            }
          ]
        }
      ],

Thoughts, @emmanuelmathot ?

[drwelby]

Any opinion on preferring

  "maximum": 10.5,
  "exclusiveMaximum": true

versus

  "exclusiveMaximum": 10.5

[m-mohr]

I'd follow JSON Schema as we already use it in other places, which (except for the outdated draft-4) use numbers instead of boolean flags: https://json-schema.org/understanding-json-schema/reference/numeric.html#range

[drwelby]

ok, didn't realize I was looking at an older draft 👍

[drwelby]

"classification:incomplete": true, is interesting to me because saying

"range": [0, null], # or json schema object...
              "name": "data",
              "description": "Actual data values in mm"

seems redundant or out of place when the data set is describing rain fall in mm and a negative depth doesn't make sense.

[m-mohr]

Yeah, I'm liking it more the more I'm thinking about it but it's less flexible and covers only some use cases, I assume. Also, it doesn't seem so wrong to exclude no-data values from statistics because they are usually always just made-up values for the file format that doesn't support encoding them properly. I guess we only need to clarify in raster that no-data values are invalid pixel values and as should not be reflected in statistics etc. On the other hand, statistics are usually real min/max values while what we want to describe here are theoretical min and max values. For example, if you have a raster with precipation values, the min and max could be 1, 5 and 10 so min/max are 1 and 10, although the potential range is 0 to infinity (mostly). But maybe that's not an issue?!