[RFC] Public API for stactools packages

[gadomski]

This is a Request For Comment following the Rust template. This is just a proposal -- please comment/suggest/advance as you see fit!

Summary

The public API of a stactools package consists of:

• Its Python API, except for functions that are only used for its CLI interface, and
• The structure of any generated STAC objects

The package's command-line interface is not part of the public API.

Each stactools package hosts examples of its own generated objects, so downstream users and other developers can "see" the public metadata API.

Motivation

The semver documentation begins by laying out the requirements for good versioning:

For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise.

As more and more folks are using/modifying stactools packages, we need to define a general public API for all stactools packages, so downstream users can have some trust in their version numbers as they use packages in their own data pipelines. This also will help developers make decisions about which features to change or which new features to include.

Guide-level explanation

The public API of a software packages defines which features/attributes downstream users can rely upon with some expectations of stability. In particular, a good public API will allow downstream users to take up new bug fixes or features without breaking existing code.

Because stactools packages are generally used to generate metadata, we consider those generated metadata as "first class" members of the package. Any breaking changes to those metadata are considered breaking changes to the public API, and require a MAJOR version change.

STAC examples

Let's use a hypothetical stactools package, stactools.foo, which provides a single function stactools.foo.stac.create_item(href: str) -> pystac.Item. For a given GeoTIFF data.tif, create_item creates an Item that looks like this:

{
  "stac_version": "1.0.0",
  "stac_extensions": [],
  "type": "Feature",
  "id": "data",
  "bbox": [
    -106,
    40,
    -105,
    41
  ],
  "geometry": {
    "type": "Polygon",
    "coordinates": [
      [
        [
          -106,
          40
        ],
        [
          -105,
          40
        ],
        [
          -105,
          41
        ],
        [
          -106,
          41
        ],
        [
          -106,
          40
        ]
      ]
    ]
  },
  "properties": {
    "datetime": "2022-03-29T22:38:00Z",
    "created": "2022-03-29T23:38:00Z"
  },
  "links": [],
  "assets": {
    "data": {
      "href": "data.tif",
      "type": "image/tiff; application=geotiff",
      "roles": [
        "data"
      ]
    }
  }
}

Breaking changes

A change to any existing attribute (except for created) is a breaking change and requires a MAJOR release, e.g. this is breaking:

5c5
<   "id": "data",
---
>   "id": "item-data",

is a breaking change. The only exception is created on non-assets (i.e. at the top level for Catalogs and Collections, and in properties for Items). E.g. this is non-breaking:

41c41
<     "created": "2022-03-29T23:38:00Z"
---
>     "created": "2022-03-29T23:40:00Z"

EDIT 2022-03-31: It is also a non-breaking changing to re-order a list where list order is not meaningful, e.g. roles.

Backwards-compatible feature additions

Adding new attributes, including extensions, is a backwards-compatible feature addition and requires a MINOR release:

3c3,5
<   "stac_extensions": [],
---
>   "stac_extensions": [
>     "https://stac-extensions.github.io/file/v2.1.0/schema.json"
>   ],
50c52,53
<       ]
---
>       ],
>       "file:size": 42

Backwards-compatible bugfixes

Changing values to make them correct is considered a backwards-compatible bugfix and requires a PATCH release. E.g. let's say the datetime had the wrong year; this would be a backwards compatible bugfix:

40c40
<     "datetime": "2022-03-29T22:38:00Z",
---
>     "datetime": "2021-03-29T22:38:00Z",

Note the use of the word correct. Merely bringing the value in line with best practices is not enough to be considered a bugfix -- the value needs to be well and truly incorrect.

Python API

The Python API is treated normally w.r.t public API. Any declared entity that does not begin with an _ is considered part of the public API. While its from a different language, I find the Cargo book's explanation of API stability helpful -- if someone has a Python version of that, please point me that way and I'll update this RFC.

Command-line interface

The command-line interface is meant to be used manually in one-off operations. If downstream users need stability and repeatability, they should use the Python API. Therefore, the command-line interface, and any Python functions used strictly for the CLI (e.g. click-decorated functions) are not part of the public API.

Unresolved questions

• Where should this be documented? https://github.com/stactools-packages/stactools-packages.github.io? https://github.com/stactools-pacakges/template? https://github.com/stac-utils/stactools? All three? Somewhere else?
• If there are multiple parties interested in updating a given stactools package, how do we resolve metadata decisions in a mutually satisfactory way?
• Is there anything we can do in stactools itself to help make this easier for package developer? (spoiler: I think there is)
• Should generated STAC objects have pointers back to the stactools package and version that generated them?

Future possibilities

• Each package could eventually produce different "flavors" of STAC objects given a certain set of conventions (e.g. Microsoft does it this way, other provider does it this other way, etc).
• Canonical examples of each package could be hosted so people could verify backwards-compatilbity themselves.