This repository has been archived by the owner on Feb 16, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 8
Add Usage Recommendations section #18
Merged
Merged
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Usage Recommendations Fragment | ||
|
||
This fragment describes conditions under which the model publishers believe the model will perform | ||
as expected. | ||
|
||
While the [Model Training] section gives users a clear picture of the training conditions, the model | ||
publisher may also have evidence or intuition that the model will also perform well when used in a | ||
different context. For instance, a model might have been trained on data from a particular | ||
geographic region and time period, but based on some combination of intuition, previous experience, | ||
and informal experimentation, the model publisher may be confident that it will perform similarly in | ||
a new geographic region. This section is meant to capture that information in a systematic way, | ||
while also providing publishers the opportunity to describe their recommendations in a more | ||
free-form style. | ||
|
||
## Directory Contents | ||
|
||
* [`usage-fragment.md`](./usage-fragment.md) - The detailed Usage Recommendations Fragment | ||
specification. | ||
|
||
[Model Training]: ../training |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
# Usage Recommendations Specification | ||
|
||
The Usage Recommendations section defines conditions under which the model publishers believe the | ||
model will perform as expected. It is important to note that these are *recommendations* and that | ||
they **in no way provide any guarantee of model performance under the given conditions**. | ||
|
||
If no Usage Recommendation section is provided, then users should assume that the model will only | ||
perform as expected under conditions that match those described in the [Model Training] section. | ||
This also applies to any fields that are not present in the Usage Recommendations section. For | ||
instance, if no `temporal` field is provided, users should assume the model will only perform as | ||
expected for data in the time range given in the [Model Training] section. | ||
|
||
## Usage Recommendation Fields | ||
|
||
| Field Name | Type | Description | | ||
|------------------|--------------------------|-------------------------------------------------------------------------| | ||
| spatial | [Spatial Extent Object] | The spatial extents within which the model should perform as expected. | | ||
| temporal | [Temporal Extent Object] | The temporal extents within which the model should perform as expected. | | ||
| description | string | A human-readable description of conditions under which the model should perform as expected. | | ||
duckontheweb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
### Spatial Extent Object | ||
|
||
| Element | Type | Description | | ||
| ------- | ------------ | -------------------------------------------------------------------- | | ||
| bbox | \[\[number]] | **REQUIRED.** Potential *spatial extents* within which the model should perform as expected. | | ||
|
||
The `bbox` element is an *array* whose elements are themselves arrays. Each inner array is a bounding | ||
box that describes a spatial extent within which the model should perform as expected. | ||
|
||
The format of the inner bounding box arrays follows the specification outlined in the **bbox** | ||
section of the [STAC Spatial Extent Object] documentation. However, the `bbox` element defined in | ||
this spec differs from the `bbox` element in the STAC Spatial Extent Object in that there is no | ||
first element that always represents the overall spatial extent. Such an overall extent might give | ||
users the incorrect impression that the model should perform as expected anywhere within that | ||
spatial extent, when in fact there may be only specific areas in which this is true. | ||
|
||
### Temporal Extent Object | ||
|
||
| Element | Type | Description | | ||
| -------- | ------------------ | --------------------------------------------------------------------- | | ||
| interval | \[\[string\|null]] | **REQUIRED.** Potential *temporal extents* within which the model should perform as expected. | | ||
|
||
The `interval` element is an *array* whose elements are themselves arrays. Each inner array is an | ||
interval array containing 2 elements, each of which may be either `null` or a timestamp. Timestamps | ||
must follow the specifications outlined in the [STAC Temporal Extent Object] documentation. | ||
|
||
The format of the inner interval arrays follows the specification outlined in the [STAC Temporal | ||
Extent Object] documentation. However, the `interval` element defined in this spec differs from | ||
the `interval` element in the STAC Temporal Extent Object in that there is no first element that | ||
always represents the overall temporal extent. Such an overall extent might give users the | ||
incorrect impression that the model should perform as expected anywhere within that temporal range, | ||
when in fact there may be only specific ranges in which this is true. | ||
|
||
|
||
[Model Training]: ../training | ||
[Spatial Extent Object]: #spatial-extent-object | ||
[Temporal Extent Object]: #temporal-extent-object | ||
[STAC Spatial Extent Object]: https://github.com/radiantearth/stac-spec/blob/v1.0.0-rc.4/collection-spec/collection-spec.md#spatial-extent-object | ||
[STAC Temporal Extent Object]: https://github.com/radiantearth/stac-spec/blob/v1.0.0-rc.4/collection-spec/collection-spec.md#temporal-extent-object |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A short sentence/paragraph could be added, noting that the section can also provide possible conditions, where the model will not perform.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated in a829170 to have separate
recommendations
andcautions
sections.