feat: Demonstrate asset datetime summary investigation

linz · Nov 10, 2021 · c5fa6ad · c5fa6ad
1 parent 614aa4c
commit c5fa6ad
Showing 1 changed file with 190 additions and 0 deletions.
diff --git a/extensions/linz/notebooks/asset-datetime-summary.ipynb b/extensions/linz/notebooks/asset-datetime-summary.ipynb
@@ -0,0 +1,190 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "4b29124d-9e52-4124-8b9f-efb60762c0dc",
+   "metadata": {},
+   "source": [
+    "# Creating asset summaries"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d38f9eb9-e10e-4903-9772-8b887e3cdef5",
+   "metadata": {},
+   "source": [
+    "We'll need a *LINZ extension schema validator:*"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "7b2d67c6-3313-4023-9752-49d133d86d59",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from json import load\n",
+    "from urllib.request import urlopen\n",
+    "\n",
+    "from jsonschema import Draft7Validator\n",
+    "\n",
+    "with urlopen(\"https://stac.linz.govt.nz/v0.0.10/linz/schema.json\") as schema_pointer:\n",
+    "    schema = load(schema_pointer)\n",
+    "\n",
+    "validator = Draft7Validator(schema)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5e32302e-6714-494f-bbe0-23a60d9b383e",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "The *collection example* is valid:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "3931b743-885d-47ec-aff4-48faa8c6da5b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with urlopen(\"https://stac.linz.govt.nz/v0.0.10/linz/examples/collection.json\") as example_pointer:\n",
+    "    example = load(example_pointer)\n",
+    "\n",
+    "validator.validate(example)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f0e21a05-f00f-4923-a457-b3a7f25907d8",
+   "metadata": {},
+   "source": [
+    "Can we *summarise assets?*"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "76ec8b0b-6d5a-4902-a0b1-6b1415b577f3",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "example[\"summaries\"][\"assets\"] = {\n",
+    "    \"created\": {\"minimum\": \"1999-01-01T00:00:00Z\", \"maximum\": \"2010-01-01T00:00:00Z\"},\n",
+    "    \"updated\": {\"minimum\": \"1999-01-02T00:00:00Z\", \"maximum\": \"2010-01-02T00:00:00Z\"},\n",
+    "}\n",
+    "validator.validate(example)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3cb90d1c-d25f-46e5-9885-c3d4f303ef09",
+   "metadata": {},
+   "source": [
+    "Yes, but *are such properties validated as normal summaries?* What if there's an invalid datetime or unexpected structure?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "2f043e5c-11a9-4ad8-9d6b-4ce8fe4cb1c0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "example[\"summaries\"][\"assets\"] = {\n",
+    "    \"created\": {\"minimum\": \"not a datetime\", \"maximum\": \"2010-01-01T00:00:00Z\"},\n",
+    "    \"updated\": None,\n",
+    "}\n",
+    "validator.validate(example)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "16355abb-5da3-4486-9ae7-4f450330c508",
+   "metadata": {},
+   "source": [
+    "Oops, that should've caused a *validation exception!* Let's instead extend the extension schema to validate asset summaries as normal summaries:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "4cd36c8b-ac96-4ea3-a4a2-0db92ef07675",
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ValidationError",
+     "evalue": "None is not valid under any of the given schemas\n\nFailed validating 'anyOf' in schema['allOf'][3]['properties']['summaries']['properties']['assets']['additionalProperties']:\n    {'anyOf': [{'allOf': [{'$ref': 'http://json-schema.org/draft-07/schema'}],\n                'minProperties': 1,\n                'title': 'JSON Schema',\n                'type': 'object'},\n               {'properties': {'maximum': {'title': 'Maximum value',\n                                           'type': ['number', 'string']},\n                               'minimum': {'title': 'Minimum value',\n                                           'type': ['number', 'string']}},\n                'required': ['minimum', 'maximum'],\n                'title': 'Range',\n                'type': 'object'},\n               {'items': {'description': 'For each field only the original '\n                                         'data type of the property can '\n                                         'occur (except for arrays), but '\n                                         \"we can't validate that in JSON \"\n                                         'Schema yet. See the sumamry '\n                                         'description in the STAC '\n                                         'specification for details.'},\n                'minItems': 1,\n                'title': 'Set of values',\n                'type': 'array'}]}\n\nOn instance['summaries']['assets']['updated']:\n    None",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mValidationError\u001b[0m                           Traceback (most recent call last)",
+      "\u001b[0;32m<ipython-input-5-a52ff90324ea>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0;34m\"$ref\"\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m\"https://schemas.stacspec.org/v1.0.0/collection-spec/json-schema/collection.json#definitions/summaries\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m }\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0mvalidator\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalidate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexample\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
+      "\u001b[0;32m/nix/store/flgn97ypbp74y7yvilld9wgggh1fzjwp-python3.8-jsonschema-3.2.0/lib/python3.8/site-packages/jsonschema/validators.py\u001b[0m in \u001b[0;36mvalidate\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    351\u001b[0m         \u001b[0;32mdef\u001b[0m \u001b[0mvalidate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    352\u001b[0m             \u001b[0;32mfor\u001b[0m \u001b[0merror\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0miter_errors\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 353\u001b[0;31m                 \u001b[0;32mraise\u001b[0m \u001b[0merror\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    354\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    355\u001b[0m         \u001b[0;32mdef\u001b[0m \u001b[0mis_type\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minstance\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
+      "\u001b[0;31mValidationError\u001b[0m: None is not valid under any of the given schemas\n\nFailed validating 'anyOf' in schema['allOf'][3]['properties']['summaries']['properties']['assets']['additionalProperties']:\n    {'anyOf': [{'allOf': [{'$ref': 'http://json-schema.org/draft-07/schema'}],\n                'minProperties': 1,\n                'title': 'JSON Schema',\n                'type': 'object'},\n               {'properties': {'maximum': {'title': 'Maximum value',\n                                           'type': ['number', 'string']},\n                               'minimum': {'title': 'Minimum value',\n                                           'type': ['number', 'string']}},\n                'required': ['minimum', 'maximum'],\n                'title': 'Range',\n                'type': 'object'},\n               {'items': {'description': 'For each field only the original '\n                                         'data type of the property can '\n                                         'occur (except for arrays), but '\n                                         \"we can't validate that in JSON \"\n                                         'Schema yet. See the sumamry '\n                                         'description in the STAC '\n                                         'specification for details.'},\n                'minItems': 1,\n                'title': 'Set of values',\n                'type': 'array'}]}\n\nOn instance['summaries']['assets']['updated']:\n    None"
+     ]
+    }
+   ],
+   "source": [
+    "schema[\"definitions\"][\"fields\"][\"properties\"][\"summaries\"][\"properties\"][\"assets\"] = {\n",
+    "    \"$ref\": \"https://schemas.stacspec.org/v1.0.0/collection-spec/json-schema/collection.json#definitions/summaries\"\n",
+    "}\n",
+    "validator.validate(example)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bdaa981e-5c50-4f9f-a7d2-6b0bef7c314f",
+   "metadata": {},
+   "source": [
+    "Looks like that worked – the assets summary is validated like any other summary. Let's make sure by reverting to the valid summary:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8e7cfdb5-1232-40fd-957b-9b01f7448401",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "example[\"summaries\"][\"assets\"] = {\n",
+    "    \"created\": {\"minimum\": \"1999-01-01T00:00:00Z\", \"maximum\": \"2010-01-01T00:00:00Z\"},\n",
+    "    \"updated\": {\"minimum\": \"1999-01-02T00:00:00Z\", \"maximum\": \"2010-01-02T00:00:00Z\"},\n",
+    "}\n",
+    "validator.validate(example)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6ee3378f-5507-4831-8611-ce56c52304ac",
+   "metadata": {},
+   "source": [
+    "And we're golden!"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python3 - stac",
+   "language": "python",
+   "name": "ipython_stac"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}