Merge pull request #3 from stacchain/verify-tree

Verify tree

Author: jonhealy1

CHANGELOG.md

@@ -9,13 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
-### Changed
-
-### Deprecated
-
-### Removed
-
-### Fixed
+- Added `verify` command to cli with accompanying script to ensure that the Merkle tree verification json produced by the `compute` command matches [#2](https://github.com/stacchain/stac-merkle-tree-cli/pull/2)
 
 ## [v0.3.0] - 2024-11-20
 

README.md

@@ -124,21 +124,27 @@ catalog/
 
 ### Basic Usage
 
-After installing the package, you can use the `stac-merkle-tree-cli` command to compute and add Merkle information to your STAC catalog.
+After installing the package, you can use the `stac-merkle-tree-cli` command to compute or verify Merkle information in your STAC catalog.
+
+### Commands:
+
+### 1. `compute`
+
+The compute command computes and adds Merkle information (`merkle:object_hash`, `merkle:root`, `merkle:hash_method`) to your STAC catalog.
 
 ```bash
-stac-merkle-tree-cli path/to/catalog_directory [OPTIONS]
+stac-merkle-tree-cli compute path/to/catalog_directory [OPTIONS]
 ```
 
 #### Parameters:
 
-- path/to/catalog_directory: (Required) Path to the root directory containing catalog.json.
+- `path/to/catalog_directory`: (Required) Path to the root directory containing `catalog.json`.
 
 #### Options:
 
-- --merkle-tree-file TEXT: (Optional) Path to the output Merkle tree structure file. Defaults to merkle_tree.json within the provided catalog_directory.
+- `--merkle-tree-file TEXT`: (Optional) Path to the output Merkle tree structure file. Defaults to `merkle_tree.json` within the provided catalog_directory.
 
-### Example
+#### Example
 
 Assuming your directory structure is as follows:
 
@@ -161,7 +167,7 @@ my_stac_catalog/
 Run the tool:
 
 ```bash
-stac-merkle-tree-cli my_stac_catalog/
+stac-merkle-tree-cli compute my_stac_catalog/
 ```
 
 Expected Output:
@@ -177,6 +183,43 @@ Processed Catalog: /path/to/my_stac_catalog/catalog.json
 Merkle tree structure saved to /path/to/my_stac_catalog/merkle_tree.json
 ```
 
+### 2. `verify`
+
+The `verify` command validates the integrity of a Merkle tree JSON file by recalculating `merkle:root` values and comparing them to the expected values.
+
+```bash
+stac-merkle-tree-cli verify path/to/merkle_tree.json
+```
+
+#### Parameters:
+
+- `path/to/merkle_tree.json`: (Required) Path to the Merkle tree JSON file to verify.
+
+#### Example:
+
+Run the command:
+
+```bash
+stac-merkle-tree-cli verify my_stac_catalog/merkle_tree.json
+```
+
+Example Output (Success):
+
+```bash
+Verification Successful: The merkle:root matches.
+```
+
+Example Output (Failure):
+
+```bash
+Verification Failed:
+ - Expected merkle:root: 5808b480d9bed10e7663d52c218571d053c7b5df42a5aefc11e216c66c711f77
+ - Calculated merkle:root: f0ed08b316b917a98c085e699c090af1cea964b697dd0bc44491ebced4d0006c
+Discrepancies found in the following nodes:
+ - Collection 'COP-DEM' has mismatched merkle:root.
+ - Catalog 'Catalogue' has mismatched merkle:root.
+```
+
 ## Merkle Tree Extension Specification
 
 This tool complies with the [Merkle Tree Extension Specification](https://github.com/stacchain/merkle-tree), which outlines how to encode STAC objects in a Merkle tree to ensure metadata integrity.
@@ -352,15 +395,19 @@ Contributions are welcome! If you encounter issues or have suggestions for impro
 
 ## Verification Steps
 
-### 1. Run the CLI Tool:
+### 1. Compute Merkle Tree
+
+Use the `compute` command to process your STAC catalog and generate a Merkle tree structure.
 
 ```bash
-stac-merkle-tree-cli path/to/catalog_directory
+stac-merkle-tree-cli compute path/to/catalog_directory
 ```
 
-### 2. Check the Output:
+#### Options:
+
+--merkle-tree-file <file_path>: Specify the output file name for the Merkle tree JSON (default is merkle_tree.json).
 
-- **Console Output**: You should see logs indicating the processing of Items, Collections, and the Catalog.
+#### Example Output:
 
 ```ruby
 Processed Item: /path/to/catalog_directory/collections/collection1/item1.json
@@ -373,17 +420,64 @@ Processed Catalog: /path/to/catalog_directory/catalog.json
 Merkle tree structure saved to /path/to/catalog_directory/merkle_tree.json
 ```
 
-- **Merkle Tree JSON**: Verify that the `merkle_tree.json` (or your specified output file) accurately represents the hierarchical structure of your STAC catalog with correct `merkle:object_hash` and `merkle:root` values.
+- The tool will generate a `merkle_tree.json` file (or the specified output file), which represents the hierarchical structure of your STAC catalog, including `merkle:object_hash` and `merkle:root` values.
+
+### 2. Verify Merkle Tree
+
+Use the verify command to validate the integrity of the generated Merkle tree JSON file.
+
+```bash
+stac-merkle-tree-cli verify path/to/catalog_directory/merkle_tree.json
+```
+
+#### Example Output (Success):
+
+```bash
+Verification Successful: The merkle:root matches.
+```
+
+#### Example Output (Failure):
+
+```bash
+Verification Failed:
+ - Expected merkle:root: 5808b480d9bed10e7663d52c218571d053c7b5df42a5aefc11e216c66c711f77
+ - Calculated merkle:root: f0ed08b316b917a98c085e699c090af1cea964b697dd0bc44491ebced4d0006c
+Discrepancies found in the following nodes:
+ - Collection 'COP-DEM' has mismatched merkle:root.
+ - Catalog 'Catalogue' has mismatched merkle:root.
+```
+
+### 3. Validate STAC Structure Updates
+
+Ensure that the STAC files (e.g., `catalog.json`, `collection.json`, item files) have been updated correctly:
+
+#### Catalog:
+
+- `catalog.json` should include:
+
+  - `merkle:object_hash`
+  - `merkle:root`
+  - `merkle:hash_method`
+
+#### Collections:
+
+- Each `collection.json` should include:
+  - `merkle:object_hash`
+  - `merkle:root`
+  - `merkle:hash_method`
+
+#### Items:
 
-### 3. Verify Integrity:
+- Each Item JSON should have `merkle:object_hash` within its properties field.
 
-- **Catalog**: Ensure that the `catalog.json` now includes `merkle:object_hash`, `merkle:root`, and `merkle:hash_method`.
+### 4. Verify Output File
 
-- **Collections**: Each `collection.json` should include `merkle:object_hash`, `merkle:root`, and `merkle:hash_method`.
+Review the generated merkle_tree.json file to confirm:
 
-- **Items**: Each Item's JSON should have `merkle:object_hash` within the properties field.
+- Proper hierarchical representation of the catalog.
+- Correct merkle:object_hash and merkle:root values for each node
 
-### 4. Run Tests:
+### 5. Run Tests:
 
 Ensure that all tests pass by executing:
 

example_catalog/catalog.json

@@ -56,7 +56,7 @@
     "https://stacchain.github.io/merkle-tree/v1.0.0/schema.json"
   ],
   "merkle:object_hash": "b14fd102417c1d673f481bc053d19946aefdc27d84c584989b23c676c897bd5a",
-  "merkle:root": "2c637f0bae066e89de80839f3468f73e396e9d1498faefc469f0fd1039e19e0c",
+  "merkle:root": "797bfc76d971db745b9b476d8c44e9fa021af889d5761476b74e982d69e7a3c2",
   "merkle:hash_method": {
     "function": "sha256",
     "fields": [

example_catalog/collections/COP-DEM/collection.json

@@ -50,7 +50,7 @@
       "href": "https://catalogue.dataspace.copernicus.eu/stac/collections/COP-DEM/items"
     }
   ],
-  "merkle:root": "fd2700dd6b39d254530a1a996d667620c4c7aacb69c86e9ea4ec71a7269f6aac",
+  "merkle:root": "aa7f89b29cb339032ec86d81d4090bdbd52199152fb657f50b08eec1b3234ee2",
   "merkle:hash_method": {
     "function": "sha256",
     "fields": [

example_catalog/merkle_tree.json

@@ -2,13 +2,13 @@
   "node_id": "Catalogue",
   "type": "Catalog",
   "merkle:object_hash": "b14fd102417c1d673f481bc053d19946aefdc27d84c584989b23c676c897bd5a",
-  "merkle:root": "2c637f0bae066e89de80839f3468f73e396e9d1498faefc469f0fd1039e19e0c",
+  "merkle:root": "797bfc76d971db745b9b476d8c44e9fa021af889d5761476b74e982d69e7a3c2",
   "children": [
     {
       "node_id": "COP-DEM",
       "type": "Collection",
       "merkle:object_hash": "17789b31f8ae304de8dbe2350a15263dbf5e31adfc0d17a997e7e55f4cfc2f53",
-      "merkle:root": "fd2700dd6b39d254530a1a996d667620c4c7aacb69c86e9ea4ec71a7269f6aac",
+      "merkle:root": "aa7f89b29cb339032ec86d81d4090bdbd52199152fb657f50b08eec1b3234ee2",
       "children": [
         {
           "node_id": "DEM1_SAR_DGE_30_20101215T103647_20130405T103047_ADS_000000_oCX9",

example_catalog_nested_items/catalog.json

@@ -56,7 +56,7 @@
     "https://stacchain.github.io/merkle-tree/v1.0.0/schema.json"
   ],
   "merkle:object_hash": "b14fd102417c1d673f481bc053d19946aefdc27d84c584989b23c676c897bd5a",
-  "merkle:root": "2c637f0bae066e89de80839f3468f73e396e9d1498faefc469f0fd1039e19e0c",
+  "merkle:root": "614dbf76bf77d22fa824867beb0980c6c2c44aa12252555a8208783923a7569d",
   "merkle:hash_method": {
     "function": "sha256",
     "fields": [

example_catalog_nested_items/collections/COP-DEM/collection.json

@@ -50,7 +50,7 @@
       "href": "https://catalogue.dataspace.copernicus.eu/stac/collections/COP-DEM/items"
     }
   ],
-  "merkle:root": "2f4aa32184fbe70bd385d5b6b6e6d4ec5eb8b2e43611b441febcdf407c4e0030",
+  "merkle:root": "7e990e84a86dc146547b6e3a3e35196f53e0123b6f216f871abd5d1f55d2ccc5",
   "merkle:hash_method": {
     "function": "sha256",
     "fields": [

example_catalog_nested_items/merkle_tree.json

@@ -2,13 +2,13 @@
   "node_id": "Catalogue",
   "type": "Catalog",
   "merkle:object_hash": "b14fd102417c1d673f481bc053d19946aefdc27d84c584989b23c676c897bd5a",
-  "merkle:root": "2c637f0bae066e89de80839f3468f73e396e9d1498faefc469f0fd1039e19e0c",
+  "merkle:root": "614dbf76bf77d22fa824867beb0980c6c2c44aa12252555a8208783923a7569d",
   "children": [
     {
       "node_id": "COP-DEM",
       "type": "Collection",
       "merkle:object_hash": "17789b31f8ae304de8dbe2350a15263dbf5e31adfc0d17a997e7e55f4cfc2f53",
-      "merkle:root": "2f4aa32184fbe70bd385d5b6b6e6d4ec5eb8b2e43611b441febcdf407c4e0030",
+      "merkle:root": "7e990e84a86dc146547b6e3a3e35196f53e0123b6f216f871abd5d1f55d2ccc5",
       "children": [
         {
           "node_id": "DEM1_SAR_DGE_30_20101212T230244_20140325T230302_ADS_000000_1jTi",

example_catalog_simple/catalog.json

@@ -0,0 +1,68 @@
+{
+  "stac_version": "1.0.0",
+  "type": "Catalog",
+  "id": "Catalogue",
+  "description": "Catalogue of data available in the platform",
+  "conformsTo": [
+    "https://api.stacspec.org/v1.0.0/core",
+    "https://api.stacspec.org/v1.0.0/collections",
+    "https://api.stacspec.org/v1.0.0/ogcapi-features",
+    "https://api.stacspec.org/v1.0.0/item-search",
+    "https://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core",
+    "https://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson"
+  ],
+  "links": [
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac",
+      "rel": "root",
+      "type": "application/json"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac",
+      "rel": "self",
+      "type": "application/json"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac/api",
+      "rel": "service-desc",
+      "type": "application/vnd.oai.openapi"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac/conformance",
+      "rel": "conformance",
+      "type": "application/json"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac/collections",
+      "rel": "data",
+      "type": "application/json"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac/search",
+      "title": "STAC search",
+      "rel": "search",
+      "type": "application/json",
+      "method": "GET"
+    },
+    {
+      "href": "https://catalogue.dataspace.copernicus.eu/stac/search",
+      "title": "STAC search",
+      "rel": "search",
+      "type": "application/json",
+      "method": "POST"
+    }
+  ],
+  "stac_extensions": [
+    "https://stacchain.github.io/merkle-tree/v1.0.0/schema.json"
+  ],
+  "merkle:object_hash": "b14fd102417c1d673f481bc053d19946aefdc27d84c584989b23c676c897bd5a",
+  "merkle:root": "f0ed08b316b917a98c085e699c090af1cea964b697dd0bc44491ebced4d0006c",
+  "merkle:hash_method": {
+    "function": "sha256",
+    "fields": [
+      "*"
+    ],
+    "ordering": "ascending",
+    "description": "Computed by including the merkle:root of collections and the catalogs own merkle:object_hash."
+  }
+}