Skip to content

Commit

Permalink
Merge branch 'master' into fix/update-collection-paths-for-odr-tde-801
Browse files Browse the repository at this point in the history
  • Loading branch information
blacha authored Sep 11, 2023
2 parents 7334e25 + 168d3f7 commit 1b3c11a
Show file tree
Hide file tree
Showing 6 changed files with 30 additions and 21 deletions.
11 changes: 10 additions & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,18 @@ jobs:
mask-aws-account-id: true
role-to-assume: ${{ secrets.AWS_CI_ROLE }}

- name: AWS Configure ODR
if: github.ref == 'refs/heads/master'
uses: aws-actions/configure-aws-credentials@v2
with:
aws-region: ap-southeast-2
mask-aws-account-id: true
role-to-assume: ${{ secrets.AWS_ODR_DATAMANAGER_ROLE }}
role-chaining: true

# Sync STAC files only on push to 'master'
- name: Sync STAC
uses: docker://ghcr.io/linz/argo-tasks:v2
if: github.ref == 'refs/heads/master'
with:
args: stac-sync /github/workspace/stac/ s3://linz-imagery/
args: stac-sync /github/workspace/stac/ s3://nz-imagery/
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# New Zealand Imagery

Toitū Te Whenua makes New Zealands most up-to-date publicly owned aerial imagery freely available to use under an open licence. You can access this through the [LINZ Data Service](https://data.linz.govt.nz/data/category/aerial-photos/?s=n), [LINZ Basemaps](https://basemaps.linz.govt.nz/#@-41.8899962,174.0492437,z5) or Amazon AWS.
Toitū Te Whenua makes New Zealand's most up-to-date publicly owned aerial imagery freely available to use under an open licence. You can access this through the [LINZ Data Service](https://data.linz.govt.nz/data/category/aerial-photos/?s=n), [LINZ Basemaps](https://basemaps.linz.govt.nz/#@-41.8899962,174.0492437,z5) or the [Registry of Open Data on AWS](https://registry.opendata.aws/nz-imagery/).

This repository contains STAC Collection metadata for each imagery dataset, as well as some guidance documentation:
- [Naming](docs/naming.md) covers the s3://linz-imagery bucket naming structure

- [Naming](docs/naming.md) covers the s3://nz-imagery bucket naming structure
- [Tools](docs/tools.md) covers some of the STAC ecosystem tools that can be used to interact with our STAC Catalog
- [Usage](docs/usage.md) shows how TIFFs can be interacted with from S3 using GDAL, QGIS, etc


## AWS Access

Toitū Te Whenua provides a public requester pays bucket `s3://linz-imagery` in `ap-southeast-2`.
Toitū Te Whenua owns and maintains a public bucket which is sponsored and shared via the [Registry of Open Data on AWS](https://registry.opendata.aws/nz-imagery/) `s3://nz-imagery` in `ap-southeast-2`.

Using the [AWS CLI](https://aws.amazon.com/cli/) anyone with a AWS account can access all of the imagery mentioned inside of this repository.
Using the [AWS CLI](https://aws.amazon.com/cli/) anyone can access all of the imagery specified in this repository.

```
aws s3 ls s3://linz-imagery --request-payer requester
aws s3 ls --no-sign-request s3://nz-imagery/
```

## License
Expand Down
4 changes: 2 additions & 2 deletions docs/naming.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ Imagery is named according to the majority region that imagery covers while also
7.5cm RGB imagery covering Auckland captured in 2022

```
s3://linz-imagery/auckland/auckland_2022_0.075m/rgb/2193/collection.json
s3://nz-imagery/auckland/auckland_2022_0.075m/rgb/2193/collection.json
```

10m Satellite RGB imagery covering the North Island captured in 2023

```
s3://linz-imagery/new-zealand/north-island_2023_10m/rgb/2193/collection.json
s3://nz-imagery/new-zealand/north-island_2023_10m/rgb/2193/collection.json
```

### Characters
Expand Down
2 changes: 1 addition & 1 deletion docs/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ A list of tools is available on [the STAC website](https://stacspec.org/en/about

To deploy STAC Browser, you can follow [this documentation](https://github.com/radiantearth/stac-browser#get-started).

For a quick start, you can use [the demo instance](https://radiantearth.github.io/stac-browser/), pass the link to the New Zealand Imagery STAC Catalog `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/catalog.json` and click on the `Load` button.
For a quick start, you can use [the demo instance](https://radiantearth.github.io/stac-browser/), pass the link to the New Zealand Imagery STAC Catalog `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/catalog.json` and click on the `Load` button.

### Filtering the data

Expand Down
18 changes: 9 additions & 9 deletions docs/usage.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Get the link to the TIFF

The STAC Catalog, https://linz-imagery.s3-ap-southeast-2.amazonaws.com/catalog.json, is the entry point of the LINZ imagery data. It points to the different STAC Collections representing a dataset.
The STAC Catalog, https://nz-imagery.s3-ap-southeast-2.amazonaws.com/catalog.json, is the entry point of the LINZ imagery data. It points to the different STAC Collections representing a dataset.

> **_Note:_** The files on our AWS S3 bucket `linz-imagery` can be accessible with a `s3` path `s3://linz-imagery` or a `https` URL `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/`.
> **_Note:_** The files on the AWS S3 bucket `nz-imagery` can be accessible with a `s3` path `s3://nz-imagery` or a `https` URL `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/`.
## Manually navigate through the STAC Catalog to a get a TIFF

Expand All @@ -20,13 +20,13 @@ This is an example using the "Wellington 0.075m Urban Aerial Photos (2021)" data
}
```

2. The Collection can be accessible from this GitHub repository, `stac/wellington/wellington_2021_0.075m/rgb/2193/collection.json` or from `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/collection.json`. From the Collection get an Item link in the `links` list:
2. The Collection can be accessible from this GitHub repository, `stac/wellington/wellington_2021_0.075m/rgb/2193/collection.json` or from `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/collection.json`. From the Collection get an Item link in the `links` list:

```json
{ "rel": "item", "href": "./BQ31_500_040071.json", "type": "application/json" }
```

3. The Item can be accessible from `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.json`. A list of assets can be found:
3. The Item can be accessible from `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.json`. A list of assets can be found:

```json
"assets": {
Expand All @@ -39,7 +39,7 @@ This is an example using the "Wellington 0.075m Urban Aerial Photos (2021)" data

```

In this case, the TIFF can be accessible from `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff`
In this case, the TIFF can be accessible from `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff`

> **_Note:_** Our TIFF files and STAC Item files use the same base name. Having the link to the STAC Item, you can determine the TIFF link by changing its suffix from `.json` to `.tiff`.
Expand All @@ -55,13 +55,13 @@ This tool is described in [this page](tools.md).

GDAL allows you to run command on a file that is not on your local machine using a virtual file systems path:

- `/vicurl/https://linz-imagery.s3-ap-southeast-2.amazonaws.com/[PATH]`
- `/vsis3/s3://linz-imagery/[PATH]`
- `/vicurl/https://nz-imagery.s3-ap-southeast-2.amazonaws.com/[PATH]`
- `/vsis3/s3://nz-imagery/[PATH]`

## gdalinfo

```bash
gdalinfo /vsicurl/https://linz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff
gdalinfo /vsicurl/https://nz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff
```

<details>
Expand Down Expand Up @@ -150,7 +150,7 @@ Band 4 Block=512x512 Type=Byte, ColorInterp=Alpha
1. In QGIS, open the "Data Source Manager" (press `ctrl+L`)
2. Select "Protocol: HTTP(S)" as the "Source Type"
3. Select "HTTP/HTTPS/FTP" as the "Protocol Type"
4. Paste the HTTPS URL to the TIFF, for example `https://linz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff`
4. Paste the HTTPS URL to the TIFF, for example `https://nz-imagery.s3-ap-southeast-2.amazonaws.com/wellington/wellington_2021_0.075m/rgb/2193/BQ31_500_040071.tiff`
5. Click on the "Add" button and wait for the file to load.

![Data Source Manager](img/usage/qgis_data-source-manager.png)
Expand Down
4 changes: 2 additions & 2 deletions template/catalog.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
{
"stac_version": "1.0.0",
"type": "Catalog",
"id": "linz-imagery",
"id": "nz-imagery",
"description": "Toitū Te Whenua Land Information New Zealand makes New Zealand's publicly owned aerial and satellite imagery archive freely available to use under an open licence. This public S3 bucket has been made available to enable bulk access and cloud-based data processing. You can also access the imagery through the LINZ Data Service or LINZ Basemaps.",
"links": [
{ "rel": "self", "href": "https://linz-imagery.s3.ap-southeast-2.amazonaws.com/catalog.json" },
{ "rel": "self", "href": "https://nz-imagery.s3.ap-southeast-2.amazonaws.com/catalog.json" },
{ "rel": "root", "href": "./catalog.json" }
]
}

0 comments on commit 1b3c11a

Please sign in to comment.