Skip to content

Getting started

Jump to bottom
lucyschrader edited this page Mar 19, 2023 · 31 revisions

On this page

About Te Papa's collections

The Collections API gives your system access to over 800,000 items in the Museum of New Zealand Te Papa Tongarewa catalogue. Over 150,000 items have images attached (a total of over 200,000 images - including over 100,000 images which may be freely downloaded).

Te Papa collects items that have iconic value to Aotearoa New Zealand:

  • Arts
  • Taonga Māori (cultural treasures)
  • Pacific
  • New Zealand heritage
  • Natural history

The catalogue contains rich metadata (descriptions) about each item, for example what a painting depicts or where a specimen was collected from.

To find out more about the data coverage, image availability, copyright, etc. see About Collections Online (the Collections Online catalogue is built using this Collections API).

Basic API usage

You need a free authentication key to use the Collections API - either a one-hour guest key (from the API browser), or register to get your own permanent key.

The best place to start is with the /object endpoint - all the images and entities branch out from there. For example:

https://data.tepapa.govt.nz/collection/object/?q=kiwi

Try out our API Browser to view the live data and to help you design common API request IRIs.

Note: It is not possible to access the API directly in your web browser without manipulating the request headers it sends. Two popular software alternatives are Postman and curl.

Requests

The Collections API is based on REST principles: data resources are accessed via standard HTTP requests in UTF-8 format to an API endpoint.

The following REST verbs are used:

  • GET - for retrieving resources
  • POST - for sending complex retrieval requests

API endpoints

This is a summary of the endpoints. For full details, parameters, error codes, etc. see the API reference documentation.

Collection resources

  • /object - the catalogue (cultural heritage items and biodiversity specimens)
  • /topic - Te Papa documents describing a topic
  • /media - images of collection items

Related entity resources

  • /agent - people and organisations associated with collection items
  • /place - locations associated with collection items
  • /document - publications associated with collection items
  • /fieldcollection - the event when a specimen was collected out in the field

Related concept resources

  • /taxon - a Linnaean biological classification used for specimens in the Te Papa collections
  • /category - an abstract topic or subject that an item is about

Structural resources

  • /group - a set of things grouped together by Te Papa staff

Search

  • /search - search across all the above resources (each resource can also be searched directly)
  • /scroll - a special search for retrieving large sets of records

API keys

To get a permanent API key: please read our Collections API Terms and Conditions (pdf), then Register for a Collections API Key.

To get a one-hour guest key for prototyping: call the API without authentication and the response will contain a guest key for you to use.

GET https://data.tepapa.govt.nz/collection/object

{
  "error":"Unauthorized",
  "guestToken":"1234567890"
}

When you receive your API key (guest or permanent), you must use it in the header of every request.

Guest key (Authorization and Bearer)

GET https://data.tepapa.govt.nz/collection/object
Authorization: Bearer 1234567890
Accept: application/json

Permanent key (x-api-key)

GET https://data.tepapa.govt.nz/collection/object
x-api-key: ABCDEFG
Accept: application/json

See below for details on Rate Limiting of your API requests.

Responses

Data formats

Each API endpoint provides one data format:

JSON

  • Default data format
  • Optionally use HTTP request header: Accept: application/json

The /object endpoint provides an additional data format:

JSON-LD

  • A special version of JSON for Linked Data - read about the Collections API Linked Data
  • Use HTTP request header: Accept: application/ld+json
  • (Note that JSON-LD is currently experimental)

Data objects

These are the main data object types, for full details see the Object Model.

Collection resources

Related entities

Related concepts

Structural types

Object model diagram

All objects contain a type and title field, so these are a useful place to start during your prototyping. Later you might choose specific fields depending on the type.

A quick way to search across both collection objects and specimens at once is to query: kiwi AND additionalType:PhysicalObject

Nested objects (for example the Person who painted this Object) contain a few, selected data fields, but to see the full nested objects you will need to request each of them from the API separately.

For example, the nested artist Person record (inside the "Northland panels" artwork Object record below) contains the name and dates to assist basic searching, and an href field containing the link to the full record in the API:

GET https://data.tepapa.govt.nz/collection/object/38664

{
    "id": 38664,
    "type": "Object",
    "collection": "Art",
    "identifier": "1978-0009-1/A-H to H-H",
    "title": "Northland panels",
    "production": [
        {
            "type": "Production",
            "title": "Colin McCahon; artist; 1958; Auckland",
            "contributor": {
                "id": 1502,
                "type": "Person",
                "title": "Colin McCahon",
                "verbatimBirthDate": "1 August 1919",
                "verbatimDeathDate": "27 May 1987",
                "pid": "tepapa:collection/agent/1502",
                "iri": "http://tepapa.govt.nz/collection/agent/1502",
                "href": "https://data.tepapa.govt.nz/collection/agent/1502"
            },
            "createdDate": "1958",
            "role": "artist",
        }
    ],
}

Following the href link retrieves the full record with all of its data fields:

GET https://data.tepapa.govt.nz/collection/agent/1502

{
    "id": 1502,
    "type": "Person",
    "title": "Colin McCahon",
    "birthPlace": "Timaru",
    "verbatimBirthDate": "1 August 1919",
    "deathPlace": "Auckland",
    "verbatimDeathDate": "27 May 1987",
    "ethnicity": [
        "New Zealand European"
    ],
    "nationality": [
        "New Zealander"
    ],
    "familyName": "McCahon",
    "givenName": "Colin",
    "gender": "Male",
    "pid": "tepapa:collection/agent/1502",
    "iri": "http://tepapa.govt.nz/collection/agent/1502",
    "href": "https://data.tepapa.govt.nz/collection/agent/1502"
}

Rate limiting

Your permanent API key is usable in small to medium applications - up to 10 requests per second. Contact us if you need higher performance.

If you get a response status code 429 it means you have sent too many requests - it resets automatically once your requests are below the rate limit again.

Searching

Specify your query terms in the q query string parameter. You can run a search against a single API endpoint or use the /search endpoint for a wide search.

GET https://data.tepapa.govt.nz/collection/search/?q=kiwi&size=1

{
  "results": [
    {
      "id": "1514",
      "type": "Category",
      "title": "kahu kiwi",
      "creditLine": "Matauranga Māori Thesaurus",
      "prefLabel": "kahu kiwi",
      "scopeNote": "Kiwi feather cloak",
      "pid": "tepapa:collection/category/1514",
      "iri": "http://tepapa.govt.nz/collection/category/1514",
      "href": "https://data.tepapa.govt.nz/collection/category/1514",
      "_meta": {
        "created": "2011-10-31T05:08:15.000+0000",
        "modified": "2012-05-24T04:29:49.000+0000",
        "qualityScore": 1
      },
      "_api": {
          "score": 18.588493
      }
    }
  ],
  "_metadata": {
    "resultset": {
      "count": 1999,
      "from": 0,
      "size": 1
    }
  }
}

Basic query syntax

Query type Example
Keyword q=kiwi
Phrase q="new plymouth"
Any word q=batch crib
All words q=batch AND crib
Wildcards q=aptery*
Range q=id:[400 TO 500]
q=_meta.created:>2016-03
Field search q=title:crib
q=prefLabel:"new plymouth"
q=title:(john AND smith)
Limit by field search q=kiwi AND collection:TaongaMāori

For more details on searching see:

Version 1 search limitations

  • Field search is not possible against nested fields, for example collection:Art is possible, but not production:mccahon or production.contributor.title:mccahon (however all nested text is searchable in general searches)
  • Some fields require the exact case, for example collection:Art is fine, but not collection:art.

Pagination

You can retrieve large result sets in chunks by making multiple requests to retrieve each 'page'. Use the from and size query string parameters (or their aliases offset and limit).

Note that from starts from 0. The default page size is 100 and the maximum is 10,000.

Scrolling for large result sets

Standard search queries (above) have a fixed maximum limit of 50,000 results - this means you can page through the first 50,000 but never retrieve the 50,001st result. We have added this limit so that we can keep the API platform more stable under heavy usage.

To page through large result sets use a _scroll endpoint instead:

  • First, POST your scroll query and specify how many minutes to keep the 'scroll' alive (how long it will take you to consume all the results - the default is 1 minute) You need to ensure redirects are disabled for this request, or you'll get a 403 Forbidden error.
  • The API response contains a 303 status code and a link to your temporary 'scroll' in the response's Location header. It also contains the first page of your results.
  • Each scroll page contains (up to) 1,000 records. We recommend using this page size or smaller for the best performance.
POST https://data.tepapa.govt.nz/collection/search/_scroll/?q=kiwi&duration=2

Status: 303
Location: /scroll/DnF1ZXJ5k...m83bzNuRVE=?duration=2
{
  "results": [
    {
    }
  ],
  "_metadata": {
    "resultset": {
      "count": 1999,
      "from": 0,
      "size": 1000
    }
  }
}
  • Make multiple GET requests against that location to gradually consume all your results (until you receive a 204 'No Content' status code).
GET https://data.tepapa.govt.nz/collection/scroll/DnF1ZXJ5k...m83bzNuRVE=?duration=2

Status: 303
Location: /scroll/DnF1ZXJ5k...m83bzNuRVE?duration=2
{
  "results": [
    {
    }
  ],
  "_metadata": {
    "resultset": {
      "count": 1999,
      "from": 1000,
      "size": 1000
    }
  }
}

GET https://data.tepapa.govt.nz/collection/scroll/DnF1ZXJ5k...m83bzNuRVE=?duration=2

Status: 204

Scroll documentation:

Image credits

API Reference

Development Project

Clone this wiki locally