Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collections tweak #619

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 54 additions & 4 deletions docs/build/collections.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,20 @@ uploaded to a collection using the CLI.

## Collections Basics

Data is stored in Collections as key-value pairs, where the key is a unique
identifier for some data (like a UUID, or timestamp). The value is always a
string - although JSON objects will be automatically serialized to a string
using the Collections API.
:::tip

The Collections API is automatically available to all Workflows and does not
require any credentials. Authentication with the OpenFn platform is managed for
you.

You can use the Collections API with any adaptor.

:::

Data is stored in as key-value pairs, where the key is a unique identifier for
some data (like a UUID, or timestamp). The value is always saved as a string
(although you can pass JSON-compatible objects directly, which will be
automatically serialized by the Collections API).

Keys can be fetched in bulk and filtered by _pattern_. For example, the pattern
`2024*` will match all keys which start with `2024`. Designing keys to have an
Expand All @@ -74,6 +84,36 @@ collections.get('openfn-patient-registrations', '2024*').then(state => {
});
```

Returned items are written to state.data as an array of `[{ key, value }]`
pairs:

```js
{
"data": {
"20240102-5901257": {
"name": "Tom Waits",
"id": "5901257",
},
"20240213-0183216": {
"name": "Billie Holiday",
"id": "0183216",
}
}
}
```

If fetching a single item (ie, no `*` in the key), it will be written directly
to `state.data` with no key:

```js
{
"data": {
"name": "Billie Holiday",
"id": "0183216",
}
}
```

Every key permanently saves its creation date, so as well as fetching by
key-pattern, you can also filter keys by date. This example fetches all keys
created before 30th September 2024:
Expand Down Expand Up @@ -115,6 +155,16 @@ collections.set('openfn-demo', 'commcare-fhir-value-mappings', {
});
```

If setting multiple values at once, pass a key generator function instead of an
id to generate a key for each item. For example, if several value are saved in
an array on `state.data`:

```js
collections.set('openfn-demo', (patient, state) => patient.id, $.patients);
```

The key generator will be called with each value and must return a string key.

## Managing Collections

Collections can be created, destroyed or renamed from the Admin menu.
Expand Down
Loading