Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create JSONata processor #165

Open
gregfurman opened this issue Nov 16, 2024 · 0 comments
Open

Create JSONata processor #165

gregfurman opened this issue Nov 16, 2024 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@gregfurman
Copy link
Collaborator

Context

JSONata is a powerful query language that can be used to operate on JSON data. Namely, it has a very rich feature library that would lend itself well to be used on Bento message data.

Motivation

  • JSONata is syntax is extremely powerful and lends itself very well to JSON data.
  • There is already a rich feature set for operating on numeric, array, string, boolean data types as well as datetime-based processing.
  • In addition, JSONata is a turing complete functional language.

Task

Create a JSONata processor component (similar to the jq processor) that can operate on Bento message data. The blues/jsonata-go library seems like the best option here.

In addition, this component should:

  • Process batches of messages.
  • Handle errors JSONata errors and when a $error](https://docs.jsonata.org/object-functions#error) is thrown.
  • Optional stretch goal: If possible, it could be useful to create some connection between JSONata and Bloblang such that we can call JSONata functions in bloblang.

Note: JSONata does not mutate JSON data, instead creating new JSON objects.

Example usage

Flatten a deeply nested array:

jsonata: |
  $.**
# In: [[[[0]]] ,1 , [2], [3, [4]], 5]
# Out:  [0, 1, 2, 3, 4, 5]

FInd the total cost of orders:

jsonata: |
   $sum(orders.(price*quantity))
# In: 
# {
#   "orders": [
#     {"price": 10, "quantity": 3},
#     {"price": 0.5, "quantity": 10},
#     {"price": 100, "quantity": 1}
#   ]
# }
# Out: 135

Reverse and reduce a list of JSON objects:

jsonata: |
   $reverse($[].id)
# In: [{"id": "id-2"}, {"id": "id-3"}, {"id": "id-4"}, {"id": "id-1"}]
# Out: ["id-1", "id-4", "id-3", "id-2"]

Sum the items array and return a new reduced object:

jsonata: |
   {"total": $sum(items)}
# In: {"items": [7, 8, 6, 5], "name": "Mr Bento"}
# Out: {"total": 26}

Get all cities in WA:

jsonata: |
   {"Cities": locations[state="WA"].name ~>  $sort() ~> $join(", ")}

# In:
# {
#   "locations": [
#     {"name": "Seattle", "state": "WA"},
#     {"name": "New York", "state": "NY"},
#     {"name": "Bellevue", "state": "WA"},
#     {"name": "Olympia", "state": "WA"}
#   ]
# }
# Out: {"Cities": "Bellevue, Olympia, Seattle"}
@gregfurman gregfurman added enhancement New feature or request good first issue Good for newcomers labels Nov 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

1 participant