faas-pipeline

Complex OpenFaaS function composition

faas-pipeline offers executing of complex pipelines on top of OpenFaaS. It does this by:

Defining its own pipeline definition language
Defining its own protocol for data exchange between functions
Accepting pipeline definitions together with initial payload

faas-pipeline can:

Process payloads concurrently. This is called ~split. The result of ~split operation is an array of results of all functions that were executed concurrently.
Process payloads sequentially. This is called ~pipe. ~pipe operation results in a new payload based on its received payload.
Include optional parameters, beside payload, when invoking each function.
Send custom metrics for each invoked function to FaaS' built-in Prometheus.

These can be combined in any number of ways. For example:

pipeline:
	~pipe:
		- ~split: # sends the received payload into fn1, fn2 and fn3
			fn1:
			fn2:
			fn3:
		- some-merging-fn: # this function acts on array of results from fn1, fn2 and fn3 with parameters of whatever1 and whatever2
			some-param: whatever1
			another-param: whatever2

Each compatible function must:

Expect to receive JSON with optional payload and params properties.
Return JSON with either error or result property and optional metrics property.

Execution stops on first error or when the entire pipeline has ran. Pipeline run thus either results in a free-form result or in a free-form error.