Skip to content

Commit

Permalink
sinks and sources doc revisions
Browse files Browse the repository at this point in the history
  • Loading branch information
cardenaso11 authored and ffakenz committed May 17, 2024
1 parent 381eac8 commit 3429c7f
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions docs/docs/getting-started/event-sinks-and-sources.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ Some use-cases exist, where many features of the Hydra platform are useful, but

2. The alternative would be to instead make do with running a full Hydra node, and preparing a persistence file before the Hydra runs, or parsing the file as it is written. This comes at the cost of control, code duplication, resource usage, and still relies on interfacing with an unstable external API (the persistence file on-disk).

To better enable these use-cases, Hydra supports the concept of alternate Event Sinks and a single alternate Event Source. These respectively represent a way to seed the initial transactions a Hydra node loads upon startup (a generalization of restoring from the persistence file), and ways to persist new transactions processed at run-time (a generalization of appending to the persistence file).
To better enable these use-cases, Hydra supports the concept of alternate Event Sinks and a single alternate Event Source. These respectively represent ways to persist new transactions processed at run-time (a generalization of appending to the persistence file), and a way to seed the initial transactions a Hydra node loads upon startup (a generalization of restoring from the persistence file).

Multiple Event Sinks may be used simultaneously, but currently only one Event Source may be used at a time. The Event Source is only loaded upon startup. Each Event Sink is run upon each new transaction. Currently, the order must be specified by customizing the order of the Event Sink list in the Hydra node source code, in the eventSinks parameter to hydrate, invoked in Hydra.Node.Run.run [here](https://github.com/SundaeSwap-finance/hydra/blob/4785bd86a03b92ba8fa8fb34c9d485a1e2f4f7d7/hydra-node/src/Hydra/Node/Run.hs#L104)

The default Hydra file-based persistence is implemented as an Event Sink and Source pair. They do not need to be used in tandem; it is possible to use the default Event Source which processes previous transactions from a file on disk, combined with an Event Sink which could, for example, store new transactions on S3, on several machines, or not at all.

Currently, there is no CLI API to toggle which sources and sinks are utilized, this must be done by the implementor of the sources and sinks. See the source and sink example, [here](https://github.com/SundaeSwap-finance/hydra/blob/4785bd86a03b92ba8fa8fb34c9d485a1e2f4f7d7/hydra-node/src/Hydra/Node/Run.hs#L97), where the Event Sinks and Source are toggled by added CLI Options.

To see an example of S3 and AWS Kinesis Event Sources and Sinks, see the doug_hydra_changes branch [here](github.com/SundaeSwap-finance/hydra). In particular, see TODO: new link once the branch is in a more publishable state. Also, I'm not sure if we have this branch public yet due to Gummiworm code living in the same code.
To see an example of S3 and AWS Kinesis Event Sources and Sinks, see the doug_hydra_changes branch [here](github.com/SundaeSwap-finance/hydra). In particular, see [hydra-node/src/Hydra/Events/AWS/Kinesis.hs](https://github.com/SundaeSwap-finance/hydra/blob/f27e51c001e7b64c3679eab4efd9f17f08db53fe/hydra-node/src/Hydra/Events/AWS/Kinesis.hs)

To construct an Event Sink, construct an EventSink e m object, where m is the monad (like IO), that the Event Sink will run in, and e is the (polymorphic) event type. There is only one field in the EventSink record, corresponding to the monadic action to take upon a new Event. An example that outputs the event to AWS Kinesis is available [here](https://github.com/SundaeSwap-finance/hydra/blob/598b20fcee9669a196781f70e02e13779967e470/hydra-node/src/Hydra/Events/AWS/Kinesis.hs#L85)

Expand Down

0 comments on commit 3429c7f

Please sign in to comment.