Streams is a light weight, simple stream processing library. While Kafka is the main use case for Streams, it is flexible enough to be used for any form of processing from any source.
You can install streams using go get
go get github.com/msales/streamsStreams breaks processing into the following basic parts.
-
Message is a message in the system, consisting of a key, value and context.
-
Sources reads and handles position from a data source.
-
Processor processes the data, optionally passing it on or marking the sources position. A sink is just a processor the does not forward the data on.
-
Pipe gives processors an abstract view of the current state, allowing Messages to flow through the system.
https://medium.com/@rafamnich/getting-started-with-streams-v3-b9ab36fb9d54