Kafka transport

[weekens]

For now Comedy uses simple point-to-point channels for message exchange. There should be an option to use Kafka message broker as an underlying transport for more advanced and robust messaging.

[ryardley]

Can I suggest the possibility to make it generic so that you could use RabbitMQ if required.

[weekens]

@ryardley, yeah, of course - there will be an abstract interface and the actual transport will be pluggable.

[ryardley]

FYI We are evaluating this framework and this was a blocker that came up.

[weekens]

Is this due to lack of message durability? Just to make it clear, what features that Kafka/RabbitMQ provides are you missing?

[ryardley]

The main gist of our hesitation is that hooking into our preexisting messaging service allows us to delegate cluster management to our orchestration provider.

[weekens]

Yes, there is often a tradeoff between flat clustering with message bus (we can name it "microservices") and hierarchical clustering with direct messaging (we can name it "actors"). Each has it's own advantages and disadvantages.

But, in fact, you can take advantage of both, if each of your micoservices is implemented with Comedy actors. On one hand, you still use your message bus and orchestration. On the other hand, you have a clear hierarchical work separation in each of your microservices with the option of further clustering.

This is exactly how we go in SAYMON. We currently have 11 top-level microservices that are managed with PM2 and exchange messages through Kafka, but each of these 11 microservice is implemented with Comedy with a built-in option to further cluster any of it's actors. Note that initially it was only 1 top-level service. Splitting it into 11 separate services was extremely easy exactly because it was initially implemented with Comedy. Also we still have an option to run all this stuff as a single process on environments with limited resources. Summing up, we have very flexible clustering thanks to actor model.

[ryardley]

Just to give a bit more background the main reason we were struggling with this is that Kubernetes has an issue where you cannot allow traffic via services in port ranges. kubernetes/kubernetes#23864 . Comedy appears to require this to manage it's actor cluster when child nodes establish a connection with the parent. We figured using an event bus or message queue might solve this problem. There might be ways around this but it has pushed us to look at other options.

[weekens]

Yes, that's right. Comedy allocates a dynamic port for each remote actor and does direct messaging between parent and child through this port.

We could implement an option of proxying message exchange with remote actor through comedy-node, but this would introduce a potential bottleneck and a single point of failure for clustered remote actors. If you still think this is a useful option, feel free to create an issue.