-
Notifications
You must be signed in to change notification settings - Fork 174
RabbitMQ
On the use of messaging services (i.e. RabbitMQ)
- Authors: Federico Stagni, Zoltan Mathe, Wojtek Ian Krzemien
Messaging systems like RabbitMQ can be useful for performing several tasks in DIRAC. This RFC is created for discussing use cases, and for integration within the DIRAC framework.
RFC 21 on the Evolution of Transformation System already gives an example of a possible use case. The idea is to achieve data driven through usage of MQ like RabbitMQ. Details are in the RFC.
RabbitMQ is the market's leader when coming to interactions with monitoring tools.
DIRAC does not provide, at the moment, a pilots logging system. Authors think that such system could be provided using a message queuing system. RabbitMQ appears to be the most suitable candidate.
Executors work in a similar fashion to MQ systems. It should be studied if MQs can do the same job, but better.
From now on, we use RabbitMQ's jargon, and we consider specifically the case of integration of RabbitMQ within the DIRAC framework. For an explanation of AMQP (one of the protocols supported by RabbitMQ, see https://www.rabbitmq.com/tutorials/amqp-concepts.html)
RabbitMQ has a server and a client part. The server part is host of the brokers (exchanges and queues), the client part is for sending (RMQ Producers) and receiving (RMQ Consumers) messages to the queues inside the servers. Accessing queues can be assimilated to accessing databases (or, better, accessing services that expose database functionalities). RMQ producers are clients, while RMQ Consumers can be considered, in DIRAC jargon, like any other DIRAC component (e.g. like agents, or better like executors).
The CS can be used for storing access info to RMQ queues. The following locations are advised:
/DIRAC/Systems/[System]/[Setup]/MessageQueueing/[QueueName]/User /DIRAC/Systems/[System]/[Setup]/MessageQueueing/[QueueName]/Host /DIRAC/Systems/[System]/[Setup]/MessageQueueing/[QueueName]/...
And
/LocalInstallation/MessageQueueing/Password
On the local cfg where DIRAC is running.
These locations are mimicking the locations that we use today for MySQL DBs and we believe that they can be enough on a first approximation. Exchanges can have similar configuration.
We'll also need to create a "MessageQueueing" utility as we have a "DB" utility, and maybe a "RabbitMQ" utility as we have a "MySQL" utility. These utilities should be used to encapsulate the RMQ logic of dealing with brokers (exchanges and queues).
In a second moment, we should also see if and how to add the installation of RabbitMQ servers to InstallTools, and of course we'll need it in the Externals.
Within the LHCbDIRAC extension of InstallTools, we already added the "consumer" component type (http://svnweb.cern.ch/world/wsvn/dirac/LHCbDIRAC/trunk/LHCbDIRAC/Core/Utilities/InstallTools.py). This can be easily backported, clearly.
Consumers can be run like any other component, so we can run/control them with runsv. But we need to be able to pass them options taken from the CS. So, we should create a "ConsumerModule" utility (similar to the "AgentModule") where we can add basic things like monitoring. We should also add a "dirac-consumer" script similar to what we have for "dirac-agent" or "dirac-service" script.
Clients should be able to put/get into RMQ queues by using username/password (this is the case for servers) but also using certificates or RFC proxies.
The implementation of RMQ is such that we need to register in its configuration the DNs of the users allowed to commit.
One of RabbitMQ’s supported plugins maps X.509 DNs to usernames: https://github.com/rabbitmq/rabbitmq-auth-mechanism-ssl RabbitMQ allows fine grained access control at the level of individual users, exchanges, and queues: http://www.rabbitmq.com/access-control.html
Then it's just finding if you can add the credentials of the user to the message, e.g. DN+ip for instance. That way we can prove origin of messages and react accordingly.
RabbitMQ uses OpenSSL for its TLS support, and you can turn on OpenSSL support for RFC proxies with an environment variable.
The exact configuration for a secure RabbitMQ server should be crafted with care.