Monitor Nodes outside the swarm cluster

[hatched-DavidMichon]

Hi,

Actually I have 3 swarm clusters, all separated in different VPC on AWS. One of the swarm is for the tools (Prometheus, Grafana, Jenkins, and so on). The 2 others are for production and staging.

Today, it's setup like this: on AWS I first created private hosted zones for my production and staging stack. On those hosted zones I created SRV entries for my exporters (node-exporters, cadvisor, etc..). Then I created scrape config that point to those SRV entries to dynamically get the list of internal node IPs so prometheus can scrape metrics from them.

This is not ideal because the list of IPs (on the SRV entries) are not build up dynamically as I add/remove nodes from my clusters. To be dynamic it would require to create a service that listen on docker events to know when nodes are added or removed to update those SRV entries. Moreover, if dockerflow/docker-flow-monitor restart, it does not get alerted from those 2 stacks because it looks only on the current swarm cluster it's running on.

To have my alerts from production and staging I have to restart dockerflow/docker-flow-swarm-listener on those 2 stacks so it send the alerts config to prometheus.

Ideally dockerflow/docker-flow-monitor should accept a list of LISTENER_ADDRESS to call dockerflow/docker-flow-swarm-listener on multiple hosts. Same for the DF_GET_NODES_URL.

What do you think? Does it makes sense for you? That would be very helpful!

[vfarcic]

The thing is that DFM calls DFSL only when it starts. After that, it assumes that DFSL will send it info about new or updated services. I don't think there is an initiative to invert the order.

[hatched-DavidMichon]

Thanks for your response @vfarcic. I don't mean of inverting the order. It's totally fine that DFM call DFSL only when it starts and then DFSL keep DFM updated. What would be beneficial is for DFM to be able to call multiple DFSL source when it starts.

[hatched-DavidMichon]

Any update on this request?

[vfarcic]

The project is mostly abandoned. I moved all my work to Kubernetes and it seems that the rest of the contributors moved as well.

[hatched-DavidMichon]

@vfarcic I see, thanks for your response. This project was abandoned because there's another similar project existing somewhere or because you (and contributors to this project) think there's no future for it? Or even more broadly that docker swarm is inferior than Kubernetes and that's the way to go?

I get all your books, fantastic job by the way. We build up our stack on docker swarm back then and of course, moving to Kubernetes is something that frequently comes in to the table. Seeing project we rely on being abandoned is kind of a red flag..

[vfarcic]

I believe that Swarm is out of the game and it was very painful for me to abandon it in my professional work. I'd love for Swarm to thrive, but the reality is different. Almost everyone adopted Kubernetes. On top of that, Kubernetes as the project continues receiving huge support (commits) from the community while development of Swarm is almost non-existent. Personally, I do not believe that Swarm has a future outside very limited use cases. Schedulers (Swarm, Mesos, Nomad, and Kubernetes) fought and only one is left alive (Kubernetes).

As for this project... It solves one of the problems of Swarm (mostly as a workaround) just like other Docker Flow projects solve others. Kubernetes' architecture already includes solutions to those problems. Prometheus, for example, can simply hook into Kube API and get all the data it needs, so there's no need for something like this project.

I would strongly recommend moving to Kubernetes.

[vfarcic]

As a side note, this project does not have to be abandoned. The fact that I don't use it anymore does not mean that others don't. That's the spirit of open source. Whoever needs something added or fixed can make a pull request. If no one does that, there is not much need for the project.

[hatched-DavidMichon]

I agree with you regarding Kubernetes. I would have liked for docker swarm to thrive but today the reality is other as you said.

And yes I totally understand that this project don't "have" to be abandoned. I was only concerned that most or all of the contributors moved away.