this is more of a question, not an issue - how to install and manage multiple nodes ?

[virtuman]

I'm super green when it comes to docker, we currently run all production with chef but containerization was a good enough reason to consider switching over to docker.

When i found your repo - it made it a lot more obvious that it is so much easier to configure and manage things but I'm having a bit of a difficulty understanding how to deploy this setup over multiple nodes (all dedicated servers). Is it even possible?

Say we want to have 2 of MYSQL + REDIS servers and 2 separate web servers
we currently also have 2 haproxy load balancers (also dedicated servers)

I couldn't understand a few things:

1. If i set all of this up the way I described above - how will the machines know about each other? Ie. if haproxy is configured on web servers - then will they automatically pick up IPs that need to be connected to haproxy for mysql availability monitoring?
2. Is there a real benefit to having mutliple NGINX + PHP-FPM + MYSQL instances running on the same node vs 1 of each instances? or is this something that's been tested and performs better when more instances are available and that they are load balanced through the local installation of haproxy?
3. With the multip-IP DNS entries - it almost seems pointless to have additional LOAD BALANCER hardware when haproxy can reside on 2 of the same physical nodes that host web servers app?
4. are there any self-hosted utilities to monitor DOCKER resource utilization / service availability / etc.?
5. what's the best way to add some customizations to the included configs? for example NGINX, ie - we use GEOIP, IMAGE RESIZE, CACHE, REALIP nginx modules. What's the best practice to customize that and save it as a reusalbe image, would it still allow us to stay in sync with your repository changes ?

thank you very much.

[nazar-pc]

1. This repository contains Consul image for DNS and HAPtoxy for load balancing - they are super simple when it comes to use and provide service discovery functionality, so that you'll be able to see the whole network topology (and even few separate topologies); each container in this setup should be cluster-aware, for instance MariaDB image is not a regular one, this is already cluster with single node, as soon as you scale it up - cluster will grow and replicate automatically, the same about Consul configuration - new nodes join cluster on scale completely automatically
2. Depends on your setup, can't say anything about your specific use case, I do not have multi-server setups yet
3. Yes, you can even have multiple HAProxy instances to increase redundancy
4. Yes, there are few of them both open source and proprietary, there was a good talk on some of Docker conferences, just search for it
5. I'd suggest to create own image based on existing

Currently there are 2 issues with multi-server (Docker Swarm) setups:

• Ceph image is not finalized (and thus not documented, crucial in order to have automatic distributed file system for Nginx and PHP-FPM), you can track it here: CephFS image #5
• Docker 1.10 released few days ago changed fundamentally how hostnames are resolved and doesn't use /etc/hosts anymore, so images will not work as cluster until support for this will be added, you can track it here: Docker 1.10 support #11

Basically, these images are aimed to provide flawless automatic clusterization with zero-configuration in simplest cases, namely you execute docker-compose scale mariadb=21 and after some time (while replication happens) you'll get 20 MariaDB Galera cluster with 21 nodes.

Any contributions are welcomed:)

[virtuman]

Thank you so very much for such detailed reply.

i am a little confused with 1. and 2. - almost seems like the contradict each other. maybe I didn't ask the questions correctly, let me try to re-phrase it:

Your images allow to deploy multiple instances of the same service-daemon on the same machine (ie. physical node). It looks very cool and easy to do this but I'm trying to understand the benefit of have multiple NGINX or MYSQL instances on the same Node. Is there a reason for having 2 nginx daemons running on the same node, or is there a performance benefit of having multiple mysql daemons running on same node? I would imagine that running multiple mysql will in turn decrease amount of resources that can be dedicated to each instance of mysql, make it "less-performant"?

and the answer 2. seems like it's a part to the answer of the 1. or did i misunderstand it?
Did you mean that this can already be deployed across multiple physical nodes with some services running on one node and other services running on the other node and that they would auto-discover themselves using CONSUL DNS? or did you mean that this hasn't been done across multiple physical nodes?

and in 5. - when you say create own image, i'm assuming you mean docker commit or do you actually mean docker save ? would that render my image to be no longer upgradable from from yours? Ie. you mentioned that there will be some fixes to a couple of problems and I would like to be able to add all your fixes to my already running cluster. Is that even an option with Docker? (as i mentioned i'm still a super new to docker)

[nazar-pc]

I do not think there is any reason to have 2 instances on the same node on the same hardware. It is possible (you can even run multiple VMs, combine them in Docker Swarm cluster and setup cross-server replication), but not necessary should be done. The reasonable use case might be 2 MariaDB instances with very different configs and residing on different physical drives within the same server.

It cannot be deployed right now out of the box with Docker 1.10. With Docker 1.9 you can, for instance, create MariaDB cluster. But for Nginx you'll need to manually prepare volumes using distributed file system (Docker docs). Ceph image in this repository aims to provide Cephfs distributed file system for these container that will just work out of the box across multiple physical hosts, but it is not finished yet (it works with Docker 1.9 though).

No, I mean write your own Dockerfile that will extend existing image from this repository, just like images from this repository extends others (MariaDB image is based on official MariaDB image and so on).

When Docker 1.10 compatibility will be implemented and Ceph image finished - you'll be able to run multi-host cluster as simple as running docker-compose up -d