Mantl Packaging

This repository contains Hammer specs for building generic Mantl utilities.

Table of Contents

• Mantl Packaging
  • Dynamic Configuration
    • Per-node Configuration
  • Packages
    • Core
      • generate-certificate
      • traefik
      • docker-gc
      • mantl-storage-setup
      • smlr
      • nomad
      • zookeepercli
    • Consul Packages
      • consul
      • consul-ui
      • consul-template
      • consul-cli
    • Vault Packages
      • vault
      • vault-mantl
    • Mantl Packages
      • mantl-dns
    • Calico packages
      • calico
    • Distributive Packages
      • distributive
      • distributive-
    • Mesos Packages
      • mesos
      • mesos-master
      • mesos-master-dynamic
      • mesos-agent
      • mesos-agent-dynamic
    • Mesos Frameworks
      • marathon
      • marathon-dynamic
    • Mantl-Cadvisor Packages
      • mantl-cadvisor
    • Beats
      • filebeat
      • journalbeat
  • Building

Dynamic Configuration

Dynamic configuration is performed with Consul. The {package}-dynamic entries in this README describe the key spaces they look for to render configuration to disk. Be aware that most of these daemons need to be restarted when configuration changes, so account for that when you're changing keys.

Per-node Configuration

In addition to the documented keys under each package, you can set per-node global options for these packages with certain flags. These will be documented in the config files if not set, but here's a short list:

Key	Description
config/nodes/{node}/external_ip	node's external IP address
config/nodes/{node}/internal_ip	node's internal IP address
config/nodes/{node}/hostname	node's hostname

Packages

Core

generate-certificate

spec

A script to generate certificates with a number of sensible defaults set.

traefik

spec

Træfɪk is a modern HTTP reverse proxy and load balancer made to deploy microservices with ease. It supports several backends (Docker, Mesos/Marathon, Consul, Etcd, Zookeeper, BoltDB, Rest API, file...) to manage its configuration automatically and dynamically.

• Traefik's README

docker-gc

spec

Systemd timers that purge unsued docker containers, images, and volumes.

To check the status of the timers: systemctl list-timers

To run manually: systemctl start docker-gc-containers.service systemctl start docker-gc-images.service systemctl start docker-gc-volumes.service

mantl-storage-setup

spec

A script that configures LVM on a system based on information in /etc/mantl/filesystems.d.

smlr

spec

smlr waits for service dependencies.

• smlr's README

nomad

spec

A Distributed, Highly Available, Datacenter-Aware Scheduler

• Nomad's Project Page

zookeepercli

spec

Zookeepercli is a simple, lightweight, dependable CLI for ZooKeeper

• zookeepercli's README

Consul Packages

consul

spec

Packages consul.io with systemd. Check /etc/sysconfig/consul for configuration.

consul-ui

spec

Standalone web UI for Consul

consul-template

spec

Packages consul-template with systemd. Check /etc/sysconfig/consul-template for configuration.

consul-cli

spec

Packages consul-cli with the currently released version.

A Distributed, Highly Available, Datacenter-Aware Scheduler

• Nomad's Project Page

Vault Packages

vault

spec

Packages vault.io with systemd. Check /etc/sysconfig/vault for configuration.

vault-mantl

spec

Packages mantl.io specific scripts for Vault.

Script	Description
vault-bootstrap.sh	Initialize vault and store keys in Consul
vault-health-check.sh	Consul health check script for Vault
vault-register-with-consul.sh	Register the Vault service with Consul
vault-unseal.sh	Read tokens from Consul and unlock Vault

Mantl Packages

mantl-dns

spec

DNS setup with dnsmasq and Consul

Calico packages

calico

spec

Distributive Packages

distributive

Package containing the Distributive binary.

distributive-

Distributive checklists for various Mantl components are included in the distributive directory.

Mesos Packages

mesos

spec

The base Mesos package, including bindings. There is no configuration in this package.

mesos-master

spec

The mesos master process. This is a configuration-only package, and will provide the mesos-master service by depending on mesos. mesos-master is configured via environment variables in /etc/sysconfig/mesos-master.

mesos-master-dynamic

spec

Makes mesos-master dynamic by populating it with consul-template (spec).

Available configuration:

Key	Description	Default
config/mesos/agents/{node}/principal and config/mesos/agents/{node}/secret	agent principal(s) and secret(s), respectively	not set
config/mesos/frameworks/{name}/principal and config/mesos/frameworks/{name}/secret	framework principal(s) and secret(s), respectively	not set
config/mesos/master/extra_options	extra command-line options to pass to mesos-master	not set
config/mesos/master/firewall_rules	see Mesos docs	{}
config/mesos/master/nodes/{node}/options	same as options, but per-node	not set
config/mesos/master/options	any key from the configuration options. Value will be uppercased to become an environment variable.	not set

This package assumes that authentication will be done globally, and so will not pay attention to unsetting the authentication per-node; it must be done globally. It also pays attention to both the authenticate_slaves and authenticate_agents flags for backwards compatibility.

This package also uses internal_ip, external_ip, and hostname from the Per-node Configuration. Do note that you can override the values set in this way in the configuration by overriding them in config/mesos/master/nodes/{node}/options.

mesos-agent

spec

The mesos agent process (formerly mesos-slave). This package name is being changed in advance of the upstream change to mesos-agent, and will call the appropriate binaries for the version of Mesos provided. This is a configuration-only package, and will provide the mesos-agent service by depending on mesos. mesos-agent is configured via environment variables in /etc/sysconfig/mesos-agent.

mesos-agent-dynamic

spec

Makes mesos-agent dynamic by populating it with consul-template (spec).

Available configuration:

Key	Description	Default
config/mesos/agent/extra_options	extra command-line options to pass to mesos-agent	not set
config/mesos/agent/firewall_rules	see Mesos docs	{}
config/mesos/agent/nodes/{node}/options	same as options, but per-node	not set
config/mesos/agent/options	any key from the configuration options. Value will be uppercased to become an environment variable.	not set
config/mesos/agents/{node}/principal and config/mesos/agents/{node}/secret	agent principal and secret, respectively. This uses the value of the Consul node to determine the key.	not set

Authentication for this package will be enabled if the principal and secret are both set.

This package also uses internal_ip and hostname from the Per-node Configuration. Do note that you can override the values set in this way in the configuration by overriding them in config/mesos/agent/nodes/{node}/options.

Mesos Frameworks

marathon

spec

Marathon, a cluster-wide init and control system for services in cgroups or Docker containers. Marathon can be controlled with environment variables in /etc/sysconfig/marathon, the available options are documented in the Marathon command-line flags documentation.

marathon-dynamic

spec

Makes marathon dynamic by populating it with consul-template (spec).

Available configuration:

Key	Description
config/marathon/options/{key}	any key from the command line flags. Value will be uppercased to become an environment variable.
config/marathon/hosts/{node}/options/{key}	the same as marathon/config/{key}, but the flags will only be applied to the specified node

Mantl-Cadvisor Packages

mantl-cadvisor

spec

cAdvisor customized for Mantl clusters

Beats

filebeat

spec

journalbeat

spec

Building

If you're on linux, run hammer to build all of the packages, which will end up in out. If you're on another platform, run ./build.sh to fire up a Vagrant VM that will provision itself with hammer and do the same.

If you add a new package, be sure to run make scripts/paths so that it will be picked up by CI.