During the 2020-2021 COVID-19 pandemic I took some time to improve my own place now that I am spending much more time in it. One thing I found myself doing was this project to better document and maintain my home network, and have data to diagnose potential issues. This meant hooking up data tracking tools like Prometheus and Grafana. This also ties into a better backup solution than I've had in the past.
I want to be able to have answers to questions such as:
- When did the Internet go down, is it on our side, or the ISP's side?
- Is it just DNS specifically or is it completely down?
- What is our Power & Water consumption?
- Is the Freezer working correctly?
- What is our air quality like?
Our current ISP is Wave-G where we have 1Gbit down & up. The fiber is terminated in our apartment building's electrical closet on each floor, it's then run to our unit via CAT-5e where it's terminated with a female ethernet port. From that we then have have CAT-5e going into our Synology RT2600ac router which acts as our Firewall, DHCP host, switch, and Wireless 5GHz WiFi access point. The RT2600ac feeds into a number of ethernet wall-ports which are all around our apartment. We use a Netgear R7000 as a switch in our tiny livingroom/office.
We have a Raspberry Pi 4 8GB as our primary server currently.
Our primary network is running on 192.168.0.1/24.
Both our Synology RT2600ac and Netgear R7000 currently run stock firmware.
The Raspberry Pi runs Raspberry Pi OS Lite x64, you can find everything directly installed via the install.sh setup/update script. This largely consists of Docker.
We use docker-compose to run all of our services. Everything inside docker runs on the 10.0.0.0/28 subnet. Each service can be accessed via servicename.raspberrypi.local.
Services:
- Pi-hole We run Pi-hole as our primary DNS server for adblocking at the DNS level. This then passes through to 8.8.8.8, Google's DNS.
- nginx-proxy
We have nginx-proxy running as an ingress (in the kubernetes sense) to expose all of our services on port 80, on different domain names. Each service must state what port they expose by using the
VIRTUAL_HOSTandVIRTUAL_PORTenvironment variables. Pi-hole then needs to know about the host names for DNS purposes via theextra_hostslist. - Grafana Grafana is our primary data visualizer, log viewer, and alerting tool. Our grafana is provisioned via the configuration files located here, all changes to dashboards and datasources must be submitted to that directory.
- Portainer We use Portainer to get an overview of our little cluster. I treat it similarly to Kubernetes Dashboard.
- Prometheus Time series Database, this is the datasource for Grafana. It scrapes point-in-time metrics from telegraf and stores them. Prometheus's configuration is located in here.
- Telegraf Telegraf collects and exposes metrics which are then scraped via Prometheus and we can then view in Grafana. Telegraf's configuration is located in here.
- Syncthing Syncthing is an open source Dropbox/One Drive style file synchronization service. We use this to continously backup our devices to the home server for onsite backup.
- Loki "Like Prometheus, but for logs." This indexes and stores logs for later viewing and alerting with Grafana. Loki's configuration is located in here.
- Promtail Watches and parses the logs via systemd-journal and passes them to Loki for storage. Promtail's configuration is located in here.
We use the Diagrams as Code library for Python to generate our home network diagrams. See diagrams/README.md for more information.
The services folder lives in /mnt/external/services on the Raspberry Pi. We connect to it via SSH with ssh pi@raspberrypi.
To push updates to the server we currently run sync.sh from the remote machine.
Then while connected to the raspberry pi, we run the following:
# Navigate to the services folder
$ cd /mnt/external/services
# Updates and runs all services
$ run.sh- Rack mounting everything.
- Get a standalone firewall and switch.
- Migrate towards Kubernetes with k3s and multiple raspberry pi's.
- Setup alerting.
- Improved offsite backup.
- VPN support. (Wireguard?)
- Media streaming.
- Better deployment for services (prior to switching to k3s).
- Better documentation, diagrams, runbooks, etc.
- Improve security.
This repository will be kept up to date as I improve my home network.