This demo showcases Firecracker's agility and high-density capabiliies. It's only been run on an EC2 I3.metal host (the defaults start 4000 microVMs) with an Ubuntu host OS, from an Ubuntu client.
Deviations form this setup will probably lead to issues and/or sub-par performance. If you want to help us support the demo on more platforms ... we take pull requests :)
Get this repo on a EC2 i3.metal instance. Open two terminals/ssh-connections to the instance.
will show a heatmap of network traffic done by each microVM.
python3 microvm-tiles.pywill control the rest of the demo.
Raise the maximum processes limit.
sudo cat >> /etc/security/limits.conf <<EOL
ec2-user soft nproc 16384
ec2-user hard nproc 16384
EOLReload the ssh session to have the new limit applied.
Create 4000 TAPs, configure networking for them and start 4k iperf3 servers each bound to their respective TAP.
sudo ./0.initial-setup.sh 4000Use 6 parallel threads to configure and start 4000 microVMs. Each thread will get an equal slice of the 4k total and sequentially configure and issue the start command for each microVM.
The script will report total duration as well as mutation rate.
# start a total of 4k uVMs from 6 parallel threads
./parallel-start-many.sh 0 4000 6
# ... wait for it ... should take around 60 seconds ... watch the heatmapEach microVM has a workload (iperf client) and will run it in a loop with a random sleep between iterations.
Looking at the heatmap you should see six 'snakes' advancing which are the microVMs that have just been powered up and are doing their first iteration of the workload. Once that's done, the random sleep will lead to random lighting of the heatmap.
Pick a number 0 <= ID < 4000. For this example 42 was chosen.
ID="42"
# get the IP for that microVM
ifconfig fc-$ID-tap0 | grep "inet "
inet 169.254.0.170 netmask 255.255.255.252 broadcast 0.0.0.0
# IP of microVM on other side is *one less*
ssh -i xenial.rootfs.id_rsa [email protected]You're now inside the microVM. Do as you please.
Let's make it stand out in the heatmap.
# stop the workload service
localhost:~# rc-service demo-workload stop
* Stopping demo-workload ... [ ok ]
# manually run iperf with a higher bandwidth than the rest
localhost:~# iperf3 -c $(./gateway-ip.sh) -b 104857600
# check out the heatmapThis microVM should now shine brighter in the heatmap.
Demonstrate the network throughput of this microVM:
localhost:~# iperf3 -c $(./gateway-ip.sh)
Connecting to host 169.254.0.170, port 5201
[ 5] local 169.254.0.169 port 53392 connected to 169.254.0.170 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 1.72 GBytes 14.8 Gbits/sec 0 952 KBytes
[ 5] 1.00-2.00 sec 1.67 GBytes 14.4 Gbits/sec 0 952 KBytes
[ 5] 2.00-3.00 sec 1.76 GBytes 15.1 Gbits/sec 0 952 KBytes
[ 5] 3.00-4.00 sec 1.69 GBytes 14.5 Gbits/sec 0 952 KBytes
[ 5] 4.00-5.00 sec 1.69 GBytes 14.5 Gbits/sec 0 952 KBytes
[ 5] 5.00-6.00 sec 1.66 GBytes 14.3 Gbits/sec 0 952 KBytes
[ 5] 6.00-7.00 sec 1.67 GBytes 14.4 Gbits/sec 0 952 KBytes
[ 5] 7.00-8.00 sec 1.77 GBytes 15.2 Gbits/sec 0 952 KBytes
[ 5] 8.00-9.00 sec 1.76 GBytes 15.1 Gbits/sec 0 952 KBytes
[ 5] 9.00-10.00 sec 1.42 GBytes 12.2 Gbits/sec 0 952 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 16.8 GBytes 14.4 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 16.8 GBytes 14.4 Gbits/sec receiver
iperf Done.To plot the boot times, on your local machine or any non-headless setup:
scp -i <identity-key> ec2-user@<i3.metal-ip>:firecracker-demo/{data.log,gnuplot.script} .
gnuplot gnuplot.script
xdg-open boot-time.png # on Ubuntu. For other distros just use your default .png viewer.