Skip to content

Home

Jump to bottom Edit New page
kjtanaka edited this page Jan 14, 2013 · 7 revisions

The goal of FutureGrid Teefaa(FG Teefaa) is to provide the scripts and methods for bare-metal provisioning, and then make it work with Resource Managers such as Torque, so that cluster users can provision their own os images as if they are using Cloud Infrastructure. FG Teefaa mainly consists of these conmonents.

  • Bootstrap - Downloads an OS image from an image source, and installs it on a bare-metal/virtual machine.
  • Snapshot - Makes a snapshot of an OS.
  • Cloudimg - Makes a cloud image from a snapshot.
  • Torque plugin - Provides scripts for Torque users to schedule and run bootstrap on cluster machines.

Motivation

  • As a user of IaaS and HPC Cluster, it would be great, if I can run a customized OS image not only on IaaS also on HPC bare-metal cluster, and send the OS image back and forth among my own local environment(Desktop/Laptop), cloud infrastructure(OpenStack, Eucalyptus) and HPC cluster.
  • As a system admin, I need some easy way to backup and restore a system. Which works on all over local machines, cloud instances and hpc bare-metal clusters. And which supports multiple distributions.

People who may want to use

  • I want to provision a system on bare-metal machines as easily as doing it on Cloud instances.
  • I want to get some bare-metal machines and build/evaluate Open Source Cloud Infrastructure such as OpenStack, Eucalyptus, OpenNebula and CloudStack.
  • I want to provision my system(Workstation/Desktop/VM) on HPC Cluster.
  • I want to provision a customized cloud-image on bare-metal machine.
  • I want to backup a system and restore it on another machine.

Get Started

A testing version is installed on india.futuregrid.org so that FutureGrid users will be able to use it soon. There are some base images are available, and here's the tutorial.

  • Login to india
ssh [email protected]
  • Write your config file(teefaa_userrc) like this.
# Provide project_id and token.
PROJECT_ID="fg-296"

# days of reservation 7 days is maximum
DAYS=1

# If you have your own costom image source, provide the list.
# If not, pick a image type listed below and uncomment it.
#IMAGE_LIST=$HOME/teefaa/image.list
IMAGE_NAME=ubuntu-12.10
#IMAGE_NAME=ubuntu-12.04
#IMAGE_NAME=centos-6.3

# Define ssh public key.
SSH_PUBKEYS="put your ssh public key to login the machine you will get."

# Define partitioning type. MBR(Master Boot Recorder) is only available,
# but GPT will be available soon. So don't change this right now.
PARTITION_TYPE="mbr"

# Define disk setting. This example sets 2G for swap, 50G for root(/) partition with ext4,
# and the rest for scratch(/scratch) partition.
disk=sda
sda1=(2 swap none)
sda2=(50 ext4 "/")
sda3=(-1 xfs "/scratch")
  • Create your job script (provisioning.pbs)
#!/bin/sh
#PBS -N PROVISIONING
#PBS -l nodes=1:ppn=8
#PBS -q provision

# Set the path to your teefaa_localrc
USERRC=/path/to/teefaa_userrc

#####  DON'T CHANGE BELOW  #####
sleep 10
# Pass your rc file to Teefaa Messenger.
cp $USERRC /tmp/userrc
sleep 10
  • Submit the job script.
qsub provisioning.pbs
  • The job script will set configuration and restart the node. Then the Teefaa Messenger(a customized netboot image) will receive and install the OS. Which will take 10 to 15 minutes usually, and once your node get ready, it will be listed on the "dispatch" queue as a job name of tfadmin(Teefaa Admin). Right now the queue is set on i132. So here's the command to check the queue.
qstat @i132
Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
13.i132                    i6_ktanaka       tfadmin                0 R dispatch
  • The list above means ktanaka got i6. i6's external name is i6r.idp.iu.futuregrid.org and external IP address is 149.165.146.6, so you can ssh to the node with this.
ssh [email protected]
or) ssh [email protected]
  • qstat command can show you how long your node have spent time. My example reserved 1day(24h), the output below means I only have a little more than 1 hour. When the time comes, the node will be shutdown, get the base image back installed and go back to the provision queue.
qstat -f 13|grep resources_used.walltime
    resources_used.walltime = 22:45:01

Make snapshot of your system

FutureGrid Teefaa provides script for making a snapshot of your system. Here's snapshotrc

# snapshotrc

SNAPSHOT_DIR="/var/lib/teefaa/snapshot"

# Define logfile.
LOGFILE=/tmp/snapshot.log

# Define the file of exclude list.
EXCLUDE_LIST=$TOP_DIR/exclude.list

If you run snapshot.sh, the script will create a snapshot in "/var/lib/teefaa/snapshot". We tested the script with Ubuntu-12.04, Ubuntu-12.10 and CentOS-6.3.

Make cloud image from your snapshot

FG Teefaa also provides script to make cloud image from your snapshot. Here's example of configfile.

# cloudimgrc

CLOUDIMG_DIR="/var/lib/teefaa/cloudimg"

# Define logfile.
LOGFILE=/tmp/cloudimg.log

Here's the command to create cloud image.

./cloudimg.sh /path/to/your-snapshot.squashfs name-of-the-image 2

This will create image file, kernel, ramdisk in "/var/lib/teefaa/cloudimg". You can upload it with euca2ools.

More about FG Teefaa

I'll work on writing deeper information about Teefaa on http://futuregrid.github.com/teefaa/ soon. But here's the requirements.

Requirements

  • Bootstrap, Snapshot and Cloudimg - Need LiveCD/Netboot with rsync, mkfs.ext4, mkfs.xfs, mksquashfs and tree.
  • Torque plugin - Need Torque(of course), PXE Server and IPMI.
Add a custom footer
Add a custom sidebar
Clone this wiki locally