Back Up Xen Image


Back Up Xen Image

Discussion

This article assumes you have the volumes for your xen images stored in LVM partitions. It also assumes you have sufficient disk space to store the image. In our practice, at Daily Data, we tend to create 4G partitions for root, then add partitions as necessary for, say, /var/lib/mysql, or /home (where I like to put web servers). This article only discusses backing up and restoring the root partition, but is applicable to all other partitions.

Note: I use cksum in the scripts I have written. The syntax is simple: cksum filename, with the result showing what the checksum of the filename is. When recovering, you can run this command on the uncompressed image before you use it, then do a comparison with the .cksum file that is stored. If they are the same, good. If not, you likely have a corrupted image.

Note: Don't forget to store your configuration files. Since they don't change that much, it is not included in the following discussion. But, saving your configuration files will help you rapidly recover/reuse a backup image.

Note: if all you want to do is copy an image from one machine to another, or even to another machine on an existing computer, rsync might be a better option: just shut down your virtual, mount the partition someplace (say, /mnt), then, rsync to the other machine. This procedure is designed to allow you to have a system in a particular state (say, a nice clean install, or a common setup) so you can recover if necessary.

Backing Up

There are many ways. Since I can shut down my virtuals, this works for me. In the following, my virtual name is virtual.example.com, and my partitions follow the xen-tools convention of virtualname-disk. Be sure and modify for your system.

Simple

  • Shut down virtual
  • move to a directory where you can store the image
  • Execute this command
dd if=/dev/virtuals/virtual.example.com-disk | bzip2 > virtual.example.com-disk.img.bz2
  • When the command completes, you should have a file, name of virtual.img.bz2 in your directory. This is an image of your partition.

Note this command assumes you have plenty of time, and not much disk space. It will shut down your virtual for an extended period of time (ie, half an hour on a dual core processor for a little 4G partition). If you need the virtual down for as little time as possible and have sufficient disk space, use some form of the following script (yes, you can set it up to accept command line parameters):

#! /bin/bash
INPUT_FILE=/dev/virtuals/virtual.example.com-disk
OUTPUT_FILE=./virtual.example.com-disk.´date +%Y%m%d´.img
VIRTUAL_NAME=/etc/xen/defiant.conf
dd if=$INPUT_FILE of=$OUTPUT_FILE
xm create VIRTUAL_NAME
cksum $OUTPUT_FILE > $OUTPUT_FILE.cksum
bzip2 $OUTPUT_FILE

Note that this will require disk space greater than the size of your virtual's partition to back up, but it is very fast. You manually shut down the virtual (xm shutdown virtualname), then execute this script. This will do the dd first (very fast), then start up the virtual (note, use the configuration file name, not the virtual name), then do the rest, which basically runs a checksum, then bzips. bzip2 seems to give very, very good results; I've seen 90% + compression on a root partition.

Less Down Time

Assuming you are using LVM, you can shut down the virtual for a very short period of time, make a snapshot and bring it back up. Then, backup the snapshot.

  1. Shut down virtual xm shutdown /etc/xen/virtual -w
  2. Create a snapshot lvcreate -s -L 1G -n snap /dev/virtuals/virtual.example.com-disk
  3. Start the virtual back up xm create /etc/xen/virtual.conf
  4. Back up the snapshot dd if=/dev/virtuals/snap | bzip2 > virtual.example.com-disk.img.bz2
  5. Remove the snapshot lvremove /dev/virtuals/snap

An LVM snapshot creates a snapshot of a partition at a given point in time. Because of the way it does it (through some PFM on files written to the original partition), a snapshot is extremely fast (by the time you have released the enter key, it is created), and uses very little disk space (you must have enough for the changes to the original disk during the time the snapshot is in existence). Thus, with a few minor steps, this can be done with very little downtime. Basically, the amount of downtime required for a reboot, plus a few seconds.

You can even perform this on a live disk, ie with the domain still running. If you do, however, and any files are open, they will be in a corrupted state, as if you had simply turned power off of a running computer. While most OS's will recover most of the time, only do this on "disposable" machines.

A script that implements this follows:

#! /bin/bash
INPUT_FILE=/dev/virtuals/virtual.example.com-disk
OUTPUT_FILE=./virtual.example.com-disk.´date +%Y%m%d´.img
VIRTUAL_NAME=example
VIRTUAL_ROOT=/dev/virtuals
xm shutdown $VIRTUAL_NAME -w
lvcreate -L 1G -n snap $INPUT_FILE
xm create /etc/xen/$VIRTUAL_NAME
dd if=$VIRTUAL_ROOT/snap | bzip2 > $OUTPUT_FILE.bz2
cksum $VIRTUAL_ROOT/snap > $OUTPUT_FILE.cksum
lvremove $VIRTUAL_ROOT/snap

Generalized Script

The following generalized script will back up a virtual with minimum downtime. It assumes the server was created with xen-create-image. Thus, the configuration file is /etc/xen/imagename.cfg, there is only one partition to back up, and it is in /dev/virtuals/imagename-disk, and it is an LVM partition. It also assumes there is storage in /home/xen-store to put the backup.

  #! /bin/bash
# you can change the following to match your system
SAVE_DIR=/home/xen-store/ # a location with room to store the compressed backup
LVM_ROOT=/dev/virtuals # the root directory of the virtuals
SNAP_NAME=snap # what we will call the snap file
# do some minor validation
if -z "$1" ; then 
echo usage: $0 virtualname
exit
fi
if ( ! xm list niska >& /dev/null ) 
then 
echo $1 is not a running virtual
exit
fi
# assumes one parameter.
VIRTUAL_NAME=$1
# The rest of the items are calculated from the above values
SNAP_PATH=$LVM_ROOT/$SNAP_NAME
OUTPUT_FILE=$SAVE_DIR´basename $1´.´date +%Y%m%d´.img
INPUT_PATH=$LVM_ROOT/$VIRTUAL_NAME-disk
# tell the user what we are doing.
echo Backing up Domain $VIRTUAL_NAME from $INPUT_PATH to $OUTPUT_FILE
# shut down the virtual, and wait until it is down before returning
xm shutdown $VIRTUAL_NAME -w
# create a snap file
lvcreate -s -L 1G -n $SNAP_NAME $INPUT_PATH
# start the virtual back up
xm create /etc/xen/$VIRTUAL_NAME.cfg
# start the copy
dd if=$SNAP_PATH | bzip2 > $OUTPUT_FILE.img.bz2
# get a checksum of the original partition
cksum $SNAP_PATH > $OUTPUT_FILE.chksum
# remove the original partition
lvremove -y $SNAP_PATH

Less storage, but more temporary disk space

You can save storage space at the expense of temporary disk space by taking the image first, then zero'ing out unused areas of the disk, then compressing it. Basically, you do everything above but, instead of dd'ing and bzipping at the same time, you do the dd first, then clear any unused disk space, then do the bzip. A simple way to clear all unused areas of a disk is to mount it, then create a file filled with zero's. Not 100% effective, but it is still pretty good. You can do this with the command dd if=/dev/zero of=/mnt/ZEROS_DELETE_ME, assuming your partition is mounted on /mnt. This will keep appending zero's to the file until it runs out of disk space and errors out. Then, simply remove the file you created an most of your unused space is filled with zero's, which compresses down nicely.

I have no massive amount of information to tell how much space you saved, but for this article, However, I did test four "dirty" virtuals (dirty meaning I had never zero'd the unused space out) and the results are in the following table. It is interesting to note that, while the times do not always increase when zeroing, the size of the resulting image does. Since dd will do a bit for bit copy, and the compression program must compress those bytes also, the backups where the time actually decreased would, I assume, be the result of truly random data in the unused space, which compression programs have to spend a lot more computing power on to reduce. Thus, under these situations, it is both faster and results in a smaller image to zero out the space before doing the compression.

Note: I use bzip2 in all of these, but gzip works just as well. You do not get as good a compression, but it is much, much faster. Just RTFM, especially the requirement to use the -c option when accepting input from a pipe (stdin).

Note: Zeroing out unused space does not seem like something to do on a working machine, once the initial zero has taken place. You will not likely add and remove a lot of files during a normal course, so it is not required to zero out empty space every time you back up.

Note: For the truly efficient minded, remove all files from /tmp and /var/log when making an archive. Archive assumes you will be using this as a replacement system. Sometimes, on a busy machine, both of these fill with information that will be useless if you try to recover from archive two years after you created the archive. There is a commented out section in the script that will do this.

  Time (hh:mm:ss) Image Size (megabytes)
  No Zero Zero Difference No Zero Zero Difference
tully 00:22:15 00:26:19 -18% 1780 1134 36%
pyanfar 00:23:21 00:18:06 22% 2996 294 90%
bluesun 00:22:01 00:19:07 13% 2592 442 83%
niska 00:10:38 00:18:30 -74% 643 231 64%
Average

-14%

68%
#! /bin/bash
date
# you can change the following to match your system
SAVE_DIR=/home/xen-store/ # a location with room to store the compressed backup
LVM_ROOT=/dev/virtuals # the root directory of the virtuals
SNAP_NAME=snap # what we will call the snap file
# do some minor validation
if -z "$1" ; then
echo usage: $0 virtualname
exit
fi
if ( ! xm list $1  >& /dev/null )
then
echo $1 is not a running virtual
exit
fi
# assumes one parameter.
VIRTUAL_NAME=$1
# The rest of the items are calculated from the above values
SNAP_PATH=$LVM_ROOT/$SNAP_NAME
OUTPUT_FILE=$SAVE_DIR´basename $1´.´date +%Y%m%d´
INPUT_PATH=$LVM_ROOT/$VIRTUAL_NAME-disk
# tell the user what we are doing.
echo Backing up Domain $VIRTUAL_NAME from $INPUT_PATH to $OUTPUT_FILE
# shut down the virtual, and wait until it is down before returning
xm shutdown $VIRTUAL_NAME -w
# create a snap file
lvcreate -s -L 1G -n $SNAP_NAME $INPUT_PATH
# start the virtual back up
xm create /etc/xen/$VIRTUAL_NAME.cfg
# start the copy
dd if=$SNAP_PATH of=$OUTPUT_FILE.img
# remove the original partition
lvremove -f $SNAP_PATH
# mount the image an zero out all unused sectors
mount -o loop $OUTPUT_FILE.img /mnt
# uncomment next line to clean up /tmp before archiving
# rm -fRv /mnt/tmp/*
# uncomment next line to clean up /var/log before archiving
# find /mnt/var/log -type f -exec rm -fv \{\} \;
dd if=/dev/zero of=/mnt/ZEROS.REMOVE_ME
rm /mnt/ZEROS.REMOVE_ME
# unmount cleaner partition
umount /mnt
# grab a checksum
cksum $OUTPUT_FILE.img > $OUTPUT_FILE.cksum
# and compress the file
bzip2 $OUTPUT_FILE.img
date

Restoring a Xen Image

This article has been moved to Restore a Xen Image

Other partitions for the Xen virtual

A note on other partitions. First, the swap. there might be some confusion here: You do not need to waste the space backing up and restoring the swap partion. Just recreate it.

  lvcreate -L 512M -n virtual-name-swap volume_group_name
mkswap /dev/volume_group_name/virtual-name-swap

There is nothing special about the swap disk: it is a place that a running system stores information to when it runs out of physical memory. On a properly set up machine, with all the resources it needs, the swap space is never used (and some systems just don't have one for this reason).

I like to use separate partitions for /home and, in some cases, other directories, most notably places in /var (/var/lib/mysql, /var/log). Things where I can not judge the size needed, and where the sizes may change over time. I stick web sites in /home/www, with logs, cgi and content in subdirectories. Thus, those grow over time. I've found a 4G root partition is more than adequate, so long as I do it this way. /tmp on some systems grows like crazy, so I just use a separate partition if I see that happening.

All of these can easily be backed up, with less traffic, etc..., simply by using a good copy program that will save permissions. I don't do that on /root as I truly do not understand /dev and /proc. Someone who knows what they are doing might be able to simply copy the files (or maybe all I need do is try it myself), but I just do the procedure outlined above until my ignorance is remedied.

However, for all other partitions, simply store the information someplace. Shut down the virtual, and mount it someplace. Then copy using something that honors links and permissions:

  # assume virtual is shut down, we are backing up /dev/virtuals/myvirtual-home
mount /dev/virtuals/myvirtual-home /mnt # mount it temporarily on /mnt
cd /home/xen-images # the partition we have created for backups
mkdir myvirtual-home # create a directory to store the info
cp -av /mnt/* myvirtual-home/ # perform the actual copy, using "archival" mode
umount /mnt # release the volume so we can restart the virtual

cp -av is a good copy for local machines, but rsync is just as good (think rsync == super copy), so replacing cp above with

  rsync -av /mnt/* myvirtual-home/

is just as good. The advantage being you can put it on a computer named mybackupserver by changing the command to:

  rsync -av /mnt/* mybackupserver:/home/xen-images/myvirtual-home/

assuming those directories exist on mybackupserver. You can also have a compressed archive using tar (this is what tar is all about). Substitute the cp (and the preceeding line that creates a directory) with:

 tar -czvf /home/xen-images/myvirtual-home.tgz /mnt/* # substitute j for z to use bzip2 instead of gzip

Now, you not only have your data, but you are using a lot less disk space.

Backing up with xenBackup script

The following technique is not tested by Daily Data yet. Please proceed with caution.

Back Up Xen Image

Last update:
2012-01-01 01:21
Author:
Rod
Revision:
1.1
Average rating: 5 (1 Vote)

You can comment this FAQ

Chuck Norris has counted to infinity. Twice.