These are just a few of the tricks I have learned about while dealing with Xen:
- Make them small. As small as you can get for the initial setup. Especially if you are using LVM. You can always increase the disk size, add memory, etc... Creating an initial small image gives you two advantages:
- all disk activity is faster, because there is less disk space to mess with. Formatting 4G of disk space vs 80G is, duh, 20 times faster.
- Backing up is faster. You can create very small images and make a backup of it in its pristine state. Then, for production, increase the size as you like. You can carry many images around on a standard thumb drive this way.
- Save often. It is so much easier when you are 90% through a complex install to go back and reload an image you created a while ago than to go over the whole installation again if you make a mistake. Again, small initial sizes help. You can make a backup in the time it takes to get a cup of coffee.
- Use pre-built images. http://jailtime.org/ has several popular images already. You simply unpack them and run. They are file images, but you can easily copy them to an LVM. Have a virtual up and running in the time it takes to download, copy, and start up. This has been moved to http://stacklet.com and is now on a monthly subscription.
- Build your own standard images. If you find you are doing something over and over, create one image, save it, then use it to create new images as you need them.
- Use Partitions. Back in the old days of computing, unix installs had multiple partitions, both as ways to keep one "directory" from filling up and managing disk space. With LVM, this is even easier
- Identify possible problems. What if a process has a bad error and puts tons of files in /tmp. Will your users store gigabytes of videos in their home directory. Is your database going to grow too big? Creating a separate partition for /home, /tmp and /var/mysql (or the postgres directory) will allow you to control your disk space usage. If you make them too small, they're LVM's. Just make them bigger.
- Keep root as small as possible. Many root partitions need less than a gigabyte. A big installation of Linux can still be less than 100M. You can back your root partition up with some imaging software, and your data via standard scripting. Better still, in case of major upgrades, you can back up your root partition, upgrade, and rapidly revert to the original image should something fail. I generally use 4G root partitions, which is entirely too much if I put /tmp and /var/log in separate partitions.
- Use multiple Networks Interface Cards. Xen has the ability to use as many NIC's as you have, and you can assign them as you like
- Use one NIC strictly for access to the server. A management console, on a private subnet.
- Identify virtuals which will be heavy network users and give them their own NIC
- If you system is internal, but has external virtuals, put the externals on a DMZ (on their own NIC), and your internals/management interface on a separate NIC plugged into a separate, internal network.
Many times you need a bridge on the DOM0 only to be passed to the DOMU's attached. The DOM0 has no need for an address on that bridge and it only provides an attack vector to the DOM0.
There are several ways you can do it. One is to assign it to a static IP that can not be reached, ie with a subnet of 255.255.255.255 (/32). Using the Debian /etc/init.d/interfaces file, you would do something similar to the following
iface eth0 inet manual auto xenbr0 iface xenbr0 inet static address 192.168.1.15 netmask 255.255.255.255
However, there is a simpler way. In this case, you simply define the bridge as manual and it is brought up, but never assigned an IP address. However, the DOMU's can use the bridge and assign their IP addresses as needed. The following example shows five interfaces; 4 are used in bridges and one is "naked".
# several bridges brought up auto, with the iface # defined as manual. This allows DOMU's to filter # through the bridge, but with no address defined # kills an attack vector. # The syntax used is: # auto xenbrX.XX # iface xenbrX.XX inet manual # bridge ports ethX.XX # Internet Bridge, no IP iface eth0.10 inet manual auto xenbr0.10 iface xenbr0.10 inet manual bridge_ports eth0.10 # DMZ bridge, no IP iface eth0.20 inet manual auto xenbr0.20 iface xenbr0.20 inet manual bridge_ports eth0.20 # LAN bridge, get IP from DHCP iface eth1.30 inet manual auto xenbr1.30 iface xenbr1.30 inet dhcp bridge_ports eth1.30 # IPMI bridge, no IP iface eth1.40 inet manual auto xenbr1.40 iface xenbr1.40 inet manual bridge_ports eth1.40 # iSCSI interface, static IP iface eth2 inet static address 10.10.10.15 netmask 255.255.255.0
Note that the vlan package is installed, and these are actually only three interfaces, eth0, eth1 and eth2. eth0 has on vlan 10 and 20, and eth1 has vlan 30 and 40. However, only vlan 30 is actually used. From this we get access (hopefully secure) to the DOM0. eth2 is a dedicated line to an iSCSI NAS device.
Of course, you could always put all vlan's on both interfaces, then bond them for even greater reliability.