Differences

This shows you the differences between two versions of the page.

--- unix:virtualization:xen:dom0 [2020/07/12 11:42] – rodolico
+++ unix:virtualization:xen:dom0 [2020/10/05 17:44] (current) – rodolico
@@ Line 1: / Line 1: @@
 ====== Xen DOM0 ======
-===== Limit DOM0 memory =====
+===== Base Install =====
-At times, the DOM0 will attempt to grab all RAM and make it where no virtuals can exist. There may be other reasons to limit the amount of memory available to the DOM0, though I generally like to give it a bunch so it can cache things.
-However, you can limit the amount of memory the DOM0 can have by adding the following to /etc/default/grub. While we're at it, I'm limiting it to 4 virtual cpu's, and disabling grub's OS Prober, which will find lvm2 and iSCSI drives (which are my virtual's boot drive).
+<code bash>
+apt -y install xen-system
-<code bash default_grub>
+# if you don't do this, it will boot into non-xen kernel
-# Limit DOM0 memory to 4096M (4G) and allow to grow to 16384M (16G) if possible
+dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen
-GRUB_CMDLINE_XEN="dom0_mem=4096M,max:4096M dom0_max_vcpus=4"
+# now, limit DOM0 memory to 4G
-# don't probe anything but the boot drive for os'
+echo '# Xen boot parameters for all Xen boots' >> /etc/default/grub
-GRUB_DISABLE_OS_PROBER=true
+echo 'GRUB_CMDLINE_XEN="dom0_mem=4096M,max:4096M"' >> /etc/default/grub
+echo '# do not probe anything but the boot drive for os' >> /etc/default/grub
+echo 'GRUB_DISABLE_OS_PROBER=true' >> /etc/default/grub
+echo 'autoballoon="0"' >> /etc/xen/xl.conf
+# and tell grub to update itself
+update-grub
+# tell Xen to use the xl tool stack. It appears with newer Debian releases, this is the default
+# so probably not necessary
+# NOT TESTED, you may have to hand edit.
+echo 'TOOLSTACK=xl' >> /etc/default/xen
+# at this point, we are ready to go, so reboot the system into the Xen kernel
+reboot
 </code>
-You also don't want xl to automatically balloon memory for DOM0, so edit /etc/xen/xl.conf and modify the 'autoballoon' directive
+Note: There is an option to limit the number of vcpu's DOM0 can have, but my notes are possibly incorrect. I had:
-<code>autoballoon="0"</code>
+<code>GRUB_CMDLINE_XEN="dom0_mem=4096M,max:4096M dom0_max_vcpus=4"</code>
-After you have added this, you need to update your grub install with the following command.
+==== Limit DOM0 memory ====
-<code bash>update-grub2</code>
+At times, the DOM0 will attempt to grab all RAM and make it where no virtuals can exist. There may be other reasons to limit the amount of memory available to the DOM0, though I generally like to give it a bunch so it can cache things.
-===== DOM0 and UEFI =====
+==== DOM0 and UEFI ====
 I have run into some serious issues trying to get a DOM0 to boot with UEFI. A fresh Devuan (Debian) install works just fine, then when the Xen kernel is brought up, the boot fails. I don't remember the exact issue, but it has been around for several years.
@@ Line 26: / Line 36: @@
 However, I my fix is to simply set the server to boot into legacy BIOS mode. As long as that is an option, it appears to be better for the time being.
+===== Adding iSCSI and NFS =====
+For some of our systems, we are using an iSCSI target for the device back ends (vbd's). We also put common configuration files on an NFS share, then mount it at /etc/xen/iscsi_configs. This allows us to migrate virtuals (DOMU's) rapidly between the Xen Hypervisors.
+Install the packages for the client (aka initiator for iSCSI)
+<code bash>
+apt -y install open-iscsi nfs-common
+</code>
+And, change /etc/iscsi/iscsid.conf to automatically start on boot. Default is to do manual startup, so uncomment automatic and comment node.startup = manual
+<file - iscsid.conf>
+# To request that the iscsi initd scripts startup a session set to "automatic".
+node.startup = automatic
+#
+# To manually startup the session set to "manual". The default is manual.
+# node.startup = manual
+</file>
+==== set up iSCSI targets ====
+This is a manual operation, but there is a script that will do it automagically for you, if you like. See
+[[unix:linux:iscsi_tricks_and_techniques|]]
+==== set up NFS mounts ====
+Now, create an entry in /etc/fstab to automount /etc/xen/iscsi_configs
+<code bash>
+mkdir -p /etc/xen/iscsi_configs
+echo 'ip.of.iscsi.target:/media/nfs_root/xen/configs /etc/xen/iscsi_configs nfs defaults 0 0' >> /etc/fstab
+mount /etc/xen/iscsi_configs
+</code>
+=== Real Life ===
+Honestly, what we do is a little more complex on the nfs mounts. We have configurations that may be different depending on which hypervisor they are one. Sometimes, when upgrading a Hypervisor, the configuration will need to be changed.
+We also like to have several utilities shared between all of our DOM0's. Installers for various operating systems, qparted to grow a partition, and systemrescuecd for just about everything.
+Finally, sometimes we want to use an FBD (File Backed Device) off an nfs mount for some DOMU's. Not as efficient, but a decent quick and dirty if we need it for some reason.
+For this, we actually create three nfs exports:
+xen-configs - stores configs for virtuals in a tree, depending on the type needed.
+xen-images - holds FBD's
+xen-store - ISO's, scripts anything that we may need for managing our work
+We mount those in /media/whatever (I just call it the name of the export) on each DOM0. For the configs, I then create a symbolic link from /media/xen-configs/4.9 to /etc/xen/iscsi_configs, with the 4.9 being the version of Xen the configuration is correct for.
 ===== Links =====
   - https://wiki.xen.org/wiki/Xen_Project_Best_Practices
+  - https://wiki.debian.org/Xen