Unix Server Tech Knowledge Base

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Setting up CARP ======

You need carp set up to tell the nodes which is primary and which is secondary.

I'm using vlan50 for my data, so we need to be able to switch between machines. The two machines will have 192.168.209.0/24 in common, configured as follows:
  * iscsi-0: 192.168.209.67
  * iscsi-1: 192.168.209.68
  * CARP: 192.168.209.64

===== Enable carp and set shared IP =====

Add the directives to load the carp module and make the alias for vlan50. the carp.preempt sets it so so when the parent interface goes down or comes up, the carp interface will do the same
<code bash>
echo 'carp_load="YES"' >> /boot/loader.conf
echo 'net.inet.carp.preempt=1' >> /etc/sysctl.conf
echo 'ifconfig_vlan50_alias0="inet vhid 1 pass iscsilan alias 10.19.209.64/24"' >> /etc/rc.conf
</code>

if you want to test this, you can run the following:

<code bash>
kldload carp  
sysctl net.inet.carp.preempt=1
ifconfig vlan50 alias vhid 1 pass iscsilan 10.19.209.64/24
</code>

===== Set devd =====

Add the following to the bottom of /etc/devd.conf. This tells the server what to do when the carp interface comes up or goes down.
<code bash>
notify 30 {
        match "system" "IFNET";
        match "subsystem" "vlan50";
        match "type" "LINK_UP";
        action "/usr/local/opt/hast/failover master";
};

notify 30 {
        match "system" "IFNET";
        match "subsystem" "vlan50";
        match "type" "LINK_DOWN";
        action "/usr/local/opt/hast/failover slave";
};

</code>

===== Create the failover script =====

This is taken mainly from the hast article at [https://opensource-information.blogspot.com/2012/08/hast-freebsd-zfs-with-carp-failover.html].

The following script is designed to bring up/shutdown the hast volumes and will eventually be used to handle the iSCSI service and NFS. It already handles the zpool itself.

Change resource list and pool name before implementing.

<code bash failover>
#!/bin/sh

# https://opensource-information.blogspot.com/2012/08/hast-freebsd-zfs-with-carp-failover.html
# Original script by Freddie Cash <fjwcash@gmail.com>
# Modified by Michael W. Lucas <mwlucas@BlackHelicopters.org>
# and Viktor Petersson <vpetersson@wireload.net>
# Modified by George Kontostanos <gkontos.mail@gmail.com>
 
# The names of the HAST resources, as listed in /etc/hast.conf
resources="disk0 disk1 disk2 disk3 disk4 disk5 disk6 disk7 disk8 disk9"
 
# delay in mounting HAST resource after becoming master
# make your best guess
delay=3
 
# logging
log="local0.debug"
name="failover"
pool="storage"
 
# end of user configurable stuff
 
case "$1" in
 master)
  logger -p $log -t $name "Switching to primary provider for ${resources}."
  sleep ${delay}
 
  # Wait for any "hastd secondary" processes to stop
  for disk in ${resources}; do
   while $( pgrep -lf "hastd: ${disk} \(secondary\)" > /dev/null 2>&1 ); do
    sleep 1
   done
 
   # Switch role for each disk
   hastctl role primary ${disk}
   if [ $? -ne 0 ]; then
    logger -p $log -t $name "Unable to change role to primary for resource ${disk}."
    exit 1
   fi
  done
 
  # Wait for the /dev/hast/* devices to appear
  for disk in ${resources}; do
   for I in $( jot 60 ); do
    [ -c "/dev/hast/${disk}" ] && break
    sleep 0.5
   done
 
   if [ ! -c "/dev/hast/${disk}" ]; then
    logger -p $log -t $name "GEOM provider /dev/hast/${disk} did not appear."
    exit 1
   fi
  done
 
  logger -p $log -t $name "Role for HAST resources ${resources} switched to primary."
 
 
  logger -p $log -t $name "Importing Pool"
  # Import ZFS pool. Do it forcibly as it remembers hostid of
                # the other cluster node.
                out=`zpool import -f "${pool}" 2>&1`
                if [ $? -ne 0 ]; then
                    logger -p local0.error -t hast "ZFS pool import for resource ${resource} failed: ${out}."
                    exit 1
                fi
                logger -p local0.debug -t hast "ZFS pool for resource ${resource} imported."
 
 ;;
 
 slave)
  logger -p $log -t $name "Switching to secondary provider for ${resources}."
 
  # Switch roles for the HAST resources
  zpool list | egrep -q "^${pool} "
         if [ $? -eq 0 ]; then
                 # Forcibly export file pool.
                 out=`zpool export -f "${pool}" 2>&1`
                  if [ $? -ne 0 ]; then
                         logger -p local0.error -t hast "Unable to export pool for resource ${resource}: ${out}."
                         exit 1
                  fi
                 logger -p local0.debug -t hast "ZFS pool for resource ${resource} exported."
         fi
  for disk in ${resources}; do
   sleep $delay
   hastctl role secondary ${disk} 2>&1
   if [ $? -ne 0 ]; then
    logger -p $log -t $name "Unable to switch role to secondary for resource ${disk}."
    exit 1
   fi
   logger -p $log -t $name "Role switched to secondary for resource ${disk}."
  done
 ;;
esac
</code>
===== References =====
  * https://opensource-information.blogspot.com/2012/08/hast-freebsd-zfs-with-carp-failover.html
  * https://wiki.freebsd.org/HAST
  * https://www.freebsd.org/doc/handbook/carp.html