IPMI Initial Configuration

IPMI Configuration for servers that support them is a Good Thing so long as you remember the system is somewhat insecure. Choose decent passwords, and if it be at all possible, use a dedicated NIC on a private subnet separate from everything else.

If you don't know, IPMI allows access to a hardware level management tool on an enabled server. This allows you to monitor temperature, power usage (Dell 2950 gives only basic info) and access to the system (BIOS) error logs. It can also give you access to Serial over LAN, replacing the need for a dedicated serial KVM.

In my case, I want to do some monitoring from a virtual machine. To do this, I simply add a virtual "NIC"  to the monitoring virtual, then use vlan tagging to give that virtual access to the IPMI network. This allows me to use the monitoring virtual to both collect data from the IPMI attached devices and to access the physical machines via SOL (Serial Over Lan).

These procedures have been tested using HP DL380 G5&6, and Dell 2950's, running the Debian distribution of Linux (currently wheezy). It should be applicable to other distributions and servers which support IPMI.

Configure IPMI in the BIOS

While the machine is still on the bench, interrupt the BIOS boot screen and configure IPMI. The two main things you need to do are set the IPAddress and set a username and password which has administrative rights. It is also very handy to give the IPMI interface a real name (the default is a logical serial number, but very difficult to identify). I generally give it something like machinename-ipmi which allows me to find it in my dhcp list rapidly.

IP Address

I like to use DHCP since my router/firewall supports sticky IP addresses. This allows me to rapidly reconfigure the entire network from the router as opposed to touching every machine if reconfiguration is required. Be aware, however, that if your router dies, it means you may lose access to the machines as soon as the DHCP leases expire. If this is a concern, put in static IP addresses.

Username and Password

It is easy to say "I'm on a protected private network, so I can just use password123 for all the passwords." However, realize that you will have several machines on this network which may not be as secure as you think, and the end results can be catastrophic. At the very least, a black hat with access can rapidly and easily shut down all of your physical machines by issuing the shutdown command via IPMI. On some systems, they can gain low level access to your hard drives and your running memory. Use good passwords.

I also add a second user with limited rights since I monitor things like power usage via IPMI using a monitoring package (Zabbix, http://zabbix.com) and the security on that publicly available server is a minor concern to me.

Install the IPMITools Package

On your server, perform the following (Debian Wheezy, adjust for other OS's)

apt-get install openipmi ipmitool libopenipmi0
modprobe ipmi_si
modprobe ipmi_devintf
echo ipmi_si >> /etc/modules
echo ipmi_devintf >> /etc/modules

If you are working on a Xen or other virtual server, install this in the root OS (DOM0 for Xen).

As root, you can now execute a shell and view/change what you need from there, simply by issuing the command

ipmitool shell

 On a different machine, you can connect using the -I and -H parameters. Note if you simply want to connect from a virtual for monitoring or managing, you can perform the install on them also. However, the ipmitool command by itself does not work (obviously, there is no IPMI beneath a virtual machine), so only the network based form of the tool can be used.

To connect over a network to a machine, use the following format

ipmitool -I lan -U username -H IPAddress command_list
ipmitool -I lanplus -U username -H IPAddress command_list

 lanplus is used on most of the newer machines.On Dell 2950's, you use lan unless you want to use SOL, but on HP DL-380's, you use lanplus for everything.

Configure IPMI via web interface

On most modern IPMI installations, there is a web server sitting on the IPMI interface. To connect, you will need the IP address of the IPMI interface (and a machine which can hit that subnet), and the username/password you set up in BIOS.

NOTE: Both IP address and username/password can be set up using IPMITool from the CLI on the machine, but it is simpler to do it from the BIOS. However, issuing ipmitool shell as root from your server will allow you to then issue lan print and user show commands from the shell. You can see your IP address and using the lan and user commands will also allow you to change them.

Now that you know the information you need, open a web browser to the proper IP address. You will most likely get a certificate error (it is self signed), but just blow through that and enter the configuration. Everyone has a different layout, even between different revisions of the same machine (DL-380 G5 looks different than the DL-380 G6, if I remember correctly). Just take your time and look around.

The important thing here is to set up SOL. Find the entry that says SOL or Serial Over Lan and configure it. Generally, there is a virtual Serial port you can attach to, and you can generally set the speed to the fastest speed listed. Be sure a user has access. Remember the serial port you use as you will need to set it in the next step.

Set up OS to redirect to SOL

At this point, IPMI is ready to give you console access on the server, but your server may not be ready for this. To set your server to allow access, you will need to edit /etc/inittab, adding a line that gives serial line access. The following script should do that.

cp /etc/inittab /etc/inittab.save
echo 'T0:23:respawn:/sbin/getty -L ttyS1 57600 vt100' >> /etc/inittab
init q

Be sure to modify the speed and serial prot name (ttys1 = second serial port)

Note that you should make sure the line does not already exist in inittab, so it is many times better to manually add it to the file

 At this point, you should be able to access the console from a remote machine using the following.

ipmitool -I lanplus -H IPAddress -U username sol activate

 If successful, you can press enter a few times and get a remote login prompt. To exit, use the special character tilde (~) followed by a period (~.). To get a list of SOL commands, use ~?.

Now, you can watch the bios screen come up on a reboot, and watch the screen/interact with it once the OS is in place. However, if you also want to watch the boot process (recommended), add the following to your /etc/default/grub (for grub2 under Debian Wheezy).

GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS1,38400n8"
GRUB_SERIAL_COMMAND="serial --speed=19200 --unit=1 --word=8 --parity=no --stop=1"

Be sure to modify the speed and serial prot name (ttys1 = second serial port)

Now run update-grub2 from the command line to reconfigure grub.

Once this is done, you can remotely connect to a serial console on the machine, issue a reboot command, and watch the entire process as if you had a crash cart connected to the server.


Tags: ipmi, SOL
Last update:
2015-04-13 08:00
Average rating:0 (0 Votes)

You cannot comment on this entry

Chuck Norris has counted to infinity. Twice.