Fun with ESXi, IPMI and Dell BMC/iDRAC – Remote Power Cycle, Console and more

Most of my Dell servers/blades in a production setting have some sort of licensed version of iDRAC which allows me to do things like power on, power off, power cycle the servers as well as pick up a remote console of the system that gives me the same look and feel as if I were standing in front of a monitor hooked up to that server.  That said, I also have a lot of one-off ESXi hosts which sit in a remote office in which we haven’t purchased the fancy iDRAC functionality.  It’s not too often that I need to power cycle these machines or connect remotely and make BIOS changes but every now and then I have to – and the last thing I want to do is drive 100KM to get to the server or spend an hour on the phone walking someone through it!

The many flavors of iDRAC

Although we didn’t purchase a version of iDRAC for these remote servers, Dell still provides us with a version that allows us to manage the server in a limited capacity.  We can see the 4 different versions and their differences of iDRAC below

Basic – The most stripped down version of iDRAC.  Provides us with basic hardware management via IPMI v2.0 and a text-based console redirection.

Express – This includes all functionality of basic and adds a web gui to help us perform remote actions.  We also get the use of the ‘racadm’ command line tools with express, along with some power monitoring and budget functionality.

Express for blades – Includes all functionality of Express and Basic, and adds a nice single user virtual console and virtual media mount/unmount capabilities.

Enterprise – All features of Express and Basic but with a dedicated NIC.  Also adds a slew of other benefits such as crash video playback, remote syslog, directory services support, multi-user console collaboration, etc.

Certainly for mission critical applications you will want to be sure you purchase the Enterprise version of iDRAC.  In my case though, the basic version that comes pre-installed on the servers will suit my needs perfectly.  I just need something to power cycle the host and possibly change a few BIOS settings remotely.

It ain’t easy being green (or blue/white or black/silver)

No matter what colors Dell are branding themselves with these days it doesn’t change the fact that performing some of these limited functions included in the Basic version is not necessarily easy!  As you can see, the Express version and above comes with a nifty web interface that can walk us through almost everything we need to do.  The Basic version though forces us to rely on IPMI and the corresponding Dell tools to gain the similar type of functionality.

IPMI is essentially a standard interface that we can use to perform out-of-band management of our systems.  It’s been around since 1998, which is an eternity in tech years and is widely supported with most hardware vendors today.

OMG, Enough already!  How do we remotely power these things off and on?

Alright, let’s cut tot he chase here!  First we need to enable IPMI over LAN within the settings of our iDRAC or BMC configuration on our servers.  Yes, this means you will need physical access to the server in order to accomplish this.  Depending on the generation of Dell server you have the path to get into the BMC/iDRAC settings may change (CTRL+E vs F2) but I’ll leave it up to you to figure that one out.  The example below is on a T320.

First up press F2 during POST to get yourself into the system settings. From here, we will need to go into ‘System Settings’ and then ‘Serial Communication’

This is where we setup the console redirection to COM2.  COM2 is what the iDRAC/BMC will use to pipe our console through so just ensure you have the Serial Communication drop-down set to ‘On with Console Redirection via COM2 ‘ and the Redirection After Boot enabled.  Feel free to change all other settings to your preference, but from what I have found, having a baud rate of 115200 and a terminal type of VT100/VT220 works out the best.

This is all that needs done in the BIOS but don’t back all the way out to a reboot.  Just save your changes until you get back to the first menu you seen, this time, select iDRAC settings.  From here we will need to do a couple of things.

First up, under ‘Network Settings’ be sure that you have replaced the default network configuration with an IP/Gateway that you can reach.  As well on this page, be sure to enable ‘IPMI over LAN’ – nothing will work if you miss this step 🙂  Another thing to to note which isn’t pictured above is the LOM settings.  Be sure to select the LOM (NIC) that you would like to use for your iDRAC functions.

Also, from the same menu where we selected ‘Network Settings’ be sure to go into ‘User Configuration’ and give your root user a password.

Congrats!   If you had to commute out to gain physical access to the servers to do the above steps I can proudly say that we are done with the need for console access!  Go ahead and go home early!

Playing around with IPMISH/IPMITOOL

So far we’ve just done the configuration but now it’s time to jump into actually performing some remote functions on these servers.  First up you will need some sort of IPMI tool.  There is a very popular Linux package which a lot of people use but in my examples I’ll be using the Dell BMC Utility – so if you want to follow along go ahead and install it.  Once your done open up a command prompt to “c:\program files (x86)\Dell\SysMgt\BMC\”.  Inside this directory you will see a bunch of files, but the ones we are going to focus on is IPMISH and IPMITOOL.

IPMISH

The first package we will look at is IPMISH.  There are many things that IPMISH can do such as modifying the front LCD text and report on power consumption, but I’ll just go over the basic commands to power on and off our server.  The syntax for IPMISH can be run in one of two ways.  We can either send the command we wish to execute along with the ip and credentials or we can enter into an interactive shell and execute multiple commands in a row to the iDRAC.  We will be using the latter method.

To connect to our iDRAC enter the following line at the command prompt

ipmish -ip IP_OF_iDRAC -u USER -p PASSWORD -interactive

This should bring you to an IPMI> prompt.  As shown below entering the ‘help’ command will give us a list of the functions available to us.

Furthermore if you wish to get more detailed help on a command we can enter “help command”, such as “help power” shown below…

So, looking at the image above we can see that there are few options in which we can power this server on and off, so go ahead and try one – to power off your host simply type “power off”.  Once your system is has shutdown we can power it back on with the command “power on”.  Alternatively we could do these in a non-interactive way by using “ipmish -ip IP_OF_iDRAC -u USER -p PASS power on”

IPMITOOL

Remember earlier when we did all of the serial console redirection and mentioned we can connect to remote TEXT based console, well, this is where IPMITOOL comes in.  Since we have done most the legwork already for console redirection all we have to simply do is connect to our console using IPMITOOL- the syntax to do so is outlined below

ipmitool -I lanplus -U root -P password -H IP_OF_iDRAC sol activate

Keep in mind that this is only a TEXT based console, so if you have already booted into your OS you will not see anything happening here.  If you go ahead and reboot you should see the post screen scrolling through as shown below

This isn’t the easiest interface to use.  Things like simply sending the F2 command to get into the BIOS is achieved through the key sequence of ESC + 2, F1 is ESX +1.  Very confusing!  There is an overview of all the key sequences here if you are looking for them.

Anyways, if you can afford a version of iDRAC above basic then I’d definitely recommend it!  As you can see things aren’t the most user-friendly, but that said, this gets the job done!   If you have any tips on how to perform remote administration on these servers in a “budget-friendly” way I’d love to hear them in the comments box below!  Thanks for reading!