Customize the VMware vSphere Health Check to only email report if there is a hardware issue.

One of the things that I love most about utilizing vCenter Server to manage my hosts is the ability to generate a nice little email alert when any hardware within my hosts decides to go a little offside.  That being said there are a lot of times when I don’t have vCenter and am trying to monitor hardware failures on the free ESXi Hypervisor.  This as always been a challenge of some sorts and usually the solution that I end up using is William Lam’s vSphere Health Check Report on top of the vSphere Management Assistant (VMA).

The vSphere Health Check report is an awesome perl script written by William Lam that generates a very thorough report containing almost everything you would need to know about a host and the VMs that are residing on it.  After the script is done running you get a nice little email such as the following….

As you can see there is a slew of information included in the health check report – and don’t take this the wrong way, this is all great information and very nice to have as a little report waiting for you in your inbox every morning  Howerver, the goal I had was to take this script, add it to a cronjob on my VMA, and modify the code so that I only get an email and the attached report if I have a hardware issue.  So to get started go ahead and get your VMA setup if you havn’t already (I think you can use any version of the remote CLI if you don’t have VMA) and go and download version 5.02 of the VMware vSphere Health Check report (The instructions to configure and install it are on the download page as well).

So, now with the customization, around Line 289 (your line numbers may differ as I’ve been in there hitting enter :)) I’ve added a few variables.  $HOSTISSUES is simply going to be a boolean variable that we will flag as yes if we run into any warnings or errors when parsing the hardware health and $emailMessage is a variable that will house those issues so we can place them directly inside the body of the message.

my $VM_TOOL="yes";
my $VMW_APP="yes";
my $VPX_SETTING="yes";

#mwpreston - additions 
my $HOSTISSUES="no"; 
my $emailMessage="";

############################
# START OF SCRIPT
############################

So on we go into the meat and bones of the script.  Around line1930 or so you should see where the script actually goes in and checks the health of the hosts.  Basically what I have done here is if the script finds any issues at all from within this section (anything that isn’t green) I will flag $HOSTISSUES to yes and append the issues to $emailMessage.   I’ve highlighted each spot below where I have added code but you can chose to add it where you like, meaning if you don’t care about a memory error, don’t include it there, etc…

######################
# HEALTH
######################
if($HOST_HEALTH eq "yes") {
	if($local_host->runtime->healthSystemRuntime) {
		if($local_host->runtime->healthSystemRuntime->hardwareStatusInfo) {
			my $hardwareStatusInfo = $local_host->runtime->healthSystemRuntime->hardwareStatusInfo;
			my ($cpuInfo,$memInfo,$storageInfo);
			$healthHardwareString .= "<tr><th align=\"left\">".$host_name."</th></tr>\n";
			my ($sensor_health_color,$sensor_health) = ("","");;

			if($hardwareStatusInfo->cpuStatusInfo) {
				$cpuInfo = $hardwareStatusInfo->cpuStatusInfo;
				foreach(@$cpuInfo) {
					$sensor_health = $_->status->key;
					if ($sensor_health =~ m/green/i) { 
						$sensor_health_color="<td bgcolor=\"$green\">OK</td>"; 
					}
					elsif ($sensor_health_color =~ m/red/i)	{ 
						$sensor_health_color="<td bgcolor=\"$red\">PROBLEM</td>";
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n"; 
					}
					elsif ($sensor_health_color =~ m/yellow/i) { 
						$sensor_health_color="<td bgcolor=\"$yellow\">WARNING</td>"; 
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n"; 
					}
					else { 
						$sensor_health_color="<td bgcolor=\"gray\">UNKNOWN</td>"; 
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n"; 
					}
					$healthHardwareString .= "<tr><td>".$_->name."</td>".$sensor_health_color."</tr>\n";		
				}
			}
			if($hardwareStatusInfo->memoryStatusInfo) {
				$memInfo = $hardwareStatusInfo->memoryStatusInfo;
				foreach(@$memInfo) {
					$sensor_health = $_->status->key;
					if ($sensor_health =~ m/green/i) { 
						$sensor_health_color="<td bgcolor=\"$green\">OK</td>"; 
					}
					elsif ($sensor_health_color =~ m/red/i) { 
						$sensor_health_color="<td bgcolor=\"$red\">PROBLEM</td>"; 
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n"; 
					}
					elsif ($sensor_health_color =~ m/yellow/i) { 
						$sensor_health_color="<td bgcolor=\"$yellow\">WARNING</td>"; 
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n"; 
					}
					else { 
						$sensor_health_color="<td bgcolor=\"gray\">UNKNOWN</td>"; 
					}
					$healthHardwareString .= "<tr><td>".$_->name."</td>".$sensor_health_color."</tr>\n";
				}
			}
			if($hardwareStatusInfo->storageStatusInfo) {
				$storageInfo = $hardwareStatusInfo->storageStatusInfo;
				foreach(@$storageInfo) {
					$sensor_health = $_->status->key;
					if ($sensor_health =~ m/green/i) { 
						$sensor_health_color="<td bgcolor=\"$green\">OK</td>"; 
					}
					elsif ($sensor_health_color =~ m/red/i) {
						$sensor_health_color="<td bgcolor=\"$red\">PROBLEM</td>";
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes"; 
						$emailMessage .= $_->name."\n";
					}
					elsif ($sensor_health_color =~ m/yellow/i) { 
						$sensor_health_color="<td bgcolor=\"$yellow\">WARNING</td>";
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes";
						$emailMessage .= $_->name."\n";
					}
					else { 
						$sensor_health_color="<td bgcolor=\"gray\">UNKNOWN</td>";
						#MWPRESTON - added following 2 lines
						$HOSTISSUES="yes";
						$emailMessage .= $_->name."\n";
					}
					$healthHardwareString .= "<tr><td>".$_->name."</td>".$sensor_health_color."</tr>\n";
				}
			}
		}
		if($local_host->runtime->healthSystemRuntime->systemHealthInfo) {
			my $sensors = $local_host->runtime->healthSystemRuntime->systemHealthInfo->numericSensorInfo;
			$healthSoftwareString .= "<tr><th align=\"left\">".$host_name."</th></tr>\n";
			my $sensor_health_color = "";
			foreach(sort {$a->name cmp $b->name} @$sensors) {
				my $sensor_health = $_->healthState->key;
				if ($sensor_health =~ m/green/) { 
					$sensor_health_color="<td bgcolor=\"$green\">OK</td>"; 
				}
				elsif ($sensor_health_color =~ m/red/) { 
					$sensor_health_color="<td bgcolor=\"$red\">PROBLEM</td>";
					#MWPRESTON - added following 2 lines
					$HOSTISSUES="yes";
					$emailMessage .= $_->name."\n";
				}
				elsif ($sensor_health_color =~ m/yellow/) { 
					$sensor_health_color="<td bgcolor=\"$yellow\">WARNING</td>"; 
					#MWPRESTON - added following 2 lines
					$HOSTISSUES="yes";
					$emailMessage .= $_->name."\n";
				}
				else { 
					$sensor_health_color="<td bgcolor=\"gray\">UNKNOWN</td>"; 
					#MWPRESTON - added following 2 lines
					$HOSTISSUES="yes";
					$emailMessage .= $_->name."\n";
				}
				my $reading;
				if(defined($_->rateUnits)) {
					$reading =  &restrict_num_decimal_digits(($_->currentReading * (10 ** $_->unitModifier)),3) . " " . $_->baseUnits . "/" . $_->rateUnits;
				} else {
					$reading =  &restrict_num_decimal_digits(($_->currentReading * (10 ** $_->unitModifier)),3) . " " . $_->baseUnits;
				}
				$healthSoftwareString .= "<tr><td>".$_->name."</td><td>".$reading."</td>".$sensor_health_color."</tr>\n";
			}	
		}
	}
}

And finally you need to modify the emailReport function in order to check for HOSTISSUES (since we don’t want to email unless we have an issue) and also attach $emailMessage to the body of the message.  This function is near the top of the script starting at roughly line 325.  My changes are below….

sub emailReport {
	#MWPRESTON - modified if statement to include HOSTISSUES
	if($email eq "yes" and $HOSTISSUES ne "no") {
		my $smtp = Net::SMTP->new($EMAIL_HOST ,Hello => $EMAIL_DOMAIN,Timeout => 30,);

		unless($smtp) {
	                die "Error: Unable to setup connection with email server: \"" . $EMAIL_HOST . "\"!\n";
        	}

		open(DATA, $report) || die("Could not open the file");
		my @report = <DATA>;
		close(DATA);

		my @EMAIL_RECIPIENTS = $smtp->recipient(@EMAIL_TO,{SkipBad => 1});
		my $boundary = 'frontier';
		$smtp->mail($EMAIL_FROM);
		$smtp->to(@EMAIL_TO);
		$smtp->data();
		$smtp->datasend('From: '.$EMAIL_FROM."\n");
		$smtp->datasend('To: '.@EMAIL_TO."\n");
		$smtp->datasend('Subject: VMware vSphere Health Check Report Completed - '.giveMeDate('MDYHMS'). " (" . $system_name . ")\n");
		$smtp->datasend("MIME-Version: 1.0\n");
		$smtp->datasend("Content-type: multipart/mixed;\n\tboundary=\"$boundary\"\n");
		$smtp->datasend("\n");
		$smtp->datasend("--$boundary\n");
		$smtp->datasend("Content-type: text/plain\n");
		$smtp->datasend("Content-Disposition: quoted-printable\n");
		#MWPRESTON - added following 2 lines.
		$smtp->datasend("\nProblems have been detected with the following\n");
		$smtp->datasend("\n$emailMessage\n");
		$smtp->datasend("\nReport $report is attached!\n");
		$smtp->datasend("--$boundary\n");
		$smtp->datasend("Content-Type: application/text; name=\"$report\"\n");
		$smtp->datasend("Content-Disposition: attachment; filename=\"$report\"\n");
		$smtp->datasend("\n");
		$smtp->datasend("@report\n");
		$smtp->datasend("--$boundary--\n");
		$smtp->dataend();
		$smtp->quit;
	}
}

If you would like the complete modified script you can get it here – honestly, I would recommend going and downloading Willams version first  and then make the required changes to it.  Since his script seems to be actively updated and worked on, whereas mine here will probably remain pretty static.  Again, I haven’t thoroughly tested this and it’s a use at your own risk type thing – and I’ve only applied these changes in version 5.02 of Williams script.

Again, any comments, concerns, questions, or if you just need help setting up the script or cron job let me know in the comments box below.