Chart

Display vSCSIStats Data With Microsoft Chart Controls

@mpoore

• 6 min read

I had to run vSCSIStats for the first time in a while the other day. Normally, trying to display the data that it outputs in a graphical form involves some cutting and pasting and a wrestling match with Excel. I recalled this time that someone had tried to automate the process in the past and a quick google search sent me to Gabe's site and an article that he wrote back in February. The Excel macro (written by Paul Dunn) that was the subject of the article looked like just the job but I couldn't get it to work for some reason. I'm not very good with vbscript and so I started to wonder if it might be possible to do it with PowerShell…

Another quick google suggested Microsoft Chart Controls for .NET 3.5 as a potentially useful tool. Originally I had planned to simply write a script that used Excel as I have done in the past. However, that approach might not have been the quickest and easiest thing to do.

Although there are quite a few people using the Chart Controls, most of the content out there focuses on coding / scripting languages other than PowerShell. The best PowerShell example that I found was in this article by Richard Macdonald. It's a great introduction and well worth a read. In fact, as you'll below, it formed the foundation of my script.

So, first things first, some pre-requisites:

That's it. Except of course that we need some data.

The environment that I'm working in uses ESX still rather than ESXi. As such, vSCSIStats is already present and easy to get to using an SSH connection the the server hosting the VM that you're interested in. Once on the server I had to obtain the worldGroupID of the VM that I wanted to monitor. This was done simply by issuing the following command:

[shell]/usr/lib/vmware/bin/vscsiStats -l[/shell]

Which returned the following output:

[shell]Virtual Machine worldGroupID: 6146, Virtual Machine Display Name: XXXXXXX {
Virtual SCSI Disk handleID: 8193
Virtual SCSI Disk handleID: 8194
Virtual SCSI Disk handleID: 8195
Virtual SCSI Disk handleID: 8196
Virtual SCSI Disk handleID: 8197
Virtual SCSI Disk handleID: 8198
}[/shell]

As you can see, the worldGroupID for the VM is 6146. The disks listed are the Hard Disks presented to the VM in the order in which they are listed in the VM's settings. A bit detective work is required to work out which disk ID maps to which drive letter in the VM. Fortunately, it was easy enough to discover that the disk that I was most interested in has an ID of 8194.

Now, to run vSCSIStats and collect some data I had to issue the following command:

[shell]/usr/lib/vmware/bin/vscsiStats -s -w 6146[/shell]

And this is the output that I received:

[shell]vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8193
Success.
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8194
Success.
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8195
Success.
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8196
Success.
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8197
Success.
vscsiStats: Starting Vscsi stats collection for worldGroup 6146, handleID 8198
Success.[/shell]

I left it running for about 30 minutes and then used the following command to export the histogram data to a CSV file:

[shell]/usr/lib/vmware/bin/vscsiStats -p all -w 6146 -c > /root/vscsiStats-export.csv[/shell]

I also used this command to stop the stats collection from continuing:

[shell]/usr/lib/vmware/bin/vscsiStats -x[/shell]

Finally, I used Veeam's FastSCP tool to copy the CSV file back to my desktop machine.

Looking inside the CSV file using a text editor, I saw this:

[text]Histogram: IO lengths of commands,virtual machine worldGroupID,6146,virtual disk handleID,8193
min,512
max,65536
mean,4456
count,13257
Frequency,Histogram Bucket Limit
6165,512
75,1024
567,2048
575,4095
2428,4096
72,8191
2270,8192
222,16383
636,16384
110,32768
16,49152
10,65535
111,65536
0,81920
0,131072
0,262144
0,524288
0,524288[/text]

This is the data that I will use to create a single histogram. The CSV file will generally contain many more datasets. One for each metric multiplied the the number of disks present. This is why it could be so tedious copying and pasting the data into Excel.

Nothing needs to be changed in the CSV file, the script that I wrote works with it as it is. All I did was place it in the same folder and the script and then executed the script from within PowerShell. There was nothing exciting to see when it ran in the console but within a few seconds a histogram was displayed using the data from above. Each dataset from the CSV file was rendered as a histogram in turn. Simply by closing the graph window, the script moved onto the next dataset and rendered it until I got to the ones that I wanted. Remember the disk handleID from earlier? It was 8194. This is the IO latency histogram for that disk:

Displaying the data and interpreting the data are two different things so I won't go into it much now. You can see though that disk IO latency varies between 0.5ms and 100ms and mostly sits at 15 – 30ms. That's what I wanted to know.

So, now for the script. As I mentioned earlier, I based it around the work that Richard Macdonald had already done so you'll see some sections of code that are very similar looking to snippets from his article. The rest I cobbled together so the whole thing is a bit clunky. I'll give it a polish sometime soon.

[ps]###########################################################
# vSCSIStats-Histograms.ps1
#
# Version:    0.1
# Author:    Michael Poore (www.vspecialist.co.uk)
#
# Uses Microsoft Chart Controls to draw charts from CSV data
# outputed by vSCSIStats
###########################################################

$thisXData = ""
$thisYData = ""
$rawData = Import-Csv "vscsistats-export.csv" -Header "Field","Value","worldGroupID","diskHandleIDHeader","diskHandleID"

$dataCollection = @()

foreach( $row in $rawData )
{
$row.Field -match "([a-zA-Z0-9-()]+)" | Out-Null
switch( $matches[0] )
{
"Histogram" {
if( test-path variable:thisHistogram )
{
$thisHistogram | Add-Member -Name XData -Value $thisXData -Membertype NoteProperty
$thisHistogram | Add-Member -Name YData -Value $thisYData -Membertype NoteProperty
$dataCollection += $thisHistogram
}
Remove-Variable -Name thisXData
Remove-Variable -Name thisYData
$thisHistogram = New-Object PSObject
$thisHistogram | Add-Member -Name Title -Value $row.Field -Membertype NoteProperty
$thisHistogram | Add-Member -Name worldGroupID -Value $row.worldGroupID -Membertype NoteProperty
$thisHistogram | Add-Member -Name diskHandleID -Value $row.diskHandleID -Membertype NoteProperty
$thisXData = @()
$thisYData = @()
}
"min" {
$thisHistogram | Add-Member -Name Min -Value $row.Value -Membertype NoteProperty
}
"max" {
$thisHistogram | Add-Member -Name Max -Value $row.Value -Membertype NoteProperty
}
"mean" {
$thisHistogram | Add-Member -Name Mean -Value $row.Value -Membertype NoteProperty
}
"count" {
$thisHistogram | Add-Member -Name Count -Value $row.Value -Membertype NoteProperty
}
"Frequency" {
#Do nothing
}
default {
#Assumed to be data
$thisXData += $row.Value
$thisYData += $row.Field
}
}
}

#Load Charting Assemblies
[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[void][Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms.DataVisualization")

foreach( $chart in $dataCollection )
{

$thisTitle = $chart.Title + " (worldGroupID " + $chart.worldGroupID + ", diskHandleID " + $chart.diskHandleID + ")"

#Create Chart Object
$thisChart = New-object System.Windows.Forms.DataVisualization.Charting.Chart
$thisChart.Width = 900
$thisChart.Height = 600
$thisChart.Left = 40
$thisChart.Top = 30

#Create ChartArea
$ChartArea = New-Object System.Windows.Forms.DataVisualization.Charting.ChartArea
$ChartArea.AxisX.Interval = 1
$ChartArea.AxisX.TextOrientation = "Rotated90"
$thisChart.ChartAreas.Add($ChartArea)

[void]$thisChart.Series.Add("Data")
$thisChart.Series["Data"].Points.DataBindXY($chart.XData, $chart.YData)

#Add Title
[void]$thisChart.Titles.Add($thisTitle)

#Change Chart Colour
$thisChart.BackColor = [System.Drawing.Color]::Transparent

#Make Bars Into 3d Cylinders
$thisChart.Series["Data"]["DrawingStyle"] = "Cylinder"

# add a save button
$SaveButton = New-Object Windows.Forms.Button
$SaveButton.Text = "Save"
$SaveButton.Top = 10
$SaveButton.Left = 900
$SaveButton.Anchor = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right
$SaveButton.add_click({$thisChart.SaveImage($Env:USERPROFILE + "Desktop" + [System.Text.RegularExpressions.Regex]::Replace($thisTitle,"[^1-9a-zA-Z_]","_") +".png", "PNG")})

# display the chart on a form
$thisChart.Anchor = [System.Windows.Forms.AnchorStyles]::Bottom -bor [System.Windows.Forms.AnchorStyles]::Right -bor [System.Windows.Forms.AnchorStyles]::Top -bor [System.Windows.Forms.AnchorStyles]::Left
$thisForm = New-Object Windows.Forms.Form
$thisForm.Text = $thisTitle
$thisForm.Width = 1000
$thisForm.Height = 700
$thisForm.controls.add($SaveButton)
$thisForm.controls.add($thisChart)
$thisForm.Add_Shown({$thisForm.Activate()})
$thisForm.ShowDialog()

}[/ps]

vSCSIStats-Histograms

Subsequently to writing this script and getting the charts and information that I wanted, I discovered that David Owen (@vMackem) has been experimenting with Chart Controls also. We compared notes over a few beers after IP Expo yesterday. His article is a good read and efficiently produces a nice pie chart from his script. I'm looking forward to hearing about his other discoveries with Chart Controls, PowerShell and PowerCLI.