Thursday, August 4, 2016

Measuring IOPS part 5: enter PowerShell

We keep our quest to understand how we can use DISKSPD to measure storage performance. As suggested by Jose Barreto and as I have discussed in the previous post of this series, we need to run the tool with a large number of workloads to get a grip of this complex matter.

To do so, I use the magic of a PowerShell script that iterates through combinations of
  • 4KB to 2MB IOs
  • 1 to 64 threads per file
  • 1 to 64 queue depth
to perform 10 seconds long read operations (I reckon this impacts me more than write operations) on a 1GB file.

I also capture latency information (by means of the –L parameter) and disable caching (adding the –h parameter).

Here’s the script. I have reused some parts of Jose Barreto’s blog code to build mine (so kudos to him for that):

$container = @()
$blocksize = 4,8,16,32,64,128,512,1024,2048
$thread = 1,2,4,8,16,32,64
$outstandingIO = 1,2,4,8,16,32,64
$duration = 10
$warmup = 5
$rest = 2
$combination = $blocksize.count * $thread.count * $outstandingIO.count
"TEST SPEC".PadRight(50,'*')
"Test started at $(Get-Date)"
"$Combination combinations".PadRight(50,'*')
"Duration of each test: $duration seconds".PadRight(50,'*')
"Warm-up time of each test: $warmup seconds".PadRight(50,'*')
"Rest between tests: $rest seconds"
"Predicted finish time: $((Get-Date) + (New-TimeSpan -seconds $($combination * ($duration + $warmup + $rest))))".PadRight(50,'*')
$a = 0 
$blocksize | % {
    $b = "-b$_" + "k"
    [int]$k = $_
    $thread | % {
        $c = "-t$_"
        [int]$t = $_
        $outstandingIO | % {
            $d ="-o$_"
            $o = $_
            Write-Progress -Id 1 -Activity ("Checking IOPS") -PercentComplete ($a / $combination * 100) -Status ("Checked {0} combinations of {1}" -f $a, $combination)   
            try {
                $ErrorActionPreference = 'Stop'
                $cmd = "diskspd $b $c $d -w0 -c1G -r -h -d$duration -W$warmup -L C:\test.dat"
                $result = Invoke-Expression $cmd
                foreach ($line in $result) {if ($line -like "total:*") { $total=$line; break } }
                foreach ($line in $result) {if ($line -like "avg.*") { $avg=$line; break } }
                [double]$mbps = $total.Split("|")[2].Trim() 
                [double]$iops = $total.Split("|")[3].Trim()
                [double]$latency = $total.Split("|")[4].Trim()
                [double]$cpu = $avg.Split("|")[1].Trim() -replace '%'   
                "Param $b, $c, $d, $iops iops, $mbps MB/sec, $latency ms, $cpu %CPU"
                $current = [PSCustomObject]@{
                    Bytes =$k
                    Threads = $t
                    Queuedepth = $o
                    iops = $iops
                    MBs = $mbps
                    Latencyms = $latency
                    PercentCPU = $cpu
            catch {
                "Param $b, $c, $d failed"
                $current = [PSCustomObject]@{
                    Bytes =$k
                    Threads = $t
                    Queuedepth = $o
                    iops = 'NA'
                    MBs = 'NA'
                    Latencyms = 'NA'
                    PercentCPU = 'NA'
            $container += $current
            Start-Sleep -Seconds $rest

That's a very simple scripts that user iteration to build a list of results combined with the test specs. Here's a screenshot of the script while running with its progress bar:

This is particularly useful for filtering the data afterward: having the results from more than 400 iterations inside a custom object, gives us the possibility to check for some empirical evidence generated from within the test.

In the next post of this series we will see what results it will give. Stay tuned!

1 comment:

  1. Why wouldn't you just use MS's code to process the results?


Related Posts Plugin for WordPress, Blogger...