Tuesday, October 8, 2013

Powershell hashtables, dictionaries and objects

In computing, a hashtable is a data structure used to implement an associative array, or, in simple words, a structure that maps keys to values. In Powershell there's a pretty quick syntax to declare a hashtable, which is:

$my_hashtable = @{}
The previous command generates an empty hashtable whose possible methods are:

PS C:\> $my_hashtable | gm |
      ? { $_.membertype -like "method" } | select name

Name
----
Add
Clear
Clone
Contains
ContainsKey
ContainsValue
CopyTo
Equals
GetEnumerator
GetHashCode
GetObjectData
GetType
OnDeserialization
Remove
ToString
The are two ways to check the type of the $my_hashtable var. The first method is

$my_hashtable  | Get-Member | select typename -Unique

TypeName
--------
System.Collections.Hashtable
The second method is:

if($my_hashtable -is [hashtable]){"True!"}
which can be shortened to:

if([hashtable]$my_hashtable){"True!"}
One common use for using associative arrays is to store inventory information, which is one of the common tasks in system administration. Let's see an example.

$my_inventory = @{}
$my_inventory.add("Computername",$(Get-wmiobject win32_operatingsystem).csname)
$my_inventory.add("OS",$(Get-wmiobject win32_operatingsystem).caption)
$my_inventory.add("Uptime",(get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime))
$my_inventory.add("Make",$(get-wmiobject win32_computersystem).model)
$my_inventory.add("Manufacturer",$(get-wmiobject win32_computersystem).manufacturer)
$my_inventory.add("MemoryGB",$(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int])
$my_inventory.add("Processes",(Get-Process).Count)
Here's the resulting associative array:

PS C:\> $my_inventory

Name                           Value
----                           -----
Make                           HP EliteBook 8460p
Manufacturer                   Hewlett-Packard
Computername                   WORKSTATION001
MemoryGB                       4
OS                             Microsoft Windows 7 Enterprise
Uptime                         15.03:14:27.3149599
Processes                      108
This was the beginning of Powershell. Over the time smart people at Microsoft have improved the syntax and since Powershell V3 the same result can be obtained with much less typing:

$my_inventory = @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname;
 OS = $(Get-wmiobject win32_operatingsystem).caption;
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime);
 Make = $(get-wmiobject win32_computersystem).model;
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer;
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int];
 Processes = (Get-Process).Count
 }
The semi-colon at the end of each line are also optional and can be skipped if you like. Once again the Powershell interpreter shows us its strength!

$my_inventory = @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }
The big drawback of using associative arrays for inventory information is that keys can't be accessed directly as properties of the object, but we have to call the 'Keys' property:

PS C:\> $my_inventory | select keys

Keys
----
{Manufacturer, MemoryGB, Uptime, Make...}
Now Powershell is an object-oriented language and therefore it's better to save our inventory information in a object (the proper name for this is PSObject) for better accessibility. Let's see how that's done:

$my_inventory = New-Object psobject -Property @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }
Using Get-Member gives us an insight look at the variable type:

PS C:\> $my_inventory | gm


   TypeName: System.Management.Automation.PSCustomObject

Name         MemberType   Definition
----         ----------   ----------
Equals       Method       bool Equals(System.Object obj)
GetHashCode  Method       int GetHashCode()
GetType      Method       type GetType()
ToString     Method       string ToString()
Computername NoteProperty System.String Computername=WORKSTATION001
Make         NoteProperty System.String Make=HP EliteBook 8460p
Manufacturer NoteProperty System.String Manufacturer=Hewlett-Packard
MemoryGB     NoteProperty System.Int32 MemoryGB=4
OS           NoteProperty System.String OS=Microsoft Windows 7 Enterprise
Processes    NoteProperty System.Int32 Processes=107
Uptime       NoteProperty System.TimeSpan Uptime=15.03:30:47.3217513
As you can see now, each key has been added as a NoteProperty to the original blank PSObject. That's cool because now each and every piece of information inside that PSObject can be retrieved in a quick manner using Select-Object:

PS C:\> $my_inventory | select computername,uptime

Computername         Uptime
------------         ------
WORKSTATION001       15.03:38:02.6384875
Over the years Powershell has evolved more and more and starting with V3 in 2012 you can use the [PSCustomObject] shortcut. Using that new syntax will allow for creation of a PSObject right from the start, skipping the step of making a call to New-Object. Let's see an example:

$my_inventory = [PSCustomObject]@{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }
The type of this PSObject stays exactly the same whether we use the older syntax (New-Object psobject -Property @{...) or the new syntax ([PSCustomObject]). On the other hand there is a remarkable difference in speed, which I am going to show you here because actions speak louder than words. If we take out the WMI queries (which are CPU intensive and could bias our test), we hardcode a fake value in a key, and use an appropriate unit of measure (ticks in this case) here's what we get:

'Ten runs for psobject + hashtable'
1..10 | % {
 measure-command -expression {
  $my_inventory = New-Object psobject -Property @{
   Computername = 'hardcoded'
  }
 } | select ticks
 }
'Ten runs for PSCustomObject'
1..10 | % {
 measure-command -expression {
  $my_inventory = [PSCustomObject]@{
   Computername = 'hardcoded'
  }
 } | select ticks
 }

Ten runs for psobject + hashtable
Ticks
-----
13637
5936
4863
6927
5336
4792
5431
4914
4598
3505
Ten runs for PSCustomObject
5688
1223
1101
1128
1081
1113
1318
1093
1073
1089
Our theory is confirmed, with 4000 ticks average for the older New-Object method and 1000 ticks average for the new [PSCustomObject] method. The two actions of setting up a hashtable and passing it to a PSObject take four times longer then just setting up a PSCustomObject. And what's more a PSCustomObject retains the order of the Noteproperties, which in some cases could be important for us. So the newest syntax [PSCustomObject] will be our preferred way of storing our inventory information most of the time.

In the meantime hashtables have evolved, I daresay, and since Powershell V3 a new keyword[ordered], which is a shortcut for OrderedDictionary, has been introduced. If you use the [Ordered] type, the key order is retained, which is a feature hashtables did not have:

PS C:\> $my_hashtable = @{"1"="Monday";"2"="Tuesday";"3"="Wednesday"}
PS C:\> $my_hashtable

Name                           Value
----                           -----
2                              Tuesday
1                              Monday
3                              Wednesday
See? The keys are sorted in an unpredictable order, which is something that can be a problem in some cases (like when listing the days of the weeks).

Using [ordered] gives us a quick way to present an object with the keys in good order:

PS C:\> $my_dictionary = [ordered]@{"1"="Monday";"2"="Tuesday";"3"="Wednesday"}
PS C:\> $my_dictionary

Name                           Value
----                           -----
1                              Monday
2                              Tuesday
3                              Wednesday
Why did I name this last variable $my_dictionary, you ask. Well, the use of the [ordered] keyword returns something different from the previous hashtables. It returns a OrderedDictionary.

PS C:\>  $my_dictionary  | gm


   TypeName: System.Collections.Specialized.OrderedDictionary

Name              MemberType            Definition
----              ----------            ----------
Add               Method                void Add(System.Object key, System.Object va
AsReadOnly        Method                System.Collections.Specialized.OrderedDictio
Clear             Method                void Clear(), void IDictionary.Clear()
Contains          Method                bool Contains(System.Object key), bool IDict
CopyTo            Method                void CopyTo(array array, int index), void IC
Equals            Method                bool Equals(System.Object obj)
GetEnumerator     Method                System.Collections.IDictionaryEnumerator Get
GetHashCode       Method                int GetHashCode()
GetObjectData     Method                void GetObjectData(System.Runtime.Serializat
GetType           Method                type GetType()
Insert            Method                void Insert(int index, System.Object key, Sy
OnDeserialization Method                void IDeserializationCallback.OnDeserializat
Remove            Method                void Remove(System.Object key), void IDictio
RemoveAt          Method                void RemoveAt(int index), void IOrderedDicti
ToString          Method                string ToString()
Item              ParameterizedProperty System.Object Item(int index) {get;set;}, Sy
Count             Property              int Count {get;}
IsFixedSize       Property              bool IsFixedSize {get;}
IsReadOnly        Property              bool IsReadOnly {get;}
IsSynchronized    Property              bool IsSynchronized {get;}
Keys              Property              System.Collections.ICollection Keys {get;}
SyncRoot          Property              System.Object SyncRoot {get;}
Values            Property              System.Collections.ICollection Values {get;}
A quick look at the properties gives us the confirmation that an ordered dictionary is more like a hashtable then like an object, since the properties have not been added to the empty canvas but just stored in the pre-existing 'Keys' property. Therefore [ordered] should be used as an 'upgrade' of an hashtable and not as an alternative to a PSCustomObject.

So, you see, we have started from the basics and we have improved our knowledge of handling object in Powershell. This is my preferred approach. Now that we can make the difference between an associative array, an ordered dictionary and a PSObject, let's see how we can use another interesting Powershell feature to improve our scripts readability.

The feature I'm talking about is named splatting. Ok, it has a funny name, but it's one of the best feature aimed at making your scripts easily understandable and way shorter.

Let's suppose we want to declare four different objects: one hashtable, one dictionary, one object which takes in a hashtable and, last but not least, a PSCustomObject. Now suppose we want to retrieve the Type of each of these four objects. The script we would write without splatting would look like this:

# Defining an associative array
$CustomObject1 = @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

# Defining a dictionary
$CustomObject2 = [ordered]@{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

# Defining a hashtable a passing it to new-psobject
$CustomObject3 = New-Object psobject -Property  @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

# Defining a PSCustomObject
$CustomObject4 = [PSCustomObject]@{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

$CustomObject1 | gm | select typename -Unique
$CustomObject2 | gm | select typename -Unique
$CustomObject3 | gm | select typename -Unique
$CustomObject4 | gm | select typename -Unique
Now this script has lot or repeated code, so why not to 'deduplicate' it? Splatting does exactly that. We store these objects in a splatted variable then pass it to the final objects. Here's the improved code:

$splatting = @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

$CustomObject1 = $splatting
$CustomObject2 = [PSCustomObject]$splatting
$CustomObject3 = New-Object psobject -Property $splatting
$CustomObject4 = [ordered]$splatting

$CustomObject1 | gm | select typename -Unique
$CustomObject2 | gm | select typename -Unique
$CustomObject3 | gm | select typename -Unique
$CustomObject4 | gm | select typename -Unique
Shorter, isn't it? Much more readable, easier to debug, easier to improve. Unfortunately there is a little problem with that. When we run the script we get an error:

At line:1 char:18
+ $CustomObject4 = [ordered]$splatting
+                  ~~~~~~~~~~~~~~~~~~~
The ordered attribute can be specified only on a hash literal node.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : OrderedAttributeOnlyOnHashLiteralNode
The problem is that we cannot use the [ordered] keyword to convert or cast a hashtable. Unfortunate, since our code looked so pretty.

The only workaround I found is to declare a new variable and to move the [ordered] keyword to it:

$splatting = @{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }

$orderedsplatting = [ordered]@{
 Computername = $(Get-wmiobject win32_operatingsystem).csname
 OS = $(Get-wmiobject win32_operatingsystem).caption
 Uptime = (get-date) - ([wmiclass]"").ConvertToDateTime((Get-wmiobject win32_operatingsystem).LastBootUpTime)
 Make = $(get-wmiobject win32_computersystem).model
 Manufacturer = $(get-wmiobject win32_computersystem).manufacturer
 MemoryGB = $(Get-WmiObject win32_computersystem).TotalPhysicalMemory/1GB -as [int]
 Processes = (Get-Process).Count
 }
    
$CustomObject1 = $splatting
$CustomObject2 = [PSCustomObject]$splatting
$CustomObject3 = New-Object psobject -Property $splatting
$CustomObject4 = $orderedsplatting

$CustomObject1 | gm | select typename -Unique
$CustomObject2 | gm | select typename -Unique
$CustomObject3 | gm | select typename -Unique
$CustomObject4 | gm | select typename -Unique
I hope this sort introduction to hashtable and PSObjects helps you to better understand how Powershell works and that you will keep this post for future reference. Do not hesitate to comment if you have any question. I am also on Twitter (@sysadm2010), should you want to get in touch with me.

4 comments:

  1. Great article! Very informative.

    ReplyDelete
  2. Awesome post, je l'avais pas encore lu :-)
    I was using the post from Glenn Sizemore to refer to this kind of information
    http://powershell.org/wp/2013/05/10/scripting-games-week-2-formatting-edition/

    I will use your post too now ;-)

    ReplyDelete
    Replies
    1. I wan't aware of the post by Glenn. Btw performancewise he gets different results from me. That's because he measure the gwmi request time too, while I filtered it out, I reckon.
      Otherwise he does a very good job of checking different IPs, which is more useful for inventorying.
      Merci FX :-)

      Delete

Related Posts Plugin for WordPress, Blogger...