Tuesday, February 14, 2017

Using the PoshRSJob module with a PowerShell function to perform a WinRM check up

As I explained in the previous post, I have written a function to test whether the WinRM service on a remote computer is able to accept connections and effectively execute commands. It all started with the finding that in very large environments with mixed OS versions, you can only be assured of the proper functioning of WinRM by trying to execute an actual command on a remote host. Only if the execution of that command succeeds you can safely state that the whole PowerShell remoting subsystem is correctly configured end-to-end.

The WinRM host process in action on the target server

Now you probably know that you have a couple of cmdlets that can be used to test this (I am thinking to Test-WSMan and Connect-WSMan), and that you can use Invoke-Command to run a block of code on a remote computer.

What I wanted to achieve here goes a bit further. I wanted a PowerShell function capable of testing all the possible configurations in a large environment with a high execution speed.

This involves testing things as the different authentication mechanisms such as Kerberos or NTLM, as well as testing against closed TCP ports, if any.

And this involves including some kind of parallelization.

Now to make a long story short, I have split the tests I perform in a function that I called Test-PSRemoting (whose full code you can find on my GitHub) in five blocks, where each block is accessed in the function through a Switch parameter.

Just to be clear, a Switch parameter is a parameter whose value is False by default and gets set to True when it is included. Here's a neat example I wrote to show you how this kind of parameter works:

function Get-SwitchValue ([switch]$switch1, [switch]$switch2)
 "Switch1 is $switch1."
 "Switch2 is $switch2."   

Get-SwitchValue -switch2

Switch1 is False.
Switch2 is True.
So as I said, there are five regions, which are only accessed if the corresponding Switch parameter is set to True.

The first Switch, named $Ping, is for the region of code where a ping is sent to the target server using the System.Net.NetworkInformation.Ping class:

The second Switch, named $Resolve, is used to query the DNS and return the IP address of the target server. This is accomplished with Resolve-DnsName with a query type set to A, so that only the IPv4 address is returned:

The third Switch, which I named $TCPCheck, is called when you want to check that the TCP ports used by the WinRM service are open on the destination server. As you might know, there are two ports for WinRM:
  • TCP port 5985 is for HTTP traffic and is used when you don't need to authenticate the target server because you can rely on Kerberos and on the Active Directory to authenticate it for you
  • TCP port 5986 is for HTTPS traffic and is used when you can't rely on an Active Directory  domain to authenticate the target server (like when it is in a Workgroup) and therefore you require that that target server identity is confirmed by a certificate issued by a trusted CA
Now, recent PowerShell versions have a native cmdlet for checking open TCP ports which is called Test-NetConnection. This cmdlet is designed in a way that it can be used to check for the standard WinRM port 5985 in a quick manner:
Test-NetConnection srv01 -CommonTCPPort WinRM
The issue is that this cmdlet seems always trying to ping the remote server before issuing the TCP connection. Since I haven't been able to determine whether using it with the InformationLevel parameter set to Quiet suppresses the ping, I have decided to fail back to use Windows Sockets through the System.Net.Sockets class provided in .NET framework. This has the important advantage of letting me use AsyncWaitHandle to handle timeouts shorter than the one of Test-NetConnection when used against a server which is unresponsive:
Measure-Command {([System.Net.Sockets.TcpClient]::new().BeginConnect('nosrv',5985,$null,$null)).AsyncWaitHandle.WaitOne(100,$False)}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 125
Ticks             : 1256004
TotalDays         : 1.45370833333333E-06
TotalHours        : 3.4889E-05
TotalMinutes      : 0.00209334
TotalSeconds      : 0.1256004
TotalMilliseconds : 125.6004

Measure-Command {Test-NetConnection nosrv -port 5985}
WARNING: Name resolution of nosrv failed -- Status: No such host is known

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 2
Milliseconds      : 306
Ticks             : 23065325
TotalDays         : 2.66959780092593E-05
TotalHours        : 0.000640703472222222
TotalMinutes      : 0.0384422083333333
TotalSeconds      : 2.3065325
TotalMilliseconds : 2306.5325
As you can see, the Test-NetConnection method is twenty times slower, and like I have said, speed is one of my main requirement for the function I am writing.

The next region, $Dcom, is where I check if DCOM can be used to retrieve the operating system of the target server as well as the name of the Active Directory Domain it belongs to. Actually this is kind of an optional part of my function: there's no link between WinRM and DCOM but it can always be interesting to know if you can switch back to DCOM/RPC to query the WMI provider on the remote host. Here's how I use the New-CimSessionOption to force my request to go over the DCOM protocol:
$SessionOp = New-CimSessionOption –Protocol DCOM
Also, always having that idea of making my function go fast, and to be robust in case the remote WMI provider is for some reasons broken, I use the New-CimSession cmdlet with a OperationTimeOut parameter set to 1 seconds. Here's the block of code for the DCOM check:

Now the final block of code. This is the part where I perform the following tests:
  • Test-WSMan with a challenge-response scheme named Negotiate, that allows me to authenticate the account I am using with Kerberos and to switch back to NTLM in case it fails
  • Test-WSMan with Negotiate on port 80, which is the old TCP port used for WinRM on Windows 2003 servers (and I have still a few of them in the place I am using this function)
  • Invoke-Command with Negotiate: in this case, since the cmdlet doesn't have a Timeout parameter, I run it in a background job which I discard after two seconds
  • Test-WSMan with Kerberos authentication
  • Test-WSMan with Kerberos authentication on port 80 for servers trunning Windows 2003 as base operating system
  • Invoke-Command with Kerberos authentication
As you can easily understand, the test with Invoke-Command is the most important part of the function since it effectively tries to retrieve the list of running services on the target server over WSMan:

Now that was for the Test-PSRemoting function.

Concerning the parallelization of the execution, I have gone down a few roads: first of all I have tried to build a quick and dirty RunspacePool but soon discovered that their implementation is so developerish that it goes well beyond what a system administrator should be able to know and understand. In the end I have decided to choose the easy path and reuse a module written and maintained by fellow MVP Boe Prox which adds a layer of abstraction to the runspaces below and makes it easy to use for the classic system administrator. The name of the module is PoshRSJob and you can find it here.

To install this module just run:

Install-Module -Name PoshRSJob
Version has the following cmdlets:

Get-Command -Module PoshRSJob

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        Get-RSJob                                    PoshRSJob
Function        Receive-RSJob                                PoshRSJob
Function        Remove-RSJob                                 PoshRSJob
Function        Start-RSJob                                  PoshRSJob
Function        Stop-RSJob                                   PoshRSJob
Function        Wait-RSJob                                   PoshRSJob
These cmdlets can be used in a oneliner fashion, just by piping Start-RSJob into Wait-RSJob and then into Receive-RSJob:

Start-RSJob -InputObject ('1/1/2017','2/2/2017') -ScriptBlock { Get-Date $_ } | Wait-RSJob | Receive-RSJob

Sunday, January 1, 2017 12:00:00 AM
Thursday, February 2, 2017 12:00:00 AM
Before you use my function, it is important to understand how this module accesses variables which should be available in the runspace. This is done through $Using, like in the example below, where I add one day to a given date:

$One = 1
Start-RSJob -InputObject ('1/1/2017','2/2/2017') -ScriptBlock { (Get-Date $_).AddDays($Using:One) } | Wait-RSJob | Receive-RSJob

Monday, January 2, 2017 12:00:00 AM
Friday, February 3, 2017 12:00:00 AM
It is also important to force Start-RSJob to evaluate any function you want to use in your parallel execution. This is done through the FunctionsToLoad parameter, which in my case I use to load the Test-PSRemoting function.

The last hint about this module is that you should make an heavy use of the Verbose parameter to follow whatever is happening in your runspaces and also to add a nice and useful progress bar via the ShowProgress switch inside the Wait-Job cmdlet.

So let's see now a few examples of how I use the PoshRSJob module to make the execution of my Test-PSremoting function lightning fast. In the first example I retrieve all the Windows Server names from the Active Directory, and I check whether they ping and can be reached via Invoke-Command, to return only those that actually passed this last test and save their data in a CSV file:

$Report = Start-RSJob -Throttle 20 -Verbose -InputObject ((Get-ADComputer -server dc01 -filter {(name -notlike 'win7*') -AND (OperatingSystem -Like "*Server*")} -searchbase "OU=SRV,DC=Domain,DC=Com").name) -FunctionsToLoad Test-PSRemoting -ScriptBlock {Test-PSRemoting $_ -Ping -Kerberos -Credential $using:cred -Verbose} | Wait-RSJob -Verbose -ShowProgress | Receive-RSJob -Verbose

$Report | ? Remoting_Kerberos -eq 'ok' | convertto-csv -Delimiter ',' -NoTypeInformation | Out-File C:\WinRM-after-gpo.csv
In the second example I use an ADSISearcher to query an old Windows 2003 Domain to retrieve all the Windows Servers and then I try all the different blocks of code (ping, name resolution, TCP check, NTLM, Kerberos) to return only the servers that actually responded to ping in a table:
$Servers = ((New-Object -typename ADSISearcher -ArgumentList @([ADSI]"LDAP://domain.com/dc=domain,dc=com","(&(&(sAMAccountType=805306369)(objectCategory=computer)(operatingSystem=*Server*)))")).FindAll()).properties.name

$Report = Start-RSJob -Throttle 20 -Verbose -InputObject $Servers -FunctionsToLoad Test-PSRemoting -ScriptBlock {Test-PSRemoting -Ping -Resolve -TCPCheck -DCOM -Negotiate -Kerberos $_ -Credential $using:cred -Verbose} | Wait-RSJob -Verbose -ShowProgress | Receive-RSJob -Verbose

$Report | ? ping -eq 'ok' | format-table * -AutoSize
For sure you can imagine here any kind of grouping of your results, with Group-Object, or you could print the results in a dynamic table with Out-GridView. What you do will depend on your needs.

Kudos to Boe for the PoshRSJob module. If you have any question on the function I wrote, or if you want to improve it, feel free to get in touch with me.

No comments:

Post a Comment

Related Posts Plugin for WordPress, Blogger...