Tuesday, December 27, 2016

A function for robust command execution in PowerShell

I have always been a great fan of a tool named RoboCopy, which I bet many of you have used countless times. Now these days I have been in need for this very same type of robustness for one of my functions since I am running it in an unreliable environment.

What I wanted to achieve with PowerShell in particular, was to fetch a great deal of data from public web servers and reuse the information in my scripts. Unfortunately when you use a cmdlet, be it Invoke-RestMethod, or Test-Connection, you can get failures which are not due to your cmdlets but to the underlying infrastructure (such as a Wi-Fi network, a distorted topology because of a flapping router or a way too busy web server).

Sure Invoke-RestMethod has a TimeoutSec parameter, but what if it fails and I really need the information coming from that website? Well, this reasoning brought me to write an advanced function that takes a command and its parameters and tries to run it a given number of times (three by default) with intervals of three seconds.

This function, which I called Start-RoboCommand (Start is an approved verb, so that my PSScriptAnalyzer is happy, and I borrowed the idea of the RoboCommand noun from RoboCopy itself), also supports an improved functioning where the command is run indefinitely, through the addition of a -Wait parameter, such as the one you can find in recent versions of Get-Content.

To finish with, I added a LogFile parameter to log errors (which is particularly important here because we are exactly dealing with commands not being successfull) and Verbose, which tells you exactly what's going wrong.


Now without further ado, here's my function:

function Start-RoboCommand {

<#
.Synopsis
   Function that tries to run a command until it succeeds or forever
.DESCRIPTION
   Function that tries to run a command until it succeeds or forever. By default this function tries to run a command three times with three seconds intervals.
.PARAMETER Command
    Command to execute
.PARAMETER Args
    Arguments to pass to the command
.PARAMETER Count
    Number of tries before throwing an error
.PARAMETER Wait
    Run the command forever even if it succeeds
.PARAMETER Delay
    Time in seconds between two tries
.PARAMETER LogFile
    The path to the error log
.EXAMPLE
   Start-RoboCommand -Command 'Invoke-RestMethod' -Args @{ URI = "http://guid.it/json"; TimeoutSec = 1 } -Count 2 -Verbose
.EXAMPLE
   Start-RoboCommand -Command 'Invoke-RestMethod' -Args @{ URI = "http://notexisting.it/json"; TimeoutSec = 1 } -Count 2 -Verbose
.EXAMPLE
   Start-RoboCommand -Command 'Invoke-RestMethod' -Args @{ URI = "http://guid.it/json"; TimeoutSec = 1 } -Wait -Verbose
.EXAMPLE
   Start-RoboCommand -Command 'Invoke-RestMethod' -Args @{ URI = "http://notexisting.it/json"; TimeoutSec = 1 } -Wait -Verbose
.EXAMPLE
   Start-RoboCommand -Command 'Test-Connection' -Args @{ ComputerName = "bing.it" } -Wait -Verbose
.EXAMPLE
   Start-RoboCommand -Command 'Test-Connection' -Args @{ ComputerName = "nocomputer" } -Wait -LogFile $Env:temp\error.log -Verbose
.EXAMPLE
   Start-RoboCommand -Command Get-Content -Args @{path='d:\inputfile.txt'} -Wait -DelaySec 2 -LogFile $Env:temp\error.log -Verbose
.NOTES
   happysysadm.com
   @sysadm2010
#>

    [CmdletBinding(SupportsShouldProcess,DefaultParameterSetName='Limited')]
    Param (
    
    [Parameter(Mandatory=$true)]
    [Alias("Cmd")]
    [string]$Command, 

    [Parameter(Mandatory=$true)]
    [hashtable]$Args, 

    [Parameter(Mandatory=$false,ParameterSetName = 'Limited')]
    [int32]$Count = 3, 

    [Parameter(Mandatory=$false,ParameterSetName = 'Forever')]
    [switch]$Wait,

    [Parameter(Mandatory=$false)]
    [int32]$DelaySec = 3,

    [Parameter(Mandatory=$false)]
    $LogFile
    )
    
    $Args.ErrorAction = "Stop"
        
    $RetryCount = 0

    $Success = $false
    
    do {

        try {

            & $command @args

            Write-Verbose "$(Get-Date) - Command $Command with arguments `"$($Args.values[0])`" succeeded."

            if(!$Wait) {
                
                $Success = $true

                }
            
            }
        
        catch {

            if($LogFile) {

                "$(Get-Date) - Error: $($_.Exception.Message) - Command: $Command - Arguments: $($Args.values[0])" | Out-File $LogFile -Append

                }
            
            if ($retrycount -ge $Count) {

                Write-Verbose "$(Get-Date) - Command $Command with arguments `"$($Args.values[0])`" failed $RetryCount times. Exiting."

                $PSCmdlet.ThrowTerminatingError($_)
                
                }

            else {

                Write-Verbose "$(Get-Date) - Command $Command with arguments `"$($Args.values[0])`" failed. Retrying in $DelaySec seconds."

                Start-Sleep -Seconds $DelaySec

                if(!$Wait) {
                
                    $RetryCount++

                    }

                }

            }

        }

    while (!$Success)

 }

Let me know how it works for you and if you have any suggestion on the logic I'll be more than happy to improve it over time. Fo sure you can find it also on my github.

Thursday, December 8, 2016

Spotlight on the PSReadline PowerShell module

The trend is clear: Microsoft has shifted some major projects, like .NET and PowerShell itself, into the open-source ecosystem, and has made them cross-platform. Today you can run your PowerShell scripts on a GUI-less Windows Server Core, or on a headless Nano Server, but also on Linux, and on a Mac.

There is a project in particular which reveals this kind of cross-pollination between OSes, and it is the PSReadline module, which is aimed at bringing the GNU Readline experience to your PowerShell console.
This module is installed by default on Windows 10 and brings some slick functionalities which are well worth a quick look.
The first functionality is the fact that with PSReadline, the console preserves command history across sessions. Ok, you were used to Get-History to find the list of the typed commands, and to use Invoke-History (aliased as 'r') to run commands found in the history. But these two cmdlets are limited to the current session:

Now with the arrival of PSReadline, which is loaded by default when you start a PowerShell console, you got the possibility to retrieve commands typed in previous sessions, even across reboots. This is achieved through log files stored inside the Application Data folder:
  • $env:APPDATA\Microsoft\Windows\PowerShell\PSReadline\ConsoleHost_history.txt for the PowerShell console host (conhost.exe)
  • $env:APPDATA\Microsoft\Windows\PowerShell\PSReadline\Windows PowerShell ISE Host_history.txt for the Integrated Scripting Environment (ISE)
  • $env:APPDATA\Microsoft\Windows\PowerShell\PSReadline\Visual Studio Code Host_history.txt for Visual Studio Code, the new toy for those into DevOps
How I discovered that? Simple. The PSReadline module comes with five cmdlets:
  • Get-PSReadlineKeyHandler: gets the key bindings for the PSReadline module
  • Get-PSReadlineOption: gets values for the options that can be configured
  • Remove-PSReadlineKeyHandler: removes a key binding
  • Set-PSReadlineKeyHandler: binds keys to user-defined or PSReadline-provided key handlers
  • Set-PSReadlineOption: customizes the behavior of command line editing in PSReadline.
If you issue “(Get-PSReadlineOption).HistorySavePath” you will get the location where the system keeps the command history for your current interpreter.

Now for some reason, the only working log between those listed above is the one for PowerShell on the command line, probably because PowerShell ISE and VSCode don't have a true console (conhost.exe) behind it:


Being the Application Data folder user-specific, you only have access to the command history for your user account: so there is one ConsoleHost_history.txt file for each user on a given computer. The permissions are set in a way that the admin can access the command history for other users, which is good for checking your systems.

Here's a script I wrote to retrieve a list of all the consolehost_history.txt files on my systems, so that I know who used PowerShell and when:
(Get-ChildItem -Path c:\users).name | % {

     Get-Item ((Get-PSReadlineOption).HistorySavePath -replace ($env:USERNAME,$_)) -ErrorAction SilentlyContinue

     } | Select-Object FullName,

                       CreationTime,

                       LastWritetime,

                       @{Name="Kbytes";Expression={ "{0:N0}" -f ($_.Length / 1Kb) }},

                       @{Name="Lines";Expression={(Get-Content $_.fullname | Measure-Object -Line).Lines}}
To prevent PowerShell from logging any command just type:
Set-PSReadlineOption –HistorySaveStyle SaveNothing
Other interesting settings that you could adopt or make custom are:
Set-PSReadLineOption -HistoryNoDuplicates
and
Set-PSReadLineOption -MaximumHistoryCount 40960
I wouldn't bother changing the HistorySaveStyle because the default parameter seems well suited to me: SaveIncrementally means that every run command is stored in the log before being actually executed.

If you want to erase you command history, you can just press ALT+F7, as you can discover by issuing:
Get-PSReadlineKeyHandler | ? Function -eq 'clearhistory'

Key    Function     Description
---    --------     -----------
Alt+F7 ClearHistory Remove all items from the command line history (not PowerShell history)
The second functionality is the possibility to access and search the history log in an interactive way. What I mean is that you can use your keyboard to search the history by pressing combinations of keys. The discovery of the existing keys is performed with:
Get-PSReadlineKeyHandler | ? function -like '*history*'

Key       Function                Description
---       --------                -----------
UpArrow   PreviousHistory         Replace the input with the previous item in the history
DownArrow NextHistory             Replace the input with the next item in the history
Ctrl+r    ReverseSearchHistory    Search history backwards interactively
Ctrl+s    ForwardSearchHistory    Search history forward interactively
Alt+F7    ClearHistory            Remove all items from the command line history (not PowerShell history)
F8        HistorySearchBackward   Search for the previous item in the history that starts with the current input - like PreviousHistory if the input is empty
Shift+F8  HistorySearchForward    Search for the next item in the history that starts with the current input - like NextHistory if the input is empty
Unbound   ViSearchHistoryBackward Starts a new seach backward in the history.
Unbound   BeginningOfHistory      Move to the first item in the history
Unbound   EndOfHistory            Move to the last item (the current input) in the history
As you can see pressing Ctrl+r will bring up a bottom-top search (identified by bck-i-search), and just start typing and PSReadline will complete the lines with commands from the history logfile:


The third functionality is the fact that PSReadLine allows you to mark, copy, and paste text in the common Windows way. It is actually just like if  you were in Word: CTRL+C copies text, CTRL+X cuts text, and CTRL+V pastes the text. CTRL+C can still be used to abort a command line, but when you select some text, with the CTRL+SHIFT+ArrowKeys key combination for instance, PSReadline will switch to CTRL+C Windows mode. Awesome.

The fourth functionality is syntax checking as you type. When PSReadline detects a syntax error it turns the grater-than-sign on the left to red, like in the following example where I forgot to close the double quotes after the $Computer variable:


If to all these functionalities you add the syntax coloring provided by PSReadline, or also the possibility to use key combinations like CTRL+Z to undo code changes, then there you are with a PowerShell console that is a delight to use. And that you can even install on your old Windows 7 by installing WMF v5 and then running the following line of code to get the module from the PowerShell Gallery:
Install-Module -Name PSReadline
Now just choose your way. Here's a comparative screenshot of the four main development environment I use:


Happy coding.
Related Posts Plugin for WordPress, Blogger...