Wednesday, December 17, 2014

How to manage the FSMO roles with Powershell

There was a time when I used legacy commands to do my daily duties. Most of you have at least used them once or more: stuff like dcpromo.exe, comp.exe, netdom.exe, xcopy.exe, tasklist.exe and taskkill.exe. It was a time when the Windows system administrators where the laughing stocks of Unix sysadmins, and for a good reason! Our skils were limited by the possibilities of the graphical user interface and we had a limited understanding of the way those old black boxes (read NT4, Windows 2000, 2003) worked under the hood. The console was just a joke, which we barely used to do a few ipconfigs or pings.

Today Microsoft is opening. There are new possibilities: Virtualization was a great (or giant?) step forward. The Cloud brought a new way of thinking to datas and to system administration. Interconnecting systems requires today a greater view of what's behind our operating systems. That's how I came to Windows Powershell. I wanted something better to do my job. I wanted to open and learn a way of managing computers which could give me satisfaction beyond that upsetting mouse clicking. Something that could make me feel prouder.

Powershell was the answer to this. A plyayful tool which in good hands can give great results.

Let me take an example: FSMO roles.

We all have an Active Directory out there indexing users, computers and resources. In these old times when you wanted to check what were the holders for each and every role you had to move down to the console and run something like this:

netdom query fsmo

Schema master     
Domain naming master
RID pool manager  
Infrastructure master

The command completed successfully.
Very static. I don't even want to show you how long and intricated was the procedure to learn how to move or seize those roles in case you needed to.

Today with Powershell we have a new way to manage those FSMO roles and make those Flexible Single Master Roles more... flexible! I want to show you now a few lines of code that are used to shuffle around your domain controllers the FSMO roles at random.

But a bit of theory first.

There are five roles: two at forest level and three at domain level. The Schema Master is the first Forest role and is in charge of keeping a writable copy of the Schema partition. The Domain Naming Master is the second forest role and keeps a writeable copy of the Configuration partition (that is the place where the logical view of your domains and trusts and the physical view of your Active Directory sites are stored). Then, at Domain level, there is the PDC Emulator, which manages time synchronization between clients and DCs (have ever heard about the 5 minutes kerberos ticket lifetime?), as well as GPO edition. The RID Master, which gives RID scopes to other DCs in the Domain. And the Infrastructure Master, which manages groups coherency between domains.

The two Forest roles can be found with Get-AdForest:

Get-ADForest | Select-Object SchemaMaster,DomainNamingMaster

SchemaMaster                         DomainNamingMaster
------------                         ------------------        
The three Domain roles can be found with Get-AdDomain:

Get-ADDomain | Select-Object PDCEmulator,RIDMaster,InfrastructureMaster

PDCEmulator             RIDMaster               InfrastructureMaster
-----------             ---------               --------------------
Once you know that a list of all your domain controllers can be generated with:

Get-ADDomainController -filter * | select -ExpandProperty Name
then you can put in place a powerful oneliner that moves each FSMO role to a random DC in less than a second. Like shuffling and serving cards for a game, but with FSMO roles instead:

0..4 | % {Move-ADDirectoryServerOperationMasterRole -Identity (Get-Random (Get-ADDomainController -filter * | select -ExpandProperty Name)) -OperationMasterRole $PSItem -Confirm:$False -Verbose}
Notice the 0..4 | % {}. This is a quick way to index the five FSMO roles without having to manually write them down: Powershell will make the translation for you:
  • PDCEmulator: 0
  • RIDMaster: 1
  • InfrastructureMaster: 2
  • SchemaMaster: 3
  • DomainNamingMaster: 4
Here's the result:

Now I could be willing to play more and shuffle my FSMO roles around every day, then send me an e-mail in case I loose a DC, so that I know what roles were on it in that particular moment. I would store the e-mail in a dedicated folder and have an history of all role moves in my Domain.

Let's prepare the e-mail. The ingredients here are a couple of variables, a bit of splatting, a PSObject and a connection to a valid mail server:

$ForestFSMORoles = Get-ADForest | Select-Object SchemaMaster,DomainNamingMaster

$DomainFSMORoles = Get-ADDomain | Select-Object PDCEmulator,RIDMaster,InfrastructureMaster

$FSMORoles = New-Object PSObject -Property @{

                    InfrastructureMaster = $DomainFSMORoles.InfrastructureMaster

                    PDCEmulator = $DomainFSMORoles.PDCEmulator

                    RIDMaster = $DomainFSMORoles.RIDMaster

                    DomainNamingMaster = $ForestFSMORoles.DomainNamingMaster

                    SchemaMaster = $ForestFSMORoles.SchemaMaster


$EmailSplatting = @{
            To = ''

            From = ''

            Subject = "FSMO Roles owners of the day for $((Get-AdForest).Name)"

            Body = $FSMORoles | ConvertTo-Html | Out-String

            SMTPServer = ''

            BodyAsHtml = $True

Send-MailMessage  @EmailSplatting
Just put the code in a .ps1, put the ps1 in a scheduled task and you're done.
What do you think? Funny, isn't it? Of course I am not reponsible of what you do with your FSMO roles. I am just showing the power of Powershell and the fun it is to make the most out of it. Limited only by our imagination.

Wednesday, December 3, 2014

Powershell oneliner to get disk usage by file extension

These days storing multimedia files (like family videos, travel pictues, etc) requires a lot of storage and since I am a big fan of SSDs (and those kind of disks are still quite expensive), I wanted to show the simple Powershell one-liner I wrote in the weekend that checks what kind of file extensions are (over)consuming space on all my local drives.

The first step is to retrieve a list of all my local drives, which resumes to a query to Win32_LogicalDisk:

(Get-CIMInstance -class Win32_LogicalDisk -Filter 'drivetype=3').DeviceID
A lot of drives, so a lot of work that Powershell will silently do for me.

Let's start to run Get-ChildItem against this list of local drives and hunt for every file in every subfolder:

Get-ChildItem $((Get-CIMInstance -class Win32_LogicalDisk -Filter 'drivetype=3').DeviceID) `
-File -Recurse
Once you know (thanks Get-Member!) that Get-ChildItem returns the file extension as a full grown property, you can ask Powershell to make the grouping for you:

Get-ChildItem $((Get-CIMInstance -class Win32_LogicalDisk -Filter 'drivetype=3').DeviceID) `
-File -Recurse |

 Group-Object Extension
The next step is to send this result down the pipeline to Select-Object, which is very good at defining what I call 'calculated properties'.

The properties I need for my report are:
- the extension itself
- the total size of all the files for the given extension
- the Count of those files

Since Group-Object returns four properties:

Count       Property   int Count {get;}                                                
Group       Property   System.Collections.ObjectModel.Collection[psobject] Group {get;}
Name        Property   string Name {get;}                                              
Values      Property   System.Collections.ArrayList Values {get;} 
we have to build on top of this to get what we want.

Let's start by letting Select-Object chew the Group properties (which is a PSObject, as you can see above) to get the total file size for each extension:

... | Select-Object ..., @{n='Size';e={$($_.Group | Measure-Object Length -Sum).Sum}}, ...

Name        Size Count
----        ---- -----
.JPG  2507712043  2704
.jpeg     319979     1
.mp4  1605261912   119
.tif    22024960     1

Very well, now we have to beautify this to make it readable. I'll perform rounding on the size of the files after having converted it to megabytes:

... | Select-Object Name, @{n='Size (MB)';e={[math]::Round((($_.Group | Measure-Object Length -Sum).Sum / 1MB), 2)}}, Count
Then I'll change the name property to Extension, suppress the dot at the beginning of the string (through some very complex regex!), and make the resulting string uppercase:

... | Select-Object @{n='Extension';e={($_.Name -replace '^\.').ToUpper()}}, ...
Here's the final result, sorted by total file size by extension:

gci $((gcim Win32_LogicalDisk -Filter 'drivetype=3').DeviceID) -File -Rec |
  Group Extension |

  Select @{
   e={($_.Name -replace '^\.').ToUpper()}

   n="Size (MB)";
   e={[math]::Round((($_.Group | Measure Length -Sum).Sum/1MB),2)}

   Count |

  Sort 'Size (MB)' -Desc

Extension Size (MB) Count
--------- --------- -----
JPG         2391,54  2704
MP4          1530,9   119
TIF              21     1
JPEG           0,31     1
Of course the one-liner can be modified to taste.

You could for instance pass to Get-ChildItem a manual list of drives or folders in the form or an array:

Get-ChildItem @('c:\','d:\','n:\pictures','n:\videos') -File -Recurse
Or you could use a different measure unit from MB. In Powershell V5 there are for instances five measure units: KB, MB, GB, TB and PB. The one you choose depends on the amount of files you have.

Powershell, anything is possible.

Friday, November 28, 2014

First look at ConvertFrom-String in Powershell v5

I've been playing with ConvertFrom-String for some days now and must admit that this cmdlet is to me one of the best improvement that came with WMF5.0 and Powershell v5.
For the moment there are not so many resources to learn from, but those that exist (especially the ones by fellow MVP François-Xavier Cart) are very well written. Let me mention them, so that this gives you a starting point for getting a good grasp on the mechanism behind it:
In addition to this posts there is a magic interface by Doug Finke on GitHub that allows you to quickly test your ConvertFrom-String template against any set of data. So, before you proceed, I definitively suggest you copy Doug's interface on your test bed to speed up your learning curve.

Ok, now that you have checked those resources and that you know that ConvertFrom-String is aimed at adding structure to unstructured string content, let's quickly get to the basic syntax.

ConvertFrom-String supports two ParameterSets:

ConvertFrom-String [-Delimiter ] [-PropertyNames ] -InputObject  []

ConvertFrom-String [-TemplateContent ] [-TemplateFile ] -InputObject  []
The cmdlet's default parameter set is FromDelimiter, which splits any line of text on whitespaces, so, while this does nothing

PS C:\> '12345' | ConvertFrom-String
this works and generates a certain amount of properties:

PS C:\> '1 2 3 4 5' | ConvertFrom-String

P1 : 1
P2 : 2
P3 : 3
P4 : 4
P5 : 5
The properties are dinamically casted to a type:

PS C:\> '1 2 3 4 5' | ConvertFrom-String | Select-Object -ExpandProperty P1 | Get-Member

   TypeName: System.Byte

PS C:\> 'a b c d e f g' | ConvertFrom-String | Select-Object -ExpandProperty P1 | Get-Member

   TypeName: System.Char
But I don't want to spend more time speaking of this, since Doug and François-Xavier did a great job of analyzing the basics of this cmdlet in their blog posts. I just suggest you get yourself familiar with the way the -TemplateFile parameter works against a txt file and the way the -TemplateContent parameter works againsts a Here-String before you proceed.

I want to try to build a concrete example of how to use ConvertFrom-String to build a structured object starting from the output of the legacy command Tracert.exe (guys, I know there is the 'Test-NetConnection -TraceRoute' option, but for the sake of this test I want to base my work on a legacy command that is not supposed to produce anything other then raw text).

Tracert prints a text which looks like this:

Tracing route to [] over a maximum of 30 hops:

  1     1 ms    <1 ms    <1 ms  HOST1 [] 
  2     1 ms     1 ms     1 ms [] 
  3    11 ms    12 ms    12 ms [] 
  4    15 ms    15 ms    12 ms 
  5     9 ms    16 ms    16 ms 
  7    40 ms    40 ms    41 ms 
  8    38 ms    39 ms    39 ms 
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
 12    38 ms    39 ms    39 ms

Trace complete
From this printed output I want to extract the name of each device I cross on my route to the target and its IP address.

As you can understand, our ConvertFrom-String template based example, which is what we want to use here to get those messy data sorted out, needs to take care of different facts which I am going to oversimplify here:
  • the device name can come with or without a DNS suffix
  • the device name can be lowercase or uppercase or a mix of both
  • the device name can be missing!!!
  • the IP address can come inside square brackets or alone
  • there can be lines with a 'Request timed out' text instead of a pair Devicename/IPaddress
  • there are additional lines of text at the beginning and at the end of the output ('Tracing route...' and 'Trace complete...')
I copy paste the output of Tracert (I suggest you use the clip command: Tracert | clip) inside Doug's ConvertFrom-String Buddy in both the Data and the Template textboxes

ConvertFrom-String Buddy interface by fellow MVP Doug Finke

then start modifying the template box. I want to create a HostInfo sequence on each line, so, following the well-known syntax
I modify the first line of the template as follow:

1     1 ms    <1 ms    <1 ms  {HostInfo*:{Computername:HOST1} [{IPAddress:}]} 
and I get the following result:

{@{ExtentText=HOST1 []; Computername=HOST1; IPAddress=}}              
{@{ []; Computername=host2; IPAddress=}}  
{@{ []; Computername=host3; IPAddress=}}
Ok, looks like ConvertFrom-String is learning from my example as expected. Great, but I want the parse engine to take also FQDNs, so I modify the second line

2     1 ms     1 ms     1 ms  {HostInfo*:{} [{IPAddress:}]} 
and this modification allow me to get the whole FQDN for the second and third record:

{@{ExtentText=HOST1 []; Computername=HOST1; IPAddress=}}                                
{@{ [];; IPAddress=}}        
{@{ [];; IPAddress=}}
This was the easy part.
Now I have to get a sequence even when the device name is missing. I must admit I had a hard time getting beyond this and was able to solve only thanks to the help of Gustavo (which you can reach at psdmfb @ microsoft com).

The key here is to tell the parse engine that the ComputerName property is optional on each line of the example, then to add a negative example for ComputerName on the third example so that Powershell does not incorrectly identify an IP as a Computername when this property is missing.

To make it simple, here's the syntax to use:

The first part of the template then becomes (notice the question marks):

  1     1 ms    <1 ms    <1 ms  {HostInfo*:{Computername?:HOST1} [{IPAddress:}]} 
  2     1 ms     1 ms     1 ms  {HostInfo*:{Computername?} [{IPAddress:}]} 
  4    15 ms    15 ms    12 ms  {HostInfo*: {IPAddress:{!Computername?:}}} 
and returns:

{@{ExtentText=HOST1 []; Computername=HOST1; IPAddress=}}
{@{ [];; IPAddress=}}
{@{ [];; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=Request timed out.; Computername=Request}}
{@{ExtentText=Request timed out.; Computername=Request}}
{@{ExtentText=Request timed out.; Computername=Request}}
{@{ExtentText=; IPAddress=}}
Woah! I got what I want plus some undesired text: {@{ExtentText=Request timed out.; Computername=Request}}

If from the Template I remove the line with

9     *        *        *     Request timed out.
I get an error:

ConvertFrom-String appears to be having trouble parsing your data using the template you’ve provided.
So I have to make my HostInfo examples more expressive so the parser knows what kind of data I am after. Nothing easier. I just have to add an example which tells ConvertFrom-String that the IP address can come alone:

14    10 ms    10 ms     7 ms  {HostInfo*:{IPAddress:}} 
and, there you are, we have a working template which output the following result:

{@{ExtentText=HOST1 []; Computername=HOST1; IPAddress=}}
{@{ [];; IPAddress=}}
{@{ [];; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
{@{ExtentText=; IPAddress=}}
What's brilliant here is that Doug's ConvertFrom-String Buddy generate all the required code for you to copy paste it into your ISE and keep up working on the resulting object. What else?

On top of this you could add any possible action, like in the following example:

$targetData = @'
Tracing route to [] over a maximum of 30 hops:

  1     1 ms    <1 ms    <1 ms  HOST1 [] 
  2     1 ms     1 ms     1 ms [] 
  3    11 ms    12 ms    12 ms [] 
  4    15 ms    15 ms    12 ms 
  5     9 ms    16 ms    16 ms 
  7    40 ms    40 ms    41 ms 
  8    38 ms    39 ms    39 ms 
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
 12    38 ms    39 ms    39 ms

Trace complete

$TemplateContent = @'
Tracing route to [] over a maximum of 30 hops:

  1     1 ms    <1 ms    <1 ms  {HostInfo*:{Computername?:HOST1} [{IPAddress:}]} 
  2     1 ms     1 ms     1 ms  {HostInfo*:{Computername?} [{IPAddress:}]} 
  4    15 ms    15 ms    12 ms  {HostInfo*: {IPAddress:{!Computername?:}}} 
  9     *        *        *     Request timed out.
  14    10 ms    10 ms     7 ms  {HostInfo*:{IPAddress:}} 

Trace complete

$targetData |
    ConvertFrom-String -TemplateContent $TemplateContent |
    Select-Object -ExpandProperty HostInfo |
    Select-Object IPAddress | % {
                                "Your code to run against $($_.IPAddress) goes here"
If you want to get in the deep of ConvertFrom-String, focus on the use of the -Debug switch:

$targetData |
    ConvertFrom-String -TemplateContent $TemplateContent -Debug

DEBUG: Property: HostInfo
Program: ESSL((Contains(Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), Dot(\.), Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), 1)): 1, 2, ...: Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?)
, Dynamic Token(\ ms\ \ )(\ ms\ \ )...ε, 3 + ε...ε, 0)
Property: Computername
Program: ESSL((EndsWith(Dot(\.), Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), Right Bracket(\]))): 0, 1, ...: ε...ε, 1 + ε...WhiteSpace(( )+), Left Bracket(\[), Number([0-9
]+(\,[0-9]{3})*(\.[0-9]+)?), 1)
Property: IPAddress
Program: ESSL((Contains(Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), Dot(\.), Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), 1)): 0, 1, ...: ε...Number([0-9]+(\,[0-9]{3})*(\.[0-9]
+)?), Dot(\.), Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), 1 + Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?), Dot(\.), Number([0-9]+(\,[0-9]{3})*(\.[0-9]+)?)...ε, 1)
As you can understand using this debugging option, and to make a long story short, ConvertFrom-String is an intelligent tool that writes regex for you! Let's make a simple example to better grasp the concept:

$targetData = @'

$TemplateContent = @'

$targetData | ConvertFrom-String -TemplateContent $TemplateContent -Debug

DEBUG: Property: Letter
Program: ESSL((EndsWith(all lower((?<![\p{Lu}\p{Ll}])(\p{Ll})+))): 0, 1, ...: ε...ε, 1 + ε...ε, 0)
The regex above is trying to match the given text against some Unicode categories. Without going too deep in this vast subject, what you must know is that each Unicode char matches one or often more than one category. An example of category is \p{L} or \p{Letter} which represents all the possible letters whatever the language. Another exampe of category is \p{N} or \p{Number} that contains any numeric char out there.
In the simple example above the ConvertFrom-String engine tries to match lowercase (\p{Ll}) and uppercase (\p{Lu}) letters. Got it?
That's all for the moment on ConvertFrom-String. The last piece of information I can give is that ConvertFrom-String has an alias: cfs.
If you want to learn more, know that there is a brilliant video on text parsing by fellow MVP Tobias Weltner (minutes 24 to 32 are specific to ConvertFrom-String) In its video, Tobias shows also the power of ISESteroids console, so it it worth checking it out.
Tobias Weltner on ConvertFrom-String with ISESteroids
If you liked this post, be kind, share!

Tuesday, November 11, 2014

Of Powershell, Pester, DSC and...

It's been since 2009 that I am using Powershell as my favourite administration tool and I must admit it has never evolved as fast as this year. A full bunch of new products and of interesting concepts are born and growing fast both in the language itself and in its surroudings, making it one of the key competence to have in the Windows IT world nowadays.
Beside this, Microsoft is pushing its Windows development as fast as it can, and the release of Windows 10 (and of its Server version), with its ton of new cmdlets, is a clear sign of the fact that people in Redmond are doing their best to build on the success of Powershell.
On top of this, the community is strongly focusing on making Powershell (which is a tool built for sysadmins and aimed at system administration) take advantage of well established software development processes like Test-Driven Development (TDD). The star on stage here is named Pester, a project started by Scott Muc a few years back, which is a Unit-Testing solution for your Powershell code.
Last but not least, by now you should have heard of Desired State Configuration, which, starting with Windows 2012 R2 and Powershell 4.0, is a configuration management solution aimed at preventing configuration drift in your environment. In a few words, DSC provides a set of PowerShell language extensions, in the form of new Windows PowerShell cmdlets, that you can use to declaratively specify how you want your environment to be configured. Talking of DSC with your colleagues, you will hear a lot about the concept of idempotency, which, to make it simple, is the way DSC ensures that the environment desired state will be reached by applying the entire configuration, regardless of the current state.
I am currently having fun testing all of this, and, let me tell you, if you are new to Powershell, it's time to get a grasp on it before it is too late.

To stimulate your curiosity, heres' some screenshots from my current labs.

Here you can see that in Windows 10 the Start button is back:

Same for the Server version:

Both come with Powershell 5.0, the ultimate version:

Installing Pester on Windows 10 is now a breeze, thanks to the Package Manager I have already talked of in a previous blog post. Just run

Find-Package Pester
to check which version is available (version 3.1.1 at the time of testing), then:

Install-Package Pester
and you'll get it installed.

Here's a screenshot of the full installation process:

The Pester module will appear in you Module folder:

These are the cmdlets it comes with (I'll come back on them in a future post):

Get-Command -Module Pester

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Function        AfterEach                                          3.1.1      Pester
Function        Assert-MockCalled                                  3.1.1      Pester
Function        Assert-VerifiableMocks                             3.1.1      Pester
Function        BeforeEach                                         3.1.1      Pester
Function        Context                                            3.1.1      Pester
Function        Describe                                           3.1.1      Pester
Function        Get-MockDynamicParameters                          3.1.1      Pester
Function        Get-TestDriveItem                                  3.1.1      Pester
Function        In                                                 3.1.1      Pester
Function        InModuleScope                                      3.1.1      Pester
Function        Invoke-Mock                                        3.1.1      Pester
Function        Invoke-Pester                                      3.1.1      Pester
Function        It                                                 3.1.1      Pester
Function        Mock                                               3.1.1      Pester
Function        New-Fixture                                        3.1.1      Pester
Function        Set-DynamicParameterVariables                      3.1.1      Pester
Function        Setup                                              3.1.1      Pester
Function        Should                                             3.1.1      Pester

So now, there are a lot of new things to learn out there. It's time for you to start having a look at all of this and that's why I am giving you a short list of recent and interesting blog posts to start from.
Stay tuned for more on Powershell!

Monday, November 10, 2014

How to use Powershell to setup a GPO for Script Execution Policy and for WinRM

Last week, during the deployment of a new Active Directory Domain, I was challenged by a colleague of mine to write a Powershell  function that performs  the two following jobs on all the computers in the new Domain:
  • set the script Execution Policy to RemoteSigned
  • enable Powershell Remoting for remote management
To be honest I never thought it had any real use to modify my Default Domain Policy to apply the above settings using Powershell instead of the GUI, since this is a kind of job you do once and for all in your Domain and Powershell brings no added value, unless.... unless you can re-use the same function for all your labs requiring a new Domain. So I accepted the challenge.
The function I came up with is the result of a lot of try and guess. I studied the existing policies on others Domain I have and came to the conclusion that the operations my function had to perform were be aimed at modifying a few registry keys, as well as to modify GptTmpl.inf, which is a INF file that keeps the security setting for a given GPO.
Let's see this in detail.
To change the Execution Policy for scripts, you have to play with the  HKLM\Software\Policies\Microsoft\Windows\Powershell key, and
  • set the value of ExecutionPolicy to RemoteSigned
  • set the value of EnableScripts to 1
Setting up Powershell Remoting configuration is a bit trickier, since you have to work on two fronts. First of all you must go under the HKLM\Software\Policies\Microsoft\Windows\WinRM\Service key and
  • set AllowAutoConfig to 1
  • set IPv4Filter and the IPv6Filter setting to * (or whatever settings is good for you)
Then you have also to set the WinRM service to start automatically, by adding it to the GptTmpl.inf file of your GPO. In my case I applied all this settings to my 'Default Domain policy' but of course you can create a dedicated GPO for this. It's up to you how to tackle the subject in your Domain.

Here's the complete function I wrote:
#Requires –Modules ActiveDirectory

function Set-PowershellDomainPolicy()
    $domain = (Get-ADDomain).forest
    $id = (Get-GPO -name 'Default Domain Policy').id
    $ExecutionPolicyParams = @{
            name='Default Domain Policy';
    try {
        Set-GPRegistryValue @ExecutionPolicyParams -ValueName ExecutionPolicy -Value RemoteSigned -Type String -ErrorAction Stop
        Set-GPRegistryValue @ExecutionPolicyParams -ValueName EnableScripts -Value 1 -Type DWord -ErrorAction Stop
        "Script execution policy changed succesfully!"
    catch { "Error changing script execution policy" }

    $RemotingParams = @{
            Name='Default Domain Policy';
            Key = 'HKLM\Software\Policies\Microsoft\Windows\WinRM\Service';
    try {
        Set-GPRegistryValue @RemotingParams -ValueName 'AllowAutoConfig' -Value 1 -Type DWord
        Set-GPRegistryValue @RemotingParams -ValueName 'IPv4Filter' -Value '*' -Type String
        Set-GPRegistryValue @RemotingParams -ValueName 'IPv6Filter' -Value '*' -Type String
        "Registry setting for Powershell Remoting OK!"
    catch { "Error enabling remoting policy" }

     #Setting up the here-string
    $inf = @'
[Service General Setting]

    try {
        $inf |
            Out-File "C:\Windows\SYSVOL\sysvol\$domain\Policies\{$id}\MACHINE\Microsoft\Windows NT\SecEdit\GptTmpl.inf" -Append -ErrorAction Stop
        "WinRM startup set to Automatic in GptTmpl.inf!"
    catch { "Error setting WinRM automatic startup"}

Which you can run by executing:
As you can see, the central cmdlet of my function is Set-GPRegistryValue, which, If I remember well, exists from Powershell 4.0. This cmdlet takes as parameter the GUID of the GPO, which I retrieve using Get-GPO.
The Policy runs in a few seconds and its ouput is shown below:
DisplayName      : Default Domain Policy
DomainName       :
Owner            : CONTOSO\Domain Admins
Id               : 31b2f340-016d-11d2-945f-00c04fb984f9
GpoStatus        : AllSettingsEnabled
Description      : 
CreationTime     : 05/11/2014 08:18:48
ModificationTime : 10/11/2014 15:44:48
UserVersion      : AD Version: 0, SysVol Version: 0
ComputerVersion  : AD Version: 25, SysVol Version: 25
WmiFilter        : 

DisplayName      : Default Domain Policy
DomainName       :
Owner            : CONTOSO\Domain Admins
Id               : 31b2f340-016d-11d2-945f-00c04fb984f9
GpoStatus        : AllSettingsEnabled
Description      : 
CreationTime     : 05/11/2014 08:18:48
ModificationTime : 10/11/2014 15:44:48
UserVersion      : AD Version: 0, SysVol Version: 0
ComputerVersion  : AD Version: 26, SysVol Version: 26
WmiFilter        : 

Script execution policy changed succesfully!

DisplayName      : Default Domain Policy
DomainName       :
Owner            : CONTOSO\Domain Admins
Id               : 31b2f340-016d-11d2-945f-00c04fb984f9
GpoStatus        : AllSettingsEnabled
Description      : 
CreationTime     : 05/11/2014 08:18:48
ModificationTime : 10/11/2014 15:44:48
UserVersion      : AD Version: 0, SysVol Version: 0
ComputerVersion  : AD Version: 27, SysVol Version: 27
WmiFilter        : 

DisplayName      : Default Domain Policy
DomainName       :
Owner            : CONTOSO\Domain Admins
Id               : 31b2f340-016d-11d2-945f-00c04fb984f9
GpoStatus        : AllSettingsEnabled
Description      : 
CreationTime     : 05/11/2014 08:18:48
ModificationTime : 10/11/2014 15:44:48
UserVersion      : AD Version: 0, SysVol Version: 0
ComputerVersion  : AD Version: 28, SysVol Version: 28
WmiFilter        : 

DisplayName      : Default Domain Policy
DomainName       :
Owner            : CONTOSO\Domain Admins
Id               : 31b2f340-016d-11d2-945f-00c04fb984f9
GpoStatus        : AllSettingsEnabled
Description      : 
CreationTime     : 05/11/2014 08:18:48
ModificationTime : 10/11/2014 15:44:48
UserVersion      : AD Version: 0, SysVol Version: 0
ComputerVersion  : AD Version: 29, SysVol Version: 29
WmiFilter        : 

Registry setting for Powershell Remoting OK!
WinRM startup set to Automatic in GptTmpl.inf!
If you want you can manually check that the settings are applied by checking the content of the INF file:
And for the other settings, check you Group Policy Management GUI:
Don't forget to run this function on a Domain Controller or on any member server having RSAT.
On your clients you must wait for 90 minutes (with a random offset of 0 to 30 minutes) for the GPO to re-apply, or just logoff and logon again.
As a side information, this GPO worked well against my new beta Windows 10 clients!
If you have any question, feel free to ask and .... please share!

Thursday, November 6, 2014

How to use Sysprep to generalize Windows Server 2012 R2 Running in a VM

Yesterday I was asked by an IT guy if the use of newsid.exe against a cloned Windows 2012 R2 virtual machine would work to prevent SID conflict. Well, what he didn't know, is that newsid is old history now (even the download is no more available) and that a new wonderful switch in the Sysprep utility is born to make the job of generalizing your master VM become a breeze. Let me quickly explain that.

The new switch for Sysprep is /mode:VM

This switch is available only through the command-line and causes Sysprep to skip the physical device recognition phase. This way deploying a clone of any VM becomes very rapid, since detecting changes in the installed hardware was time-consuming.

Here's the complete Sysprep syntax to use. Open an elevated command prompt, move to c:\windows\system32\sysprep, then run:

sysprep.exe /oobe /generalize /shutdown /mode:vm

and you are done. The master virtual machine will shut down and you can keep it as base host from which to deploy any other VM. Each time you start a new clone you'll be asked about your regional settings and input language, and you'll need to enter your activation key again.

Hope this helps.

Sunday, November 2, 2014

Generating a runner pace-band with Powershell

Powershell is about added value. Powershell is built upon .NET. And Powershell is flexible. Three statements. Once you know those, you know that you can cross the borders of what Powershell was meant for, invent something new, and still get great results.

I am a runner. Some days ago I have been running an half-marathon, and, during preparation, I found myself in the need of a wrist pace-band showing split times for each km, in order to evite erratic running speeds.

Of course there are pace-band generators on the web, like the one at RunnersWorld, but I took the decision that I wanted to make one myself with my preferred tool: Powershell.

There are several obstacles to reaching the desired result. I'll walk trough the required steps and we will together build a Powershell function named Get-Pace.

The Powershell function need of course to take in some parameters. The first three parameters are just used for customising your pace-band, so they'll be of type System.String.
  • RunnerName
  • RaceName
  • RaceDate
Then there are the parameters used for actual split pace calculation:
  • Distance as System.Decimal
  • Time as System.TimeSpan
  • Unit as System.String
  • Open as System.Management.Automation.SwitchParameter
We will be casting the values for these variables into the required types by using some well known accelerators.

In Powershell, to cast a Decimal you can use:

[decimal]$Distance = 42.195

[decimal]$Distance = '42.195'
They both will convert the passed string to a decimal.

Now we want to tell Powershell that we want to set the expected duration of the race as a TimeSpan.
When casting a TimeSpan, Powershell expects you to pass the string in the one of the following formats:

[timespan]$Time = '1:3:4'
will set a TimeSpan of 1 hours, 3 minutes and 4 seconds;

[timespan]$Time = '1:3'
will set a TimeSpan of 1 minutes and 3 seconds;

[timespan]$Time = '3'
will set a TimeSpan of three days;

[timespan]$Time = '3:1:2:3'
will set a TimeSpan of 3 days, 1 hours, 2 minutes and 3 seconds;

When setting a TimeSpan, always remember to enclose the string into quotes, otherwise the interpreter won't be able to do the required conversion and throw the following error:

+ [timespan]$Time = 1:3:4
+                   ~~~~~
    + CategoryInfo          : ObjectNotFound: (1:3:4:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

The unit must accept only one of two possible value: miles or kilometers. To do so, best option is to use ValidateSet for parameter validation:

[ValidateSet('km', 'mile')][string]$Unit = 'km'
Powerful, isn't it? Parameter validation should be one of your closest allies when working on a Powershell function, remember.
Now there is an unfortunate problem when generating a pace-band. One of historical kind. Let me explain that. Most runners need a pace-band when running a marathon. A marathon has a weird distance. More than two milleniums ago, Pheidippides ran from the city of Marathon to Athens for roughly 40 km or 25 miles. When in 1908 the Olympics were held in London, the race was extended to 26.2 miles, or 42.195 km, so the runners could cross the finish line in front of the royal family's viewing box.
This brings us to the fact that while a race like a 10k has an exact number of split, races like marathon or half-marathon have a last split of shorter length. And we need to take this in consideration in our script.
To determine if a race has only full splits, first we need to round its distance to the closest smaller integer. Two correct ways to do that:

$FullSplits = [decimal]::Floor($Distance)

$FullSplits = [System.Math]::Floor($distance)
Then to get the last incomplete split length, just use:

$LastIncompleteSplit = $Distance - $FullSplits
After having determined the number of split, next step is to calculate the number of seconds per split: 
$TimeSpanPerSplit = New-TimeSpan -seconds ($Time.TotalSeconds/$Distance)
and convert it to speed using a simple formula you all know:

$Speed = 60/$TimespanPerSplit.TotalMinutes
Now, since Powershell is object-based, we proceed to the set-up of the array that will contain split information to print on the pace-band.

For each split we are going to show the three basic information each runner needs:
  1. Split number
  2. Target split time
  3. Cumulative time
Since we are printing the data on the wrist band as a string, we can take advange of the -f operator to format data in a readable way while running.

Let's see that:

# Creating an empty array for storing full splits
$Splits = @()
# Populating the array with split and cumulate time association
1..$FullSplits | % {
    $CumulateTime = $TimespanPerSplit.TotalSeconds * $_
    $CumulateTimeSpan = New-TimeSpan -Seconds $CumulateTime
    $Splits += "{0:D2} {1} {2:D2}h{3:D2}'{4:D2}''" -f $_,$Unit,$CumulateTimeSpan.Hours,$CumulateTimeSpan.Minutes,$CumulateTimeSpan.Seconds
Notice the use of :D2 to tell -f to pad hours, minutes and seconds with a zero in case they have one digit only, so that 0h25'7'' will appear as 00h25'07''. Useful.

In case there is an additional incomplete split, like in a half marathon where the last split is 97.5 meters long, we repeat the step once more, so to complete the array. Thanks to the if(){} condition, this step will be skipped for races like a 10k or a 20k, because $FullSpits and $Distance will have equal value.

if($Distance -gt $FullSplits)
    $CumulateTimeFinal = $TimespanPerSplit.TotalSeconds * $Distance
    $CumulateTimeFinalSpan = New-TimeSpan -Seconds $CumulateTimeFinal
    $Splits += "{0:N2} {1} {2:D2}h{3:D2}'{4:D2}''" -f $LastIncompleteSplit,$Unit,$CumulateTimeFinalSpan.Hours,$CumulateTimeFinalSpan.Minutes,$CumulateTimeFinalSpan.Seconds
In the next part of the Get-Pace function we are going to generate one text header showing the average target pace:

$Header1 = "{0}'{1}''/{2}" -f $TimespanPerSplit.Minutes,$TimespanPerSplit.Seconds,$Unit
... and one with the average speed: in kmh if we specified 'km' as measure unit, in mph if we chose 'miles'. Notice here the use of Switch(){} statement, which replaces the use of multiple sequential if(){} conditions.

Switch ($Unit)
    "km" {$suffix = 'kmh'}
    "mile" {$suffix = 'mph'}
$Header2 = "$([System.Math]::Round($Speed,2))$suffix"
We continue setting up a side text containing the name and the date of the race:
$SideText = $RaceName + ' ' + $RaceDate
Then we for sure need a footer to encourage the runner to stick to the target pace. Here it comes:
$Footer = "Go $RunnerName!!!"
That was the easy part of the function.

Today we want to push our Powershell a little farther and see if we are capable of outputting the $Array object as well as $Header1, $Header2, $SideText and $Footer to a printable image.

First step is to load the assembly:
Add-Type -AssemblyName System.Drawing
The second step is to set the path and name of the image file:
$FileName = "$home\pace-band.Png"
The third step is to set the image height and width, trying to get something printable. This is very empiric, since the the resulting image size in centimeters will depend upon your DPI.... but this is another story and out of scope here, so take these values as granted but modify them if they don't work for you:
$PngHeight = 20 * 38
$PngWidth = 5 * 38
$Png = New-Object System.Drawing.Bitmap $PngWidth,$PngHeight
Now let's choose the font type and size for our wrist-band:
$FontSmall = New-Object System.Drawing.Font Consolas,8
$FontBig = New-Object System.Drawing.Font Consolas,14
$FontSideText = New-Object System.Drawing.Font Consolas,14
Next is to choose both foreground and background colors:
$BrushBgColor = [System.Drawing.Brushes]::LightYellow
$BrushFgColor = [System.Drawing.Brushes]::DarkBlue
Time to setup the background image:
# Creating a graphic from the image and putting some colors
$Graphics = [System.Drawing.Graphics]::FromImage($Png)
Once you are here, you have to print on the pace-band each split as a single line and move down on the paper. I did the math, so they could be wrong, but normally this worked well on my screen:
# Adding one line to the pace-band for each split
0..$FullSplits | % {
    $splitlength = $Graphics.MeasureString($Splits[$_],$FontSmall)
    $Graphics.DrawString($Splits[$_],$FontSmall,$BrushFgColor,($PngWidth-$splitlength.Width)/2,$PngHeight / ($Distance) + ($splitlength.Height*$_) + 75) 
Let's now add the two headers as well as the footer text. Notice the use of MeasureString to measure string length, in order to center text on the pace-band with the help of the following formula: (width of the image - string length) / 2. Notice also how I assigned the DirectionVertical flag to my side text, so that it appears vertically centered. Resulting code is here:
# Measuring headers, sidetext and footer in order to center them on pace-band
$Header1Length = $Graphics.MeasureString($Header1,$FontBig)
$Header2Length = $Graphics.MeasureString($Header2,$FontBig)
$FooterLength = $Graphics.MeasureString($Footer.ToUpper(),$FontBig)
$SideTextLength = $Graphics.MeasureString($SideText,$FontSideText)

# Writing headers, footer and sidetext
$Graphics.DrawString($Footer.ToUpper(),$FontBig,$BrushFgColor,($PngWidth-$FooterLength.Width)/2,($SplitLength.Height * $Distance) + 125)
$DrawFormat = New-Object System.Drawing.StringFormat("DirectionVertical");
Last two steps, dispose the object that consume memory, such as fonts and graphics (this is robust programming, guys!) and save:
You could now call the function with the following parameters:
Get-Pace -RunnerName Carlo -RaceName "Marathon" -RaceDate "01/XI/2014" -Distance "42.195" -Time "2:59:0" -Unit "km" -Open
and get a printable image like this:

Print it, cut it, cover both front and back with sticky tape, place the band on your wrist and then tape the top edge over the bottom tab.

Of course you could improve this function by adding parameters like the destination file, or the colors to use for your pace-band. I leave that to you on purpose, so you can increase your Powershell skills, but feel free to ask questions as soon as they arise. Enjoy and share!

Wednesday, October 22, 2014

Reading large text files with Powershell

Any sysadmin out there knows that log files are an invaluable asset for troubleshooting issues on their servers. Sometimes though these kind of files can be very large and become difficult to handle, as I had the occasion to notice in a Powershell forum discussion one week ago.

The thing to understand is that the right tool to use in Powershell is determined from the kind of file you are dealing with.

Here's the bunch of options you have for reading text files:

  • Get-Content
  • Get-Content -Raw
  • Get-Content -ReadCount
  • Switch -File
  • [System.IO.File]::ReadAllText()
  • [System.IO.File]::ReadAllLines()
  • [System.IO.File]::ReadLines()
  • [System.IO.File]::OpenText().readtoend()
Get-Content is your out-of-the-box option. It works well with small log files and its use is pretty straightforward, but on large log files it can be very slow.

Get-Content .\small.csv

Let's suppose we have four logfiles of different size and number of lines:

- logfile0.log, a small logfile: 4 lines and 1KB size
- logfile1.log, a medium size logfile: 700K lines and 160MB size
- logfile2.log, a large size logfile: 300K lines and 1.2GB size
- logfile3.log, a logfile containing more than a million of lines: 1.4M lines and 650MB size

Let's see what Get-Content can do against these files:

(Measure-Command {Get-Content .\logfile0.log}).TotalSeconds

(Measure-Command {Get-Content .\logfile1.log}).TotalSeconds

(Measure-Command {Get-Content .\logfile2.log}).TotalSeconds

(Measure-Command {Get-Content .\logfile3.log}).TotalSeconds

Pretty swift on the smaller ones, but way too slow on that last large file, which, with its over one million lines becomes difficult to deal with.

Let's see than how the other methods I listed above perform against it. Doing some performance testing is not only fun, but it helps better understanding how your systems work.

As we just saw, with its basic syntax Get-Content performs poorly:

(Measure-Command {Get-Content .\logfile3.log}).TotalSeconds

Since Powershell v3, Get-Content has a -Raw parameter, which makes it to read text files in a text stream, keeping newline character intact. As you can see below, it is way faster than Get-Content:

(Measure-Command {Get-Content .\logfile3.log -Raw}).TotalSeconds

Nonetheless it a huge impact on your memory use:

Happily enough we can dispose data in memory by forcing the Garbage Collector to do its job before continuing our tests:


Let's move on to the third option: the -ReadCount parameter. As it was showed in the mentioned forum discussion, this switch can speed Get-Content beyond what one could expect, given that you find the right value for it:

(Measure-Command {Get-Content .\logfile3.log -ReadCount 10}).TotalSeconds

(Measure-Command {Get-Content .\logfile3.log -ReadCount 100}).TotalSeconds

(Measure-Command {Get-Content .\logfile3.log -ReadCount 1000}).TotalSeconds

(Measure-Command {Get-Content .\logfile3.log -ReadCount 10000}).TotalSeconds

This -ReadCount switch helps speeding up the process by controlling the number of records Get-Content writes into the pipeline at a time. From my experience, reading the file by chunks of 1000 or 10000 lines usually gives the best results and has low to no impact on CPU and memory even for such a large logfile.

The problem with -ReadCount is that it takes a bit of guessing to find out the right value for a specific file, since performance varies not only with the size of the file but also with the size of each record.

An interesting alternative to Get-Content, which you should know, is the use of the Switch keyword:

(Measure-Command {Switch -File .\logfile3.log {default {$PSItem}}}).TotalSeconds

Not so bad, isn't it? Let's remark that the impact on CPU usage of Switch is slightly higher than Get-Content -ReadCount:

Let's see now if we can have better results by crossing the unwelcoming bridge to .NET.

First option is the ReadAllText() method:

(Measure-Command {[System.IO.File]::ReadAllText('C:\temp\get-content\logfile3.log')}).TotalSeconds

ReadAllText() is very fast but has a temporary impact on system memory.

Second option is ReadAllLines():

(Measure-Command {[System.IO.File]::ReadAllLines('C:\temp\get-content\logfile3.log')}).TotalSeconds

Not as good as ReadAllText(), but it has a slightly less aggressive impact on memory allocation:

Third option is ReadLine(), which differs from ReadAllLines() in the fact that it allows you to start enumerating the collection of strings before the whole collection is returned:

(Measure-Command {[System.IO.File]::ReadLines('C:\temp\get-content\logfile3.log')}).TotalSeconds

Impressive result here.

Let's move to the fourth and last option which is OpenText().readtoend():

(Measure-Command {[System.IO.File]::OpenText('C:\temp\get-content\logfile3.log').readtoend()}).TotalSeconds

Great performance, but, same as for the two other .NET methods, a lot of RAM has to be allocated since they read the entire file into memory and your code could in some cases throw an exception of type System.OutOfMemoryException. Also, you have noticed it, when using .NET, the file path must be fully specified or Powershell won't be able to find it.

In conclusion, seeking through the file with Get-Content -ReadCount seems to be the seconds fastest way to go through a large text file. First option with large files is [System.IO.File]::ReadLines():

  1. [System.IO.File]::ReadLines(): 6 seconds
  2. Get-Content -ReadCount 1000: 7 seconds
  3. Switch -File: 10 seconds
  4. [System.IO.File]::OpenText().readtoend(): 10 seconds
  5. [System.IO.File]::ReadAllText(): 13 seconds
  6. [System.IO.File]::ReadAllLines(): 16 seconds
  7. Get-Content -Raw: 46 seconds
  8. Get-Content: 229 seconds
If you liked this post, please share, or leave a comment if your tests gave you different results.

Wednesday, October 1, 2014

Powershell MVP, once again!

I am so honored to say that I have received the Microsoft MVP award for the second year following my contributions to the Powershell Community and on this very same blog.
It's definitively been a long year, full of intense moments, like in February when I spoke on Powershell in front of several hundreds of people during the Techdays in Paris with my friend Fabien Dibot and for the first time I met the MVP project coordinators Martine and Marjorie.
Or like when I took part in coaching the competing teams during the 2014 Powershell Winter Scripting Games and came to work with brilliant people like Mike F Robbins, Boe Prox, Emin Atac, Jeff Wouters and Jan Egil Ring, just to mention a few.
It's always nice to see that the effort I make to publish good content have once again been noticed and that the Community appreciate them. I will do my best to improve content quality up to the highest standards, so stay tuned for more on Powershell and System Administration!

Friday, August 22, 2014

How to disable IPv6 with Powershell

The weekend is approaching fast. I have decided to end the week with a post on how to disable IPv6 on your Windows computer with a simple Powershell oneliner, since I had to do it in a specif infrastructure design this week and wanted to share the how-to. Not that disable IPv6 is something that I would recommend, since, as Microsoft reminds us, "IPv6 is a mandatory part of the Windows operating system" and "Microsoft does not perform any testing to determine the effects of disabling IPv6. If IPv6 is disabled on Windows 7, Windows Vista, Windows Server 2008 R2, or Windows Server 2008, or later versions, some components will not function."

Despite that, there are exceptions, and it could be useful to know how to do it, especially if you are running Windows in Server Core mode. Here's the quick oneliner in charge of it:
New-ItemProperty 'HKLM:\SYSTEM\CurrentControlSet\Services\Tcpip6\Parameters\' `
     -Name  'DisabledComponents' -Value '0xffffffff' -PropertyType 'DWord'
As you can see this oneliner modifies the DisabledComponents registry value, which disables all IPv6 components and affects all network interfaces on your host except the IPv6 loopback interface.

Note that you must restart your computer for these changes to take effect:
As a side note, let me tell you that you can force pings to use IPv4 or IPv6 just by appending a -4 or -6 switch to your PING command:
ping localhost -4

Pinging [] with 32 bytes of data:
Reply from bytes=32 time=1ms TTL=128
Reply from bytes=32 time=1ms TTL=128
Reply from bytes=32 time=1ms TTL=128
Reply from bytes=32 time=1ms TTL=128

Ping statistics for
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 1ms, Average = 0ms

ping localhost -6

Pinging [::1] with 32 bytes of data:
Reply from ::1: time=1ms
Reply from ::1: time=1ms
Reply from ::1: time=1ms
Reply from ::1: time=1ms

Ping statistics for ::1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms
I hope you have learned something! Stay tuned for more Powershell tips!
Related Posts Plugin for WordPress, Blogger...