Live PowerShell With Karl Prosser

ST POP Connector for Exchange server is designed as a replacement of the POP3 Connector in Small Business Server,
but can be used with any Exchange or SMTP server.  ST POP Connector provides the following benefits over the default POP3 Connector in SBS

• Download email as often as once per minute
• Supports leaving emails on the POP server for a specified number of days or indefinitely
• Ability to only download emails from the POP server from a particular point in time, preventing emails that have already been downloaded from from being duplicated (This is very helpful if the POP state file is lost and you are setting up the mail accounts for a second time or if you are transitioning from using Outlook to download the POP email and keeping a copy on the server)
• Ability to archive messages
• Supports all versions of Exchange or any SMTP server
• Supports downloading from GMail, Hotmail, Yahoo and other online accounts with POP access
• Supports SSL connections
• Detailed logging and debugging viewable in a log file and Event Viewer
• Large attachments

There are also very interesting possibilities and uses used in conjunctions with a product like Gmail for your domain where you can have Google for your domain being the main place email is landing, giving you a free “online backup” as well as not requiring an enterprise grade always on internet connection for your SMB exchange server, and also not having to waste incoming bandwidth of the 90% of mail traffic which is spam. Using this you’d also get free spam protection from Google mail. However you still will be using exchange and outlook thus getting the best of both worlds, having that enterprise quality exchange/outlook business experience.

We are providing ST POP Connector for FREE for up to 3 mailboxes and
the full version is free for Microsoft MVP’s.  Just email us with a link to your MVP profile to info at shelltools dot net.

To read more or to Download it go to http://www.popconnector.com

I was having an odd behavior with Portable PowerShell where if i started it from CMD it would take turns between giving a CMD.exe prompt and a PowerShell Prompt and take turns running them.

So in this test app, you can see we are first in CMD, then we run this application then after than its alternating between CMD and PowerShell Prompts and you see the expression 1 worked in PowerShell while having a problem in CMD.EXE

So i hear you say what’s the source code for this little demo? – I’m glad you asked

class Program
    {
        static void Main(string[] args)
        {
            var p = new System.Diagnostics.ProcessStartInfo("powershell");
            p.UseShellExecute = false;
            System.Diagnostics.Process.Start(p);
        }
    }

So why is this happening. Well normally when a console application calls another and they run using the same console window the first one is effectively paused while the 2nd one is running, then when the 2nd one exits it resumes.

so the flow here if it were working would be

CMD –> TESTAPP –> POWERSHELL

CMD naturally is waiting for testapp to exit, then in the sample code TESTAPP starts powershell, but i forgot to waitfor powershell to exit before proceding, so TestApp and PowerShell are sharing the same console for a split second then TestAPP Exits, and when TestApp exits CMD regains control of the console.. Now you have too totally separate programs CMD and POWERSHELL competing and using the same physical console window.

The fix is really simple.

.

class Program
    {
        static void Main(string[] args)
        {
            var p = new System.Diagnostics.ProcessStartInfo("powershell");
            p.UseShellExecute = false;
            var proc = System.Diagnostics.Process.Start(p);
            proc.WaitForExit();
        }
    }

Not really a biggie, but i thought it would be an interesting thing to share, and probably the cause of a bit of hair pulling  by many.

I’ve been looking through a bunch of my old scripts (pre 2007) and found an interesting new-pscustomobject. I wouldn’t recommend you use this since now there are better ways in V2, and if you wanted to do anything more its better to use or build a more elegant custom object DSL/DSV that takes into account many aspects. This script was originally a one liner but i split it up to make it more pretty for the blog.

function new-pscustomobject ([string[]] $names , [object[]] $values )
{$obj = 1| select-object $names;
0..($names.length-1) | % { $obj.$($names[$_]) = $values[$_] };$obj
}

and you could use it like

new-pscustomobject -names name, age, height -values "karl", 29, 195

Whats interesting here are a few things.. one using select-object as a quick and dirty way of creating a stub object with properties without values.. the 1 | pipeing is just to get it to run once and create one object… it really could be anything. Another thing is using the 0..(whatever) to create a collection, and then use foreach to iterate over that. Its very “PowerShelleze” but a fair bit slower than a foreach ($x in $y) or a for loop. I think the real “dynamic language” feature this shows up, is address properties of an object dynamically with $obj.$(expression) where expression calculates the name of the property. That feature of powershell comes in handy so often for me, and cuts down many a 200 line C# solution into a one liner.

There is alot of confusion about which PowerShell V2 to download. The problem is if you search for Download PowerShell V2, the most prominent hits are PowerShell V2 CTP – basically an Old Beta. Microsoft has to keep that up because there are production things that actually use that (like outlook@edu) . The best URL path to follow is to go to Microsofts official PowerShell Page at http://www.microsoft.com/powershell and follow the download page links from there.
if you want to go directly to the V2 Download page now though here is a direct link http://support.microsoft.com/kb/968929

One of the reasons why it doesn’t show up is the install isn’t just PowerShell but packaged with WinRM (and optionally Bits 4.0) as something called “Windows Management Framework” . This is good to know and remember since that is what will show up in your add/remove programs if you ever need to remove it.

Have you ever just wanted to run a line or two of C# from within PowerShell and consume the resulting objects from PowerShell? Before V2 you had to do some codedom yourself Plus write a full dotnet class yourself. With V2 you can just do add-type but still you have to write a full class just to run an expression.

Anyway how about a year ago after getting fed up with doing that, and also after seeing Mono’s C# commandline interface i thought lets do something quick and dirty that could do this..

with the script that will follow you can do stuff like (c being an alias for the function that compiles and runs a C# expression

(c DateTime.Now).adddays(5)
(c "new{a=1,b=2,c=3}").b
c 'from x in Directory.GetFiles(@"f:\downloads") where x.Contains("win") select x'

an interesting thing i found out, was that with the C# compiler in memory i can’t create more than one anonymous type (2nd example above) with the same signature (i.e int,int,int ) so the compiler will try to create the exact same type again.

So it may seem pointless to call DateTime.Now when you can just do [DateTime]::Now but thats not the point, the point was it was running C#, returning an object that you could immediately use in powershell. I threw a linq example in there just to show you a glimpse of the potential.

So how do you do this. You could even do this in V1 with codedom generation but here i’m using PowerShell V2

function run-csharpexpression([string] $expression )
{
$global:ccounter = [int]$ccounter + 1
$local:name  =  [system.guid]::NewGuid().tostring().replace('-','_').insert(0,"csharpexpr")
$local:ns = "ShellTools.DynamicCSharpExpression.N${ccounter}"

$local:template = @"
using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;

namespace $ns
{
public static class Runner
{
public static object RunExpression()
{
return [[EXPRESSION]];
}
}
}
"@

$local:source = $local:template.replace("[[EXPRESSION]]",$expression)

add-Type $local:source -Language CsharpVersion3 | out-Null
invoke-Expression ('[' + $local:ns + '.Runner]::RunExpression()')
}

Thanks to Oising http://www.nivot.org for playing around with this when i was building it.

So if i wanted to really turn this into something industrial quality what would i do?

• try to work out a way that i could create the same anonymous type twice..
• deal with exceptions well
• make a version that you can pass in objects from powershell to the expression, and that they could even be put in as fields on the runner class so you could reference them by name from C#
• Make a version that is designed to be run as part of the pipeline, where the function wrapper would manage the PROCESS block and call the csharp expression once every PROCESS block invocation.
• I would possibly make a special version of Generics since generics interop is often one of the harder things in Dotnet
• I’d have it keep a track of all the classes it has made, and have functions to look at them and interact with them and maybe reuse them. Particularly if they are parameterized.

Of course if you were so pathologically inclined you could very easily adapt this to make a VB.NET version too.

Enjoy. And if anybody feels inspired to make a better more industrial quality version i’d love to have a look at it.

PowerShell has some really good build in language features for dealing with regular expressions -Replace , -Match . Not to mention regex filtering is available to many cmdlets etc, and if thats not enough you can just invoke the powershell Regex classes.

However when dealing with more than the first match.. or where the replace was not just a simple string replace.There are many cases where a simple string substitution was not enough and i really needed to apply some logic at each match to work out what it was to replace it with.

The problem was it took me about 5 or 6 lines of code each time, and broke my “keep everything in the pipeline” goal as well as my workflow. So here is a function to do this. In reality it could be a lot more robust, and could be part of a whole family of such functions carefully sculpted for your regex needs.

More importantly than regex however, this is a good example of refactoring logic in powershell into a one liner, and how using using scriptblock in a “closure” manner where the end user passes in a scriptblock they wrote and gets some automatic variables from the scope of your function. Lets look at the function and an example.

function replace-regexgroup ([regex]$regex, [string]$text ,[scriptblock] $replaceexpression)
{
$regex.Replace($text,{
$thematch = $args[0]
$groupnames = $regex.GetGroupNames()
for ($count = 0; $count -lt ( $thematch.groups.count) ; $count++)
{
set-variable -name $($groupnames[$count]) -visibility private -value $($thematch.groups[$count] )
}
if ($replaceexpression -ne $Null) { &$replaceexpression}
} )
}

So the user passes in the regular expression, the text that they want to do the replace on and a scriptblock which is the expression that will be run for each match, and the result of that expression will be what the each match is replaced with. You can see how i use set-variable to set a variable in the of my function based on the name of each regex match (i really should also do a $1 , $2 , $3 as many people are used to in jquery and such. ) Then i call the scriptblock with &$replaceexpression . I haven’t decided in scenarios like this whether to invoke such scriptblocks with & which will create a child scope for them to run in or rather execute then in my scope with the dotsource operator “.”.

So how do we use this. Well in the first scenario when i made this, i was exporting wiki contents from a desktop app that used HTML stored in a database. I wanted a backup of this content in plain html and it had links such as shown below

$example = @"
<P><a href="wiki://284_636">links to test page 2</a></P>
<P><a href="wiki://109_49">
"@

and i needed to find all those Wiki links and needed to know and transform it based on the two numbers on either side on the _ (in the first case 284 and 636) which i did with the following line of code.

replace-regexgroup 'wiki://(?<wholething>(?<folder>\d+)_(?<page>\d+))' $example { "$folder/$page/index.html" }

You can see that the regex named different groups and that my scriptblock is simply one string with variable expandsion inside using the automagically created variables $folder and $page based on the regex groups.

Here is a link to the script on PoshCode .

I hope that this is more than just a good starting point for a regular expression function but something that helps teach the value of scriptblocks and how they can be used to make reusable code that simplifies your workflow.

-Karl

UPDATE FEB 2010 We don’t need to rely on things like thinapp or app-v anymore. At ShellTools we have build our own Portable PowerShell framework. Its currently in Beta – see http://www.portablepowershell.com for more details.

What is it?

Portable PowerShell is software that allows you to run PowerShell on machines that don’t have PowerShell installed that you can run from a Machine that doesn’t have PowerShell on it, from a USB stick, on a machine that has a different version of PowerShell, a preinstall environment like BartPE, or WinPE or when booted to a windows 7 recovery DVD.

Why would I want it?

Administrators often cannot get their key tools, such as PowerShell installed in the environments they are working. This may be because a server installation has not yet been tested or approved, or the IT pro is working on a client’s environment, or a desktop support engineer is working on a machine that will likely not ever have PowerShell installed or enabled. Additionally you may want to run a different version of PowerShell than that installed, being able to do side by side work or comparisons when developing scripts. Having a thumb drive with portable PowerShell and all your own custom scripts with you at all times would be a dream come true to many administrators and IT pros.

How does it work?

You simple take our “PortablePowerShell” zip file and unzip it on a computer that already has PowerShell and Run “MakePortable.exe” and that will produce a folder containing everything you need to run PowerShell portable. From there you can just run it, or then copy it onto other computers, or onto a USB stick.

Does it use App-V or Thinapp?

Though we experimented with ThinApp and App-V for this very purpose a few years ago, we do not use either of those technologies as the past on licensing costs would be prohibitive for the majority of our customers. Instead we have crafted our own “Application Virtualization” particularly targeting the needs of the PowerShell engine.

What about PowerShell V2

In the future after the full release of PowerShell V2 we will release a version that will be able to produce a Portable PowerShell V2, however we made a Portable WinRM so V2 remoting will be limited or non-existent, however we are investigating what is needed to ensure that PowerShell V2 remoting will work.

What about Third Party Snapins?

Snap-in DLLs can be distributed with Portable PowerShell, and we have a configuration tool, and an XML configuration file that ensures that snapins get registered with the Portable PowerShell. However if the third party snapin’s installer puts other files in different places and maintains its own product registry , those snapins may not work. We do however provide a way that you can create and maintain virtual registry settings. We will set up a forum where instructions of including different snapins can be shared with the community. We are confident the majority of Third Party snapins will be able to be run, and we are willing to update the product to accommodate important but “difficult” snapins.

Other Features include ability to produce “custom portable PowerShell’s with special logging, specific scripts build in, scheduling etc.

So when can I get this and how much will it Cost?

ShellTools debuted with bring PowerShell Analyzer then PowerShellPlus, which is now produced and sold by Idera, successfully to market. We have a commitment to our customers, and we want to ensure that what we make lasts, and is feasible to maintain and support. We also learnt that bringing a product to market requires a lot more effort that just excellent software engineering. So basically we are now starting the process of working out whether ShellTools Portable PowerShell is commercially feasible and that we should bring it to market, or whether we just keep it as our own internal tool.

To help us decide could you please fill out the following survey? Additionally if you leave your email address with us in the survey you will likely be invited into the private Beta.

Link to Survey

Embedded Survey

Thanks,
The ShellTools Team.

A picture says a thousand words

Search results from the PowerShell Code Repository right there on your desktop in windows 7 thanks to federated search and Jaykul adding OpenSearch support to PoshCode. YAY

So how do you get this. Simply download this link  and save it as fileextention .OSDX to your windows 7 machine. Then Run it. 

That is a simple OSDX file that is just an XML file describing the OpenSearch interface. the full text is here. and it comes from PoshCode itself.

<?xml version="1.0" encoding="UTF-8" ?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/" xmlns:moz="http://www.mozilla.org/2006/browser/search/">
  <ShortName>PoshCode</ShortName>
  <Description>PowerShell script repository search interface</Description>
  <Tags>PowerShell Scripts Scripting Code Modules</Tags>
  <Contact>feedback@poshcode.org</Contact>
  <Url type="text/html" template="http://poshcode.org/search/{searchTerms}" />
  <Url type="application/rss+xml" template="http://poshcode.org/api/{searchTerms}" />
  <Image height="16" width="16" type="image/vnd.microsoft.icon">http://poshcode.org/favicon.ico</Image>
<!–   <Url type="application/x-suggestions+json" template="http://localhost:7402/content/notes.opensearchsuggestions.json?qt={searchTerms}"/>

  –>
  </OpenSearchDescription>

How many times have you had something like:

get-thousandsofitems | % { do-something $_ } | whatever

Only to have some exception happen in do-something and it throws the whole thing. When really you don’t care, you’d rather just be able to know what failed and have it move on. This happens a fair bit with Get-WMIobject etc, because one of the machines may be off, firewall blocked, some sort of permissions issues – things that bubble up even when you have the preference set to SilentyContinue.

So what do we normally do in this scenario. We either pipeline it and make it look like a VBscript, or ruin our beautiful single liner by making that script complicated with V1 traps and the likes, and we usually get the error handling wrong wrong a number of times along the way.

Well with PowerShell you can always adjust and expand the language. There is nothing particular special about the foreach-object Cmdlet. In its common use you can replicate it with a function as simple as the one below.

function myforeach-object([scriptblock]$process) { process { &$process } }

So how about making a looping function that deals with the exceptions for you, you say? Great Idea!, Now you are cooking. The following function is pretty much a simple V1 function, though it should really be completed and production readied into a V2 Advanced function with all the modern trimmings and trappings, however I DO use V2 TRY/CATCH syntax instead of V1 Trapping, because its more sane to us who don’t have a secret VBscript fetish.

function foreach-withexception ([scriptblock]$process,$outputexception)
{
  begin { $global:foreachex = @() }
  process {
            try
            {
            $local:inputitem = $_
            &$process
            }
            catch
            {
            $local:exceptionitem = 1 | select-object object,exception
            $local:exceptionitem.object = $local:inputitem
            $local:exceptionitem.exception = $_
            $global:foreachex += $local:exceptionitem
            }
          }
  end {}
}
set-alias %ex foreach-withexception

So i love my aliases , %ex for foreach with exceptions. Really I wish i could come up with a better name for the long function. I also keep a %i for my foreach-index function as well. Basically you use this just like foreach-object (though i haven’t implemented a begin and end area in this), and exceptions get eaten, but after the fact you can access the object that caused the error, and the error in the global $foreachex variable. At a later date i’m going to add an option paramter variable that if can put the exception in instead of a naughty global variable. But as Jeffrey Snover says “To ship is to choose” though for me it really means “this function does what i need it to now, i need to go back and solve the Real problem i’m solving that this function really helps with”

here is an example

100..-5 | %ex {  "yo $_" ;  1 / $_ }

$foreachex

normally 0/0 would throw a fuss , and kill the whole thing, but it just works on , and you can check to see if there are any errors in $foreachex Normally the things that i need this for though are “loose web services”, WMI, or networking sort of things. but really it simplifies my one liners, and gets things done quicker.

@poshcode | Download

I’ve always liked the fact that I can extend PowerShell, whether its just a little bit of syntax here or there, or a full blown DSL, and end up with something that is still naturally powershelly. Jaykul (Joel Bennet) has really used PowerShell in such a way. really making something that is pithy, follows a DRY philosophy, and is METAPROGRAMMING on two levels (the auto generation of the PowerShell functions and parameters, and of course the XAML). Its not only a beautiful example of PowerShell as a DSL, but also ultimately useful. I really think it has the potential to become the tcl/tk of PowerShell and the primary admin GUI scripting framework. I’m so impressed with it, we at ShellTools are definitely going to adjust our PowerShell WPF designer to support Boots. Boots is also now a codeplex project

Below is an example of the script, and a few screenshots and videos from Jaykul’s blog.

Boots {
   StackPanel -Margin 10 {
      TextBlock "The Question" -FontSize 42 -FontWeight Bold -Foreground "#FF0088"
      TextBlock -FontSize 24 {
         Hyperlink {
            Bold "Q. "
            "Can PowerBoots do async threads?"
         } -NavigateUri " " -On_RequestNavigate { $global:Answer[0].Visibility = "Visible" }
      }
      TextBlock -FontSize 16 {
         Span "A. " -FontSize 24 -FontWeight Bold
         "Oh yes we can!"
      } -OV global:Answer -Visibility Collapsed
   }
} 

.

.