Test-Infecting a Powershell Script

Writing Powershell scripts is generally a lot of fun, but once in a while you might come across a need to become test-infected, that is, become a little obsessive about how well your script is tested before you distribute it.  There might be several reasons for getting infected: you’ve been burned, you have to distribute your scripts to people that don’t have quick and easy access to you, or the intent of the script has rather serious overtones and/or consequences should something go wrong.

Not to say there is anything wrong with _always_ being test-infected.  But sometimes you just want to have fun with the script and jam it out there for other Powershell hacks like me to have a look at.  There is real value in that.

Caveats aside, let’s look at test-infecting an existing script.  Steve Murawski gleefully (via twitter) offered his script Show-ADObject.ps1 as lab rat for my intended infection.  Here’s what happened.

My stated starting point was the existing Show-ADObject script.  To infect the script, the first step I undertook was to create a test script that describes the intended behaviour.  The second step was to refactor the target script until such a time that a) all test cases are automated and b) all test cases pass.

After running Steve’s script, I came up with seven behaviour-defining test cases.  I chose to describe each one of those test cases using the given-when-then phrasing from behaviour-driven development (BDD) since the appropriate aliases are built into the PSExpect testing framework.

Steve’s code had some dependencies that I have inoculated against infection through isolation.  In other words, I made them out of scope of my tests.  In particular, the graphing routine and the graphing class library that was used.  Consequently none of the following defining behaviours refer to the actual visual graph itself – only the items that are being fed into that graphing routine.

Written this way, the tests should read as naturally as plain text.  When you say each test, don’t skip to the text in quotes – read them with the words given-when-then included.

    # first defining test case - collect the nodes for a valid AD class name
    #
    #
    given "an empty list of objects to map and a valid class name 
that I know has entries"
when "I request to view the node map for that class" then "there should be more than zero nodes to view" # second defining test case - collect the nodes for a valid set of AD class names # # given "an empty list of objects to map and a list of valid class names
that I know each have entries"
when "I request to view the node map for those classes" then "there should be more nodes than were discovered with
just a single class name"
# third defining test case - display a list of AD class names # # given "an empty list of AD class names" when "I request the list of valid AD class names" then "there should be more than zero valid AD class names returned" then "known class names like group, organizationalunit should be in the list" # fourth defining test case - adding color to the nodes # # given "an empty list of objects to map and a list of valid class names
that I know each have entries"
when "I request to view a colorized node map for those classes" then "there should be more than zero nodes to view and filtering only nodes
without color should not yield any objects"
# fifth defining test case - displaying help and usage examples # # when "I request to view the help and usage examples" then "the answer should be text and contain key help contents" # sixth defining test case - performing the check for required files # when all files are in place # given "a list of files that must be available for the script to perform
as expected and that all the files are in place"
when "I request to check to see if those files exist" then "the answer from the check should be true and the missing files report
should be empty"
# seventh defining test case - performing the check for required files # when all files are _NOT_ in place # given "a list of files that must be available for the script to perform
as expected and that _not_ all the files are in place"
when "I request to check to see if those files exist" then "the answer from the check should be false and the missing files report
should contain the number of missing files"

These seven defined behaviours form the backlog of work that I have to do in order to make the infection complete.  The next step is to provide the script blocks for each of the given-when-then phrases, in effect, automating the test case.  It’s best to implement one test case at a time so that you can get to the first passing test quicker – you’ll enjoy the positive feedback you get when you work this way.

    given "an empty list of objects to map and a valid class name `
        that I know has entries"
{
        $script:ObjectsToMap = @()
        $script:TargetADClass = "group"
    }
    when "I request to view the node map for that class" {
        $script:ObjectsToMap = Get-ADObjectsToMap $TargetADClass 
    }
    then "there should be more than zero nodes to view" {
        $script:NumberOfNodesFromOneClass = `
            $script:ObjectsToMap.GetUpperBound(0) + 1
        Assert ($script:NumberOfNodesFromOneClass -gt 0) `
            -Label ("ObjectsToMap.Single.Count:" + `
            $script:NumberOfNodesFromOneClass)
    }

Writing these code blocks is a translation exercise – you translate the natural language in text into Powershell script, calling the target script in order to exercise the true target of your test.  In this case, the target of the test is the function Get-ADObjectsToMap.  I’ve refactored its signature based on what the test needed.  Sure, there is a design element to the translation and you might come up with a different signature, but the point is – the function signature needs to support the test.  So the function Get-ADObjectsToMap isn’t allowed to perform any input or output – it must be written to accept input from the test script and then it must return something that the test script can inspect.  This enables the test oracle – the indicator of the pass/fail of the test – to be automated.

The ‘then’ clause has one test condition in it, confirming that at least something came back from the dip into Active Directory. The test condition is the line that starts with ‘Assert’, that is, a call to one of the functions in PSExpect.  I thought about adding another test condition to confirm that a specific item was being returned, but rejected the idea since I thought that was more a test of the Get-QADObject cmdlet than it was of my script.  Borderline.  At least this way, I’ve not injected any test condition that is specific to my Active Directory.

Running the script the first time failed miserably since I hadn’t created the target function with the right signature yet.  So parsing errors everywhere.  Next step: fix the parsing errors so that the script actually runs cleanly, albeit still fails the test.  This is an important step in test-driven development – getting to the first clean fail.  Next step after that, perhaps obviously, is to write the script so that the test passes.

Querying for group
03/01/2009 3:20:29 PM,SHOULDPASS,PASSED,PSpec,an empty list of objects to map and a valid class name that I know has entries
03/01/2009 3:20:29 PM,SHOULDPASS,PASSED,PSpec,I request to view the node map for that class
03/01/2009 3:20:42 PM,SHOULDPASS,PASSED,PSpec,there should be more than zero nodes to view
03/01/2009 3:20:42 PM,SHOULDPASS,PASSED,ObjectsToMap.Single.Count:237

Now you can choose to move on to the next test case, or you can choose to refactor the code written so far in order to improve its quality.  I recommend you refactor immediately so that it doesn’t feel like work later on.  Keep your focus tight.  Refactor to eliminate side effects, improve modularity, improve readability, etc.  Do this as you go instead of waiting until the end.

The process really doesn’t change for the remaining test cases with the exception that, every once in a while, you may have to go back to a previously-passing test and revise it based on what you have learned from getting other test cases to pass.  This was certainly required in this case once it got time to handle the coloring of the nodes that were intended to be graphed.

Download both the test script and the (refactored) target script here:

http://www3.telus.net/~amgeras/samples/Test-ShowADObjectR.ps1

http://www3.telus.net/~amgeras/samples/Show-ADObjectR.ps1

Summary

Steps for test-infecting a script:

  1. Write a test script that describes the intended behaviour.  The script should not run since you haven’t written the target script yet.
  2. Get the test script running by filling in _just enough_ of the target script so that there are no syntax or parsing errors – the test script should run smoothly, but still fail the test cases.  The PSExpect testing framework was designed to be able to highlight test case failures yet still have the test script run smoothly.
  3. Choose a single defining test case – ideally the most significant or intended-behaviour-defining one – and write the target script so that the test case passes.  Avoid over-engineering the target script to handle the other test cases – instead, focus on the one test case that you’ve chosen and get it passing, quickest way possible.
  4. Refactor your target script to meet your personal or enterprise standards.  If in Step 3 you’ve got the test case passing in the simplest possible way, Step 4 is to keep the test passing but to improve its quality.
  5. Choose the next-best defining test case and repeat Steps 3 and 4.  This time, you will need to ensure that the first test case stays passing.  This might mean refactoring your target script some more in order to handle the new test case.  It may also mean refactoring your first test case.  By the end of this step, though, all the tests you’ve tried to get passing, should be.
  6. Rinse. Lather. Repeat.  That is, repeat Step 5 until you’ve run out of behaviour-defining test cases.
  7. Refactor.  This time, add some test cases that are only there to confirm the stated quality.  These are tests like how to deal with null inputs, how to deal with exceptions, etc.   These are supporting test cases since they don’t describe core intended behaviour, nonetheless may be necessary to confirm the intended design or quality level.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s