Behaviour Driven Development with Powershell

or, A Testament to the Flexibility of Powershell

The greatest impact that the Functional Testing Tools Workshop in Portland (October, 2007) had on me was to raise my awareness of behaviour-driven development, in particular, the ‘given-when-then’ pattern for structuring examples.

I will defer to other blogs on describing given-when-then in detail, you’ll see what it means soon enough in the examples below.  For now, think of given-when-then as a means of translating the meaning of a user story into a related set of acceptance criteria using examples.  Something that has always been a bit awkward for business users and developers alike.

Given you adopt the story-telling convention popularized by Mike Cohn (I think),

As a <blank>
I want to <blank>
So that <blank>

when you want to translate the story into a set of related acceptance criteria using examples,
then use the following convention popularized by BDD proponents such as Dan North:

Given <a known set of conditions or state>
When <something happens>
Then <check for an expected set of conditions or state>

There are tools that help you do this and to make statements worded like this executable – in that case, they almost become executable requirements.  They are certainly executable examples.  The most-often-used example is for a simple account manager class (I’ve seen this example on a number of blogs but didn’t pay attention to where it originated, but I think it was Dan North’s again).  I’m not the only person looking at this – there is a NBehave project on CodePlex that provides a mechanism for writing tests in this manner.  And there are many ruby examples in both rspec and rbehave.

# Story: Transfer to cash account from savings account
# As a savings account holder
# I want to transfer money from my savings account
# So that I can get cash easily from an ATM

# Scenario 1: Savings account is in credit
given "My savings account balance is 100 and my cash account balance is 10"
when "I transfer 20 to my cash account from savings"
then "my cash account balance should be 30 and my savings account balance should be 80"

given "My savings account balance is 400 and my cash account balance is 100"
when "I transfer 100 to my cash account"
then "My cash account balance should be 200 and my savings account balance should be 300"

My opinion is that this is a clear way of expressing an example, albeit all that I have really seen are simplistic ones.  I’m going to enjoy working through more complex examples on an upcoming project.

Now on to Powershell.  It was always my intention to introduce some sort of behaviour-driven development helper functions in PSExpect, but I was never certain of what that could be.  Until I went to the Functional Testing Tools conference in Portland.  I love it when I can take some time off to think, and be inspired by a group of brilliant people.

Turns out we can make those examples above executable by writing a function and adding a set of aliases to our Powershell environment.  The function accepts a string parameter and a script block parameter.  All that it does is write out the string to a log file and then run the script block.  I’ve called it Invoke-Block for lack of imagination, and it uses the PSExpect logging mechanism so that the same log file shows the results of assertions.

function global:Invoke-Block([string]$Label, [ScriptBlock]$Script)
{

    # build the log entry and write it out
    $entry = BuildLogEntry $Intention.ShouldPass $ResultPrefix.Passed "PSpec" $Label
    $added = $Assertions.Add($entry)
    if ($LogFileName -ne $null) {
        $logEntry = BuildMessage $entry
        $logEntry | out-file $LogFileName -append -width $LogFileLineLength
    }
   
    # now run the script block
    if ($Script)
    {
        &$Script
    }
}

set-alias given invoke-block
set-alias when invoke-block
set-alias then invoke-block

The aliases enable using the given-when-then triad in a Powershell script without generating any errors.  This means that the executable version of the example is the same as above, with the script blocks added in.  The example relies on a class library from the system under test (SUT) that the script loads using reflection (not shown) but the calling syntax to that library is hidden using a set of Powershell functions that utilize a verb-noun naming convention: create-account, transfer-money, set-accountbalance, and test-accountbalance (these are elements in the grammar of the domain-specific language, or ubiquitous language for this particular system under development).

# Story: Transfer to cash account from savings account
# As a savings account holder
# I want to transfer money from my savings account
# So that I can get cash easily from an ATM

# Scenario 1: Savings account is in credit
given "My savings account balance is 100 and my cash account balance is 10" {
    set-variable -name SvAccount -value (create-account "Savings" 100) -scope script
    set-variable -name csAccount -value (create-account "Cash" 10) -scope script
}

when "I transfer 20 to my cash account from savings" {
    transfer-money -From $SvAccount -To $CsAccount -Amount 20
}

then "my cash account balance should be 30 and my savings account balance should be 80" {
    test-accountbalance -Account $SvAccount -Expected 80 -Label "Savings.Balance.After"
    test-accountbalance -Account $CsAccount -Expected 30 -Label "Cash.Balance.After"
}

given "My savings account balance is 400 and my cash account balance is 100" {
    set-accountbalance -Account $SvAccount -Amount 400
    set-accountbalance -Account $CsAccount -Amount 100
}

when "I transfer 100 to my cash account" {
    transfer-money -From $SvAccount -To $CsAccount -Amount 100
}

then "My cash account balance should be 200 and my savings account balance should be 300" {
    test-accountbalance -Account $SvAccount -Expected 300 -Label "Savings.Balance.After"
    test-accountbalance -Account $CsAccount -Expected 200 -Label "Cash.Balance.After"
}

RaiseAssertions

The Output

29/11/2007 1:56:13 PM,SHOULDPASS,PASSED,PSpec,My savings account balance is 100 and my cash account balance is 10
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,PSpec,I transfer 20 to my cash account from savings
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,PSpec,my cash account balance should be 30 and my savings account balance should be 80
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,Savings.Balance.After
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,Cash.Balance.After
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,PSpec,My savings account balance is 400 and my cash account balance is 100
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,PSpec,I transfer 100 to my cash account
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,PSpec,My cash account balance should be 200 and my savings account balance should be 300
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,Savings.Balance.After
29/11/2007 1:56:14 PM,SHOULDPASS,PASSED,Cash.Balance.After

The output above is from the log file that PSExpect generates on every run.  That whole SHOULDPASS concept is there to support a programming style where you first create some tests that you expect to fail – you mark them SHOULDFAIL and then PSExpect prints those assertion failures in YELLOW instead of RED like it would if an assertion labelled with SHOULDPASS fails.  I’ve come to rely on log files to use as evidence of passing tests, and to support debugging (since I tend to avoid using a step-through debugging tool on my own code; it’s an anti-pattern for my personal coding style since using a tool like that means I’m confused and I don’t have enough tests to show me the way).

A Way of Explanation

Yes, this really executes and the output demonstrates that each of the checks pass – this example is available in the most recent upload of PSExpect on the CodePlex site in the ‘samples’ directory.  A couple of things worth pointing out …

First, the starting curly brace of the script block must be on the same line as the call to invoke-block, otherwise Powershell parses the function call without the script block and errors abound.  A minor inconvenience if that isn’t already your scripting convention.

Second, the unfortunate use of set-variable in the first ‘given’ script block.  That’s really just in order to set the scope of variables (hence the use of the -scope parameter).  Alternatively, you could declare the variable and instantiate it before any of the ‘given’ clauses.  I don’t think it matters, but this was the way that it was done in the NBehave example on the CodePlex site so I’ve copied that approach in my example.

Third, those wrapper functions.  I’ve mentioned in other blog entries that Powershell uses the verb-noun naming convention for all the built-in cmdlets and that you are forced into this same convention when writing your own custom cmdlets.  Well you should follow that form when writing Powershell functions too and write your functions like they are prototypes of cmdlets.  Much better all-around style, and you don’t lose any ground making changes to scripts if you choose to promote a function to a cmdlet.  I prefer to use the wrapper functions in test scripts since they enhance the readability of the script for non-developers.  I’ve rarely specified the wrapper functions ahead of time.  Instead, I write the test scripts first and determine what wrapper functions I need based on the tests that I want to run.  Yup, test-driven test helper functions.

If you like behaviour-driven development, and you’re working with a .NET class library, then using the aliases for given-when-then as specified above seems a lot easier than writing compiled code versions of the test scripts.  Further, and I shouldn’t even say this, you don’t even need PSExpect to do this – you can write you own version of invoke-block and set up your own aliases to match your own style.  Meta-programming is just one of the advantages of a dynamic language.

Next Steps: Hyper-Testability

But there’s more that can be done if you are using Powershell.

Visualize if the test authoring helper functions such as test-accountbalance, set-accountbalance, transfer-money, create-account were Powershell cmdlets instead of Powershell functions.  It’s nice to have those tester helper functions initially in Powershell because they can be quickly modified to match the needs of the test authors, however, they will eventually stabilize.  That’s when it’s time to make them cmdlets and release them along with the rest of the code.  Release the testing support tools as part of the working software you’re delivering (at least when you’re releasing into a test environment).

Having the cmdlets for supporting the automated testing (behaviour-driven or otherwise) makes any number of scenarios possible – and certainly enhances the testability of the system.  And I’m willing to bet that there is massive overlap between the cmdlets used for system administration and the cmdlets used for testing.  In other words, if you’re planning to use cmdlets (and an MMC snap-in) to support the administration of your product, then taking it just-a-little further and creating cmdlets for testing purposes isn’t all that much more work.  And the result is something that I’m starting to label "hyper-testability" – where an empowered, educated tester can write any number of scenarios using building blocks provided by the system under test.

Three cheers for empowered testers, testable systems, and oh yes – three cheers for enlightened discourse among a group of really smart people that were kind enough to let me listen in.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s