Testing Applications with Powershell Download – PSExpect Edition

Like many of you, I noticed the download that was made available on the MSDN Tester Centre the other day – Testing Applications with Powershell – Sample Code.  This link, along with downloads and article such as the UI Automation Specification and associated Community Promise gives me some hope that it might be possible to run the same test script both with and without the UI.

That’s my hope for testing tools.  Oh yeah, and that we can do that using business-oriented grammar in the scripts.  And that we can define, scope, and reuse elements of that grammar in exploratory testing, semi-automated style.

Anyway, baby steps … baby steps.

The post was great in that it raised the awareness of how Powershell might be used for testing.  All of my examples in PSExpect are against libraries or other scripts.  This example automates the UI.  And that’s cool if not entirely necessary in all contexts.

I replaced the verdicts within the posted sample code with calls to PSExpect Assert* functions. There were two of them, depicted below with the original code hashed (commented) out:

$result = get-listBox $lb "222"
#    if ($result -ge 0) {
#      write-host "Found ‘222’ in ListBox!"
#    }
#    else {
#      write-host "Did NOT find ‘222’ in ListBox"
#      $pass = $false
#    }
AssertEqual -Expected 0 -Actual $result -Label "TC-1" -Intent $Intention.ShouldPass

$text = get-textBox $tb
#    if ($text -eq "ad") {
#      write-host "Found ‘ad’ in TextBox!"
#    }
#    else {
#      write-host "Did NOT find ‘ad’ in TextBox"
#      $pass = $false
#    }
AssertEqual -Expected "ad" -Actual $text -Label "TC-2" -Intent $Intention.ShouldPass

and then there is the way that the test script reports back to the tester about the status of the test run:

#    if ($pass) {
#      write-host "`nTest scenario result = Pass" -foregroundcolor green
#    }
#    else {
#      write-host "`nTest scenario result = * FAIL *" -foregroundcolor red
#    }
RaiseAssertions

Running the revised test script and running it against the sample application to be tested generates the following two lines of output:

02/06/2008 12:39:26 PM,SHOULDPASS,PASSED,TC-1
02/06/2008 12:39:28 PM,SHOULDPASS,PASSED,TC-2

Since the tests pass, they show up as green.  You can find the entire script at(http://www3.telus.net/~amgeras/samples/Test-TheApp.ps1).  You will find that the script requires the PSExpect library in a specific location – just change that location to get the script to run (one line to edit).

We did drop a number of lines of script completing this refactoring. We also gained functionality since when you run the script, you are also generating a log file in behind the scenes that could be posted in the test results repository.  That’s all good.  But there are other refactorings that we might consider that I will tackle when I get a moment.

First, from a testing perspective, the functions get-listBox is counter-intuitive since it returns an integer when given an item that it is supposed to contain.  I understand why, but the consequences to the test script are obvious when you compare the two AssertEqual statements above.

In the first AssertEqual, the expected result is ‘0’ yet we are really looking for the value ‘222’.  In the second AssertEqual, we are looking for ‘ad’ and that is exactly what we truly are looking for.  That’s more intuitive.  So in the next refactoring I would remove the comparator in the function and leave that comparison for the tester.

Second, and this only reflects my bias towards domain-driven design thinking, is that the sample application mimics some function that doesn’t come through in the test script unless you look really close.   It gets the point across for people like me to talk around, so it was fit for purpose. 

But …

If the screen functionality could be represented using a verb-noun naming convention, then that would provide the means for separating the test runner from the test actions from the set of verdicts and that would make it even more meaningful, and the test script more readable.

For example,

start-productSearch -Category $null -Item $null
test-productSearchResults -Expected ‘Error’

start-productSearch -Category ‘Product Name’ -Item ‘ad’
test-productSearchParam -Expected ‘ad’
test-productSearchResults -Expected ‘222’

Now, the test runner is still the Powershell script (as it was before we did any refactoring).  The screen action is start-productSearch.  And the verdicts are test-productSearchParam and test-productSearchResults, both implemented in the two AssertEqual statements above but whose meanings were hidden in the interpretation of the script.

And this test script is now much more reflective of what the screen does and how we’re testing it.  It communicates better to the testers, and we have this function start-productSearch that we could reuse in other , longer-running scenarios of the larger application.

Right now, these last two refactorings are musings, but I’ll post them and my findings along the way as soon as I get a chance.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s