I know we’re not supposed to come up with solutions before fully understanding the problem, but I’ve got just enough experience and wide-eyed insensibility to believe that I’ve seen something that just might work: testing mashups.
Microsoft Popfly is a set of tools for creating web pages and the like from pre-defined blocks. The one aspect that I am particularly intrigued by is called a mashup. You use a tool called the Mashup Creator to build a web page comprised of blocks that come from disparate sources. For example, build a page that lists your Facebook friends’ statuses and your Twitter friends’ most recent updates. I’m assuming it’s called a ‘mashup’ after the term that’s been coined for mixing songs into single tracks (some of my favourites have sampled from 5 or more songs).
Now imagine if the "blocks" used to build a mashup were Powershell cmdlets instead of web applications and that those cmdlets were designed to expose the functionality of an application under test. Each "mashup" you create is therefore another application scenario. Imagine that some of the blocks you use on a mashup are data sources so that you could make the scenario data-driven.
So, why do I think this might be a good idea?
First, it may help non-programmers to develop automated test scenarios that are executable. Second, the economics of building cmdlets for the application just changed: not only might the cmdlets be useful for system administration, but now they will also be useful in writing automated tests for the application.
Why cmdlets? Well they exhibit all the behaviour that I would want from blocks for building tests from – they have a naming convention, there are behavioural standards that they adhere to, and there is a mechanism already in place for stringing them together and running them as one scenario, so they’re not just commands that can be invoked. If the ‘noun’ part of the cmdlet always refers to something from the business domain, then we’re working at the same level of abstraction as (hopefully) the requirements are written – so as testers we’re grounded, too. There is a tie-in to the whole world of domain-specific language and domain-driven development that just can’t be ignored.
Now these aren’t the only tests that would need to be considered for any system under development. A test strategy for any software product is going to include extensive developer (unit and low-level integration) testing and manual functional and acceptance testing. I get that. At the same time, automating more test scenarios means they are shareable and immediately executable by anyone on the team, enhancing communications and reducing communication overhead. That’s a good thing, and worth striving for.