Otherwise known as the functional testing tools workshop, this workshop will be held in advance of the Agile 2008 conference in Toronto this coming August.
My goals/intentions for a next-generation functional testing tool are simple:
- test-first support
- test-last support
- exploratory testing support
- one environment for all three of the above
In other words, I want it all. Because I believe that testers need to have all styles/techniques in their personal portfolio, and I don’t want them to have to switch between tools when they choose the style that best fits their current test mission.
So what does that mean?
Test-first support means elaborating requirements and designs using tests. That may or may not be a specification, that’s a semantic I’m choosing not to care about. I just want to elaborate the requirements using examples and I want to run those examples and fulfill that happy test-driven development-with-customer-tests cycle. Ditto for designs. An environment that supports this would allow me to craft these examples in the absence of the system and it needs to do that because I’m not describing a system (yet). I’m describing a domain. I envision doing things like registering verbs, registering nouns and then constructing examples using verb-noun combinations of those registered verbs and nouns in specific sequences. The nouns have state-like attributes so that I can describe things like ‘good customer’ and ‘bad customer’ without a ton of syntax. Domain-specific, yes. Language? Script? Don’t know. I know it’s not XML because ‘flow’ doesn’t just appear out of an XML document, and it’s important that sequence and control does come easily when reading whatever artifact I’m describing.
Test-last support means verifying and validating a system under test, again using examples. I may choose to run these as manual or automated tests. Same thing – register verbs, nouns, build tests from verb-noun combinations in sequence. With states. You record-playback-refactor within the context of a single verb-noun combination; building larger scripts later, and only once those verb-noun combinations are registered. In the age of Supervising Controller or Model-View-Presenter or Model-View-Controller, I believe these verb-noun combinations should work inside or outside the presentation layer; so back-end business process testing can be done at a different time than the front-end human-computer interaction testing. But the scenarios we use at the back-end can in fact be re-run to include the front-end when the time is right.
Exploratory testing support means interacting with the system to explore its functionality, maintaining state as I go, providing me with logs and the ability to adjust that system state as I go. I can adapt those logs for auditors to see later. I can share the logs with other testers, and they are more productive because of that. In other words, I can share what I have learned about the domain and the system with others on the team. I would expect that again, the common verbs, nouns, and states that have been registered form the backdrop for this.
And yes, I want to switch between any of test-first, test-last, and exploratory testing, even within a single project. Because all requirements are not equal, and the micro-context for any given requirement might just implicate one testing style over the other.
Maybe this is too abstract, but we have to start somewhere. I’d like to think that we start by confirming support for all schools of testing, and to quote Jerry Weinberg at CAST 2008 – "Business is too complicated for any one school of testing. Learn from them all. Taste everything, only swallow what you want/need."