5 Phases of the Attitude to Software Testing

Boris Bezier (1990) described 5 incremental phases to the attitude of testing.

Phase 0 Attitude - "There's no difference between testing and debugging. Other than in support of debugging, testing has no purpose." The purpose of debugging is to find an error that led to a programming failure. Defects may be stumbled upon but there is no formalised effort to find them.

Phase 1 Attitude - "The purpose of testing is to show that software works." At this phase, there is a marked distinction between debugging and testing. The premise is that the software is correct which may obscure us from discovering defects. For example, subconsciously choosing to run positive test scenarios or designing tests that should not fail. This will not result in scenarios that reveal deeply hidden defects.

Phase 2 Attitude - "The purpose of testing is to show that the software doesn't work." This is a very different mind-set. It assumes the software doesn't work and challenges the tester to find its defects. With this approach, we will consciously design strong, revealing test cases that evaluate the system in its breadth and depth.  The tester reveals a bug, the developer fixes it, the tester designs and executes another test intended to demonstrate another bug – this may lead to circle of ever more “extreme” tests. Taken to the limit, this phase may be prolonged more than necessary, resulting in reliable software but missed business opportunities. The problem with this attitude is knowing when to stop.

Phase 3 Attitude - "The purpose of testing is not to prove anything, but to reduce the perceived risk of not working to an acceptable value." Testing can never prove software is correct. To do so would require us to test every possible valid combination of input data and every possible invalid combination of input data. This level provides developers with information on the software deficiencies and to provide decision makers with a snap shot of the state of the quality of a system at a moment in time. Testing, pass or fail, impacts our perception of the risk about a software product. The more we test with "harsh" tests our confidence of the software increases. We will use risk to decide to release when that confidence is high enough.

Phase 4 Attitude - "Testing is not an act. It is a mental discipline that results in low-risk software without much testing effort." The objective is to reduce the testing effort. Another objective is to reduce the volume of defects. The focus is on making software more testable from its inception. This includes reviews and inspections of its requirements, design, and code. In addition, it means writing code that incorporates facilities the tester can easily use to interrogate it while it is executing. Furthermore, it means writing code that is self-diagnosing, that reports errors rather than requiring testers to discover them.

Each of the above phase is an accumulation of the previous phase attitude and approach. Phase 0 involves no planned test effort but will result in the greatest test cost in customer complaints and fixes. The planned testing increases at Phase 2 and then reduces at Phase 3 onwards, with an increase in quality. While Phase 3 and 4 will involve an increase in the organisation's infrastructure, it might not take a massive over-head to move between Phase 1 and 2, or Phase 2 and 3. 

I can relate to these attitude phases on two levels: My own experience of my attitude to testing and subsequent testing activities has progressed from Phase 0 through to Phase 4. I also have witnessed organisations with attitudes to testing at each of Beizer's Phases. 

Can you relate to Beizer's phases of testing attitude? Do you know where you are personally and where your organisation is at? What do you need to do to progress? If you are operating within a regulated environment, what phase is acceptable for you?