Exploratory testing: Asking new questions

In psychology, for many years academics operated under the assumption that people had personality traits. Regardless of whether they are biologically inherent (nature) or culturually developed (nurture), psychologists generally agreed that the traits formed an individual's personality. Some researchers tried to show that people behave differently in different contexts, by administering questionaires about hypothetical examples. The results were mixed. Then, Pervin (1976) asked four people to list some real experiences from their life and describe how they acted and felt. Even with this small sample size, the researcher was able to demonstrate that people act differently in different situations. The difference in approaches was that he did not try to fit the participants' answers into a predetermined list of questions; he let them write about whatever they wanted.

Quantitative research begins with the assumption that the researcher knows exactly what to measure. Qualitative research begins with the idea that the previous quantitative research is measuring the wrong constructs - that the questions on the surveys are not getting to the meat of the issue. The qualitative researcher then takes an adaptive approach to investigate what the real questions should be.

The same investigative approaches can be applied to researching the behavior of software. If you do not agree with the way previous research has been performed (for example, you think a different approach or looking in a different area will find new bugs) then you can investigate the phenomenon with exploratory techniques. The survey (scripted test plan) cannot be written ahead of time, because you don't know what the important questions (tests) are.

But the lack of a scripted test plan doesn't mean you go in blind, 'randomly' testing. You still intelligently probe the system with knowledge of software and how software fails (Whittaker, 2004), the domain and scenarios (Kaner & Bach, 2004, slide 78 & 82), and the team that created the software under test (Kaner & Bach, 2004, slide 63 & 64).

Kaner, C. & Bach, J. (2004). The nature of exploratory testing. Exploratory & Risk-Based Testing. Retrieved on May 26, 2005 from here. (PDF)

Pervin, L. A. (1976). A free-response description approach to the analysis of person-situation interaction. Journal of Personality and Social Psychology, 34(3), 465-474.

Whittaker, J. A. (2004). Software testing as an art, a craft and a discipline. DoD Software Tech News, 7(2), 9-11.