Quantitative versus Qualitative software testing criteria? Aren't they complementary...
[textile]It seemed that my perspective on 'quantified' in my SMART thinking post was interpreted as meaning that I was only interested in quantitative methods... when in-fact I see qualitative and quantitative as complementary methods of test-analysis.
This became clear to me thanks to a discussion on the agile-testing mailing list involving Brian Marick and Michael Bolton... also following discussions with colleagues... So, I have revised my written definition of measurable to clarify my point of view:
Can be quantified in some way (even if it is in terms of TRUE/FALSE against one or more qualitative measures) and does not contradict another item under review. Succinctly, it is 'reasonably' unambiguous
I was wrong to presume that it went without saying that there is a complimentary relationship between qualitative and quantitative criteria. I was also wrong to presume that it is obvious that 'reasonable' ambiguity is acceptable... where 'reasonable' is relative to your circumstances.
My current view (and it has been so for some time) is that if a test either passes (even if only with caveats) or fails, I need some translation from the qualitative measure into a boolean quantification (TRUE/FALSE)... even if I arrive at that through a qualitative assessment of the result or take a judgement that it 'nearly' passes and hence I qualify the PASS status with caveats...
The lesson I have learned - make sure there is sufficient context to my arguments to minimise ambiguity over what my perspective is on a potentially contentious issue.