Best Practices and System Testing
Augustâ€™s Software Test & Performance magazine caused me to reflect on best practices. Matthew Heusser article (p.42) refers to the context-driven school of testing saying â€œThere are good practices in context, but there are no best practicesâ€. While I completely agree that it is senseless to speak about a practice without its context, I believe that the word combination â€œbest practicesâ€ is used by most people as a name for â€œgood practices in certain contextâ€. It is no good to look for â€œbestâ€ and â€œpracticeâ€ in a dictionary; it is rather a stable word combination now: looking for â€œhotâ€ and â€œdogâ€ in a dictionary doesnâ€™t help much for the meaning of â€œhot dogâ€. If somebody understands it directly, as the â€œbestâ€ practice to blindly apply â€“ it is definitely a problem.
It is true that context usually isnâ€™t defined well for practices. Quite often it is impossible to define context for a practice completely â€“ you could have millions of exceptions for one practice. So its application to a context is usually left to a reader.
Still another article in the issue, â€œRaising the Curtain on System Testingâ€ by Ambighananthan Ragavan (p.21-26), looks like Matthew and the context-driven school warn against. While there are some good points in the article, some statements startled me. Without even clear definition (I havenâ€™t quite realized how Ambighananthan sees the difference between performance and concurrency testing), the author makes statements like: â€œcomparatively speaking, you should choose the number of test cases in ratios similar to these below:
I wouldnâ€™t say something like this even for a very well define type of systems: depending on context it can differ drastically.
Then Ambighananthan wrote: â€œHereâ€™s a heretical thought: If you really want to make a product totally free from defects, you canâ€¦ If we truly make an effort and maintain a never-say-die attitude, I believe that a product can be made 100 percent defect-freeâ€. I still donâ€™t quite understand how a person doing system (load, performance, etc.) testing can say that. Perhaps limited experience, just never worked with distributed business systems involving multiple third-party tools? Ambighananthanâ€™s bio states â€œhas been involved in NMS product testing for the past three years.â€ NMS probably stands for â€œnetwork management systemâ€. I never was involved in testing such products, perhaps they are simple enough and you can exactly define what amount of each type of testing is necessaryâ€¦ Although I never thought about them this way.