Testing Thoughts

I am fond of quoting. Not sure why, but I like to quote. I guess it gives some legitimating to what I am saying: Things are much more truer when said by other people, and if this other is famous there’s no possible rebuttal! Unless, of course, there is a conflicting quote by someone not less famous. Then… we’re quit. In order to foster my quoting addiction, and to help me remembering who was that said a smart phrase, I’ll type in this page my favorite quotes. I’ll try to categorize them and put source and date, but mostly I’ll just write a quote and a name. Because I don’t plan to read again all the papers , I’ll jot the quotes down as I remember or use them on my day-to-day. See if it helps you, too!

“Software systems are discrete state systems that do not have the repetitive structure found in computer circuitry. (…) The number of states in software systems is one order of magnitude larger than the number of states in the nonrepetitive parts of computers.” Dr. David Parnas, in Software Aspects of Strategic Defense Systems (links to my post on the paper).

“Contrary to much of our profession’s folklore, a good testing question doesn’t necessarily have a definitive, expected result. Tests that are designed to confirm a prevailing theory tend not to reveal new information.” Michael Bolton, in “How Testers Think“.

“I hasten to point out that, really, it’s neither the testers nor the developers who make the determination whether it’s a bug worth fixing or not. Ultimately it’s the project owner (the project manager, or on Agile teams, the Customer) that makes the decision. (…) The choice to fix a bug is usually far more of a business decision than a technical one.” Michael Bolton, in the comments for “How Testers Think“, telling me to stop discussing already!

“The only system which is truly secure is one which is switched off and unplugged, locked in a titanium lined safe, buried in a concrete bunker, and is surrounded by nerve gas and very highly paid armed guards. Even then, I wouldn’t stake my life on it.” Gene Spafford, quoted in the Apache Administrator’s Handbook, lowering the hopes of these too trustful in their 5 or 6 security tests…

On “incidents of software failure”, more commonly called bugs by the rest of us:

“Those of us who are software professionals know better: the most competent programmers in the world cannot avoid such problems.” Dr. David Parnas, in Software Aspects of Strategic Defense Systems (links to my post on the paper).

On software and programs evolution:

“(…) as programs undergo continual change their structure decays to the point that it is very hard to add something new or change something already there without affecting seemingly unrelated parts of the program in a way that causes errors.” Daniel M. Berry, in “The Inevitable Pain of Software Development, Including of Extreme Programming, Caused by Requirements Volatility“, explaining the Belady-Lehman Upswing

“The addition of any function not visualized in the original design will inevitably degenerate structure. Repairs also, will tend to cause deviation from structural regularity since, except under conditions of the strictest control, any repair or patch will be made in the simplest and quickest way. No search will be made for a fix that maintains structural integrity.” Belady and Lehman (see previous quote), in “Programming System Dynamics or The Metadynamics of Systems in Maintenance and Growth” - this paper I have not read yet.

Graphical User Interfaces:

“Graphical the testing of GUIs is quite challenging. In particular, the testing of GUIs is more complex than testing conventional software, for not only does the underlying software have to be tested but the GUI itself must be exercised and tested to check for bugs in the GUI implementation. Even when tools are used to generate GUIs automatically, they are not bug free, and these bugs may manifest themselves in the generated GUI, leading to software failures.”
Atif Memon, in “Using a goal-driven approach to generate test cases for GUIs”

“To complicate matters further, the space of possible GUI states is extremely large.”
Gregory Kaphammer, after quoting the above Memon, in “Software testing”

On the dangers of being a ‘good’ tester:

“Sometimes familiarity with the technical details of a system can hide problems that are obvious to those that don’t know the technology, the requirements documents, and the test scripts. As testers it is important that we be careful not to let our familiarity with a system make us blind to to bugs — things that bug our users.”
Ben Simo, here.