Software Testing Quotes

I am fond of quotes. Not sure why, but I like to quote. I guess it gives some legitimating to what I am saying: Things are much more truer when said by other people, and if this other is famous there’s no possible rebuttal! 😉 Unless, of course, there is a conflicting quote by someone not less famous. Then… we’re quit. :) In order to foster my quoting addiction, and to help me remembering who was that said a smart phrase, I’ll type in this page my favorite quotes. I’ll try to categorize them and put source and date, but mostly I’ll just write a quote and a name. Because I don’t plan to read again all the papers :), I’ll jot the quotes down as I remember or use them on my day-to-day. See if it helps you, too!

Software systems are discrete state systems that do not have the repetitive structure found in computer circuitry. (...) The number of states in software systems is one order of magnitude larger than the number of states in the nonrepetitive parts of computers.” Dr. David Parnas, in Software Aspects of Strategic Defense Systems (links to my post on the paper).

Contrary to much of our profession's folklore, a good testing question doesn't necessarily have a definitive, expected result. Tests that are designed to confirm a prevailing theory tend not to reveal new information.Michael Bolton, in “How Testers Think“.

I hasten to point out that, really, it's neither the testers nor the developers who make the determination whether it's a bug worth fixing or not. Ultimately it's the project owner (the project manager, or on Agile teams, the Customer) that makes the decision. (...) The choice to fix a bug is usually far more of a business decision than a technical one.Michael Bolton, in the comments for “How Testers Think“, telling me to stop discussing already!

The only system which is truly secure is one which is switched off and unplugged, locked in a titanium lined safe, buried in a concrete bunker, and is surrounded by nerve gas and very highly paid armed guards. Even then, I wouldn't stake my life on it.Gene Spafford, quoted in the Apache Administrator’s Handbook, lowering the hopes of these too trustful in their 5 or 6 security tests…

Before software can be reusable it first has to be usable.” – Ralph Johnson.
And he wrote Design Patterns, so he would have a say in reuse…

If you don’t care about quality, you can meet any other requirement” – Gerald M. Weinberg

Pay attention to zeros. If there is a zero, someone will divide by it.” – Cem Kaner

On “incidents of software failure”, more commonly called bugs by the rest of us:

Those of us who are software professionals know better: the most competent programmers in the world cannot avoid such problems.” Dr. David Parnas, in Software Aspects of Strategic Defense Systems (links to my post on the paper).

On software and programs evolution:

(...) as programs undergo continual change their structure decays to the point that it is very hard to add something new or change something already there without affecting seemingly unrelated parts of the program in a way that causes errors.” Daniel M. Berry, in “The Inevitable Pain of Software Development, Including of Extreme Programming, Caused by Requirements Volatility“, explaining the Belady-Lehman Upswing

The addition of any function not visualized in the original design will inevitably degenerate structure. Repairs also, will tend to cause deviation from structural regularity since, except under conditions of the strictest control, any repair or patch will be made in the simplest and quickest way. No search will be made for a fix that maintains structural integrity.Belady and Lehman (see previous quote), in “Programming System Dynamics or The Metadynamics of Systems in Maintenance and Growth” – this paper I have not read yet.

Graphical User Interfaces:

Graphical the testing of GUIs is quite challenging. In particular, the testing of GUIs is more complex than testing conventional software, for not only does the underlying software have to be tested but the GUI itself must be exercised and tested to check for bugs in the GUI implementation. Even when tools are used to generate GUIs automatically, they are not bug free, and these bugs may manifest themselves in the generated GUI, leading to software failures.
Atif Memon, in “Using a goal-driven approach to generate test cases for GUIs”

To complicate matters further, the space of possible GUI states is extremely large.
Gregory Kaphammer, after quoting the above Memon, in “Software testing”

On the dangers of being a ‘good’ tester:

Sometimes familiarity with the technical details of a system can hide problems that are obvious to those that don't know the technology, the requirements documents, and the test scripts. As testers it is important that we be careful not to let our familiarity with a system make us blind to to bugs -- things that bug our users.
Ben Simo, here.

Fundamentally you can’t do your own QA, it’s a question of seeing you own blind spots.” Ron Avitzur, “The Graphing Calculator Storyvideo

Testers and Developers:

... we have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We're more of a testing, a quality software organization than we're a software organization.
Bill Gates on trustworthy computing (Information Week, May 2002)

On avoiding unrecoverable error (or: have a strategy that allows recovery of any bad thing that happens):

Just like with golf, technology is as much about ensuring that your bad hits are recoverable as it is ensuring that you make great ones.“, by AKF consulting, here.

Scripted or Exploratory?

(pure) ~Scripted and ~exploratory tests are two ARTIFICIAL ends of testing continuum. In real life I am doing neither. I just do testing.“, by Vipul Kocher on Twitter.

Exploratory Testing is not so much a thing that you do; it's far more a way that you think.“, by Michael Bolton on Twitter.

We don’t need humans doing something that a machine can do. What we do want the human testers doing is exploratory testing. Exploratory testing is a creative endeavor in which human testers explore the behavior of the system in an attempt to characterize its behaviors; both documented and undocumented.“, by Uncle Bob Martin on an interview.

Exploratory testing is self-managed process; scripted testing is managed from outside.“, by James Bach during RST class, mentioned by Aleksis Tulonen.

Irreproducible Bugs:

Irreproducible bugs become highly reproducible right after delivery to the customer”, Michael Stahl’s derivative of Murphy’s Law

Best Practices:

"industry best practice" is more likely to mean industry lowest common denominator”, by Jason Yip on Twitter.

If you're relentlessly focused on lowering cost, you'll quickly become oblivious to opportunities to increase value.”, Michael Bolton on Twitter.

The amount of formatting and verbosity applied to a test plan is inversely proportional to the amount of actual good testing represented by such plan.”, David Gilbert on his blog.

People, team, managers:

In risk based testing, it's more important to have the right people involved than it is to follow the process exactly right.“, Rex Black

If you’re too busy hiding your lack of knowledge, you won’t feel comfortable asking questions. You’ll think you should already know the answers. Let me tell you a secret: no one knows all the answers. No one. Not knowing the answers isn’t a sign of weakness—not asking questions is.“, Elisabeth Hendrickson

If you're gonna manage testers or any highly cognitive work, then you need to participate in the work.” Interview with James Bach

Measurement and metrics:

Storytelling strikes me as a more powerful tool than quantification or measurement for what we do.“, Mr Alan Cooper, on Twitter

In some situations, the most important objective of testing is to find as many important bugs as possible. In other situations, finding bugs is not important at all. In yet other situations, bug-finding is only one of a number of important objectives. The wise test professional knows which situation she is in.“, Rex Black, March 2010 Rexism

Quality is dead:

When it comes to some programs (...), you are more in a position of being used than you are of being a user.“, Manual for the Franklin ACE 100

Notetaking and Documents:

The paperwork always have to take a back seat to the testing itself.
The testing itself is paramount, and if the paperwork gets in the way, the paperwork needs change.
” Interview with James Bach

What is part of testing?

Actual tests are part of planning too. At least, they can be, if done consciously.” Shmuel Gershon (me!) quoted by Ben Simo at a Weekend Testing session in 2011

More…

There are obvious variables that are inputs and outputs, there are also subtle variables (or hidden variable) that are not directly visible to us but are visible only for their effects to the output variables.” Dale Emery on this webcast with Elisabeth Hendrickson. I add: There are hidden variabled that don’t have any visible effect on output as well…

User actions are always 'variables'.” Elisabeth Hendrickson on the same webcast with Dale Emery

Rules and Procedures may be dumb, but they spare you from thinking.” Scott Simons in an NPR broadcast, cited by Barry Schwartz’s TED Talk

(Last update: 14/Nov/12)

12 thoughts on “Software Testing Quotes”

  1. No, I will not succumb to a mere bug! I will find the steps, it’s a fight until the end…

    …but when I don’t succeed (and it happens, unfortunately), the best thing I can do is to report the bug with the best, most appropriate steps that I could find in a relevant time.

  2. I have much to learn from many of these quotes…

    I especially like this one: “Irreproducible bugs become highly reproducible right after delivery to the customer” – and this is so true, that’s why I like to find proper steps for the bug and report it, even though it’s hard to discover.

Leave a Reply

Things to share about Software Testing