Recently a big argument was held (link in Portuguese, requires registration) on whether “testers need to know programming” or not. I made some contributions to the discussion, and I’ll try to translate my point of view to English here.
Please note that there were a lot of people participating, and the post below does not present the entire debate or even an entire opinion, just the messages I wrote as answers. I hope you enjoy them.

The proposed question was:

Do testers need programming skills? Until a few years ago, most people would definitely answer NO… But I’ve known both people that seek testing jobs as an alternative to coding, and testers that write code routinely. So, do testers need or not to know programming?

—

Do testers need programming skills? My comments:
The question as it is posed takes some assumptions that may be incorrect: Saying that “until a few years ago, most people would definitely answer NO” is like deciding the answer nowadays is YES .
First, from other discussions on this (there was one not long ago in the software-testing mailing list too), it sounds more like whoever used to say “NO” a few years ago still says “NO”, and whoever used to say “YES” still says “YES” . There was no significant technology or methodology novelty that caused a change in the approach to this (Cem Kaner says testing practice has evolved only a little over the years).
What does happen is that some people confuse the software testing we (software testers) do with the unit testing and other techniques that enhance and support development (and are mostly done programmatically). It is true that many testers work on this kind of tasks in their day-to-day, but I believe this is far from the essence of our job — and the distinction of testing and checking by Michael Bolton may help understanding that.
Second; if anything, the advance of years made testing without programming knowledge even easier. Jerry Weinberg describes the tests he did at the beginning of his career as written in punch cards on assembly and given for the machine to execute. At these times, it was much harder (bordering unfeasible) to test an app without knowing to program in the language, or details about the data types and system limitations. With time the advancements in interfaces made it, if anything, much easier to test without programming knowledge. Moreover, most languages are today capable of more or less the same things, so when testing a web page most tests will be similar if it was written in PHP or ASP.NET, and many tests will be similar for an app written on C# or Delphi. And in order to learn about internal details, you don’t need to know the code either — memory, performance and process analyzers can help with detailed views about the resources used.

That’s not to say a tester should not know how to code . Programming has many advantages to a tester:

• Being able to understand the type of problems that afflicts applications (knowing “what the bug is doing to the program” ).
  • Knowing about how software is built helps when building a mental model of the software functioning (where does the program makes decisions? when does it asks for resource? how does it communicates with libraries?).
    • Knowledge about the internal code of the application being tested does not necessarily helps, though. At times, this can be harmful as it limits the tests done more than it expands them. But this is another discussion, on the values of Black-Box testing and Glass-Box testing.
• Gives the option to automate tasks or functions when these are needed in a test (automated tests, automated loads, automated performance probes…).
  • And as important, the creation of little tools that aid the testing process (helping quick configuration of a system or gathering data for reports) can help the whole team, and make a tester an important player in the group.
• It helps communication with developers. Programmers are human beings that are easy to talk to when you know their language … Being on the same speaking-context as them really helps. And as it happens in many other geek linguistic elites, some programmers will respect code-speakers more than non-speakers.

So, should testers know how to program?
Well, it definitely helps in many facets. My non-authoritative answer is that it helps so much that no tester should ignore this area, given the opportunity.

—

With so many advantages, should we hire only the testers that know how to program?

Cem Kaner commented once that he “will not accept M.Sc. or doctoral students if they have weak communication skills. My experience is that these are more important, for success in my lab, than programming skills”. Kaner’s vision, on the articles that he writes about recruiting (this book, and this article), is that the teams should be heterogeneous and not everybody should have coding backgrounds — the “domain knowledge” about the area and business of the application are essential for effective tests.
For example, when building a team for testing an app that does stock market transactions, which kind of player would you recruit? People who know how to program, or that know the stock market trade? The best answer, I believe Cem would say, is “people from both sides”. Personally, if I had to choose one of the extremes, I would pick the stock market expert. Hey, but he doesn’t know how to code!! Well, then end user doesn’t know either.
In the comments to Testers don’t think like Developers think like Computers I was asked about recruiting programming testers, and this is the answered there (notice the example reuse ):

Coding isn’t a required background for testers. Coding skills will help on coding tasks, and sometimes with system understanding.
But a diverse background will make the team thrive and explore the system in fruitful ways.
A domain expert can bring much more value than a B.S. in Computer science.
For example, when building an application for trading stocks, a tester with a buying/selling stocks background will find valuable information.
(See lesson 235 in “Lessons Learned in Software Testing” by Cem Kaner, James Bach and Bret Pettichord – “Staff the testing team with diverse background”).

Of course, there is no 1-size-fits-all, and different projects may require a different team profile. Just as different managers and different contexts may.
In essence, I would agree that teams should be encouraged to hire non-coder testers. Or, if not encouraged, the practice should be considered perfectly acceptable.

Another point:
When I instruct testers, one of my first instructions is that operating system, programming language or data base limitations don’t matter. Be careful of *why* the software does the things it does… What matters is what the user wants, and knowledge about technology limitations can often come in the way.
In his article “A Letter to the Programmer“, Michael Bolton comments that the limit of text entry in the chat window of Skype is 32768 chars and a little problem he found with that. In the comments, a programmer quickly jumps to say that 32678 is the standard limit with the text_box element! This programmer did not notice that this doesn’t matter. If there is a problem here, a user doesn’t want to know about API limitations — and testers with coding skills risk “believing the system” instead of “defending the user”.

—

But what about TDD and Unit Tests?

It was commented in the thread that job opening notices more often than not require knowledge in coding language and in automation tools. From this, one sees that the focus on automation is very strong when recruiting testers. But, is this due to a sincere attempt to increase value? This tendency can very well be focusing cost reduction instead (which is good too, but can be harmful when done wrong).
This brings us back to the confusion commented above about the difference between “real tests” and tests (checks) done during programming that enhance and support development (Unit Tests, TDD suites etc). In this video commented on the thread (here, in Portuguese) a programmer explaining unit testing and TDD maintains that if you have great automatic tests that support your code, you don’t need a software testing team.
Ouch, that is a big mix-up!
When TDD tests are mixed up with the tests software testers do, testers are being approached as a programmer’s baby-sitter, looking over the developer shoulders to tell about coding errors. But our real function isn’t (solely) checking whether the “if x < 20″ was coded correctly, but to ask whether “20″ is enough, and in what extreme business conditions there will be 23 required, or when each of these 20 data-structs will need to be bigger.

Programmers make well to test their code thoroughly before releasing for testing, that way we can skip the dumb tests and focus on the real questions the customer is not asking. As it happens, for these questions, most automatic checks are less valuable than manual tests — be it because the automation will not find a bug again, because these tests do not evolve together with the system, or because the automation is not thinking beyond its code. If one could automate everything a tester does and think, then it would have been done and we testers would be having an easy time supervising robots . But you can’t, sapient testing is a human’s thing. See what James Bach writes in his article “Manual Tests Cannot Be Automated“.

(
Last, full disclosure:
- I have a computer systems development engineer degree, and worked as programmer for many years. Knowing how to program in many languages helps my testing job every day.
- Unrelated to this post, and not depending on me , most new hires in my department at work have background and/or formation in programming, this is a prerequisite.
)

So, a few hours before January is over, I’ll transpose here an answer to Testing.StackExchange about the last decade on testing:

Question: What are the most important software testing developments of the decade?

My Answer:
The question asks about the most important developments… Not the best or the worst, the beneficial or the harmful.
I’ll try to answer here with considerations by me and others I found on the net. Not everybody will agree that all these are good — even I don’t agree with all — but my approach here is more of a reporter than a judge.

1. Blogs were the most important testing development, at least for me .
   It helps me grow my testing philosophy. It helps by being a quick and instrumental medium to quality discussions. Following the great authors as they write is a great experience (many authors started blogging only during the 2000′s, like James Bach and Michael Bolton).
   One great thing about blogs is that they allow us to understand the flow of ideas as they are being built upon — instead of simply receiving the ideas later, in book, as was the custom 10 years earlier.
   • Tweeting, forums, and other collaboration platforms of Web2.0 are cool too, but I believe their impact to be smaller.
   • Blogging also allows a simple guy like me to share his Testing Thoughts :).

Other points:

1. The Context-Driven School gathered momentum with this name, and became well known. I couldn’t find tracks on the full history of the context-driven school, but the earliest mentions I could find to it with this name are from the very late nineties. It is clear that Cem and James were publishing context-driven papers from the early 90′s (see here), maybe before — but the first places where I found the “context-driven” name where from 1999 (at the software-testing list. This seems corroborated by C2′s page).
   • Again, I don’t mean to say that it started on this decade. I believe the work of the context-driven people was context-driven for the last 50 years , and all had always sought excellency. It is not new from the 2000′s, but it looks like the movement got a name — and exposure as a movement — only in the last 10 years.
2. Certifications gathered momentum too. Without entering in the good/bad discussion, it is a movement that had a lot of action in the past 10 years and affected the way we discuss software testing today. It wouldn’t be fair to count what happened in the 2k’s without mentioning them.
   • The ISTQB was founded in 2002, and is a popular certification.
   • The ISEB was doing certifications before (so this too, isn’t entirely a new thing), but all got much more impetus.
3. Greater awareness and recognition of the benefits not only of Exploratory Testing, but of testing in general, and testing as a career too.
4. Michael Bolton started consulting in the testing arena, but moreover started writing and publishing. His essays about testing and checking are very cool, and Jon Bach considered it “one of the most (in)famous and important posts to come along in our industry in a long time“.

For the next 10 years…

1. The weekend testing meetings in India were highly praised by James, Michael and Pradeep on twitter. Seems like they believe it will make an impact on testing in the next years.
2. Testing.StackExchange appeared too late in 2009. But… will it appear in the 2020 list? I hope yes.

Demetri Martin

One best friend of mine introduced me to Mitch Hedberg and Demetri Martin, great one-liner comedians. They are/were two funny men!! Three, actually, if you count my friend which is also funny. 

After hearing the disks for over a year, not only the jokes aren’t any less funny, but I’ve started to find subliminal testing messages in them .
I’m writing down these “insights” because I find value in them. And even if they fail to teach you something… Hey! At least the jokes are pretty funny!
So here go some favorite quotes, and their parallel in testing: 

. 

1) “A drunk driver is very dangerous. So is a drunk backseat driver, if he’s persuasive!
'Dude, make a left.'
'But those are trees…!'
'Trust me...'“ 

We are testers. As such, most of the time we aren’t driving the company: Essential operational decisions are made by someone else (project manager, product owner, CEO, whatever you call him).
But if we are not the drivers, our position is certainly close to this of a backseat driver: we have our maps, good knowledge of the area and our experience; and we give advice that is used to take real decisions. So don’t be a drunken backseat driver, it’s dangerous! If you are inebriated by an obsessive desire to fix a bug, or by a personal quest against/for colleague, by a blind belief in a set of metrics or a unfounded trust in a fictitious “Best Practice”, you’ll be taking your company directly into the trees.
And, as with any drunk person… Whenever your judgment isn’t objective… At least recognize/admit that you may be drunk! Will make it easier for everyone else. 

. 

2) “I used to play sports... Then I realized you can buy trophies. Now I'm good at everything!“ 

A plaque, a crown, a card, a trophy… It is common to find people who treasure “victory symbols”, I know I do. They prove you’ve mastered a skill.
Trophies are nice and great, but only if they are accompanied by skills and real world practice. Trophies that can be attained without these traits are just empty cups*. 

One should beware when dealing with “achievement symbols”… At times, acquiring the symbol does not mean acquiring the achievement or skill too!
For example, just as buying the trophy doesn’t make Demetri good at tennis, getting a testing certification won’t make a tester good at testing… The certifications syllabi try to teach only specific lexicon and terms definition, but not the real practice of testing, because they don’t/can’t cover the interactions between persons and players.
For some people, the joke could be read as “I used to practice and study... Then I realized you can pass a certification. Now I can prove mastership - without the effort of gaining it!” Be sure to be from the ones who keep learning and carrying the skill. 

. 

3) “I am afraid of sharks, but only in a water situation. If I saw a shark on the street, I'd be like 'What? F#*k you!'
It's funny, that's like the opposite of how I am with lions!“ 

What’s the Best Practice for dealing with sharks? “Escaping” or “screaming”, right?
What if the shark’s on the sand? Or dead? Not hungry? Not dangerous? You have an anti-bite clothe? Anti-shark cage? Suddenly the ‘Best Practice’ looks more like a ‘so-so advice’, uh? 

The business of testing isn’t different. By looking at our surroundings and context, we can learn important details about our problems – and only then we can try to find an action path that solves it.
Without the context analysis, we may come with solutions that don’t address the real problem or does it badly. One size only seldom fits all. 

. 

4) “If I have to move up in a building, I choose the elevator over the escalator. Because one time I was riding the escalator and I tripped. I fell down the stairs... for an hour and a half.“

Automation is all nice and fine… Until you trip on it! Then it takes long time to recover from the fall (you could be long done by this time) and the fix to the system can be so crooked that it lets you prone to further tripping.
The elevator is a much more robust automation (not only it goes up and down, but it is much safer to trip in it ).
Anyway, who said you have to automate? An equally good way to go up and down is the stairs! You don’t need to wait for them to come, they are as quick as you need (or can make) them to be, you can stop to look around or change direction in the middle of the way, they don’t limit weight or number of users and they are never out-of-order.
Slipping, well, can hurt a bit . Just remember that manually doing things gives you many opportunities a machine cannot provide. 

. 

5) “I saw a sign on this door; it said, 'Exit Only'. So, I entered it and went up to the guy working there, and I was like, 'I have some good news. You have severely underestimated this door over here by, like, 100%, man!'“

Labeling is a natural instinct: The Bible depicts Adam naming animals as his first action in Eden, and Aristotle has been classifying everything since the 300 BC.
But the labels we use can limit ourselves or our tools.
“I'm a tester, I can't code“.
“I'm an engineer, I can't sell“.
“I'm a newbie, I can't help“.
Good news! You’ve underestimated yourself by at least 100%! 

. 

6) “I want to make a revolving door that says 'Pull' on it, just see how obedient people are.“ 

Meeting a Test Case that says “1) Reach to revolving door”, “2) Pull the door to open”. What do you do?
Maybe nothing is wrong. Maybe there are good reasons to make people walk backwards at the entrance.
Or maybe there isn’t, and then, who’s decides? The test case? The design? The developer? The tester?
What are your ways to discover about the door product and its users before deciding on obeying or not?

. 

7) “I was walking down the street, and this guy waved to me. Then he came up to me and said, 'I'm sorry, I thought you were someone else.' I said, 'I am.'“ 

Wow, that’s a funny joke!
Beginning a task with a very clear objective is usually a good thing – but failing to identify or even acknowledge alternative outcomes may not be.
A good friend was working in an academic project, trying to prove a theory about the nature of lasers. However, while progressing in the research, the calculations made clear that the theory wasn’t right. He was desolated – all this work and nothing to publish! He failed, momentarily, to notice that demonstrating a theory wrong has as much scientific value as proving it right, and the publication will advance science just as well.
Testing present us with many similar situations. We can start a test chasing a specific bug, and miss other bugs that may appear. Or we can keep our mind open for different outcomes, and grow your tests as we go from these outcomes. A different bug is hiding there, believe it . 

. 

8 ) “My plumbing is all screwed up. Because it turns out, I do not own a garbage disposal.“ 

How many times we rely on a garbage disposal when we don’t have one?
- Don’t worry, release it that way, we have a fantastic relationship with this customer. What if you don’t?
- It’s fine to release that way, the customer has a great quality assurance team and they’ll catch it if there’s a problem. What if they don’t?
- No problem, we can work Sundays — our testing department understands our needs and won’t care. What if they do care?
- This looks like a very nasty bug. But customers will always have the underlying framework, so it is fine. Will they always? 

. 

9) “I'm in a weird position, because I like rainbows, but I'm not gay. So whenever I go out wearing a rainbow shirt, I have to put 'Not gay'. But I'm not against gays, so under that I'll have to put '... but supportive'. It's weird how one group of people took refracted light. That's very greedy, gays.“ 

Once in a while a group of people will come, wrap common sense ideas in a new named package, and claim ownership, which is as reasonable as taking sole ownership of refracted light.
I could parallel this one to many things or groups, but I chose one where this happens with a lot of enthusiasm… The Agile movement.
Suddenly, you can’t focus on customer and organize a software project in a change-adapting way, without being called Agile. That group just took no-nonsense practices that had been used for decades and re-branded them as if they had invented it!
Don’t get me wrong – it is my opinion that what Agile preaches are often-good-practices-for-many-contexts and there are testimonies of companies, projects and souls being saved from a bureaucracy and poor quality hell by adopting Agile as a new approach. But this does not mean that anyone who is delivering quality in a “release early release often” way is Agile.
For example, even being a total newbie and never part of an Agile team until now, my personal testing approach focus on happy customers, working software, individual interactions and change — without me needing to be an Agilist.
More experienced great people had practiced all that, and test-first, and XP for decades. 

More than that, not only the practices’ refracted light was taken, but even the word “agile” itself was expropriated – see James Bach’s note at his ‘Who stole Agile’ post. 

——
Hope you enjoyed the article, or at least Demetri’s jokes.
If you have more insights and funny stuff to share, don’t hesitate to contact me – I promise to laugh.

Instead of a new post, I revisited and modified last month’s post, About youTesting with uTest.
It has now more content, and still has a discussion of pay-per-bug models.

The initial opinions are still there. While the pay-per-bug model presented by uTest is certainly innovative and interesting; the model still misses a lot. It will certainly be center of discussion many times in many circuits .

To some extent, trying to get quality assessment and counseling from isolated bugs from isolated testers is like getting medical advice from many different isolated doctors without them examining you personally even once.
But, to the other extent, bug reports do consist an important tool for quality evaluation and direction assessment. And well formed groups can benefit from the yet another list of bugs received, as long as they don’t confuse them with complete product testing.

In one of my discussions of the matter with friends, I was encouraged to ask the people working in uTest what they think of these points and how do they and the companies solve the apparent drawbacks.
It is a great idea, as they seem to be very forthcoming and they do have a lot of experience in software development and quality. The CEO, Mr Doron Reuveni, tweets and sports and seems approachable. Not many companies have such accessible CEOs, and I’ll try my luck with him. If he replies and answers our questions, it will be very cool, and I’ll add the info to the uTest posts here.

He writes books, software, and gives interviews about the craft and business of software. Not only that, but (not surprisingly) he’s also got a blog.

Two months ago he wrote that reading your colleagues code check-ins is a good practice, and I think this is good advice. And good advice for software testers too: I read the bugs submitted by my team on a regular base, and it’s been very enlightening (often to me, at times to them too).

I don’t like copy-paste, but as an experience on the parallels between development/testing, I’ll copy his entire post, just changing the parts that are development-related to testing-related words.
“Great artists steal“, right? Let’s see if it is intelligible (my changes are in blue).

Do you work as part of a software testing team?  Here’s a piece of advice for you:

Read the bugs.

Every morning before you start your own testing tasks, use your favorite bug tracking tool to look at all the bugs that everybody else submitted the day before.

There is a reasonable chance that this advice is worth exactly how much you paid me for it. 

Still, testers at many of the best teams I have known do make this their habit.

Reading the bugs is very likely to produce two benefits:

1. The bug might get better. You might find something in the bug description that needs to be fixed. Or you may find a different direction to investigate and find the inherent fault that causes the failure. Or you may suggest another implication of the defect that increases the customer impact perception… So on.
2. You might learn something. Maybe one of your coworkers is using a technique you don’t know about. Maybe someone else has a tool you can use. Or maybe reading the bugs simply gives you a deeper understanding of the project you are working on.

Of course, use some common sense about whether this habit is practical in your situation. For example, if you work on a team of a thousand testers, this probably won’t work well for you. Then again, I assume if you work on a team of a thousand testers, almost nothing works well for you, so this habit might fit right in. 

(Posted with Eric’s permission)

I enjoyed the story very much. It is very exciting to see the passion he had (has) for his software and how he was committed to it. Plus, Ron is a great story teller.
The graphing calculator had all the ingredients of a cool app. It scratched a developer’s personal itch, and is a great example of NeoVictorian computing: built for people, built by people, crafted in workshop, inspired.
Actually, if we’re commenting on NeoVictorianism, Ron was one that really “woke up one day to find himself living in the software factory“. The night got very cold, they said the factory is going to close and he should move somewhere else. The cool part? He kept doing his individual craftsmanship inside the corporation. Secretly.

Now to our matter, testing. He’s got some very good comments on Software Testing (which he calls in the story as QA. Not his fault, but ours). So nice comments, that I’ll just transcript:
At (~20:24), “Our biggest problem was QA“. “Fundamentally, you can’t do your own QA — it’s a question of seeing your own blind spots“, and “it is expensive to do it well“.
At a moment in the story, two testing engineers came to offer help, as in their official work “they were really bored because they spend lots of times developing automatic test tools and now they’re at the point where they push the button and the test starts in the morning and they come back in the evening to see which API’s break“.
These testers were successful because they badly wanted to explore the new software, and also because they had good mathematical experience/background — closely fit skilled testers!

Ron later speaks about usability (~35:57).
He describes the process of the usability studies they conducted after they’ve been programming with users in mind. Ron (which understands well the importance of usability: “In a classroom, any time spent frustrated with the computer is time taken away from teaching“) says that even after all the attention to usability they had, they were surprised by the results of the study: “Sitting behind a two-way mirror, watching first-time users struggle with our software, reminded me that programmers are the least qualified people to design software for novices. Humbled after five days of this, Greg and I went back and painstakingly added feedback to the software, as if we were standing next to users, explaining it ourselves.” In the movie, he adds that they were “pounding on the glass” and that “all engineers should be forced to go through this, watching real users actually trying to use their software“.

There’s another interesting comment at the very beginning (~02:30). Ron comments on how straightforward typewriting work was on the firsts Macs, and how a newbie could go from zero to full speed in less than 5 minutes on MacWrite.
This is very curious: Even today, the computer is more useful as a typewriter than anything else. The uses had varied much, and nowadays professionals use the computer for every nitty-gritty important detail, but still, the first thing that comes to mind to most people when they think “computer” is “writing”. From journals to books to presentations, my perception is that the only thing people can do (on a computer) with no complex instructions is publishing. Want to publish? Open this app and start typing. But you want to render a 3D object? Search for a court lawsuit? Build a repository of data? Out of luck, will take years to learn it well.
Our hardware has evolved, and the purposes we use the computer for have changed too. But the form has stayed the same as 30 years ago. And (not) surprisingly, the interfaces of our apps still resemble the interface of that first MacWrite. Time to change how we interact with computers? I hope so. Science Daily had long ago an article that said “While much about the computer has changed over the last three decades–greater power, faster speeds, more memory… – what has not changed is the user interface.” They offered a gesture interface proposition, and other technologies like multitouch or brain-controlled interfaces are being researched. From my part, I’m ready to test the new paradigm as soon as it’s ready .

This is the full video, this is the link for the graphing calculator storry in text form (10 min instead of 55 ), and go here for a demo of the product at Apple’s launch.

Enjoy, and please let know your testing thoughts in the comments!

When not from a lecture focus, then from a side comment or explanation. Below you’ll find some insights I gained from today’s lecture. When not from a lecture focus, the ideas come from a side comment or explanation:

1. Map everything.
   Vipul’s “I have a dream” talk described an utopic system, in which every single bit of information is not only saved, but mapped and related to relevant bits. In this dream, every single information is mapped, and you can traverse all info and predict taks efforts and chain-reactions.
   As he said, all the technology needed for such a system exists, it just have to be done (in an efficient way).    

   As such, we can start to gain the benefits of the systems by implementing easy thing. Like, for example, Requirements mapping to code. Why is that everybody talks about mapping the test-cases to the requirements, but no one mentions the Requirement to Source Code mapping?
   Such mapping would be very good to:

   • Prevent Gold Plating (number #30 here) (and see a programmer’s rant here);
   • Show the risks (for the application) of changing a requirement (and estimate the change effort);
   • Show the risks (for the requirements) of changing the application (due to a bug fix, for example).

   A Requirement mapping effort is not complete by merely mapping test-cases.

2. 
   Combine testing components.
   If two components are codependent (as in one’s functionality affects the other’s and vice versa), most of the times it is better to think of them as one (although composed) component.
   Whenever you have to do something in one, include the other. It sounds obvious when you speak about testing tasks, like doing a regression and/or a specific test. But it can also be true when you are dividing any ownership and resources. Keep close things close.
3. 
   Combine tests.
   Combine tests into composed scenarios. Run a number of tests – but do them sequencially as part of of a use-case. Instead of testing the find feature, do a combo hit: Open a document, find a text, replace it, save it again under the same name – 4 tests in 1!
   You still got to do all the tests, but the scenario combinations increase incidental complexity ,which usually causes smarter bugs.
4. 
   Software security does not end in the software.
   Business are made of software and data. Be it a collection of different software by different company, or one single solution by one company, the business workflow has a lot of stops and transitions. It is not enough to have a software adequately tested for security, if the workflow (wich is made of people) allows security problems. Authentication, authorization, configuration and more are problems that need to be dealt on the workflow level. This business process needs to be built with a security mindset since the beginning. As Vipul said a day before: “Security is not supposed to be tested, it is supposed to be tested for. And the same is true for performance and usability”.
5. 
   Models always lie.
   I am afraid of models. Since I discovered that many problems with requirements come from people thinking this representation is the real thing (instead of just a reflecting image with imperfections), I am afraid of models. In todays’ Test-Cases from UML lecture by Ofer Prat, he gave a wonderful example about modeling:
   London’s Underground was always mapped in it’s geographical form, it was a model drawn as close to reality as possible. Such a model was complex and almost unusable. Over the time, it evolved in a simpler and easy to follow model, but by then it was not accurate anymore – see this page for a graphical history.  

   So, models (and requirements) are meant to be straightforward directions. As such, they may lack from the accuracy and/or truth that reality presents.  If software is behaving in a bad way, and this bad way fits the requirements, it is time to re-tune the model.

More to come tomorrow!

For testers, thinking like developers is evil. If you think like a programmer, you’ll start excusing the software and will forgive the system’s bugs.

I am reading the very cool book “Dreaming in Code” by Scott Rosenberg, and I just understood a little bit more on why’s so bad sharing the developers mindset.

Scott starts its book with Chapter 0, a common practice in computer literature. When enumerating the reasons, he explains that “Programmers count from zero, not from one. (…) Why do programmers count from zero? Because computers count from zero!” and that developers are taught to think like computers.

This is a very interesting and basic concept.
Dr. Parnas in his “Strategic Defense System” paper (a real favorite of mine ) has a whole chapter called Why conventional software development does not produce reliable programs, in which he clarifies this problem — developers thinking (or trying to think) like a computer:

As we continue in our attempt to “think like a computer”, the amount we have to remember grows and grows. The simple rules defining how we got to certain points in a program become more complex as we branch there from other points. (…) Because one needs to remember so much (…), new problems are created when old problems are corrected.

The beauty in all this is that Testers should not exercise the programming rationale (when analyzing a software) because they’ll be mimicking the rationale of the very same software they are testing. They are limiting themselves to what the program knows about itself, instead of expanding their logic to people-centered matters — i.e. how the software will be used and what it is really meant to do.

Back to “Dreaming in Code”, Scott has an almost poetical way of writing the concepts developed by Dr. Parnas.
Because computers think from zero, “programming include little offsets, translations of ‘+1′ or ‘-1′”, he says. So:

There’s a space between (…) the way the machine counts and thinks and the way we count and think. When you search for explanations for software’s bugs and delay and stubborn resistance to human desires, that space is where you’ll find them.

Poetry.

I am fond of quoting.
Not sure why, but I like to quote. I guess it gives some legitimating to what I am saying.

So, the quote collection is available at this address: http://testing.gershon.info/quote-collection/. It will grow slowly, please check it regularly.

1) “I’ve never seen the same solution work in every place; or in more than one place”.
The above, is quoted (almost) literally from Bernard’s “Systems of Systems” lecture. He described the well known problem, but was unable to suggest a solution.
Any solution you find to any problem, can never work 1-to-1 in your problem. Adapt. Don’t be afraid to change you environment/process too, but usually both should change in order to succeed.

2) Critical tests are done at the end. Live with it – plan for it.
On the “Performance Tests” lecture, Lior Katz said that it is a fact for almost all systems that you can not do proper stress/load/performance tests until the system is stable — which usually means an advanced stage of the project.
This leads to important/critical bugs to be found at critical/important phases. One way to deal with it is to hope nothing very bad will happen… the other is to be conscious of the danger and be prepared to deal with urgent problems in the last milestones.

3) Care for the important level of your abstraction.
Apart from the obvious eye-opener we had in the “Building Blocks” lecture, which taught us that GUI automation should be done in layers, like any other automation, another insight was that you should not care (so much) about ugliness or repetition in the abstraction layer. Three reasons for that:
   1) The most important is that the test-cases layer is easy to implement and easy to maintain (or maintenance-less). That way new test-cases can be added quickly and by unskilled persons.
   2) The abstraction layer is always finite (as it deals with a finite number of states, or components).
   3) From another lecture, “Automation from day one”: “Do not test a system with an equally complex system“. If sometimes repetition or duplication can help your final layer to be simpler, it is better.

1) When you measure an object’s weight, the weight do not gets affected. Similarly, when you measure software’s quality, you don’t have influence on the amount of quality.
That’s from Rex Black’s lecture (10 worst things).
 The lecture was your standard lecture in which a lot of common-sense ideas are put on a slide, but the clarity, order and side-info gives all a better meaning and things get adjusted on your head. I like these lectures, they help to organize your mind, and surely help you to articulate these thoughts later.
 The idea in bold above is a very smart one: Don’t try or pretend to be the Quality Factor in the product you are testing. Don’t be a Quality Assuror. Just because you are a Testing Expert, and know how to measure the quality of an application, does not mean you have the power (or meanings) to increase its quality. Testing is not to improve quality, testing is to give a quality measurement. It is a decision making tool.

2) Don’t be afraid of producing a large amount of test cases. If you need to discard a part, at least you’ll know exactly what you’re giving up.
That’s pretty much self-explanatory. It was one of Vipul Kocher’s points on the “Verb and Noun technique” lecture.
This technique allows for a quick and easy mass-production of test cases. And, for those who are afraid of being overwhelmed by all the test cases, the answer is: “No one is telling you to test them all. But now your choice is conscious.”

3) – Company ‘X’ has no written development requirements whatsoever. It is a bad thing?
   – Yes!
   – It depends.
The above dialog happened during the “V Model” lecture by Geoff. He was talking about a branch of some company (he named it, I wont. Just know that it is quite big…) that systematically don’t have written requirements.
The categorical reply that one participant gave (and most if not all held too) was the obvious – you’ve got to have requirements to be a good boy. How can I test without requirements?
That’s why the answer is so surprising and so refreshing. If an answer is categorical, it is probably wrong. Your cognitive bias is fooling you again.
If this company has a process, on which all stakeholders agree, and on which the testing department has agreed – then it doesn’t matter so much whether there is place for a ‘Write Requirements’ checkbox on it. If the testing department agrees to test without requirements as we know them, perfect.
Summing up, not everything that is different is bad. If it is well analyzed and well controlled, any process can give you results. You can decide on what is important and how much you want to risk.

3.5) You can use a process model as a display of your place on the project, without having to follow the process.
That’s actually the summary of the lecture. Geoff presented a case study, where although the process was not led by the V-Model principles, and did not follow the V-Model path, the V-Model was used as a clever instrument to show all involved parts what are the dependencies between development and testing. It is a clear diagram to show what parts of testing can/cannot be done with/without proper effort from the development side.
Of course… this rule is true for any other diagram/process, not only V-Model. So you probably can choose the model that better suits your necessities. Just be sure not to be biased – anyone can pick the model that fits its own interest and try to apply it to the company reality… Managers should be smart enough to spot where the discrepancies are.

4) Don’t be the one to harms MS Windows, or your own application.
On the “Testing the Windows* registry” lecture, Michael Stahl explained that messing with the Windows Registry can surely disrupt your OS system. Or your application. There are even disclaimers on this everywhere. But he also explained that it only messes the system if you (ok, the programmer) let it happen. If all registry inputs are checked and validated before being used, nothing bad will/shall happen.
There is no excuse – the registry is under the developer control (not the other way around)!

Joel says that nothing improves morale and efficiency like private walled offices. I’ve worked in both way, and in two occasions companies I worked switched methods (from closed environments to open spaces) - in this experience, the gains of working on wall-less and open spaces are visible. Suddenly, everything is quick and everything is clear. No more leaving things to deal later. No more company-wide procrastination.
My advice: break those doors and those walls.

Everybody loves Joel.

Everybody loves his articles, his jokes and his books.
More than that, people love to disagree with him, so they can look smart and judgmental. The funniest part is seeing the judgmental faces they do.

In 2000, Joel wrote a cool article on “Top Five (Wrong) Reasons You Don’t Have Testers“.
He’s got some great information there, and in this post I’ll just comment it (see the ‘Annotations’ tag?).

Ok, so go on (link) and read the paper.
In his paper, Joel debunk 5 miths of companies who won’t hire testers:

1. Bugs come from lazy programmers.
2. My software is on the web. I can fix bugs in a second.
3. My customers will test the software for me.
4. Anybody qualified to be a good tester doesn’t want to work as a tester.
5. I can’t afford testers!

Joel has intelligent advice on the first three points (you’ve read them, right?), so I’ll not discuss them. As for the last two… lots of comments!

Regarding point 4, Joel’s article is a bit outdated.

• Joel says that “most people who are that smart will tend to get bored with day-to-day testing, so the best testers tend to last for about 3 or 4 months and then move on“, and gives some ways to make your testing scene look more inviting.
  C’mon! If your testers become bored after 3 months, you are probably hiring the wrong people. Good testers are passionate about tests, and know what it takes to be a tester. They know the work can become boring if they let it, so they manage the work in a way it will never be boring. Good testers are built for testing, even before you hire them.
  Myself, I started to work on testing out of curiosity, after a few years of development. Man, this is fun! The analytics mindset of a tester cannot be compared to the algorithmic mindset of a developer. They’re two different things, that level on the same job-satisfaction. Humm? What do I mean with Passionate Testers? Read the blogs of Braidy Tester, and of Pradeep, you’ll know what I mean.
  So, Joel, hiring those non-traditional workers as your test-base will just give you big turnovers and you’ll never gather that cool techie testing group that laugh about bugs found.
  A funny quote from the article: “Recognize that you will have a lot of turnover among your top testers. Hire aggressively to keep a steady inflow of people.“. I doubt anyone would treat the development team like that. It should not be different in the testers team – - recruit a winning team, treat them like kings, and you can expect them to serve the company for years to follow.

Last point, point  5. Again, not a very accurate/scientific approach:

• We can summarize Joel’s reply in this point as “Don’t skimp on testers, because you can find some cheap testers”. That’s the same as saying “Don’t skimp on Customer Service, you can hire any outsorced company on India to answer your phones.”. Ahh! Doesn’t make that much sense now, does it?
  The real reason not to economize on testers is that bugs and defects will cost much more than your testing team if you don’t test. If your customer find a bug, one of these which you could have avoided, the sum of costs of all the activities related to fixing and re-releasing your product is big – - very big (not to mention the harm to your reputation!). If your software is a shelf-product, it is even harder and expensive. If you are on hardware, the mere thought is making you pale.

Of course, I am not expecting any answer from Joel, but I like to believe that he agrees with me.