Recently a big argument was held (link in Portuguese, requires registration) on whether “testers need to know programming” or not. I made some contributions to the discussion, and I’ll try to translate my point of view to English here.
Please note that there were a lot of people participating, and the post below does not present the entire debate or even an entire opinion, just the messages I wrote as answers. I hope you enjoy them.

The proposed question was:

Do testers need programming skills? Until a few years ago, most people would definitely answer NO… But I’ve known both people that seek testing jobs as an alternative to coding, and testers that write code routinely. So, do testers need or not to know programming?

—

Do testers need programming skills? My comments:
The question as it is posed takes some assumptions that may be incorrect: Saying that “until a few years ago, most people would definitely answer NO” is like deciding the answer nowadays is YES .
First, from other discussions on this (there was one not long ago in the software-testing mailing list too), it sounds more like whoever used to say “NO” a few years ago still says “NO”, and whoever used to say “YES” still says “YES” . There was no significant technology or methodology novelty that caused a change in the approach to this (Cem Kaner says testing practice has evolved only a little over the years).
What does happen is that some people confuse the software testing we (software testers) do with the unit testing and other techniques that enhance and support development (and are mostly done programmatically). It is true that many testers work on this kind of tasks in their day-to-day, but I believe this is far from the essence of our job — and the distinction of testing and checking by Michael Bolton may help understanding that.
Second; if anything, the advance of years made testing without programming knowledge even easier. Jerry Weinberg describes the tests he did at the beginning of his career as written in punch cards on assembly and given for the machine to execute. At these times, it was much harder (bordering unfeasible) to test an app without knowing to program in the language, or details about the data types and system limitations. With time the advancements in interfaces made it, if anything, much easier to test without programming knowledge. Moreover, most languages are today capable of more or less the same things, so when testing a web page most tests will be similar if it was written in PHP or ASP.NET, and many tests will be similar for an app written on C# or Delphi. And in order to learn about internal details, you don’t need to know the code either — memory, performance and process analyzers can help with detailed views about the resources used.

That’s not to say a tester should not know how to code . Programming has many advantages to a tester:

• Being able to understand the type of problems that afflicts applications (knowing “what the bug is doing to the program” ).
  • Knowing about how software is built helps when building a mental model of the software functioning (where does the program makes decisions? when does it asks for resource? how does it communicates with libraries?).
    • Knowledge about the internal code of the application being tested does not necessarily helps, though. At times, this can be harmful as it limits the tests done more than it expands them. But this is another discussion, on the values of Black-Box testing and Glass-Box testing.
• Gives the option to automate tasks or functions when these are needed in a test (automated tests, automated loads, automated performance probes…).
  • And as important, the creation of little tools that aid the testing process (helping quick configuration of a system or gathering data for reports) can help the whole team, and make a tester an important player in the group.
• It helps communication with developers. Programmers are human beings that are easy to talk to when you know their language … Being on the same speaking-context as them really helps. And as it happens in many other geek linguistic elites, some programmers will respect code-speakers more than non-speakers.

So, should testers know how to program?
Well, it definitely helps in many facets. My non-authoritative answer is that it helps so much that no tester should ignore this area, given the opportunity.

—

With so many advantages, should we hire only the testers that know how to program?

Cem Kaner commented once that he “will not accept M.Sc. or doctoral students if they have weak communication skills. My experience is that these are more important, for success in my lab, than programming skills”. Kaner’s vision, on the articles that he writes about recruiting (this book, and this article), is that the teams should be heterogeneous and not everybody should have coding backgrounds — the “domain knowledge” about the area and business of the application are essential for effective tests.
For example, when building a team for testing an app that does stock market transactions, which kind of player would you recruit? People who know how to program, or that know the stock market trade? The best answer, I believe Cem would say, is “people from both sides”. Personally, if I had to choose one of the extremes, I would pick the stock market expert. Hey, but he doesn’t know how to code!! Well, then end user doesn’t know either.
In the comments to Testers don’t think like Developers think like Computers I was asked about recruiting programming testers, and this is the answered there (notice the example reuse ):

Coding isn’t a required background for testers. Coding skills will help on coding tasks, and sometimes with system understanding.
But a diverse background will make the team thrive and explore the system in fruitful ways.
A domain expert can bring much more value than a B.S. in Computer science.
For example, when building an application for trading stocks, a tester with a buying/selling stocks background will find valuable information.
(See lesson 235 in “Lessons Learned in Software Testing” by Cem Kaner, James Bach and Bret Pettichord – “Staff the testing team with diverse background”).

Of course, there is no 1-size-fits-all, and different projects may require a different team profile. Just as different managers and different contexts may.
In essence, I would agree that teams should be encouraged to hire non-coder testers. Or, if not encouraged, the practice should be considered perfectly acceptable.

Another point:
When I instruct testers, one of my first instructions is that operating system, programming language or data base limitations don’t matter. Be careful of *why* the software does the things it does… What matters is what the user wants, and knowledge about technology limitations can often come in the way.
In his article “A Letter to the Programmer“, Michael Bolton comments that the limit of text entry in the chat window of Skype is 32768 chars and a little problem he found with that. In the comments, a programmer quickly jumps to say that 32678 is the standard limit with the text_box element! This programmer did not notice that this doesn’t matter. If there is a problem here, a user doesn’t want to know about API limitations — and testers with coding skills risk “believing the system” instead of “defending the user”.

—

But what about TDD and Unit Tests?

It was commented in the thread that job opening notices more often than not require knowledge in coding language and in automation tools. From this, one sees that the focus on automation is very strong when recruiting testers. But, is this due to a sincere attempt to increase value? This tendency can very well be focusing cost reduction instead (which is good too, but can be harmful when done wrong).
This brings us back to the confusion commented above about the difference between “real tests” and tests (checks) done during programming that enhance and support development (Unit Tests, TDD suites etc). In this video commented on the thread (here, in Portuguese) a programmer explaining unit testing and TDD maintains that if you have great automatic tests that support your code, you don’t need a software testing team.
Ouch, that is a big mix-up!
When TDD tests are mixed up with the tests software testers do, testers are being approached as a programmer’s baby-sitter, looking over the developer shoulders to tell about coding errors. But our real function isn’t (solely) checking whether the “if x < 20″ was coded correctly, but to ask whether “20″ is enough, and in what extreme business conditions there will be 23 required, or when each of these 20 data-structs will need to be bigger.

Programmers make well to test their code thoroughly before releasing for testing, that way we can skip the dumb tests and focus on the real questions the customer is not asking. As it happens, for these questions, most automatic checks are less valuable than manual tests — be it because the automation will not find a bug again, because these tests do not evolve together with the system, or because the automation is not thinking beyond its code. If one could automate everything a tester does and think, then it would have been done and we testers would be having an easy time supervising robots . But you can’t, sapient testing is a human’s thing. See what James Bach writes in his article “Manual Tests Cannot Be Automated“.

(
Last, full disclosure:
- I have a computer systems development engineer degree, and worked as programmer for many years. Knowing how to program in many languages helps my testing job every day.
- Unrelated to this post, and not depending on me , most new hires in my department at work have background and/or formation in programming, this is a prerequisite.
)

This was an interesting exercise.
What took my interest in this one?
- First of all, I liked Pradeep’s post about learning to code.
Learning to code is an important skill for a tester. I can relate to it because I am trying to learn PowerShell. I program in some languages (C#, C++, PHP, VbScript…) and have a programming background (after my Computer Systems Engineer degree, the ‘natural’ path was to code, and I worked as a programmer for many years before finding Software Testing), but PowerShell has different approach and paradigms and learning it will definitely be fun.
- Second, I had an impromptu vacation day, and I thought it would be educational to use it for a peer testing puzzle. I thought I could win this one quickly (I was mistaken, it took me long).

So I downloaded Pradeep’s application, and got to work!
Rather than just trying to solve the puzzle, I looked at it as if the mission was:
“This is commercial 'roulette' style game. Stakeholders want to know it the game logic can be broken/learnt, which could mean a substantial loss of money when people start winning every time.”

Testing Hint: Cem Kaner says “Testing is an empirical, technical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek“. So it is important to set straight what is the information to seek. Which kind of bugs to look for?

In this blog I describe my attemp to answer this question in an exploratory way.

Now, Stop the press!
You may want to go to Pradeeps post, download the file, and try it for yourself before reading my solution. This post will be waiting right here when you’re done.

…

Here is the report of what I did, the line of thinking during my experiment, and the conclusion:

• 1. First, I ordered my environment to allow efficient and organized work:
  1. Increased the Screen Buffer Size of the Command line I was using to 500 (it turned out that I would do well on less, even half, than that (see point 2.b). But it didn’t hurt).
  2. Renamed the executable to be shorter and without number version: ren fnemo_1.7.6.exe fnemo.exe
  3. Changed the prompt to something shorter, cleaner, and that gave me context about this task against the other Cmd lines windows open: prompt $Cfnemo$F$S$G

Testing Hint: Organizing your testing environment before you start will likely help you during your tests. After you’re in the heat of fight, it will be harder to stop and get organized.

• I tried some basic executions of the program, to grasp the feeling of what it does and how hard it is to find Nemo. This also taught me what are the messages returned by the application when Nemo is found, and when Nemo isn’t found. They’re “Found Nemo this time!, Nemo Gill Bubbles SharkTooth Flow Phamplet Stinger” and “Ah! bad luck, didnt find Nemo this time!” respectively.
  1. I also learnt that Pradeep used Perl and Perl2Exe to do the app.
  2. Additionally, the application clears the screen at every execution, so the big command line buffer is not so helpful.

Testing Hint: Planning or scheduling your testing before you experiment a bit with the software can be done. But you risk being completely wrong in your assumptions about the app. In our case, there were no big surprises, but there could be.

• Sarmila commented on the Challenge page about disassembling the executable. Although by learning all the assembly one can learn about the rules that move the fishes around, it would be extremely difficult and overkill. But this made me think on how Perl2Exe works… Maybe it just wraps the Perl interpreter and the Perl script together? If so, what if the script is stored internally in clear? I tried to open the app in an Hex editor, but no, the Perl script isn’t in clear text there, it is obfuscated.
  • I also tried the useful Strings tool by SysInternals. My intention was to see if, maybe, the array of fishes could be seen at the app code, in order to learn the initial order of fishes. It couldn’t.
  • That ended my cheating session , from here on I used only a black box functional approach.
    • Well, it is not really cheating because there were no rules against that. If you do a full reverse engineer of the code, the answer is legit too. But I preferred the straightforward exploratory approach: learning from the software functionality as I use it, and use this learning to redirect my next steps.
• .
• By now, I knew that Nemo changed places between plays.
  1. Nemo also change places between invalid parameters (which may or not be a bug).
  2. Moreover, the other fishes change places too, even their placement related to Nemos’ placement changes.
  3. So I tried to see if Nemo will return to the same place in any consistent way or number of times. Running that is easy, you just enter “3″, “3″, “3″… and count where Nemo was found and where he wasn’t.
     • I discovered there is regularity (for “3″, its every 2, 3, 17, 5, 10, 5…), but the regularity was irregular enough, and also changed for different positions. This was likely a consequence of the real logic, rather than the logic itself. This correlates to the movement of fishes, but does not cause it.

Testing Hint: Take note of your steps while testing. Using the Session Tester tool is a great idea. For this session, I used MS OneNote because it supports rich text, and I could use tables.
Is that like saying that rich texts helps rich tests? .

• Another detail that helped me in the game was knowing the developer and the purpose of the challenge. This puts a lot of content in the context. Pradeep is a rather playful and would put some tricks into it.
  1. First I tried to see if Nemo could sometimes be in any placement bigger than 7 (there are 7 fishes, places 1 to 7, but there were no rules as to where the fishes were limited to be).
  2. I tried in a toss of up to 220 attempts, and Nemo wasn’t in place 8 even once. So I let this idea on StandBy for now.
  3. Then I tried… what if the “Minimum attempts” input at the beginning of the app affected where Nemo appears?
     • 10 is the default for the “Minimum Attempts“. Nemo is at place 1 at the beginning there.
     • 11 attempts chosen: Nemo is at place 1 at the beginning too. Maybe it does not affect?
     • 12 attempts chosen: Nemo is at place 4! Bingo!
     • 13, 14, 15, 16 showed that the original places were cycling at ’3′ intervals. That means that when trying to find Nemo on the default 10 attempts, it will be in the same placements as 10, 13, 16, 19, 22, 25, 28, 31, 34, 37, 40, 43, 46, 49, 52…

Testing Hint: let the test and what you learn during the testing guide you. New information you find should set you in a new direction.

• You’re seeing that “220 attempts” above, and thinking if I really entered each number manually.
  1. By this time, I decided that I was in need of some automation to insert inputs to the application.
  2. The original app had no apparent automation capabilities, so I decided to use the internal input redirection of the Command line:
     • By using the “<” operator, I could redirect an input file “input.txt” to the app using “fnemo.exe < input.txt“
     • I extended that to “fnemo.exe < input.txt > output.txt“, which gives us with a complete dump of results that can be searched with Notepad.
  3. We had an easy automation now, and this can help our sapient testing. I was exploring, and using automation to help me try different things help me explore my possibilities and get a bigger picture of the situation when needed.

Testing Hint: Don’t be afraid of automation. It can help you talk with the software in different ways and interact with it differently. It helps a lot with data generation and logging, too. Another hint: Never let the automation take control of the testing. It is there to help you, not to drive the effort.

• With the knowledge and tool I had so far, I did a lot of trials in order to find where Nemo is at each iteration. This could give me a hint if he moved in any regular or consistent manner.
  • So I made a big file of winning moves (a lot of trial and error went in it).
• .
• It was now time to model the results:
  • First attempts at modeling the results:
  • I got this: Nemo’s position: 1, 3, 5, 3, 2, 7, 3, 4, 7, 4, 4, 1, 5, 5, 2, 5, 6…
    • Nemo was certainly jumping around without any clear regularity, even when I did it twice the times there was no repetition.
• On the Second modeling attempt, I tried to see if a more visual model could help:

• 1	2	3	4	5	6	7
  X
  					X
  				X
  		X
  					X
  						X

• Even in a table with many more results, it made no sense.

Testing Hint: When you look at a problem or thing, different models will give you different view and different details. Multiple viewpoints help — keep changing the viewpoint until you learn enough.

One interesting thing that happened by this time, is that in my manual trials, I was finding Nemo pretty easily.
Intuitively, I was seeing a pattern on the game, a pattern that was not conscious or explicit. It was as if my mind had already learned the rules without noticing them.
It reminded me of what Douglas Adams wrote in Dirk Gently’s Holistic Detective Agency:
“A ball flying through the air is responding to the force and direction with which it was thrown, the action of gravity, the friction of the air which it must expend its energy on overcoming, the turbulence of the air around its surface, and the rate and direction of the ball's spin.
And yet, someone who might have difficulty consciously trying to work out what 3 x 4 x 5 comes to would have no trouble in doing differential calculus and a whole host of related calculations so astoundingly fast that they can actually catch a flying ball.
People who call this instinct are merely giving the phenomenon a name, not explaining anything.”

• I still thought that looking at the other fishes will bring the breakthrough needed.
  1. In order to learn the movement of the fishes, I did a table of their placement in relation to Nemo at every run.
     1. i. This is what I got:

     2. Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
        Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
        Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
        Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
        Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
        Nemo	Phamplet	Gill	Bubbles	Flow	Stinger	SharkTooth
        Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger

     3. See anything interesting? The rows repeat themselves alternatingly! So the fishes were not really moving around, they were following Nemo!
        1. My take is that Pradeep has two different arrays: One for Odd rows, the other for Even rows.
           1. This may not be true, but it doesn’t matter for us. When testing, we not always know what exactly the programmer wrote in the code, but we infer a mental model. If it suits the needs, it is a good model even when not the real thing.
              1. Many times I think these fake mental models are even better than the real thing. It’s the best way for them to act as an intuitive Oracle when analyzing an application.

Testing hint: When you learn and infer about software, you make assumptions and a mental model of what the software does inside itself. This model does not need to be an exact depiction of the code underneath the app. As long as it answers your questions, and you change this model constantly as you learn, it is good.

• From this part on, I started to analyze Even rows and Odd rows separately.
  • Bingo! Seeing the pattern was easy now.
  • Odd rows follow this cyclic sequence: +4, +4, +1
  • Even rows follow this cyclic sequence: +0, +4, +4
• .
• But what with the differences when using different “Minimum Attempts“?
  • A quick look at the tables I had gathered showed that they follow the same rules, but start at different places.
• .
• At the end of this trials, these are the set of rules that allow me to know where’s Nemo at any given time:
• a. Where does Nemo start?
  1. “Minimum Attempts” of 10, 13, 16, 19…

  2. 	Place 1	Place 2	Place 3	Place 4	Place 5	Place 6	Place 7
     First Line	Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
     Second Line	Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles	Flow

  1. “Minimum Attempts” of 11, 14, 17, 20…

  2. 	Place 1	Place 2	Place 3	Place 4	Place 5	Place 6	Place 7
     First Line	Nemo	Gill	Bubbles	SharkTooth	Flow	Phamplet	Stinger
     Second Line	Gill	Bubbles	Flow	Stinger	SharkTooth	Nemo	Phamplet

  3. “Minimum Attempts” of 12, 15, 18, 21….

  4. 	Place 1	Place 2	Place 3	Place 4	Place 5	Place 6	Place 7
     First Line	Flow	Phamplet	Stinger	Nemo	Gill	Bubbles	SharkTooth
     Second Line	Stinger	SharkTooth	Nemo	Phamplet	Gill	Bubbles	Flow

• b. Where does Nemo go?
  1. i. “Minimum Attempts” of 10, 13, 16, 19… Follow this cyclic rule, starting at step 1.
  2. ii. “Minimum Attempts” of 11, 14, 17, 20… Follow this cyclic rule, starting at step 2.
  3. iii. “Minimum Attempts” of 12, 15, 18, 21… Follow this cyclic rule, starting at step 3.
  4. Odd Rows: +4 | +4 | +1
  5. Even Rows: +0 | +4 | +4
• .
• How do I know it is right?
  • Proof!  I built another automation aid: An Excel spreadsheet that can tell you the position of nemo in all trials given an initial “Minimum Attempt“. It works!
  • You can download the spreadsheet to try it here. Choose any number of initial attempts, it will tell you all Nemo’s position accurately.
  • Is this the real logic behind Pradeep’s challenge? It may not be. It is probable that pradeep did something else, much simpler, and that only translates into this. However, this answers the mission of predicting Nemo’s moves.
    • I am sure that analyzing the findings above one can discover what was really that Pradeep did. I’ll just ask Pradeep on a mail for a suggestion. Testing hint: You can always reach out to the developer and ask him about his software and/or code!

Testing hint: Again, your mental model doesn’t always needs to be exact. As long as it works and answers the questions you have. Keep in mind that this mental model should be flexible and evolve with your knowledge.

• Conclusion sent to the ‘roulette’ stakeholders:
  Until a higher amount of randomness is added to the game, it is too risky to release the game as is now. A malicious player can find the logic of the game after some hours of work, and a pair of smart malicious players in even less time.

Thanks Pradeep, for the nice challenge.
It was a very nice experiment, and I learnt a lot. The best part, for me, was organizing my thoughts in this report, and I hope you like it too.

Request: Are you a lawyer? Please send me corrections

Matters related to law, and all the discussions around it, interest me much — especially when related to Software.
This made me read about the subject and keep contact with the legal representatives within the company I work for. This also motivated me to learn and lecture about the legal guidelines in software development adopted by our company, and to lecture about legal matters on software in general at the last Israeli SIGiST conference.
Most important than all, this made people share with me a lot of comments, questions and stories pertaining to the law.

For example, one colleague brought to my attention a case in which he and some friends had bought a PlayStation 3 in the local Office Depot website for 220NIS – when the normal price is almost tenfold! They believed it was some special sale promotion, but at the end Office Depot announced it as a typing mistake and cancelled the sale after it had been acknowledged (sale confirmation by email).
(There is an article about the episode in Hebrew here if you want to see it).

What is my opinion on the legal aspects of the story? I was asked.
I don’t have one, as I am not a legal professional, I answered. I am, though, a Testing professional, and here is the tester rambling I sent by mail commenting the occurrence:

——
In fact, electronic transactions are special and different that normal stores. They can be built without any human interaction whatsoever, and the lack of human monitoring can bring new and unexpected problems.
Note that I can’t have an official Legal opinion, as I am not a lawyer. I’ll just write what I learn.
A usual understanding in contract law is that once both parts show agreement to the deal, then the deal is closed and valid (I think a lawyer would explain this by stating that there is a mutual offer and acceptance of goods or services). Such agreement is shown, for example, by accepting the payment money.
In such a case as this story, the electronic sale often includes a debit in the buyer’s Credit Card (that turns into a credit at the cancellation of sale). If a service provider accepts payment for a service, he is in obligation to provide the service at the agreed terms. If he didn’t agreed to the terms, he would not have taken the payment. So in our case the fact that a payment transaction was completed supposedly forces the two parts to honor it (the store should not cancel the delivery just as you should not cancel the money debit).

However, apart from the amateur philosophy, we can find a bit more of info in the net about this sort of things…
a) Not only the payment debit can be viewed as a contract acceptance, but also the confirmation mail you receive when buying online can be seen as such, depending on what it writes.
b) The Terms of Service of the website can be used for or against the store, depending on their presence and the judge interpretation of them.

• For example:
• Buy.Com (huge company, 290 million revenue) announced at $164 a $588 monitor. They tried to cancel the sales, and lost more than $60000 in a lawsuit (CNET link).
• On the other hand, Amazon.Com (huger, 19 billion revenue) announced at $11 a $449 handheld (CNET link). They cancelled the sales successfully, claiming that the Terms of Use state that there is no binding contract until the confirmation of shipment arrives.
• Buy.Com had no such protection in their terms, and after the lawsuit they updated the terms to include “have the right to refuse or cancel any such orders whether or not the order has been confirmed and your credit card charged”.
• If it will always help? Not sure. In this link a representative of the FTC says that it will depend greatly on the state the lawsuit is taking place.

In the US, there is a concept of “loss of bargain”, that could allow one to buy the PS3 at normal price and sue for the difference (which represents the bargain lost due to the store’s malpractice). Of course, if the judge identifies the buyer as having bought it to exploit a weakness or obvious typo, the attempt can backfire…

Now that we learned some principles of the domain… We go for the most important…
As I said, I have no legal opinion, but I do have a tester opinion.
First, this shows (again) that there is no final answer in the law yet for new technologies and new problems.

And I now know that for assessing risk on an e-store product, one has to answer these questions too:

• Does the system have any protection against mistakes
• This can be done by issuing an alert if there is a big discrepancy after:
• comparing the product to similar ones
• or comparing to the previous price
• or comparing to the price in the normal (brick) store
• Does the system require a manual human confirmation before sending the payment or confirmation mail?
• This can be bad for a large operation, but for small-medium business, can be a good way of catching mistakes
• Does the system alerts the administrator if there is a sudden surge in the purchases of a newly added product?
• Do the Terms and Conditions protect the company in case of mistakes?
• Are the Terms and Conditions clear to the end user? At least in a visible place?
• Is the user obliged to mark an acceptance to the Terms during log-on, or it is just a link in the page footer?

These are some of the questions I would ask when faced with such a product. There are more, certainly — please share your own ones in the comments or by email

——
One last comment…
Sometimes the typos make the prices higher than the norm… See this picture, a real example from Amazon selling this $400 card for a whopping $5 million!! That’s a lot of money for a graphics card…

Hummm… would they cancel this sale as a mistake too?

My colleague Issi Hazan was asked to teach the ITCQB syllabus to testing-oriented-students at TechCareer, but he bravely thought of pushing them to real tests instead, even before they find a job – this would allow them to build experience and get a job more easily (one way to solve the “hard to get a job without experience, hard to get experience without a job” problem).
So we built a lecture on how to “create a tester portfolio“. We explain how testing can be done even outside an enterprise environment: Lots of Open Source projects are seeking for good information from software testers, and a good record there is certainly sure to help. Crowdsourcing is also a  possible way to work on testing before getting a real job (see two posts on crowdsourced tests here and here).

The presentation is available below in flash format, and can be downloaded here: “Create your testing portfolio“.

But “PowerPoint is Evil™“, so I feel compelled to write it in real words too.
It goes like this:

Introduction

If you are searching for your first job in software testing, one of the big challenges you will face is how to have your résumé stand out. Either if this is your first job ever, or you’re changing career from a different background, the question remains: How can your résumé compete with seasoned testers’?

The solution to the 1st challenge constitutes your second one: actually acquiring this experience that differentiates the seasoned testers. And this is hard.
People looking for jobs often complain they “can't get a first job without experience, and can't get experience without getting a first job“. The logic sounds leak proof solid… where it not for the fact that, well, most people do get a first job.

So how can a newbie gain this differentiation?
Many people seek certifications, hoping that the additional line or two in the CV with the uppercase LETTERS will cover for the lack of skills and practice. Now, here’s a secret: they won’t.
Certifications do not give you skills or wisdom; they give you a determined lexicon and specific definition for terms. Some employers may want that, but they know it does not equal practice.
Software testing is an art. Young artists do not collect “painter certifications” or “canvas master certifications“… What young artists do is paint a lot, toss all the colored canvases in a big binder, and show them to galleries. Gallery owners can through this portfolio see their practice, evolution, style and commitment.

This is our suggestion to you: start testing, and be prepared to explain your tests to prospective employers.

Meet your Pseudo-Employer

Testing without having a tester job sounds farfetched, but it is actually within reach.
Of course, you can enter a bug-hunting-spree and find defects in any software from your desktop or the web. While you can learn techniques and tools this way, this method misses the most important (and fun) part of testing: The interactions.
The interactions with people involved in the software will give you focus on what is important for them to know. It will also redirect your tests as you go with feedback and comments, making sure you are always providing useful data. Last, being in touch with this human side of the software will give you the opportunity to explain, discuss and participate in fix/no-fix decisions – transforming your findings from simple bugs to valuable information on quality and risks.

And there are many ways to be involved with testing in the development of software.
Here are a few options:

–> Academy Projects

There is a lot of interesting software being done as final projects for Computer Science graduations. These students do all the work themselves, and they will be certainly glad to have help in testing the software.
One of the great things here is that the interaction is all 1 to 1 and face to face, allowing it to be a closer and more dynamic relationship. Feedback and bug-fixing times are quick; you can suggest a change and then see it done the day after.
Posting your ‘testing offer’ as a note in the wallboard, or asking from one of the project mentors to point you a needy student, are all good ways to get started.
And, for what it’s worth, you may appear in the credits on the project report!

–> Open Source

Sourceforge has hundreds of thousands of open source software projects under development. You can pick one that suits your interests and limitations by using their filtered search, and you can also take a look at their “Help Wanted” section, which many times ask for help in testing.
The open source community is thirsty for contributors, and they respect sound advice. If you show your competence and are provide consistent views/comments, the community will pay attention – and they don’t care about your degree, background or years of experience.
And, for what it’s worth, you may be given credits (or ‘tip of hats‘) on the release notes!

–> Crowdsourcing

There are websites where you can receive money for reporting bugs. The two examples I know of: uTest and Flash Mob Testing.
Testing in this environment may pose problems (see this post for a list of concerns) and the interaction you have here is by far less involved and meaningful than the previous two options.
We listed this in the slides because the couple dollars are tempting and people may prefer this option over the others.

Note that here, you won’t get public credit for your work, in case you wanted to showcase it in an interview.

–> Volunteer for a company

This one wasn’t at the slides’ first version, because I had forgotten about this history completely.
When I was at the university, I had a friend working for a software company, a small one without a testing team. To exercise my ‘criticism’ (I didn’t know I was ‘testing’, back then) I started to write him comments on their webpage and their application, notes that they triaged in their bug meetings and fixed in the next release (sometimes).
So it was volunteer testing for a real company.
(This entire story slipped my mind until 3 weeks ago I had a course in the same building as this company is located. It all came back to memory suddenly: that was my first testing gig!! ).

In summary

There are many ways to test without a testing job. Find or invent the option that better fits you, and start differentiating yourself as a real tester.
Try it!
With a little work from your side, you can demonstrate to employers your capabilities, style and passion. And having volunteered to test and provide data isn’t something a recruiting manager sees every day, so it is likely to attract his attention.

The next steps

After you’ve decided where to start testing, following a consistent way of work will make your testing much more efficient.
The slides include many suggestion on how to get a good start: Follow the forums discussing the software (and ask questions!), be sure to read the bug submitting guidelines so your information is treated with care, and find a lot of bugs

An Alternative

When applying for a job, capitalize on the experience you’ve already got.
Software projects need domain expertise just as they need testing expertise. If you were a stock-broker, a software house developing stock-market software can have great benefit from your testing. If you speak languages, a lot of specific software can benefit from this trait.
See my comment in this post: The entire experience you bring to a team is valuable; recruiters aren’t always only looking for super-testing-wizes.

Good luck!

This is an interesting topic:
I’ve been involved lately in many conversations about uTest, or more specifically about its model.
uTest is a website where companies can post their software, along with some guidelines on focus areas, and users around the world can download the app, find bugs and get paid for bugs reported (as long as the bugs are accepted by the posting company).
There is a lot of confusion/discussion around the good parts and the bad parts of the model, so I will share here some of the points I had taken from these conversations (thanks to all the friends who shared insights with me)… Some attentive readers will notice the article is an almost copy paste from a reply in the software-testing group.

Please note that I am not saying “uTest considered harmful” or “don’t use uTest” or anything like that.
Please note that I am not saying “uTest is great” or “use uTest” or anything like that.
All I want to point here are some of the strengths and some of the weaknesses of the model, so every one (both testers and companies) can decide for his own context. I welcome debate over any of these points, and will update my post accordingly.

–> (A) Here we go then. Some strenghts of the model:

1. Crowdsourcing provides diversity:
   James Whittaker in his “Future of Testing” webinar considered this model of testing (there I heard the ‘crowdsourcing tests’ term the first time) the next evolution in software testing, as it allows a huge number of testers to test in very diverse environments and configurations.
   Full disclosure: The Webinar was given for uTest and at their website.
2. The managers are nice guys trying to create a nice community.
   In the case of uTest, they have blogs with guest writers such as James Whittaker and others. They are on twitter and occasionally twit a testing article (more often they do brand promotion, which is fine too), they offer bug battles and discussion forums.
   And at these venues they do good. I have heard that Doron Reuveni is a really nice guy, and I believe it from his twits and posts.
   Of course, people tell me that all this community building is about money and business, but that’s completely legitimate. I take money for testing at my job, too. Making money doesn’t make anything bad.
3. Simplified development cycle:
   For some companies, it appears that crowdsourcing makes their development lifecycle more efficient. See this article on how crowdsourcing supposedly slashes development time.
   The points commented in the article are a bit dangerous, in light of the concerns raised below, but for companies that have yet to build a testing team, starting to receive bugs before the testing team is complete can be a good thing.

–> (B) On the other hand, the uTest model has some weaknesses, like:

1. “Software testing” misinterpretation:
   The idea of this model is ‘pay per bug’. Companies are outsourcing testing by receiving bug reports from the community.
   But submitting bugs is only one part of the work that a tester does. Testing (functional too) requires involvement with the development life cycle. Testing requires understanding of the user needs and business needs.
   By putting the testers far away from the company and their decisions or business view, the testers have little option to ask good questions. They have little space to provide real information about risk (that’s what testing is about, right?), and thus they are likely to practice poor testing.
   Moreover, testers get their money for reporting bugs, so this will become the ultimate purpose of submitting them. In any other context many would agree that bugs aren’t there just to be reported or even just to be fixed — bugs raise questions about the application, about the value it provides and about the general direction the product is going.
   When you take the testers out of the fixing decision, you miss this healthy discussion. In this model, if a bug doesn’t get fixed, the tester isn’t probably alerted about this fact. What’s more, the tester may not even care, as he already cashed the bug’s money and “fixing or not fixing is a business decision where we don’t have a say”, right?     And with that, reporting well detailed and complete written bugs isn’t a goal for pay per bug testers too: If you spend valuable time writing a bug report with detailed information, you are not making money! Plus, another person may log this exact bug with a poor report before you finish your perfect report (you don’t receive money in this case).
   Sure, the model does not foster or defend bad practices on purpose. But it has its way to motivate them, in a sense.
2. Poor metrics (paying for bug reported):
   Companies like uTest didn’t invent the “bug quantity measurement metric“, of course. It is an old approach that has failed in many places from what I learnt (might’ve been successful in other places) and is definitely not suitable for all contexts.
   Even if applying companies receive great comprehensive dashboards displaying different quality aspects of the software, at the end of the day they are only seeing bug numbers.”You get what you pay for” — the company is paying for bug quantity and testers (as anyone else) will give you what you measure them for, a large quantity of bugs. It is easy to confuse this data with information about risk or information about value. uTest writes in the Terms of Use: “uTest makes no promises or warranties as to any specified quality standards, testing coverage, maturity, quality measurements, the general quality of services or feedback, or results provided on or through its website.” Now, this is only a standard disclaimer, and some may say these measurements are rubbish anyway, but what uTest is saying, in essence, is “you get only bug numbers, nothing else“, which is true. Companies must understand that bug reports don’t make for a quality or value assessment. See point (C/5) for a related comment.
3. Quick and Dirty is rewarded:
   The rules are: time consuming bugs pay the same as quick bugs. From their Terms of Use: “uTest shall ONLY pay you for bugs reported and approved by the applicable Company and NOT for the time you incurred by analyzing bugs or time incurred for any other purpose.”
   Which type of testers receive a higher score at this model? Testers that report simple bugs will be at higher ranks than the ones that hunt and investigate and find hard bugs.
   Consider this together with point (C/2)… and you’ll have testers that log a large number of shallow bugs.
   This kind of behavior is acquired by whoever practices it… And by the end of the day, some testers with high scores at uTest may have become accustomed to quick and dirty work.And what about bugs that don’t easily reproduce?
   Bugs that don’t always reproduce need special attention and a lot of time invested. Forget about those in crowdsourcing, as more attention equals less money… One is risking that the ones to find the tricky bugs will be the end customers.

–> (C) Some other points should also be considered:

1. Bad disputes fostering:
   Any discussion translates into money. And money, being a terrible motive , can make disputes between a company and a tester biased on both sides.
   Software companies will want fewer bugs, and software testers will want more. None of the two is a good goal, and both can lead to wrong stands.
2. Bad attitude from testers causing bad results:
   Testers from a ‘real world’ project might wish to receive better built software so they can jump into important matters instead of clinging at innumerous surface bugs.
   In the pay-per-bug model, testers will leave a mature software to go test an immature one (more bugs = more money)… so the tester tests worse apps and the software company loses the testers exactly when it more needs them (when the app is stabilizing, testers will migrate to a new, immature one).
3. Disruptive competition between testers:
   In the company I work for, if I share information with a colleague and he uses it to find a bug before me, everybody is happy (as there was a collaborative effort that generated a useful piece of info). Testers in a pay-per-bug competition will be less inclined to share information, as bugs opened by someone else mean less money and less score points.
   In their model, you compete with other testers, and this enters the testing culture of the participants. And testers who compete are les useful in a team than testers who collaborate.
4. The score feedback isn’t objective:
   An award winning uTest tester commented in one of the online discussion that the score for testers doesn’t represent testing skill.
   The skill set required to be a good tester isn’t exercised in such simple view of testing as ‘receive the app, try it, list the bugs’.
   And also, don’t forget that there’s money involved and this usually undermines objectivity .From uTest Terms of Use: “Testers shall have no right to either control or influence their fee rate or feedback regarding their reputation or success.“. Of course, here it is simply a technicality, explaining that the decision over what is a bug is in the hand of the software company hiring the tester.
   But this is an approach where one cannot get more reputation by learning more skills. Instead, one has to wait until someone decides how good he is, with few chances to appeal. Different people in different companies think differently, and the tester has no control over this process here.
5. Companies get the wrong picture:
   A large amount of bugs and activity from uTest-like websites is likely to inebriate the software developing company into thinking they’re getting great value (just as a large amount of activity around automation does in other cases).
6. ‘Unfair’ and harmful competition with your in-house testers:
   For companies that do have testers in-house, sending the application to be tested at crowdsourcing models may pose additional problems:
   - How do the ‘inside’ testers feel about that?
   - What happens when the outside users find bugs they missed?
   - And for the same bug… how come someone from the outside get paid for reporting it? What conflict of interest may appear when a tester realizes he can suddenly make money if he (or his wife, or his brother) reports a bug he found during work?
   - Does a bug-finding competition affects how the in-house testers do their work?
7. Update about different types of test:
   The first version of this article said that “There are companies that value regression tests, for example. As these usually generate fewer bugs, testers at pay-per-bug will not perform them. The same is true with other types of tests that raise questions and not bugs.”.
   Update: I recently discovered that companies that want to run regression or a set of specific tests do use uTest. They post the application with a list of scripted tests for people to follow, and pay for the completion of the scripted tests. So there are ways to do these tests.
   (However, in one such example that I looked at, there were no bugs reported in the script executions. What would be the results if instead of paying for the tests passing, the company asked for bugs? But this problem isn’t exclusive of uTest, it happens everywhere, I guess )

…

There may be more and other points for and against the model.
Please write them on the comments or in a private mail, get the conversation going on this. I may change my mind on this matter too, why not?