Archive for category Ask the Tester

How do you evaluate testers? A question from StackExchange

I frequent many of online forums. It’s a great way to grow and learn, as the discussions are fantastic opportunities to challenge us with questions we don’t face on our day to day work. I also enjoy the questions that I do encounter in my day to day, because when answering these (or reading answers) for a context different from mine, I discover new ways of thinking about matters I do out of routine.

In English, an interesting site is the Testing.StackExchange” Questions&Answers site. Unfortunately the site is less active than it could be, but there are good questions and very good answers in the list. Names like Alan Page, Michael Bolton and BJ Rollison answer questions there, so it’s a good place for you to ask yours.

This week an interesting question popped at the Testing.StackExchange forum: How can you evaluate the performance of a tester? How can you compare testers’ efficiency?
This is a question that entertains me once a year, when my company does employee evaluations, and my trial at answering it is in this post.
Can you suggest other points of view? What else you recommend to consider? How do you evaluate testers?

Read the rest of this entry »

Tags: , , , , , , , , , , ,

Should/Need Testers know how to Program (a Testing Question from Brazil)

There’s a very active Brazilian software testing discussion mailing list in Portuguese, called DFTestes.
When I say “very active”, I mean an average of 215 messages/month in 2009, and January 2010 has got more than 404 messages. Compared with other mailing list I participate, this is the most active one, and a tip of hat goes to the testers and list-admins that make the discussions interesting and vibrant.

Recently a big argument was held (link in Portuguese, requires registration) on whether “testers need to know programming” or not. I made some contributions to the discussion, and I’ll try to translate my point of view to English here.
Please note that there were a lot of people participating, and the post below does not present the entire debate or even an entire opinion, just the messages I wrote as answers. I hope you enjoy them.

The proposed question was:

Do testers need programming skills? Until a few years ago, most people would definitely answer NO… But I’ve known both people that seek testing jobs as an alternative to coding, and testers that write code routinely. So, do testers need or not to know programming?

Read the rest of this entry »

Tags: , , , , , , , , , , ,

Finding Nemo – Solving Pradeep's software testing challenge (an exploratory approach)

Back on December, Pradeep Soundararajan set a challenge up in his blog.
He built an application with the description below:

This was an interesting exercise.
What took my interest in this one?
– First of all, I liked Pradeep’s post about learning to code.
Learning to code is an important skill for a tester. I can relate to it because I am trying to learn PowerShell. I program in some languages (C#, C++, PHP, VbScript…) and have a programming background (after my Computer Systems Engineer degree, the ‘natural’ path was to code, and I worked as a programmer for many years before finding Software Testing), but PowerShell has different approach and paradigms and learning it will definitely be fun.
– Second, I had an impromptu vacation day, and I thought it would be educational to use it for a peer testing puzzle. I thought I could win this one quickly (I was mistaken, it took me long). 🙂

So I downloaded Pradeep’s application, and got to work!
Rather than just trying to solve the puzzle, I looked at it as if the mission was:
This is commercial 'roulette' style game. Stakeholders want to know it the game logic can be broken/learnt, which could mean a substantial loss of money when people start winning every time.

Testing Hint: Cem Kaner says “Testing is an empirical, technical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek“. So it is important to set straight what is the information to seek. Which kind of bugs to look for?

In this blog I describe my attemp to answer this question in an exploratory way. Read the rest of this entry »

Tags: , , , , , , , , , , ,

Testing questions and the Laws of eCommerce

(Disclosure: I am not a lawyer!) (Request: Are you a lawyer? Please send me corrections 🙂)

Matters related to law, and all the discussions around it, interest me much — especially when related to Software.
This made me read about the subject and keep contact with the legal representatives within the company I work for. This also motivated me to learn and lecture about the legal guidelines in software development adopted by our company, and to lecture about legal matters on software in general at the last Israeli SIGiST conference.
Most important than all, this made people share with me a lot of comments, questions and stories pertaining to the law.

For example, one colleague brought to my attention a case in which he and some friends had bought a PlayStation 3 in the local Office Depot website for 220NIS — when the normal price is almost tenfold! They believed it was some special sale promotion, but at the end Office Depot announced it as a typing mistake and cancelled the sale after it had been acknowledged (sale confirmation by email). 🙁
(There is an article about the episode in Hebrew here if you want to see it).

What is my opinion on the legal aspects of the story? I was asked.
I don’t have one, as I am not a legal professional, I answered. I am, though, a Testing professional, and here is the tester rambling I sent by mail commenting the occurrence:

Read the rest of this entry »

Tags: , , , ,

"Create your testing portfolio" presentation

Some colleagues and I do voluntary work at TechCareer, helping immigrants learn matters related to technology and score a career in the Israeli Hi-Tech industry either as programmers or testers. It is a very good project, with nice leaders.

My colleague Issi Hazan was asked to teach the ITCQB syllabus to testing-oriented-students at TechCareer, but he bravely thought of pushing them to real tests instead, even before they find a job – this would allow them to build experience and get a job more easily (one way to solve the “hard to get a job without experience, hard to get experience without a job” problem).
So we built a lecture on how to “create a tester portfolio“. We explain how testing can be done even outside an enterprise environment: Lots of Open Source projects are seeking for good information from software testers, and a good record there is certainly sure to help. Crowdsourcing is also a  possible way to work on testing before getting a real job (see two posts on crowdsourced tests here and here).

The presentation is available below in flash format, and can be downloaded here: “Create your testing portfolio“.

But PowerPoint is Evil™, so I feel compelled to write it in real words 🙂 too.
It goes like this:


If you are searching for your first job in software testing, one of the big challenges you will face is how to have your résumé stand out. Either if this is your first job ever, or you’re changing career from a different background, the question remains: How can your résumé compete with seasoned testers’?

The solution to the 1st challenge constitutes your second one: actually acquiring this experience that differentiates the seasoned testers. And this is hard.
People looking for jobs often complain they “can't get a first job without experience, and can't get experience without getting a first job“. The logic sounds leak proof solid… where it not for the fact that, well, most people do get a first job. 🙂

So how can a newbie gain this differentiation?
Many people seek certifications, hoping that the additional line or two in the CV with the uppercase LETTERS will cover for the lack of skills and practice. Now, here’s a secret: they won’t.
Certifications do not give you skills or wisdom; they give you a determined lexicon and specific definition for terms. Some employers may want that, but they know it does not equal practice.
Software testing is an art. Young artists do not collect “painter certifications” or “canvas master certifications“… What young artists do is paint a lot, toss all the colored canvases in a big binder, and show them to galleries. Gallery owners can through this portfolio see their practice, evolution, style and commitment.

This is our suggestion to you: start testing, and be prepared to explain your tests to prospective employers.

Meet your Pseudo-Employer

Testing without having a tester job sounds farfetched, but it is actually within reach.
Of course, you can enter a bug-hunting-spree and find defects in any software from your desktop or the web. While you can learn techniques and tools this way, this method misses the most important (and fun) part of testing: The interactions.
The interactions with people involved in the software will give you focus on what is important for them to know. It will also redirect your tests as you go with feedback and comments, making sure you are always providing useful data. Last, being in touch with this human side of the software will give you the opportunity to explain, discuss and participate in fix/no-fix decisions – transforming your findings from simple bugs to valuable information on quality and risks.

And there are many ways to be involved with testing in the development of software.
Here are a few options:

–> Academy Projects

There is a lot of interesting software being done as final projects for Computer Science graduations. These students do all the work themselves, and they will be certainly glad to have help in testing the software.
One of the great things here is that the interaction is all 1 to 1 and face to face, allowing it to be a closer and more dynamic relationship. Feedback and bug-fixing times are quick; you can suggest a change and then see it done the day after.
Posting your ‘testing offer’ as a note in the wallboard, or asking from one of the project mentors to point you a needy student, are all good ways to get started.
And, for what it’s worth, you may appear in the credits on the project report! 🙂

–> Open Source

Sourceforge has hundreds of thousands of open source software projects under development. You can pick one that suits your interests and limitations by using their filtered search, and you can also take a look at their “Help Wanted” section, which many times ask for help in testing.
The open source community is thirsty for contributors, and they respect sound advice. If you show your competence and are provide consistent views/comments, the community will pay attention – and they don’t care about your degree, background or years of experience.
And, for what it’s worth, you may be given credits (or ‘tip of hats) on the release notes! 🙂

–> Crowdsourcing

There are websites where you can receive money for reporting bugs. The two examples I know of: uTest and Flash Mob Testing.
Testing in this environment may pose problems (see this post for a list of concerns) and the interaction you have here is by far less involved and meaningful than the previous two options.
We listed this in the slides because the couple dollars are tempting and people may prefer this option over the others.

Note that here, you won’t get public credit for your work, in case you wanted to showcase it in an interview.

–> Volunteer for a company

This one wasn’t at the slides’ first version, because I had forgotten about this history completely.
When I was at the university, I had a friend working for a software company, a small one without a testing team. To exercise my ‘criticism’ (I didn’t know I was ‘testing’, back then) I started to write him comments on their webpage and their application, notes that they triaged in their bug meetings and fixed in the next release (sometimes).
So it was volunteer testing for a real company.
(This entire story slipped my mind until 3 weeks ago I had a course in the same building as this company is located. It all came back to memory suddenly: that was my first testing gig!! 🙂 ).

In summary

There are many ways to test without a testing job. Find or invent the option that better fits you, and start differentiating yourself as a real tester.
Try it! 🙂
With a little work from your side, you can demonstrate to employers your capabilities, style and passion. And having volunteered to test and provide data isn’t something a recruiting manager sees every day, so it is likely to attract his attention.

The next steps

After you’ve decided where to start testing, following a consistent way of work will make your testing much more efficient.
The slides include many suggestion on how to get a good start: Follow the forums discussing the software (and ask questions!), be sure to read the bug submitting guidelines so your information is treated with care, and find a lot of bugs 🙂

An Alternative

When applying for a job, capitalize on the experience you’ve already got.
Software projects need domain expertise just as they need testing expertise. If you were a stock-broker, a software house developing stock-market software can have great benefit from your testing. If you speak languages, a lot of specific software can benefit from this trait.
See my comment in this post: The entire experience you bring to a team is valuable; recruiters aren’t always only looking for super-testing-wizes.

Good luck! 🙂

About youTesting with uTest

(Note: This post,originally from July, was re-written in August. Only format/wording changes, with additions to make it clearer)

This is an interesting topic:
I’ve been involved lately in many conversations about uTest, or more specifically about its model.
uTest is a website where companies can post their software, along with some guidelines on focus areas, and users around the world can download the app, find bugs and get paid for bugs reported (as long as the bugs are accepted by the posting company).
There is a lot of confusion/discussion around the good parts and the bad parts of the model, so I will share here some of the points I had taken from these conversations (thanks to all the friends who shared insights with me)… Some attentive readers will notice the article is an almost copy paste from a reply in the software-testing group.

Please note that I am not saying “uTest considered harmful” or “don’t use uTest” or anything like that.
Please note that I am not saying “uTest is great” or “use uTest” or anything like that.
All I want to point here are some of the strengths and some of the weaknesses of the model, so every one (both testers and companies) can decide for his own context. I welcome debate over any of these points, and will update my post accordingly. Read the rest of this entry »

Tags: , , , , ,