Category Archives: Test Insight

Brightly Cool Testing Concepts

Train Trainings, or missing the train for fun and profit

Yes, please Sir, you may take your train in platform 1 or 2 to Amsterdam Central.
These were the words that gave me a breath of confidence inside the Amsterdam Airport train station. Amidst many signs in the undecipherable Dutch language (insert jokes about Hebrew letters here), numbers are easy and I know these from tender age. Off I go to platform 1 (where a computer screen did indeed say “Amsterdam Central” in large fonts) hopping in the first train that arrives.

After a couple stations in which the small computer screen in the wagon did not show anything resembling “Amsterdam Central”, I asked the young boy sharing the bench with me whether I’m on the right direction. Turns out I was not, ha! Had to have waited for a later train in that now far-away platform 1.
In the last moments before the train left with me for another city altogether (Utrecht), I got off the train and a station operator helped me to a platform that had a direct train to Maastricht (for EuroSTAR 2015!). And I felt back in control.

Ok, not a remarkable story. Why am I breaking a 4 years blog-hiatus with such inanities (other than to humour a good friend)?
It’s because I’ll try to tie it with testing and learning. Let’s try it together:

Meanwhile I texted my Wife, explained that I’d lost time by getting in the wrong train. “OTOH,” she asked, “if you got on an earlier train and then did an exchange, you may have saved time. Right?
And that’s the point: I don’t know.

I learned a few things about Netherlands trains during the 20 minutes this happened: that platforms are used for many different lines and directions, that even though the main sign for a platform points to one destination there are smaller trains that also stop there before the big ones, that even though we are told to change trains in a station we can exchange in others… but I still didn’t know whether what I did was a delay or a lucky win. Even though I am empirically experiencing the trains first hand, I’ll have to wait to meet a friend to show me the train rails and timetables before I can start to understand more of what had happened. Either that, or opening the info by myself to make sense of it, or alternatively spending a couple days doing the trajectory between the stations taking notes.

Sitting in the train right now, it felt similar to what we feel testing software. We act, the application reacts, we do one thing and the software replies with another. Sometimes we go in a wrong direction, take notes of the new learnings and then go back. And, after a while testing, a question pops: We’ve been learning all day long, now this peculiar behavior — is it a bug or not? Is it a problem or perhaps a lucky feature?
And often we don’t know. Only after we admit that our learning is not complete, we can look for a friend to help us with the answer, or identify the materials needed to fill the knowledge gap, or decide on the next empirical steps required to elucidate the question… or at least get closer to an answer or recommendation.

Software is tricky (it’s invisible, it’s intangible) and it is infinite. Learning software is infinite as well. If we fall in the trap of feeling in control, we’ll not notice when big questions are yet unanswered.

Well, I should post this online, before I miss my station.

The Big Exploratory Testing Rolling Strategy Dice

Marlon Brando said in a role he played: “People win with loaded dice“…
This is the exercise we’re building today, loaded dices – for the win!

…I get by with a little help from my friends. One of the friends at work who pushes me into all sort of good experiences is Issi Hazan. At this example, he invited me to join him in a testing class for the Tech-Career project (a social wellbeing program in Israel that works with immigrants from Ethiopia to form them and prepare them for a job in the High-Tech industry. They were mentioned at this old post about building a portfolio, and at work we volunteer by giving them coding and testing classes).

Issi’s aim is to teach the group strategies for Exploratory Testing.
But how to do that in a single short out-of-the blue session? How do you convey the strategy, heuristics, coverage models, quality criteria in a practical easy-to-get form? How to make it dynamic and not-boring?

In this post we want to run through you this idea for an aiding tool for the instruction — and the execution too — of tests of exploratory nature. Continue reading The Big Exploratory Testing Rolling Strategy Dice

Lightning Talks night with EuroStar 2010 (the Rebel Alliance, a conference after the conference with beer)

I am back from EuroStar.
I’ve got lots to write about EuroStar: the people, the lectures, the Test Lab, the venue. However, if my mails are any indication, the thing people are waiting most to read about is the “Rebel Alliance” night.

This night, which went by many names (“Rebel Alliance”, “Oprørsalliancen”, “Danish Alliance”), was an informal meeting of friends. As we did at StarEast, it was “a mini conference after the conference, with beer”. We had there very special people and some famous names from the software testing world, and we spent the night talking testing, debating testing, listening to lightning talks on testing, playing games and debating some more. I am a boring nerd, so for me it was the best party ever.
The content and energy were fantastic, it was a remarkable evening. In this post I’ve collected some pics, videos and links.

Who was there? I know I’ll miss a name, but here goes a list (random order):
Jesper L Ottosen | Joris Meerts | Dorothy Graham | James Lyndsay
Bart Knaack | Zeger Van Hese | Martin Jansson | Henrik Andersson
Michael Bolton | Andy Glover | John Stevenson | Rob Lambert
Carsten Feilberg | Ajay Balamurugadas | Markus Gaertner | Henrik Emilsson
Julian Harty | Rob Sabourin | Rikard Edgren | Shmuel Gershon
Lynn McKee | Rob Lugton

Continue reading Lightning Talks night with EuroStar 2010 (the Rebel Alliance, a conference after the conference with beer)

Alternative Paths for Self-Education in Software Testing by Markus Gaertner (EuroStar talk)

Markus Gartner is very skilled in speed-notes. He’s been around EuroStar speed-noting all the session’s he’s been into. He calls it live blogging, but I use the term Speed-Note because, well, I think I coined it ;).
He was live-blogging all the talks, but… who is live-blogging Markus’s talk? Who watches the watchmen? Well, I took the challenge. What you see below are my notes from his ~40 minutes talk.

Alternative Paths for Self-Education in Software Testing by Markus Gaertner

Markus’s main motivation for organizing this talk was noticing an economic downtime a few years back, where conventional commercial education was costly to follow and pay for by companies or individuals. But even in this situation, if you’re in a position where you want to learn and educate yourself, there are other ways…
Markus’s first conventional/official training course in Software Testing was in 2007, 1 year after starting his work as a tester. By that time he already had got responsibility leading people and projects, and he attributes this to spending that year learning about testing from books and people, thinking “how can I advance in this?”

Who is responsible for your education?
One can assign the responsibility over her education on the employer, on teachers, certifications, family… Markus main message on the lecture is that YOU are responsible for your own education. (editor’s note: I agree. In fact, I believe it can’t work any other way).

Shino (as he’s called informally by school friends) presented some options to self-education:

  • Feedback.
    • By writing a personal journal one can assess how his thoughts change over time — that’s self-feedback. By writing to a blog or magazine you receive a different feedback, which is external feedback by other people reading your stuff. For even more direct feedback, you can write on mailing lists or present at conferences. And there’s also an environments built specifically for getting quick and easy feedback, and it’s called, guess what, Social Media. On Twitter, on LinkedIn, the Software Testing Club or Weekend Testing, you can get immediate feedback to questions and affirmations (editor’s note: some of the links point to my own profiles).
  • Learn to Program
    • Programming can be a very useful skill for a tester.
      You can try to learn some scripting languages to help you during your tests, or study design patterns to understand more about software construction. Learning about programming techniques and technologies or pairing with a programmer can give you a great understanding of what goes on in the code.

There are many more options to learn and advance. Markus divided them by two:

  • *HYPOTHESIS* – Left Side of the Brain
    • Books
      • There are many books you can read, but Markus mentioned specifically three.
      • Testing Computer Software – Cem Kaner
        • Gives a very good understanding of software testing and it structure. Markus heard rumors that a 3rd edition might be on the way. (editor’s note: Can it be? Remember, you heard the scoop here first! :) )
      • A Practitioner’s Guide to Software Test Design – Lee Copeland
        • This book structures techniques of software technique, in a very short form. This is Markus preferred and recommended book. (editor’s note: I like this book a lot, thanks Justin for the great gift. But the book I like most is Testing Comp Software by Kaner (and thanks Hayim for this gift!))
      • Lessons Learned in Software Testing – Kaner/Bach/Pettichord
        • An easy reading book with a multitude of different aspects and ideas about testing in all its aspects.
      • Secrets of a Buccaneer-Scholar
        • This book by James Bach includes heuristics for learning (like the SACKED SCOWS, Long Leash Heuristic, Obsess and Forget…) that can be used in most contexts.
    • RST
      • It’s a commercial course by James Bach and Michael Bolton. You have to take the course frontally in a class room. But the slides are available online (!) at and they include models, heuristics and other valuable points from the course.
    • BBST
      • Developed by James Bach and Cem Kaner, this course is an in-depth view of software techniques and our work as testers (it covers Bug Advocacy and Testing Techniques). One of the ways to take the course is by looking at the videos at the BBST website, the other is by taking the course with the AST association, which include — in addition to that material — collaboration and feedback with peers and instructors.
  • *SYNTHESIS* – Right Side of the Brain
    • Testing Challenges
      • Solving challenges as a game may condition you to solve challenges in real life. Markus reminded us that James Bach is willing to share testing challenges over Skype to anyone who contacts him ( Eusebiu Blindu, for example, has challenges on his website as well (link, link, link).
    • Dojos
      • On coding dojos, programmers pair and try to solve one specific coding problem. They include collaboration, a safe environment and a deliberate practice. Markus cloned this idea over to testing, and in Testing Dojos testers solve a problem (test this, evaluate tools, learn new approaches…) in pairs or individually to sharpen their skills.
    • Weekend Testing
      • A weekend testing session is an online ~2 hours session of testing where people work on their own or pairing. They are like dojos in matter of mission, and include a multi-person discussion over Skype on the mission and the results. (editor’s note: It’s certainly a fun way to spend the weekend! I was there and recommend).
    • Miagi-Do
      • This ‘school’ for software testing was founded by Matt Heusser, is non-commercial and zero-profit, and aims to improve testing skills through practice. By proving skills when solving testing challenges you earn colored belts. Markus is a Black-Belt Miagi-Do tester and can be contacted for more information or for initiation. (editor’s note: While is it true that there isn’t much you can do with the belts :), there’s much you can do with the skills.)

Summarizing his talk, Markus says that there are different styles for learning, and you are responsible to discover what your own style is and how you will advance on it. Markus recommends, in any case, trying as many types of learning as you can.

How do you evaluate testers? A question from StackExchange

I frequent many of online forums. It’s a great way to grow and learn, as the discussions are fantastic opportunities to challenge us with questions we don’t face on our day to day work. I also enjoy the questions that I do encounter in my day to day, because when answering these (or reading answers) for a context different from mine, I discover new ways of thinking about matters I do out of routine.

In English, an interesting site is the Testing.StackExchange” Questions&Answers site. Unfortunately the site is less active than it could be, but there are good questions and very good answers in the list. Names like Alan Page, Michael Bolton and BJ Rollison answer questions there, so it’s a good place for you to ask yours.

This week an interesting question popped at the Testing.StackExchange forum: How can you evaluate the performance of a tester? How can you compare testers’ efficiency?
This is a question that entertains me once a year, when my company does employee evaluations, and my trial at answering it is in this post.
Can you suggest other points of view? What else you recommend to consider? How do you evaluate testers?

Continue reading How do you evaluate testers? A question from StackExchange

The effect of time. And Shana Tova UMetuka (Happy and Sweet Year)

שנה הלכה שנה באה
אני כפי ארימה.
שנה טובה לך, אבא,
שנה טובה לך, אמא.(link)
“For auld lang syne, my dear,
for auld lang syne,
we’ll take a cup of kindness yet,
for auld lang syne. (link)”


Tomorrow we celebrate in Israel the New Year in the Hebrew calendar. Yes, 4 months before December ;), and guess what… we are now celebrating year 5771! The difference between the years counted in Gregorian calendar and the Hebrew calendar is ~3760 years.

New years, calendars and dates are extremely important for Software Testing.

Let me tell you a true story about value, about a bug. It was not a bug I found, but in fact a bug I had created. Continue reading The effect of time. And Shana Tova UMetuka (Happy and Sweet Year)

STAREast 2010 – Bugs on a Plane!

Last month I visited the STAR East conference. It was over two weeks ago, which on the internet age makes it old news.
I follow my own clock, however, and will start posting about STAR East only now. Scott Rosenberg would call this type of blogging ‘history’ instead of ‘journalism’ (link), and it sounds just as fine.

In next blog posts, I’ll write about what happened at STAR. What happens in Orlando doesn’t stay there.
But on this one, it’s a bit about what happened in the way there.

For some time before travelling, I’ve been thinking about physical problems that resemble computer bugs. “Spirits in the material world”. For example:

In this picture you can (barely) see a water bottle that was forgotten by a worker on my neighbors’ roof. Just like a programmer would forget a temporary variable she invented to get some refreshment from the code inherent limitations, a worker will forget his own refreshment once it isn’t needed anymore.

Continue reading STAREast 2010 – Bugs on a Plane!