Posts Tagged testing

2020 – the year that tested us. Happy New Testing Year!

This weekend I recalled that up to 10 years ago I used to write New Year posts. There was little to no action in the past 10 years on this blog, but another decade – and 2021 – deserves a “hello again” post!

2020 was by all means special. You may think it was bad, it was good, neutral – but you will agree it was special. 2020 made us rethink the way we live and relate, and the way we study and work.

A month ago, I presented at TestFlix, an online video conference to which The Test Tribe was kind enough to invite me. The conference was scheduled on Shabbath, so I risked missing the conference (and my own talk)… luckily it continued after the Shabbath was over and I got to attend a couple hours of it – including the weird experience of listening to myself.
As far as online conferences went, it was pretty fun, with a small expo and different chat rooms. You look around your room and there’s no one other than your coffee mug, and yet you’re in touch with people around the globe.
EuroSTAR had a similar conference recently, for which you can see the stunning (stunning, I say! Content is top-notch, but just look at that graphic design!) program here.

Everything is online now! The joys of remote working from home sitting in a small room from dawn to dusk!
We’re a household of 10, with 6-7 of the kids studying over Zoom/Teams/Classroom, encouraging a tired and sweating D-Link router to make it through one more Mb – You can do it, little D-Link, you’re a big Cisco!

Testing in the time of Corona… Covid-19 gave us opportunities for many interesting conversations on Testing. From the discussion on whether testing more causes more bugs (after all, looking at your clock makes it tick slower, right?), to the realization that testing is the most important thing ever, all the way through rethinking your work routine.

On a practical and personal note, one of the things that working remotely demonstrated is, well, that testing remotely is possible in embedded and invisible systems. That statement is not as trivial as it seems: You see, working with embedded software means your product does not fit in the contents of a USB-Key, or in a GitHub repo. The embedded product (software) is located in a big board with a specific processor, usually connected to a bigger power supply and having all sort of flimsy wires tied to weirdly named devices (JTAG? Aardvark? Saleae? Lauterbach? Kvaser? Who’s naming these things??) that you connect to your computer when you test. It looks so scary, that once we needed one abroad for a business trip and we mailed it instead of taking it on the plane because it looked so much like a bomb 🙂 .

CC-BY-NC, SNAFU.org

The quickest way to test this software is to be relatively close to it so you can turn it off and on, look at the LEDs, disconnect cables, reconnect cables, plug in other analyzers, look at it angrily to see if it finally obeys, nudge it in despair, and so on. Can we do this with no hands?

In 2015 I presented at EuroSTAR a talk called ‘Demystifying Embedded Software‘ (it was frontal, not online, as the internet didn’t exist back then). We discussed the many similarities between a desktop/mobile application and embedded software running on a specialized processor and an OS, and also the differences (which include optimizations of storage and power, runtime accessibility, unusual interfaces and fuzzy IO, and limitations in the software’s lifecycle).

I debated with myself briefly on whether to include the fact that embedded software needs you near. I was testing sensor subsystems at the time, so tests included shaking, rotating, and flashing lights at our product – certainly much harder to do from a distance. However, we were able to work from a distance – as we had robots to rotate, lightboxes to illuminate, coils to reorient, and all sorts of fancy toys. The conclusion was clear: It is possible to test a system like that remotely (so this didn’t make the slides), it is much harder too.

Jump forward 5 years and we must test things remotely 24/6. I am not testing sensors anymore, but the above still holds true: While I can test remotely, it requires fancy solutions as a medium.

While everyone will quickly tell you that that makes the work slower, I now discovered slowness is not the major drawback. The problem is that mediation means talking with the product through a translator. And if in person you can get a feeling of the software’s mood (is it slowing down? is it in trouble? did it restart surreptitiously?), having an SSH commanding an automatic query makes it much harder to notice the software’s reactions (is the network slow? is the serial interface in trouble? did the JTAG interface flicker?).

If on one hand I have now a full remote setup that allows me to control the product, on the other hand I still haven’t overcome the alienation that distance brings: I can’t feel the product due to the noise caused by the input and output toolchains. Monitoring the tools in use is becoming as important as monitoring the product itself.

“Monitoring the tools in use is becoming as important as monitoring the product itself.”

— Previous Paragraph

My lesson? When working remotely, we have to be double alert.

It is an interesting place to be, and it will hopefully retranslate into sharper instincts once we come back to normal.

Tags: ,

The Big Exploratory Testing Rolling Strategy Dice

Marlon Brando said in a role he played: “People win with loaded dice“…
This is the exercise we’re building today, loaded dices – for the win!


…I get by with a little help from my friends. One of the friends at work who pushes me into all sort of good experiences is Issi Hazan. At this example, he invited me to join him in a testing class for the Tech-Career project (a social wellbeing program in Israel that works with immigrants from Ethiopia to form them and prepare them for a job in the High-Tech industry. They were mentioned at this old post about building a portfolio, and at work we volunteer by giving them coding and testing classes).

Issi’s aim is to teach the group strategies for Exploratory Testing.
But how to do that in a single short out-of-the blue session? How do you convey the strategy, heuristics, coverage models, quality criteria in a practical easy-to-get form? How to make it dynamic and not-boring?

In this post we want to run through you this idea for an aiding tool for the instruction — and the execution too — of tests of exploratory nature. Read the rest of this entry »

Tags: , , , , , , , ,

STAREast 2010 – Bugs on a Plane!

Last month I visited the STAR East conference. It was over two weeks ago, which on the internet age makes it old news.
I follow my own clock, however, and will start posting about STAR East only now. Scott Rosenberg would call this type of blogging ‘history’ instead of ‘journalism’ (link), and it sounds just as fine.

In next blog posts, I’ll write about what happened at STAR. What happens in Orlando doesn’t stay there.
But on this one, it’s a bit about what happened in the way there.

For some time before travelling, I’ve been thinking about physical problems that resemble computer bugs. “Spirits in the material world”. For example:

In this picture you can (barely) see a water bottle that was forgotten by a worker on my neighbors’ roof. Just like a programmer would forget a temporary variable she invented to get some refreshment from the code inherent limitations, a worker will forget his own refreshment once it isn’t needed anymore.

Read the rest of this entry »

Tags: , , , , , , , , , , , ,

Should/Need Testers know how to Program (a Testing Question from Brazil)

There’s a very active Brazilian software testing discussion mailing list in Portuguese, called DFTestes.
When I say “very active”, I mean an average of 215 messages/month in 2009, and January 2010 has got more than 404 messages. Compared with other mailing list I participate, this is the most active one, and a tip of hat goes to the testers and list-admins that make the discussions interesting and vibrant.

Recently a big argument was held (link in Portuguese, requires registration) on whether “testers need to know programming” or not. I made some contributions to the discussion, and I’ll try to translate my point of view to English here.
Please note that there were a lot of people participating, and the post below does not present the entire debate or even an entire opinion, just the messages I wrote as answers. I hope you enjoy them.

The proposed question was:

Do testers need programming skills? Until a few years ago, most people would definitely answer NO… But I’ve known both people that seek testing jobs as an alternative to coding, and testers that write code routinely. So, do testers need or not to know programming?

Read the rest of this entry »

Tags: , , , , , , , , , , ,

Finding Nemo – Solving Pradeep's software testing challenge (an exploratory approach)

Back on December, Pradeep Soundararajan set a challenge up in his blog.
He built an application with the description below:

This was an interesting exercise.
What took my interest in this one?
– First of all, I liked Pradeep’s post about learning to code.
Learning to code is an important skill for a tester. I can relate to it because I am trying to learn PowerShell. I program in some languages (C#, C++, PHP, VbScript…) and have a programming background (after my Computer Systems Engineer degree, the ‘natural’ path was to code, and I worked as a programmer for many years before finding Software Testing), but PowerShell has different approach and paradigms and learning it will definitely be fun.
– Second, I had an impromptu vacation day, and I thought it would be educational to use it for a peer testing puzzle. I thought I could win this one quickly (I was mistaken, it took me long). 🙂

So I downloaded Pradeep’s application, and got to work!
Rather than just trying to solve the puzzle, I looked at it as if the mission was:
This is commercial 'roulette' style game. Stakeholders want to know it the game logic can be broken/learnt, which could mean a substantial loss of money when people start winning every time.

Testing Hint: Cem Kaner says “Testing is an empirical, technical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek“. So it is important to set straight what is the information to seek. Which kind of bugs to look for?

In this blog I describe my attemp to answer this question in an exploratory way. Read the rest of this entry »

Tags: , , , , , , , , , , ,

Yay, another Happy New Testing Year! A decade in review…

This is our fourth Happy New testing Year post, after this onethis one and this one. 🙂

So, a few hours before January is over, I’ll transpose here an answer to Testing.StackExchange about the last decade on testing:

Question: What are the most important software testing developments of the decade?

My Answer:
The question asks about the most important developments… Not the best or the worst, the beneficial or the harmful.
I’ll try to answer here with considerations by me and others I found on the net. Not everybody will agree that all these are good — even I don’t agree with all 🙂 — but my approach here is more of a reporter than a judge. Read the rest of this entry »

Tags: , , , , , , , , , , ,

Software Testing is Funny! with Demetri Martin

Demetri Martin

One best friend of mine introduced me to Mitch Hedberg and Demetri Martin, great one-liner comedians. They are/were two funny men!! Three, actually, if you count my friend which is also funny. 

After hearing the disks for over a year, not only the jokes aren’t any less funny, but I’ve started to find subliminal testing messages in them 🙂 .
I’m writing down these “insights” because I find value in them. And even if they fail to teach you something… Hey! At least the jokes are pretty funny! 🙂
So here go some favorite quotes, and their parallel in testing: 

Read the rest of this entry »

Tags: , , , , , , ,