Exploratory Testing – How to Think Beyond Traditional Testing Boundaries?

“I really appreciate your concern for the product and making us helpful in an understanding end-user perspective. It’s going to be very helpful. Thanks for the good work and keep it up!!!”

This was the last e-mail of an email chain with 21 emails from our client. It was midnight, and our product release was delayed due to a critical bug we found. You might think, what is new in that? It may happen many times. But, this was really different as the critical bug we reported was not a result of any documented test case.

After completing regression testing for the last time that evening, I was just playing with the product.  What does that mean? You are free to do what you are not supposed to do.  Based on my experience and project knowledge, I had some ideas on how to test the product apart from our typical test repository, called Exploratory Testing.  The exploratory testing done found a critical bug related to a server hang issue while doing something unexpected.

Being a fan of exploratory testing, I love to explore the product in different ways. For me, the definition of software is:

“It should do what it is supposed to do, and it should not do what it is not supposed to do.”

Limiting testing boundaries to check whether products that are supposed to work are working makes you an incomplete tester. In fact, a tester’s life starts when documented regression testing ends and results are updated. Looking at products from different perspectives and understanding end-user requirements in different scenarios make a big difference. So today, let’s understand together, how that difference can be made:

What You Will Learn:

How to look product from different perspectives?

#1. Understand customer/end user

Software testing is all about checking the quality of the product in terms of customer satisfaction. How do you know a customer’s viewpoint? The answer is simple – you have to be the customer. OK, let me make a correction. Being customer will not be enough. You need to understand how the customer wants to handle the product. No two customers who bought same raw materials will prepare the same recipe. Yes, the product we develop/deliver is a raw material for customer’s businesses and they have a different mindset while using it.

As a software tester, we need to check the purpose of the product and not the object or aspect of it.

Let me give you some real-life practical examples:

Understanding stakeholders and a wide range of their expectations should be the baseline of exploratory testing.

#2. A mindset

While looking for (let’s say) a job employment ad, do you see that jackpot and in between the pages with the bold font? Most of us do not (believe me, it’s true). Because we have instructed our mind to look for what is useful or to be checked. Anything else is of no use, so the mind denies us of recognizing it.

Open your mind, and do not set any expectations when you start exploring a product. Always remember, it’s not OK if the product is doing what it is supposed to do. It is also important that it should not do what it is not supposed to do.

I remember one classic example:

In Linux, ‘cat’ command is used to check the content of a file and the ‘ls’ command is to check the content of the directory. Working with Linux and being in software testing for five years, I never thought to do cat <dir name> because my mind was set; if I needed dir content, I need to use ‘ls’. That worked, but the reverse side of the expectation is that the product was not supposed to behave the way it was not supposed to, was wrong. One of our customers, who did not know Linux well, did cat <dir name> by mistake and the system crashed. We paid for this mindset.

Always be ready to make mistakes with the software because that is what end user is going to do. To test the software, you have been trained, but the end user will not be as trained as you or he/she will not be as much of a technical expert as you. Also, he/she will do anything with the software when they are in trouble. Think about those scenarios, and provide testing feedback. Life of the software and yours (as a tester) will rock.

#3. Know the competitors

While testing any software application for your client, did you ever attempt to know and understand other software with the same purpose? Did you ever suggest some useful functionality you observed in a competitor’s product? It does not fall in our job description, is the typical answer. But do you know the benefit of doing it?

Here are few real-life examples to make you understand the point:

Everyone wants to have something extra for what they pay for. Our analysis of competitive software can work the same way for us. The customer always likes to hear valuable suggestions – mainly comparative suggestions to make the product more useful or marketable.

Also, this kind of comparison and analysis of the same range of products makes our analysis more powerful and eventually we create a treasure to which we can go back at any moment and find something useful.

Over to you

Ending this post here, knowing it is incomplete.  Do you want to add something or want to wait for the continued part in the next post? I welcome your suggestion and inputs to make the software testing community make more effective and productive.

Thank you for reading and sharing appreciation and views. We are overwhelmed with the positive responses from the readers.

About the Author: This is a guest post by Bhumika Mehta. She is a project lead, carrying 7 years of experience in software testing. She appreciates good ideas, innovations and risks too. And of course, she hates monotonic work, people and environment.