Cognitive Bias in Software Testing: Why Do Testers Miss Bugs?

This article will help you find out about Cognitive Bias in Software Testing: Are you Influenced by it? Let’s get started.

The testing world is moving at a much quicker pace with technological advancements in order to ensure “quality at the speed of light”.

“Continuous Integration, Digital transformation, life-cycle automation, shifting quality to the left to minimize costs” etc. are some of the magical words that are swinging around.

While we speak about these, the underlying question –“Why and how the defect was missed” still continues to be heard and remains unanswered as well.

Cognitive Bias in Software Testing

And at times it feels as if the most obvious defects have slipped through.

But why?

Though we all might wish to think that we are very logical, structured and rational, the unfortunate fact is that we are all under the influence of Cognitive Biases which affect our thought processes in our day to day life including our professional work too.

Cognitive Bias – A Brief Description

According to Wikipedia – “A cognitive bias is a systematic pattern of deviation from norm or rationality in judgement. Individuals create their own “subjective social reality” from their perception of the input.

An individual’s construction of social reality, not the objective input, may dictate their behavior in the social world.

Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgement, illogical interpretation, or irrationality. “

Alright… That’s a good definition… But how does it impact thinking and what it means for testers in the testing world?

Well!! When the testers approach any testing, they are already influenced by their own biases – framing thoughts and judgments based on what to look for, where there would be potential defects coming up, who is developing it, the entire history of the program etc., and the list goes on.

It is very much imperative for us to know about the different types of biases so that we can be much more aware and indeed think of managing them effectively.

Types of Cognitive Biases in Software Testing to Look For

There are different Cognitive Biases that we need to look for within ourselves and some of them are explained below in detail.

#1) Resemblance Bias

It is very easy for humans to judge a situation based on the resemblance of a similar situation.

For Example, as testers, we often tend to think that web applications will have similar kind of errors while client-server applications would have a similar set of errors.

As testers, we would naturally only be looking out for similar errors only depending on the nature of the project.

Unfortunately, because of this nature, at times we tend to miss out on the most obvious ones just because our mind did not allow us to think that.

#2) Congruence Bias

This is a behaviour where our mind refuses to think of alternatives.

This means that testers tend to validate only the expected behavior and as a result, the negative validations are missed out.

When test cases are being written, we tend to cover all the requirements with their expected behaviors and miss out on the negative flows because not all the negative flows are mentioned specifically in the requirements.

They are implicit within the requirements and it is practically not possible to document all the user behaviors as well.

#3) Confirmation Bias

This is a tendency to search for and interpret the information by confirming our beliefs and hypothesis.

Typically, in the testing world, I am sure that we would have come across situations where we thought that one specific developer’s code by default would have more defects when compared to the others and hence we would have spent a lot of time testing the module developed by him.

Being under the influence of these beliefs will tend to increase the risk of missing out on defects in the modules developed by others.

#4) The Bandwagon Effect

The bandwagon effect asserts the conduct or beliefs spread among the people.

When a certain number of people believe in something, then it automatically increases the probability of the other person also believing the same. This happens many times in our day-to-day life.

A most common example is when we buy some products. Rather than independently selecting a product we usually go with the belief of others.

Exactly the same behavior is displayed in the testing world as well. In our peer group, if some people feel that one particular module is defect-free, we unknowingly tend to believe the same and the focus we give to that module during validation goes down considerably.

#5) In-Attentional Blindness

This is a behavior where testers tend to miss the most obvious defects when we are not looking for them.

Imagine a situation, where, you ask a group of people to count the number of persons coming in with a particular dress color, you could observe that people will be so engrossed in counting that they miss seeing any other important or significant thing around them.

To relate this to the testing world, in an enhancement project for example, where one of the screens was developed new then it is a natural tendency of the testers to focus on the newly developed screen more and miss out on the other critical integrations.

#6) Negativity Bias

The negativity bias is a human tendency to give more psychological weightage to bad experiences than a good ones.

Where does this come into the testing world? It is very difficult to convince the testers to provide sign-off for a build to production since their focus would only be on the defects they have uncovered.

There is no time when they can certify that there are absolutely no defects in the products. This is one of the main reasons for which the decision to Go-Live for a product depends on the Product or the business manager though recommendations would be cited by the test managers.

Conclusion

I hope you got a better idea of Cognitive bias in Software testing, its influence, and what can be done to eliminate this influence?

An important fact to realize is that we are blind towards our own biases while we can identify the biases of others (this in itself is a blind spot cognitive bias).

However, we could be more conscious and to a large extent, we can extend more thought on these wherever required.

About the Author:

This interesting post is written by STH author Geethanjali S – A Certified Scrum Master and PMP Certified having 18+ years of experience in Quality Assurance and Engineering. A Transformational leader with rich experience in leading and delivering QA transformations, Global rollout programs, Mergers & Integration related programs.

Do you remember being influenced by Cognitive Bias? Have you noticed Cognitive bias in your peer group? Feel free to express your thoughts in the comments section below!! We would love to hear from you.