Cognitive Bias in Software Testing: Why Do Testers Miss Bugs?

Cognitive Bias in Software Testing: Have You been Influenced?

The testing world is moving at a very quicker pace with technological advancements in order to ensure “quality at the speed of light”.

“Continuous Integration, Digital transformation, life-cycle automation, shifting quality to the left to minimize costs” etc are some of the magical words that are swinging around. While we speak about these, the underlying question –“Why and how the defect was missed” still continues to be heard and remains unanswered as well. 

Cognitive Bias in Software Testing

And at time it feels as if the most obvious defects have slipped through.

But Why?

Though we all might wish to think that we are very logical, structured and rational, the sad fact is that we are all under the influence of Cognitive Biases which influence our thought processes in our day to day life including our professional work too.

Cognitive Bias – A Brief Description

As per Wikipedia – “A cognitive bias is a systematic pattern of deviation from norm or rationality in judgment. Individuals create their own “subjective social reality” from their perception of the input.

An individual’s construction of social reality, not the objective input, may dictate their behavior in the social world. Thus, cognitive biases may sometimes lead to perceptual distortion, inaccurate judgment, illogical interpretation, or  irrationality. “

Alright… That’s for a good definition… But how does it impact thinking and what it means for the testers in the testing world?

Well !! When the testers approach any testing, they are already influenced by their own biases – framing thoughts and judgments based on what to look for, where there would be potential defects coming up, who is developing it, the entire history of the program etc., and the list goes on.

It is very much imperative for us to know about the different types of biases so that we can be much more aware and indeed think of managing them effectively.

Types of Cognitive Biases in Software Testing to Look For

There are different Cognitive Biases that we need to look for within ourselves and some of them are explained below in detail.

Cognitive Bias

#1) Resemblance Bias

It is very easy for humans to judge a situation based on the resemblance of a similar situation.

For Example, as testers, we often tend to think that web applications will have similar kind of errors while client-server applications would have a similar set of errors.

We, as testers, naturally would be looking out only for those similar errors depending on the nature of the project. Unfortunately, because of this nature, at times we tend to miss out on the most obvious ones just because our mind did not allow us to think that.

#2) Congruence Bias

This is the behavior where our mind refuses to think of alternatives.

It means, testers tend to validate only the expected behavior and as a result, the negative validations are missed out.

When test cases are being written, we tend to cover all the requirements with their expected behaviors and miss out on the negative flows because not all the negative flows are mentioned specifically in the requirements.

They are implicit within the requirement and it is practically not possible to document all the user behaviors as well.

#3) Confirmation Bias

This is a tendency to search for and interpret the information by confirming our beliefs and hypothesis.

Typically, in the testing world, am sure that we would have situations where we thought that one specific developer’s code by default would have more defects when compared to the others and hence we would have spent a lot of time on testing the module developed by him.

Being under the influence of these beliefs will tend to increase the risk of missing out on defects in the modules developed by others.

#4) The Bandwagon Effect

The bandwagon effect asserts the conduct or beliefs spread among the people.

When a certain number of people believe in something, then it automatically increases the probability of the other person also believing the same. This happens many times in our day-to-day life.

A most common example is when we buy some products. Rather than independently selecting a product we usually go with the belief of others.

Exactly the same behavior is displayed in the testing world as well. In our peer group, if some people feel that one particular module is defect-free, we unknowingly tend to believe the same and the focus we give to that module during validation goes down considerably.

#5) In-Attentional Blindness

This is a behavior where testers, tend to miss the most obvious defects when we are not looking for them.

Imagine a situation, where, you ask a group of people to count the number of persons coming in with a particular dress color, you could observe that people will be so engrossed in counting that they miss seeing any other important or significant thing around them.

To relate this to the testing world, in an enhancement project for example, where one of the screens was developed new then it is a natural tendency of the testers to focus on the newly developed screen more and miss out on the other critical integrations.

#6) Negativity Bias

The negativity bias is a human tendency to give more psychological weightage to bad experiences than a good ones.

Where does this come into the testing world? It is very difficult to convince the testers to provide sign-off for a build to production since their focus would only be on the defects they have uncovered.

There is no time when they can certify that there are absolutely no defects in the products. This is one of the main reasons for which the decision to Go-Live for a product depends on the Product or the business manager though recommendations would be cited by the test managers.

Conclusion

Hope you would have got a better idea of Cognitive bias in Software testing, its influences, and what can be done to eliminate the influence?

And an important fact to realize is that we are blind towards our own biases while we can identify the biases of others (this in itself is a blind spot cognitive bias). However, we could be more conscious and to a large extent, we can give extend more thought on these wherever required.

Do you remember being influenced by Cognitive Bias? Have you noticed Cognitive bias in your peer group? Feel free to express your thoughts in the comments section below!!

About the Author:

This interesting post is written by STH author Geethanjali S – A Certified Scrum Master and PMP Certified having 18+ years of experience in Quality Assurance and Engineering. A Transformational leader having rich experience in leading and delivering QA transformations, Global rollout programs, Mergers & Integration related programs.

Recommended Reading

19 thoughts on “Cognitive Bias in Software Testing: Why Do Testers Miss Bugs?”

  1. Thank you for explaining each bias points. Completely new topic.
    Further relating and detailing each points with testing world is very helpful.

    Individual may need to learn about these biases and practice to come out of.

    Reply
  2. Very informative! Comparing the testing concepts with the real world scenarios is nice.

    Reply
  3. I always suggest myself and my team not to get influenced by any while performing the testing, looks like the proper term I should be using is Cognitive Bias. Thanks for introducing this topic, detailing out with examples, will share this with team and share the feedback.

    Reply
  4. Nice One! Keep on sharing such an interesting and useful information.

    Reply
  5. A great article, defining an existing problem, is half the solution. I would be interested in a future article on the purpose of tests cases and the construct of those test cases.

    Reply
  6. Thanks for sharing such essential knowledge of Cognitive Bias in Software testing. This article is very extraordinary. I recommend SmartMentors.net for Quality Testing training.

    Reply
  7. Great article. There are many concepts from behavioral economics that are useful for software testing and engineering.

    Reply
  8. Hey thanks for sharing the information about software courses. it’s very useful for us , i would like some facilitate for my web site,
    please check once

    Reply
  9. A really informative article. Recently, we had a session as a part of the software testing consulting services session by the digital group, where we were told about how congruence bias prevents us from experimenting with new ideas. It was a session to help us switch to automation with ease.

    Reply
  10. New Topic and Useful Information. Good Write.

    Reply
  11. Thanks for the informative and helpful post, obviously in your blog everything is good.

    Reply
  12. Wonderful article! We will be linking to this great post on our website. Keep up the good writing. Visit us for the best custom tent, Custom pop-up tent, custom tent canopy, and display.

    Reply
  13. This is an excellent article! It was a pleasure to read this article. Thank you for making this information available to us. It’s been an incredible year! Thank you for making our lives a little easier. Beaver Architect

    Reply
  14. Thanks for the informative and helpful post, obviously in your blog everything is good.

    Reply
  15. Each of our high-quality custom printed canopy tents is printed by a prestigious brand, making them ideal for both outdoor and indoor events, as well as highlighting competition events with a promotional design. This product is great for your business since it is simple to set up, carry ahead, and looks beautiful.

    Reply
  16. Though we all might wish to think that we are very logical, structured and rational, the sad fact is that we are all under the influence of Cognitive Biases which influence our thought processes in our day to day life including our professional work too.

    Reply
  17. Product is very high quality and the colors are sharp and superb- very happy with the purchase

    Reply

Leave a Comment