Test Coverage – What You Should Really Test?

Scene 8 years ago: when I was having 2 years of experience in software testing industry I was thinking that it was alright if I don’t know some fundamentals about software testing as something one can learn with experience and me too will do.

I was right – and wrong as well.

Right because with experience, you learn to see bigger picture, you understand real meaning of “Critical Situation” and you understand the end user more. Wrong because none of the mentioned things require experience, but early learning, which I understood very late. That is the encouraging factor to write this post.

test coverage

Here is my experience from one of the interviews at that time:

How do you make sure that test coverage is complete or maximum? I was asked.

Ummmm……I make sure that I test every functionality, I said.

So you are saying that once you test all the functionalities, you feel that you have covered maximum of application/product, in terms of testing, the interviewer backfired.

Ummm…well….ummm….yes, because when you test all the functionalities, you are confident about application/product’s behaviour, I supported my answer.

I agree, but my question is – will it give you confidence that your test coverage is maximum or complete? the interviewer was in no mood to let me go.

Now, I was growing impatient, unknown about the fact that I was going to learn one of the most important topic about software testing – Test coverage”.

Rather than reproducing the excerpts of interview, I am summarizing here, what I learned about Testing Coverage, on that day. But before that let’s be clear on one point – test coverage does not ever mean to know whether you are testing enough or not, neither it means you are testing perfectly or not. Having said that, presenting my understanding about test coverage.

Also read => How to improve code coverage.

Test coverage can have different meaning in different context. Let’s discover those contexts one-by-one:

Product coverage – What aspects of product did you look at?

Yes, when test coverage is being measured in terms of product, the main area to focus on is – which areas of product you have tested and which remains untested?

Example #1: If “knife” is a product, you are testing; just do not concentrate on checking whether it cuts the vegetables/fruits properly. There are other aspects to look for such as – user should be able to handle it comfortably.

Example #2: If “notepad” is an application, you are testing, checking relevant features is a must thing. But other aspects to be taken care are – application responds properly while using other applications simultaneously, application does not crash when user tries to do something unusual, user is provided proper warning/error messages, user is able to understand and use the application easily, help content is available when required.


If you don’t look into the mentioned scenarios, you can’t claim that the test coverage for the application is complete.

Risk coverage – What risks have you tested for?

Risk coverage is an another aspect to have complete test coverage. You can’t tag the product or application as “tested”, until you test the associated risks too. Coverage of associated risks is an important factor of overall test coverage.

Example #1: While testing an airplane, if a tester tells you that he/she has fully tested the internal system of airplane and it’s working fine but only flying capability of airplane was not covered while testing – what would be your reaction? Well, that is what risk coverage means. Identifying risk as per the application/product and testing it thoroughly is always a good practice.

Example #2: While testing an e-commerce site, tester considered every effective factor but did not consider the risk of numbers of users accessing the website simultaneously and on the Super OFFER day, the not considered risk proved to be a huge mistake.

Also read => Risk based testing – How to Manage Risks During Test Planning Phase and risk management at test execution phase.

Requirements coverage – What requirements have you tested for?

If a product or application is developed very well but if it’s not matching with customer’s requirements then it’s of no use. Requirement coverage while testing is as important as product coverage.

Example #1: Excited for the family function, you asked tailor to stitch your dress and put on those peacock blue show buttons on neck line. While stitching the dress, tailor thought that putting those buttons on neck line will not look good so he stitched a golden colored border instead. On the trial day, definitely, the unhappy customer shouted at tailor for not sticking to the requirements.

Example #2 : While testing a chat application, tester took care of all the important points like multiple users chatting in a group, two users chatting independently, all types of emoticons available, updates sent to user immediately etc. but forgot to look into requirement document, which clearly mentioned that when two users chat independently, video call option should be enabled. The client marketed the chat application claiming that it would allow calling, while two users chat independently. You can imagine what would have happen to the chat application.

Read also => How to Create Requirements Traceability Matrix

In conclusion

Test coverage does not end with the mentioned contexts. There are many other points that should be considered when it comes to test coverage but let’s keep that discussion for another day.

About author: This article is written by STH team member Bhumika M. She is a project lead, carrying 10+ years of software testing experience.

Pour in your comments and views about the post as well as test coverage. As usual, we love to hear from you.




Recommended reading

18 comments ↓

#1 Umesh patil

very good examples to explain the points

#2 Atul Sharma

This blog is really owsm. It shows that how should a tester think over the application.

#3 JIJI

Hi,

am new in software testing. I began testing in MTM (Micro Soft Test Manager). So am only familiar with that. Is any other tools better than this? If I am joining to another company as the testing resource , is that easy for me with the knowledge in MTM? or it will be totally different? Currently am doing manual testing.

please review and advice.

#4 sindhura

Very nice article…

#5 Neha Mutalik

Nicely Written… Great Read…

#6 RushiSK

Worth article on Test Coverage. Very well explained with example.

#7 Bhumika Mehta

All Readers,

Thanks for your continuous readership and glad that you liked the post and it was informative.

Thanks,
Bhumika

#8 Anand Awate

Superb article..

#9 Russel

As a newcomer in software testing, this is a really big help. Thanks!

#10 Lalitha

Thank You,
For giving such a helpful article for beginners in testing.

#11 swapnajit

It was informative and descriptive. Thanks for your valuable effort.

#12 monis

As,iam studying in Bsc, I thougt of giving my talk on this particular topic please help me out …

#13 Nagarjuna

Could you please gimme a one word answer to specify Test Coverage ??

#14 Katrine

So funny)
I have got almost the same years experience (3 years) as you mention (2 years) in the beginning and have got the same question on the interview about coverage) So sorry that I don`t read this article some days before.

#15 mary

As a beginner , i got a good example to improve on manual testing

Thanks

#16 praveen

Bhoomika……..well said about test coverage,thank you

my question is what is the exact difference between smoke and sanity testing with example

can plz share ur answer with us

thanks

#17 london

HI, I WANT TO SWITCH CAREER TO SOFTWARE TESTING AND I AM FROM FINANCE BACKGROUND. CAN ANY ONE GUIDE ME WHERE I CAN GET GOOD FREE ONLINE TUTORIALS FOR SOFTWARE TESTING. AS MENTIONED ABOVE VERY HARD TO FIND THE RIGHT INSTITUTE SO PLANNING TO GIVE IT A TRY MYSELF FIRST.

I AM TAKING HELP FROM GURU 99 FOR SOFTWARE TESTING BUT I GUESS IT IS JUST FOR THE BEGINERS/FRESHERS ONLY.

THANKS IN ADVANCE

#18 Patrick

It is very useful, thanks a lot.
For the Risk coverage, it’s seem that we should cover/define high priority tezt cases regarding main function, as the example ‘airplane’.
So can we call risk coverage as main function of application coverage?

Leave a Comment