How to Find Maximum Valid Defects in Any Application?

Advice from a person passionate about testing:

Have you ever been in a situation where you’re out of college and hired by an organization for the role of a Quality Assurance Analyst or in general terms as a “Software Tester” instead of a developer?

Have you also been advised time and again by a bunch of people called “seniors” to opt for a project that will provide you Software Development experience instead of testing?

Have you been told that it’s okay to be a tester in the initial phase of your career, but later you must move to a development role or that your growth will be stunted if you stick to the testing role?

 Or have you been told that there is very little skill involved with testing?

If you find yourself nodding for any of the above questions, brace you! Because here’s an advice from a person passionate about testing and intending to remain in the same field for a very long time!

What is this article about?

This article not only showcases why testing is equally or more important than the development, but also that discovering defects or bugs requires vision, fore-thought, talent and a tremendous amount of skill.

I have also tried to express a few suggestions/ tricks to be able to dab on the software’s weakness and expose defects.

See Also => More tips to find bugs in an application

What is Testing?

If we go by a technical definition, then testing is a detailed investigation process whereby a determination is made whether a piece of code or a product that is intended for customer use is meeting the specifications it’s required to.

Simply put, testing can be broadly defined as a practice to figure out if “something is not right” or “something can be even better”.

Needless to say, this process must be carried out with the motive of discovering errors or rather, with the aim of ensuring that the product or feature is meeting / exceeding the expectations.

Importance of Testing

When you closely think of it – isn’t testing deeply rooted in our daily lives? Let’s take a very common example of shopping. When you as a consumer go to an apparel store, you have a set of ideas in mind before purchasing apparel.

The first and foremost thing any consumer does is to try on the outfit and check if any alterations are required in size. Doesn’t this activity in some way comprise of testing or validation and isn’t it important?

Likewise, in software – Bugs or errors will inevitably exist in any feature or product. No matter how perfectly a feature or program is coded, there is always something amiss. This probably has nothing to do with the capabilities of the developers, but most often because the piece of software turns out to be rather complex and grows beyond the control of the developers to handle this complexity.

Have you also given a thought of how many implementation/ design defects can arise out of sheer complexity?

Interestingly even 50% of development time is spent on testing to make sure the code is working as per design. Secondly, the cost of fixing a bug or an error is much lower in an early stage of the SDLC, than at the later stage.

Another key aspect to consider is that – In a market of constantly emerging technologies, with organizations developing several competitive products and pitting them against each other, what kind of product do you think a customer or consumer would prefer?

Isn’t it safe to say that a customer will want a reliable, stable, easy to use product? Here is where we as testers step in and play an invaluable role in the quality of the software or the product.

Thirdly, for any high-level product demos either to the stakeholders or the customers, the particular technical representatives firstly approach the test team for their help and rely heavily on their reviews to create customer demos. It’s easy to see the technical bandwidth involved in being a tester.

Also while testing, have you ever asked yourself this simple question – why are we doing something this way and not the other way? If so, then you are in a good place to even suggest enhancements, wish-lists and in some cases, kick start a Proof of Concept!

What Is A Defect?

A Defect can, therefore, be described as a program or feature failing to meet it’s expected results or an unsatisfactory condition that prevents the user goal from being achieved.

With the discussion above, as testers or as I would prefer to put it – as “test developers” the primary goal for a tester is to expose as many defects as possible as early as possible. This requires not only a keen eye but a bent of mind that can drive down to the core of the system and determine which cases can cause a failure.

While developers have their knowledge limited to only the feature or program they develop, as test developers we are those fortunate few who have the opportunity to gain complete end to end knowledge of the software/product and in some cases, we have the platform to even drill down to the architecture level to expose potential problems.

So who do you think has more skill and knowledge?

Best Practices To Maximize The Chances Of finding Valid Bugs

While each individual has their own distinctive style for testing, here below, I am attempting to pass on some suggestions and tricks to ensure good test coverage and maximum cum valid defects finding through my experience.

#1) Research And Self-study

This mainly entails gaining in-depth knowledge of the feature being implemented.

• As a test representative, the most common remark that is made by the development team in most forums is that the testers do not have a complete understanding of the feature, due to which they find it challenging to determine what is a program defect and what is not.
• As a tester, even if you are not directly involved in technical HLD review meetings, it’s very important to reach out to the concerned developers and request them to provide a copy of the HLD, Use case documents or any similar technical input documentation which will serve as a good knowledge base to understand the feature deeply.
• Another excellent practice is to have meetings with the developers and discuss the technical aspects of the feature being implemented. Communication and more of it serves an invaluable medium to gain in-depth knowledge, a good foothold on the on not only the product/software that you’re part of but also consumers and cross products.
• Also, understand how a customer would exploit that functionality. Remember you are an interface between your organization and the customers using the software, so you bear a huge responsibility for understanding the product not only from the technical perspective but also from the user perspective.
• If the feature is an implementation of any existing technologies that are being used, make sure you understand those underlying technologies.

For Example, I had a similar experience when we were providing support for a networking concept which was an IEEE standard. I went to the IEEE website and searched for various articles related to it to gain a thorough understanding of the concept.

This helped me catch defects that were completely out of the specified use cases by the development team but very valid in terms of the actual functionality specified by IEEE in a real-time scenario.

Of course, it’s obvious that this helped me expose a maximum number of defects than any other team member.

#2) Planning Test Scenarios

Needless to mention, the research and knowledge part of it is inevitable in any phase of testing and bug finding. After the preliminary stage of self-education, a good amount of thought must be invested in planning the test scenarios.

• It is an obvious fact – test case planning is a very important phase in the testing life-cycle. The test cases you design and plan are a direct report of the quality of the product and the extent of testing performed. The test cases outlined have an exponentially proportional relationship to the stability of the software that will go out to the customers.
• After having read a use case document or any technical functional input document, spend a lot of thought in designing the test cases. There must be a good distribution of happy path scenarios and error path scenarios. Also, record some effort on the alternate path scenarios and creative/ad-hoc/out of box scenarios.
• Be watchful while describing scenarios and ensure that they are valid. Again have multiple reviews conducted with the development team, architects and ensure that you have their review comments incorporated in your test cases. This will ensure that no time is wasted on creating or enhancing test cases which are invalid.

#3) Investigating Setup Readiness

At most times in the testing cycle, the test team has some amount of time on hands before the actual testing commences.

• When there are times like this, it’s a good practice to evaluate the infrastructure that will be used for testing. Testers find themselves in situations where many of their defects will be setup related and are rejected as there is no fix that can be delivered from the code point of view.
• Testers often find it frustrating in the testing cycle to find most of their defects getting rejected due to setup issues even if the test goal is hindered in actuality. Neither there is any test progress, nor there are valid defects to indicate why there is a hindrance. Some other times it so happens that a particular test gives a particular erroneous result the first time, but gives a different result in every other run. This makes the defect replication extremely challenging. Most developers do not have the patience and inclination to figure out something going on the wrong side of the feature they implement. Make sure to reach out to get the required help to stabilize the infrastructure as early as possible.

#4) Testing Techniques To Find Defects

Getting to the most interesting part of our job– Testing and in turn finding defects! We are those fortunate people who make developers improvise their code quality to a large extent.

• Discovering the weakness: While you test, it’s very important to have the bent of mind to drill down to the deepest level and make an attempt to bring out the point where the code might “break”.
• Happy Path testing: The first thing to do would be to keep an eye on the number of days allotted to complete testing for a piece of code. In those days, keep aside a reasonable portion of time for happy path scenarios and start with them first. It is likely that you may not find many bugs here because all developers make sure the unit tests the happy paths effectively.
• Error Path testing: Next move on to the error path scenarios. Make sure you spend a good chunk of time for testing the error path scenarios. This kind of testing is often the most fun part is you can get as creative as you like! You’re most like to find issues here.
• Ad-hoc/Creative testing:  Although very commonly this segment of testing can be classified under the error path testing, it’s good to dedicate some last few days for this activity. I have found several valid defects with ad-hoc testing and negative testing.
• Communication with the developers before filing a defect: Although as testers we are not entitled to touch base with the developers and can open defects in case we find an issue, I make a strong recommendation to all my juniors to definitely communicate on the issue observed.

There are 2 main reasons for this:

a) Most developers would not want to have defects against their name due to tight schedules and may reject the defect asking for more information, just to buy some time.

In certain other cases, they may genuinely not understand the steps to recreate a scenario that the tester has followed and may return the defect for replication, again burdening the tester.

b) The second reason is that most often we work with teams that are globally distributed and the fix may need to come from another developer sitting in another part of the world. So if the problem is clear to a local developer that you initially filed the defect against, he will make it a point to add his findings to the defect and re-direct to the person who has to fix it.

This avoids time delays in generating a fix for the defect as the developers in a different time zone will already have some minimal information he needs to work on a fix, again preventing several exchanges between the tester and developer and reducing the age of the defect.

Logs: It is not only important for a tester to provide clear steps for a problem replication, but also very important for a tester himself keep an eye on the logs that are generated while running a test. Providing proper logs in a defect will help to fix it faster. It’s an absolute must for a tester to get acquainted with the log information.

Again – communicate with the developers to understand what is going on. In my experience, I have always seen that the developers encouraging testers to understand the code flow and logging. Once you’re well acquainted with the logs, you can immediately detect where a test fails by its stack trace. Sometimes you can also easily detect where to catch more defects!

Here is another example of why getting familiar with the logs is important. During our testing cycle many times we have observed that a particular test flow wise, the feature or software behave as per the specifications.

However, a field issue was caught at the customer end that during the entire scenario flow, the customer observed a series of null pointer exceptions in the logs! This came back as a critical test escape because this is certainly unacceptable! People had to be immediately deployed to develop scripts or tools that would give a direct report of the Null Pointer Exceptions.

Keep an eye on what defects are getting filed: Not to be snoopy, but always keep a watch on the defects that are getting filed to avoid duplication and in turn rejection. This will reduce the number of duplicate defects drastically.

Run logs: This is a practice that may appear as an overhead to the tester, but in my view, it is helpful in the long run. Run logs are nothing but like a diary keeping an account of the day’s work and progress.

Run logs can contain all the information about test scenarios run, their behavior under certain conditions,  the expected output, the actual output, all the flows followed by the tester, defect information of any particular scenario.

Keep a neat, understandable record of all of this and you’ll never find difficulty in reporting status to the management or even, remember if you have run a particular scenario or not, a year later. It’s all there, in print!

Conclusion

The above pointers can greatly influence testers new to the industry to develop the mindset that is needed for ensuring that the quality of the product is measurably high in standard.

Personally, the above practices have helped me immensely in being one of the most sorted out testers in the unit I am part of. Also remember, the majority of Subject Matter Experts in any organization are mainly testers.

So enjoy and happy bug finding!

About the Author: This is a guest post by Sneha Nadig. She is having 7+ years of Software Testing experience and currently working as a test lead in a top MNC.

Do let us know in the comments below if you find these tips useful. Expecting more tips/suggestions from our expert readers.