What is Longevity Testing? How to Catch the Bugs Before the Customer Finds It

This article explains the meaning of “Longevity Testing” and how it helps to assess the stability of the System or the Product and reduce the defects found by the customer i.e. Catch the bugs in-house before the customer finds it”.

By the end of this article QA Managers, Leads and Testers will have a fair knowledge about:  

What You Will Learn:

What is Longevity Testing?

Longevity Testing is a Testing Activity:

Flow diagram of handling Customer reported issues (Fig. 1)

Background to Longevity Testing

#1) Usually, in the first few weeks of the Product deployment or after an upgrade to the latest Software release at the customer site, all things run well. However, over a period of few weeks, a customer starts reporting the issues.

#2) Many of the issues may be simple features as they are reported by the customer and are not easily reproducible in-house. They need a lot of time and careful analysis by Expert Team across the spectrum. Hint:Time=$$$!!!

#3) One or more of the following happens when customer(s) find the defect (Fig. 1)

4) The higher percentage of such issues reported by a customer(s) are related to typical System or Product stability in combination with customer topology, infrastructure, traffic, and application specific.

Why is Longevity Testing required?

1) Any ‘Defect’ that arises out of Customer reported the issue is usually a Test Escape.

2) Any such defects cost bottom-line $$$ to the Customer as well as the Engineering Organization that provides solutions and services to the customers.

3) In a normal scenario, the defect should have been noticed internally during various testing cycles including Regression Testing by one or more testers from the Testing Team depending upon the complexity of the issue.

4) Most importantly, such defects arising out of customer reported issues also point out an appropriate test scenario or a test case from being missed out at the point of Test Plan execution.

5) Many of the Testers must have experienced that a particular feature is failing at customer site but passing in-house in various Testbeds like

6) Key observations to be considered

Considering few Test gaps as mentioned above =>



Planning and Executing Longevity Tests

It is important that QA Managers, Leads, and Testers include Longevity Testing as part of their overall Test Strategy.

Planning

Understanding Customer Verticals:

Customers usually fall into one of the below broader verticals:

Activities

#1) Develop a separate Test Plan and Test Case for Longevity Testing. This will also help to track the test execution, bugs logging, and verification

#2) Identify test cases based on Test Escape Analysis inputs – usually bug scrub of EFDs or CFDs

#3) It is very important that QA team mimics test beds of one or more verticals depending on the organization’s line of business with number of verticals

#4) Dedicated Test Bed(s) should have

#5) Appropriate tools for generating Load, Stress and Real-time Traffic

#6) Identify Manual execution resource

#7) Identify Automation resource/strategy for faster and repeated execution

#8) Identify START and END of Longevity Testing for a given release

Two approaches for START and END of Longevity Testing:

I) Approach 1:

II) Approach 2:

#9) Bug verification for resolved defects

#10) Move Longevity Testing to Regression for subsequent Regression Testing

Execution

What are the Pros and Cons of Longevity testing?

Pros

Cons

Conclusion

Many of the ‘Defects’ that arise out of Customer reported issues are primarily due to Test Escape. This, in turn, begs for a lot of questions like Test Plan development, review, coverage and execution.

Externally Found Defects (EFD) or Customer Found Defects (CFD) have a business ($$$) impact for the Customer as well as for the Product Organization.

Longevity Testing being unique, should help any Product organization to improve Customer Satisfaction by the way of identifying and resolving defects before customer catches them. Longevity Testing also helps improve stability resulting in robust quality System or Product.

About the author: This article is written by STH author Vinayak. He is having 12 years of QA/testing experience in Fortune 500 companies.

Let us know if you have any questions or suggestions about this article.