What is Endurance Testing in Software Testing (Examples)

Introduction to Software Endurance Testing:

In this article, we will discuss and explore Endurance software testing in detail.

Initially, to start with, let’s try to understand what Endurance testing is exactly all about, and then we will explore all the details about it. I’m sure that this article will be a perfect guide for those who are new to the Endurance Test Concept.

This article will give you a complete overview of Endurance Testing, including its meaning, need, duration, benefits, challenges, and steps involved along with examples in simple terms to enable you to understand the concept perfectly.

What is Endurance Testing?

The word Endurance means ‘the capacity of something to last or to withstand the wear and tear.’ You can simply call it as durability, lasting power, or longevity.

When it comes to the world of software, an Endurance test (soak testing longevity testing, or capacity testing) is a type of non-functional testing that is done to check if the software system can sustain under a huge expected load continued over a long period.

It is one of the performance testing types and a sub-type of load testing.

Why Endurance Testing?

The main purpose of performing this testing is to identify any potential memory leaks. So, during this testing, memory utilization is closely monitored. Hope you are aware of the concept of memory leaks. Still, let me briefly what a memory leak is to those who may be new to this term.

A memory leak is a failure in a software program to release discarded memory, causing impaired performance or failure.

A memory leak might not have a short-term impact but in the long-term, it slows down the system which leaves no free memory and thus finally leads to application or system crash. A memory leak is an issue that becomes apparent only after a certain time.

Hence, we need endurance testing to uncover such issues.

However, it might be possible that with a huge significant load, your application works well for some period say, 1 hour. But, when exposed to the same amount of load continuously for a longer period say, 3-4 hours, your application crashes down due to resourcing issues and insufficient disk space.

Another important issue that endurance software testing targets to identify is its performance degradation. It needs to be ensured that the throughput or response times after prolonged use are equivalent to or improved than at the beginning of the test.

Database connection issues are also identified in the Endurance Test. If the database connection is not closed successfully then it might result in a system crash.

Endurance test also verifies if there is a proper connection closure between the layers of the system which would otherwise cause certain modules of the system to hang.

In a nutshell, we can say that the endurance test aims to determine how the system behaves for a long period of sustained use. The major issues that are identified as a part of Endurance testing are memory leaks, performance degradation, database connection issues, etc.

Duration of an Endurance Test

If it’s a 24*7 nonstop operation project, then ideally an Endurance Test should run for 72 hours. The reason is that the application should be capable enough to get run unattended on weekends (as weekend support is costly for organizations).

So, we need to test the system exposed to a significant load for a period slightly larger than a weekend. However, as running the test for 72 hours could be too cumbersome, generally, we run the endurance test for 12 hours.

Moreover, there is no hard and fast rule for the duration of a soak test. It majorly depends on factors like clients’ involvement, actual production use, etc. At times, the endurance tests also do run for a year!

Benefits of Endurance Testing

The benefits derived from this testing are listed below:

• It ensures the fitness of an application.
• It uncovers those bugs which otherwise cannot be found by any other performance test. For Example, you might not find the memory leak issue during volume testing or stress testing.
• Makes the application robust: It identifies the performance degradation issues that could occur under huge continuous load and then fixing those issues makes the application more robust.
• It tells about the behavior of the system under a load in the long run. In other words, it checks the system’s sustainability over time.
• The endurance test result data can be used for the customer to validate or enhance their infrastructure needs.

Challenges in Endurance Testing

The challenges involved in this testing include:

• It is a very time-consuming one. So, it is difficult to carry out Endurance testing on a project which has very stringent timelines.
• Can’t be done manually. It requires an automation tool and an expert knowing that tool.
• It is often hard to determine how much load is worth applying.
• If the test environment is not isolated properly from the live production environment, then the application or network failures in the endurance test can hamper the entire working system and result in permanent data loss/data corruption.
• Unhandled exceptions are observed by the customer.

An Example

An example where Endurance testing is required and can be used is the Banking application.

On the closing days of the bank, the application is tested to know if the system can sustain under continuous expected load or a large number of transactions for long.

This is a perfect example of an Endurance test.

Steps to Perform this Test

Below is the approach for performing this testing:

#1) Establish the Test Environment:

This involves figuring out which hardware, software, database (along with its size as it grows with the duration of the test), and OS is required to perform the Endurance test.

It also involves creating a team for performing the endurance testing and assigning the roles & responsibilities within the team. The test environment should be ready before the execution of the test and it should be isolated properly from the actual live system.

#2) Creation of Test Plan and Test Scenarios:

The test cases should be designed, reviewed and finalized. The test execution strategy should also be built up in this step.

The breakpoint of the application should be determined, and it should be finalized how much load would be put on the application during an Endurance test.

#3) Test cycle Estimation:

It involves analyzing what would be the duration of each test phase and how many test cycles are required.

#4) Risk Analysis:

This is a very important step in this testing. The test cases are prioritized based on the risk factor.

Risks and issues that a tester might come across during the Endurance test are mentioned below:

• Is the performance test going to remain consistent with time?
• Are there any other minor issues which are not yet been deducted?
• Is there any external interference that has not yet addressed?

#5) Test Schedule:

Determine the budget, deliverables, and timeframes.

#6) Test Execution:

This involves finally kicking off the endurance test.

#7) Test Cycle Closure:

Close the test cycle based on the exit criteria set up in the test planning phase. It could be based on the defects found, the duration of the test, etc.

Endurance Testing Tools

There are multiple tools available in the market.

Enlisted below are some of the most popular and useful Endurance Test tools:

• Apache JMeter
• LoadRunner
• LoadStorm
• LoadUI
• Appvance
• OpenSTA
• WebLOAD
• Rational Performance Tester

Conclusion

In this article, we have learned about what is Endurance Testing. How does it need to be done? Its advantages and disadvantages, and the various tools available.

Some of the key takeaways are:

• Endurance testing (soak testing longevity testing or capacity testing) is a type of non-functional testing that is done to check if the software system can sustain under a huge expected load continued over a long period of time.
• It is a subset of Load Testing.
• The major issues which are identified as a part of this testing are memory leaks, performance degradation, database connection issues, etc.
• The duration of the Endurance test depends upon the business, project, and client requirements. It may last for 5-10 hours a few days or a month or sometimes even a year.
• It makes the application more robust and prepares it to sustain under continuous heavy loads.
• As it is very time-consuming, it should be avoided to do the Endurance test manually. It is mostly automated.
• It should start with establishing an isolated test environment, then creating test plans, estimate test cycles duration, analyze the risk, prepare the test schedule, execute the endurance test and then finally close the test cycle.
• Some of the great tools for Endurance tests are Apache Jmeter, LoadStorm, LoadRunner, LoadUI, Appvance, OpenSTA, WebLoad, and IBM Rational Performance Tester.

Further Reading:

• Top 15 Performance Testing Tools of 2018: Load Testing Tools List
• STH Detailed article on Performance Testing – With Examples
• 4 Endurance test tips you need to know when performing this test through Apache JMeter