This Comprehensive Guide on Soak Testing Explains What is Soak Testing, Why do we need it, its Application, Advantages, Best Practices and Disadvantages:
Various types of testing need to be performed while testing a software application. Functional and Non-functional testing are the two broad categories into which we can categorize the testing types.
Functional testing, as the name itself suggests, relates to testing the functionality of the application. Non-functional testing, on the other hand, covers all the other testings (usability, performance, etc.), other than Functional testing.
What You Will Learn:
Soak Testing – A Complete Guide
This tutorial introduces you to the concepts of Soak testing which is a type of Performance testing.
As seen in the image above, we can say Soak testing is a type of Non- functional testing.
What Is Soak Testing
It is a type of performance testing to check if an Application Under Test (AUT) can withstand continuous load for a pre-determined time frame. This is a non-functional type of testing. It is also termed as ‘Endurance Testing’ or ‘Longevity Testing’.
If you go by its literal name, then the word ‘soak’ in itself holds the meaning of what this testing intends to do. Thus, subjecting an application for a specific period to a high load is what this testing is all about.
One may wonder, what could be the difference if an application is subjected to load for an hour or maybe 20 hours. But yes, it does hold significance.
This can be explained better with a real-world scenario. If a rope is pulled from both the ends by two people for some time, then it might just resist the pressure, however, if the same is continued for days, then the rope might just break by giving in to the pressure from either end.
So is the case with the software. When we subject an application to a high load (a few hundred or thousand users) then it may just work fine for an hour. However, when the same application is subjected to the load for 20 hours, it may collapse altogether.
Continuous heavy traffic for a long duration can cause varying issues in the application. Thus, there arises the need for Soak Testing.
In this testing, the basic concept is to load the application with expected users but for an extended period. This helps in identifying the various underlying issues that would otherwise go undetected until the actual scenario occurs in the live application.
Need For Soak Testing
To understand the need we must also be aware of the possible issues that an application might go through in case it encounters heavy load for a long duration.
Let us go through the various reasons that make Soak testing necessary.
#1) It is mainly required to identify issues like incorrect memory management, database connection issues, degrading application response time, etc.
Each of these issues has been explained below:
- Incorrect memory management may involve issues like a memory being allocated for use but never released or when resources use more memory than is required. When such scenarios continue for a long duration, then it might lead to the system running out of memory resulting in an application to stop responding.
- Database connection issues – Error arising while closing a database connection may, in the long run, result in the application crashing altogether.
- Degrading application response time – At times an application for some reason might become less efficient and its response time might increase. Over a course of time, this may cause the application to stop responding.
To avoid such situations from arising, we would prefer to Soak test our application. It helps in identifying such underlying issues that might otherwise go undetected.
#2) Soak Test helps to determine if our application is ready to take up the load for a sustained period.
#3) It enables the team to take corrective action based on how the system responds to the Soak tests.
When To Begin Soak Test?
Ideally speaking, like any other performance testing, this testing should be done during product development along with functional testing. However, this is seldom done. The reason is obvious i.e. managing the project cost.
Thus, the focus mainly lies in functional testing and all forms of Performance testing are generally given a back seat and taken up close to the release date of the application.
In general, Soak testing is taken up just before the application being released to the client. But this has a big disadvantage that relates to the fixing of the issue.
When any performance issue is found at a later stage, it may be difficult to fix it as that could involve a major code change that might not be possible considering the closeness of the Application delivery date.
Thus, it is always advisable that this testing should be performed well on time so that the identified issues can be addressed.
Soak Testing Strategy
Just as a Test Strategy is prepared for testing an application, a strategy is prepared beforehand to perform Soak testing, and this is very much needed.
Let’s take a look at what goes into the Soak Testing Strategy preparation.
Before starting the Soak Test, the team has to determine the load for which the application needs to be Soak Tested. The duration for which it needs to be tested must also be predetermined. Generally, this is provided by the development team.
The test team should decide on the scenarios that they plan to the Soak Test. This would, in turn, depend on the Client commitment and requirement of the Application under Test.
As Soak testing is mainly focussed on identifying memory and resource leak issues it is important to know the memory and database consumption beforehand against those available.
The environment details like the OS, the device, etc. on which Soak testing would be performed should also be decided.
Last but not the least the risk(s) that are involved should also be taken into consideration. A backup plan should always be made for such situations. For instance, if the database crashes while testing, then what other alternatives are available in its place and so on.
Scenarios For Soak Testing
When an e-commerce website announces an online sale of its products, then it is natural for the website to get loaded during the sale period that might span for 3-5 days. In such a situation, the website should be Soak tested to avoid any unexpected crash.
During the closing of a financial year, a bank website may have to face a very high load for a continuous period. In such a situation the website must have been Soak tested to avoid any unexpected crash of the web application.
When an application is designed to handle a predetermined load for a continuous predetermined period, then it becomes necessary to test the application for a load at least 2X its known load handling capability.
For example, if a website is known to handle a 500 user load for a continuous period of 15 hours, then the application should also be Soak tested for 1000 users for 15 hours. This would help us to know if the application would respond abnormally when forced to twice its load capacity.
- Soak Testing should always be done by knowing the idle load limit of the application, both, in terms of the users and the time duration. This is required to be known as the target is to load the application with the expected users but for a long duration.
- It is advisable to run Soak tests at night or if even longer duration testing is to be done, then it is advisable to do so over weekends. The reason is obvious i.e. during work hours resources get tied up whereas at night or non-work hours the test servers may be available to be used for long durations. Thus non-work hours is the ideal time for such tests.
- Risks associated when Soak testing an application should always be analyzed and a mitigation plan should be ready for the same any incident.
Soak Testing Constraints
- The long duration required for testing an application is one big constraint generally due to the unavailability of time. Thus Soak testing may at times be avoided due to a shortage of time.
- The test environment needs to be carefully selected so that any other type of testing being performed on the Application does not get impacted. This may happen, as testing the application for heavy load for long-duration could result in issues.
- The time for Soak testing has to be carefully decided and should mainly be off-work hours (like a weekend or night hours after work closure).
- Generally, automation tools are required for Soak testing as the tests have to be run for long durations with a large number of users.
Disadvantages Of Soak Testing
- Project timelines may get impacted due to Soak testing as the time required for the same is generally high.
- Resources get tied up for the testing duration as there is high memory utilization due to a large number of users accessing the application.
Through this tutorial, we learned what Soak testing is and what makes it necessary to perform this testing.
Now with this understanding of what Soak Testing is and what kind of issues it helps in identifying, we can very well understand the need for performing the same. Especially in times when the entire world is always connected this testing becomes a must.
We saw when we should begin to Soak test along with the approach that should be followed. Scenarios, best practices, and the constraints associated with it were also discussed here.
We hope this tutorial helped you in understanding what is Soak testing and must have enhanced your knowledge on the same.