Stability Testing is a part of Performance Testing. This Tutorial Explains the Objectives, Importance, and Need of Stability Testing With Examples:
Stability testing is ‘Non-Functional’ testing. Here the team does not test the features and basic functionality of the site; but tests the features of the software product like ‘robustness’, ‘error handling’, ‘reliability’, and product stability’.
Let’s consider a person purchasing a new product from the market (For Example, a mobile phone). The buyer will expect the product to perform evenly for a long time, without any failures. Similarly, users of the website and mobile app will expect the corresponding website or mobile apps to perform with high reliability and stability.
Now, take the case of users trying to purchase commodities from e-commerce portals. If a huge number of users use the same portal concurrently, the chances of performance degradation for the website are high. The users may also experience a slow response time or even memory leakage during the session.
These issues raise threats to both developers and stakeholders. It’s the responsibility of the testing team to find those issues and report to the development team before it reaches the end-users. This type of testing will come under performance testing. Stability testing is a part of performance testing.
What You Will Learn:
- Overview Of Stability Testing
Overview Of Stability Testing
As mentioned earlier, Stability Testing is also defined as ‘Non-Functional’ testing. Here, the testing teams check the robustness, error handling, reliability, and product stability.
This testing is also termed as ‘Endurance Testing’ or ‘Soak Testing’. It is also known as the ‘Extreme Load Testing’.
In this type of testing, testers will check how the online portal responds when lots of active users use the website concurrently. Testers also need to check the system response in such an environment.
On many occasions, websites need to run continuously for weeks (even months) without the opportunity to restart the server. Active users for such websites (users using the website currently) can be tremendous and each user will expect a seamless performance.
The role of the testers is to provide confidence to the developers and ensure the end-users that they can use a system as indented, without failure or memory leakage even in high traffic. For this purpose, the application is subjected to maximum load (till the breakpoint of the application) and the behavior of the system is checked under such conditions.
This testing is usually done before the release of the software. Testers need to ensure that the application is capable of handling the anticipated load to the website. Sometimes the system can crash due to a heavy load or can become slow or even show unexpected behaviors.
Software stability highly depends on the seamless performance of the system under the above-mentioned stressful conditions. So, stability testing plays a very important role.
Objectives Of Stability Testing
The objectives are enlisted below:
- To find the durability of the system.
- Find the stability of the application and thereby increase the confidence of the developer.
- Find the defect in the system in a stressful environment.
- Overall evaluation and effectiveness of the product.
- To make sure that the system can handle a large program.
- To test the response time of an application.
- To check the database connectivity.
Facts About Stability Testing
Some useful facts are enlisted below:
- Stability testing demands an appropriate testing environment.
- For getting better results, stability testing needs a well-planned and structured approach.
- It is a time-consuming process. Time taken for the testing process may vary based on the client’s requirement, type of product, and company policies.
- Isolation of the system is important in this form of testing. While doing stability testing of the application, there are chances for data permanently getting lost or corrupted.
- Endurance testing may cause failures of application components, so end users may observe unhandled exceptions.
Difference Between Stability And Reliability In Software System
There is a substantial difference between reliability and stability in a software application. This can be explained with the help of an example.
Consider a user installed a new app on his/her mobile phone and launched it. If the application installed is crashing after every 3 minutes of usage, it will certainly annoy the user. But if the user could restore the data without any loss after relaunching the app, the application will not lose its reliability. Such an application can be regarded as reliable but cannot be considered stable.
On the other hand, take the scenario in which the application data is not getting saved properly. Here, the application functions well and does not crash (like every 3 minutes). Such an application can be or cannot be regarded as reliable but can be considered as ‘stable’.
Reliability and Stability are like two sides of the same coin. So just remember the fact that both reliability and stability are important for a product from a business perspective.
Examples of Stability testing:
It’s a common approach that, when a user purchases a new mobile phone, he/she knowingly or unknowingly performs a stability test. The user will save many images, photos, videos, documents, etc in the device memory and they will check if too many data stored has affected the performance of the device or not. This is a particular example of stability testing.
Due to immense data stored, our device may hang up sometimes, then the user needs to delete some data or clear temp files to make the device regain the performance. After this testing, the user will have a clear idea about the capacity of the system.
Another real-time example is online purchase portals. During a ‘sale/festival’ season many people purchase commodities from these types of web portals. The performance of a website must satisfy user expectations.
So, the testers need to test the site by keeping in mind the expected ‘rush’ which is likely to happen in these sites during those days.
Stability Testing Of An Internet Connection
Here, the test case will be to verify how stable our internet connection is. While establishing a connection over the internet and requesting web pages, due to packet loss, users have to suffer inconsistent latency in viewing web pages.
Sometimes, packet loss occurs when it fails to reach the destination. It is due to packet loss that many of us may have come across similar issues while we are using Skype video calls, playing games over the internet, or randomly being disconnected from the internet.
Prerequisites for testing internet connectivity:
- To test the internet stability, we need to have a web browser in which the firewall is disabled.
- Chose a web address (For Example, https://www.google.com/) which is unlikely to fail.
- Use Google Spreadsheet or Microsoft Excel to record the result, since it is easy to understand and more readable for the users.
- Double-check the internet connectivity, devices and check the connections again. We will then perform the below-mentioned tests.
Methods used for testing internet connectivity:
The best way to test connectivity is to visit SpeedTest.(before further reading please visit the site). In SpeedTest.net we have a provision for choosing servers. Choose and run the server which is near you.
After that, the site will perform some calculations based on some predefined algorithms and show the report which determines the quality of our internet connection immediately. In the report, the packet loss percentage is included. It should be ‘0%’.
‘0%’ packet loss determines the high stability of the connection. Any number greater than ‘0’ shows that the connectivity is unstable.
The second way is using ‘cmd’ prompt and type in the command ‘ping'(Please refer to the below figure). Here, using command prompt we can test the stability and the latency of the internet in real-time also.
In the figure please check the ‘Ping Statistics’ section.
Number of packets sent = 4
Number of packets received = 4
Number of packets lost = 0
The result shows that the connection is highly stable.
Step 1: Testers will choose any of the two methods explained above.
Step 2: Testers will run the process and record the needed information that is obtained as a response to internet connectivity, in a spreadsheet.
Step 3: They will repeat the process 3 or four times a week.
(Since Stability testing is done over a period of time, testers will have to schedule the process at least more than two times a week.)
Step 4: The results are recorded in spreadsheets.
Testers need to note down the date of running the process. Compare the result obtained on each date to get an idea about the stability of our current network.
Note: Breakpoint is the state of the system at which the system will collapse when a further load is given to it. It defines the capacity of the system.
The specification document is a document given to the testers by the team leader and will contain details about the anticipated load in the system. In stability testing, the testers will check the system breakpoint based on the guidelines provided to them in the specification document.
The testing team will check the system by testing the application with the load above/below the breakpoint specified in the specification document. This is different from the case of Load Testing.
In stability testing, we will use only the expected load for testing, but in load testing, an unexpected load is given to the application, and testers check the capacity of the application.
Software Testing Life Cycle Based On Stability Testing
The different phases of the Software Testing Life Cycle are enlisted below:
- Requirement Analysis
- Test Plan
- Test Case Development
- Test Environment Setup
- Test Case Execution
- Test Closure
Let us understand all the above phases in detail.
#1) Requirement Analysis
In this phase, the testing team will decide the different types of testing to be performed in the application. It purely depends on client requirements and the type of application. For Example, the testers are testing a banking application then the most priority testing, in this case, is security testing. If the testers are testing a real-estate application, they will give priority to functional testing.
#2) Test Planning
In this phase, scopes of testing are discussed. Testers will discuss the need for automation. For stability testing, the test processes are tedious and have to be repeated many times over a certain duration, automation will be a good choice. ‘LoadStrom’ is a good tool to perform stability testing using automation.
In this phase, we will discuss the budget and time limit for testing with the client. Since the testing is time-consuming, the budget and time limit should meet the test plan.
#3) Test Case Development
Test case for testing the application is created in this phase.
#4) Test Environment
The test environment is a vital factor for testing stability. We need a proper test environment which is the replica of our production environment. The original environment should be unused because during the testing the site may sometimes crash or even data loss.
#5) Test Execution
In this phase, test cases are executed and the test results are verified. This is a time-consuming phase. The general issues faced by testers in this phase are memory leakage, problems with data connectivity, slow response time, etc.
#6) Test Closure
In this phase, all the team members will meet and discuss the exit criteria used in the project. Exit criteria depend on factors like the number of defects found and the time used in testing.
Tools Used In Stability Testing
Following tools are used:
- Rational Performance Tester
How to use Apache JMeter for Endurance Testing?
Apache JMeter is a good tool for Endurance Testing. Before starting the testing, testers must have good knowledge about the business objectives. After that testers will create test scripts. Then we will configure the thread group settings in JMeter.
While doing endurance testing using JMeter, we must specify the following factors:
- The number of threads: This indicates the expected number of users on the site.
- Ramp-up period: This indicates the time taken by each thread to complete. If we have 5 threads, the ramp-up period is 50 seconds.
- Loop-count: This indicates the number of times the process is repeated. For endurance testing, it is set to forever.
- Scheduler: In this testing, we will use scheduler functionality. We need to specify the scheduler configuration according to the requirement.
Many applications are improperly designed and do not release the device memory after the usage. This will gradually lead to memory loss. We can overcome the issue with stability testing. So, Stability testing is very important. It is non-functional testing. It is only concerned with the characteristic of the application. Here, testing is not about the behavior of the system.
Hope you understood the importance and need for Stability Testing.