What is Regression Testing? Definition, Tools, Method, and Example

What is Regression Testing?

Regression Testing is a type of testing that is done to verify that a code change in the software does not impact the existing functionality of the product. This is to make sure the product works fine with new functionality, bug fixes or any change in the existing feature. Previously executed test cases are re-executed in order to verify the impact of change.

What is Regression Testing

Regression Testing is a Software Testing type in which test cases are re-executed in order to check whether the previous functionality of the application is working fine and the new changes have not introduced any new bugs.

This test can be performed on a new build when there is a significant change in the original functionality that too even in a single bug fix.

Regression means retesting the unchanged parts of the application.

Tutorials Covered In This Series

Tutorial #1: What is Regression Testing (This Tutorial)
Tutorial #2: Regression Test Tools
Tutorial #3: Retest Vs Regression Testing
Tutorial #4: Automated Regression Testing in Agile


Regression Test Overview

Regression test is like a verification method. Test cases are generally automated as test cases are required to execute again and again and running the same test cases again and again manually is time-consuming and tedious one too.

For Example, Consider a product X, in which one of the functionality is to trigger confirmation, acceptance, and dispatched emails when Confirm, Accept and Dispatch buttons are clicked.

Some issue occurs in the confirmation email and in order to fix the same, some code changes are done. In this case, not only the Confirmation emails need to be tested but Acceptance and Dispatched emails also needs to be tested to ensure that the change in the code has not affected them.

Regression Testing is not dependent on any programming language like Java, C++, C#, etc. It is a testing method which is used to test the product for modifications or for any updates being done. It verifies that any modification in a product does not affect the existing modules of the product.

Verifying that the bugs are fixed and the newly added features have not created any problem in the previous working version of the software.

Testers perform Functional Testing when a new build is available for verification. The intent of this test is to verify the changes made in the existing functionality and the newly added functionality as well.

When this test is done, the tester should verify whether the existing functionality is working as expected and the new changes have not introduced any defect in functionality that was working before this change.

Regression test should be a part of the Release Cycle and must be considered in the test estimation.

When To Perform This Test?

Regression Testing is usually performed after verification of changes or new functionality. But this is not the case always. For the release that is taking months to complete, regression tests must be incorporated in the daily test cycle. For weekly releases, regression tests can be performed when the Functional Testing is over for the changes.

Regression checking is a variation of retest (which is simply to repeat a test). When Retesting, the reason can be anything. Say, you were testing a particular feature and it was the end of the day- you could not finish testing and had to stop the process without deciding if the test passed/failed.

The next day when you come back, you perform the test once more – that means you are repeating a test you performed before. The simple act of repeating a test is a Retest.

Regression test at its core is a retest of sorts. It is only for the special occasion that something in the application/code has changed. It might be code, design or anything at all that dictates the overall framework of the system.

A Retest that is conducted in this situation to make sure that the said change has not made an impact on anything that was already working before is called Regression Test. The most common reasons why this might be conducted are because new versions of the code have been created (increase in scope/requirement) or bugs have been fixed.

Can Regression Testing Be Performed Manually?

I was just teaching one of these days in my class, and a question came to me – “Can regression be done manually?”

I answered the question and we moved on in the class. Everything seemed OK, but somehow this question nagged me for quite a while later.

Over the many batches, this question comes multiple times in various different ways. Some of them are:

  • To perform the test execution do we need a tool?
  • How is Regression Testing performed?
  • Even after an entire round of testing– newcomers find it difficult to discern what exactly Regression test is?

And of course, the original question:

  • Can this Testing be performed manually?

To begin with, Test execution is a simple act of using your Test cases and performing those steps on the AUT, supplying the test data and comparing the result obtained on the AUT with the expected result mentioned in your test cases.

Depending on the comparison result, we set the status of the test case pass/fail. Test execution is as simple as that, there are no special tools necessary for this process.


Automated Regression Testing Tools

Automated Regression Test is the testing area where we can automate most of the testing efforts. We run all the previously executed test cases on a new build.

This means that we have a test case set available and running these test cases manually is time-consuming. We know the expected results, so automating these test cases is time-saving and is an efficient regression test method. The extent of automation depends upon the number of test cases that are going to remain applicable overtime.

If test cases are varying from time to time, the application scope goes on increasing and then automation of regression procedure will be the waste of time.

Most of the Regression test tools are record and playback type.  You will record the test cases by navigating through the AUT (application under test) and verify whether the expected results are coming or not.


Recommended Tools

#1) Ranorex Studio

ranorex-studio

Enhance Software Quality and maximize your resources with this powerful Automated Regression Testing tool. You can execute more test cases in a fraction of the time with Ranorex which is up to 78% efficiency increase over Manual Testing.

=> Visit Ranorex Studio Website


#2) Katalon Studio

Katalon Studio

Katalon Studio is a free, all-in-one test Automation tool for your website, API, desktop, and mobile applications.

It allows you to quickly design and automate regression test cases as well as run scripts on multiple devices or cross-browser in parallel. You can also review test results with a comprehensive and customizable test report in Logs, HTML, CSV, or PDF formats, and then forward it as an email attachment through Katalon Studio.


Other Tools

Most of these are Functional and Regression test tools.

Recommended Reading => Check Here For The List Of Top Regression Tools

Adding and updating Regression test cases in an Automation test suite is a cumbersome task. While selecting an Automation tool for Regression tests, you should check if the tool allows you to add or update the test cases easily.

In most cases, we need to update automated Regression test cases frequently due to frequent changes in the system.


WATCH THE VIDEO

For a more detailed explanation of the definition with an example, please check the following Regression Test video:

Why The Regression Test?

Regression is initiated when a programmer fixes any bug or adds a new code for new functionality to the system.

Regression testing

There can be many dependencies in the newly added and existing functionality.

It is a quality measure to check whether the new code complies with the old code so that the unmodified code is not getting affected. Most of the time the testing team has the task to check the last minute changes in the system.

In such a situation, testing only affected application area is necessary to complete the testing process on time by covering all the major system aspects.

This test is very important when there is a continuous change/improvement added in the application. The new functionality should not negatively affect the existing tested code.

Regression is required to find the bugs that occurred because of a change in the code. If this testing is not done, the product might get critical issues in the live environment and that indeed can lead the customer into trouble.

While testing any online website, a tester reports an issue that the Price of the Product is not shown correctly i.e. it shows lesser price than the actual price of the Product, and it needs to be fixed soon.

Once the developer fixes the issue, it needs to be re-tested and Regression Testing is also required as verifying the price at the reported page would have got corrected but it might be showing an incorrect price at the summary page where the total is shown along with the other charges or the mail sent to the customer still has the incorrect price.

Now, in this case, the customer will have to bear the loss if this testing is not performed as the site calculates the total cost with the incorrect price and the same price goes to a customer by email. Once the customer accepts, the Product is sold online at a lower price, it will be a loss for the customer.

So, this testing plays a big role and is very much required and important as well.

Types Of Regression Testing

Given below are the various types of Regression :

  • Unit Regression
  • Partial Regression
  • Complete Regression

#1) Unit Regression

Unit Regression is done during the Unit Testing phase and code is tested in isolation i.e. any dependencies on the unit to be tested are blocked so that the unit can be tested individually without any discrepancy.

#2) Partial Regression

Partial Regression is done to verify that the code works fine even when the changes have been done in the code and that unit is integrated with the unchanged or already existing code.

#3)  Complete Regression

Complete Regression is done when a change in the code is done on a number of modules and also if the change impact of a change in any other module is uncertain. Product as a whole is regressed to check any changes because of the changed code.

How Much Regression Is Required?

This depends upon the scope of newly added features.

If the scope of a fix or feature is too large, then the application area getting affected is also quite large and the testing should be performed thoroughly including all the application test cases. But this can be effectively decided when the tester gets input from a developer about the scope, nature, and the amount of change.

As these are repetitive tests, test cases can be automated so that a set of test cases alone can be easily executed on a new build.

Regression test cases need to be selected very carefully so that maximum functionality is covered in a minimum set of test cases. These set of test cases need continuous improvements for newly added functionality.

It becomes very difficult when the application scope is very huge and there are continuous increments or patches to the system. In such cases, selective tests need to be executed in order to save testing cost and time. These selective test cases are picked based on the enhancements done to the system and the parts where it can affect the most.

What Do We Do In Regression Check?

  • Re-run the previously conducted tests
  • Compare the current results with previously executed test results

This is a continuous process performed at various stages throughout the software testing lifecycle.

A best practice is to conduct a Regression test after the Sanity or Smoke Testing and at the end of Functional testing for a short release.

In order to conduct effective testing, the regression Test Plan should be created. This plan should outline the regression testing strategy and the exit criteria. Performance Testing is also a part of this test to make sure that the system performance is not affected due to the changes made in the system components.

Best practices: Run automated test cases every day in the evening so that any regression side effects can be fixed in the next day build. This way it reduces the release risk by covering almost all regression defects at an early stage rather than finding and fixing those at the end of the release cycle.

Regression Testing Techniques

Given below are the various techniques.

  • Retest all
  • Regression Test Selection
  • Test case Prioritization
  • Hybrid

Regression testing techniques

#1) Retest All

As the name itself suggests, the entire test cases in the test suite are re-executed to ensure that there are no bugs that have occurred because of a change in the code. This is an expensive method as it requires more time and resources when compared to the other techniques.

#2) Regression Test Selection

In this method, test cases are selected from the test suite to be re-executed. Not the entire suite is re-executed. Selection of test cases is done on the basis of code change in the module.

Test cases are divided into two categories, one is Reusable test cases and another one is Obsolete test cases. The reusable test cases can be used in future regression cycles whereas obsolete ones are not used in the upcoming regression cycles.

#3) Test Case Prioritization

Test cases with high Priority are executed first than the ones with medium and low priority. The priority of test case depends on its criticality and its impact on the product and also on the functionality of the product which is used more often.

#4) Hybrid

The hybrid technique is a combination of Regression test selection and Test case Prioritization. Rather than selecting the entire test suite, select only the test cases which are re-executed depending on their priority.

How To Select A Regression Test Suite?

Most of the bugs found in the production environment occur because of the changes did or bugs fixed at the eleventh hour i.e. the changes done at a later stage. The bug fix at the last stage might create other issues/bugs in the Product. That’s why Regression checking is very important before releasing a Product.

Below is a list of test cases that can be used while performing this Test:

  • Functionalities which are frequently used.
  • Test cases which cover the module where the changes have been done.
  • Complex test cases.
  • Integration test cases which include all the major components.
  • Test cases for the core functionality or feature of the Product.
  • Priority 1 and Priority 2 test cases should be included.
  • Test cases which frequently fail or recent testing defects were found in the same.

How To Perform Regression Testing?

Now that we have established what regression means, it is apparent that it is testing also – simply repeating in a specific situation for a specific reason. Therefore, we can safely derive that the same method applies for testing in the first place can be applied to this too.

Therefore, if testing can be done manually then Regression Testing can be too. The use of a tool is not necessary. However, as time goes on applications get piled on with more and more functionality which keeps increasing the scope of regression. To make the most of the time, this testing is most often Automated.

Given below are the various steps involved in performing this Testing

  • Prepare a Test suite for Regression considering the points mentioned in “How to select Regression Test suite”?
  • Automate all the test cases of the test suite.
  • Update the Regression suite whenever it is required like if any new defect which is not covered in the test case is found, and a test case for the same should be updated in the test suite so that the testing is not missed for the same next time. The regression test suite should be managed properly by continuously updating the test cases.
  • Execute the Regression test cases whenever there is any change in the code, the bug is fixed, new functionality is added, an enhancement to the existing functionality is done, etc.
  • Create a test execution Report which includes the Pass/Fails status of the executed test cases.

For Example:

Let me explain this with an example. Please examine the below situation:

Release 1 Statistics 
Application NameXYZ
Version/Release Number1
No. of Requirements (Scope)10
No. of Test Cases/Tests100
No. of days it takes to Develop5
No. of days it takes to Test5
No. of Testers3
Release 2 Statistics 
Application NameXYZ
Version/Release Number2
No. of Requirements (Scope)10+ 5 new Requirements
No. of Test cases/Tests100+ 50 new
No. of days it takes to Develop2.5 (since this half the amount of work than earlier)
No. of days it takes to Test5(for the existing 100 TCs) + 2.5 (for new Requirements)
No. of Testers3
Release 3 Statistics 
Application NameXYZ
Version/Release Number3
No. of Requirements (Scope)10+ 5 + 5 new requirements
No. of Test cases/Tests100+ 50+ 50 new
No. of days it takes to Develop2.5 (since this half the amount of work than earlier)
No. of days it takes to Test7.5 (for the existing 150 TCs) + 2.5 (for new Requirements)
No. of Testers3

The following are the observations we can make from the above situation:

  • As the releases grow the functionality grows.
  • The development time does not necessarily grow with releases, but the testing time does
  • No company/its management will be ready to invest more time in testing and less for development
  • We cannot even reduce the time it takes to test by increasing the test team size because more people means more money and new people also means lots of training and maybe also a compromise in quality as the new people might not be at par with the required knowledge levels immediately.
  • The other alternative clearly is to reduce the amount of regression. But that could be risky for the software product.

For all these reasons, Regression Testing is a good candidate for Automation Testing, but it does not have to be done only that way.

Basic Steps to Perform Regression Tests

Every time the software undergoes a change and a new version/release comes up, the following are the steps you can take to carry out this type of testing:

regression testing process

  • Understand what kind of changes have been made to the software
  • Analyze and determine what modules/parts of the software might be impacted – the development and BA teams can be instrumental in providing this information
  • Take a look at your test cases and determine if you will have to do a full, partial or unit regression. Identify the ones that will fit your situation
  • Schedule the time and test away!

Regression In Agile

Agile is an adaptive approach which follows an iterative and incremental method. The product is developed in short iterations called sprint which lasts for 2- 4 weeks. In agile, there is a number of iterations, hence this testing plays a significant role as the new functionality or code change is done in the iterations.

The Regression test suite should be prepared from the initial phase and should be updated with each sprint.

In Agile, Regression check is covered under two categories:

  • Sprint Level Regression
  • End to End Regression

#1) Sprint Level Regression

Sprint Level Regression is done mainly for the new functionality or the enhancement that is done in the latest sprint. Test cases from the test suite are selected as per the newly added functionality or the enhancement that is done.

#2) End-to-End Regression

End-to-End Regression includes all the test cases that are to be re-executed to test the complete product end to end by covering all the core functionalities of the Product.

As Agile has short sprints and it goes on, it is very much required to automate the test suite, the test cases are executed again and that too needs to be completed in a short span of time. Automating the test cases reduces the time of execution and defect slippage.

Advantages

Given below are the various advantages of Regression test

  • It improves the quality of the Product.
  • It ensures that any bug fix or enhancement that is done does not impact the existing functionality of the Product.
  • Automation tools can be used for this testing.
  • It makes sure that issues which are already fixed do not occur again.

Disadvantages

Though there are several advantages, there are some disadvantages as well. They are:

  • It has to be done for a small change in the code as well because even a small change in the code can create issues in the existing functionality.
  • If in case automation is not used in the Project for this testing, it will be a time consuming and tedious task to execute the test cases again and again.

Regression Of GUI Application

It is difficult to perform a GUI (Graphical User Interface) Regression test when the GUI structure is modified. The test cases written on old GUI either become obsolete or need to be modified.

Re-using the regression test cases means GUI test cases are modified according to the new GUI. But this task becomes a cumbersome one if you have a large set of GUI test cases.

Difference Between Regression And Re-testing

Re-testing is done for the test cases which fail during the execution and the bug raised for the same has been fixed whereas Regression check is not limited to the bug fix as it covers other test cases as well to ensure that the bug fix has not impacted any other functionality of the Product.

Regression Test Plan Template (TOC)

1. Document History

2. References

3. Regression Test Plan

3.1. Introduction

3.2. Purpose

3.3. Test Strategy

3.4. Feature to be tested

3.5. Resource Requirement

3.5.1. Hardware Requirement

3.5.2. Software Requirement

3.6. Test Schedule

3.7. Change Request

3.8. Entry /Exit criteria

3.8.1. Entry Criteria for this Testing

3.8.2. Exit Criteria for this Testing

3.9. Assumption/Constraints

3.10. Test Cases

3.11. Risk /Assumptions

3.12. Tools

4. Approval/Acceptance

Let's take a look at each of them in detail.

#1) Document History

Document history consists of a record of the first draft and all the updated ones in the below-given format.

VersionDateAuthorComment
1DD/MM/YYABCApproved
2DD/MM/YYABCUpdated for the added feature

#2) References

References column keep a track of all the reference documents used or required for the Project while creating a test plan.

NoDocumentLocation
1SRS documentShared drive

#3) Regression Test Plan

3.1. Introduction

This document describes the change/update/enhancement in the Product to be tested and the approach used for this testing. All the code changes, enhancements, updates, added features are outlined to be tested. Test cases used for Unit Testing and Integration Testing can be used to create a test suite for Regression.

3.2. Purpose

Purpose of Regression Test Plan is to describe what exactly and how testing would be performed to accomplish the results. Regression check is done to ensure that no other functionality of the product is hampered because of the code change.

3.3. Test Strategy

Test Strategy describes the approach which will be used to perform this testing and that includes the technique that will be used, what will be the completion criteria, who will be performing which activity, who will write the test scripts, which regression tool will be used, steps to cover the risks like resource crunch, delay in production, etc.

3.4. Features to be tested

Feature/components of the product to be tested are listed here. In regression, all the test cases are re-executed or the ones which affect the existing functionality are chosen depending on the fix/update or enhancement done.

3.5. Resource Requirement

3.5.1. Hardware Requirement:

Hardware Requirement is identified here like computers, laptop, Modems, Mac book, Smartphone, etc.

3.5.2. Software Requirement:

Software Requirement is identified like which Operating system and browsers will be required.

3.6. Test Schedule

Test schedule defines the estimated time for performing the testing activities.

For Example How many resources will perform a testing activity and that too in how much time?

3.7. Change Request

CR details are mentioned for which Regression would be performed.

S.NoCR DescriptionRegression Test Suite
1
2

3.8. Entry/Exit Criteria

3.8.1. Entry Criteria for this testing:

Entry criteria for the Product to start Regression check are defined.

For Example:

  • Coding changes/enhancement/addition of new feature should be completed.
  • Regression test Plan should be approved.

3.8.2. Exit Criteria for this testing:

Here the exit criteria for Regression are defined.

For Example:

  • Regression testing should be completed.
  • Any new critical bugs found during this testing should be closed.
  • Test Report should be ready.

3.9. Test Cases

Regression Test cases are defined here.

3.10. Risk/Assumptions

Any risk & assumptions are identified and a contingency plan is prepared for the same.

3.11. Tools

Tools to be used in the Project are identified. Such as:

  • Automation tool
  • Bug Reporting tool

#4) Approval/Acceptance

Names and Designation of the people are listed here:

NameApproved/RejectedSignatureDate

Conclusion

Regression Testing is one of the important aspects as it helps to deliver a quality product by making sure that any change in the code whether it’s small or large does not affect the existing or old functionality.

A lot of automation tools are available for automating the regression test cases, however, a tool should be selected as per the Project requirement. A tool should have the ability to update the test suite as the Regression test suite needs to be updated frequently.

With that, we wrap this topic up and hope there will be much better clarity on the subject from now on.

Please let us know your Regression related questions and comments. How did you tackle your Regression Testing tasks?

Recommended Reading

124 thoughts on “What is Regression Testing? Definition, Tools, Method, and Example”

  1. Thank you so much for this precious article. I was having doubt that in the interview point how to answer to this type of questions like, What is regression testing and how to perform it? Now i got clear answer thank u.

  2. Hello, i really thankful to you for sending a valuable articles on important topics.
    Presently i am searching job in testing field. I would like to be perfect in automation tool QTP.
    My request is, could you plz provide important interview qns. on QTP?

  3. fine definition for regression check
    and also clearing doubts on each topics.
    i have a doubt clarify this please through mail,
    “a step by step procedures for using a test case to test a specific unit of code, function or capability?”

  4. In your answer you had mentioned Regression as verification method.
    but regression checking comes under validation….and will be validating the application has not got any new bugs due to the new enhancements or fixing bugs.
    kindly clarify me.
    Thanks for your collection of material…its very useful.

  5. @hyma

    Reply to Database Migration testing…

    Your company has an enterprise product for example say Bubbly Version 2.5 and this product has been released in the market and in use by number of customers. Now your company has developed a beta version say Bubbly Version 3.5 which has added some more new features in it. Now if your company targets the existing customer to upgrade to the new version the customers would definitely ask one question ” WHAT ABOUT THE VALUABLE DATA WHICH WE HAD IN THE PRODUCT??”. Now here comes data migration concept in which the script is written in such a way that running the script will just upgrade the application version from Bubbly 2.5 to Bubbly 3.5 and the whole data which exists in the earlier version(Bubbly 2.5) is retained in the newer version(Bubbly 3.5).
    Hope this clears you….. :)

  6. @Anitha

    Regression check is again a chain process friend. Whenever a change in requirement or change in build or change in code or bugs found and fix occurs, we use to perform regression (i.e. execute all the test cases) to ensure it had not affected the application in any way. The process starts with VERIFICATION and goes on to VALIDATION, thereby enhancing the quality of the software…

  7. hi…
    I have little doubt on concepts of retesting and regression.Say,retest is done on some modules of build1 and Regression is done on (modified) build2.If we get bugs on build2,can we retest it(retest some modules)?

    Thank u in advance

  8. Hi,

    Just want to clear you what exactly the difference between retest & regression.
    Retest – Testing within the same build and no change in the build number also.
    Regression – Testing in different build numbers or on different versions of the build.

    Hope you can understand now and map to your questions.

    Cheers,
    Nani

  9. Hi ,
    During regression check whether we have to test all the executed(passed) tests or any selected tests .Please clarify my doubt asap.

  10. It’s really fentastic and good article .

    I have some doubts in Regression, verification and validation tests. Now i can clearly understand concepts.

    Thanks.

  11. Hi,
    Which testing would i do if my application is being migrated from e.g. 9i to 10g without any changes in application.

  12. hi vijay,
    can u please clarify?
    1.how to do backend testing?
    2.tell me one scenario how to do backend testing?
    3.what is the uses of backend testing?
    pl….clarify

    thanks in advance

  13. 1.how to do backend testing in qtp?with simply examples?
    2.simply tell about test case-senario with examples?
    3.what is the diff regression-retesting?
    4.tell about sanity,sanitation testing?both are one or diff?
    5.how many types of testing in sys testing?

  14. Hi Abhay,

    QTP is a Functional Testing tool u can use only regression checking through QTP not Load Testing so u can use LoadRunner and QEngine, Jmeter …like lot of Performance tools is there are clear now.
    please let me know if u have any clarifications
    Thanks,
    Suresh
    bsuresh7282@gmail.com

  15. retesing: means testing the same functionality once again after the bug is fixed by developers.

    regression testing: done by two scenarios
    1.test the any dependency functionalities are effecting after the bug is fixed.
    2.test the enhancement features(i.e test the existing functionalities while adding new functionalities to it).

  16. Hi all,

    can any one please tell me if we have 100 test cases for a build and that build needs some modification. After modification, we need to perform regression testing, My question is thatdo we need to execute all the test cases again?

    Regards,
    KAT22

  17. Kindly send me the code how to run a login page with different user name & password no of times after recording sample Flight application of QTP using VBscript before closing the application.

    Thanks
    Devender Chandel

  18. It was very good content about the regression.
    But i need best automation and regression tool for wap site testing. Any one can give me the link please.

    Regards
    BVB

  19. To satyanarayana,
    In a regression check we have to do testing for that particular bug fixed part & all other part effected / connected that bug fixed part.
    Please correct me if i m wrong..

  20. Hi,article is really useful.Thanks.. Suppose there is the code for a proramme.It is changed now in order to add some funcionality,do i need to do regression check for Old code or new cod or entire code?

  21. Hi Pratap,
    When a new functionality is added, you need to test the new functionality completely. In addition you need to perform a regression checking on the other parts of the code to ensure that newly added functionality does not have any negative impact on the existing functionality

Leave a Comment