Functional Testing Vs Performance Testing: Should It Be Done Simultaneously?

Functional Testing Vs Performance Testing:

Differences between Performance Testing, Load Testing and Stress Testing were explained with examples in our last tutorial.

Software Testing covers a wide range of areas where any verification or validation of software functionality can occur. Occasionally, non-functional aspects become less concerning the functional aspects. They are not performed practically; simultaneously during software testing.

Must Read => Entire Series of Performance Testing Tutorials

This article explains the added benefits of the quality of the software product during various scenarios in the software testing life-cycle when both functional and non-functional are taken simultaneously. 

functional performance testing 5

Quick difference between Performance Testing and Functional Testing

Sl NOFunctional Testing
Performance Testing
1To verify the accuracy of the software with definite inputs against expected outputTo verify the behavior of the system at various load conditions
2It can be manual or automated
It can beperformed effectively if automated
3One user performing all the operations
Several users performing desired operations
4Involvement required from Customer, Tester and DeveloperInvolvement required from Customer, Tester, Developer, DBA and N/W Management team
5Production sized test environment not mandatory and H/W requirements are minimalRequies close to production test environment & several H/W facilities to populate the load

Why functional testing and performance testing should be done simultaneously?

 

Functional testing becomes much more important for any software pre-release. Actual results-based verification and validation in the replicated production or test environment are where the testing usually happens.

Defect leakage can become one of the greatest issues:

Testers have more responsibility than developers in terms of the quality of the product. Basically, they don’t want the tested product to have defect leakage. Testers generally tend to only perform functional testing to achieve this.

functional performance testing 1

The following is a conversation between a Test Manager and a Tester:

(Test Manager is referred to as ‘TM’ and Tester as ‘TR’)

TM: Hey buddy… How are we doing in the product ‘A’ testing?

TR: Yep… We are progressing through in a greater fashion.

TM: That’s fantastic… And what is our scope in terms of performance testing while functional testing is under execution?

TR: We aren’t covering them, our deliverables are supposed to be only in the functional area and not on the non-functional area. Also, the test environment we’re using is not an exact replica of the production.

There are a few questions from the above conversation to be considered:

  • Does functional testing have a dependent factor over performance?
  • What if the performance of the software is degraded, but the delivery of the product happens without checking the performance?
  • Performance testing – is it co-existing within the functional testing process?

It has become a general practice for testers not to work on the non-functional aspects unless they are requested to do so. It’s common to avoid non-functional testing until the client has reported issues with the performance of the software under test.

So, there are 2 questions for you to consider:

  1. Performance – does it impact functional testing?
  2. Do we keep performance testing as a separate deliverable, even if it worries the client?

Performance testing is important!

Software works based on various architectures and following models, including:

  1. Required response reply models
  2. Transactions based systems
  3. Load-based systems
  4. Data replication based systems

The above mentioned systematic model’s functional testing behavior depends on the performance of the system.

functional performance testing 2

The automation point-of-view requires much attention towards performance testing.

The following is a conversation between a client and the Test Manager.

(Client is referred to as ‘CL’ and Test manager as ‘TM’)

CL: Hence coming to the solution we have requested, I hope there will be multiple iterations of the testing which is happening currently.

TM: Yes, this can be done. As you have said, there will be a higher probability of the iterative testing, we would like to propose automation to deal with the functional (regression) testing.

CL: OK great, please send us your approach so we may approve this. Automation will have a much higher output with minimal effort.

TM: Exactly. We will work on the approach and get back to you with a Proof of Concept.

From the above conversation, it is clear that the clients’ need is to optimize efficiency.

Case study

Company ABC works on a project for developing Software A. Testing the Software A is being done by the company XYZ.

The contract for Company ABC and XYZ has some restrictions for their collaboration. Any discussion between the 2 companies should happen once-a-week or three-times-a-month. The system works on a model of request-response mode. The development phase has been completed by Company ABC.

Now it’s the time for Company XYZ to perform the formal functional testing on Software A. XYZ starts working on testing Software A. They have given a clean chit on the software and have given the ‘Go’ for live implementation after 2 cycles of testing.

In spite of the quality certificate from the testing team, the live implementation did not go well. There were lots of post-production bugs. There were large numbers of issues faced by the clients, including a break in functionality for the end-to-end business processes.

So now what is the problem?

functional performance testing 3


  • Is it a problem with a restriction on collaboration between the development and testing team?
  • Is it that the requirements were not captured 100%?
  • Is it that the product was not tested in a proper test environment?
  • Or any other causes?

After careful research and analysis, the following were inferred:

functional performance testing 4

  1. There were few of the dependent and interdependent applications which had performance issues while fetching the responses.
  2. The test inputs used were not absolute.
  3. The robustness of the software was not taken care of.
  4. Lots of sync issues between the multiple independent applications.
  5. The software testing had done multiple re-works which were not considered.

Hence after the remedial actions planning team stepped in, the following were suggested:

  1. The interaction between the development team and the testing team has to be increased.
  2. All dependent applications need to be connected and included in the system functional testing
  3. The request and response time-out value needs to be increased to give room to non-production environments
  4. Various input ranging between simple to the complex has to be used in functional testing
  5. Non-functional testing, especially the performance and load testing, has to be done as advised by the remedial team.
  6. In addition to system testing, system integration testing has to be performed.
  7. A minimal time-gap between any two testing iterations has to be provided. This is for re-testing the previously identified bugs.
  8. All bugs identified in previous iterations should be fixed in the current iteration.

The testing team implemented all of the actions proposed and there were a great number of defects uncovered in little time.

Observations:

  1. The live implementation schedule of the software improved significantly by optimizing the test-cycle times.
  2. There was good progress in the optimization of software quality. Hence there was a tremendous decrease in the support tickets post-implementation.
  3. Re-works were decreased and it was testing iterations instead of re-work. Between the different iterations, there were better improvements in quality observed.

Conclusion

Performing non-functional testing during functional test execution is more advantageous and will add more benefits to the overall software quality. This will identify performance bugs (restricted to the testing environment and dependency) and hence will be reducing situations of functional issue assumptions.

Sufficient planning for performing functional and non-functional testing (to a minimum level) has to be done in order to keep a strong relationship among the other stakeholders of the project.

About Author: This is an article written by Nagarajan. He is working as a test lead with over 6 years of Testing experience in various functional areas like Banking, Airlines, Telecom in terms of both manual and automation.

Our upcoming tutorial will explain more about Performance Test Plan and Test Strategy.

PREV Tutorial | NEXT Tutorial

Recommended Reading


10 thoughts on “Functional Testing Vs Performance Testing: Should It Be Done Simultaneously?”

  1. Rashmi Tiwari/Vamsi /Komal,

    Thanks for reading the post. Please wait for further more post in our website.

    We are planning to provide all in all guidelines to all varieties of testing including mobile, iOS, Android

  2. Hi Nagarajan,

    Nice post.

    But, I am little contradicting here. Performance testing should be done on finished product i.e., after all the bugs identified in functional testing are closed and not to be done parallel to functional testing.

    Coz, any defect identified in parallel will affect other testing.

  3. Hi Balaji,

    Sure. It is the fact. But when there was some module had good performance that has degraded at a later point of time due to some non functional parameters, they can be addressed much in advance to avoid them.

    After functional testing has been completed, performance testing itself can be performed using specialized tools like loadrunner.

    But it is always better to set threshold for any performance and report now and then as a good practice.

  4. I like the article on this site, but this article is little confusing.. it started with the right thing talking about the problem then in between there are generic problem of testing away from the main topic of performance testing..

    Doing parallel with functional may help for a stable product with minimum functionality affected..

    We may do a little functional test with performance test.
    These are my views

  5. Hi Nagarajan,

    It is well said that performance testing should be done after the application has been developed but it is better if performance testing starts initially parallel to the functional testing to avoid any future defects.

  6. Hi

    That is a very interesting article.
    However, one question arises, what if there are functional bugs. Then it will definitely interfere with Performance Testing.

Leave a Comment