“We are always taught how to write functional test cases. Why is that? Is ‘non-functional testing’ carried out without documentation (in other words, on an ad-hoc basis) or is that a separate process that is much more difficult to understand? How are test cases written for different kinds of testing that happens on the application?”
This is one of the most original, distinctive and out-of-the-box questions I have been asked in the recent times. Thus, this is the topic for today and this is what we are going to illuminate in the following article.
How come we never get to see and practice writing non-functional test cases?
Let’s start with what we know and as always a practical scenario.
Functional and Non-functional Testing:
Example: The following are the steps to be performed on an Online Banking application in order to perform a transfer. Let us use that as our test of reference.
- Login to the site
- Choose the bank account
- Choose the payee (this payee could belong to the same bank or a different one- this depends on your data choice to execute this step. In any case, choose one. Also, we are going to assume that the payee is already added.)
- Enter the amount to be transferred (positive value, with in limit, correct format, etc.)
- Click Transfer and check if the confirmation is received, account balance has been updated and all that.
This is the functional test case, correct?
On the very application, on the very transfers’ page, let us say we are performing Performance, Security and Usability Testing. These are non-functional testing types, correct? How would we write the test cases?
#1. Usability Testing
Usability testing is a genre of software testing that deals with the user experience. These are few of the questions that we it tries to answer.
- How easy is the application to use?
- How satisfying is the experience of using the system?
- If not right familiar right away, how easy is to learn?
More info on it is here: Usability Testing Guide
How would a user determine the answers for the above questions in the context of usability testing?
The user would perform the exact same steps to make as in the functional test case. Am I right?
#2. Performance Testing
There are very many variations to performance testing, but at its core, it is used to get statistics about the system, its resource utilization, response time, network consumption etc. at various load points.
Check out to know more about it at: Performance Testing Tutorials
Now, If I were to test the performance of the transfers’ transaction I would have 10, 20, 30, 100…1000…etc users perform transfer operation simultaneously or incrementally depending on what I want to target and gather data about..
What steps would each user perform in order to use the transfers while the performance test is in progress?
The same exact steps as the functional test, correct?
#3. Security Testing
Security Testing is a branch of QA that helps make the software systems hack proof. It involves identifies vulnerabilities(potential problem areas in the software system), exploiting them through penetration or white hat testing technique and when loop-holes are find, they are worked on.
Find more information at: Security testing Guide
When I want to check if transfers are hack proof and are directed correctly to the intended recipients and that there are no black spots in the entire process? I would perform the transfer while the monitoring for security leaks goes on in parallel.
Therefore, in effect I am carrying out the exact same steps I would normally do in case of a functional test case.
I guess we have enough to establish that the steps in all the situations are the same. The method and the intention behind the process is what different.
Let’s take a comparative look:
|Type of Testing||Who?||Why? Intention|
|Functional testing||QA testers||Accuracy|
|Usability||QA testers or real time users||Ease of use|
|Ease of learning|
|Performance||Performance testers via a tool that treats the operation as a transaction performed by a certain number of concurrent users while the tester is analyzing all the logistics||Response time|
|Network usage etc.|
|Security||Scanning tools and other monitoring system by specialized security experts||Hack safe|
|Payee and the payer identity protection etc.|
What is interesting to note is that, no matter what form of testing we want to do, all the steps are the same. The real difference is that:
- Who performs these steps?
- What is the intention, or in other words what am I trying to achieve via this test?
- The tools and techniques used.
Coming back to our question, why do we never learn to write non-functional test cases with every detailed step there is to it?
It’s because at their very core, test steps for variations on test types on a certain function are all the same, functional or not. It is the intention that makes a difference and may be the method.
We hope that this process of delving deep into this topic have been as much fun to you as it has been presenting it to you all. We would love to hear your feedback and thoughts on this subject.
About the author: This reader’s question is answered by STH team member Swati S.
How do you handle non-functional testing in your teams? And as always, let us know if you agree or disagree or have anything to add to what we have going on here.