Is Non-functional Testing Always Carried out without Documentation and Test Cases? Why?

“We are always taught how to write functional test cases. Why is that? Is ‘non-functional testing’ carried out without documentation (in other words, on an ad-hoc basis) or is that a separate process that is much more difficult to understand? How are test cases written for different kinds of testing that happens on the application?”

This is one of the most original, distinctive and out-of-the-box questions I have been asked in the recent times. Thus, this is the topic for today and this is what we are going to illuminate in the following article.

How come we never get to see and practice writing non-functional test cases?

non functional testing

Let’s start with what we know and as always a practical scenario.

Functional and Non-functional Testing:

Example: The following are the steps to be performed on an Online Banking application in order to perform a transfer. Let us use that as our test of reference.

  1. Login to the site
  2. Choose the bank account
  3. Choose the payee (this payee could belong to the same bank or a different one- this depends on your data choice to execute this step. In any case, choose one. Also, we are going to assume that the payee is already added.)
  4. Enter the amount to be transferred (positive value, with in limit, correct format, etc.)
  5. Click Transfer and check if the confirmation is received, account balance has been updated and all that.

This is the functional test case, correct?

On the very application, on the very transfers’ page, let us say we are performing Performance, Security and Usability Testing. These are non-functional testing types, correct? How would we write the test cases?

#1. Usability Testing

Usability testing is a genre of software testing that deals with the user experience. These are few of the questions that we it tries to answer.

  • How easy is the application to use?
  • How satisfying is the experience of using the system?
  • If not right familiar right away, how easy is to learn?

More info on it is here: Usability Testing Guide

How would a user determine the answers for the above questions in the context of usability testing?

The user would perform the exact same steps to make as in the functional test case. Am I right?

------------

#2. Performance Testing

There are very many variations to performance testing, but at its core, it is used to get statistics about the system, its resource utilization, response time, network consumption etc. at various load points.

Check out to know more about it at: Performance Testing Tutorials

Now, If I were to test the performance of the transfers’ transaction I would have 10, 20, 30, 100…1000…etc users perform transfer operation simultaneously or incrementally depending on what I want to target and gather data about..

What steps would each user perform in order to use the transfers while the performance test is in progress?

The same exact steps as the functional test, correct?

#3. Security Testing

Security Testing is a branch of QA that helps make the software systems hack proof. It involves identifies vulnerabilities(potential problem areas in the software system), exploiting them through penetration or white hat testing technique and when loop-holes are find, they are worked on.

Find more information at: Security testing Guide

When I want to check if transfers are hack proof and are directed correctly to the intended recipients and that there are no black spots in the entire process? I would perform the transfer while the monitoring for security leaks goes on in parallel.

Therefore, in effect I am carrying out the exact same steps I would normally do in case of a functional test case.

I guess we have enough to establish that the steps in all the situations are the same. The method and the intention behind the process is what different.

Let’s take a comparative look:

Type of TestingWho?Why? Intention
Functional testingQA testersAccuracy
Efficiency
Business applicability
UsabilityQA testers or real time usersEase of use
Ease of learning
Efficiency
PerformancePerformance testers via a tool that treats the operation as a transaction performed by a certain number of concurrent users while the tester is analyzing all the logisticsResponse time
Network usage etc.
SecurityScanning tools and other monitoring system by specialized security expertsHack safe
Payee and the payer identity protection etc.

What is interesting to note is that, no matter what form of testing we want to do, all the steps are the same. The real difference is that:

  1. Who performs these steps?
  2. What is the intention, or in other words what am I trying to achieve via this test?
  3. The tools and techniques used.

In conclusion:

Coming back to our question, why do we never learn to write non-functional test cases with every detailed step there is to it?

It’s because at their very core, test steps for variations on test types on a certain function are all the same, functional or not. It is the intention that makes a difference and may be the method.

We hope that this process of delving deep into this topic have been as much fun to you as it has been presenting it to you all. We would love to hear your feedback and thoughts on this subject.

About the author: This reader’s question is answered by STH team member Swati S.

How do you handle non-functional testing in your teams? And as always, let us know if you agree or disagree or have anything to add to what we have going on here.

Recommended reading

13 comments ↓

#1 Varun patil

thanks for sharing.

here is my view – write standard test cases based on non functional requirements. if no requirements, create questions and get answers to have at least something to compare.

something is better than nothing. isn’t it?

#2 Alberico Lima

I agree with Varun, we always can go throught a checklist to verify some of the non-functional aspects. Those checklists can be considered a kind of test specification.

#3 vinayak jadhav

@varun, @alberico

I think auther has given the right answer..otherwise we dont need context driven testers..

#4 LivingTester

Ad-hoc-testing can do more and its good maintaining some test cases for Non-Functional test to check out. Or test it with Effective manner……XD

#5 Thanh

We are making things complicated by talking about functional vs non-functional and test cases vs no test cases.

Why don’t just call it as “things” to test and what we actually need is test ideas/test scenarios. There are some cases when it’s hard to right a step by step but it’s fine as long as we know what test we need to perform.

#6 Somesh Bose

Do you feel Exhaustive testing is possible ?
A-Yes
B-No

#7 Swati Seela

@Varun patil: Completely agree…thanks for stopping by!

#8 Swati Seela

@ LivingTester: Ad-hoc is a great option. Its only flip side is that it is not a 100% measurable and controllable…

#9 Swati Seela

@Thanh: I believe, you and I are both keeping things simple..except that you call them ‘things to test’ and I call them ‘test cases’..its all in the name :)

#10 Swati Seela

@ Alberico Lima, @vinayak: thank you for your valuable comments and readership

#11 Swati Seela

@somesh: I would say “No” even though I would like to be optimistic and say “Yes”. Exhaustive testing needs indefinite amount of time..which no one has :)

Thanks for stopping by!

#12 farukh

hi,
Can you please teach me, how to I create user follow chat for an e-commerce website ?

#13 Tester

Thats the reason we need to understand the TMM process.

Leave a Comment