Topics we will cover in this article:
– Application Testing
– Categories of Applications
– Application Testing Methodologies
– Application Testing Tools
– Software Test Plan
– Application Testing Cycles
– Application Testing – Best Practices
Introduction to Application Testing
Application Testing is such an activity that is performed frequently by almost every software tester in his career. These two words are extremely broad in practical aspect. However, only the core and most important areas will be discussed here. The purpose of this article is to touch all the primary areas so that the readers will get all the basic briefing at a single place.
In a one-liner, we can say that application testing is a process through which the functionality, usability, and consistency of the entire application are tested.
What You Will Learn:
Whether it is small calculator software with only the basic arithmetic operations or an online enterprise solution; there are three categories of applications:
For desktop applications, testing should take into account the UI, business logic, database, reports, roles and rights, integrity, usability, functionality, performance, security, hardware & software compatibility and data flow.
For web applications, testers should give sufficient importance to performance, load, and security of the application. Other main testing types covered under web application testing are functional testing, cross-browser testing, UAT, Beta testing, regression testing, compatibility testing, smoke testing, exploratory testing, compatibility & Multilanguage support testing and stress testing.
For mobile applications, the main types of testing that should be done are UI testing, Rule-based testing, regression, functional and security testing.
So AUT (application under test) is either desktop software or a website or a mobile app.
This is a well-known and well-discussed aspect that there are only 3 universally accepted testing methodologies:
a. Black Box: In black-box testing, the AUT is validated against its requirements considering the inputs and expected outputs, regardless of how the inputs are transformed into outputs. Testers are least concerned with internal structure or code that implements the business logic of the application.
There are four primary techniques to design test cases for black box testing:
i. BVA (Boundary Value Analysis)
ii. EP (Equivalence Partitioning)
iii. Decision Tables
iv. State Transition Tables (and diagrams)
Black box testing is commonly employed for functional, non-functional and regression testing.
b. White Box: Primary focus of this methodology is to validate how the business logic of the application is implemented by code/program.
The internal structure of the application is tested here and the techniques available to do so are:
Both the above-listed techniques contain several other strategies that may be discussed in some other article. Some techniques are discussed in ‘Test Case Design Techniques’ topic.
c. Grey Box: Practically speaking, this is a mixture of the black box and white box.
In this methodology, mainly the tester tests the application with black box approach. But, for some business critical or vulnerable modules of an application, testing is done through a white box.
There are a lot of application testing tools available in the market today. These include both paid and open source tools. Moreover, some tools are purpose specific e.g. UI testing, Functional Testing, DB Testing, Load Testing, Performance, Security Testing and Link validation testing etc. However, some tools are strong enough to provide the facility of testing several major aspects of an application. The most important concept in ‘Application Testing’ is functional testing. So, our focus will be on functional testing tools.
Here is the list of some most important and fundamental features that are provided by almost all of the ‘Functional Testing’ tools.
Different vendors provide some specific features that make their product unique to other competitor products. But the five features listed above are the most common and can be found in almost all the functional testing tools.
Following is the list of few widely used Functional Testing tools.
For any activity, some planning is always required and same is true for software testing. Without a proper plan, there is always a high risk of getting distracted during the testing. If this risk becomes a fact, the results could be horrible.
b. Objectives: This section describes the goals of testing activity e.g. validation of bug fixes, new features added or revamp of AUT, etc.
c. Focus: This section describes what aspect of application will be included in the testing e.g. security, functionality, usability, reliability, performance or efficiency etc.
d. Approach: This section describes what testing methodology will be adopted for which areas of AUT. For example, in the STP of an ERP application; the approach section may contain the information that black box testing will be the approach for payroll. On the other hand, for reports, the approach will be grey box testing.
e. Schedule: This section describes that who will be doing what, where, when and how on the AUT. Schedule section is, in fact, a ‘4Ws and 1H’ of the STP. Normally, the schedule is prepared as a simple table, but every organization may have its own customized format according to their own needs.
Once the test plan is ready and application is under development, testers do design and document the test cases. In the “Application Testing – Methodologies” section above, I have listed the TC design techniques.
Once the AUT is ready for testing, the practical phase of the testing cycle starts in which testers actually execute the test cases on AUT. Keep in mind that here the testing cycle is discussed regardless of Testing Levels (Unit, Module, Integration, System and User Acceptance) and Testing Environments (Dev, QA, Client’s Replica, Live).
a. Smoke Testing: This is the very first testing cycle. The purpose of smoke testing is to verify that there are no crashes in the application and it is suitable for further testing. This step is wide and shallow.
b. Sanity Testing: This is the second testing cycle. Its purpose is to verify that a specific module is working properly and is suitable for complete testing. This step is narrow and deep.
Tip: Usually there is not ample amount of time available to run the above two cycles separately. So, a mixture of both these cycles is adopted in practice.
c. Functional Testing: The proper and full-fledged testing of application is performed in this application test cycle. The primary focus of this activity is to verify that the business logic of the application is working as expected.
d. Regression Testing: This is the final application cycle. Here the bug-fixes and/or updates are verified. Moreover, regression testing also ensures that there is no malfunctioning in other areas of AUT due to fixes and changes.
Bugs are logged in every testing cycle. There is no distinct borderline between the testing cycles. For example, in Regression, the Functionality is also verified and it may also require smoke, sanity or their merger first.
We have talked above about four different application testing cycles. We also need to understand here that each application test cycle has certain steps involved in it. Generally, any testing cycle has the steps as presented in below image:
I think hundreds of articles are available about this on the internet. Every article suggests a different number of best practices ranging from 7 to 30 (that I have seen so far). However, I have just 5 tips for readers.
Furthermore, you should prioritize the test cases well and cover the main business flows first.
Application Testing is a vast subject and it is the primary activity for almost all the software testers. In this article, I have provided the overview of most fundamental and necessary areas that fall under application testing. It involves strategies, phenomena, approaches, tools, technologies, and guidelines. I have addressed the conceptual and practical insight of application testing along with its most prominent areas of concern.