Topics we will cover in this article:
– Application Testing
– Categories of Applications
– Application Testing Methodologies
– Application Testing Tools
– Software Test Plan
– Application Testing Cycles
– Application Testing – Best Practices
Application Testing is an activity that every software tester performs daily in his career. These two words are extremely broad in practical aspect. However, only the core and most important areas will be discussed here. The purpose of this article is to touch all the primary areas so that the readers will get all the basic briefing at a single place.
Categories of Applications
For desktop applications, testing should take into account the UI, business logic, database, reports, roles and rights, integrity, usability and data flow. For web applications, along with all these major areas; testers should give sufficient importance to performance, load and security of the application. So AUT is either desktop software or a website.
Application Testing Methodologies
This is a well-known and well discussed aspect; there are only 3 universally accepted methodologies;
a. Black Box: In black-box testing, the AUT is validated against its requirements considering the inputs and expected outputs, regardless of how the inputs are transformed into outputs. Testers are least concerned with internal structure or code that implements the business logic of the application. There are four primary techniques to design test cases for black box testing;
i. BVA (Boundary value Analysis)
ii. EP (Equivalence Partitioning)
iii. Decision Tables
iv. State Transition Tables (and diagrams)
a. White Box: Primary focus of this methodology is to validate, how the business logic of application is implemented by code. Internal structure of the application is tested and the techniques available to do so are;
i. Code Coverage
ii. Path Coverage
Both the above listed techniques contain several other strategies that may be discussed in some other article. Some techniques are discussed in ‘Test Case Design Techniques’ topic.
a. Grey Box: Practically speaking, this is a mixture of black box and white box. In this methodology, mainly the tester tests the application as in black box. But, for some business critical or vulnerable modules of application; testing is done as white box.
Application Testing Tools
According to the best of my knowledge, there are at least 50 testing tools available in market today. These include both paid and open source tools. Moreover, some tools are purpose specific e.g. UI testing, Functional Testing, DB Testing, Load Testing, Performance, Security Testing and Link validation testing etc. However, some tools are strong and provide the facility of testing several major aspects of an application. The general concept of ‘Application Testing’ is its functional testing. So, our focus will be on functional testing tools.
Here is the list of some most important and fundamental features that are provided by almost all of the ‘Functional Testing’ tools.
a. Record and Play
b. Parametrize the Values
c. Script Editor
d. Run (the test or script, with debug and update modes)
c. Report of Run session
Different vendors provide some specific features that make their product unique to other competitor products. But the five features listed above are the most common and can be found in almost all the functional testing tools.
Following is the list of few widely used Functional Testing tools.
Software Test Plan (STP)
For any activity, some planning is always required and same is true for software testing. Without proper plan there is always high risk of getting distracted during the testing. If this risk becomes a fact, the results could be horrible.
Following are the 5 main parts of a good Test Plan:
i. Overview of AUT
ii. Features (or areas) to be tested
iii. Exclusions (features or areas not to be tested) with reason
iv. Dependencies (of testing activities on each other, if any)
b. Objectives: This section describes the goals of testing activity e.g. validation of bug fixes, new features added or revamp of AUT etc.
c. Focus: This section describes what aspect of application will be included in the testing e.g. security, functionality, usability, reliability, performance or efficiency etc.
d. Approach: This section describes what testing methodology will be adopted for which areas of AUT. For example, in the STP of an ERP application; the approach section may contain the information that black box testing will be approach for payroll. On the other hand, for reports the approach will be grey box testing.
e. Schedule: This sections describes that who will be doing what and where on the AUT, when and how. Schedule section is, in fact, a ‘4Ws and H’ of the STP. Normally it is a simple table, but every organization may have its own customized format according to their own needs. Once the test plan is ready and application is under development; testers design and document the test cases. In the “Application Testing – Methodologies” section above, I have listed the TC design techniques.
Once the AUT is ready for testing, the practical phase of testing cycle starts in which testers actually execute the test cases on AUT. Keep in mind that here the testing cycle is discussed regardless of Testing Levels (Unit, Module, Integration, System and User Acceptance) and Testing Environments (Dev, QA, Client’s Replica, Live).
a. Smoke Testing: The very first testing cycle that is wide and shallow in approach. The purpose of smoke testing is to verify that there are no crashes in the application and it is suitable for further testing.
b. Sanity Testing: The second testing cycle that is narrow and deep in its approach. Its purpose is to verify that a specific module is working properly and is suitable for complete testing.
Tip: Usually there is not ample amount of time available to run these two cycles separately. So, a mixture of both these cycles is adopted in practical.
c. Functional Testing: The proper and full fledged testing of application is performed in this cycle. The primary focus of this activity is to verify the business logic of the application.
d. Regression Testing: This is the final cycle of testing in which the bug-fixes and/or updates are verified. Moreover, regression testing also ensures that there is no malfunctioning in other areas of AUT due to fixes and changes.
Bugs are logged in every testing cycle. There is no distinct border line between the testing cycles. For example, in Regression the Functionality is also verified and it may also require smoke, sanity or their merger first.
Application Testing – Best Practices
I think, hundreds of articles are available about this on internet. Every article suggests different number of best practices ranging from 7 to 30 (that I have seen so far). However, I have just 5 tips for readers.
- Plan Properly
- Test Keenly
- Log the bugs Clearly
- Do Regression Test Efficiently
- Improve above four skills Continuously
Conclusion: Application Testing is a vast subject and the primary activity of any software tester. In this article, I have provided the overview of some most fundamental and necessary areas that fall under this topic. Application Testing involves strategies, phenomena, approaches, tools, technologies and guidelines. However, I have addressed the conceptual and practical insight of its salient concerns.