Software Testing Methodologies For Robust Software Delivery

This tutorial explains various Software Testing Methodologies employed in the industry to produce stable and robust software:

Testing is an essential part of the Software Development Process. A robust and stable software product can be delivered with the use of standard testing methodologies that will help to predict the timeline of the software system.

A software application may turn even more complex with a large number of platforms and devices. More importantly, it is required to ensure whether it meets the specified requirements and can be efficiently installed and operated on the user’s machine or not.

Testing methodologies are the methods that are used to test the functional and non-functional requirements of a Product. Each method has its own defined deliverables to ensure that the expected product is delivered to the customer.

Software Testing Methodologies

Methodologies can be considered as the set of testing mechanisms used in the software development lifecycle from Unit Testing to System Testing. Selecting an appropriate testing methodology is considered to be the core of the testing process.

With the means of security, compatibility, and usability, a software product should be tested by using the proper testing methodology. Let us learn about these Testing Methodologies in detail-

Functional Testing

Functional testing is a type of black-box testing. It does not require any knowledge of code. Functional testing is done to test the functionality of the product i.e. the product functions as expected by the customer.

Functionality is verified by providing the input data and verifying that the output is as expected as in the requirement document. Functionality testing covers the functionality of a product, usability i.e. application or product is easy to use, etc.

Types of Functional Testing:

• Unit Testing
• Integration Testing
• System Testing
• Acceptance Testing

Testing is performed in the below order:

Unit Testing

Unit testing is the testing of individual components of a product. This testing is done by the developer itself. Unit test cases are prepared before the test execution by the developers only. It is very important to execute all the unit test cases as it helps to detect the defect at an early stage.

Moreover, if the defect is not detected in unit testing, then it could lead to Major and Critical issues at the later stage which indeed would cost high to fix the same and will also take more time than the estimated time.

While doing unit testing if the developer integrates two components to test to save time and the other component is not yet ready, then the developer uses stubs or drivers to perform his testing. If a defect occurs on the integrated system, then it becomes difficult to find the module, because of which the issue has occurred.

Thus, testing a component individually is very important.

Stubs and drivers are basically used when one component calls the function from another component that is not yet developed, thus to create the test environment for testing of the unit, stubs and drivers are used by the developers.

Stubs

A stub is a dummy code that has the same Input parameters as the actual module but has a simplified behavior. Stub follows a top-down integration approach i.e. stub is used when the sub-modules are not developed.

Example: If there are three modules i.e. module X, module Y, module Z. Module X is developed and it calls a function from module Y & Z but modules Y and Z are under development. In this case, the stub of module Y & Z is created to test module X

Drivers

The driver follows the bottom-up integration approach. It is used when the main module is not ready and the other module needs to be tested. In that case, a dummy is created for module X so that Module Y & Z can call the function from Module X.

Integration Testing

Integration testing is the testing when two or more modules are integrated or merged to test if the product works fine. Modules are integrated in a planned manner as per the integration plan.

Example: The computer which we use has individual components as a Monitor, Screen, Key Board, Mouse. Once the monitor and keyboard are ready, they should be integrated to verify if the output is as expected.

Approaches used in Integration Testing are:

1. Top-Down approach
2. Bottom-up approach
3. Mixed approach
4. Big Bang approach

Top-down approach

In Top-down integration testing, the top-level module is developed and tested first. After that immediate sub-modules are integrated with the top-level module and are tested. Stubs are required to complete the integration testing with a top-level module in case the module to be integrated is not developed and tested.

To test module 2 in integration testing when modules 4 & 5 are not developed, stubs are created to test the same in the top-down approach.

Bottom-up Approach

In the bottom-up approach, sub-modules are developed and tested first and the whole system is tested.

To test Module C, the Driver of module A is created so that function can be called.

Mixed Approach

The mixed approach is a combination of both top-down and bottom-up approaches. In the top-down approach, testing can start only when the top modules are developed and the unit tested.

Same way, bottom-up approach testing, can start only when the modules at the bottom have been developed and tested. The mixed approach overcomes this shortcoming as in the mixed approach, testing can start anytime once the modules are developed.

Big Bang Approach

In the Big bang approach, all the modules are integrated into one go and are tested. A drawback of this method is that if any error is detected while testing, it would be difficult to find the error-causing module. Once the defect cause is detected, it will be fixed but it would cost high & will be time-consuming as the defect found is at a later stage.

System Testing

In system testing, a system as a whole i.e. a completely developed system is tested to find defects. All the modules are integrated to test as a complete system. Scenarios prepared for system testing are run to find the defects.

All the modules of a product are integrated and tested to verify that the built product is as per the customer requirement. End-to-end testing is performed to cover the complete scenarios.

Acceptance Testing

Once the System testing is complete, the Product is ready to be released to the Production environment. But before deployment, the customer’s acceptance and approval are required. The customer verifies whether the product is all that they wanted or not. For this acceptance testing is done.

Types of Acceptance Testing:

Alpha Testing: Alpha testing is done internally by the members of the company who developed the product. This testing is not performed by the developers and testers who developed and tested the product.

Beta Testing: Beta testing is done by the actual users in the Production environment. Beta version of the application is released in the real environment to get feedback from the users which indeed helps to reduce the chances of failure of the product in the production environment.

User Acceptance Testing: Testing is done at the user end wherein the prepared UAT test cases are executed. Replica of the production environment is created for the customer to perform the user acceptance testing.

Non-Functional Testing

Non-functional testing is done to verify the non-functional aspects of a product. E.g. Performance of any product comes under non-functional activity.

Non-functional testing is as important as functional testing is. If it is not done properly, it can lead to major issues in the Product.

Example: If Performance testing is not done for new functionality added to the application, when the application goes live it becomes too slow to work with and the customer gets affected with the same.

Types of Non-functional Testing

• Performance Testing
• Load Testing
• Stress Testing
• Volume Testing
• Security Testing
• Usability Testing
• Compatibility Testing
• Installation Testing
• Uninstallation Testing
• Recovery Testing
• Documentation Testing

Performance Testing

Performance testing is done to verify if the system meets the non-functional requirement identified in the SRS document. It verifies how the system behaves and performs like response time, throughput, etc.

There are various types of Performance testing done as given below:

Load Testing

Load testing is done to check how the system responds when the load is increased on the system. When the load is at a peak, it verifies whether the application works as expected or not?

Basically, the load is constantly increased on the system by increasing the number of users using the same application at the same time till the time the load reaches its threshold.

Example: If the application’s response time is 30 seconds with the number of logged-in users as 1000, then testing is done by increasing the number of users on the site till the number reaches 1000. It verifies that the response time is not increased as the load increases.

Stress Testing

Stress testing is done beyond the system’s capacity to check at which point it fails. In this, invalid or illegal inputs are used to test so as to stress the capabilities of a product. Stress is increased on the system by increasing the number of users till the time the application breaks.

Example: If the application’s response time is 30 seconds with the number of logged-in users as 1000, then testing is done by increasing the number of users on the site till the number reaches (more than 1000 users) wherein the application breaks down. The number of users is increased beyond the capability of the product to handle it.

Volume Testing

Volume testing is done with a huge amount of data to verify the efficiency & response time of the software and also to check for any data loss.

Security Testing

Security testing is one of the important testing methods as security is a crucial aspect of the Product. It is done to verify if the application is secured or not.

Few examples of what is being tested while performing security testing are:

1. Security testing verifies that only authorized users should be able to access data.
2. Authentication is verified i.e. the user should be able to log in with his credentials. An incorrect username or password should not allow the user to log in.
3. Secured and unsecured pages in the application should be verified.
4. For financial sites, the session should timeout after a few minutes if no activity is being performed.

Usability Testing

Usability Testing is done to check how user-friendly the application is. User Interface is checked to verify that it meets the customer requirement. Display screens, messages, navigations, links, orphan pages are a few to name among those that are tested while doing usability testing.

Compatibility Testing

Compatibility testing verifies whether the application/Product is compatible with all the hardware/software platforms or not. The application should be compatible with all browsers, operating systems, mobile devices.

Example: The application might show all the images perfectly in one browser whereas, in another browser, the images are shown distorted. So, it’s important to verify the application on all the browsers and operating systems that the customer has asked for in his requirement.

Installation Testing

Installation testing is one of the most important testings as Installation is the very first interaction of the user with the product and it’s important that the user does not face difficulty in installing the same.

The software can be installed via CD, Internet, and network location.

Example: If an installation is done through the Internet, then test cases should be included for:

• Bad network speed
• Broken connection.
• Size and approximate time taken.
• Concurrent installation/downloads.

The Installation of Software requires the use of License keys. Thus different types of licensing are:

• Node-Locked
• Floating
• Named User
• Temp/Evaluation Licenses

Uninstallation Testing

The purpose of Uninstallation testing is to verify that uninstalling the product does not cause any errors and all the components are removed.

Uninstallation testing ensures that all the data, executables, and DLL files are removed. Uninstallation of the application is verified by checking the software in Add/Remove programs menu, and through the manual deletion of files.

Recovery Testing

Recovery testing is done to verify that the data is recovered if any fault occurs; application crashes or power goes off. To test the recovery of data, the software is forcefully failed to check if data is recovered successfully. E.g. Printer can be disconnected to check if the system hangs or can be shut down to check data loss.

Documentation Testing

Documentation testing is done to verify that all the technical/maintenance documents exist and are consistent. If the requirement specifies the types of audience for which a specific manual is required, and it is checked for such compliance.

Testing Techniques

Basically, there are 3 testing techniques that are used for testing. They are White Box Testing, Black Box Testing, and Grey Box Testing. Each of the testing techniques is briefed below for your better understanding.

#1) White Box Testing

The white box testing technique is used to examine the program structure and business logic, it validates the code or program of an application. It is also called Clear Box Testing, Glass Box Testing, or Open Box Testing.

White Box Testing Techniques include:

• Statement Coverage: Examines all the programming statements.
• Branch Coverage: Series of running tests to ensure if all the branches are tested.
• Path Coverage: Tests all the possible paths to cover each statement and branch.

#2) Black Box Testing

The Black Box testing method is used to test the functionality of an application based on the requirement specification. Unlike White Box Testing it does not focus on the internal structure/code of the application.

Black Box Techniques include:

• Boundary Value analysis
• Equivalence Partitioning(Equivalence Class Partitioning)
• Decision Tables
• Domain Tests
• State Models
• Exploratory Testing(Requires less preparation and also helps to find the defects quickly).

#3) Grey Box Testing

This method of testing is performed with less information about the internal structure of an application. Generally, this is performed like Black Box Testing only but for some critical areas of application, White Box Testing is used.

Models in SDLC

Selecting proper testing methodologies also incorporates choosing a proper model in SDLC.

The models include:

• Waterfall model
• V model
• Agile model
• Spiral model
• RAD

Let’s have closer look at each Software Development Models with a brief explanation.

#1) Waterfall Model

The  Waterfall model is the basic life cycle model which was developed by Winston Royce in 1970. This model represents multiple stages or processes in a sequential manner that flows progressively downward.

This approach is useful when requirements are well known, technology is understood and the resources with required expertise are available.

The waterfall model is defined by the following stages:

• Requirement Gathering and Analysis: Capture and analyze all the requirements and make sure whether they are testable or not.
• System Design: Create and document design based on requirement analysis. Define the hardware and software requirements.
• Implementation: Create robust code for components as per the design and integrate them.
• System Testing: Integrated components form a whole system, this phase is performed to ensure whether the system is working as per the requirements, tracking and reporting the testing progress.
• System Deployment: Make sure if the system is stable with zero bugs, all test criteria had been
  met, ensure Environment Setup, etc.
• System Maintenance: Makes sure if the application is working efficiently as per the requirement with the suitable environment. In case a defect is found then that should be fixed and deployed (updated) in the environment.

Advantages of Waterfall Model:

• Simple and easy to understand.
• Easy to manage as each phase has its own specific deliverables.
• Overlapping of stages is avoided.
• Good for small projects.

Disadvantages of Waterfall Model:

• Increase in the amount of risk and uncertainty.
• Once entered into the Testing phase, cannot change anything in the previous stages e.g Design and Coding, etc.
• Not good for complex and large projects.
• Not suitable where the requirements keep changing.

#2) V Model

V Model is an extension of the Waterfall Model where the process execution takes place in a sequential style in V-Shape and is also known as the Verification and Validation Model. In this approach, there exists a directly associated testing phase in every single phase of the development cycle.

It has been proven beneficial and cost-efficient than the waterfall model as the testing is performed at each development phase rather than at the end of the development cycle.

V Model is classified into 3 Phases.

• Verification Phase
• Coding Phase
• Validation Phase

a) Verification Phase:

• Business Requirement Analysis: Communicate with the customer to understand their expectations and requirements.
• System Design: Design a complete system and its components along with the hardware and software requirements.
• Architectural Design: In this phase architectural specifications are captured. This is also known as high-level Design.
• Module Design: This is also known as Low-Level Design, Detailed internal design for all the specified system modules.

b) Coding Phase:

This phase contains the actual coding phase in the development lifecycle. Programming languages should be chosen based on the system and architectural design specified in the previous phase technology platform. Coding is performed according to the standards and guidelines that are pre-defined.

c) Validation Phase:

• Unit Testing: Performed on an individual module to eliminate the bugs at the early stage.
• Integration Testing: Performed to test the communication between different modules in the system.
• System Testing: System Testing is performed on a system as a whole.
• Acceptance Testing: This is associated with the business requirements. It is performed in a user environment from the user’s point of view.

Advantages of V model

• Simple, easy to use, and understand.
• Overlapping is avoided as phases are executed one at a time.
• Easy to manage and suitable for small projects.

The disadvantages of the V Model are more or less similar to the disadvantages of the Waterfall model.

#3) Agile Model

Agile Model shows an iterative and incremental approach. This approach breaks the product into small incremental units to provide iterations. Then each iteration involves steps like Planning, Requirement Analysis, Design, Coding, Unit Testing, Acceptance Testing, etc.

This approach also allows continuous interaction with the customer for their feedback and corrections in the requirements at regular intervals.

The following diagram will help you to understand the Agile Model approach more precisely:

The following image will show the iteration cycle in the Agile Model:

Advantages of the Agile model:

• A realistic approach to software development.
• Promotes teamwork.
• Eliminates mismatch between requirements and test cases.
• Rapid and requires a minimum amount of resources.
• Suitable for large and long-term projects.
• Good for changing requirements.
• Easy to manage.

Disadvantages of the Agile model:

• Not suitable for complex projects.
• Requires a heavy interaction with the customer which may cause delay.
• Misguidance of requirements may cause the incorrect development of the software product.
• Increased maintainability risk.
• Handover to another team may be quite challenging.

#4) Spiral Model

The spiral model incorporates an iterative development approach along with the systematic approach of the waterfall model. It is similar to the incremental model and emphasizes Risk Analysis.

Spiral Model has four stages:

• Planning Phase
• Risk Analysis
• Engineering Phase
• Evaluation Phase

1) Planning Phase: In this phase, the requirements are gathered and reviewed to finalize the test case.

2) Risk Analysis: This stage includes identifying, monitoring, and estimating management risks. Requirements are analyzed to identify the risks using techniques like brainstorming, walkthrough, etc.

3) Engineering Phase: In this phase, the software is developed and tested at the end.

4) Evaluation Phase: This is the last stage where a customer evaluates the output of a project and gives their feedback for either the next spiral or approval.

Pictorial representation of Spiral Model:

When to use the Spiral model:

• For high-risk projects.
• When the requirements are complex.
• If a project is large.
• Have a sufficient amount of time for getting user’s feedback for the next spiral.
• Requires significant changes due to research and exploration.
• Users are not sure of their needs.

Advantages of Spiral Model:

• Avoidance of risk as it involves a high amount of risk analysis.
• Rapid Development.
• Changes in requirements are accommodated easily.
• Requirements can be acquired more accurately.

Disadvantages of the Spiral model:

• Complex management.
• Not suitable for small projects.
• May involve no. of spirals(indefinite).
• Costly.
• Requires a high amount of risk analysis and expertise for their project’s success.

#5) RAD Model

Rapid Application Development(RAD) is a type of incremental model. In this approach, components are developed in parallel.

This is a rapid approach and it can give a fast product to the customer to provide feedback.

Phases in RAD are as follows:

• Business Modeling: Identifies vital information and its flow between various business channels.
• Data Modeling: Information gathered in the previous stage is used to define data objects required for the business.
• Process modeling: Data objects are converted to get business objectives and flow of information.
• Application Generation: In this phase, automation tools are used to convert the process model into actual code.
• Testing and Turnover: Tests all the components of a system, hence overall testing time is reduced.

 Advantages of RAD Model:

• Progress can be measured.
• Reduces development time.
• Increased reusability.
• Quick initial reviews.
• Enhances customer feedback.

Disadvantages of RAD Model:

• Requires high skilled resources.
• High-cost estimation.
• Not applicable for cheaper projects.
• High dependency on modeling skills.
• Only a modularized system can be built using RAD.

Difference Between Software Testing Methodologies And Strategies

The answer to this is not much complex as there is a simple difference between both.

Testing Methodologies are the methods or approaches to testing that includes Unit testing through System Testing.

Testing Strategies is an overview of the key issues that occur in the testing process and is to be taken into consideration by the project manager, a team of developers, and testers.

Conclusion

Software testing methodologies guide the testing of an application in such a way that the product developed is as expected by the customer. They ensure that every bit of a product or application is what the customer has asked for.

Choosing a proper Testing Methodology is the action or set of actions that lie at the core of the testing process. This may even be a versatile activity that changes according to the business requirements and timeline of the software product.

However, one can choose single or even multiple Software Testing Methodologies to have a more flexible and efficient end product that satisfies the customer’s needs and expectations within the desired time.

Being a tester, the topics discussed may be similar to you, but we believe that this tutorial would have helped you learn the basics of Testing.