Difference Between Test Plan, Test Strategy, Test Case, Test Script, Test Scenario And Test Condition

Understand the difference between Test Plan, Test Strategy, Test Case, Test Script, Test Scenario, and Test Condition with examples.

Software Testing includes several basic and important concepts that every software tester should know.

This article will explain the various concepts in Software Testing along with their comparison.

To make things easier for you, I have included detailed explanations of the differences between Test Plan and Test Strategy, Test Case and Test Script, Test Scenario and Test Condition, and Test Procedure and Test Suite.

=> Click Here For Complete Test Plan Tutorial Series

Question: “We almost have an overload of technical terms when working in an IT environment. Technical terminology is used to identify and address processes, documents, tasks, and all other related elements. What’s the best way to remember, comprehend, and utilize them correctly in every situation?”

The above question asked by Sasi C. is the most often asked in our Software Testing class and I always tell our participants that with experience we hardly notice these words and that they become a part of our vocabulary.

Yet, these terms are commonly misunderstood, so my goal in this article is to provide clear definitions for them.

Understanding the Variances With Examples

Enlisted below are the various Software Testing concepts along with their comparison.

Let’s start!!

Test Plan And Test Strategy

Test Strategy and Test plan are two important documents in the testing life cycle of any project. Here we are trying to give you an in-depth knowledge of test strategy and test plan documents.

Test Plan

A Test Plan is a document that outlines the scope, objective, and approach for testing a software application. The Test Plan is a term used to describe a document that needs to be delivered.

The Test Plan is a document that lists all the activities in a QA project, schedules them, and defines the scope of the project, roles & responsibilities, risks, entry & exit criteria, test objective, and anything else that you can think of.

I like to describe the Test Plan as a ‘super document’ that contains everything you need to know.

Designing the Test Plan will be guided by the requirements. When assigning tasks to the test engineers, a replacement had to be made for one of the testers due to some circumstances. Here, the Test Plan gets updated.

The Test Strategy outlines the testing approach and everything else that surrounds it. It differs from the Test Plan, in the sense that a Test Strategy is only a subset of the test plan. The document is both hardcore and somewhat generic and static. People argue about when to employ test strategy or plan – but I fail to see any significant variation.

Example: The Test Plan gives information about who is going to test at what time. A designated “X tester will test module 1,” for instance. If tester Y replaces X, the test plan must be revised.

Test Plan Document

Test Plan is a document that provides complete information about testing tasks related to a Software Project. It provides details like the scope of the testing, types of testing, objectives, test method, testing effort, risks & contingencies, release criteria, test deliverables, etc. It monitors the tests scheduled for execution after coding.

The test plan is subject to change. Initially, a draft test plan will be developed based on project clarity. This initial plan will be modified as the project progresses. The test team manager or test lead can prepare the test plan document. It describes the specifications and is subject to change based on the same.

What to test, when to test, who will test, and how to test will be defined in the test plan. Test Plan will sort out a list of issues, dependencies, and underlying risks.

Types Of Test Plans

Test Plans can be of different types based on the stage of testing. Initially, there will be a master test plan for the entire project execution. Testers have the option to create separate test plans for specific testing types like system testing, system integration testing, user acceptance testing, etc.

Another approach is to have separate test plans for functional and non-functional testing. In this approach performance, testing will have a separate test plan.

Contents of Test Plan Document (IEEE-829 test plan structure)

It is difficult to draw a clear format for the test plan. The test plan format may vary depending on the project at hand. IEEE has defined a standard for test plans, which is described as the IEEE-829 test plan structure.

Please find below IEEE recommendations for a standard test plan content:

1. Test Plan Identifier
2. Introduction
3. Test Items
4. Software Risk Issues
5. Features to be tested
6. Features not to be tested
7. Approach
8. Item Pass/Fail Criteria (or) Acceptance Criteria
9. Suspension Criteria and Resumption Requirements
10. Test Deliverables
11. Test Tasks
12. Environmental Requirements
13. Staffing and Training needs
14. Responsibilities
15. Schedule
16. Approvals

Suggested Read => Test Plan Tutorial – A Perfect Guide

Test Strategy

Test Strategy is a set of guidelines that explain the test design and determine how testing needs to be done.

Example: A Test Strategy includes details like “Individual modules are to be tested by the test team members”. It doesn’t matter who tests it, so it can remain generic and the change in team members can stay static.

Test Strategy Document

The purpose of the test strategy is to define the testing approach, the types of tests, test environments, and tools to be used for testing, and the high-level details of how the test strategy will be aligned with other processes. The test strategy document is a living document and will be updated** when we get more clarity on requirements, SLA parameters, test environment build management approach, etc.

Test strategy is intended for the complete project team that comprises Project Sponsors, Business SMEs, Application/ Integration Development, System Integration Partners, Data Conversion Teams, Build/Release Management Teams such as technical leads, architecture leads, and deployment and infrastructure teams.

** Some people believe that once defined, the test strategy should never be changed. Testing projects generally undergo updates as the project advances.

Below are the important sections that a test strategy document should have:

#1) Project Overview

This section can begin by summarizing the organization, followed by a brief description of the project at hand. It can include the below details.

• What was the need for the project?
• What objectives the project will attain?

Table of Acronyms: It is better to include a table with acronyms that the document reader might come up with while referring to the document.

#2) Requirements Scope

Requirement scope can include Application Scope and Functional scope

Application Scope defines the system under test and the impact on the system because of new or changed functionality. Related systems can also be defined.

Functional Scope defines the impact on different modules within the system. Each system related to functionality will be explained here.

#3) High-Level Test Plan

The Test Plan is a separate document. In the test strategy, a high-level test plan can be included. A high-level test plan can include test objectives and test scope. Test scope should define both in-scope and out-of-scope activities.

#4) Test Approach

This section outlines the testing approach that the team will follow during the testing life cycle.

As per the above diagram, testing will be conducted in two phases, i.e. Test Strategy & Planning and Test Execution. The Test Strategy & Planning phase will be one time for an overall program, whereas Test execution phases will be repeated for each cycle of the overall program.

The above diagram shows different stages and deliverables (outcomes) in each phase of the execution approach.

The test approach should include the below sub-sections:

a) Test Schedule: Explain the proposed project timeline in this subsection.

b) Functional Testing Approach: Using this subsection summarize each phase and the respective entry and exit criteria. Different testing phases are Unit Testing, System Testing, System Integration testing, User Acceptance Testing, and End-to-End testing.

c) Testing Key Performance Indicators:

• Test Case Prioritization: Define the test case prioritization approach so that, with time constraints, the test team can execute high-priority scenarios. There should be an agreement between project stakeholders regarding the risks involved in not executing all the planned scenarios.
• Defect Prioritization: Defect prioritization strategy is the next topic to cover here. Define priority level and describe each level as critical, high, medium, etc.
• Defect Turnaround Time: Defect turnaround time is defined as the time between when the defect was first raised and when the defect is fixed and comes for a retest. Quick turnaround ensures speedy testing and adherence to the project timeline. Define the turnaround time for each level of defect priority.

#5) Test Coverage

This section describes the processes that the QA team will follow to optimize the coverage of business/functional requirements in test scenarios and test cases. Requirement Traceability Matrix (RTM) can trace all the requirements with respective test scenarios and test cases.

#6) Test Environment

Define the different QA environments available. Specify which testing will be done in each environment and by whom. Create an environment backup plan to take care of emergencies. Access to each environment should be regulated and called out with clarity.

Testing tools that are going to be used also can be mentioned in this section.

#7) QA Deliverables and Metrics

Listed below are all the QA deliverables:

List of all the QA metrics:

#8) Defect Management

Clearly define a defect management strategy by creating a defect workflow, defect tracking methodology, & defect triage process. Mention defect responsibility for each tester’s roles. Periodic defect analysis and root cause analysis will improve the overall quality of testing.

#9) Communication Management

Set guidelines for status reports, status meetings, and onsite-offshore communication.

#10) Assumptions, Risks, and Dependencies

Describe the assumptions on which the project is based. These may include timing, resources, and system capabilities. Describe any dependencies such as other projects, availability of temporary resources, and other deadlines that may impact the project.

#11) Appendix

Include stuff like roles & responsibilities, work time zone, and references in this section.

Further reading => Guide to Writing a Good Test Strategy document.

Test Plan Vs Test Strategy

The difference between these two documents is subtle. A test strategy is a high-level static document about the project. On the other hand, the test plan will specify what to test, when to test, and how to test.

Test Case And Test Script

These two terms can be used interchangeably. Yes, I am saying there is no difference. The test case is a sequence of steps that help us perform a certain test on the application. The test script is also the same thing.

Now, there is one school of thought that a test case is a term used in the manual testing environment, and a test script is used in an automation environment. This is partly true, because of the comfort level of the testers in the respective fields and also of how the tools refer to the tests (some call test scripts and some call them test cases).

So in effect, test script and test case both are steps to be performed on an application to validate its functionality, whether manually or through automation.

Further Reading => How to Write Effective Test Cases? and Test Case Example Template.

Test Scenario And Test Condition

Test Scenario is a way to define all the ways to test an application. It is a single statement to cover all ways to test an application.

Test Condition is the specification that a tester must follow for testing an application.

This is a one-line pointer that testers create as an initial, transitional step into the test design phase. This is mostly a one-line definition of “What” we are going to test concerning a certain feature. Usually, test scenarios are input for the creation of test cases.

In agile projects, test scenarios are the only test design outputs and no test cases are written following these. A test scenario might result in multiple tests.

Examples of Test Scenarios:

• Validate if the Admin can add a new country
• Validate if the Admin can delete an existing country
• Validate if an existing country can be updated

Test conditions are more specific. It can be approximately defined as the target of a given test.

Example Test Condition: In the above example, if we were to test the scenario 1, we can test the following conditions:

• Enter the country’s name as “India”(valid)and check for the addition of the country
• Enter a blank and check if the country gets added.
• In each case, the specific data is described and the goal of the test is much more precise.

Further reading => 180+ Sample Test Scenarios for Testing Web and Desktop Application

Test Procedure And Test Suite

The test procedure is a combination of test cases based on a certain logical reason, like executing an end-to-end situation or something to that effect. The order in which the test cases are to be run is fixed.

Test Procedure: It is nothing but the Test Life Cycle. There are 10 steps in the Testing Life Cycle.

They are:

1. Effort Estimation
2. Project Initiation
3. System Study
4. Test Plan
5. Design Test Case
6. Test Automation
7. Execute Test Cases
8. Report Defects
9. Regression Testing
10. Analysis and Summary Report

To illustrate, if I wanted to assess the email-sending functionality on Gmail.com, the sequence of test cases I would merge to create a test procedure is as follows:

1. The test to check the login
2. The email composition test
3. The test to attach one/more attachments
4. Formatting the email in the required way by using various options
5. Adding contacts or email addresses to the To, BCC, CC fields
6. Emailing and making sure it is shown in the “Sent Mail” section

All the test cases above are grouped to achieve a certain target at the end of them. Also, test procedures have a few test cases combined at any point in time.

The Test suite is the list of all the test cases that have to be executed as a part of a test cycle or a regression phase, etc. There is no logical grouping based on functionality. The order in which the constituent test cases get executed may or may not be important.

Test Suite: The Test Suite is a container that has a set of tests that helps the testers to execute and report the test execution status. It can take any of the three states, i.e. Active, in progress, and completed.

Example of the Test Suite: If an application’s current version is 2.0. The previous version 1.0 might have had 1000 test cases to test it entirely. For version 2, there are 500 test cases to just test the new functionality that is added in the new version.

So, the current test suite would be 1000+500 test cases that include both regression and the new functionality. The suite is a combination too, but we are not trying to achieve a target function.

Test suites can contain 100s or even 1000s of test cases.

Also Read =>> Comprehensive Overview of White Box Testing

Conclusion

Software Testing Concepts play a major role in the Software Testing Life Cycle.

A clear understanding of the above-discussed concepts, along with their comparison, is very important for every Software Tester to carry out the testing process effectively.

Usually, articles like these are excellent starting points for deeper discussions. Please contribute your thoughts, agreements, disagreements, and anything else in the comments below. We look forward to your feedback.

We also welcome your questions about software testing or anything related to your testing career. We will address these in more detail in our upcoming posts in the same series.

Happy Reading!!

=> Visit Here For Complete Test Plan Tutorial Series

PREV Tutorial | NEXT Tutorial