A strategy plan for defining the testing approach, what you want to accomplish and how you are going to achieve it.
This document removes all uncertainty or vague requirement statements with a clear plan of approach for achieving the test objectives. Test strategy is one of the most important documents for the QA team.
What You Will Learn:
Writing a Test Strategy effectively is a skill every tester should achieve in their career. It initiates your thought process
It helps test managers to get the clear state of the project at any point. The chances of missing any test activity are very low when there is a proper test strategy in place.
Test execution without any plan rarely works. I know teams who write strategy document but never refer it back while test execution. Testing strategy plan must be discussed with the whole team so that the team will be consistent with approach and responsibilities.
In tight deadlines, you can’t just waive any testing activity due to time pressure. At least it must go through a formal process before doing so.
What is a Test Strategy?
Test strategy means “How you are going to test the application?” You need to mention the exact process/strategy that you are going to follow when you will get the application for testing.
I see many companies follow test strategy template very strictly. Even without any standard template, you can keep this test strategy document simple but still effective.
Test Strategy vs. Test Plan
Over the years, I see a lot of confusion between these two documents. So let’s start with basic definitions. Generally, it doesn't matter which comes first. Test planning document is a combination of strategy plugged with an overall project plan. According to IEEE Standard 829-2008, strategy plan is a sub-item of a test plan.
Every organization has their own standards and processes to maintain these documents. Some organizations include strategy details in test plan itself (here is a good example of this). Some organizations list strategy as a subsection in a testing plan but details are separated out in different test strategy document.
Project scope and test focus are defined in the test plan. Basically, it deals with test coverage, features to be tested, features not to be tested, estimation, scheduling and resource management.
Whereas the test strategy defines guidelines for test approach to be followed in order to achieve the test objectives and execution of test types defined in the testing plan. It deals with test objective, approach, test environment, automation strategy and tools, and risk analysis with a contingency plan.
To summarize test plan is a vision of what you want to achieve and test strategy is an action plan designed to achieve this vision!
Hope this will clear all your doubts. James Bach has more discussion on this topic here.
The process to develop a good test strategy document
Don't just follow the templates without understanding what works best for your project. Every client has its own requirements and you must stick to the things that work perfectly for you. Do not copy any organization or any standard blindly. Always make sure if that is helping you and your processes.
Below is a sample strategy template that will outline what should be covered in this plan along with some examples to illustrate what makes sense to cover under each component.
Test Strategy in STLC:
Common sections of test strategy document
Step #1 – Scope and Overview:
Project overview along with information on who should use this document. Also, include details like who will review and approve this document. Define testing activities and phases to be carried out with timelines with respect to overall project timelines defined in the test plan.
Step #2 – Test Approach:
Define testing process, level of testing, roles, and responsibilities of every team member. For every test type defined in test plan (Example: unit, integration, system, regression, installation/uninstallation, usability, load, performance, and security testing) describe why it should be conducted along with details like when to start, test owner, responsibilities, testing approach and details of automation strategy and tool if applicable.
In test execution there are various activities like adding new defects, defect triage, defect assignments, re-testing, regression testing and finally test sign-off. You must define exact steps to be followed for each activity. You can follow the same process which worked for you in your previous test cycles.
A Visio presentation of all these activities including a number of testers and who will work on what activity is very helpful to quickly understand roles and responsibilities in the team.
Example: defect management cycle – mention the process to log the new defect. Where to log in, how to log new defects, what should be the defect status, who should do defect triage, whom to assign defects after triage etc.
Also, define change management process. This includes defining change request submission, template to be used, and process to handle the request.
Step #3 – Test Environment:
Test environment setup should outline information about a number of environments and required setup for each environment. Example: one test environment for functional test team and another for UAT team. Define the number of users supported on each environment, access roles for each user, software and hardware requirements like operating system, memory, free disk space, number of systems etc.
Defining test data requirements is equally important. Provide clear instruction on how to create test data (either generate data or use production data by masking fields for privacy).
Define test data backup and restore strategy. Test environment database may run into problems due to unhandled conditions in the code. I remember the problems we faced on one of the projects when there was no database backup strategy defined and we lost whole data due to code issues.
Backup and restore process should define who will take backups when to take a backup, what to include in backup when to restore the database, who will restore it and data masking steps to be followed if the database is restored.
Step #4 – Testing Tools:
Define test management and automation tools required for test execution. For performance, load and security testing describe the test approach and tools required. Mention whether it is open source or commercial tool and how many users are supported on it and plan accordingly.
Whether you are doing manual or automated testing (or a combination) a test management tool like TestRail can help you organize the team and increase productivity.
For example, you can record details about test cases or scenarios with screenshots and expected results, and then set up test plans and test runs. When testing starts, track the status of individual tests and measure progress with informative dashboards and activity reports.
Deliver fast feedback on application quality with TestRail’s powerful reports and metrics. Track team workload to adjust assignments and resources, and work more productively with personalized to-do lists, filters, and email notifications.
Step #5 – Release Control:
As mentioned in our last UAT article, unplanned release cycle could result in different software versions on test and UAT environments. Release management plan with proper version history will ensure test execution of all modifications in that release.
Example: set build management process which will answer – where new build should make available, where it should be deployed, when to get the new build, from where to get the production build, who will give the go, the no-go signal for production release etc.
Step #6 – Risk Analysis:
List all risks that you envision. Provide a clear plan to mitigate these risks and also a contingency plan in case if you see these risks in reality.
Step #7 – Review and Approvals:
When all these activities are defined in test strategy plan it needs to be reviewed for sign-off by all entities involved in project management, business team, development team, and system administration (or environment management) team.
Summary of review changes should be tracked at the beginning of the document along with approver name, date and comment. Also, it's a living document meaning this should be continuously reviewed and updated with the testing process enhancements.
Simple Tips to Write Test Strategy Document
1) Include product background in test strategy document. In the first paragraph of your test strategy document answer – Why stakeholders want to develop this project? This will help to understand and prioritize things quickly.
2) List all important features you are going to test. If you think some features are not part of this release then mention those features under “Features not to be tested” label.
3) Write down the test approach for your project. Clearly, mention what types of testing you are going to conduct?
I.e. Functional testing, UI testing, Integration testing, Load/Stress testing, Security testing etc.
4) Answer questions like How you are going to perform functional testing? Manual or automation testing? Are you going to execute all test cases from your test management tool?
5) Which bug tracking tool you are going to use? What will be the process when you will find a new bug?
6) What are your test entry and exit criteria?
7) How will you track your testing progress? What metrics are you going to use for tracking test completion?
8 ) Task distribution – Define roles and responsibilities of each team member.
9) What documents will you produce during and after the testing phase?
10) What all risk you see in test completion?
Test strategy is not a piece of paper. It’s the reflection of whole QA activities in software testing life cycle. Refer this document time to time in test execution process and follow plan till the software release.
When the project nears the release date it's fairly easy to cut on testing activities by ignoring what you have defined in test strategy document. But it is advisable to discuss with your team whether or not cutting down on any particular activity will help for release without any potential risk of major issues post-release.
Most of the agile teams cut down on writing strategy document as team focus is on test execution rather than documentation. But having a basic test strategy plan always helps to clearly plan and mitigate risks involved in the project. Agile teams can capture and document all high-level activities to complete test execution on time without any issues.
I'm sure developing a good test strategy plan and committing to follow it will definitely improve the testing process and quality of the software. It would be my pleasure if this article inspires you to write a test strategy plan for your project!
If you like this post please consider sharing it with your friends!