What is Test Strategy?
A strategy plan for defining testing approach, what you want to accomplish and how you are going to achieve it.
This document removes all uncertainty or vague requirement statements with a clear plan of approach for achieving the test objectives. Test strategy is one of the most important documents for QA team. Writing it effectively is a skill every tester should achieve in their career.
It initiates your thought process which helps to discover many missing requirements. Thinking and test planning activities help team to define testing scope and test coverage. It helps test managers to get clear state of the project at any point. The chances of missing any test activity are very low when there is a proper test strategy in place.
Test execution without any plan rarely works. I know teams who write strategy document but never refer it back while test execution. Testing strategy plan must be discussed with whole team so that the team will be consistent on approach and responsibilities. In tight deadlines, you can’t just waive any testing activity due to time pressure. At least it must go through a formal process before doing so.
Test Strategy vs. Test Plan:
Over the years, I see a lot of confusion between these two documents. So let’s start with basic definitions. Generally it doesn’t matter which comes first. Test planning document is a combination of strategy plugged with overall project plan. According to IEEE Standard 829-2008, strategy plan is a sub item of test plan.
Every organization has their own standards and processes to maintain these documents. Some organizations include strategy details in test plan itself (here is good example of this). Some organizations list strategy as a subsection in testing plan but details are separated out in different test strategy document.
Project scope and test focus is defined in test plan. Basically, it deals with test coverage, features to be tested, features not to be tested, estimation, scheduling and resource management.
Whereas test strategy defines guidelines for test approach to be followed in order to achieve the test objectives and execution of test types defined in testing plan. It deals with test objective, approach, test environment, automation strategy and tools, and risk analysis with contingency plan.
To summarize test plan is a vision of what you want to achieve and test strategy is an action plan designed to achieve this vision!
Hope this will clear all your doubts. James Bach has more discussion on this topic here.
Process to develop a good test strategy document:
Don’t just follow the templates without understanding what works best for your project. Every client has its own requirements and you must stick to the things that work perfectly for you. Do not copy any organization or any standard blindly. Always make sure if that is helping for you and your processes.
Below is a sample strategy template that will outline what should be covered in this plan along with some examples to illustrate what make sense to cover under each component.
Test Strategy in STLC:
Common sections of test strategy document:
Step #1 – Scope and Overview:
Project overview along with information on who should use this document. Also include details like who will review and approve this document. Define testing activities and phases to be carried out with timelines with respect to overall project timelines defined in test plan.
Step #2 – Test Approach:
Define testing process, level of testing, roles and responsibilities of every team member. For every test type defined in test plan (e.g. unit, integration, system, regression, installation/uninstallation, usability, load, performance, and security testing) describe why it should be conducted along with details like when to start, test owner, responsibilities, testing approach and details of automation strategy and tool if applicable.
In test execution there are various activities like adding new defects, defect triage, defect assignments, re-testing, regression testing and finally test sign-off. You must define exact steps to be followed for each activity. You can follow same process which worked for you in your previous test cycles. A visio presentation of all these activities including number of testers and who will work on what activity is very helpful to quickly understand roles and responsibilities in the team.
E.g. defect management cycle – mention the process to log new defect. Where to log it, how to log new defects, what should be the defect status, who should do defect triage, whom to assign defects after triage etc.
Also define change management process. This includes defining change request submission, template to be used, and process to handle the request.
Step #3 – Test Environment:
Test environment setup should outline information about number of environments and required setup for each environment. E.g. one test environment for functional test team and another for UAT team. Define number of users supported on each environment, access roles for each user, software and hardware requirements like operating system, memory, free disk space, number of systems etc.
Defining test data requirements is equally important. Provide clear instruction on how to create test data (either generate data or use production data by masking fields for privacy).
Define test data backup and restore strategy. Test environment database may run into problems due to unhandled conditions in the code. I remember the problems we faced on one of the projects when there was no database backup strategy defined and we lost whole data due to code issues. Backup and restore process should define who will take backups, when to take backup, what to include in backup, when to restore database, who will restore it and data masking steps to be followed if database is restored.
Step #4 – Testing Tools:
Define test management and automation tools required for test execution. For performance, load and security testing describe the test approach and tools required. Mention whether it is open source or commercial tool and how many users are supported on it and plan accordingly.
Step #5 – Release Control:
As mentioned in our last UAT article, unplanned release cycle could result into different software versions on test and UAT environments. Release management plan with proper version history will ensure test execution of all modifications in that release.
E.g. set build management process which will answer – where new build should made available, where it should be deployed, when to get new build, from where to get the production build, who will give go, no-go signal for production release etc.
Step #6 – Risk Analysis:
List all risks that you envision. Provide a clear plan to mitigate these risks and also a contingency plan in case if you see these risks in reality.
Step #7 – Review and Approvals:
When all these activities are defined in test strategy plan it needs to be reviewed for sign-off by all entities involved like project management, business team, development team and system administration (or environment management) team. Summary of review changes should be tracked at the begging of the document along with approver name, date and comment. Also it’s a living document meaning this should be continuously reviewed and updated with the testing process enhancements.
Test strategy is not a piece of paper. It’s the reflection of whole QA activities in software testing life cycle. Refer this document time to time in test execution process and follow plan till the software release. When project nears the release date it’s fairly easy cut on testing activities by ignoring what you have defined in test strategy document. But it is advisable to discuss with your team whether or not cutting down on any particular activity will help for release without any potential risk of major issues post release.
Most of the agile teams cut down on writing strategy document as team focus is on test execution rather than documentation. But having a basic test strategy plan always help to clearly plan and mitigate risks involved in the project. Agile teams can capture and document all high level activities to complete test execution on time without any issues.
I’m sure developing a good test strategy plan and committing to follow it will definitely improve testing process and quality of the software. It would be my pleasure if this article inspires you to write a test strategy plan for your project!
If you like this post please consider sharing it with your friends!