Test Management Tutorial: An Ultimate Guide To Test Management

This is a Test Management Tutorial for Software Testing. It includes Test Management Phases, Tools and Test Management Vs Organizational Structure:

Test Management is the process of managing all the test related activities, documents, and other related work. Organizational Structures refer to a hierarchy of teams or employees working on particular projects.

Do you think organizational structure affects test management?

If your answer is no, we will see why? If yes, let’s see how it affects. To find the relation between these two, we need to understand these topics clearly and then explore the relationship between Test Management and Organizational Structure.

Introduction To Test Management

Test Management means managing the entire process of software testing for a particular project. The test management process is applied to the whole software development life cycle. Hence, ideally, as soon as the software development process starts the test management process should also start.

Test Manager had the following responsibilities-

• The Test Manager should ensure consistency and quality of these work products.
• Work with Test Analyst and Technical Test Analyst to select and customize the appropriate template.
• Work with Test Analyst and Technical Test Analyst to establish standards of these products, like levels of detailed degree.
• Review the work products using appropriate techniques.

Test Management Components

Test Management is divided into 5 parts for better understanding:

1. Test Documentation
2. Test Estimation
3. Test Metrics
4. Measurement of Test Progress
5. Metrics for Monitoring the Testing Lifecycle

#1) Test Documentation

There are three types of Test Documentation that are listed below:

• Test Policy
• Test Strategy
• Master Test Plan

#1) Test Policy: 

• Summarizes value that the organization derives from testing.
• Defines testing policies.
• Describes how to evaluate the effectiveness of testing.
• Outlines the Test Process.
• Specify how the organization will improve the Test Process?

#2) Test Strategy:

• Describes the general test methodologies, that are used to manage Project and Product Risks.
• Analytical Strategies: Like Risk-Based Testing.
• Model-Based Strategy: Like an operational profile where the test team develops a model based on actual and accepted situations of environment, input, and conditions.
• Methodological Strategy: Quality characteristics where the test team uses a set of test conditions, checklist or collection of generalized, logical tests.
• Process or Standard-Compliant techniques: Follows a set of the process like SCRUM/Agile.
• Reactive Strategies: Using defect based ATTACKS LIKE EXPLORATORY TESTING.
• Consultative Strategy: Like user-directed testing where the test team relies on the input of one or more stakeholders to determine test conditions like Outsourced Compatibility Testing.
• Also describes:
  • Integration procedures
  • Test Specification techniques
  • Independence of testing
  • Mandatory and optional standards
  • Test environment
  • Tools
  • Reusability of software products
  • Retesting and Regression.

#3) Master Test Plan:

• It covers all testing tasks that need to be done.
• It discusses how testing will implement Test Strategy and Policy.
• If something is not described, then the Test Plan should describe why and the mitigation plan for that.
• Contents of the Test Plan are:
  • Items to be tested
  • Quality characteristics to be tested.
  • Schedule
  • Execution cycle
  • Defect variables
  • Test items in scope
  • Exit criteria
  • Project risks
  • Overall governance of testing efforts,
  • Roles and responsibilities
  • Input and output

#2) Test Estimation

General Points:

• Is a management activity
• It is based on experience.
• It provides a specific and detailed catalog of costs, resources, tasks, and people.
• Estimation once prepared, must be delivered to management along with the justification.
• The final estimate represents the best possible balance of organizational and project goals.
• The estimate is based on information available at the time, it was prepared.
• In order to remain accurate, estimates should be updated to reflect new and changed information.

Factors affecting Test Estimation:

• Required level of Quality
• Size of the system
• Historical data
• Process factors like strategy, development, and lifecycle
• Material factors like test environment, automation, tools, and data
• People factor
• Complexity of process
• Training and KT(Knowledge Transfer)
• Assimilation and development of new tools and technology, process or techniques.
• The requirement of a higher degree of the detailed Test Specification.
• Timing of component arrival
• Test data.

Guesses:

• Work breakdown structure
• Team Estimation session
• Tester – Developer ratio
• Organization history
• Function point analysis, LOC.

Test Estimation is further explained later in the tutorial.

#3) Test Metrics

• What gets measured, is considered as done?
• What does not measure, is easy to get ignored?
• A limited set of useful metrics should be defined.
• Only those metrics should be defined whose interpretation is agreed upon by all.
• Reporting and merging of metrics should be automated.
• The Manager should validate the information in metric.

Project Metric: % of pass, fail executed etc.

Product Metric:

• Attributes of the product
• Defect Density

Process Metric: Measures capability of testing like % of the defect.

People: Capability of the individual.

Test Progress Metric:

• The number of Test Conditions/Cases, Planned vs Executed.
• Total defect categorized down by severity, priority, current state and effect subsystem.
• The number of changes Required, Accepted, Build and Tested.
• Planned vs Actual Cost.
• Planned vs Actual Duration
• Planned vs Actual Testing milestone.
• Product Quality Risk status
• % loss of Test effort, Cost or Time.

#4) Measurement Of Test Progress

Product Risks:

• % of Risk covered.
• % of Risk for fail test
• % Risk identified by the individual.

Defects:

• The number of defects found vs the number of defects submitted.
• Mean-time of failure arrival rate
• Defects in the particular test items.
• Detection of RCA(Root Cause Analysis)
• The defect is Test releases.
• Defect in phase
• Priority and Severity
• Report Rejects vs Duplicate
• Time-taken to resolve
• The number of new defects introduced due to the fixing of old defects.

Test:

• Total number of the Test pass, fail, runner, blocked
• The total number of Regression test cases.

Coverage:

• Requirement and Design coverage
• Risk coverage
• Environment Configuration coverage
• Code coverage

#5) Metrics For Monitoring The Testing Lifecycle

Monitor Test Plan

• Number of Risk and Requirement
• Defect discovery
• Plan vs Actual efforts.

Monitor Test Design

• The number of defects found during design.

Monitor Test Analysis

• Number of conditions
• Number of defects in Analysis

Monitor Test Implementation

• % of environment configuration
• % of test case automated.

Monitor Execution

• % of Passed, Failed, No run, Blocked test cases
• % Test cases covered
• Planned vs Actual defects resolved
• % of Plan vs Actual coverage

Monitor Closure

• % of Test cases pass, ail
• % of Test cases checked into the reusable category
• % of Test cases automated.
• The number of Defects Resolved/Not Resolved.
• % of Test work product

The test monitoring and control phase discussed below further explains this topic.

Test Management Phases

During the Test Management process, one must consider the following points. In other words, the following are the different phases of the Test Management process:

1. Risk Analysis
2. Test Estimation
3. Test Planning
4. Test Organization
5. Test Monitoring and control
6. Issues management
7. Test report

You can notice that the first four phases are more about planning and the remaining three are about execution. Hence, we can divide the complete test management process into two parts i.e. Planning and Execution.

Let’s explore the various Test Management Phases in detail.

#1) Risk Analysis

This phase includes finding out the risk factors and possible solutions. If risk analysis is done thoroughly, we can avoid future failures or at least some kind of solution might be available.

Risk is something that may or may not happen. But if it happens then what will be its impact? It may badly affect the quality of software, the reputation of the company and much more.

Risk factors should be found out to avoid this bad impact. Risk analysis should be done for finding out risk factors. There are two types of risks i.e. Project risks and Product risks. Project risks are the risks that are related to the work process and Product risk are risks that are related to the developed product.

#2) Test Estimation

Test estimation is about the prediction of the time required for each test activity/phase. As this is an estimate it cannot be accurate. For better test estimation we can study the past projects of our company or we can consult with the team members who are going to be responsible for that work or test phase.

#3) Test Planning

Test planning itself is a long process. It includes defining test objectives, test scope, test strategy, time scheduling, resources, communication approach, etc. Requirements should be very clear for defining test objectives and scope. The test plan is for testers, users and the project team members.

The test plan describes the role of testing in the project. The test plan also includes the roles and responsibilities, list of features that are going to be tested and are not going to be tested, test environment, list of tools and assumptions if any.

#4) Test Organization

During the test planning phase, we have planned all possible things about testing.

Hence we need skilled team members to execute this plan or to make the plan a successful one. Test organization is all about building the perfect test team for a successful project.

#5) Test Monitoring And Control

While testing work is in progress or while the testers are executing the test plan, all these work progress must be monitored. One should keep track of all this testing work. If test monitoring is done, then the test team and test manager will get feedback on how the testing progress is?

Using this feedback, the test manager can guide the team members to improve the quality of further testing work. With the help of test monitoring, the project team will get visibility on the test results. It also helps to know about test coverage.

For large projects, test monitoring is done using an automated tool, as the gathering of data will be easier. For small projects, one person will gather all the data or documents that are related to test progress. For gathering test progress information, we can take the help of the IEEE 829 test log template. This was all about Test Monitoring.

Let’s see what Test control is? Project work will not always go as we have planned. There might be some differences between the plan and the actual work. To minimize or remove these differences, we need to do some changes and that is how we control the test work.

#6) Issues Management

Issues can be any problem that occurs during the software development and testing process. It can be the smallest reason because of which we are not able to develop/deliver a quality product. Some issues are a show-stopper i.e. without solving that issue we will not be able to proceed with the further process.

Issue management is all about how we handle these issues/problems. We can also call it as Incident management. Issue management requires better planning for the process of resolving issues. Better issue management depends on the skill and experience of the testing manager.

How do these issues occur?

There can be several reasons for an issue to occur. Some issues are related to strategy and some are related to the definition, HR, scheduling, etc.

Strategy Issues:

Examples:

• The project runs out of funds.
• Poor project communication.
• The project management process is not according to the stated standards.

Definition Issues: Issues that are related to requirements.

Examples: Unclear requirements. A lot of issues can be introduced because of unclear requirements.

Scheduling Issues: This is the most common type of issue. Employees have to struggle to meet the deadline.

HR Issues:

Examples:

• There is a lack of skill in the team.
• Wrong employee mapping for work.

There can be a lot more types of issues and we cannot mention all of them here. Thus issue management is about logging, tracking and resolving issues.

#7) Test Report

Test report helps to identify test coverage, quality of the developed product and the required process improvements. We can decide ‘how much testing is required?’

If enough testing is done, then we can submit this test report to the stakeholders or clients. So that they will also get to know the quality of the product and have an idea of how much testing is performed on the product.

Test Management Tools

Test management gets complicated as we proceed in our software development process and that is one of the major reasons for which so many test management tools are available nowadays.

These tools will aid in the last four phases of the test management process (Test Organization, Test Monitoring & Control, Issues management, and Test report). As these tools help for the important phases of test management, they should be considered first in the project.

Enlisted below are the most popular Test Management Tools:

1. qTest
2. PractiTest
3. Zephyr
4. Test Collab
5. TestFLO for JIRA
6. XQual
7. Xray – Cutting Edge Test Management
8. TestRail
9. QACoverage
10. Requirements and Test Management for Jira (RTM)
11. SPIRATEST by Inflectra
12. Kualitee
13. aqua
14. Testpad
15. JunoOne

=> Click here for detailed reviews of TOP Test Management Tools

Organizational Structures

Let’s see the different organizational structures.

There might be certain rules for organizational structures or there might be some ideal structures, but irrespective of that every organization can have its structure. There are so many organizational structures and each one have their advantages and disadvantages.

Here, we will discuss some of them.

Firstly we will see the simplest organizational structure that is used for small projects.

In this structure, both the testers and programmers are reporting to the Development Manager.

1. The development manager has good control over project activities.
2. There will be less possibility of a communication gap between the testing and development teams.
3. Also in meetings, it is good for deciding the deadlines for the development manager as he/she has complete knowledge about the testing and development work.
4. Teamwork will be efficient, because of minimal layers.

Disadvantages of this Structure include:

1. As there is no testing manager, there is a possibility that testing will be considered late in the project.
2. There is another possibility that testing will get less importance to the project. It can be considered late in the project.

Generally in small organizations for small projects, it happens that the development team takes more time than mentioned and the testing team has to suffer i.e. the testing team will have to test the product by the deadline, so that the testing team gets less time to test the product.

In this structure, for completing a project successfully, the development manager has to keep in mind that his aim is not to just complete the project but to develop quality software.

The second most commonly used Organizational Structure:

This is the most common type of organizational structure. In this structure, the testers are reporting to the Test Managers and the developers are reporting to the Development Manager. Both the Test Manager and the Development Manager are reporting to the Project Manager.

Further Reading => Review of QA Sphere Test Management Tool- Must Read Guide

The Test Manager will be responsible for all test-related activities and it is the responsibility of the Development Manager to get the software to develop. The Project Manager will control both the testing and development activities.

Advantages:

• Unlike the previous structure, here in this structure, there are different managers for testing and development, hence both of them can focus on their work. They will remain dedicated to their work and there will be fewer distractions for them.
• In this structure, the testing activities cannot be neglected or it cannot be considered late in the project. This means that both testing and development will get equal importance.
• When it comes to making critical decisions, advantageously, the testing team has independence.

Disadvantages:

• There is a possibility of a communication gap because of multiple levels.

Test Management Vs Organizational Structures

Organizational structures directly affect Test management. Different organizational structures have a different impact on test management, hence test management varies according to the skill and experience of the Testing manager as well as according to the testing manager’s position in the organizational structure.

We have seen two organizational structures here. In the first structure, the development manager and the testing manager is the same person, hence it affects test management. The development manager has the aim of developing software, and while doing this he/she has to look at the testing work as well.

Thus at times he/she might give biased opinions. He/she may just overlook the issue and go ahead. This way it may affect test management. An independent test manager will be able to provide more justice and test management will be better with independent test managers.

Conclusion

We have seen both the topics i.e. test management and organizational structures separately and along with the relation between these two. We can conclude that organizational structures affect test management.

While comparing both the structures mentioned above, in the second structure, test management will be handled better than the first one. The reason behind this might be a dedicated test manager.

Organizational structures differ from one organization to another. Although there is some defined process for test management (or teams might be using test management tools), test management will differ because of different organizational structures, test managers, test manager’s skills and experience.