What is Requirements Traceability Matrix (RTM) in Software Testing: Step-by-step guide to creating Traceability Matrix with examples and sample template
Today’s tutorial is about an important QC tool, that is either over-simplified (read overlooked) or over-emphasized – i.e Traceability Matrix (TM).
Most often, the making, reviewing or sharing of a Traceability Matrix is not one of the primary QA process deliverables – so it is not majorly concentrated on, thus causing the under-emphasis. On the contrary, some clients expect a TM to reveal earth-shattering facets about their product (under test) and are disappointed.
“When used right, a Traceability Matrix can be your GPS for your QA journey”.
As is a general practice at STH, we will see the “What” and “How” aspects about a TM in this article.
What You Will Learn:
- What is the Requirement Traceability Matrix?
- Test coverage and Requirement Traceability
- What is Test coverage?
- Types of Requirement Specifications
- Challenges for Requirement collection
- How Requirement Traceability Can Help
- Advantages of RTM and Test Coverage
- Challenges in Test Coverage
- How to Create a Requirements Traceability Matrix
What is the Requirement Traceability Matrix?
In Requirement Traceability Matrix or RTM, we set up a process of documenting the links between the user requirements proposed by the client to the system being built. In short, it’s a high-level document to map and trace user requirements with test cases to ensure that for each and every requirement adequate level of testing is being achieved.
The process to review all the test cases that are defined for any requirement is called Traceability. Traceability enables to determine which requirements spawned the most number of defects during the testing process.
The focus of any testing engagement is and should be maximum test coverage. By coverage, it simply means that we need to test everything there is to be tested. The aim of any testing project should be 100% test coverage.
Requirements Traceability Matrix establishes a way to make sure we place checks on the coverage aspect. It helps in creating a snapshot to identify coverage gaps. In short, it can also be referred to as a metrics which determines the number of Test cases Run, Passed, Failed or Blocked etc. for every requirement.
Why is Requirement Traceability required?
Requirement Traceability Matrix helps to link the requirements, test cases, and defects accurately. The whole of the application is tested by having requirement traceability (End to End testing of an application is achieved).
Requirement Traceability assures good ‘Quality’ of the application as all the features are tested. Quality control can be achieved as software gets tested for unforeseen scenarios with minimal defects and all functional and non-functional requirements being satisfied.
Requirement Traceability Matrix aids for software application getting tested in the stipulated time duration, the scope of the project is well determined and its implementation is achieved as per the customer requirements and needs and cost of the project is well controlled.
Defect Leakages are prevented as a whole of the application is tested for its requirements.
Types of Traceability Matrix
In ‘Forward Traceability’ Requirements to the Test cases. It ensures that the project progresses as per the desired direction and that every requirement is tested thoroughly.
The Test Cases are mapped with the Requirements in ‘Backward Traceability’. Its main purpose is to ensure that the current product being developed is on the right track. It also helps to determine that no extra unspecified functionalities are added and thus the scope of the project is affected.
(Forward + Backward): A Good Traceability matrix has references from test cases to requirements and vice versa (requirements to test cases). This is referred to as ‘Bi-Directional’ Traceability. It ensures that all the Test cases can be traced to requirements and each and every requirement specified has accurate and valid Test cases for them.
Examples of RTM
#1) Business Requirement:
BR1: Writing emails option should be available.
Test Scenario(technical specification) for BR1
TS1: Compose mail option is provided.
Test Case 1 (TS1.TC1): Compose mail option is enabled and works successfully.
Test Case 2 (TS1.TC2): Compose mail option is disabled.
After executing the test cases if any defects are found that too can be listed and mapped with the business requirements, test scenarios and test cases.
Example: If TS1.TC1 fails i.e. Compose mail option though enabled does not work properly then a defect can be logged. Suppose the defect ID auto-generated or manually assigned number is D01, then this can be mapped with BR1, TS1, and TS1.TC1 numbers.
Thus all requirements can be represented in a table format.
|Business Requirement #||Test Scenario #||Test Case #||Defects #|
Test coverage and Requirement Traceability
What is Test coverage?
Test coverage states which requirements of the customers are to be verified when the testing phase starts. Test coverage is a term that determines whether the test cases are written and executed ensure to test the software application completely, in such a way that minimal or NIL defects are reported.
How to achieve Test Coverage?
The maximum Test Coverage can be achieved by establishing good ‘Requirement Traceability’.
- Mapping all internal defects to the test cases designed
- Mapping all the customer reported defects (CRD) to individual test cases for future regression test suite
Types of Requirement Specifications
#1) Business Requirements:
The actual customers’ requirements are listed down in a document known as Business Requirements Document (BRS). This BRS is minutely derived high-level requirement list, after a brief interaction with the client.
It is usually prepared by ‘Business Analysts’ or the project ‘Architect’ (depending upon organization or project structure). The ‘Software Requirement Specifications’ (SRS) document is derived from BRS.
#2) Software Requirements Specification Document (SRS):
It is a detailed document which contains all the meticulous details of all functional and non-functional requirements. This SRS is the baseline for designing and developing the software application.
#3) Project Requirement Documents (PRD):
The PRD is a reference document for all the team members in a project to tell them exactly what a product should do. It can be divided into the sections like Purpose of the product, Product Features, Release Criteria and Budgeting & Schedule of the project.
#4) Use Case Document:
It is the document that helps in designing and implementing the software as per the business needs. It maps the interactions between an actor and an event with a role that needs to be performed to achieve a goal. It is a detailed step-by-step description of how a task needs to be performed.
Role: Download Game
Game download is successful.
Use Cases may also be a part included in the SRS document as per organizations work process.
#5) Defect Verification Document:
It is documented containing all the details related to defects. The team can maintain a ‘Defect Verification’ document for fixing and retesting of the defects. The testers can refer ‘Defect Verification’ document, when they want to verify if the defects are fixed or not, retest defects on different OS, device, different system configuration etc.
The ‘Defect Verification’ document is handy and important when there are a dedicated defect fixing and verification phase.
#6) User Stories:
The user story is a primarily used in ‘Agile’ development to describe a software feature from an end-user perspective. User stories define the types of users and in what way and why they want a certain feature. The requirement is simplified by creating user stories.
Currently, all of the software industries are moving towards the use of User Stories and Agile Development and corresponding software tools for recording the requirements.
Challenges for Requirement collection
1) The requirements collected must be detailed, unambiguous, accurate and well specified. But there is NO appropriate measure for calculating these details, unambiguousness, accuracy and well-defined specifications that are needed for the requirement collection.
2) The interpretation of the ‘Business Analyst’ or ‘Product Owner’ whoever provides the requirements information is critical. Similarly, the team who receives the information has to raise appropriate clarifications in order to understand the expectations of the stakeholder’s.
The understanding must be in sync with both the business needs and the actual efforts required for application implementation.
3) The information should also be derived as per the point of view of the end user.
4) Stakeholders’ state conflicting or contradicting requirements at different times.
5) End-user point-of-view is not considered due to multiple reasons and further stakeholders think they “completely” understand what is required for a product, which generally is not the case.
6) Resources lack skills for application developed.
7) Frequent ‘Scope’ changes of application or priority change for modules.
8) Missed, implicit or undocumented requirements.
9) Inconsistent or vague requirements determined by the customers.
10) The conclusion of all the factors stated above is that the ‘Success’ or ‘Failure’ of a project depends considerably on a requirement.
How Requirement Traceability Can Help
#1) Where is a requirement implemented?
Requirement: Implement ‘Compose mail’ Functionality in a mail application.
Implementation: Where on the main page the ‘Compose mail’ button should be placed and accessed.
#2) Is a requirement necessary?
Requirement: Implement ‘Compose mail’ Functionality in a mail application to certain users only.
Implementation: As per user access rights if the email inbox is ‘Read-only’ then in this case ‘Compose mail’ button won’t be required.
#3) How do I interpret a requirement?
Requirement: ‘Compose mail’ Functionality in a mail application with fonts and attachments.
Implementation: When ‘Compose mail’ is clicked what all features should be provided?
- Text Body to write emails and edit in different font types and also bold, italics, underline them
- Types of attachments (Images, documents, other emails etc.)
- Size of attachments(Maximum size allowed)
Thus the requirements get broken down to sub-requirements.
#4) What design decisions affect the implementation of a requirement?
Requirement: All elements ‘Inbox’, ‘Sent mail’, ‘Drafts’, ‘Spam’ , ‘Trash’ , etc. should be clearly visible.
Implementation: The elements to be visible should be displayed in ‘Tree’ format or ‘Tab’ format.
#5) Are all requirements allocated?
Requirement: ‘Trash’ mail option is provided.
Implementation: If the ‘Trash’ mail option has been provided, then the ‘Delete’ mails option (requirement) must be implemented initially and should be working accurately. If the ‘Delete’ mail option is working properly, then only the deleted emails will be collected in ‘Trash’ and implementing the ‘Trash’ mail option(requirement) will make sense(will be useful).
Advantages of RTM and Test Coverage
1) The build developed and tested has the required functionality which meets the ‘customers’/ ‘users’ needs and expectations. The customer must get what he wants. To surprise the customer with an application that does not do what it’s expected to do is not a satisfying experience for anyone.
2) The end product (software application) developed and delivered to the customer must encompass only the functionality that’s needed and expected. Extra features provided in the software application may seem attractive initially until there is an overhead of time, money, and effort to develop it.
The extra feature may also become a source of defects, which can cause problems for a customer after installation.
3) Developer’s initial task gets defined clearly as they work first on implementing the requirements, which are of high- priority, as per the customer requirement. If customer’s high–priority requirements are clearly specified, then those code components can be developed and implemented on first priority.
Thus it is ensured that the chances of the end-product being shipped to the customer is as per the topmost requirements and is on schedule.
4) Testers verify first the most important functionality implement by developers. As the verification (testing) of the priority software component is done first it helps to determine when and if the first versions of the system are ready to be released.
5) Accurate test plans, test cases are written and executed which verify that all of the application requirements are implemented correctly. Test cases mapping with the requirements helps to ensure that no major defects are missed. It further helps in implementing a quality product as per the customer expectations.
6) In case there is ‘change request’ from the client, all of the application components that are affected by the change request get modified and nothing gets overlooked. This further enhances in evaluating, the impact a change request does to the software application.
7) A seemingly simple change request might implicate modifications that need to be done to several parts of the application. It’s better to derive a conclusion on how much efforts will be required before agreeing to make the change.
Challenges in Test Coverage
#1) Good Communication channel:
If there are any changes which are suggested by the stakeholders, the same needs to be communicated to the development and testing teams in the earlier phases of development. Without this on time development, testing of application and capturing /fixing of defects cannot be ensured.
#2) Prioritizing the Test Scenarios is important:
Identifying which are high-priority, complex, and important test scenarios is a difficult task. Trying to test all of the test scenarios is almost an unachievable task. The goal of testing the scenarios must be very clear from the business and end-user point of view.
#3) Process Implementation:
Testing process must be clearly defined considering the factors like technical infrastructure and implementations, team skills, past experiences, organizational structures and processes followed, project estimations related to cost, time and resources and location of the team as per the time zones.
A uniform processes implementation considering the mentioned factors ensures every individual concerned with the project is on the same page. This helps in a smooth flow of for all the processes related to application development.
#4) Availability of Resources:
Resources are of two types, skilled-domain specific testers and the testing tools used by the testers. If the testers have proper knowledge of the domain they can write and implement effective test scenarios and scripts. To implement these scenarios and scripts the testers should be well equipped with appropriate ‘Testing Tools’.
Good implementation and on time delivery of the application to the customer can be ensured by the only skilled tester and appropriate testing tools.
#5) Effective Test Strategy implementation:
‘Test strategy’ in itself is a big and a separate topic of discussion. But here for ‘Test Coverage’ an effective test strategy implementation ensures that the ‘Quality’ of the application is good and it is maintained over the period of time everywhere.
An effective ‘Test Strategy’ plays a major role in planning ahead for all kinds of critical challenges, which further helps in developing a better application.
How to Create a Requirements Traceability Matrix
To being with we need to know exactly what is it that needs to be tracked or traced.
Testers start writing their test scenarios/objectives and eventually the test cases based on some input documents – Business requirements document, Functional Specifications document and Technical design document (optional).
Let’s suppose, the following is our Business requirements document (BRD): (Download this sample BRD in excel format)
(Click any image to enlarge)
The below is our Functional Specifications document (FSD) based on the interpretation of the Business requirements document (BRD) and the adaptation of it to computer applications. Ideally, all the aspects of FSD need to be addressed in the BRD. But for simplicity’s sake, I have only used the points 1 and 2.
Sample FSD from Above BRD: (Download this sample FSD in excel format)
Note: the BRD and FSD are not documented by QA teams. We are mere, the consumers of the documents along with the other projects teams.
Based on the above two input documents, as the QA team, we came up with the below list of high-level scenarios for us to test.
Sample Test Scenarios from the Above BRD and FSD: (Download this sample test Scenarios file)
Once we arrive here, now would be a good time to start creating the requirements traceability matrix.
I personally prefer a very simple excel sheet with columns for each document that we wish to track. Since the business requirements and functional requirements are not numbered uniquely we are going to use the section numbers in the document to track. (You can choose to track based on line numbers or bulleted-point numbers etc. depending on what makes the most sense for your case in particular.)
Here is how a simple Traceability Matrix would look for our example:
The above document establishes a trace between, the BRD to FSD and eventually to the test scenarios. By creating a document like this, we can make sure every aspect of the initial requirements has been taken into consideration by the testing team for creating their test suites.
You can leave it this way. However, in order to make it more readable, I prefer including the section names. This will enhance understanding when this document is shared with the client or any other teams.
The outcome is as below:
Again, the choice to use the former format or the later is yours.
This is the preliminary version of your TM but generally, does not serve its purpose when you stop here. Maximum benefits can be reaped from it when you extrapolate it all the way to defects.
Let’s see how.
For each test scenario that you came up with, you are going to have at least 1 or more test cases. So, include another column when you get there and write the test case IDs as shown below:
At this stage, the Traceability Matrix can be used to find gaps. For example, in the above Traceability Matrix, you see that there are no test cases written for FSD section 1.2.
As a general rule, any empty spaces in the Traceability Matrix are potential areas for investigation. So a gap like this can mean one of the two things:
- The test team has somehow missed considering the “Existing user” functionality.
- The “Existing user” functionality has been deferred to later or removed from the application’s functionality requirements. In this case, the TM shows an inconsistency in the FSD or BRD – which means that an update on FSD and/or BRD documents should be done.
If it is scenario 1, it will indicate the places where the test team needs to work some more to ensure 100% coverage.
In scenarios 2, TM not just shows gaps it points to incorrect documentation that needs immediate correction.
Let us now expand the TM to include test case execution status and defects.
The below version of the Traceability Matrix is generally prepared during or after test execution:
Download requirements traceability matrix template:
Important Points to Note
The following are the important points to note about this version of the Traceability Matrix:
1) The execution status is also displayed. During execution, it gives a consolidated snapshot of how work is progressing.
2) Defects: When this column is used to establish the backward traceability we can tell that the “New user” functionality is the most flawed. Instead of reporting that so and so test cases failed, TM provides a transparency back to the business requirement that has most defects thus showcasing the Quality in terms of what the client desires.
3) As a further step, you can color code the defect ID to represent their states. For example, defect ID in red can mean it is still Open, in a green can mean it is closed. When this is done, the TM works as a health check report displaying the status of the defects corresponding to a certain BRD or FSD functionality is being open or closed.
4) If there is a technical design document or use cases or any other artifacts that you would like to track you can always expand the above-created document to suit your needs by adding additional columns.
To sum up, RTM helps in:
- Ensuring 100% test coverage
- Showing requirement/document inconsistencies
- Displaying the overall defect/execution status with a focus on business requirements.
- If a certain business and/or functional requirement were to change, a TM helps estimate or analyze the impact on the QA team’s work in terms of revisiting/reworking on the test cases.
- A Traceability Matrix is not a manual testing specific tool, it can be used for automation projects as well. For an automation project, the test case ID can indicate the automation test script name.
- It is also not a tool that can be used just by the QAs. The development team can use the same to map BRD/FSD requirements to blocks/units/conditions of code created to make sure all the requirements are developed.
- Test management tools like HP ALM come with the inbuilt traceability feature.
An important point to note is that the way you maintain and update your Traceability Matrix determines the effectiveness of its use. If not updated often or updated incorrectly the tool is a burden instead of being a help and creates the impression that the tool by itself is not worthy of using.
Requirement Traceability Matrix is the means to map and trace all of the client’s requirements with the test cases and discovered defects. It is a single document which serves the main purpose that no test cases are missed and thus every functionality of the application is covered and tested.
Good ‘Test Coverage’ which is planned ahead of time prevents repetitive tasks in testing phases and Defect leakages. A high defect count indicates that testing is done well and thus ‘Quality’ of the application is going up. Similarly, a very low defect count indicates testing is not done up to the mark and this hampers the ‘Quality’ of the application in a negative way.
If the Test coverage is done thoroughly then a low defect count can be justified and this defect count can be considered as supporting statistics and not a primary one. Quality of an application is termed as ‘Good’ or ‘Satisfying’ when the test coverage is maximized and defect count is minimized.
Have you created a Requirement Traceability Matrix in your projects? How similar or different is it from what we have created in this article? Please share your experiences, comments, thoughts, and feedback on this article through your comments.