Test Case Management Using TestRail: A Complete Hands-on Review Tutorial and WalkThrough
TestRail tool provides web-based test case management which is a combination of sophisticated project management capabilities with features optimized for testing.
This tool can be used for any type of project including Agile Development and Testing Methodology.
While TestRail is primarily used for software testing, it is also flexible enough to be used in any type of QA process.
Let's explore this tool in detail with a hands-on TestRail Review Tutorial!!
What you will learn in this tutorial:
- Creating TestRail Account
- Adding a Project
- Adding Test Suites
- Adding Test Cases
- Adding Test Run
- Executing Test cases
- Reports with test Run and Results
What You Will Learn:
Functions of TestRail
The Primary Functions of TestRail:
- Document test cases with steps, expected results, screenshots, and much more.
- Organize test cases into test suites and sections.
- Assign test cases for execution and manage team workloads.
- Track the results of test runs in real time.
- Review progress toward milestones.
- Generate reports on a variety of metrics.
TestRail supports every type of software testing. You can use it to organize manual/script-based testing, to schedule and report the results of exploratory testing, and to integrate with the test automation tools.
TestRail also integrates with defect tracking tools out-of-the-box and includes an open API, so that you can create your own custom integrations. This flexibility is the key reason for which the teams choose TestRail over the other test case management solutions.
The most important factor is fast, lightweight UI which is easy to learn and use, with little or no training required. Moreover, this is a powerful tool with advanced features such as customizable reports.
Given below is an example project in TestRail. The project overview window summarizes the daily testing progress at a glance, including the number of test cases, passed, blocked, the ones needing retest, or failed.
In the middle of the screen, you can see Test Runs and Milestones. A test run is used to group test cases for execution, whereas milestone is used to group test runs for a specific purpose, such as a software release.
To follow along with this walkthrough, you get a free TestRail trial version here.
You can choose the hosted Cloud edition for instant setup, or the Server edition to install on your own computer. You just need to complete the required fields as shown below.
For the Cloud edition, you have an extra step of choosing a web address where you’ll access your online instance.
You’ll receive an email with a link to confirm your free trial. Click the link to create your TestRail account. You’ll be automatically redirected to your trial TestRail instance once it is ready. This process should take just a few minutes.
Depending on your location, you may need to complete a Data Processing Agreement to confirm compliance with the General Data Protection Regulation (GDPR).
Once you’ve done that, setup is complete and you’re ready to get started!
Step-by-Step Getting Started
#1) The screen that you see below is the TestRail Dashboard.
The dashboard shows an overview of your projects, recent activities, and any “todos” assigned to you. Notice the “Welcome to TestRail” notification at the bottom of the screen with suggested steps to get started. In this walkthrough, we’ll complete the first four steps.
#2) Click the Administration tab. You need to come here to do things like add users and roles, extend your trial subscription, configure custom fields, set up integrations, and more. Click on Users and Roles, and you’ll see that you have been added as the administrator.
Click the Roles tab, and you’ll see the predefined roles i.e. Read-only, Tester, Designer, and Lead. Click the pencil icon to view the rights assigned to each role. It’s easy to change the default descriptions, create additional roles, add one or more users, assign them to roles, organize them in groups, etc.
#3) Use the Dashboard tab to return to the Dashboard. This is where you’ll manage and track your testing projects. Let’s start by creating a project. Click the Add Project button to do so.
#4) Give your project a name, then choose a storage option, as shown below. For greater flexibility, you should choose the third option: use multiple test suites to manage cases.
This allows you to start with a single test suite and then add more test suites in the future if required.
#5) Click Add Project.
The dashboard will appear with your new project (if it doesn’t, simply click the Dashboard tab). You can change the project’s name or delete it later if you wish. The screenshot below shows the dashboard for an example project with multiple test suites and another project with a single repository.
Notice how the available options change depending on the type of project.
#6) Click the Test Suites link under your new project. If this is your first project, then the Test Suites view will appear with a single default suite, called Master. Simply click the name of the suite to edit its sections and test cases.
Otherwise, click the Add Test Suite button to add a test suite to your new project, as shown in the image below.
#7) Now let’s add your first test case. Once the following message appears, click Add Test Case.
#8) A detailed Test Case view appears, as shown below. Let’s add a simple test named “Login.”
#9) The following four fields are required and can be used to sort and filter test cases: Section, Template, Type, and Priority. The optional “Automation Type” field is for integration with a test automation tool.
- Section defaults to Test Cases. A section is created automatically for each new project.
- Template defaults to Test Case (Text). TestRail comes with three built-in test case templates:
- Test Case (Text): Includes text areas for pre-conditions, test steps, and the expected result (shown above). You can add screenshots to these text areas.
- Test Case (Steps): Includes a text area for pre-conditions, plus rows for individual test steps with the expected result for each step. You can add a screenshot to each individual step.
- Exploratory Session: Includes text areas for the mission and goals of the exploratory session.
- Choose a Type for the test, such as Regression, Functional, Performance, or Automated.
- Set a Priority, such as Critical, High, Medium, or Low.
Recommended Reading => I suggest reading this step-by-step guide on how to write effective test cases.
#10) Now you can complete the test case with pre-conditions, steps, and the expected results. Once you’re done defining the test, click on Add Test Case. The test case summary appears, as shown below:
#11) Let’s add a few more test cases.
Click the Test Cases link to display the test case menu as shown below. All we really need right now is the title for each test case, so let’s do this quickly using the Test Case Menu. Just click the Add Case link at the bottom of the list of test cases to add a title.
Click the green check mark or press Enter to save and go to the next case. (Note that you can also import test cases from a CSV or XML file).
#12) After creating your test cases, the next step is to build a test run. This is a set of tests that you want to use for a specific purpose such as regression testing, smoke testing, new features testing, risk-based testing, acceptance or in-sprint testing.
For each test run, you can create a name & description, link to a milestone, identify which test cases to include, and assign the run to a particular user or group for execution. Click the Test Runs & Results Tab, and then click the Add Test Run button.
If prompted to select a test suite, choose “Master,” and then click OK.
#13) The Add Test Run screen appears, as shown below. As we chose the multiple test suites option earlier, the name defaults to the name of the test suite. Otherwise, it defaults to “Test Run <date>.” You also have the option of assigning the test run to a Milestone.
Use the Assign To field to assign the test run to a user. Let’s go ahead and choose the option to Include all test cases, and then click on Add Test Run.
Further Reading => Handling the Test Execution Phase in STLC
#14) Now the Test Runs & Results screen appears. If you are following along with this walkthrough, you will see a single test run, “Master,” that is zero percent (0%) complete. The sample screen below shows a project with four runs in progress and several completed runs.
To view or update the progress of a test run, simply click its name.
#15) The screenshot below shows the status of a test run in progress.
As each test is executed, a tester can update its status as passed, failed, etc. It’s also possible to set the status of multiple tests at once. If you’re following along with the walkthrough, use the drop-down to set the status of your login test case to Passed.
#16) The Add Result window will appear, where you can add comments about the test, assign it to another team member, attach a screenshot, and even push the defect to your integrated issue tracker.
For Example, assume that you’re using Jira for issue tracking. After you submit your result, the test case is updated with the defect ID in Jira, and the Jira issue remains connected to the test case through the TestRail API. Any updates to the issue in Jira will also update TestRail.
After the defect is fixed, you can use TestRail’s re-run feature to re-execute the test and enter the new results.
#17) Click Add Result to close the window and return to the test run in progress. Notice that the pie chart has been updated to reflect the status change.
#18) As you’ve got a test result, you can explore many customizable reports within TestRail. The sample screen below shows the reports available from the test run. More reports are available from the Reports tab.
Although it isn’t necessary to set up milestones to execute test runs, it’s a good practice.
Milestones allow you to track progress across multiple test runs for goals such as a software release. Use the Milestones tab to add them. The sample screen below shows a project with three open milestones and two completed milestones.
Once all tests are complete in a test run, you can lock the run which will prevent future changes. Hence, even if a test case changes for a future run, its definition is preserved for the current run in case you need to audit the results later.
With all these features, it’s easy to see how TestRail can significantly boost a team’s testing productivity.
If you are still managing test cases using spreadsheets, I'll suggest giving TestRail a try to see what all you can achieve for your project.
Feel free to share your feedback/queries with us in the comments section below!