TestRail Review Tutorial: Learn End-to-End Test Case Management

Test Case Management Using TestRail: A Complete Hands-on Review Tutorial and WalkThrough

The TestRail tool provides web-based test case management which is a combination of sophisticated project management capabilities with features optimized for testing.

This tool can be used for any type of project including Agile Development and Testing Methodology.

While TestRail is primarily used for software testing, it is also flexible enough to be used in any type of QA process.

Let’s explore this tool in detail with an extensive TestRail Review Tutorial!

TestRail Test Case Management

You can learn about the following in this tutorial:

• Creating a TestRail Account
• Adding a Project
• Adding Test Suites
• Adding Test Cases
• Adding Test Run
• Executing Test cases
• Reports with test Run and Results

Functions of TestRail

Primary Functions of TestRail

• Document test cases with steps, expected results, screenshots, and much more.
• Organize test cases into test suites and sections.
• Assign test cases to execute and manage team workloads.
• Track the results of test runs in real-time.
• Review progress towards milestones.
• Generate reports on a variety of metrics.

TestRail supports all types of software testing. You can use it to organize manual/script-based testing, schedule and report the results of exploratory testing, and integrate with the test automation tools.

TestRail also integrates with defect tracking tools out-of-the-box and includes an open API so that you can create your own custom integrations. This flexibility is a key reason for which teams choose TestRail over other test case management solutions.

The most important factor is fast, lightweight UI which is easy to learn and use, with little or no training required. Moreover, this is a powerful tool with advanced features such as customizable reports.

Given below is an example project from TestRail. The project overview window summarizes the daily testing progress at a glance, including the number of test cases, passed, blocked, the ones needing retest, or failed.

In the middle of the screen, you can see Test Runs and Milestones. A test run is used to group test cases for execution, whereas a milestone is used to group test runs for a specific purpose, such as a software release.

TestRail Walkthrough

To follow along with this walkthrough, you can get a free TestRail trial version here.

You can choose the hosted Cloud edition for instant setup, or the Server edition to install on your own computer. You just need to complete the required fields as shown below.

For the Cloud edition, you have the extra step of choosing a web address where you’ll access your online instance.

You’ll receive an email with a link to confirm your free trial. Click the link to create your TestRail account.

You’ll be automatically redirected to your trial TestRail instance once it is ready. The process should take just a few minutes.

Depending on your location, you may need to complete a Data Processing Agreement to confirm compliance with the General Data Protection Regulation (GDPR).

Once you’ve done that, the setup is complete and you’re ready to get started!

Step-by-Step Getting Started

1) The screen that you see below is the TestRail Dashboard.

The dashboard shows an overview of your projects, recent activities, and any “to-dos” assigned to you. Notice the “Welcome to TestRail” notification at the bottom of the screen with the suggested steps to get started. In this walkthrough, we’ll complete the first four steps.

2) Click on the Administration tab. You need to come here to do things like add users and roles, extend your trial subscription, configure custom fields, set up integrations, and more.

Click on Users and Roles and you’ll see that you have been added as an administrator.

Click on the Roles tab and you’ll see the predefined roles i.e. Read-only, Tester, Designer, and Lead. Click on the pencil icon to view the rights assigned to each role.

It’s easy to change the default descriptions, create additional roles, add one or more users, assign them to roles, organize them into groups, etc.

3) Use the Dashboard tab to return to the Dashboard. This is where you’ll manage and track your testing projects. Let’s start by creating a project. Click the Add Project button to do so.

4) Give your project a name, then choose a storage option, as shown below. For greater flexibility, you should choose the third option: use multiple test suites to manage cases.

This allows you to start with a single test suite and then add more test suites in the future if required.

5) Click Add Project.

The dashboard will appear with your new project (if it doesn’t, simply click the Dashboard tab). You can change the project name or delete it later if you wish. The screenshot below shows the dashboard for an example project with multiple test suites and another project with a single repository.

Note how the available options change depending on the type of project.

6) Click on the Test Suites link under your new project. If this is your first project, then the Test Suites view will appear with a single default suite, called Master.

Simply click on the name of the suite to edit its sections and test cases.

Alternatively, click the Add Test Suite button to add a test suite to your new project, as shown in the image given below.

7) Now let’s add your first test case. Once the following message appears, click Add Test Case.

8) A detailed Test Case view appears, as shown below. Let’s add a simple test named “Login.”

9) The following four fields are required and can be used to sort and filter test cases: Sections, Templates, Types, and Priority. The optional “Automation Type” field is for integration with a test automation tool.

1. Section defaults to Test Cases. A section is created automatically for each new project.
2. Template defaults to Test Case (Text). TestRail comes with three built-in test case templates:
   • Test Case (Text): Includes text areas for pre-conditions, test steps, and the expected result (shown above). You can add screenshots to these text areas.
   • Test Case (Steps): Includes a text area for pre-conditions, plus rows for individual test steps with the expected result for each step. You can add a screenshot of each individual step.
   • Exploratory Session: Includes text areas for the mission and goals of the exploratory session.
3. Choose a Type for the test, such as Regression, Functional, Performance, or Automated.
4. Set a Priority, such as Critical, High, Medium, or Low.

Recommended Reading => I suggest reading this step-by-step guide on how to write effective test cases.

10) Now you can complete the test case with pre-conditions, steps, and the expected results. Once you’re done defining the test, click on Add Test Case. The test case summary appears, as shown below:

11) Let’s add a few more test cases.

Click the Test Cases link to display the test case menu as shown below. All we really need right now is the title for each test case, so let’s do this quickly using the Test Case Menu.

Just click the Add Case link at the bottom of the list of test cases to add a title.

Click the green checkmark or press Enter to save and go to the next case. (Note that you can also import test cases from a CSV or XML file).

12) After creating your test cases, the next step is to build a test run. This is a set of tests that you want to use for a specific purpose such as regression testing, smoke testing, new features testing, risk-based testing, acceptance or in-sprint testing.

For each test run, you can create a name & description, link to a milestone, identify which test cases to include, and assign the run to a particular user or group for execution.

Click on the Test Runs & Results Tab, and then click the Add Test Run button.

If prompted to select a test suite, choose “Master,” and then click OK.

13) The Add Test Run screen appears, as shown below. As we chose the multiple test suites option earlier, the name defaults to the name of the test suite.

Otherwise, it defaults to “Test Run <date>.” You also have the option of assigning the test run to a Milestone.

Use the Assign To field to assign a test run to a user. Please go ahead and choose the option to Include all test cases and then click on Add Test Run.

Further Reading => Handling the Test Execution Phase in STLC

14) Now the Test Runs & Results screen appears. If you are following along with this walkthrough, you will see a single test run, “Master,” that is zero percent (0%) complete.

The sample screen below shows a project with four runs in progress and several completed runs.

To view or update the progress of a test run, simply click on its name.

15) The screenshot below shows the status of the test run in progress.

Once each test is executed, the tester can update its status as passed, failed, etc. It’s also possible to set the status of multiple tests at once.

If you’re following along with the walkthrough, use the drop-down to set the status of your login test case to Passed.

16) The Add Results window will appear where you can add comments about the test, assign it to another team member, attach a screenshot, and even push the defect to your integrated issue tracker.

For example, assume that you’re using Jira for issue tracking. Once you submit your results, the test case will be updated with the defect ID in Jira.

The Jira issue will remain connected to the test case through the TestRail API. Any updates on the issue in Jira will also update TestRail.

After the defect is fixed, you can use TestRail’s re-run feature to re-execute the test and enter the new results.

17) Click Add Results to close the window and return to the test run in progress. Note that the pie chart has been updated to reflect the status change.

18) As you’ve got a test result, you can explore many customizable reports within TestRail. The sample screen below shows the reports available from the test run. More reports are available from the Reports tab.

Milestone Setup

Although it isn’t necessary to set up milestones to execute test runs, it’s good practice.

Milestones allow you to track progress across multiple test runs for goals such as software releases. Use the Milestones tab to add them.

The sample screen given below shows a project with three open milestones and two completed milestones.

Once all tests are completed in a test run, you can lock in the run which will prevent future changes.

Hence, even if a test case changes for a future run, its definition is preserved for the current run in case you need to audit the results later.

Conclusion

With all these amazing features, it’s easy to see how TestRail can significantly boost a team’s testing productivity.

If you are still managing test cases using spreadsheets, I’d suggest giving TestRail a try to see what all you can achieve for your project.

Further reading =>> Top Test Case Management Tools

Feel free to share your feedback/queries with us in the comments section below! We would love to hear from you.