Testim.io Tutorial: Speed-up Authoring and Execution of Automated Tests

In this hands-on tutorial, you will learn how to build stable tests in minutes using Testim automation tool and use machine learning process to speed-up the authoring, execution, and maintenance of automated tests.

When we hear the phrase “Automation Tool”, the major considerations that come to mind with regards to our tests include:

• Authoring and Execution
• Stability
• Reusability
• Extensibility
• Maintenance
• Troubleshooting
• Reporting
• CI/CD Integration. There are many tools in the market which address one or a few of the above factors but not all of them. This is where Testim.io comes into the picture.

Testim.IO Tool Tutorial

Testim.io is not a tool but a platform that includes almost all the prime factors like Authoring & Execution, maintenance, troubleshooting, reporting, etc., of automated tests. With the introduction to the concept of Dynamic locators, they focus on more testing types like functional testing, end-to-end testing, and UI testing.

It increases the extensibility and stability of your test suites. It also gives teams, organizations, the flexibility to extend the functionalities of the platform using complex programming logic with JavaScript and HTML.

Here is a detailed insight on Testim.io and how it helps you overcome some of the challenges that customers face during automation.

How is this tool different from other tools?

To understand how Testim differentiates itself from other tools, let’s take a step back at how automation tools have evolved over the past decade.

In the past 10 years, there have been several vendor tools and open source tools that tried to address the various problems related to automated testing. However, one major problem that no one has addressed so far is the use of static ID’s.

For example, say a developer changes the ID name or some attribute of an element on a web page. The automated tests immediately break due to this change, as we are still referring to the old ID name. This leads to unstable tests and the testers have to spend a lot of time maintaining them.

They introduced the concept of Dynamic locators. Artificial Intelligence (AI) underneath the platform in real-time, analyzes all the DOM objects of a page and extracts the objects and their properties.

Finally, the AI decides the best location strategy to locate a particular element based on this analysis.

Due to this, even if a developer changes the attribute of an element, the test still continues to run and this leads to more stable tests. As a result of this, the authoring and execution of automated tests are much faster and more stable.

Testim.io is a Platform and NOT a Record nor a Playback Tool

First of all, this is not a tool but a platform.

It has robust features which not only help to automate your tests but it also takes care of the other important aspects of your tests such as running your tests locally and in the cloud (their private grid or 3rd party grid), integrating with CI/CD systems, capturing logs and screenshots of test runs, giving detailed reporting of the test runs with graphs, statistics, and other related information.

Finally, making the capture and reporting of bugs very simple by integrating them with the widely used bug tracking tools.

Secondly, it gives the organizations and the teams the flexibility to build their own wrappers around the platform by using complex programming logic with JavaScript and HTML. In this way, anyone can extend the functionality of the platform.

Finally, the platform aligns with the Shift-Left paradigm where the Developers and Testers can start writing the tests right from the start by working on different branches without overwriting each other’s changes. It also helps teams collaborate and gives flexibility for anyone to write automated tests via its built-in functionalities.

This way, even a non-technical team member can make valuable contributions to your automated tests. The platform helps to instil the concept that automated tests should be simple, effective, stable, and collaborative.

Addressing Key Factors

How does Testim Address the key factors that are vital to Automated Tests?

Authoring and Execution

Automated tests need to be easy to author and execute from anywhere and at any time.

Testim helps to do this in the following ways:

1) Creating a Test

We create a new Test by clicking on “Create New” or “New Test”.

2) Recording and Playback a test

Once we click the “Record” button, we can record different user actions in our application.

After recording the different actions, click on the “Stop Recording” button to finish recording our tests. Use the “Play” button to replay the recorded test and the “Save” button to save the tests.

3) Validation and Assertions

This platform helps to make validation of different attributes of an element and really simple APIs.

Various options for users such as:

• Adding Custom Validation using JavaScript and HTML
• Validate element visibility
• Validate element text
• Pixel level validation
• API level validation

4) Screenshots

While each test is recorded, the platform takes a screenshot of all the Pass and Failed results of each and every step. As a result, the users find it easier to troubleshoot the problems and understand what happens underneath the hood.

For Example, In the below screenshot, we will notice that the attributes of the “Select Destination” button have changed but the test still ran because the tool AI used, Dynamic Locators. Also, the functionality of the system did not change but only the attribute of an element.

You can also take a screenshot of what exactly happened while running the test by displaying the Expected image (Baseline) and the Actual image.

(Note: Click on the image below for an enlarged view)

5) Feedback on Each Step

The user also gets feedback on each step in terms of whether the tests Passed or Failed by showing a “Green” or “Red icon” on the top left portion of each step as shown below:

6) Labelling tests

Testim provides a feature to label each and every test that a user creates.

There are 2 reasons for which we may want to label a test:

• Helps identify the reason for which the test was created in the first place.
• Helps run the tests with the same label all at once through the CLI feature.

The way we create labels is by clicking on the “Label” button and either selecting an existing label or creating a new one.

7) User Documentation

They provide users with all the documentation that they will need to use the different features of this platform.

Most of the answers about using this platform can be found by clicking on the “Educate” tab and by visiting the documentation site as shown below:

Due to the above reasons, Testim.io helps to make the authoring and execution of tests really fast and simple. Within a matter of seconds, a user can record, replay, and save the tests.

Stability

Testim uses Dynamic locators instead of Static ids. Due to this, the tests are more stable and the more we run the tests the smarter the AI becomes in ensuring more stability to the automated tests.

A user will be able to see the different location strategies that their AI uses to locate an element in the IDE, through the Properties panel, for each element that we interact with via our automated tests.

As you will notice, the AI parses through all the DOM objects and lists them in the Properties Panel along with the rankings of each and every location strategy for that particular element. In this way, even if the attribute of an element changes, then the AI can use a different location strategy from the already parsed list of DOM objects.

Thus, a user does not have to worry about flaky tests.

Reusability

One of the best practices of writing automated tests is creating reusable components that can be used in different parts of our test suite.

Why is this important?

Creating reusable components is important because:

• It helps to increase the readability of the automated tests.
• Saves effort by not repeating the same set of steps in different parts of the tests.
• Any changes to the reusable step need to be done only in one place and it is reflected throughout the tests, across different projects.
• Makes automated tests more extensible.

Testim helps to ensure Reusability by “Grouping” and “Parameterization”.

1) Grouping

Any number of related steps can be grouped into one reusable component.

For Example – The “Login” scenario is one of the most commonly used steps in any application. The way in which we can create a reusable “Login” step would be to select the steps that we want to group together and then click on “Add new Group” as shown below:

2) Parameterization

This platform gives the option of testing the application through various input combinations via Parameterization.

This can be achieved in various ways.

One way to do this is to give all the input parameters that we would need to test the application in the form of a JSON file in the Setup step (first step of our tests) as shown below:

Extensibility

This tool gives flexibility for organizations to extend the functionalities of this platform using JavaScript and HTML. This way, with any functionality that this platform does not handle, the user can write their own code to build a robust automation framework

For Example – Say we want to validate the “Select Destination” button from our previous examples.

The way to do this would be.

• Click on “Add custom action”.

• Give a name to the New Step and click on “Confirm”.

• Click on “PARAMS” and Select “HTML” for this example.
• Add Custom Code.

• A new step with Custom Code gets added to the list of already existing steps.

Maintenance

One of the most important factors related to automated tests is “Maintenance”.

A lot of effort is spent on maintaining the tests rather than writing the actual tests. This leads to a wastage of valuable effort and time from the resources who could have rather spent that time testing the actual application.

Testim helps to minimize the effort required to maintain automated tests in the following ways:

1) Version Control

At any given time, it is important to have the logs of what changes were made to a particular test. This way we can always revert back to the older version of the test as and when required. This platform provides this functionality by showing all the version history by going to the Properties panel of the setup step and clicking on “See old revisions”.

2) Branching

In the “Shift Left Paradigm” where both Development and Testing must start in parallel as early as possible in the software development lifecycle.

Keeping this in mind, they have provided the functionality to the teams to create separate branches for each team member and work on the same projects and tests. This way, no one can overwrite the changes of the other team members and the teams can work on the same code base at any time.

You just need to select “Fork” to create a new branch and you can also switch between existing branches.

Users have the option of scheduling their tests. This helps to run the tests automatically on a certain day and time without any manual intervention. You will also be notified via email in case of any errors.

Troubleshooting

As testers, we spend a considerable amount of time troubleshooting the issues. To help with troubleshooting, this platform offers different options for users to narrow down the scope of the problem.

These options include:

1) Screenshots

The screenshot feature explained in the “Authoring and Execution” section helps the users to know what was the baseline image and what is the actual image found.

2) Properties Panel

The property panel helps to capture the error messages and display them to the user.

For example, when an element is not visible and the test fails, we get the below error message in the panel.

3) Test Logs

Logs are a rich source of information on what happens underneath the UI. The tool provides test logs when the user runs the tests on our grid or a 3rd party grid. The option can be found in the properties panel of the setup step.

4) Documentation

Testim put a lot of effort into documenting most of the features of the tool in the User Documentation that is found under the ducate”tabReporting

This is one of the most important aspects of your tests to get visibility on what tests ran, passed, failed, duration of the run, what browser combinations, and so on.

Testim.io provides 2 types of the Reporting feature:

1) Reporting of Suite and Test Runs

When users click on the “Runs” option in the side menu, they can see all the suite and test runs with all the required details as shown below. This will help to get visibility on the test runs and help in easier reporting.

2) Reporting Bugs

We spend a considerable amount of time in a day reporting bugs. This is one of the most critical and time-consuming parts of testing.

Testim.io helps to reduce bug reporting time and increase productivity by giving a chrome extension to capture and report bugs quickly. You can capture screenshots and videos of the bug and record an actual automated test with this extension. It integrates with most of the widely used bug tracking systems.

After capturing the defect and clicking on “Publish”, it automatically populates all the required steps in the Bug Description. It also adds screenshots, videos, and automated tests with the defect.

(Note: Click on the image below for an enlarged view)

CI/CD Integration

This tool integrates with different CI/CD systems such as Jenkins, Travis CI, Circle CI, and other build tools. It also integrates with 3rd party grids hosting their own private cloud to run tests. All this is done via the CLI (Command Line Interface) that we provide on the platform.

Clicking on the “Settings” icon from the side menu opens up the CLI.

This is pre-populated with the commands that need to run the tests on CI/CD systems, 3rd party grids, or their grids. We just have to click on “Copy” and literally, paste the command in whichever build tool or grid the organization is using.

CLI also accepts other parameters to run specific tests, to run tests in parallel, to override the base URL and a lot more functionalities.

Conclusion

In a nutshell, the Testim.io platform helps you overcome some of the major challenges with automation as described in the above sections. With this, you can make automation simpler & more stable.

This platform will help:

• Non-technical team members to get involved in automation and write solid tests.
• Technical team members use complex programming logic to extend the functionalities of this platform.
• Authoring and execution of tests is super fast.
• Tests are more stable with the use of “Dynamic Locators”.

Overall it is a platform that helps in various facets of test automation and they are in the process of building more functionalities to help the agile teams and organizations collaborate with the motto of ONE TEAM and ONE GOAL.

=> You can explore more about this tool here.

Feel free to ask if you have any queries and also share your thoughts in the comments section as we would love to know what challenges you face with automation and how you managed to solve them.