A Hands-on Review Tutorial of The TestProject Test Automation Platform:
Test automation is essential in the world of continuous deployment and short release cycles. And, we all as software testers come across many challenges when it comes to automation testing.
Today, we are going to review one of the popular test automation tools – TestProject. It's a community-powered free end-to-end test automation platform to simplify automation testing effort for mobile and web applications.
What You Will Learn:
What Is TestProject?
TestProject is an end to end framework for test automation of mobile and web apps. It aims to simplify your test automation experience while offering infinite customization through a gallery of add-ons.
What makes this tool really cool is that it’s a first of its kind free community-powered cloud platform that can be used to record, develop and analyze test automation.
Another cool thing about TestProject is that it easily supports so many platforms like Windows, macOS, Linux, iOS, Android, Webservices.
With TestProject, you get a complete test automation framework that allows you to
- Record tests with a beautiful cloud-based recorder for web and mobile applications
- Schedule tests for execution on any browsers or devices you own
- Integrate tests to your CI/CD pipeline with out-of-the-box integrations to Jenkins and other tools
- Extend TestProject with a coded test SDK, REST API, and numerous community Add-Ons
- Analyze results with complete analytics and drill-down reports
This wealth of features empowers the entire team where nontechnical users can record tests and developers can complete them with their code. This way, it is very helpful for the agile teams as it allows them to easily collaborate and share the test artifacts.
TestProject is built on top of standard open-source automation tools like Selenium and Appium. Unlike those open source tools running on their own, it removes all the complexities of installing and managing drivers for each platform and browser that you use.
With the installation of a single executable, you get the ability to use all the browsers on the machine and any attached mobile devices to record, execute, debug and develop your test.
This tool has been built in stealth for almost two years and is already being successfully used in more than 2000 organizations worldwide. Tricentis has recently acquired TestProject and committed to further accelerate development while keeping TestProject free test automation tool for everyone.
Features of TestProject
#1) In-browser Smart Test Recorder
We would say that this feature is the unique selling point of this tool. To create automated tests, you do not need to write a single line of code. With TestProject’s Smart Test Recorder feature, you can create automated tests just by recording the steps performed on the browser.
Wherever you need to perform complex steps that go beyond a traditional recorder, you can lean on any of the existing add-ons built by the community.
For each step that you perform, it will be recorded as a test and automatically Selenium or Appium code will be generated in the background. You can explore more about this feature in the latter part of this tutorial where we have demonstrated how does this tool work.
So, this feature makes the test automation simple for anyone. You can effortlessly create automated tests for Web, Android, and iOS applications.
Let us go in-depth of this recorder functionality and explore more about it.
#1) Mobile Screen Mirroring:
This option allows you to mirror your mobile screen in the browser, seeing exactly what is on the device in real-time. Just connect your mobile device to your machine through a USB cable and you are good to go!
#2) Element Inspector:
You can test any element in your app. You can select from different built-in or suggested actions, validations, attributes, save element.
#3) Element Explorer:
Another feature is element explorer. This can be used to simply view and examine the structure of your mobile application screens. You can analyze the element hierarchy, select from different locator strategies, and see all available properties of UI components.
#4) Test Editor:
This is the most helpful feature of a test recorder that aids in creating, editing and organizing your tests. At each step, you can add comments, select elements, create parameters, perform certain actions, define failure behavior, take screenshots, set timeouts and do much more.
You can change the order of steps, insert steps in between, add or delete steps.
#5) Other features making recorder even more powerful:
The recorder automatically suggests add-ons that can be helpful for your tests. TestProject does the AI-based matching, analyzes the elements in your application and then based upon this analysis, it recommends add-ons that can enhance your tests. You can instantly add these addons and extend your tests.
Another powerful way that extends your tests is nested tests. For the complex scenario, you can record an entire test as a single step and fit that into your complete test. This reduces the amount of maintenance required when any of these test steps change.
Furthermore, you have the code export feature. You can generate the source code for your recorded test, have a developer modify it, then upload back to TestProject. I have explained in detail about this code export feature in the latter part of this article where I have demonstrated how to use this tool.
#2) Powerful Addons
TestProject gives lots of awesome addons that help to make your tests very powerful. Addons allow you to greatly expand what you can do with your tests. They are the set of reusable coded actions that can be integrated into any test.
You can create your addons using TestProject SDKs or you can use addons that have been created and shared by the test automation community. They will save a lot of valuable development time for you.
Creating new addons:
Use existing addons:
Addon actions can be used seamlessly in your test. You can create and use them in the same way you would interact with built-in actions. If you see the list of addons, you will find that there are a lot of really useful addons provided by the community.
For example, the jRand addon gives you instant access to random data generators for a wide variety of test data.
You have many addons available based on your needs. For example, if you are working with jquery, this tool offers you 4-5 jquery addons that you can use. If you want to generate random email ids and passwords for testing, you can use Generate Random Credential addon.
Some of the featured add-ons are Advance element finder, JS actions, Element Extensions, and Generic Extensions. As the TestProject community continues to grow, the number of possibilities will continue to increase, making the possibilities of what this tool can do truly endless.
Suppose if you have some scenario where you want to call an API directly in your test instead of sending a request to the web page. By adding the RESTful API addon, you can simply do this action in your test.
For example, you could create a user by sending the request to the API, then in the very next step log in through the UI using that exact user’s credentials.
Below screen print is a demonstration on how to use RESTful API in your test:
With the single click at the install button, you can get any addon you want in a snap. Store it in the test project repository and it is ready for anyone in your team to use.
#3) Code Generation Feature
This feature allows you to generate Java code from the recorded iOS, Android and Web tests. You can make any modifications to the code, extend your tests and reupload them as coded tests to TestProject. This allows developers to engage in developing and maintaining the tests created by non-technical testers.
For those who do not want to use recorder but first want to code their tests, the TestProject offers SDK for Java and .NET core. The SDK provides a convenient, standardized wrapper around Appium and Selenium that is 100% compatible with TestProject. You can go to the developer's tab and generate the developer key.
This key is used for debugging and debugging TestProject coded tests and add-ons locally. And, you can start coding your test with TestProject.
Using the mix and match of both ‘record & playback’ feature and code development feature helps greatly in achieving the test automation goals.
While the tool historically had both a free and a paid pricing plan, we have heard that after the Tricentis acquisition of TestProject, they have committed all current features of TestProject to be available for free.
From our conversation with the TestProject CEO Mark Kardashov, we have learned that the only feature of TestProject that will cost money, will be the access to the live support that can be added into the app.
Pros of TestProject
#1) FREE end-to-end test automation platform: The biggest advantage of this tool is that it offers a free forever plan. You get so many exciting features that typically cost thousands of dollars at no cost!
#2) Highly collaborative environment: You can easily add an unlimited number of team members in your project and enjoy a great collaboration experience. All you need to do is to just add their email addresses and access permissions and then they can immediately start contributing to the project.
This makes TestProject very useful for Agile teams and CI/CD based projects.
Agents installed on the local machine allows to set up and run tests in any browser or device, but these tests are stored in a shared cloud-based testing repository where team members can work together in running and maintaining them.
#3) Ultimate user-friendliness: Both the user interface and user experience of this tool are just awesome. At every step, you will be guided with tooltips and short tutorials that will guide you through working with this tool and exploring its features.
#4) One to One customer support: Just a day after I created my account on TestProject, I got an email from the customer success team to ask about my progress with the tool so far and to tell them about my test automation needs so that they can help to make the best out of TestProject.
They also schedule a 1:1 online meeting to demonstrate TestProject’s features and capabilities or even help you start creating tests.
Additionally, in the support section of the website, you can also submit the ticket for any issue or problem you face while using the tool. We also learned that TestProject is launching a community forum and enhanced online documentation in the coming months.
Let me share my experience with their customer support. Once I was having a technical issue with the tool. I logged a ticket at the support center and I got the response within an hour.
That response gave the very detailed and perfect solution for my issue. This quick acknowledgment and response by the TestProject team are appreciable.
This level of customer support is rarely seen in free to use tools.
#5) Great community support: The community support for this too is also great. You can access the forum and the blog where you can go through the general announcements and commonly asked questions.
In TestProject’s blogs and forums, you will get tutorials for creating coded tests using TestProject's SDK and you will also get tutorials for using and developing addons.
#6) Compatibly with various Platforms: The smart test recorder allows you to effortlessly create automated tests for web, Android and iOS (in fact, iOS on windows).
#7) Saves time and effort: There are a lot of features in this tool that save your time and efforts to a great extent. For example, the automatic code generation feature saves your struggle of manually developing the code. Similarly, the automatic test document generation and manual test generation saves your time and effort that you would have put on creating documents.
Cons of TestProject
We did not find any cons as such in this tool. But there are certain limitations that can be addressed:
#1) Currently, it supports code generation only in the Java programming language. More programming languages should be added. Furthermore, it seems like the TestProject team is already working on this. I saw under the code generation functionality that C# programming language is coming soon.
#2) If your agent gets stopped, it does not show any warning sign or notification. It would be better if it pops up a message informing the user so the user can look into it and restart the agent.
Having discussed enough of this tool, its features, pricing, advantages, and limitations, let us now proceed with the practical demonstration of using this tool.
Getting Started with TestProject: Installation and Setup
TestProject is very easy to install and setup. The installation wizard guides you very smoothly through every step required for setup. The best part is that you just need to download one single exe and follow some easy steps to get started. There are no perquisites or dependencies to install anything else to get started with this tool.
Let me walk you through installation and setup steps:
Step #1) You can do free signup on their website TestProject
Step #2) Upon signup, you will get the confirmation email. Once you do the confirmation from your email account, it will sign you in and take you to your TestProject account homepage.
Step #3) What we liked about this tool is its user-friendliness. The signup is very smooth and once you are logged in, it will take you to a quick tour of a few necessary steps that are helpful to start your first automation project.
First, you need to install the TestProject agent. I am using the Windows machine here, so I am going to install the agent for Windows.
Just click on ‘Download and Install’ and then an agent exe will start downloading
This agent incorporates all the drivers and SDKs you would need for test automation.
Step #4) Once the exe download is complete, click on it and start the installation wizard.
Click on Next and just follow the Agent setup steps as directed through the wizard.
Step #5) Once the agent installation is finished, you need to click on the ‘Agent is installed’ button.
Step #6) Once you click on Agent is installed, you will be asked to register your agent. Give any alias name to your agent, check for its availability and then click on Register and Finish:
Step #7) Then you will get a message saying ‘Waiting for Agent to complete registration’.
Step #8) Finally, you will get the success message and you are now ready to go to the testing dashboard.
Hands-On With TestProject
Let us start working with TestProject and share our experience with it.
The green icon highlighted in the below screenshot (next to the bell icon in the top bar) shows that your TestProject agent is up and running. You will also get your first demo project created under the ‘My Projects’ section.
Once you will click on Next on your FirstProject, it will ask you to add team members. You can simply do so by entering their email IDs, set their access and click Finish. They will receive the invitation link and join immediately to start collaborating.
Now, click on Open Project to get started:
Now you will see the list of Tests:
On the left side of the navigation pane, you will see options like Test & Jobs, Elements, Applications, Data sources, and Parameters.
The Test Folders section displays the folders that can be used to easily organize your tests.
Elements section contains the UI elements that are mapped during the creation of tests, so you can manage and update elements that are used throughout tests for easy maintenance.
Under the Applications section, you will be having Web or Mobile test targets. Every test is associated with a specific app.
Data sources contain external data files to allow data-driven automation.
Under the Parameters section, you can manage global parameters for the tests.
Under the Demo Project (My first Project), there will be a demo test that performs some basic operations and validations on the TestProject login page.
On the top menu bar, you will see the option to create a New Test from where you can create your own test for the app you want to test.
Let us first explore this simple demo test to see what all functionalities this tool offers.
As you can see below, there are many steps in this step which are testing validating the Login and logout functionality. First, it navigates to the test project app URL and then clicks on the username textbox, enters the username, similarly does for the password, then clicks the Login button, validates if the login is successful, performs some other steps and validations and then logs out.
Clicking on each step gives complete details about that step. For example, below here I clicked on the first step i.e. to Navigate to app URL and then it gave me complete details about the step.
The best thing is that it's so easy and fun to play around the test. You can easily customize it by changing the parameter values, add your own parameters, disable any steps, add new steps, etc.
As you can see in the below screenshot, you can define the on failure behavior, number of step reattempts and a lot more. Inside input parameters, you can click on the +icon and see, add or edit the parameters and their values.
You also get an option to make parameter values secret to store sensitive information such as passwords and tokens.
If you will click at the three dots corresponding to the test, you will find so many options like a copy to project, move to folder, change input parameters, duplicate test, view data source template, generated code, manual test, test document.
I just loved the Test Document option here. As a tester, this option is so useful and a great time saver. I just had to click on ‘Test Document’ and it auto-generated very well-defined test documentation in excel where I got all the test details, and test steps details documented perfectly.
So, with TestProject you do not need to spend time on documentation. If you have an automated test, you can simply click on this option and generate the test document. Below image shows how an autogenerated test document looks like:
Similarly, you can use the ‘Manual Test’ option, what it does is that it converts the automation test to manual test document which contains all the steps, inputs and actions enlisted. So, if you want to perform that test manually, you can simply refer to this manual document and go ahead.
All you need to do is to just fill the test results in the doc. This way, it saves so much of the time which otherwise would have been put in converting the automation test to manual test and creating the document from scratch.
See below the manual doc generated. It contains everything test step description, test data, expected result, comments. All you need to do is to just fill the actual result and pass/fail status when you perform this test manually.
Another very useful option here is ‘Generated Code’. You can generate source code corresponding to any automated test and can use it for further development or modifications. Currently, it generates code in Java language. You can choose among different browsers as seen below:
Once you will click the ‘Generate button’, you will get the source code emailed to your account.
And, as we discussed earlier in this article, the tool also gives you a powerful recording feature. You can use that to modify an existing test or create a new test.
Let us now see how to create a new test with TestProject.
Then, it will ask you to add the web application to the testing list
Once you click on Finish, it will ask you what you would like to do with the recording. You can create the test using TestProject recorder functionality, edit the test manually, or create a manual test. I selected the record option to explore the recorder functionality of this tool. Let's see how it works:
Once I selected ‘record’ and clicked ‘Start Testing’, it opened the Google Chrome browser in a new window and took me at the application URL i.e. Google search page. The recorder recorded all the actions performed and automatically created test steps. As you can see below, a total of 7 steps are created. Once, you stop the recording, the test will be saved.
The best part is that at every step, you can select and freeze any element using double shift and then select the actions or validations that are required to be performed. For example, at step number 5 in this test, I select the below highlighted div and then performed an action to validate if it contains the text ‘Software Testing Help?’.
Let us also see the code is generated for this test. I used the ‘generated code’ option and got the code in an email:
And, below is how the code looks in Eclipse:
At any point in time, I can modify this code and reupload in TestProject to be executed again as a coded test.
In this tutorial, we discussed the main features of the TestProject automation tool along with its pros and cons. We also saw the practical demonstration of the functionalities.
We have explored a lot!
We are sure that now you will be excited to use this cool tool. If you are someone who works or wants to work in test automation, this tool is worth trying.
Check it out here and let us know if you have any queries in the process!