All You Need to Know About codeless or Scriptless Test Automation Framework and the Best Tools:
This article will give you a complete overview of Scriptless Automation Framework in simple terms.
The design goals, Implementation requirements, Benefits & Myths of Scriptless Automation are explained with perfect examples for your easy understanding.
While concluding this article, you will get a good insight into all the answers to the ‘why’, ‘what’, ‘when’ and ‘how’ questions related to this framework.
What You Will Learn:
- Levels of Automation
- Best Scriptless Test Automation Tools
- Design Goals
- Insight into Implementation Requirements
- Benefits of Scriptless Automation
- Myths About Scriptless Automation
- Recommended Reading
Levels of Automation
Let’s take a quick look at the levels of automation, before diving deep into the codeless Test Automation Framework.
As the name itself suggests, this requires creating a program/software to automate tests or processes.
Example: If you have to perform unit testing of a particular function or a class, then you would need to create test programs to test your function or class. There is no way to automate your units tests without coding.
In general, if any automation tool or framework mandates the testers to create some code or script for providing test input, execution and reporting of automated tests, then such tools are categorized in this level.
In this level, you do not need any coding or scripting knowledge to create automated tests. However, as a framework, it is a good practice to provide the capability of customization.
For advanced users, customization can be done by writing their own code for covering some unique test scenarios.
Advanced Automation using Artificial Intelligence (AI)
This is the highest level of automation that can be achieved:
- Create test scripts from test case documents (probably using Natural Language Processing technology).
- Auto-correct test scripts based on the application changes for effective regression testing
- Auto-generate test scripts based on the user actions/use cases on the application under test. This is the advanced version of the traditional record and playback feature.
- Auto-validation of the test results based on Machine learning without user inputs on what needs to be validated and test analytics.
In this article, we would be focusing on the level-2 automation which is Scriptless.
Before that, let's see the list of some of the top Scriptless Automation Tools.
Best Scriptless Test Automation Tools
The top 4 tools that fall under this category are stated below for your reference.
TestCraft is a scriptless Selenium test automation platform. Its revolutionary AI technology and unique visual modeling allow for faster test creation and execution while eliminating test maintenance overhead. Testers create fully automated test scenarios without coding. Customers find bugs quickly, release more frequently, integrate with the CI/CD approach and improve the overall quality of their digital products.
- No programming skills required. Leverage manual testers’ business knowledge and allow them to create codeless automated test scenarios
- Substantially reduce maintenance costs. Scripts are automatically adjusted to change due to our AI algorithm. Major changes are adapted with a few clicks
- Selenium-based. Quickly integrate and leverage modules developed by the community
- Provides immediate time to value. Nothing to install (SaaS). Easy to master
Some of the important design goals that should be considered are briefed below.
#1) Input Driven
Any input that needs to be provided to run automated tests should be coming outside the ‘coding’ part of your automation framework. Input can be your test steps, test conditions, test data, expected results etc.
For every type of input, the framework should support in the form of keywords that are easy to use and thereby understand the context too.
#2) Allow Customization
This clearly means that you should provide options to customize the handling of test scripts for advanced users so that they can include unique scenarios. Advanced users can plug-in their test scripts (in the form of a program) to the framework’s test input.
This is a really important design goal as understanding how to set up, configure and use automation framework it should not become very difficult than coding.
One such Example would be to provide a template that has all the required test inputs with default values (but relevant to the context).
In this way, the users would quickly understand how to provide test inputs based on their application needs. Another way would be to provide a high level of abstraction in the test inputs.
Example: Let's consider the button click simulation. Here, as a user, I am only interested in a simple action called click a button.
The framework should be capable of performing that action by internally handling different methods of identifying a button control followed by the action. In fact, this is somewhat related to the Input driven goal where you have to define keywords that are easy to understand as in this example, ClickButton.
#4) Platform Agnostic
Must be capable of running on different PC and Mobile platforms. Different browsers in case of scriptless automation developed for web applications. As part of this goal, you should also consider supporting the common test inputs across different platforms.
#5) Technology Agnostic
In today’s scenario, we all know that technologies are vast and there are a lot of options to develop applications.
No matter what technology is used to develop an application under test (under a given application type such as web or mobile), the automation framework should be capable of testing those applications with some minimal or no changes to test inputs and the framework engine itself.
#6) Test Environment Agnostic
Should be capable of running on cloud and on-premises, virtual environment.
Should provide options to plug-in the other software components to cover complex test scenarios.
Let’s take an Example of the e-commerce website which has both front end application and web services and you want to test the functionality of your application along with the performance. This requirement would definitely need plug-ins for tools that can test web services and performance of the website as well.
Insight into Implementation Requirements
Before taking a look at the specifics of implementation, let us first understand the high-level architectural components of Scriptless Automation Framework.
There are 3 primary components (of course, the components can be further split into the lower level, but that is not the purpose for now!) that we are going to see to get a big picture of the framework:
#1) Framework Core Engine: This is the core component of the framework that does all the heavy lifting job of automation logic, validations, interfaces to the rest of the components of the architecture and third-party apps/scripts for customized test configuration and executions.
All the test input actions are forwarded to the action handler to process the actions on the AUT (Application Under Test). For developing this engine, it is important to choose the right programming knowledge and technology by considering the design goals mentioned above.
Depending on the smart selection of technology, you will be able to meet most of your design goals thereby providing the best possible benefits to the consumers of the automation framework.
#2) Test Input Interface: This is the front end component from the consumer's point of view and this is where you will provide all your test inputs such as test case steps, test conditions, test data, expected results etc.
Remember the design goal of simplicity, this component should be designed in such a way in which the consumers can quickly adapt and start using your framework. Failing to meet this goal would result in less usage of your framework itself.
The interface to the core engine is the interpreter which typically follows some protocol to exchange information on test inputs.
#3) Test Output Interface: This is the component which is responsible for outputting your test results, test logs, defect logging, reporting, and dashboards. The framework core engine will forward the results to the Output logger which is then rendered by the Output interface.
Out of the above components, the consumers would be typically interfacing with the Test input interface and Test output interface. Hence, we are going to explore these 2 components only.
Test Input Interface
Test inputs can be provided in different ways. However, it entirely depends on what type of consumers are we targeting for this framework.
- Files such as XML, Excel; Schema such as test data fields, test steps, data types and so on would be defined by the framework engine and that has to be followed while providing the test inputs.
- GUI based input where you will provide the relevant test fields so that the consumers can provide all the test details for execution.
- Visual editor based input where the consumers can provide in the form of flow charts (supporting basic logic such as conditions and loops). In fact, this is the most trending way for providing input interface though it is comparatively time-consuming for developing such interface.
The typical feature set that needs to be provided are:
- Ability to group test cases based on test types, test priority and so on.
- Must be able to create custom test suites for running smoke & regression tests, selective tests etc.
- Test inputs that are platform agnostic.
- Ability to execute a single test case multiple times.
- Should be able to execute a single test case for different test values.
- Ability to launch another executable for a specific test step or test case. E.g. Interface to Jenkins to automatically launch test execution.
- Test configurations for execution and output logging. Example: Timeouts.
- Provision to set up ‘test script library’ for re-usability across teams and projects that are deemed as necessary.
- Ability to import test data inputs from templates that are created in the past.
- Provision to import test case documents and map them to the test data inputs. Remember the ‘Extensibility’ design goal. Yes, this is the first step towards the next level of automation!
Test Output Interface
From the Scriptless Automation point of view, you must also ensure that you are able to view the test execution results in a seamless manner without any need to write special code.
The typical feature set that needs to be provided are:
- Ability to view test results, test logs, framework logs etc.
- Generation of various test reports and dashboards.
- Ability to create custom reports based on specific requirements.
- Must be able to stream test results at run time through GUI.
- Ability to interface with standard test case management tools.
- Should be able to export test results, reports and dashboards in different formats such as Excel, doc, pdf etc.
- Ability to filter and parse test logs for specific requirements such as Performance test engineering etc.
- Provision to push all test data (results, logs, selective data points) to advanced modules such as analytics. Remember the ‘Extensibility’ design goal. Yes, this is yet another first step towards the next level of automation!
Benefits of Scriptless Automation
Enlisted below are the main benefits of creating Scriptless Framework:
- It is not mandatory for the testers to have any programming or scripting skills.
- Apart from the testers (who are the primary consumers of Scriptless automation), the framework usage can be extended to the Product owners (in the Agile world), and Business Analysts as well for setting up and executing the User Acceptance Tests (UAT).
- Quicker development of automated tests that perfectly fit into the Agile process.
- Offers flexibility for both native and advanced users.
- Less ‘test’ related errors as the underlying scriptless framework would have been tested thoroughly. This, in turn, reduces the time spent on debugging and fixing test related issues.
- Easier maintenance of the overall automation suite.
Myths About Scriptless Automation
Given below are a few Myths about this framework.
#1) SAF is 100% scriptless: Actually, this is not 100% true. Though the goal is to make it 100% codeless, it is practically not possible to achieve this feat.
The primary reason for this is the presence of various complexities in the Application Under Test (AUT), technologies and the programming languages that are used in developing the software. Hence, one of the design goals that we have mentioned is about providing flexibility for customization.
#2) For using SAF, the testers do not need any programming or coding knowledge: Again, this is not 100% true. Actually, they do require some basic concepts of programming and logical skills.
When it comes to web or mobile based applications, the testers also need to know the basics of objects and classes that need to be included as a part of customized scripts.
There are some tools like TestCraft which, in fact, does not need the QA professional to have coding knowledge. Their drag and drop plus on-the-fly features allow for anyone to understand how to run tests.
#3) SAF is all about record and playback which can be relied on: No. Record and playback will just give the user a working script with a hardcoded value of inputs and flows which has to be later customized.
In any way, it does NOT cover all possible scenarios of user inputs and test steps.
Given below are some examples of Scriptless Automation for your easy understanding.
#1) Here is a test execution example
- Enter User in Username field
- Enter Password in Password field
- Click Login button
Each of the above are the keywords that are supported by Scriptless frameworks.
As a tester, you have to just pass the values as shown below:
Enterpassword dummypwd, mask=true
The words in italics are the keywords that accept arguments to perform actions.
Note: EnterPassword accepts 2 arguments. The second one is the flag to indicate masking of characters as it is typed in the password field.
#2) Let’s say, you want to automate a Payment Transaction Flow:
- Select Account type
- Enter amount for goods purchased
- Enter PIN
- Print Receipt
EnterPIN 1234, mask=true
PrintReceipt Customercopy=false, Merchantcopy=true
The words in italics are the keywords that accept arguments to perform actions.
Note: EnterPIN accepts 2 arguments. The second one is the flag to indicate masking of PIN digits as it is typed in the PIN entry field. PrintReceipt also has 2 arguments.
If you notice the flow above, this is a typical transaction flow. You could use the entire flow as a “transaction test template” for various test cases. This is how you make the generation of test steps and test inputs easy for the testers.
Reusability of keywords, group of keywords and flow based keywords can be done in Scriptless Automation Framework (SAF).
From this article, you would have figured out that developing a code-less or script-less automation framework is a challenging and equally interesting task.
You can develop such frameworks for any kind of application testing i.e. web-based, mobile apps, Web services, GUI, database, embedded device applications etc.
However, utmost care should be taken to develop a stable, reliable and high-performance tool. This is just a word of caution to choose the right technology and scripting/programming language to develop a wonderful and useful framework.
Do you have any experiences in Scriptless Automation? Feel free to share your thoughts with us in the comments section below.