Learn What Are Different Types of Automation Testing with Some Misconceptions About Test Automation:
In this second part of test automation tutorials series, I will briefly describe the types of automated tests and then most importantly I will clear some misconceptions about test automation.
What is automation testing?
Automation testing can be defined as a way to run a set of tests over and over again without having to execute them manually. Introducing automation tests in your test strategy is a way to save money and time.
What You Will Learn:
Types of Automation Testing
Types of automation tests define what kind of test suites can be automated. Many testers confuse this topic with types of automation frameworks which defines how you will design your test suite into an automation pack which can be executed conveniently.
In this article, we will take a close look at the Automation testing types and in the end, will give a brief about automation frameworks.
Let’s understand the above classifications in detail:
Automation based on the type of testing
Automation of Functional Tests:
Functional tests are written to test the business logic behind an application. Automating these mean writing scripts to validate the business logic and functionality expected from the application.
Automation of Non-Functional Tests:
Non-functional tests define the non-business requirements of the application. These are requirements related to performance, security, databases etc. These requirements can remain constant or can be scaled as per the size of the software.
Automation based on the phase of testing
Automation of Unit Tests:
These tests are run during the development phase itself, ideally by the dev after completion of development and before handing over the system to the testers for testing.
Automation of API Tests:
API tests are run during the integration phase. These may be run by development or testing and can be run before or after the UI layer is built for the application. These tests target the testing based on request and response on which the application is built.
Automation of UI based tests:
UI Based tests are run during the test execution phase. These are specifically run by testers and are run only once the UI of the application is handed over to them. These test the functionality and business logic of the application from the front end of the application.
Automation based on the type of tests
Unit Tests are tests built to test the code of an application and are usually built into the code itself. They target the coding standards like how methods and functions etc. are written.
These tests are more often written by the developers themselves, however, in today’s time automation testers may also be asked to write them.
Executing these tests and getting no bugs from them will mean that your code will compile and run without code issues. These tests usually do not target functional aspects of the application and as they target code, it is more appropriate to automate them so that they can be run as and when required by the developer.
The smoke test is a famous test performed in the test life cycle. These are post-build tests, they are executed immediately after any build is given out of the application to ensure that the application is still functioning after the build is done.
This is a small test suite and is something that will be executed multiple times and hence it makes sense to automate it. These tests will usually be of a functional nature and depending on the type of application a tool can be picked for them.
API testing has become very famous in the past few years. Applications built on API architecture can perform this testing.
In API testing the testers validate the business layer of the application by checking the request-response combinations for the various API’s on which the application is built. API Tests can also be done as part of the integration tests below.
Integration test as the name suggests means testing the application by integrating all the modules and checking the functionality of the application.
Integration testing can be done through API testing or can be done through the UI layer of the application.
UI tests are done from the UI layer or the frontend of the application. These may target testing the functionality or simply test the UI elements of an application.
Automating the UI to test the functionality is a common practice. However, automating the GUI features is one of the more complicated automation.
One of the most commonly automated test suites is the regression test suite. Regression, as you may already know, is the test done at the end of testing a new module to ensure none of the existing modules have been affected by it.
It is repeated after each new iteration of testing and the main test cases stay fixed with usually few new additions after a new iteration. As it is frequently run almost all test teams try to automate this pack.
Automation as Continuous integration:
Continuous Integration may again be running of the automated regression tests itself, however, in achieving CI, we enable the regression or identified test suite to be run every time new deployment is done.
Security testing can be both functional as well as the non-functional type of testing which involves testing the application for vulnerabilities. Functional tests will compose of tests related to authorization etc., whereas non-functional requirements maybe test for SQL injection, cross-site scripting etc.
Performance Tests and Quality control:
Performance tests are non-functional tests which target requirements like testing of load, stress, scalability of the application.
Acceptance tests again fall under the functional tests which are usually done to ensure the acceptance criteria given by the client has been fulfilled.
Above we have described the type of tests that can be automated and various classifications of the same, all classifications eventually will lead to the same end results of a test suite being automated. As we said earlier a little understanding on how these are different from frameworks.
Once you have identified the tests you want to automate from the above classification you will need to design your logic in a manner to execute these tests smoothly and without much manual intervention. This design of a manual test suite into an automated test suite is where frameworks come in.
Now we will see more details of 3 Top Test Automation Types
- Unit Testing
- API Testing
- GUI Testing
#1. Automated Unit Tests
Automated Unit tests are written to test on the code level. Bugs are identified in the functions, methods, and routines written by developers.
Some companies ask developers to do the unit testing themselves and some hire specialized test automation resources. These resources have access to source code and they write unit tests to break the production code. Due to the presence of unit tests, whenever the code compiles, all unit tests run and tell us the result that whether all functionality is working. If any unit test fails, that means there is now a bug present in the production code.
Some of the most popular tools present in the market are NUnit and JUnit. Microsoft also provided its own framework for unit testing called MSTest. Go through the websites of these tools and they will provide examples and tutorials on how to write unit tests.
#2. Automated Web Service / API Tests
An Application Programming Interface (API) makes it possible for software to talk to other software applications. Just like any other software, APIs need to be tested. In this type of testing, GUI is usually not involved.
What we test here is usually the functionality, compliance and security issues. In web applications, we can test the Request and Response of our application that whether they are secure and encrypted or not.
This is one of the examples where we can use API Testing. The most popular tool for API testing is SOAPUI which has both free and paid versions. There are other tools as well, which you can use according to your need.
#3. Automated GUI Tests.
This type of automated testing is the toughest form of automation because it involves testing of a User interface of the application.
It is tough because GUI’s are highly subject to change. But this type of testing is also closest to what users will do with our application. Since the user will use the mouse and keyboard, automated GUI tests also mimic the same behavior by making use of mouse and keyboard to click or write to objects present on the user interface. Due to this, we can find bugs early and it can be used in many scenarios such as regression testing or filling up forms which takes too much time.
Like types of automation tests, there are multiple types of frameworks also.
Some commonly used automation frameworks are:
- Linear (Record and playback)
- Keyword Driven
- Data Driven
- Page Object Model
Further Reading=> Automation Frameworks.
So as you can see the first step in the process of automation is to identify the type of automation, then you can identify the framework to design and keeping these in mind you can select tools to suit your needs.
Based on the type of testing you are targeting and also the type of framework you may want to build around it following tools are available to use:
Selenium: Very powerful tool for testing of Web Applications. Provides multiple browser support.
Junit and Nunit: Tools majorly used for Unit testing by developers.
QTP: Great tool for non-web applications and comes with a built-in object repository.
Sikuli: Open source tool for GUI testing.
Soap UI: Tool for API testing.
Rest Assured: Library to create an API test framework.
Appium: Tool that supports mobile testing and supports native app testing, hybrid and mobile web application testing.
Jmeter: A tool used for performance tests.
Test NG: Test NG is not an automation tool in itself, however, it provides great support to automation frameworks built with selenium, appium, rest assured etc.
Further Reading=> Test Automation Tools
Misconceptions about Automation Testing
Over the years, I have heard some misconceptions about test automation. I think I should clear them too in this article.
Misconception #1. Automation is here to replace manual testers.
Test automation is for helping the testers to make testing faster and in a more reliable manner. It can never replace humans.
Think of test automation as a car. If you walk, you will take around 20 minutes to reach your home. But if you use a car, you will reach in two minutes. The driver of the car is still you, a human, but.. the car helps the human to achieve his/her goal faster. Also, most of your energy is saved, because you didn’t walk. So you can use this energy to perform more important things.
Same goes with automation testing. You use it to quickly test most of your repeated, long and boring tests and save your time and energy to focus and test new and important functionality.
As James Bach said a wonderful quote:
“ Tools don’t test. Only people test. Tools only perform actions that “help” people test. “
Tools can click on objects. But where to click will always be told by a manual tester. I think you get my point now.
Misconception #2. Everything under the sun can be automated
If you try to automate 100% of your test cases, maybe you will be able to do so, but if that you could do, then our first point becomes false. Because if everything is automated, what will manual tester do?
Actually, the point is, you cannot automate 100% of your test cases. Because of we, as testers, believe that no application can be 100% tested. There will always be some scenarios which we will miss. There will always be bugs that come only when your application will be used by clients.
So if the application cannot be 100% tested, how you can promise 100% automation?
Also, there is a very thin chance that you will be able to automate all of your existing test cases. There are always scenarios which are difficult to automate and are easier to do manually.
For example, One user will enter the data, the second user will approve the data, the third user will view the data and the fourth user is prohibited to view the data. These scenarios can be automated, but they will take a lot of time and effort. So it will be easier if you just do that manually.
Remember, we use cars to go distances, but there may be lengthy signals on the way, there will be fuel consumption, there will be parking space issues, parking charges and a lot more headache. In some scenarios, we just walk and reach to our destination :).
So, you should not try to automate everything. Only automate those scenarios which are important and take a lot of time to do manually.
Misconception #3. Automation only involves recording and playback.
Please don’t live in a fantasy world. This fantasy is actually created by false advertisements from different automation tool vendors. They say you just record and playback your steps and your test cases will be automated. Well, that’s a Big Lie!
Automation is everything but recording and playback. Pure automation engineers normally don’t use recording and playback feature at all. Recording and playback are generally used to get an idea of how the tool is generating a script for our steps.
Once we get to know the scripting, we always use scripting to create automated tests. Remember, you must know programming if you want to do test automation. On the other hand, don’t be dis-hearted if you don’t know programming. Because like any other task, the programming can also be learned with practice and dedication.
I have known people, who are not even from a Computer science background, but they learn to programme and now they are awesome automation engineers. At Microsoft, they hire testers who can do programming. They are called SDET (Software development engineers for test). The first line of Job description says “SDET’s write a lot of code….”.
Please learn to programme, don’t run away from it. It will make you an amazing tester.
I hope this article will help you to clear some concepts related to test automation.
We have covered a high level of various types of automation testing, with various ways to classify.
The main classifications are:
- Automation based on Type of testing (Functional or non-functional)
- Automation based on the phase of testing (Unit, API or UI)
- Automation based on the various types of tests (Multiple testing types)
We have also listed down various tools which can be used for these types of automated testing.
In the next article, I will discuss how to start test automation in your organization step by step.
Feel free to give your feedback and suggestions, it will help us to improve in our future articles.