Learn the Different Types of Automation Testing with Some Misconceptions About Test Automation:
In this second part of test automation tutorials series, I will briefly describe the types of automated tests and then most importantly I will clear some misconceptions about test automation.
What is Automation Testing?
Automation testing can be defined as a way to run a set of tests over and over again without having to execute them manually. Introducing automation tests in your test strategy is a way to save money and time.
What You Will Learn:
Types of Automation Testing
Types of automation tests define what kind of test suites can be automated. Many testers confuse this topic with the types of automation frameworks which define how you will design your test suite into an automation pack which can be executed conveniently.
In this article, we will take a close look at the Automation testing types and in the end, will take a brief look at the automation frameworks.
Let’s understand the above classifications in detail:
Automation Based on the Type of Testing
Automation of Functional Tests:
Functional tests are written to test the business logic behind an application. Automating these mean writing scripts to validate the business logic and the functionality expected from the application.
Automation of Non-Functional Tests:
Non-functional tests define the non-business requirements of the application. These are the requirements related to performance, security, databases, etc. These requirements can remain constant or can be scaled as per the size of the software.
Automation Based on the Phase of Testing
Automation of Unit Tests:
These tests are run during the development phase itself, ideally by the dev after the completion of development and before handing over the system to the testers for testing.
Automation of API Tests:
API tests are run during the integration phase. These may be run by the development or testing team and can be run before or after the UI layer is built for the application. These tests target the testing based on the request and response on which the application is built.
Automation of UI based tests:
UI Based tests are run during the test execution phase. These are specifically run by the testers and are run only once before the UI of the application is handed over to them. These test the functionality and business logic of the application from the front end of the application.
Automation Based on the Type of Tests
Unit Tests are the tests that are built to test the code of an application and are usually built into the code itself. They target the coding standards like how the methods and functions are written.
These tests are more often written by the developers themselves, however, in today’s world, automation testers may also be asked to write them.
Executing these tests and getting no bugs from them will mean that your code will compile and run without any code issues. These tests usually do not target the functional aspects of the application and as they target code, it is more appropriate to automate them so that they can be run as and when required by the developer.
The smoke test is a famous test performed in the test life cycle. These are post-build tests, they are executed immediately after any build is given out of the application to ensure that the application is still functioning after the build is done.
This is a small test suite and is something that will be executed multiple times and thereby it makes sense to automate it. These tests will usually be of a functional nature and depending on the type of application a tool can be picked for them.
API testing has become very famous in the past few years. Applications built on the API architecture can perform this testing.
In API testing, the testers validate the business layer of the application by checking the request-response combinations for the various API’s on which the application is built. API Tests can also be done as a part of the integration tests below.
Integration test as the name itself suggests means testing the application by integrating all the modules and checking the functionality of the application.
Integration testing can be done through API testing or can be done through the UI layer of the application.
UI tests are done from the UI layer or the frontend of the application. These may target testing the functionality or simply test the UI elements of an application.
Automating the UI to test the functionality is a common practice. However, automating the GUI features is one of the more complicated automation.
One of the most commonly automated test suites is the regression test suite. Regression, as you may already know, is the test that is done at the end of testing a new module to ensure that none of the existing modules have been affected by it.
It is repeated after each new iteration of testing and the main test cases stay fixed with usually a few new additions after a new iteration. As it is frequently run almost all the test teams try to automate this pack.
Automation as Continuous integration:
Continuous Integration may again be running on the automated regression tests itself, however, in achieving CI, we enable the regression or identified test suite to be run every time when a new deployment is done.
Security testing can be both functional as well as a non-functional type of testing which involves testing the application for vulnerabilities. Functional tests will compose of tests related to authorization etc., whereas non-functional requirements maybe test for SQL injection, cross-site scripting, etc.
Performance Tests and Quality control:
Performance tests are non-functional tests which target the requirements like testing of load, stress, scalability of the application.
Acceptance tests again fall under functional tests which are usually done to ensure if the acceptance criteria given by the client has been fulfilled.
So far, we have described the type of tests that can be automated and various classifications of the same, all classifications eventually will lead to the same end results of a test suite being automated. As we said earlier a little understanding is required on how these are different from frameworks.
Once you have identified the tests that you want to automate from the above classification, then you will need to design your logic in a manner to execute these tests smoothly, without much manual intervention. This design of a manual test suite into an automated test suite is where the frameworks come in.
Now we will explore Top 3 Test Automation Types
- Unit Testing
- API Testing
- GUI Testing
#1) Automated Unit Tests
Automated Unit tests are written to test the code level. Bugs are identified in the functions, methods, and routines written by the developers.
Some companies ask the developers to do the unit testing themselves and some hire specialized test automation resources. These resources have access to source code and they write unit tests to break the production code.
Due to the presence of unit tests, whenever the code compiles, all unit tests run and tell us the result that if all the functionality are working. If any unit test fails, then it means that there is now a bug present in the production code.
Some of the most popular tools present in the market include NUnit and JUnit. Microsoft also provides its own framework for unit testing called MSTest. Go through the websites of these tools and they will provide you more examples and tutorials on how to write unit tests.
#2) Automated Web Service / API Tests
An Application Programming Interface (API) makes it possible for the software to talk to other software applications. Just like any other software, APIs need to be tested. In this type of testing, GUI is usually not involved.
What we test here is usually the functionality, compliance and security issues. In web applications, we can test the Request and Response of our application that if they are secure and encrypted or not.
This is one of the examples where we can use API Testing. The most popular tool for API testing is SOAPUI which has both free and paid versions. There are other tools as well, which you can use according to your need.
#3) Automated GUI Tests.
This type of automated testing is the toughest form of automation as it involves testing of a User interface of the application.
It is tough as the GUI’s are highly subject to change. But this type of testing is also closest to what the users will do with our application. As the user will use the mouse and keyboard, automated GUI tests also mimic the same behavior by making use of mouse and keyboard to click or write to objects present on the user interface.
Due to this, we can find bugs early and it can be used in many scenarios such as regression testing or filling up forms which takes too much time.
The most popular GUI testing tools include Micro Focus Unified Functional Testing (UFT), Selenium, Test Complete and Microsoft Coded UI (which is a part of Visual Studio ultimate and premium editions).
Just like the types of automation tests, there are multiple types of frameworks as well.
Some commonly used automation frameworks include:
- Linear (Record and playback)
- Keyword Driven
- Data Driven
- Page Object Model
Further Reading=> Automation Frameworks
As you can see the first step in the process of automation is to identify the type of automation, then you can identify the framework to design and keeping these in mind you can select the tools that suit your needs.
Based on the type of testing you are targeting and the type of framework that you may want to build around it, the following tools are available to use:
- Selenium: Very powerful tool for testing of Web Applications. Provides multiple browser support.
- Junit and Nunit: Tools majorly used for Unit testing by the developers.
- QTP: Great tool for non-web applications and comes with a built-in object repository.
- Sikuli: Open source tool for GUI testing.
- Soap UI: Tool for API testing.
- Rest Assured: Library to create an API test framework.
- Appium: Tool that supports mobile testing, native app testing, hybrid, and mobile web application testing.
- Jmeter: A tool that is used for performance tests.
- TestNG: TestNG is not an automation tool in itself, however, it provides great support to automation frameworks built with selenium, appium, rest assured, etc.
Further Reading=> Test Automation Tools
Misconceptions About Automation Testing
Over the years, I have heard some misconceptions about test automation. I think I should clear them too in this article.
Misconception #1. Automation is here to replace manual testers.
Test automation is for helping the testers to make testing faster and in a much reliable manner. It can never replace humans.
Think of test automation as a car. If you walk, you will take around 20 minutes to reach your home. But if you use a car, you will reach in two minutes. The driver of the car is still you, a human, but.. the car helps the human to achieve his/her goal faster. Also, most of your energy is saved, as you didn’t walk. Thus you can use this energy to perform more important things.
Same goes with automation testing. You use it to quickly test most of your repeated, long and boring tests and save your time and energy to focus & test new and important functionality.
As James Bach said a wonderful quote:
“ Tools don’t test. Only people test. Tools only perform actions that “help” people test. “
Tools can click on objects. But where to click will always be told by a manual tester. I think you get my point now.
Misconception #2. Everything under the sun can be automated
If you try to automate 100% of your test cases, maybe you will be able to do so, but if you could do that, then our first point becomes false. If everything is automated, then what will a manual tester do?
Actually, the point is, that you cannot automate 100% of your test cases. Because we, as testers, believe that no application can be 100% tested. There will always be some scenarios which we will miss. There will always be bugs that come only when your application will be used by the clients.
If the application cannot be 100% tested, then how you can promise 100% automation?
Also, there is a very thin chance that you will be able to automate all of your existing test cases. There are always scenarios which are difficult to automate and are easier to do manually.
For example, One user will enter the data, the second user will approve the data, the third user will view the data and the fourth user is prohibited to view the data. These scenarios can be automated, but they will take a lot of time and effort. So it will be easier if you just do that manually.
Remember, we use cars to go to distances, but there may be lengthy signals on the way, there will be fuel consumption, there will be parking space issues, parking charges, and a lot more headache. In some scenarios, we just walk and reach our destination :).
Thus, you should not try to automate everything. Only automate those scenarios which are important and those which take a lot of time to do manually.
Misconception #3. Automation only involves recording and playback.
Please don’t live in a fantasy world. This fantasy is actually created by false advertisements from different automation tool vendors. They say you just record and playback your steps and your test cases will be automated. Well, that’s a Big Lie!
Automation is everything and not just recording and playback. Pure automation engineers normally don’t use recording and playback feature at all. Recording and playback are generally used to get an idea of how the tool is generating a script for our steps.
Once we get to know the scripting, we always use scripting to create automated tests. Remember, you must know programming if you want to do test automation. On the other hand, don’t be dis-hearted if you don’t know programming. Just like any other task, programming can also be learned with practice and dedication.
I know people, who are not even from a Computer science background, but they learn to programme and now they are awesome automation engineers. At Microsoft, they hire testers who can do programming. They are called SDET (Software development engineers for test). The first line of the Job description says “SDET’s write a lot of code….”.
Please learn to programme, don’t run away from it. It will make you an amazing tester.
I hope this article would have helped you to clear some concepts related to test automation.
We have covered a high level of various types of automation testing, with various ways to classify.
The main classifications include:
- Automation based on Type of testing (Functional or non-functional).
- Automation based on the Phase of testing (Unit, API or UI).
- Automation based on the various types of tests (Multiple testing types).
We have also listed down the various tools that can be used for these types of automated testing.
In our upcoming article, we will discuss the step by step procedure of how to start test automation in your organization.