Recently we have been receiving too many questions on when and how to automate the testing process. Instead of answering them individually, we thought it would be better to have a discussion here.
This tutorial will shed light on when to automate? how to automate? Or should we automate our testing work?
It would always be better to start a meaningful discussion on such a vast topic to get in-depth ideas and thoughts from experts from different areas and their experience in Automation Testing.
What You Will Learn:
Tips Before Automating Your Testing Work
Why Automation Testing?
#1) You have some new releases and bug fixes in a working module. So how will you ensure that the new bug fixes have not introduced any new bugs in the previous working functionality? Hence, you also need to test the previous functionality.
So will you manually test all the module functionality every time you have some bug fixes or a new functionality added? Well, you might do it manually but then you are not testing effectively. Effective in terms of company cost, resources, time, etc.
Here comes the need for Automation.
– Automate your Testing procedure when you have a lot of regression work.
#2) You are testing a web application where there might be thousands of users interacting with your application simultaneously.
How will you test such a web application? How will you create those many users manually and simultaneously? Well, a very difficult task if done manually.
– Automate your load testing work for creating virtual users to check the load capacity of your application.
#3) You are testing an application where the code is changing frequently. You almost have the same GUI but there are more functional changes, thus testing rework is more.
– Automate your testing work when your GUI is almost frozen but you have a lot of frequent functional changes.
What are the Risks Associated with Automation Testing?
There are some distinct situations where you can think of automating your testing work. We have covered some risks of Automation Testing here. If you have made the decision of automation or are going to take it sooner then think of the following scenarios first.
#1) Do you have skilled resources?
For automation, you need to have people who have some programming knowledge.
Think of your resources. Do they have sufficient programming knowledge for automation testing?
If not, do they have technical capabilities or programming background so that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes, only then you need to think about automating your work.
#2) Initial cost for Automation is very high
It’s a well known fact that Manual Testing has too many costs associated with hiring skilled manual testers. If you are thinking automation will be the solution for you, then think twice.
Automation cost is too high for the initial setup i.e., cost associated with automation tool purchase, training and maintenance of test scripts is very high.
There are many unsatisfied customers regretting their decision to automate their work. If you are spending too much and merely getting some good looking testing tools and some basic automation scripts then what is the use of automation?
#3) Do not think to Automate your UI if it is not fixed.
Beware before automating the user interface. If the user interface is changing extensively, the cost associated with script maintenance will be very high. Hence, basic UI automation is sufficient in such cases.
#4) Is your application stable enough to automate further testing work?
It would be a bad idea to automate testing work in the early development cycle (Unless it is an agile environment). Script maintenance costs will be very high in such cases.
#5) Are you thinking of 100% Automation?
Please stop dreaming. You cannot 100% automate your testing work. Certainly, you have areas like performance testing, regression testing, load/stress testing where you can have a chance of reaching nearly to 100% automation.
Areas like User interface, documentation, installation, compatibility, and recovery where testing must be done manually.
#6) Do not Automate tests that run once
Identify application areas and test cases that might be running once and not included in the regression. Avoid automating such modules or test cases.
#7) Does your automation suite have a long lifetime?
Every automation script suite should have enough life that its building cost should definitely be less than that of manual execution cost. It is a bit difficult to analyze the effective cost of each automation script suite.
Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (general assumption depends on specific application complexity) to have a good ROI.
Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise, your tool will remain on the shelf giving you no ROI.
Handing over expensive automation tools to unskilled staff will lead to frustration. Before purchasing automation tools, make sure that the tool is the best fit for your requirements. You cannot have a tool that will 100% match with your requirements.
Find out the limitations of the tool that best matches your requirements and then use manual testing techniques to overcome those testing tool limitations. An open-source tool is also a good option to start with automation. To learn more about choosing automation tools, read our previous posts here and here.
Instead of relying 100% on either manual or automation use the best combination of manual and Automation Testing. This is the best solution for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.
We would like to hear about your experience in Automation Testing. Any practical experience will always be helpful for our readers.