Recently we have been receiving too many questions on when and how to automate the testing process. Instead of answering them individually, we thought it would be better to have a discussion here.
This tutorial will shed light on when to automate? how to automate? Or should we automate our testing work?
It would always be better to start a meaningful discussion on such a vast topic to get in-depth ideas and thoughts from experts from different areas and their experience in Automation Testing.
What You Will Learn:
Tips Before Automating Your Testing Work
Why Automation Testing?
#1) You have some new releases and bug fixes in a working module. So how will you ensure that the new bug fixes have not introduced any new bugs in the previous working functionality? Hence, you also need to test the previous functionality.
So will you manually test all the module functionality every time you have some bug fixes or a new functionality added? Well, you might do it manually but then you are not testing effectively. Effective in terms of company cost, resources, time, etc.
Here comes the need for Automation.
– Automate your Testing procedure when you have a lot of regression work.
#2) You are testing a web application where there might be thousands of users interacting with your application simultaneously.
How will you test such a web application? How will you create those many users manually and simultaneously? Well, a very difficult task if done manually.
– Automate your load testing work for creating virtual users to check the load capacity of your application.
#3) You are testing an application where the code is changing frequently. You almost have the same GUI but there are more functional changes, thus testing rework is more.
– Automate your testing work when your GUI is almost frozen but you have a lot of frequent functional changes.
What are the Risks Associated with Automation Testing?
There are some distinct situations where you can think of automating your testing work. We have covered some risks of Automation Testing here. If you have made the decision of automation or are going to take it sooner then think of the following scenarios first.
#1) Do you have skilled resources?
For automation, you need to have people who have some programming knowledge.
Think of your resources. Do they have sufficient programming knowledge for automation testing?
If not, do they have technical capabilities or programming background so that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes, only then you need to think about automating your work.
#2) Initial cost for Automation is very high
It’s a well known fact that Manual Testing has too many costs associated with hiring skilled manual testers. If you are thinking automation will be the solution for you, then think twice.
Automation cost is too high for the initial setup i.e., cost associated with automation tool purchase, training and maintenance of test scripts is very high.
There are many unsatisfied customers regretting their decision to automate their work. If you are spending too much and merely getting some good looking testing tools and some basic automation scripts then what is the use of automation?
#3) Do not think to Automate your UI if it is not fixed.
Beware before automating the user interface. If the user interface is changing extensively, the cost associated with script maintenance will be very high. Hence, basic UI automation is sufficient in such cases.
#4) Is your application stable enough to automate further testing work?
It would be a bad idea to automate testing work in the early development cycle (Unless it is an agile environment). Script maintenance costs will be very high in such cases.
#5) Are you thinking of 100% Automation?
Please stop dreaming. You cannot 100% automate your testing work. Certainly, you have areas like performance testing, regression testing, load/stress testing where you can have a chance of reaching nearly to 100% automation.
Areas like User interface, documentation, installation, compatibility, and recovery where testing must be done manually.
#6) Do not Automate tests that run once
Identify application areas and test cases that might be running once and not included in the regression. Avoid automating such modules or test cases.
#7) Does your automation suite have a long lifetime?
Every automation script suite should have enough life that its building cost should definitely be less than that of manual execution cost. It is a bit difficult to analyze the effective cost of each automation script suite.
Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (general assumption depends on specific application complexity) to have a good ROI.
Conclusion
Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise, your tool will remain on the shelf giving you no ROI.
Handing over expensive automation tools to unskilled staff will lead to frustration. Before purchasing automation tools, make sure that the tool is the best fit for your requirements. You cannot have a tool that will 100% match with your requirements.
Find out the limitations of the tool that best matches your requirements and then use manual testing techniques to overcome those testing tool limitations. An open-source tool is also a good option to start with automation. To learn more about choosing automation tools, read our previous posts here and here.
Instead of relying 100% on either manual or automation use the best combination of manual and Automation Testing. This is the best solution for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.
We would like to hear about your experience in Automation Testing. Any practical experience will always be helpful for our readers.
Hi,
I am looking for a open source Functionality Testing Tool for Windows based applciation, can any one please help me in this.
Regards,
Deepesh
I want to do certification in Software testing, can someone guide me @ the same.
Regards,
Jayaprada
@Jayaprada
go for ISTqb if 1-3
go for cste if >3 yrs of exp..
gud lck
Deepesh
pls use google will u find lots of freeware for functional tool..but no guarantee when it will crash and no support as well if u face any implication..
so plan n choose..
happy googling dude
It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Can anybody explain it in a detailed manner. specially the sentence “Unless it is agile environment”.
In agile both development and testing go parallel.So we can not wait until dev cycle completion.Thats what he meant here.
Does any one know about Perl Automation of Browseer based application on Linux platform?
Does any one know about Perl Automation of Browser based application on Linux platform?
Hi,
Is there any possibility of shifting my domain from QA to development.currently i am a fesher and joined in QA domain.I would like to know ,whether there is a chance of shifting from testing to development after 1 year
Hello All,
I have started learning perl scripting language and i want to learn about widndows automation using perl, can anybody help me in that, it would be great if i get some sample scripts and docs.
What are the testing that cannot be automated?
Can any one tell me
Tests that are not advisable to automate or ROI is very high if we plan to automate
1. Only one time tests
2. Reviewing release notes, documentation
3. UI related tests (e.g. Light grey color background and UI is not disrupted across various browsers)
4. Coverage is less for tests and custom controls being used by Dev. This can still be automate dif we have expertise in creating Win 32libraries and dlls.
can any one tell which are tools not consist of programming knowledge
hi vijay,
its really a good explanation of automation.i can agree with ur point Instead of relying 100% on either manual or automation use the best combination of manual and automation testing.it was absolutely a right thing to do…
thanx for giving such an information about automation tool
A realistic evaluation of automation candidates needs to be done before the effort is started. From experience, automating highly volatile areas will only result in much more time being spent on automation script maintenance per release, and just as many or more bugs due to less time allocated to manual testing.
That is really a good analysis, particularly all the risks u mentioned in Automation are really woth to consider before automation work.
Thanks for the great advice again.
for the question; why AT, when AT, and Risks in AT ?
best answer
thank you
Hi Vijay,
The information you provided here is very good, worth for the test Managers, leads & testers who want to start automation testing for their projects.
Thank you
Venu
sir, i have knowledge about c,c++.which coarse is beneficial for getting job,right now i m jobless.
Nice one..
The discussion is really informative.
It is not wise to automate 100% of the test cases, the reasons are,
1. We will never be able to cover some uncovered areas in the application if we only execute automation scripts.
2. Manual testing is always necessary to uncover new bugs and enhancements in the application.
3. At least 30% of the test cases should be covered in manual testing.
4.The more the no of automation scripts the more the maintenance cost would be.
5.ROI will be very less if more new features are added in the application.
6) Do not automate tests that run once: – I couldn’t agree more, automation is ideal for routinary work.
hi am working on testing. any doubts cn reach me vamsiroyal2000@gmail.com
nice discussion. Thanks to provide
Hello All,
which programming language we should know for automation testing
Firstly, Management should be educated enough how automation works then slowly you can see the fruitful results.
Please spell check before you publish online. It helps build credibility.
Nice and short article. Well, there are many other aspects to think of when automation. The articles seems to talk mainly about GUI and load testing, but does not talk anything about under the hood automation. Like automating message flows through MQ, Webservices, backend which can be done simultaneously as the dev is writing code via pair programming. Also, a lot of focus has been on selecting automation tool. Many a times for non-GUI automation, it is best to build your own test harness using high level programming language like JAVA and other supporting components like Cucumber/Gradle/Maven/Spring etc
Overall, it is a nice article.
Hello Everyone I want to do in crrtification in testing course plz I want to know that mannual testing course is better or automation testing course is better for future plz help me I m fresher candidate for testing course.
hi ,,i am dev.. now iam adoing a job as a Tester..but how can i know the automation testing. and my currently company is mobile application based. that is Android and ios. finall y my query is…i need to learn automation …can u tellme..which automation course would be curring running for market value…
Thanks for very informative article. Disagree for one point, It is possible to have 100% automated test when you have a well defined process and stable SUT
can u tell me which are all application we cant perform automation testing
I recently joined automation project, I am very new in IT field so got project based on mt training. I have never seen that we can automate same flow with different language. ie. my company has web based regression flow, but their website is in 32 different language, so they want to automate those regression flow. Can we do that? Any help would be appreciated.
Thanks!
It is possible to start with automation testing in mainframe application, performance and load testing??
A great list of tips, Vijay. Before starting automation, these tips are a great checklist to make sure the foundation is laid strong for any project requiring automation.
With emerging need of Internet of Things testing, it’s become even more important to understand these questions – Why, When and how to automate?
yes, a satisfy our thought, but i thing nothing impossible in the world, and i try …. not only try i will working on automation testing and that challenging for me……..
hiiiiii…….!!!!!!……..frds…..GOOD JOB…….I have a definition of AT……..In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.
Hi Friends…
This is a good discussion on Automation Testing.
I think that automation testing starts after the one round manual testing because automation testing performs mostly on stable builds.
We use a tool that automates the work done by Manual testers. It doesn’t require any knowledge of programming. Just simple thing is that the system need to be configured with Latest Java. Now a days its very easy to download or update to latest Java version. So my take on automation is 40% and 60% on Manual testing (I consider manual testing as mother of all testing).
when to automation testing use .
Hello All,
Have you all tried the selenium ?
The effectiveness of testing the applications varies widely among companies and organizations. s. Test automation is often the answer. Test automation means using a software tool like selenium. We at Vatsa solutions use various tools and frameworks for testing.
what is the feasibility of automation testing in Magento 2 and if yes than how to achive?
Thanks for sharing this article
Thank you for sharing this great article! You have raised many good points and a great tips checklist for automation testing.
Hi, I am an Automation tester since 4 years. I got a request from the project to analyze a text document which has report in plain text and see what went wrong.
I don’t think that is what automation is used for. We need eyes to see and analyse the text document.
Any thoughts?
Mam, Can You please help me, How I test the automation test?
Excellent post. I work in a company where the testers are not skilled and as a short-term solution we are going to use a web UI testing tool to ease the burden and slowly apply Selenium Webdriver using Python.
i want to study testing tool automation,before knowing testool.i need to study overall core java or some topics
HI, Very Interesting article. I will share my ideas here.
Test Automation should always start when our application is baselined stable. I meant here, while your web Application is unstable then probably selenium test automation is not making any impact. We first have a base lined prodcut and we may start the Test Automation.
but the question is How to begin with Test Script? so, for this you need to learn your application first and then identify the autimation test candidate cases which can be automated. Always remember you can not automate each and every cases and you need to select the Automation cases. Lets first contact Manual Team and get all Test cases, then you go-trough the cases and after reviewing you can easily identify the automation cases. Incase of diubt you may discuss with team or your senior.