10 Tips you should read before automating your testing work

I was getting too many questions on when and how to automate the testing process. Instead of answering them individually I thought it would be better to have some discussion here. I will put my thoughts about when to automate, how to automate or should we automate our testing work? I know there some of our readers are smarter than me. So it would be always a good idea to start a meaningful discussion on such vast topic to get in-depth idea and thoughts from experts from different areas and their experience in automation testing.

Why Automation testing?
1) You have some new releases and bug fixes in a working module. So how will you ensure that the new bug fixes have not introduced any new bug in previous working functionality? You need to test the previous functionality also. So will you test manually all the module functionality every time you have some bug fixes or new functionality addition? Well, you might do it manually but then you are not doing testing effectively. Effective in terms of company cost, resources, Time etc. Here comes need of Automation.
– So automate your testing procedure when you have the lot of regression work.

2) You are testing a web application where there might be thousands of users interacting with your application simultaneously. How will you test such a web application? How will you create those many users manually and simultaneously? Well, very difficult task if done manually.
– Automate your load testing work for creating virtual users to check load capacity of your application.

3) You are testing application where the code is changing frequently. You have almost same GUI but functional changes are more so testing rework is more.
– Automate your testing work when your GUI is almost frozen but you have a lot of frequently functional changes.

What are the Risks associated in Automation Testing?
There are some distinct situations where you can think of automating your testing work. I have covered some risks of automation testing here. If you have taken decision of automation or are going to take sooner then think of following scenarios first.

1) Do you have skilled resources?
For automation, you need to have persons having some programming knowledge. Think of your resources. Do they have sufficient programming knowledge for automation testing? If not do they have technical capabilities or programming background that they can easily adapt to the new technologies? Are you going to invest money to build a good automation team? If your answer is yes then only think to automate your work.

2) Initial cost for Automation is very high:
I agree that manual testing has too much cost associated to hire skilled manual testers. And if you are thinking automation will be the solution for you, Think twice. Automation cost is too high for the initial setup i.e. cost associated to automation tool purchase, training and maintenance of test scripts are very high.
There are many unsatisfied customers regretting on their decision to automate their work. If you are spending too much and getting merely some good looking testing tools and some basic automation scripts then what is the use of automation?


3) Do not think to automate your UI if it is not fixed:
Beware before automating user interface. If a user interface is changing extensively, the cost associated with script maintenance will be very high. Basic UI automation is sufficient in such cases.

4) Is your application is stable enough to automate further testing work?
It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Script maintenance cost will be very high in such cases.

5) Are you thinking of 100% automation?
Please stop dreaming. You cannot 100% automate your testing work. Certainly, you have areas like performance testing, regression testing, load/stress testing where you can have a chance of reaching near to 100% automation. Areas like User interface, documentation, installation, compatibility, and recovery where testing must be done manually.

6) Do not automate tests that run once:
Identify application areas and test cases that might be running once and not included in the regression. Avoid automating such modules or test cases.

7) Will your automation suite be having long lifetime?
Every automation script suite should have enough lifetime that its building cost should be definitely less than that of manual execution cost. This is bit difficult to analyze the effective cost of each automation script suite. Approximately your automation suite should be used or run at least 15 to 20 times for separate builds (General assumption. depends on specific application complexity) to have good ROI.

Here is the conclusion:

Automation testing is the best way to accomplish most of the testing goals and effective use of resources and time. But you should be cautious before choosing the automation tool. Be sure to have skilled staff before deciding to automate your testing work. Otherwise, your tool will remain on the shelf giving you no ROI. Handing over the expensive automation tools to unskilled staff will lead to frustration. Before purchasing the automation tools make sure that tool is the best fit for your requirements. You cannot have the tool that will 100% match with your requirements. So find out the limitations of the tool that is the best match with your requirements and then use manual testing techniques to overcome those testing tool limitations. Open source tool is also a good option to start with automation. To know more on choosing automation tools read my previous posts here and here.

Instead of relying 100% on either manual or automation use the best combination of manual and automation testing. This is the best solution (I think) for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.

Over to you. I would like to here your experience about automation testing. Any practical experience will always be helpful for our readers.




Recommended reading

103 comments ↓

#1 Krupa

I think Vijay is right. Automation testing is used in regression, performance and in load/stress testing. My opinion is before automating any feature of your application, the staff should have good knowledge of their application. Other point is automation test script should get updated when there is any changed in the functionality.

#2 sailasri

Yes i agree with the point ,Instead of relying 100% on either manual or automation use the best combination of manual and automation testing

#3 Faizal

That is really a good analysis, particularly all the risks u mentioned in Automation are really woth to consider before automation work.
Last week my boss asked me to have some RnD on automation tools and he told me to come up with some feasible tools.
It’s great that I found this article and now I will study my project according to all the mentioned risks to see how can I have my project work automated.
Thanks for the great advice again.

#4 Vikas

Its really a good discussion!!
Right now i m involved in manual only……so its a good way to start & have knowledge of automation testg….which i m going 2 encounter shortly….
Thank$ a lot man for all this
:-)

#5 Rajneesh

Hi,

This is very good!

I am a software developer having experience of 4 years, and now I have just got interest towards testing.

Most of the companies don’t try to analysis between manual and automation testing, at what level they require and end up spending more money than ROI……

It’s the responsibility of test engineer to put some view over that what percentage it requires manual and what percentage it require automation and do the best practice in view of the power/ capability of the testing tool.

We can make this discussion more focused, if we discuss some perfect percentage of stats involved in manual / automation testing.

Thanks
Gera

#6 sridevi

Hi,

I am working as a manual tester in a company since one year,actually I want to know about manual test case templete.in manual testing what are the steps we should follow. We are writing bug report in a excel sheet and maintain the power point to rectify reports. Our domain is medical insurance policies.
Thanks & Regards.
sridevi

#7 bindu

hello vijay,
you are giving very clear idea abt each and every topic of testing at the same time can u tell me qtp faqs
website so that i can attend interview on qtp as i am working as manual tester.

#8 tulasiram

hello
this is very good information abt the automation of the tool . we are writing the test case on automobile compay in excel sheet .

#9 raghava

Hai vijay , i am not agreeing u r answer. This is the also one reason but it s not giving total picture in my view there are so many reasons to go to automation tool. Just for understanding i can give one example : In the Mainframe applications we use lots of millions of records we can test as testdata . So in this case in manual testing it’s less scope to found the bugs. So in this case we use Aumation Tool for efficitive testing and Quality. if any concers let me know. Meanwhile u r answer is also good. Thx.

#10 Veni

Hi,
This is Veni.Vijay,u r giving very clear information.its good.Iam working in company since 6 months as a test engineer.Please tell me about Database testing.Its compulsary for me

#11 Varsha

Hi Vijay

You have given some very good points to ponder before going for automation. My company is going to automate the testing work soon. We are looking at open source tools like selenium & canoo webtest.
Can u suggest some good links where I can get more info on these tools? Also pls suggest any other good open source tools for functional testing from your experience. Our domain is business performance management. Thanks.

#12 Gatari Mishra

Can sumbody please tell me the process of reporting bug in QTP… , I am a new user so please help me out..

Regards

Gayatri

#13 Ashwin Channe

Can anybody expalin about “Different Types of Metrics”

#14 Arun

Hi vijay
Well really ur explaination well statisfie me. then i want to know how to implement the automation testing when our software designed by Third party tools,For example if the s/w developed in .net but the grids are designed by Far Point then hoe can to identify the records in that grids . its very importan doubt for me.

#15 Purvi

hi all,
can you please tell me how automation testing can be done? Do we need to have some software for it?

#16 shridhar

hi,
I would like to know more abt manual testing & need few templates for test cases & bug report

#17 shiva

do they write test case for automated testing??..plz mail me this answer or post it

#18 Krishna

Nice and very informative article.

#19 Manan

Good Analysis. Only one point I would like to defer is that even though GUI is not fixed (But you have rough idea), you can go ahead and automate if functionality is fixed. But if you are carrying out functional testing and are trying to automate when functionality itself is not fixed then effort in maintaining the script might increase.

#20 Surya

Hi all, Good analysis. we write test cases for automation, right, first write for smoke test(major functionality),secondly for functionality and regression so, my doubt is that these testcases consist of +ve,_ve,gui,database or we write seperately?

#21 Mandeep $ Naveen

This has added up in ma knowledge. This information is very fruitful to us. I gained lot of things from this. Prior to this what I was thinking that Automation is the most important aspect in testing. I paid no attention to manual one. but Manual has its own importance in its own aspects.
I like this line very much “Instead of relying 100% on either manual or automation use the best combination of manual and automation testing. This is the best solution (I think) for every project. Automation suite will not find all the bugs and cannot be a replacement for real testers. Ad-hoc testing is also necessary in many cases.”

#22 Anish Bindal

In addition to points mentioned above, I believe we can follow some other ways to automate applicaitons where UI changes, complexity are often changinf instead of waiting to get it completely fixed. This results in close deadline of the product and we never automate. Keyword based automation is the fastest way I came across for such cases. Control information is outside the scripts and small scripting can help you to update it everytime in just couple of minutes.

#23 vasant kumar

The info given is very productive Pls let me know about Automation testing in detail(ie; scripting purpose in QTP,Loadrunner,Silk & Rationale) what languages do u need to know to automate test cases….. thank u

#24 Surya

nice, Hi all, Good analysis. we write test cases for automation, right, first write for smoke test(major functionality),secondly for functionality and regression so, my doubt is that these testcases consist of +ve,_ve,gui,database or we write seperately?

#25 Vishal Chhaperia

Hi Vijay,

can you pls tell me what does ROI stands for??

email: bishalchaperia@gmail.com

#26 CHRR

what is automation test plan template

#27 CHRR

how to write automation test cases from FRS docs

#28 Rahul

@Vishal Chhaperia
ROI–Return On Investment

#29 Vishal Chhaperia

Thanks Rahul…………

Call : 9830704327

mail : bishalchaperia@gmail.com

#30 CHRR

diff betwen automation and manual test case template?

#31 Sanjay

how we will differenciate defect,bug and error ,its a genuan for interview.
with warm regards
sanjay bahttnagar

#32 Sanjay

Hi to all,
can u plz tell me, what is ROI

#33 Ryan

Sanjay, that was answered above by Rahul.

ROI = Return on Investment.

The ratio of the cost savings achieved relative to the investment, in the form of time and resources dedicated to undertaking such a task.

#34 chssekharrao

Can sumbody please tell me the process of reporting bug in QTP… , I am a new user so please help me out..

do they write test case for automated testing??..plz mail me this answer or post it

#35 Greg McNelly

Functional testing of services is another area where you may realistically automate 100% of your tests, because you do not have the number or complexity of issues that you may have with a GUI.

Over the past few years, I have become more appreciative of the value of test automation that goes beyond regression testing. Automating tests forces a more detailed analysis and understanding of the system under test; you help prove your understanding of the system by automating it. Automation also improves the behavior of the development team; they tend to test more and earlier if the automated tests are made available to them.

Having programming skills within your testing group and having a good, well-defined testing process will protect you from test automation traps.

#36 balaji

hi good explanation but i need more about QTP can any one help me out

#37 dsreddy

HI,
THIS SITE IS TOO GOOD FOR TESTERS

#38 Vijay

I am also interested in the field of Software Testing, please guide me from your side & provide suggestions for switching my Networking domain to Testing domain…

#39 Saurabh

Very Good infact Excellent.

We cannot have 100% automated website or project. Some manual efforts are very neccesary.

Tools are important but Manual testers with some Knowledge of code is also equally important in this scenario.

I am a software tester from the last 14 months (previously i was in coding line but switched to testing line) and i have a good knowledge of QTP9.2 but i realy more on Manual testing at initial stage and in final stage. In middle i use tools for my satisfaction.

If u have any question mail to me i will do my best to answer you.

#40 Preety

Hi All,
Hope you all are doing fine
Can anyone of you help me out in converting the QTP generated Test results into HTML.

Thanks

#41 Anil

We want to automate the regression testing process for a website where there many links similar to a storage management,what tools you would suggest?

#42 Sachin

Excellent Work
Really important to think over all issues before automating our modules.
It helped lot.

#43 Aravind Sudam

Excellent post. It gives indepth idea for all the new testers about automation. Good Work

#44 chetana

there is a web application which dosent have much user actions.its a HR related application which gives information about different pollicies in US.is it good to automate such kind of applications?

#45 Arul

which testing best for releasing bug free application (automation are manual testing )

#46 rahul

@Arul
u r diverting urself..each testing has its own life in testing cycle..
in the initial phase there in no meaning for Automation testing at all..its only manual and manual..
and suppose there is only one version(release ) of a software then again automation doesnot play any role..
its all depands on s.w..
is it clear..for more info ping me back
rahulkumarbangalore@yahoo.co.in

#47 rahul

@Chetana..
i have answered for Arul..same applicable to u..

#48 rahul

@balaji
please use ankur jain blog for QTP..its best

#49 Madhuri

Hi Vijay,

Please give more information about Silk Test tool

#50 Venkat

hey very useful points are discussed here..
but cam i know what is the difference between defects,bugs and errors..

Thank you
Venkat

#51 Deepesh

Hi,

I am looking for a open source Functionality Testing Tool for Windows based applciation, can any one please help me in this.

Regards,
Deepesh

#52 Jayaprada

I want to do certification in Software testing, can someone guide me @ the same.

Regards,
Jayaprada

#53 Rahul

@Jayaprada
go for ISTqb if 1-3
go for cste if >3 yrs of exp..
gud lck

#54 Rahul

Deepesh
pls use google will u find lots of freeware for functional tool..but no guarantee when it will crash and no support as well if u face any implication..
so plan n choose..
happy googling dude

#55 Saroj Patnaik

It would be bad idea to automate testing work in early development cycle (Unless it is agile environment). Can anybody explain it in a detailed manner. specially the sentence “Unless it is agile environment”.

#56 ganesh

Does any one know about Perl Automation of Browseer based application on Linux platform?

#57 ganesh

Does any one know about Perl Automation of Browser based application on Linux platform?

#58 Avinash

Hi,

Is there any possibility of shifting my domain from QA to development.currently i am a fesher and joined in QA domain.I would like to know ,whether there is a chance of shifting from testing to development after 1 year

#59 sneha

Hello All,
I have started learning perl scripting language and i want to learn about widndows automation using perl, can anybody help me in that, it would be great if i get some sample scripts and docs.

#60 Bhavya U S

What are the testing that cannot be automated?
Can any one tell me

#61 Anish Kumar Bindal

Tests that are not advisable to automate or ROI is very high if we plan to automate
1. Only one time tests
2. Reviewing release notes, documentation
3. UI related tests (e.g. Light grey color background and UI is not disrupted across various browsers)
4. Coverage is less for tests and custom controls being used by Dev. This can still be automate dif we have expertise in creating Win 32libraries and dlls.

#62 chandra sekar

can any one tell which are tools not consist of programming knowledge

#63 Twinkle chowdary

hi vijay,
its really a good explanation of automation.i can agree with ur point Instead of relying 100% on either manual or automation use the best combination of manual and automation testing.it was absolutely a right thing to do…
thanx for giving such an information about automation tool

#64 Joe

A realistic evaluation of automation candidates needs to be done before the effort is started. From experience, automating highly volatile areas will only result in much more time being spent on automation script maintenance per release, and just as many or more bugs due to less time allocated to manual testing.

#65 sowmya

veryyyyyy bad

#66 sowmya

gud mrng have a nice day………
i’m very feel about ur bad comments

#67 sowmya

testing is nothing automated u r really waste bhavya from us

#68 teja

hiiiiiiiii sneha

#69 sowmya

thank u rahul dude

#70 sowmya

thank u rahul dude……hw r u

#71 narapareddy

thanks dude

#72 Sudheer

That is really a good analysis, particularly all the risks u mentioned in Automation are really woth to consider before automation work.

Thanks for the great advice again.

#73 kiranR

for the question; why AT, when AT, and Risks in AT ?
best answer

thank you

#74 venu

Hi Vijay,

The information you provided here is very good, worth for the test Managers, leads & testers who want to start automation testing for their projects.

Thank you
Venu

#75 ankit

sir, i have knowledge about c,c++.which coarse is beneficial for getting job,right now i m jobless.

#76 Amitabh Panda

Nice one..
The discussion is really informative.
It is not wise to automate 100% of the test cases, the reasons are,

1. We will never be able to cover some uncovered areas in the application if we only execute automation scripts.
2. Manual testing is always necessary to uncover new bugs and enhancements in the application.
3. At least 30% of the test cases should be covered in manual testing.
4.The more the no of automation scripts the more the maintenance cost would be.
5.ROI will be very less if more new features are added in the application.

#77 MSI Data

6) Do not automate tests that run once: – I couldn’t agree more, automation is ideal for routinary work.

#78 vamsi singasani

hi am working on testing. any doubts cn reach me vamsiroyal2000@gmail.com

#79 karimulla shaik

nice discussion. Thanks to provide

#80 Robin Mehta

Hello All,

which programming language we should know for automation testing

#81 Unknown

Firstly, Management should be educated enough how automation works then slowly you can see the fruitful results.

#82 Tcorn

Please spell check before you publish online. It helps build credibility.

#83 Mansoor Shaikh

Nice and short article. Well, there are many other aspects to think of when automation. The articles seems to talk mainly about GUI and load testing, but does not talk anything about under the hood automation. Like automating message flows through MQ, Webservices, backend which can be done simultaneously as the dev is writing code via pair programming. Also, a lot of focus has been on selecting automation tool. Many a times for non-GUI automation, it is best to build your own test harness using high level programming language like JAVA and other supporting components like Cucumber/Gradle/Maven/Spring etc

Overall, it is a nice article.

#84 Nishu Sharma

Hello Everyone I want to do in crrtification in testing course plz I want to know that mannual testing course is better or automation testing course is better for future plz help me I m fresher candidate for testing course.

#85 dev

hi ,,i am dev.. now iam adoing a job as a Tester..but how can i know the automation testing. and my currently company is mobile application based. that is Android and ios. finall y my query is…i need to learn automation …can u tellme..which automation course would be curring running for market value…

#86 Jins

Thanks for very informative article. Disagree for one point, It is possible to have 100% automated test when you have a well defined process and stable SUT

#87 anisha

can u tell me which are all application we cant perform automation testing

#88 Mike

I recently joined automation project, I am very new in IT field so got project based on mt training. I have never seen that we can automate same flow with different language. ie. my company has web based regression flow, but their website is in 32 different language, so they want to automate those regression flow. Can we do that? Any help would be appreciated.
Thanks!

#89 Puru

It is possible to start with automation testing in mainframe application, performance and load testing??

#90 Sanjay Zalavadia

A great list of tips, Vijay. Before starting automation, these tips are a great checklist to make sure the foundation is laid strong for any project requiring automation.

With emerging need of Internet of Things testing, it’s become even more important to understand these questions – Why, When and how to automate?

#91 Bhagwan

yes, a satisfy our thought, but i thing nothing impossible in the world, and i try …. not only try i will working on automation testing and that challenging for me……..

#92 Surekha

hiiiiii…….!!!!!!……..frds…..GOOD JOB…….I have a definition of AT……..In software testing, test automation is the use of special software (separate from the software being tested) to control the execution of tests and the comparison of actual outcomes with predicted outcomes.

#93 Surekha

hiiiiii…….!!!!!!……..frds…..GOOD JOB…….I want features of automation testing

#94 Ravi Singh Jaiswar

Hi Friends…
This is a good discussion on Automation Testing.
I think that automation testing starts after the one round manual testing because automation testing performs mostly on stable builds.

#95 Rajkumar

We use a tool that automates the work done by Manual testers. It doesn’t require any knowledge of programming. Just simple thing is that the system need to be configured with Latest Java. Now a days its very easy to download or update to latest Java version. So my take on automation is 40% and 60% on Manual testing (I consider manual testing as mother of all testing).

#96 prashant pandey

when to automation testing use .

#97 Neha Joshi

Hello All,

Have you all tried the selenium ?
The effectiveness of testing the applications varies widely among companies and organizations. s. Test automation is often the answer. Test automation means using a software tool like selenium. We at Vatsa solutions use various tools and frameworks for testing.

#98 Viraj

what is the feasibility of automation testing in Magento 2 and if yes than how to achive?

#99 Toms

Return Of Investment

#100 nisha

Thanks for sharing this article

#101 ECM TechNews

Thank you for sharing this great article! You have raised many good points and a great tips checklist for automation testing.

#102 raj

FUCKKKKKKKKKKKKK with all those points

#103 Vinuta

Hi, I am an Automation tester since 4 years. I got a request from the project to analyze a text document which has report in plain text and see what went wrong.

I don’t think that is what automation is used for. We need eyes to see and analyse the text document.

Any thoughts?

Leave a Comment