Automated Testing – How to Choose the Best Automation Testing Tool

What is Automation Testing?

A series of software programs to validate test output against specified test conditions. It’s the best way of executing repetitive test cases using some software/tool which controls the test execution.

In simple words – Test automation is getting rid of repetitive manual tests and replacing those with systematic programs using automation tools.

Is Automated Testing a Solution for You?

I’ve worked on many projects in my career. When you work on same project for more than one year you strongly start feeling need of automating some tasks. You start thinking to introduce automation testing on the project if it’s not been considered till now by project management.

One year is enough time for anyone to know the ins and outs of any project. Once you know the project functionality in detail it becomes easier to decide which repetitive tasks need to be automated.

Some testers also get bored doing same repetitive tasks again and again and they strongly start feeling the need of test automation.

Does that mean you should jump into automation testing right away? Definitely not! There are many criteria you need to work on before deciding whether automation is a solution for you.

Automated Testing

When Does Test Automation Make Sense?

  • When there are many repetitive tests
  • When there are frequent regression testing iterations
  • When you need to simulate large number of users who are using the application resources
  • When AUT is having comparatively stable UI
  • When you have large set of BVT cases
  • When you can’t rely solely on manual test execution for critical functionality

Further readingWhen should you go for automation?

Once you know that it is the right time to invest your time and money in a good automation tool you can start looking for the best automation tool that suite your needs.

Further readingTips you read should before starting automated testing

How to Select Automation Tool for Your Project?

Automation testing success largely depends on the selection of right testing tools. It takes lot of time to evaluate relevant automation tools available in the market. But this is a must one time exercise that will benefit your project in long run.

There were few situations where I got chance to review and select automation tool for my projects. The task was difficult as we had to manage our testing needs and cost restrictions but it was a worth experience.

Here are the criteria you need to consider before selecting any testing tool:

1) Do you have necessary skilled resource to allocate for automation tasks?

2) What is your budget?

3) Does the tool satisfy your testing needs? Is it suitable for the project environment and technology you are using? Does it support all tools and objects used in the code? Sometime you may get stuck for small tests due to inabilities of the tool to identify the objects used in the application.

I consider above three factors as most important for selecting any tool.

4) Does the tool provide you the free trial version so that you can evaluate it before making a decision? Also does the tool have all features available in trial version?

5) Is the current tool version stable? Is the vendor company established with good customer support as well as online help resources and user manual?

6) How is the tool learning curve? Is the learning time acceptable for your goals?

------------

7) Do you want automation tool for only your project needs or you are looking for a common tool for all projects in your company? It would be a good choice if you select a tool that supports most of the coding languages on your projects.

8 ) Which testing types does it support? Tool which supports maximum testing types (Unit, functional, regression etc.) is always a better choice. Warning – Don’t go for a tool just because it is supporting all testing types. It’s also important that the tool should be powerful enough to automate your complex requirements.

9) Does the tool support easy interface to create and maintain test scripts? Record and playback tool with abilities to edit recorded scripts could be a good solution.

10)  Does it provide simple interface yet powerful features to accomplish complex tasks?

11)  How easy it is to provide input test data for complex or load tests? Tool supporting test data input from various data files such as Excel, XML, text file etc. would be a big relief for the automation the testers.

12)  Does it provide the powerful reporting with graphical interface? Clear and concise reports will always help you to conclude the test results quickly.

13)  Does it integrate well with your other testing tools like project planning and test management tools?

You may also want to consider other criteria like:

14)  Tool vendor refund policy

15)  Existing customer reviews for the tool

16)  Is the vendor providing initial training?

Tips: Requirement gathering is by far the most important step for selecting the right tool. Make sure to categorize your requirements in must have, nice to have, and not required feature categories. This will help you to evaluate the tool quickly. Remember you won’t find a tool already available in the market which will support all your automation needs!

Best Automation Tools:

HP QTP and Selenium are the two most popular functional testing options available currently. QTP is a best functional testing tool supported on wide range of coding languages and platforms whereas Selenium is the best open source functional web testing tool.

In next article we’ll discuss manual and automation testing challenges.

Feel free to discuss your thoughts/questions on automation testing!

 



Get FREE eBook + Blog Updates By Email!

Subscribe to get software testing awesome articles and free resources. Enter your email address and click 'SIGN UP NOW' button.


63 comments ↓

#1 shweta on 09.21.07 at 5:05 am

Hello Sir,

Your information is indeed very hepful.I have to start with automated testing in my company.Can u tell me
exact steps to test static & dynamic websites using Winrunner.How to write test scrips????As far as i know they r automatically generated.

#2 shweta on 09.21.07 at 5:06 am

Hello Sir,

Your information is indeed very hepful.How to write test scrips????As far as i know they r automatically generated.

#3 10 Tips you should read before automating your testing work on 10.26.07 at 1:49 pm

[...] option to start with automation. To know more on choosing automation tools read my previous posts here and [...]

#4 Aps on 11.19.07 at 8:21 am

Plz suggest me a good training institute in Mumbai for Software testing….I have heard abt Seed Infotech….Plz advice….thank u

#5 QuickTest Pro - QTP Functional testing tool review on 11.27.07 at 10:30 am

[...] are hundreds of automation tools in the market. Choosing automation tool for your organization is really a difficult [...]

#6 sameer on 02.19.08 at 11:24 am

Hello sir,
i want to know about automation tools…. how these are working & how we check e-commerce sites through automation …Is it possible if so, please tell me the tools which are used for that

Thank you sir
sameer

#7 Lakshmi on 02.19.08 at 12:11 pm

Hi Sameer !!!

Have a glance at this site …

opensourcetesting.org

where you can get lot of information regarding variety of tools for software testing…

You can choose one of them wrt your requirement.

#8 sameer on 02.26.08 at 9:43 am

hi Lakshmi
Thanks 4 giving me information…
but can u tel me which tool is used to test the e-commerce site … i am very confused
plz help me out

#9 Madhu DDL.... on 02.29.08 at 9:08 am

The explanation about each and every topic is Excellent.

The logic U R using while explaining is nice.

Bye………………….Bye………………..

#10 Prasad Gali on 03.11.08 at 8:42 am

This site is simply superb for both manual and automation testers interms of explanation of the concepts…

And also this site provides valuable information for the testers…

#11 poonam on 03.14.08 at 5:34 am

Hello, can u tell me which tool is useful & easy to test load & performance of any web application?
I’m doing manual testing, i never used any automation tool but now i want to use any tool which is useful & easy to use. Please help me.

#12 Anjali Awasthi on 04.11.08 at 9:18 am

Dear sir,
I am working in a product based company as as SQA in manual testing.But i have an issue is that how can i automate the product testing as this produst is a winCE product and also mouse is not working for this……could u plz help ????

#13 george on 04.11.08 at 10:20 am

@poonam,
For load\performance testing You can use VSTS, it is very easy and userfriendly reporting also.
cheers
george

#14 sameer on 04.14.08 at 1:04 pm

Dear poonam
Could you plz tell me what is VSTS..

#15 Suresh Balakrishnan on 06.05.08 at 7:25 am

Hi All,

This is Suresh Working as a Test Engineer especially Performance Load Test in my Concern. right now i m doing Load Test using QEngine tool with 500 virtual Users. QEngine is wonderful tool for Performance Testing so any one interested contact me i ll help u related to Performance Testing (QEngine)

Thanks,
Suresh
bsuresh7282@gmail.com

#16 poonam gulhane on 06.09.08 at 5:01 am

Hello Suresh,
I am working as a QA. Also i want to use QEngine to test web application. I downloaded trial version but i dont understant how to use it. Please can u tell me the procedure? Please help me to use QEngine.

Brgds
Poonam

#17 Beena on 06.09.08 at 3:11 pm

thanx….

#18 Rashmi on 06.10.08 at 9:33 am

Hi ,
I would like to know what is metrics and what are the types of metrics related to software testing. Please expalin it with example. i have read it several tiemes and i am curious to know about it.
Thanks,
Rashmi

#19 Suresh Balakrishnan on 06.11.08 at 5:36 am

Hi Poonam,

Please mail me i ll sent through mail related Testing and QEngine Queuries.

Thanks,
Suresh

#20 Jalina on 07.14.08 at 4:13 am

Wow this is an excellent site. I have a quick question. I do lot of regression testing manually. Will like to automate that. Company is willing to go further with my initiative. Can someonehelp me how to go about it?

Thanks….

#21 vikram on 08.08.08 at 3:10 pm

Hi ,
Our company works on opensource platform / Environment,we using ubuntu linux(os),
we have some product ERP-openbravo opensource ERP,Sugar CRM and CMS all software are opensource,

so our testing team want use opensource tool for
Fuctional,regression,stress,performance,load testing
which tool we have to use,
we know mercury interactive tool but they r not opensource.

we found some tool from net like
for testmanagement -testopia(succeed)
defect tracking – bugzilla(succeed)

fuctionality testing-selenium(not well succeed)
Jcrawller(load,performance)(not well succeed)
Plz suggest any opensource tool which have good document on net.

Thanks and Regards,
Vikram

#22 sachin goel on 12.08.08 at 10:10 am

cld u plz tell me what is performance testing. &how does it

#23 neha on 04.02.09 at 7:13 am

Plz suggest me a good training institute in Mumbai for QTP….
Plz advice
thank u

#24 priya on 04.10.09 at 6:42 am

Plz suggest me a good training institute in Mumbai/hyderabad for QTP….
Plz advice
thank u

#25 SADHANA on 05.25.09 at 10:47 am

HAI…

I WANT TO LEARN T QTP9.2 ND JAVASCRIPT PLZ GUIDE ME IN CHOOSING GOOD INSTITUTE IN MUMBAI..

#26 Jagdish Reddy on 05.25.09 at 11:34 am

Hi Priya,

I know the institutes in Hyderabad but dont know in Mumbai. Let me know you want only QTP training else other tools also? send me a mail on Srireddy333@gmail.com for any info

#27 Kalyan on 07.27.09 at 10:26 am

what are the advantages of QEngine over QTP?

#28 Vidya on 09.08.09 at 7:00 pm

Hi All,

We are conducting QTP weekend batch if intrested contact me on 9324260594 , 9819045845

Thanks

#29 Priyanka on 10.26.09 at 6:46 am

Which is the best open source tools available for web application testing, also one which can be easily used by a person having 0% technical knowledge Plz Reply ASAP

#30 karthik on 12.08.09 at 7:43 am

Hi,
I was experienced in QTP recently i have shifted another comp where i do not have QTP ..Nw am landing the situation to automate in selenium but selenium not supporting vb script… can any 1 tell me that automation tool for functionality testing which support vb script and should be opensource one(other than QTP)

#31 deepa on 01.29.10 at 10:58 am

Please can you suggest which open source automation tool is used for ecommerce application

#32 mani on 03.08.10 at 12:14 pm

Plz suggest more info on website testing

#33 Allan Pereira on 03.08.10 at 2:48 pm

Hi,
I would like to know if there are any tools available for testing Browser based applications for In-Flight Entertainment System.

The applications run under simulated environment which is RACK Hardware just like the one which is installed on an Airplane.

With Regards,
Allan Pereira

#34 arokia on 03.25.10 at 2:30 pm

please give ans. for automation

#35 sonia on 05.09.10 at 7:30 pm

hi
i am very new to testing i would to know about testing scrpit

#36 Muhammad on 07.26.10 at 8:03 am

how to test the web application manually with example taking one sample project

#37 Muhammad on 07.26.10 at 8:05 am

interview questions for manual testing

#38 Shawn on 11.03.10 at 1:21 pm

Hi

We are currently doing only manual testing in our organization. I have been tasked to complete a Concept Document for the acquisition of automated test tools.

Please assist in providing me an example of such a document.

#39 Ramakrishna on 11.26.10 at 4:58 am

Hi Shweta
HP Quick Test Professional Is One Of the Best Tool In the Market, And .If U use HP Quick Test Professional Record Option Scripts will auto Generated .If u want U can Edit This script In VB .
Can Download And Try The trail version

#40 Anvesh on 12.08.10 at 11:49 am

hi friends,
recently i joined as test engineer (automation)in a product based company.whether it is having bright future or not

#41 RAvichandra on 01.11.11 at 7:43 am

hi
for learning QTP Institution is not better because its huge crowd and also not better environment to learn my openion is to better to join who professional in QTP
in bangalore i know

Mr.Prabhakar Gudi
contact no:9886032977
6+ years exp in QTP workin in top MNC

#42 Pratik on 01.11.11 at 9:06 am

Hi,

Could you please tell me the list of performance testing tools, that include load testing as well as stress testing.

#43 Bibekannada Mohanty on 02.08.11 at 12:38 pm

Can any one please help me…

1) What all are the things can be automated in an E-Commerce domain project….

#44 anil on 03.23.11 at 1:03 pm

hi,
can anyone tell me the how to find the new technique for web based application on online to panilkumar0907@gmail.com

#45 naresh on 07.18.11 at 7:39 am

can any one tell me is there any automation tool is available from microsoft .

#46 viki on 08.01.11 at 3:21 am

hi
please, send the Advantage – Disadvantages of Manual and Automation testing, also the step-by-step process to automate your project work.
1′s again please

#47 Prathamesh Sakpal on 05.21.12 at 11:09 am

Hello sir,
I am working on client server base application project.
Please give about guidence to choose automation tool for testing.
Please mail me to my email id-
sakpal.prathamesh@yahoo.com
Thanking u.

#48 saritha on 06.21.12 at 12:07 pm

i am a beginner in testing plz suggest me which topics i need to learn..

#49 ankit on 08.17.12 at 12:36 pm

hi all
how to learn Load UI any idea.(any specific link,Tutorial)
If any buddy have a idea plz give me a reply
my mail is ankit.nriitm@gmail.com

#50 ankit on 08.17.12 at 12:42 pm

Any boddy tell me how to use Load UI testing tools.

#51 Madhu on 11.26.12 at 5:34 am

Hi,
Please suggest me open source Automation tool for stand alone applications.Particularly for health care domain.

#52 priyanka on 02.18.13 at 10:15 am

sir i also want to start automation testing in my company ,on wesales project getting confused which tool to use and =how to do it ,the project is based on .net platform can u just help me out plz

#53 sathya on 02.19.13 at 4:25 pm

hi sir, can u plz tell me about web elements??

#54 sathya on 02.19.13 at 4:27 pm

can u plz send me d answer to dis mail m.sathya005@gmail.com

#55 Kusum on 07.04.13 at 7:08 am

Can You please suggest me name of automation testing tool to test WPF application?

I want to knowabout the trial version of the particular tool,Please send me the answer to this mail -kusum.s@veniteck.com

#56 khushi on 07.04.13 at 7:09 am

Can You please suggest me name of automation testing tool to test WPF application?

I want to knowabout the trial version of the particular tool,Please send me the answer to this mail -kusum.s@veniteck.com

#57 Srinivasulu on 07.16.13 at 4:51 am

Hi This is srinu . Present am working as a Manual Test Engineer . I need to learn Automation tool . Present which automation is mostly running on S/W Filed ? Is there any one can train on selenium who working on selenium Test Engineer .

#58 Pulkeet on 08.07.13 at 4:13 am

Hi,
Pls give me some tutorials of QTP,QC,LR.
pls share it on my email id : pulkeetgarg2007@gmail.com

#59 Dariusz Cieslak on 10.19.13 at 1:41 pm

Good article, however I cannot find any reference to non-determinic test types.

We, at randomtest.met believe the most effective way to track software quality during development is to eploy: automation+randomness+design by contract.

You don’t have to pay time for scripting and managing user interface actions, you embed assertions in source code instead. Random input-based coverage triggers some of written assertions (sometimes crashes your app) and you see your stats at the end of the day.

That number shows you (rught) estimation of the quality of software you have. A good input for decision on a release.

#60 Raghav on 10.24.13 at 2:12 pm

Hi,

I have been working on manual testing,
I want learn any automation testing tool.
Where could i get proper knowledge on these tool.
which tools are preferable.

#61 Tool Seeker on 12.12.13 at 5:55 am

Can someone suggest me a good automation testing tool for the software that uses following technologies,

1. JQuery Version 1.7.2
JQuery Tools Version Tools v1.2.7
jQuery Validation Version 1.9.0

2.HTML 4.1 & CSS 2

3. Ajax calls

4. Java scripting

5. Browsers : chrome V30 and later , Ie9

I have had looked at Selenium but it says it does not support Java. Please suggest me a good tool as early as possible

#62 Deyan on 01.30.14 at 12:58 pm

Hi Man,

It is really a great site and it has been useful to me.

I am new to automation and want to start automating some parts of a banking program but I am confused which Automation Program should I best use.

We use Oracle 12c for Database,
Weblogic to read the information and use Oracle 11g Forms to display the data.
For the reports we use Oracle Apex, SQL Plus and RDF.

If you could guide me through which programs I should choose from.

Really appreciate it.

Regards,
Deyan

#63 Seema on 02.12.14 at 8:53 am

Hi,

We are into build native Mobile apps.
It includes features like Messaging, Interacting to backend system in Cloud. We use Titanium as our IDE to build native app for iOS and Android phone.

We want to choose a good automation tool which would help us in functional testing as well as reduce the time during regression.
1. Functional Testing, using Record Playback/ object driven tools
2. Re usability of test cases across different platform
3. Support to Performance/Load test is a plus

Could you please advice me on few tools we could look into?

We have looked at SOASTA & Ranorex so far.

Hope to hear from you guys soon..

Thanks,
Seema