The Ultimate Guide to Start Automation Testing on Your Project

Do you want to start automation testing on your project but struggling with the most basic steps like:

  • How to introduce test automation on your project?
  • How to select the best and the right automation tool?
  • How to develop scripts effectively?
  • How to execute and maintain test scripts?
  • and finally what are the best practices you need to follow for successful automation testing?

Today, we are starting a short but useful series of tutorials on “Getting started with Automation Testing”. In this automation testing tutorial series we will answer all above questions with step by step simple examples.

This is the first tutorial in 7 part “The Ultimate Guide to Automation Testing” series.

=> Here is the list of 7 Best Tutorials on Starting Test Automation on Your Project

Tutorial #1. Best Guide to Start Test Automation on Your Project
Tutorial #2. Types of Automated Tests and Some Misconceptions
Tutorial #3. 10 Steps to Introduce Test Automation on Your Project
Tutorial #4. The A to Z Guide on Selecting the Best Automation Tool
Tutorial #5. Script Development and Automation Frameworks
Tutorial #6. Execution and reporting of Test Automation
Tutorial #7. Best Practices and Strategies of Test Automation

 

automation testing guide

What is test automation and why we need it?

If software can do anything then, why not make software test software?

Does this statement sound logical to you? If yes, then congratulations, you are now thinking about automation testing.

Imagine yourself first day at your job as an SQA. You are presented with the application to be tested. It’s an ERP application containing 100s of forms and thousands of reports. You begin your exploratory testing by opening a form which contains around 50 different fields. You try to enter random data in this form which took around 20 minutes. Then you press submit. Wolla!! An error message is shown which looks like an unhandled exception. You become very happy. You proudly note down the steps and report the bug in your bug management system. Great effort, you feel really confident and energetic. You continue the testing until the day ends and find some more bugs. “Amazing first day”, you thought.

Now comes the next day, the developer has fixed the issue and releases the new version of build. You test the same form with same steps and you found that bug is fixed. You mark it fixed. Great effort. You have contributed to the quality of the product by identifying that bug and since this bug is fixed, the quality is improved.

Now comes the third day, developer has again released a newer version. Now you again have to test that form to make sure no regression issue is found. Same 20 minutes. Now you feel a little bored.

Now imagine 1 month from now on, newer versions are constantly releasing and on every release, you have to test this lengthy form plus 100 of other forms like this, just to make sure no regression is there.

Now you feel angry. You feel tired. You begin to skip steps. You fill around only 50% of the total fields. Your accuracy is not the same, your energy is not the same and definitely your steps are not the same.

And one day, the client reports the same bug in the same form. You feel pathetic. You feel unconfident now. You think you are not competent enough. Managers are questioning your ability.

I have news for you; this is the story of 90% of the manual testers out there. You are no different.

Regression issues are the most painful issues. We are humans. We cannot do the same things with same energy, speed and accuracy every day. This is what machines do. This is what automation is required for. To repeat the same steps with same speed, accuracy and energy as they were repeated the first time. I hope you get my point.

automation testing a friend

Whenever such situation arises, you should automate your test case. Test automation is your friend. It will help you to focus on new functionality while taking care about the regressions. With automation you can fill that form in less than 3 minutes. The script will fill all the fields and will tell you the result along with screenshots. In case of failure, it can pinpoint the location where the test case failed, thus helping you to reproduce it with ease.

------------

Some other scenarios where we need Automation

The above scenario is not the only case when you will need automation testing. There are situations, which cannot be tested manually. For example,

  1. Comparing two images pixel by pixel
  2. Comparing two spreadsheets containing thousands of rows and columns.
  3. Testing the application under the load of 100,000 users.
  4. Performance Benchmarks.
  5. Testing the application on different browsers and on different operating systems in parallel.

These situations require, and should be, tested by tools.

A simple example of Test Automation

When you are testing a software (on the web or desktop), you normally use a mouse and keyboard to perform your steps. Automation tool mimics the same steps by using a scripting or a programming language.

For example, if you are testing a calculator and the test case is that you have to add two numbers and see the result. The script will perform the same steps by making use of your mouse and keyboard.

The example is below.

Manual Test Case Steps:

  1. Launch Calculator
  2. Press 2
  3. Press +
  4. Press 3
  5. Press =
  6. The screen should display 5.
  7. Close Calculator.

Automation Script:

//the example is written in MS Coded UI using c# language.
[TestMethod]
public void TestCalculator()
{

//launch the application
var app = ApplicationUnderTest.Launch("C:\\Windows\\System32\\calc.exe");

//do all the operations
Mouse.Click(button2);
Mouse.Click(buttonAdd);
Mouse.Click(button3);
Mouse.Click(buttonEqual);

//evaluate the results
Assert.AreEqual("5", txtResult.DisplayText,”Calculator is not showing 5);

//close the application
app.Close();
}

The above script is just the duplication of your manual steps. The script is easy to create and easy to understand.

What are Assertions

The second last line of the script needs some explanation.

Assert.AreEqual(“5”, txtResult.DisplayText,”Calculator is not showing 5);

In every test case, we have some expected or predicted result in the end. In the above script we have the expectation that “5” should be shown on the screen. The actual outcome is the result which is displayed on the screen. In every test case we compare the expected outcome with actual outcome.

Same goes with automation testing. The only difference is, when we do that comparison in test automation, it is called something else in every tool.

Some tools call it “Assertion”, some call it “check point” and some call it “validation”. But basically this is the comparison. If this comparison fails, for e.g. screen is showing 15 instead of 5 then this assertion/checkpoint/validation fails and your test case is marked failed. When a test case is failing due to the assertion that means you have detected a bug through test automation. You report it to your bug management system just like you normally do in manual testing.

In the above script, we have performed the assertion in the second last line. 5 is the expected outcome, txtResult. DisplayText is the actual outcome and if they are not equal, we will be shown a message that “Calculator is not showing 5”.

Automation Testing is the Cost-effective method for regression testing

Test automation costs are higher initially. There is the cost of the tool, then there is a cost of the test automation resource and his/her training. But when scripts are ready, they can be executed hundreds of times repeatedly with the same accuracy and rather quickly. This also saves many hours of manual testing. So the cost gradually decreases, and ultimately it becomes the cost-effective method of regression testing.

Conclusion

I can sum up this article for you with 7 points.

Test Automation:

  1. Is the testing which is done programmatically.
  2. Uses tool to control the execution of tests.
  3. Compares expected outcomes with actual outcomes (Assertions).
  4. Can automate some repetitive but necessary tasks (e.g. Your regression test cases)
  5. Can automate some tasks which are difficult to do manually ( e.g. Load testing scenarios)
  6. Scripts, can run quickly and repeatedly
  7. Is cost effective in the long run.

Test Automation is explained simply in this article, but that doesn’t mean it is always simple to do. There are challenges, risks and many other obstacles. There are numerous ways by which test automation can go wrong, but if all goes well, then the benefits of test automation are huge.

Still to come in this series:

In the upcoming tutorials, we will talk about many aspects related to automation testing.

These include:

  1. Types of automated tests and some Misconceptions about automation testing
  2. How to introduce test automation in your organization and avoiding common pitfalls when doing test automation.
  3. The tool selection process and comparison of various automation tools.
  4. Script Development and Automation Frameworks with examples.
  5. Execution and reporting of Test Automation
  6. Best Practices and Strategies of Test Automation

About the author: Mohammad Saad, who is working as a Test Automation Lead in a leading MNC, is helping us for this “getting started with automation testing” in-depth tutorials series. His extensive experience in test automation field includes – initiating test automation on a project, designing test automation frameworks and best practices in order to carry out test automation for the long run, managing a centralized test automation department in company and automating Build Acceptance testing. He is using Microsoft Test Manager, MS Coded UI Test, Selenium and Visual Studio Tools for Test Automation.

You can suggest what other topics you want to know about in test automation in the comments section. I would love to have your feedback.

Recommended reading

47 comments ↓

#1 komal

Good article. I specifically need guidance on tool selection process. Can you help on this?

#2 Koteshwari

Good article. Thank You

#3 Mangala Hegde

Hi,

Article was really helpful.Looking forward to read more articles on “Guide to Start Automation Testing on Your Project”…That would be very helpful to switch to automation.

#4 Nikita

Nice Article. I need guidance on writing test scenarios in Behat. Can you help on this?

Share any article with Behat, Mink & Selenium. How to use it with the project with examples?

#5 Mohammad Saad

@komal: Thank you. For tool selection process, please wait for 4th Article in this series. The article name is “The tool selection process and comparison of various automation tools.”

#6 kamran

very good article specially for new comers who want to make their career in test automation field..i need a complete detail article about test automation in automotive industries.which tools are used and mostly which targets are needed test automation.thanks.

#7 Mohammad Saad

@Koteshwari: Thank you

#8 Awais Khalil

Simple and to the point.
Good Stuff.
Waiting for more on it in advance level.

#9 Mohammad Saad

@Komal: Thank you. Please wait for article number 4 in this series

#10 Mohammad Saad

@Nikita: Thank you. This series is a general guideline on Test Automation. We will not target any particular tool. So unfortunately I will not be able to help. A quick google search showed me this article.
http://behat.readthedocs.org/en/v2.5/quick_intro.html

#11 Pramod Mallick

Indeed a good article, thanks for the effort you put to consolidate those basic steps together.

It would be better if you add the tool name in the subject line which code is given as example (Selenium/QTP/RFT)

The topic is not elaborate enough to justify the title. I mean guide to start automation should include a bit of manual test case , considering and filtering those for regression suite and then identify keywords, then prepare test data , then record and play/ script code then enhance those with programming logic and write reporting module , then outputs, then maintaince and debug … something like this. But maximum text covers why we need automation, advantages of automation etc.. Just a thought for improvement no other intention!.Correct me if I am wrong.

#12 Dilip Pal

I have started Test Automation using Microsoft Test Manager, MS Coded UI Test, and Visual Studio Tools. I am Looking forward to read more articles on the same.

#13 Mohammad Saad

Pramod Mallick: Thank you for your feedback. This is just the first article in the series of 7 articles about test automation. Your questions will be answered in upcoming Articles.

#14 Mohammad Saad

Pramod Mallick: Please see the comments on the top of automation script. The tool name is mentioned there. (MS Coded UI)

#15 Biswaranjan

I need little more information on working of Selenium. How to use it with the project with examples?Some good sources to get some hands on this tool(Sites information, Prerequisite for the same)

#16 Mohammad Saad

@Biswaranjan

please see these articles for selenium
http://www.softwaretestinghelp.com/selenium-tutorial-1/

#17 Pramod Mallicki

Mohammad Saad:
Thanks for your comments. I am eager to read next part of the article.
If possible, cover the differences between coded UI and other tools.
Example: QTP vs Coded UI – including technology, simplicity, cost, versioning, maintaing, compatibility, depedency etc..

Thanks,
Pramod Mallick

#18 Luke

Thank you very much for this tutorial and initiative. I’ve started learning Automation Testing with OpenSource Selenium2 (WebDriver) requiring Java programming knowledge and now I’m trying to get familiar with UFT ver: 12.02 from HP (Unified Functional Testing) former-QTP that requires now knowledge of Visual Basic scripting. I’ve read all the comments above and now my questions are:
1. Are MS Coded UI and Microsoft Test Manager related to UFT or are they totally different and require to learn other scripting languages?
2.Which one’s better for WebApp and Mobile Testing?

Thank you,
Luke

#19 Mohammad Saad

@Luke: Thank you for your feedback. The answer to your first question is that MS Coded UI and MS Test Manager are totally different tools. These are created by Microsoft. UFT on the other hand is owned by HP. You need to learn either C# or VB.Net to learn Coded UI. MS Test manager is a test case and bug management tool so you don’t need to learn any programming language for that.

The answer to your second question is described in detail in article number 4 “The tool selection process and comparison of various tools” which will be published shortly.

#20 Mohammad Saad

@Pramod Mallicki These differences are described in detail in article number 4 “The tool selection process and comparison of various tools” which will be published shortly.

#21 Mansur

Useful Automation Guide Indeed And thanks for it :)

#22 Vivek

Mohammad, you’re doing great job. Welcome one and hats-off to STH. Hoping that your service will continue henceforth.

#23 Keyur Makwana

One more thing it is fine as you said we need to introduce automation for project/product which we required more regression testing but I need to know about if project is small size and not much regression testing work then automation is feasible or not ? If yes then why or not then why ?

#24 Mohammad Saad

@Keyur Makwana: Your question will be answered in article number 2 and article number 3 in this series. You will get to know that when you will need automation and when you can avoid automation all together.

#25 kv

wt are the entry criteria for automation testing

#26 Waheed Abbas

this is a fantastic article!! thanks saad and admin.
Keep sharing .

#27 Luke

@Mohammad Saad Thank you for your prompt and clear answer. I’m looking forward for your upcoming articles.

Thank You,
Luke

#28 Sunil

Nice start to explain Automation in easy way from the scratch.

#29 lalitha

Nice Article .I need some information for microsoft test manager.

#30 Dorothy Graham

Thanks for this basic intro to automation, Mohammed. I look forward to your other articles.

I hope that our wiki on test automation will also be useful (TestAutomationPatterns.wikispaces.com).

I will also be interested to see if you reference my 2 books on test automation! ;-)

#31 Hitesh

Great start!!! You explained automation concept so simply and precisely that I got it in first read. Please publish the remaining articles soon. I can hardly wait to delve deeper into this. I am itching to introduce automation at my work. I know a little bit about Selenium but there are so many other options as well that I know nothing about. I look forward to learning more about all the other tools.

#32 Mohammad Saad

Hitesh: Thank you for the compliments. You will get an idea about different tools in article number 3 . “The tool selection process and comparison of various automation tools.” . Keep reading.

#33 Tina

Very comprehensive article. Would be very much interested in having some tutorials in MTM. Some comprehensive information about Microsoft Test Manager would be great.

#34 Rajesh

Very good article, thank you

#35 Occygen

OCCYGEN is a test management tool. In a more general way, it is a tool for QA / Validation activities in a company. Occygen enhances the productivity of testers and improve visibility on the test run progress. It is a project management tool and a rich internet application that can be accessed through any web browser, no installation required. On server side, it can be deployed under Windows or Linux machine. It offers customization of the fields and test case execution using Nosetests based on Python Scripts. Can be integrated with ticket management softwares like Mantis, Bugzilla and Jira.

#36 Abhi

Good reading your article…I am looking for more information on how to start writing automation.How to develop the test framework

#37 Vimal

First i would to say many thanks to Mohammad Saad for great information on Automation Testing. i cant wait to read for next Articles . Best of luck.

#38 Thulani

Great article indeed Automation is explained very simple and understandable.

#39 karthikeyan

Is scripts generated automatically using tools or someone has to manually write. Pls help

#40 ramya

why you people delete orange hrm portial

#41 Karthik Rajesh

Nice Article..could you please add articles on Automation Estimation Techniques

#42 Arati

Hi,
Thanks for all these tutorial series.

I am about to start Automation testing in my organization. This tutorials are really big help.

Looking forward to get good knowldge on selenium.

#43 Shanthya

Hi,

Your tutorials are very useful. Could you please add article about selenium in Visual Studio using C#

#44 shahid qureshi

saad sir……..what a example you dealed with…awsome

#45 Nandini

Awesome article!.. Am new to automation.Could you help on tool selection(open source) for the application which is functionality oriented.

#46 Shriprakash Saroj

Very good article…………

#47 sun

nice article

Leave a Comment