How to Select Correct Test Cases for Automation Testing (and Ultimately Achieve a Positive Automation ROI)

Now is the era of test automation. Most of the testing projects are trying to translate their manual test cases to automated one to improve productivity and coverage.

One of the key steps to commence automation testing is – selecting the appropriate test cases and determining the ROI (Return of Investment).

What to expect from this article?
In this article, I have tried to cite down few important points based on my experience to help you select the correct candidate of automation and determine various other factors which will yield better test results and benefits.

manual to automation testing process

Why Automated Testing?

Automation does not overpower or replaces manual testing but it compliments it. Like manual, automation too needs a strategy with proper planning, monitoring & control. Automation, when implemented correctly, can become an asset to the team, project and ultimately to the organization.

There are many advantages of automation; here are few important to mention:

Many a time stakeholders feel that test automation acts as a support tool for manual testing, so it’s vital to understand that automation is the best way to increase the effectiveness, efficiency and coverage of testing. It not only saves time but also improves accuracy as repetitive tasks via manual approach can prone to human errors and can be time consuming.

Candidates of Automation

Basic Mistake to Avoid:
One of the most basic mistakes which testers make is NOT Selecting the correct test cases for automation.

Don’t just select any test suite. Analyze the test cases thoroughly and select the candidates for automation considering the most important factor i.e. ROI. First we should understand and find out the ways for getting a higher and positive ROI.

(ROI – Return on Investment – It is a calculation of benefits in terms of cost saving, increased efficiency and quality)

There is no standard procedure of determining the correct test cases for automation. It all depends upon the application you are testing.

Based on my experience, I have tried to chalk down few steps which may provide some insight for selecting the test cases and ultimately progress to achieve a positive ROI for automation.

See Also = > How to Translate Manual Test Cases into Automation Scripts?

How to Select Correct Test Cases for Automation Testing

Step 1:

Identify the parameters on which you will base your test case as a candidate for automation.

As of now I am identifying the below parameters, you can have your own parameters depending on your application.

  • Test case executed with different set of data
  • Test case executed with different browser
  • Test case executed with different environment
  • Test case executed with complex business logic
  • Test case executed with different set of users
  • Test case Involves large amount of data
  • Test case has any dependency
  • Test case requires Special data

Step 2:

Break each application into modules. For each module, analyze and try to identify the test cases which should be automated based on the parameters. This list will vary for projects to projects and can also be enhanced to suite your needs:

Fig 1.0 (click on image to enlarge view)

Manual to automation testing

Y – Yes
N – No

In a similar way, for all modules, this list can be used to identify the automation candidate test cases.

Step 3:

Consolidate and group the number of test cases for each module shown below

Fig 2.0 (click on image to enlarge view)

Manual to automation testing 1

Figure 2.0 is quite straight forward and self explanatory. Here I am trying to quantify the details and giving an estimate to finish the testing manually.

Step 4:

Once you have identified all the granular level details, you can present them in the below way. We are now progressing to calculate the ROI.

Fig 3.0:

Manual to automation testing 2

------------

We should also take into account the below attributes which forms the basis of deterring the ROI:

  • Purchasing and licensing cost of the tool
  • Time to develop the scripts
  • Time to maintain the scripts.
  • Time to analyze the results manually and automatically
  • Time and cost to train the resources.
  • Management overheads

Test Automation ROI Calculation Example:

In most of the cases, ROI is calculated for 5 years, but it’s not mandatory. Based on the above factors, let me try to elaborate the calculation of ROI for 5 years. As usual you can always tailor and enhance it.

(click on image to enlarge view)

Manual to automation testing 3

*ROI = (Cumulative saving / Investment through automation)*100

Manual to Automation Testing – What are the Process Challenges?

I have tried to cite the points, which I feel, are a big challenge, when we try to automate a test suite.

#1. Automation Need: Every test team is unique and has an exclusive need for automation. We cannot develop a fix standard, but we can tailor a standard which will suite our need.  Because of this reason, automation does require a good support from the management as well as from the development team.

#2. Automating the complete application: Automating 100% application is a big task. Not that it is impossible, but it requires proper planning and monitoring and of course; some time. There are lots of permutations and combinations of data, n number of environments with n number of authentication and authorization attributes which needs to be validated and hence requires a strategy to automate.

#3. Manual Vs Automation mentality:We normally automate which is important and repetitive, but we prefer to test the important functionality manually”. Confused? Even I am!! But this is a fact. We should have a criteria which will decide which are the important test cases. These criteria can be based on multiple factors like complex business logics, areas which are of more interest to clients, risk prone areas etc.

#4. Deciding on the framework: Designing the framework is the most important facet of automation. I believe we should devote relatively more time to develop the framework than to script. Whenever we develop the automation plan, framework designing should be the main focus. PLAN to design the framework. Identify and make a checklist of the items which will form the framework. If the framework is rock solid, scripting and maintaining becomes easy.

#5. Knowledge of the team: Whenever we think of automation, we immediately jump to learn the programming language or scripting language. Learning these language will definitely help but more emphasis should be on building and developing logic. Automation should not be the responsibility of some handful resources, rather entire team should contribute towards it. This will help not only to enhance the skills of the resources, but also keep them motivated.

#6. Reporting: Every tool has a standard to report the test results. To customize it; is a challenging task. Reporting the test results also requires coordination and maintenance which adds to the cost.

#7. Trust: We should trust our automation stuff. We invest man hours to build an automation suite but still we don’t believe in the test results. Efforts should be given to maintain the scripts. Also we should see that the team who is doing the manual testing of the application, should be involved to automate it as they know their application. Most of the time, a third team does the automation so the actual testing team is not aware of the scripts and finally end up with running the tests manually because they feel follow up on the scripts,  add up to their tasks

See Also => Manual and Automation Testing Challenges.

Conclusion:

In most of the cases, we prefer to automate the regression suite (here are some challenges in automating regression suite in agile environment) as it contains bigger number of test cases. In that case we can break down the regression suits into smaller suits and decide to run the appropriate suite as per the release requirement. Suppose a regression suite contains 1500 test cases, you can break it to 3 suits of 500 test cases per suit and automate it.

Instead of automating the entire suite, you can opt for phase wise automation. In other words, you can follow the prototype model for developing the automation suite. Create a structure or framework with implementation of fewer numbers of test cases and start using that and gradually enhance it by adding more test cases to it.

We should follow the Deming wheel (PDCA cycle) for automation as well. Being an ongoing activity, emphasis should be laid on building the framework properly which will result in ease of maintenance and implementing new features. It requires proper support from the dev team and from management as well. We should encourage the testing team to contribute the most for automation testing as they know their product more than any one else.

About the Author: This is a guest article by Shilpa Chatterjee Roy. She is working in software testing field for the past 8.5 years in various domains.

Hope we have simplified this ‘manual to automation testing’ process. Please feel free to share your experience and thoughts on how you overcome the process challenges.

Recommended reading

26 comments ↓

#1 smita

The automation ROI example provided is very helpful. It shows how automation is clearly a winner over long run.
thanks for explaining this in so simple words.

#2 Kirti Jadhav

challenges are identified correctly and also like the solutions provided to those.
great article.

#3 Niraj

Excellent blog, everything is very systematically that a fresher can also get idea from basic to advance level concept. Great Work, Keep on going.

#4 Sunil

Really nice work.
Great help. Keep it up , please..

Thanks,
sunil

#5 Sujata

Really a great help to identify and differentiate between the two testing processes.

Thanks.
Sujata

#6 Raj

Thanks for your patience providing detailed information. It also helps those who are facing interviews

#7 bibhishan

Thanks sir,

sir can we use selenium web driver for small application?
suppose my application is like online project management system.
please help me i am confuse

#8 Shilpa Chatterjee Roy

Thanks All for commenting on my article!
@Bibhishan,

Selenium 2 (web driver) is a browser automation tool so you can use it no matter how big your application is. But before putting your efforts into it, its better to identify how frequently you actually run the automation candidate test cases. Sometimes it happens that for smaller applications, when you run manually the tests are executed in lesser time, so management are not so keen to automate those stuffs, but if you could present the benefits of automating that small application, it should not be a problem.
As far as cost is concerned, selenium being an open source tool, should not be a concern.

#9 sai

Impressed by metrics given.Sounds good and practical,Keep posting this kind of articles.It helps testers to gain more knowledge on Metrics.

#10 Mann

Really Nice “Thanks for sharing :-)“

#11 Srividya

Really nice article. Thanks for sharing and helping us to improve the new latest concepts. keep us shared.

#12 Sadeesh Kumar

A Good and Perfect Article, Every Management who looks for Automation would have read this!! Good Work Roy:)
Keep us posted a very good information like this!!!

#13 srikanth

Really it’s good blog…. thank u sir …..

#14 Rakesh

I need a Best testcase template. Can u please share the same to my mail. rahulrakesh16@gmail.com

Thanks in advance.

Rakesh S M

#15 Bibhishan

Thank mam,

For this great information…

we always looking for this kind of great information…

#16 Manjunatha

It’s really good article. It gives clear idea about selection of test cases for Automation script and also ROI

#17 Priya

Excellent article. This is what I was looking for.

#18 nagi

how to know the load generator configuration….?

#19 rengarajan

I need a Best testcase template. Can u please share the same to my mail. rengarajan.m@sigmainfo.net

#20 ramya

I fed up with ur online class. I paid the money but after two days of class I cant login into goto training session. The following message is displaying “The organizer has canceled your registration, so you cannot join this training session. Please contact the organizer with any questions.”

#21 Kshitij

Hi,

Can STH, have date the article was published. Something published 4-6 years ago, might not make a good read today.

Kshitij.

#22 Vijay

@ Kshitij
The best way to find the date is to check the first comment :)

#23 Sandeep Duve

Really Nice article.

#24 Stephen Foster

Shilpa, thanks for the article. I am not able to figure out the formula for the Cumultive Savings field in your ROI spreadsheet.

Could you please provide it to me?

Thank you!

Stephen

sfosterqa@gmail.com

#25 Rajat

very Useful Article. Thank you for sharing.

#26 aneesh patel

How to identify regression test cases in both manually or automation?

Leave a Comment