How Does Test Planning Differ for Manual and Automation Projects?

We all agree that Automation projects are different in nature from Manual testing ones. Although, autonomous automation projects don’t really exist (or should not exist ideally), both manual and automation projects are dealt differently when being planned.

A mis-planned project inevitably gets mis-executed; this not only affects the current project and casts a shadow on the individual’s capabilities, but can also lead to the loss of confidence in the team for the client/management- affecting further business.  I would rather say that we testers be safe than sorry.

A good Dilbert comic about planning..

project planning dilbert

Before we go any further, I want to establish what this article is NOT going to be about.

1) This is not an in-depth discussion of automation frameworks. Different projects use different frameworks depending on the nature of their AUTs, architecture, complexities, team’s expertise etc. The information regarding the frameworks can be found at the below links:

Test automation frameworks part 1 and part 2.

2) This is also not about template, format or creation of a test plan document. We are going to address the pre-documentation considerations for an automation project, more in the lines of a feasibility analysis.

3) This is also not tool specific

Every activity in the SDLC takes time, effort, infrastructure- in other words- MONEY.

For a manual testing project the cost consuming factors are:

  1. People
  2. Tools – Test/defect management
  3. Infrastructure – environment
  4. Time
  5. Training

For an automation project, in addition to the above items it needs expenditure for:

  1. Automation tools
  2. Add –in for test management tool integration
  3. Add-in to support AUT (like SAP, oracle etc)
  4. Framework set up
  5. Tool specific training

Given these circumstances, does the success of an automation project depend on how well you have written the code, how many reusable components you wrote or in how few lines of code you achieved the desired result?

No.

There is one and only question that determines the success – “Are you able to generate a better ROI (Return on Investment) in comparison to the manual route”? – If not immediately, eventually.

If the answer to this question is “NO” then you have planned the automation project incorrectly.

Normally, a test plan has the following sections. We are going to discuss each one of them focusing on automation specific aspects to consider:

automation test plan

Automation Testing Test Plan Sections

Section #1: Scope

  • Choose the test cases/scenarios that are to be regressed over and over across multiple cycles.
  • Sometimes the simplest of test cases need lots of complicated solutions to be automated. If these are just for a one time use, it obviously does not make sense. Reusability should be your focus.
  • Automation Testing does not/cannot perform exploratory testing.

Section #2: Test strategy

  • This section is referred to as Framework in the automation world.  Some frameworks are extremely challenging to create and also are effective – but time, effort and competency wise they are demanding.  Always look for a middle ground and do the best you can without jeopardizing over utilization of resources.
  • Decide on coding best practices to be used, naming conventions, locations for test assets to be stored, format of test results , etc. to maintain uniformity and increase productivity.

Section #3: Resources/roles and responsibilities


  • The first step in this direction is to understand the team’s capabilities and anticipate ahead the scope of automation coming into picture. This will help choose a team that suits both the automation and manual testing needs. Also, pick people who have the right attitude – those do not think that manual testing is beneath their stature.
  • Choose a team well versed with AUT, test management, defect management and other SDLC activities
  • Section #1: Scope

Section #4: Tools

Pick automation tools based on the following rules:

  • Does the company already have licenses for a certain tool, try and see if you can use it
  • Look for open source(but reliable) tools
  • Do the team members know the tool already or do we need to bring in someone new? Or train the existing ones?

Section #5: Schedules

  • Include time for code-walkthroughs and inspection of the automation scripts
  • Maintain the scripts on a timely basis. If you create a piece of code that you are not going to use for the next 6 months or so, make sure to periodically maintain it to lessen its chances of failure.

Section #6: Environment

  • The target environment that your AUT is going to run and the automation tool that you want to use should be compatible. This is one of the factors to be considered pre-licensing for the tool.
  • Also, analyze if the rest of the management tools in place and the automation tool you are trying to bring in are inter-connectible for additional benefit.

Section #7: Deliverables

  • Your test scripts are your deliverables. However, not everyone is automation/programming language savvy. So, plan on creating a “How-to” document that will help the current users and future team members to be able to understand this script even when you are not around.
  • Include comments in your script too.

Section #8: Risks

If you are going to propose an automation solution, be sure to choose cost effective tools and solutions to make sure that the automation endeavor does not burden the project. It is important to set the expectation that ROI for an automation project cannot be positive immediately but can be clearly seen over long periods of time.

Therefore, if you propose automating a system, pick the one that is

  • Stable and not too much maintenance
  • Has a scope for huge regression suites
  • Does not have too much of manual intervention or does not depend of a human’s intuition

Section #9: Test data

  • Take into consideration the security aspects of the data
  • Do not hard code any test data into the scripts. This just leads to too much script maintenance and might induce errors during modification.
  • Be very specific. For a manual test step – ‘enter first name’, you can say enter any 5 character name. While testing, a tester can type “Swati” or “seela” or anything else. But for a tool it can’t make such suppositions. Therefore, provide exact values.

Section #10: Reports/results

  • Script execution results are also technical and might not be easily understood by the rest of the teams. Plan on writing detailed results to notepad or excel sheets as an additional measure.
  • Detailed framework documents, review results, defect reports, execution status reports are also expected.

We, as automation enthusiasts might think that clients/management don’t easily buy the automation proposals.

However, when our ultimate goal is to maximize ROI through automation, we are in perfect harmony with the management/client’s goals too. This will ensure that we not only get to automate our project, but will be able to do so, with lots of consent, co-operation and excitement.

Planning and thorough analysis on all the factors listed above can be our ally through this journey. Again, ROI means everything.

This post is written by STH authors team member Swati Seela.

Have queries or things to discuss? Feel free to post in below comments.




Recommended reading

10 comments ↓

#1 Gourav

You focused on automation ROI that is the best part of this panning difference.

Success of automation depends on gain in ROI but at the same time one should also consider that automation takes some time to get into positive ROI.

Nice article.

#2 Aniruddha

Thanks,
Article is good.
How will you compare the ROI on manual testing and Automation testing. How will you calculate ROI for both types of testing.

Thanks
Aniruddha

#3 JJ Hill

I am sure I would have been fascinated by the content of this article but the URL includes a defect, and I couldn’t take the rest of it seriously. ;)

#4 Aman

its nice But if you can talk about testing so you should go with ROI – Risk.

Thanks
aman

#5 Shanmugavel.C

Well. Nice article. Always the automation leads thinking of.
Always management forgot to understand the efforts and pain involved during automation and simply neglecting the resources who put their best work. If ROI is not coming eventually, then absolutely it’s not the actual resource’s mistake.
Anyway good article…

#6 kumar

Hi, i want to do a project on automation testing(selenium).
Where can i get it.

#7 Deyan

Hi Man,

It is really a great site and it has been useful to me.

I am new to automation and want to start automating some parts of a banking program but I am confused which Automation Program should I best use.

We use Oracle 12c for Database,
Weblogic to read the information and use Oracle 11g Forms to display the data.
For the reports we use Oracle Apex, SQL Plus and RDF.

If you could guide me through which programs I should choose from.

Really appreciate it.

Regards,
Deyan

#8 Razi

Good help.But tell me how we will handle a brand new application which we dont know which functions are reusable and which is just one time only .Also the requirements are changing during development and the business line is not sure if they want these functions or not .They will decide few functions upon completion of the project .

#9 Eshan Sarpotdar

Very nice post Swati, however, there is more to Test Automation. Though we are incorporating high-end test strategies, it is still unclear that errors may arise during final stage.

Following up on this, I came across and registered for a webinar on “Automation for Test Automation” It would impart the process in developing an automation suite having a self-maintaining automation mechanism and which would work for any customization and configuration. Here is the registration link – http://j.mp/1itquYj

#10 Jay

Plz provide Bugzilla testing tutorials with examples

Leave a Comment