7 Factors Affecting Test Estimation of Selenium Automation Project – Selenium Tutorial #32

The tutorial will discuss different factors that influence test estimation in Selenium automation. It’s time to begin.

In the last couple of Selenium tutorials, we learned about automation testing using Cucumber and Selenium tools. We also discussed the integration of Selenium WebDriver with Cucumber. Here, we will discuss the factors that affect the effort estimation of Selenium Automation.

Planning and estimation are the two most important aspects of a software development lifecycle.

I feel that in the software industry, there are no foolproof methods of doing anything. Every project is exclusive and has different sets of complexity and environmental factors. Hence, implementing the estimation and planning strategy should be a collaborative effort of the individual teams with proper interventions of seniors and management support.

Test Estimation of Selenium Automation Project

Before you estimate any project, it is crucial to understand every phase that your project will go through, so that you can give a correct and justified estimation.

Estimation can not only be done for the manual testing process but estimation techniques are applied to test automation as well. With the rising popularity of Selenium, I’m discussing key factors for estimating a Selenium project.

Let’s Start!!

I assume that we are beginning the Automation initiative from square one and have no preexisting framework.

Factors Affecting Estimation of Selenium Automation

Below, we explain the various factors that you should consider for estimating the “Selenium” specific project.

#1) Scope of the Project

Scope typically means identifying the correct test cases for automation. Apply the “Divide & Rule” strategy to accomplish it. Break your application into small chunks or modules and analyze each of them to come up with the appropriate test cases for automation.

The steps involved are:

1. Identify the various factors which will form the basis for identifying the candidate test cases.
2. Break the application into smaller modules.
3. Analyze each module to identify the candidate test cases.
4. Calculate ROI.

For more details on how to identify the correct test case, please see my previous paper: Selection of correct test cases

#2) Complexity of the Application

The steps involved here are:

1. Determine the size of the application based on the number of test cases that need to be automated.
2. Size complexity through the Fibonacci series.
3. Identify the verification points and checkpoints for each test case.

Here, we have to establish the definition of big/medium and small-sized applications. This definition differs from an individual/group perspective. How you classify your application can also depend upon the number of test cases.

Example:

If your application has 300–500 test cases to automate, you can consider it as a small-sized application. If the number of test cases exceeds 1500, it can be classified as complex. This factor can be different for different applications.

For some, 1500 test cases to automate can be considered small/medium scale. So once you have identified the exact number of test cases, scale it to small/medium or large. Your strategy towards estimating this effort will depend hugely on these criteria.

You also have to consider the different checkpoints and verification points for your test case. A test case can have over 1 checkpoint, but only 1 verification point.

If you have over 1 verification point, it is recommended to bifurcate into separate test cases. This will ease the maintenance and enhancement of your test suite.

#3) Use of Supporting Tools/Technologies

The steps involved here are:

1. Identify the framework and automation needs.
2. Based on the needs, analyze and identify the tools to be used.
3. Identify the dependencies/implications of using the tool.

Selenium alone is not sufficient to build a framework or complete the automation. Selenium (Web driver) will only script the test case. However, there are other tasks as well, like reporting the result, tracking the logs, taking screenshots, etc.

To achieve this, you need separate tools that will be integrated with your framework. So it is important here to identify these supporting entities that will best suit your requirements and will help you get a positive ROI.

#4) Implementing the Framework

The steps involved are:

1. Identify the input (pattern in which data is fed into the script) and output (reports/test results) of your automation suite.
2. Design your input files. This may range from a simple text file to a complex Excel file. It is the file that will have your test data.
3. Design the folder structure based on your input parameters.
4. Implement the reporting feature (either in some Excel file or using any tool like ReportNG).
5. Determine/implement loggers in your framework.
6. Implement the build tool in your framework.
7. Implement the unit test framework (Junit or TestNG).

There are many other requirements apart from just scripting in test automation with Selenium, like reading the data from a file, reporting/tracking the test results, tracking logs, triggering the scripts based on the input conditions and environment, etc.

We need a structure that will take care of all these scripts. This structure is nothing but your framework.

Web applications are complex by nature because they involve lots of supporting tools and technology to implement. Similarly, implementing the framework in Selenium is also tricky as it involves other tools to integrate.

Since we know Selenium is NOT a tool but a collection/group of jar files, it is configured and not “Installed”, Selenium itself is not strong enough to build a complex framework. It requires a list of third-party tools to build a framework.

We have to remember here that there is nothing “Ready-made” in Selenium. For everything, we have to code, so provisions in estimation should be there for googling the errors and troubleshooting.

Here we understand that Framework building is the most important aspect of your Automation effort. If your framework is rock solid, maintenance and enhancement will be easier. Especially in the Agile era, if your framework is good, you can integrate your tests in all the sprints easily.

It wouldn’t be wrong if I say that this factor in designing the Framework should be the most important aspect of estimation. If needed (like in complex applications) this factor should again be broken down into a separate WBS and estimation should be done.

#5) Learning and Training

Learning Selenium differs from learning any other automation tool. It involves learning a programming language than just a scripting language (though script language helps while building a framework like you want to write a script that would invoke your automated scripts after doing the environment setting changes).

In case we are combining WebDriver with Java, I would say that if one is well versed with core Java, they are in a very good shape to start with Selenium automation.

Along with learning Java, provisions should be there to learn other technologies like ANT / Maven (for building), TestNG/JUnit (unit test framework), Log4J (for Logging), reporting (for reporting), etc. This list may grow based on the level of the framework. The more this list grows, the more time it will take.

If the management has decided to go with selenium, these learning activities can be done along with the planning activity. Because there is no limit to learning these technologies, it is suggested to have a definite plan (syllabus) ready for the team. Clear direction allows them to start their learning process and gain unanimous agreement.

Practically, we testers are not very keen to learn a full-fledged programming language and we feel this is for developers. But now we have to change this mentality and should learn the programming language to be equally important as learning the new testing process.

This will not only increase the tester’s knowledge about the language and automation but also will give them a chance to understand how the application works internally, which may increase their scope to find new bugs.

#6) Environment Setup

Environment set-up deals with (not limited to):-

• Setting up the code in the test environment.
• Setting up code in the production environment.
• Writing scripts for triggering the automated tests.
• Developing the logic for reporting.
• Establishing the frequency of running the scripts and developing the logic for their implementation.
• Creating text/excel files to enter test data and test cases.
• Creating property files to track the environment and credentials.

#7) Coding/Scripting and Review

Before you write your tests, there are 2 prerequisites:

1. Candidate test cases should be handy
2. Framework is ready

Identify the different actions that your web application does. It can be simple actions like navigation, clicking, entering text; or a complex action like connecting to a database, handle flash, or Ajax.

Analyze each test case individually, identifying the actions performed and estimating the hours required for each case. Adding up all the hours in the test suite will give you the precise total.

Provisions should be there for review as well. The reviews are simply code reviews which can be done either by a peer or a developer. Pair programming is a fast option, but if it is not possible, based on the resources or organization’ review strategy, hours should be allocated for it.

More details about each factor affecting estimation:

Factor #1: Scope

Meaning: Identifying candidate test cases for automation through ROI

Steps Involved:

1. Identify the various factors which will form the basis of identifying the candidate test cases.
2. Break the application into smaller modules.
3. Analyze each module to identify the candidate test cases.
4. Calculate the ROI.

Factor #2: Complexity

Meaning: Establish the definition of straightforward and small sized application

Steps Involved:

1. Size the application based on the number of test cases that need to be automated.
2. Size complexity through Fibonacci series.
3. Identify the verification point and checkpoint of each test case.

Deliverable: Size of the application – Small, medium or Big.

There are several test cases, along with their corresponding checkpoint and verification point.

Remarks: Recommended – A test case can have multiple check point but only 1 verification point. If a test case has over 1 verification point, it should be bifurcated into a separate test case.

Factor #3: Supporting Tools

Meaning: Selenium itself is not strong enough to build a complex framework. It requires a list of framework tools for building a framework.

Steps Involved:

1. Finalized IDE
2. Finalized unit test tool
3. Finalized logger
4. Finalized reporting tool
5. Finalized build tool

Deliverable: List of tools needed to create the framework.

Remarks

Examples:

• Eclipse / RAD – as IDE
• Ant / Maven – As build tool
• jUnit / TestNG – as unit test framework
• Log4j – as Logger
• ReportiNG – as reporting tool
• Text files – for tracking the environments/credentials
• Excel files – for tracking the test data
• Perl / Python – for setting up an environment and triggering the test scripts.

Factor #4: Implementing Framework

Meaning: Creation of structure

Steps Involved:

1. Design your input files.
2. Design the folder structure.
3. Determine/implement logger in your framework.
4. Implement the build tool in your framework.
5. Implement the unit test framework

Deliverable:

• Framework and folder structure created in the IDE.
• Excel sheet containing your input data.
• Property files containing environment-related data and credentials.

Remarks: This is the most crucial step. It is advisable to include some buffer time while estimating because some time troubleshooting takes more time than expected.

Factor #5: Environment set up

Meaning: Deals with code set up and downloading/preparing for the code deployment

Steps Involved:

1. Prepare the input file and report.
2. Create a triggering script.

Deliverable: Environment ready

Remarks: We should try to build our framework in such a way that with least hassle, our code is deployed into the said environment/box. I should not be wrong if I say that with minimal entries into our text files (which have the URL and credentials) our code should be ready to run and ROCK!

Factor #6: Learning and training

Factor #7: Coding /Scripting and Review

Meaning: Writing the actual test scripts and reviewing them

Steps Involved:

1. Test cases and framework are ready.
2. Take/divide the test cases, convert them into automated scripts and track your progress.

Deliverable: Automated test scripts

Remarks: The whole team should participate in writing the test scripts using the implemented framework. So, while estimating, everyone in the team should take their efforts into consideration.

Conclusion

Having considered all these points, do not forget to include management overhead and some buffer time in your final Selenium automation estimation.

The best and proven way to do any estimation is to follow the WBS (Work Breakdown Structure) mechanism. This is straightforward and serves the purpose of implementing the automation estimation needs.

The factors mentioned above are based on my experience, but there can be some other entities as well which might affect the strategy.

The thumb rule here is “Identify certain criteria, divide your modules or test case on those criteria; and scale it”. Based on your scaled figure, you can come up with an accurate estimation.

Next Tutorial #33: We will conclude our most comprehensive Selenium online training free tutorials series with the last tutorial, i.e. “Selenium testing interview questions with answers”.

Let us know if you have any other tips for effort estimation for Selenium projects in the comments section below. We would love to hear from you.