In the last couple of Selenium tutorials, we learned about automation testing using Cucumber and Selenium tool. We also discussed about the integration of Selenium WebDriver with Cucumber.
In this tutorial, we will discuss different factors affecting effort estimation of Selenium automation.
Planning and estimation are two most important aspect of a software development lifecycle.
I personally feel that in the software industry, there are no bulletproof methods of doing anything. Since every project is exclusive and has different sets of complexity and environmental factors, implementing the estimation and planning strategy should be a collaborative effort of the individual teams with proper interventions of seniors and management support.
Before you begin with estimating any project, it is imperial to understand each and every phase that your project will be going through, so that you can give a correct and a justified estimation.
Estimation can not only be done for the manual testing process but in this era of automation, estimation techniques are applied to test automation as well. Now Selenium gaining a momentum and popularity in the market, I am trying to write about some factors which should be taken into consideration while estimating a Selenium project.
I am assuming that we are starting the Automation initiative from the scratch and that we have no ready-made framework available.
What You Will Learn:
The various factors which effect and which you should consider for estimation of “Selenium” specific project are explained below:
Scope typically means identifying the correct test cases for automation. Apply “Divide & rule” strategy to accomplish it. Break your application in small chunks or modules and analyze each of them to come up with the appropriate test cases for automation.
The steps involved are:
For more details of how to identify the correct test case, please see my previous paper: Selection of correct test cases for Automation
Steps involved here are:
Here we have to establish the definition of big/medium and small sized application. This definition differs from an individual/group perspective. How you classify your application, depends can also be dependent upon the number of test cases.
If your application has 300 – 500 test cases to automate, you can consider it as the small-sized application. If the test cases are over 1500, it can be classified as complex. This factor can be different for the different application. For some, 1500 test cases to automate can be considered as small / medium scaled. So once you have identified the exact number of test cases, scale it to small/medium or large. Your strategy towards estimating the effort will hugely dependent on these criteria.
You have to also consider the different checkpoints and verification points for your test case. A test case can have more than 1 checkpoint
but will have only 1 verification point. In case you have more than 1 verification point, it is recommended to bifurcate into separate test cases. This will also ease your maintenance and enhancement of your test suite.
Steps involved here are:
Selenium alone is not sufficient to build a framework or complete the automation. Selenium (Web driver) will only script the test case, but there are other tasks as well, like reporting the result, tracking the logs, taking screenshots etc.
To achieve these you need separate tools that will be integrated with your framework. So it is important here to identify these supporting entities which will best suit your requirement and will help to get a positive ROI
Here comes the tricky part J the steps involved are!!
There are many other requirements apart from just scripting in test automation with Selenium, like reading the data from a file, reporting / tracking the test results, tracking logs, trigger the scripts based on the input conditions and environment etc. So we need a structure that will take care of all these scripts. This structure is nothing but your Framework.
Web applications are complex by nature because it involves lots of supporting tools and technology to implement. In a similar way, implementing the framework in Selenium is also tricky (I will not say complex) as it involves other tools to integrate. Since we know Selenium is NOT a tool but actually a collection/group of jar files, it is configured and not “Installed”, Selenium itself is not strong enough to build a complex framework. It requires a list of third-party tools for building a framework.
We have to remember here that there is nothing “Ready-made” in Selenium. For everything, we have to code, so provisions in estimation should be there for googling the errors and troubleshooting.
Here we have to understand that that Framework building is the most important aspect of your Automation effort. If your framework is rock solid, maintenance and enhancement become easier especially in the era of Agile, if your framework is good, you can integrate your tests in all the sprints easily.
I won’t be wrong if I say that this particular factor of designing the Framework should be the most important aspect of estimation. If needed (like in complex application) this factor should be again broken down into a separate WBS and estimation should be done.
Learning Selenium is a bit different than learning any other automation tool. It basically involves learning a programming language than just a scripting language (though script language helps while building a framework like you want to write a script that would invoke your automated scripts after doing the environment setting changes).
In case we are combining WebDriver with java, I would say that if one is well versed with core java, they are in a very good shape to start with Selenium automation.
Along with learning java, provisions should be there to learn other technologies like ANT / Maven( for building), TestNG/jUnit ( unit test framework), Log4J( for Logging), reporting ( for reporting) etc. this list may grow based on the level of the framework. The more this list grows, the more time it would take.
If the management has decided to go with selenium, these learning activities can be done parallel with the planning activity. Because there is no limit to learning these technologies, it is suggested to have a definite plan (syllabus) ready for the team so that they can initiate their learning process in a definite direction and everybody is on the same page.
Practically speaking, we testers do not have a very much keen in learning a full-fledged programming language and we feel this is developers piece of cake. But now we have to change this mentality and should consider learning the programming language to be equally important as learning the new testing process. This will not only increase tester’s knowledge about the language and automation but also will give a chance to understand how the application works internally which may increase their scope to find new bugs.
Environment set up deals with (not limited to):-
Before you actually start writing your tests, there are 2 prerequisites:
Identify different actions that your web application does. It can be simple actions like navigation, clicking, entering text; or a complex action like connect to a database, handle flash or Ajax. Take one test case at a time, and identify what all action that particular test case does and estimate hours accordingly for per test case. The sum of all the hours for the entire test suite will give you the exact number.
Provision should be there for Review as well. The reviews are simply the code review which can be done either by a peer or a developer. Pair programming is the best option which is quick, but if it is not possible, based on the available resources or organizations review strategy, hours should be allocated for it.
More details about each factor affecting estimation:
Factor #1: Scope
Meaning: Identifying the candidate test cases for automation through the ROI
Deliverable: List of test cases that needs to automate.
Remarks: It is important to freeze your scope once you go ahead with other steps of estimation.
Factor #2: Complexity
Meaning: Establish the definition of straightforward and small sized application.
Deliverable: Size of the application – Small, medium or Big.
A number of test cases and their corresponding checkpoint and verification point.
Remarks: Recommended – A test case can have multiple check point but only 1 verification point. If a test case has more than 1 verification point, it should be bifurcated into a separate test case.
Factor #3: Supporting tools
Meaning: Selenium itself is not strong enough to build a complex framework. It requires a list of framework tools for building a framework.
Deliverable: List of tools needed to create the framework.
Factor #4: Implementing Framework
Meaning: Creation of structure
Remarks: This is the most crucial step. It is advisable to include some buffer time while estimating because some time troubleshooting takes more time than expected.
Factor #5: Environment set up
Meaning: Deals with code set up and downloading/preparing for the code deployment
Deliverable: Environment ready
Remarks: We should try to build our framework in such a way that with least hassle, our code is deployed into the said environment/box.
I should not be wrong if I say that with minimal entries into our text files (which have the URL and credentials) our code should be ready to run and ROCK!
Factor #6: Learning & training
Meaning: Learning a programming language and other supporting technologies
Steps Involved: Prepare a plan as per your automation needs and share it with the team and encourage them to learn and proceed as per the syllabus.
Deliverable: Training Plan and its tracker which will track the progress of the team.
Remarks: Emphasis should be on building logics rather learning syntax.
Factor #7: Coding / scripting and Review
Meaning: Writing the actual test scripts and reviewing them
Deliverable: Automated test scripts
Remarks: the Whole team should participate in writing the test scripts using the implemented framework. So while estimating, efforts from the whole team should be taken into consideration.
Having said about all these points, do not forget to include Management overhead and some buffer time in your final Selenium automation estimation. The best and the proven way to do any estimation is to follow the WBS (Work Break down Structure) mechanism. This is straight forward and serves the purpose of implementing the automation estimation needs.
The factors mentioned above are the ones based on my experience, but there can be other entities as well which might affect the strategy.
The thumb rule here is “Identify certain criteria, divide your modules or test case on those criteria; and scale it”. Based on your scaled figure – you can come to an accurate estimation.
Next Tutorial #33: We will be concluding our most comprehensive Selenium online training free tutorials series with last tutorial i.e. “Selenium testing interview questions with answers”.
Let us know if you have any other tips for effort estimation of Selenium projects.