For the success of any project test estimation and proper execution is equally important as the development cycle. Sticking to the estimation is very important to build a good reputation with the client.
Experience plays a major role in estimating “software testing efforts”. Working on varied projects helps to prepare an accurate estimation of the testing cycle. Obviously one cannot just blindly put some number of days for any testing task. Test estimation should be realistic and accurate.
In this article, I am trying to put some points in a very simple manner, which are helpful to prepare accurate test estimation.
What You Will Learn:
- Brief Description of the Test Estimation Process
- The basic prerequisites of the test estimation process
- Most Prominent Test Estimation Techniques
- Details of the test point estimation technique
- Test Estimation Examples
- Use Case point estimation method
- Work-phase breakdown technique
- 9 General Tips on How to Estimate Testing Time Accurately
- Recommended Reading
Brief Description of the Test Estimation Process
“Estimation is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable.” [Reference: Wikipedia]
We all come across different tasks and duties and deadlines throughout our lives as professionals, now there are two approaches to find a solution to a problem.
A first approach is a reactive approach whereby we try to find a solution to the problem at hand only after it arrives.
In the second approach which can be called a proactive approach whereby we first prepare ourselves well before the problem arrives with our past experiences and then with our past experience, we try to find a solution to the challenge when it arrives.
Estimation can thus be considered as a technique which is applied when we take a proactive approach to the problem.
Thus Estimation can be used to predict how much effort with respect to time and cost would be required to complete a defined task.
Once the testing team is able to make an estimate of the problem at hand then it is easier for them to come up with a solution that would be optimum to the problem at hand.
The practice of estimation can be defined then more formally as an approximate computation of the probable cost of a piece of work.
The basic prerequisites of the test estimation process
1) Insights gathered from working with past experience: It is always a good practice to spend some time, recalling about past projects which posed challenges similar to the current endeavor at hand.
2) The available documents or artifacts: The test management repository tools come in handy in these types of scenarios as they store the requirement and clarification documents. These documents can be referred by the testing team to clearly define the scope of the project.
3) Assumptions about the type of work: Past working experience helps in making assumptions about the project. This is where hiring experienced professionals matters most.
The testing managers can pick up the brains of these people for delivering the desired results.
4) Calculation of potential risks and threats: The testing team also needs to visualize the potential risks and threats and pitfalls which lie may lie for the team in future.
5) Determining whether the documents have been baselined: The testing team also needs to determine if the requirements have been baselined or not. If the documents are not baselined then it is important to determine the frequency of the changes.
6) All responsibilities and dependencies should be clear: The organization should clearly define the roles and responsibilities for all the persons who would be performing the estimation process.
7) Documentation and tracking of the estimation records: All the relevant information to the estimation process should be documented.
8) Activities which are required to be performed during the test estimation process
- Organize the team that would perform estimations
- Decompose the project into project phases and subsequent constituent activities
- Compute the estimation based upon previous projects and professional experience
- Prioritize the possible threats and come up with the approaches to mitigate those risks
- Review and document the relevant part of the work
- Submit the work to the relevant stakeholders
Most Prominent Test Estimation Techniques
Some of the most important techniques for test estimation are:
- Test point estimation
- Work-phase based estimation
- Use case point estimation
How and where we use these techniques:
#1) Test Point estimation is a simple and easily understandable estimation technique which is widely used across the software testing spectrum. Iterative phases and simplicity are the most important features of this particular technique.
#2) Work-phase based estimation is the estimation technique which is used whereby a guess estimate is made on a particular phase (normally the shortest and simplest of the phases) and then the testing team gradually adds on other phases into the initial estimation and finally comes up with an appropriate estimation.
#3) Use-Case Point estimation technique is the estimation on the use cases where the unadjusted actor weights and unadjusted use case weights are used to determine the software testing estimation.
Details of the test point estimation technique
The test point estimation technique is done by following the listed steps:-
(The following weights which may vary from project to project could be considered under this paradigm – Some of these weights are the weight for the programming language based upon the complexity of the code, application weight based upon the type of application and test weights which are assigned based upon the different phases of software testing.)
Unprocessed Test Points are multiplied by CWF to obtain the testing size in Test Point’s Size.
The Productivity Factor indicates the amount of time for a test engineer to complete the testing of one Test Point
Testing Effort in Person Hours is computed by multiplying the Test Point size by the Productivity factor.
For computation of the test point estimation technique, we consider the following variables.
- Test requirement complexity
- Interface with other requirements
- Total number of verification points
- Baseline test data
We then need to consider weight vectors for each of the data variables and organize them in the following manner.
Adjustment factor = Average of (product of complexity weight and factor weight) / 30
Adjustment Test Point for Test case design = Total Test Point X (1 + Adjustment factor for Test Case design)
Adjusted Test Point for Test case execution = Total Test Point X (1 + Adjustment factor for Test Case execution)
Total Test Point (normalized) X (1 + Adjustment factor for Test Case design/execution) = Adjusted Test Point for Test Case design/execution
Total effort in Person Hours (PH) = Number of Normalized Test points / Productivity (in Normalized Test points per Person Hours)
Test Estimation Examples
Let us try to apply the above formulation into another practical use.
Suppose we end up with a test requirement whereby we have 5 test scenarios to test.
Now say Test scenario 1 has got 5 test expected results, test scenario 2 6 test expected results, test scenario 3 only 2 test expected results, test scenario 4 9 test expected results, test scenario 5 also 9 test expected results, respectively.
So we classify the test scenarios in three classes i.e. complex, simple and moderate based upon the total number of expected results present in these three classes.
Complex classes will have more than 7 expected results whereas the simple ones will consist of less than 5 expected results and the moderate scenarios would consist of between 4 to 7 expected results.
We thus classify test scenario 1 and test scenario 2 as moderate scenarios, the scenario 5 and scenario 6 as complex ones and test scenario 3 as simple.
We now will apply test points to all these scenarios. We apply 5 test points for complex classes, 3 for moderate ones and 2 for the simple scenarios.
We multiply the assumed test points with the total number of expected results in all these test scenarios. So we end up with the following approximations.
Scenario 1: 3 test points * 5 test expected results = Adjusted test points = 25
Scenario 2: 3 test points * 6 test expected results = Adjusted test points = 30
Scenario 3: 2 test points * 2 test expected results = Adjusted test points = 4
Scenario 4: 5 test points * 9 test expected results = Adjusted test points = 45
Scenario 5: 5 test points * 9 test expected results = Adjusted test points = 45
So considering that we need to apply say 5 Person Hours for each adjusted test point we end up getting the following approximate result.
Test Scenario 1: 25 adjusted test points * 5 person hours = 125 person hours
Test Scenario 2: 30 adjusted test points * 5 person hours = 150 person hours
Test Scenario 3: 4 adjusted test points * 5 person hours = 20 person hours
Test Scenario 4: 45 adjusted test points * 5 person hours = 225 person hours
Test Scenario 5: 45 adjusted test points * 5 person hours = 225 person hours
So total approximated person-hours is: 745 person hours
Use Case point estimation method
Use-Case Point Method is based on the use cases where we calculate the overall test estimation effort based on the use cases or the requirements.
Here is the detailed process of the Use case point estimation method:
An example of the same is that say in a particular requirement we have 5 use cases, use case 1, use case 2,…, use case 5 respectively. Now let us consider that use case 1 consists of 6 actors, use case 2 consists of 15 actors, use cases 3, 4 and 5, 3, 4 and 5 actors respectively.
We consider any use case which involves the total number of actors as less than 5 as negative, any use case with total number of actors is equal to or more than 5 and less than or equal to 10 as positive and any use case with more than 10 actors as exceptional.
We decide to assign 2 points to the exceptional use cases, 1 to the positive ones and -1 for the negative ones.
Thus we categorize the Use cases 1 and 5 as positive, use case 2 as exceptional and use case 3, 4 as negative respectively based on our above-stated assumptions.
So the Unprocessed actor weights = Use case 1 = (total number of actors) 5 * 1(the assigned point) = 5. Similarly
Use case 2 = 15 * 2 = 30 .
Repeating the process for rest of the use cases we receive the Unprocessed actor weights = 33
Unprocessed use case weight = total no. of use cases = 5
Unprocessed use case point = Unadjusted actor weights + Unadjusted use case weight = 33 + 5 = 38
Processed use case point = 38 * [0.65+ (0.01 * 50] = 26.7 or 28 person hours approximately
Work-phase breakdown technique
The work phase breakdown technique can be described in the following steps.
- Break down the overall work into phases.
- Start with the simplest phase and assign an approximate estimation value to it.
- Then proceed with by identifying the next possible phase which could be commenced once this phase completes.
- Derive a possible set of approximation values that could be applied to this phase and choose the maximum value amongst all the derived approximation values.
- Sum up the approximated estimation value by adding the current phase effort estimation value to the already existing value.
- Continue steps 3 to 5 until all the phases identified in the first step are exhausted.
- Accept the final approximate estimate value as the ultimate.
Suppose in a requirement there are 5 required phases. So in the initial phase 1 we assume that total efforts needed are 35 person-hours and then we commence the next phase 2 for which we have 4 comparative assumptions of 35 , 45 , 55 and 65 respectively.
So we consider the 65 person-hour which is the maximum value here. In phase 3 , 4 ,5 we come up with estimates (12 , 33, 43 , 54) , (15 , 10 , 7 , 8) and (2 , 16 , 5 , 13) respectively. By applying the said principle we end up with 185 person hours respectively.
I am putting information on – How to estimate testing efforts for any testing task, which I learned from my experience.
9 General Tips on How to Estimate Testing Time Accurately
Factors Affecting Software Test Estimation, and General Tips to Estimate Accurately:
#1) Think of Some Buffer Time
The estimation should include some buffer. But do not add a buffer, which is not realistic. Having a buffer in the estimation enables to cope with any delays that may occur. Having a buffer also helps to ensure maximum test coverage.
#2) Consider the Bug Cycle
The test estimation also includes the bug cycle. The actual test cycle may take more days than estimated. To avoid this, we should consider the fact that test cycle depends on the stability of the build. If the build is not stable, then developers may need more time to fix and obviously, the testing cycle gets extended automatically.
#3) Availability of All the Resources for Estimated Period
The test estimation should consider all the leaves planned by the team members (typically long leaves) in the next few weeks or next few months. This will ensure that the estimations are realistic. The estimation should consider some fixed number of resources for a test cycle. If the number of resources reduces then the estimation should be re-visited and updated accordingly.
#4) Can We Do Parallel Testing?
Do you have some previous versions of same product so that you can compare the output? If yes, then this can make your testing task bit easier. You should think the estimation based on your product version.
#5) Estimations Can Go Wrong – So re-visit the estimations frequently in initial stages before you commit it.
In early stages, we should frequently re-visit the test estimations and make modification if needed. We should not extend the estimation once we freeze it unless there are major changes in requirement.
#6) Think of Your Past Experience to Make Judgments!
Experiences from past projects play a vital role while preparing the time estimates. We can try to avoid all the difficulties or issues that were faced in past projects. We can analyze how the previous estimates were and how much they helped to deliver the product on time.
#7) Consider the Scope of Project
Know what is the end objective of the project and list of all final deliverables. Factors to be considered for small and large projects differ a lot. The Large project typically includes setting up a test bed, generating test data, test scripts etc. Hence the estimations should be based on all these factors. Whereas in small projects, typically the test cycle include test cases writing, execution and regression.
#8) Are You Going to Perform Load Testing?
If you need to put considerable time on performance testing then estimate accordingly. Estimations for projects, which involve load testing, should be considered differently.
#9) Do You Know Your Team?
If you know strengths and weaknesses of individuals working in your team then you can estimate testing tasks more precisely. While estimating one should consider the fact that all resources may not yield same productivity level. Some people can execute faster compared to others. Though this is not a major factor, it adds up to the total delay in deliverables.
Software test estimation is the practice which requires the involvement of experienced professionals as well as the introduction of industry-wide best practices like test case point and use case point methods.
It is also important for adopting an open mind for customizing the required processes. The successful implementation of these processes lead to an overall improvement of the testing process.
This is a guest article by Author “N. Sandhya Rani”.