Brief Description of the Test Estimation Process
“Estimation is the process of finding an estimate, or approximation, which is a value that is usable for some purpose even if input data may be incomplete, uncertain, or unstable.” [Reference: Wikipedia]
We all come across different tasks and duties and deadlines throughout our lives as professionals, now there are two approaches to find a solution to a problem.
First approach is a reactive approach whereby we try to find a solution to the problem at hand only after it arrives.
In the second approach which can be called a proactive approach whereby we first prepare ourselves well before the problem arrives with our past experiences and then with our past experience we try to find a solution to the challenge when it arrives.
Estimation can thus be considered as a technique which is applied when we take a proactive approach to the problem.
Thus Estimation can be used to predict how much effort with respect to time and cost would be required to complete a defined task.
Once the testing team is able to make an estimate about the problem at hand then it is easier for them to come up with a solution that would be optimum to the problem at hand.
The practice of estimation can be defined then more formally as an approximate computation of the probable cost of a piece of work.
The basic prerequisites of the test estimation process
1) Insights gathered from working with past experience: It is always a good practice to spend some time, recalling about past projects which posed challenges similar to the current endeavor at hand.
2) The available documents or artifacts: The test management repository tools come in handy in these types of scenarios as they store the requirement and clarification documents. These documents can be referred by the testing team to clearly define the scope of the project.
3) Assumptions about the type of work: Past working experience helps in making assumptions about the project. This is where hiring experienced professionals matters most.
The testing managers can pick up the brains of these people for delivering the desired results.
4) Calculation of potential risks and threats: The testing team also needs to visualize the potential risks and threats and pitfalls which lie may lie for the team in future.
5) Determining whether the documents have been base lined: The testing team also needs to determine if the requirements have been base lined or not. If the documents are not base lined then it is important to determine the frequency of the changes.
6) All responsibilities and dependencies should be clear: The organization should clearly define the roles and responsibilities for all the persons who would be performing the estimation process.
7) Documentation and tracking of the estimation records: All the relevant information to the estimation process should be documented.
8) Activities which are required to be performed during the test estimation process
- Organize the team that would perform estimations
- Decompose the project into project phases and subsequent constituent activities
- Compute the estimation based upon previous projects and professional experience
- Prioritize the possible threats and come up with the approaches to mitigate those risks
- Review and document the relevant part of the work
- Submit the work to the relevant stakeholders
A brief description of some of the most Prominent Test estimation techniques
Some of the most important techniques during test estimation are:
- Test point estimation
- Work-phase based estimation
- Use case point estimation
How and where we use these techniques
#1) Test Point estimation is a simple and easily understandable estimation technique which is widely used across the software testing spectrum. Iterative phases and simplicity are the most important features of this particular technique.
#2) Work-phase based estimation is the estimation technique which is used whereby a guess estimate is made on a particular phase (normally the shortest and simplest of the phases) and then the testing team gradually adds on other phases into the initial estimation and finally comes up with an appropriate estimation.
#3) Use-Case Point estimation technique is the estimation on the use cases where the unadjusted actor weights and unadjusted use case weights are used to determine the software testing estimation.
Details of the test point estimation technique
The test point estimation technique is done by following the listed steps:-
(The following weights which may vary from project to project could be considered under this paradigm – Some of these weights are weight for the programming language based upon the complexity of the code, application weight based upon the type of application and test weights which are assigned based upon the different phases of software testing.)
Unprocessed Test Points are multiplied by CWF to obtain the testing size in Test Point’s Size.
The Productivity Factor indicates the amount of time for a test engineer to complete the testing of one Test Point
Testing Effort in Person Hours is computed by multiplying the Test Point size by the Productivity factor.
For computation of the test point estimation technique we consider the following variables.
- Test requirement complexity
- Interface with other requirements
- Total number of verification points
- Baseline test data
We then need to consider weight vectors for each of the data variables and organize them in the following manner.
Adjustment factor = Average of (product of complexity weight and factor weight) / 30
Adjustment Test Point for Test case design = Total Test Point X (1 + Adjustment factor for Test Case design)
Adjusted Test Point for Test case execution = Total Test Point X (1 + Adjustment factor for Test Case execution)
Total Test Point (normalized) X (1 + Adjustment factor for Test Case design/execution) = Adjusted Test Point for Test Case design/execution
Total effort in Person Hours (PH) = Number of Normalized Test points / Productivity (in Normalized Test points per Person Hours)
Let us try to apply the above formulation into another practical use.
Suppose we end up with a test requirement whereby we have 5 test scenarios to test
Now say Test scenario 1 has got 5 test expected results, test scenario 2 6 test expected results, test scenario 3 only 2 test expected results, test scenario 4 9 test expected results, test scenario 5 also 9 test expected results, respectively.
So we classify the test scenarios in three classes i.e. complex, simple and moderate based upon the total number of expected results present in these three classes.
Complex classes will have more than 7 expected results whereas the simple ones will consist of less than 5 expected results and the moderate scenarios would consist of between 4 to 7 expected results.
We thus classify test scenario 1 and test scenario 2 as moderate scenarios, the scenario 5 and scenario 6 as complex ones and test scenario 3 as simple.
We now will apply test points for all these scenarios. We apply 5 test points for complex classes, 3 for moderate ones and 2 for the simple scenarios.
We multiply the assumed test points with the total number of expected results in all these test scenarios. So we end up with the following approximations.
Scenario 1: 3 test points * 5 test expected results = Adjusted test points = 25
Scenario 2: 3 test points * 6 test expected results = Adjusted test points = 30
Scenario 3: 2 test points * 2 test expected results = Adjusted test points = 4
Scenario 4: 5 test points * 9 test expected results = Adjusted test points = 45
Scenario 5: 5 test points * 9 test expected results = Adjusted test points = 45
So considering that we need to apply say 5 Person Hours for each adjusted test point we end up getting the following approximate result.
Test Scenario 1: 25 adjusted test points * 5 person hours = 125 person hours
Test Scenario 2: 30 adjusted test points * 5 person hours = 150 person hours
Test Scenario 3: 4 adjusted test points * 5 person hours = 20 person hours
Test Scenario 4: 45 adjusted test points * 5 person hours = 225 person hours
Test Scenario 5: 45 adjusted test points * 5 person hours = 225 person hours
So total approximated person hours is: 745 person hours
Use Case point estimation method
Use-Case Point Method is based on the use cases where we calculate the overall test estimation effort based upon the use cases or the requirements.
Here is the detailed process for the Use case point estimation method.
An example of the same is that say in a particular requirement we have 5 use cases, use case 1, use case 2,…, use case 5 respectively. Now let us consider that use case 1 consists of 6 actors, use case 2 consists of 15 actors, use cases 3, 4 and 5, 3, 4 and 5 actors respectively.
We consider any use case which involves total number of actors as less than 5 as negative, any use case with total number of actors is equal to or more than 5 and less than or equal to 10 as positive and any use case with more than 10 actors as exceptional.
We decide to assign 2 points to the exceptional use cases, 1 to the positive ones and -1 for the negative ones.
Thus we categorize the Use cases 1 and 5 as positive, use case 2 as exceptional and use case 3, 4 as negative respectively based on our above stated assumptions.
So the Unprocessed actor weights = Use case 1 = (total number of actors) 5 * 1(the assigned point) = 5. Similarly
Use case 2 = 15 * 2 = 30 .
Repeating the process for rest of the use cases we receive the Unprocessed actor weights = 33
Unprocessed use case weight = total no. of use cases = 5
Unprocessed use case point = Unadjusted actor weights + Unadjusted use case weight = 33 + 5 = 38
Processed use case point = 38 * [0.65+ (0.01 * 50] = 26.7 or 28 person hours approximately
Work-phase breakdown technique:
The work phase breakdown technique can be described in the following steps.
- Break down the overall work into phases.
- Start with the simplest phase and assign an approximate estimation value to it.
- Then proceed with by identifying the next possible phase which could be commenced once this phase completes.
- Derive a possible set of approximation values that could be applied to this phase and choose the maximum value amongst all the derived approximation values.
- Sum up the approximated estimation value by adding the current phase effort estimation value to the already existing value.
- Continue steps 3 to 5 until all the phases identified in the first step are exhausted.
- Accept the final approximate estimate value as the ultimate.
Suppose in a requirement there are 5 required phases. So in the initial phase 1 we assume that total efforts needed are 35 person hours and then we commence the next phase 2 for which we have 4 comparative assumptions of 35 , 45 , 55 and 65 respectively.
So we consider the 65 person hour which is the maximum value here. In phase 3 , 4 ,5 we come up with estimates (12 , 33, 43 , 54) , (15 , 10 , 7 , 8) and (2 , 16 , 5 , 13) respectively. By applying the said principle we end up with 185 person hours respectively.
Software test estimation is the practice which requires the involvement of experienced professionals as well as the introduction of industry wide best practices like test case point and use case point methods.
It is also important for adopting an open mind for customizing the required processes. The successful implementation of these processes lead to an overall improvement of the testing process.