What is Software testing estimation? Why it is needed?
Software testing is an integral part of the Software Development Life Cycle (SDLC). In recent times, the increasing software complexity has led to an increase in testing complexity as well. Estimating testing effort accurately hence has become an even more important factor towards project success.
Test estimation is required to know the answers to these two questions -
How long will the testing process take? and,
How much will it cost?
Finding the answers to these questions will feedback to the test planning and will help to determine testing duration and cost.
What to estimate?
Software Testing Estimation techniques are the effort estimation techniques that calculate the approximate :
Time - How much time testing will take?
Team Count- How many QA members are required?
Cost - What will be the overall testing cost?
Skills - What shall be the skill set of the QA team?
Effort estimation in software testing
Some of the most popular and effective techniques in software testing estimation are explained in the below section. Though these techniques are explained from a software testing estimation standpoint, these can also be used for estimating any complex work or project.
Types of estimation techniques in software testing
1. Work Breakdown Structure (WBS) :
Break the work into smaller modules for easy & accurate estimations. In this technique, a testing task is broken down into smaller modules and those modules are further divided into measurable sub-modules. Each sub-module is, in turn, divided into functionalities and they are split into sub-functionalities. The outcome is a very detailed, tightly coupled, traceable yet easy to understand, and manageable hierarchical map of project functionality. Using any other estimation technique, these modules are estimated to get actual effort. The benefit of using a work breakdown structure (WBS) as a prerequisite with other techniques is that breaking the complex project into smaller modules gives more accurate and measurable estimates. The estimation process becomes simpler because it is easier to test and estimate smaller tasks. The other benefit of WBS is that sub-modules and functionalities are easily distributed among the team members, can be easily tracked and measured against original estimation.
2. 3-Point software testing Estimation:
The statistical way of estimation.
Similar to the WBS, it also divides the task into smaller sub-tasks. But, this is a statistical method where three possible scenarios should be estimated for each sub-task based on prior experience or best guesses.
a = the best case estimate
m = the most likely estimate
w = the worst case estimate
A. The best-case estimate: It assumes that the project aspects like the presence of a highly skilled team, availability of necessary resources, everything will go right and the project will face no blockers. Now using any other estimation technique project is estimated for the best case to come to a value. Let's assume this number is 50 man-days for our further calculation.
M. The most likely estimate: It assumes that when the team is skilled, with enough project resources, most of the things will go well with very few blockers. Now using any other estimation technique project is estimated for the most likely case to come to a value. Let's assume this number is 70 man-days for our further calculation.
W. The worst-case estimate: When the team is insufficiently skilled, the project has limited resources and most things will not go as planned. Now using any other estimation technique project is estimated for the worst-case to come to a value. Let's assume this number is 100 man-days for our further calculation.
Estimate Calculations
B, M, W respectively are used to calculate ‘E’ and ‘SD’.
‘E’ for estimation, and
‘SD’ for standard deviation - measures the variability or uncertainty in the estimate
E= (B+4M+W)/6 For above example, E = (50+(4*70)c+100)/6 = 71.6 man-hours SD = (W - B)/6 For above example, E = (100-50)/6 = 8.3 Therefore, the nearest approximation for the testing estimate can be considered to vary within the range of E+SD to E-SD man-days. For the above example, the estimate will vary from 63.3 to 79.9 man-hours.
3. Function Point Analysis (FPA) :
Estimation from the functionality standpoint. Function point analysis is based on the specifications of the project or design. Similar to the WBS, tasks are divided into modules and each model is given a function point, depending on its complexity. Modules with higher complexity have higher points. This technique indicates software functionality from the user’s point of view and estimates the value of Total Effort with respect to time, cost, or size of the task. 4
Total Effort = Total FP x Estimate per FP
Estimate per FP is defined by the test manager on the basis of team experience and skill, with respect to time, money, or size. For instance, 10hours/points or $100/points. The FP for each module = No. of modules of a certain difficulty x FP for that module. Each module’s FP is then added to have the total FP.
4. Delphi Technique
This technique is a group estimation technique and is one of the most popular where estimates are derived after multiple rounds of questionnaires sent to a panel of experts. It has the following steps -
An expert panel makes forecasts, with reasons, based on the results of multiple rounds of questionnaires regarding how many hours a certain task or project will take under the guidance of the manager.
After the first round, the experts are allowed to revise their estimates based on how they interpret group responses, accounting for other experts’ judgment.
Rounds are repeated till the range of forecasts decrease and an average value is reached.
This method is simple and reliable as the experts are highly experienced in the subject area. The resulting estimates from this technique reflect the consensus estimation of the group of experts.
5. Agile Estimation
In the previous techniques, details and requirements are defined before we plan the schedule and budget. This has some drawbacks because the software industry is constantly changing and hence the use of the previous techniques is decreasing. The test estimation techniques in agile support continuous delivery. In these techniques, the currently available data and prior experience are used for estimation and new information is continuously integrated into the project to refine the process of estimation. Some of the widely used agile estimation techniques are -
Planning Poker: It is a consensus-based technique for estimating, mostly used to estimate the effort or relative size of testing by breaking project work into sprints. At the start of each sprint, estimates are done for the user stories (smallest measurable user requirement) and priorities are defined. The team members use a deck of cards with numbers on them from 0 to 21 in the Fibonacci sequence (0, 1, 2, 3, 5, 8, 13, 21). These numbers represent ‘Story Points’. The meeting moderator describes a story and asks agile team members to estimate the effort privately without consulting any other team member. Estimates are done on the story-pointing scale. Members then asked to hold up the card at the same time showing the effort they think is required for the story. This consensus repeats with discussion until all the votes are accounted for and reasoned for.
T-Shirt Sizing: Sometimes, the story-pointing scale is overwhelming for the team members to estimate. In such cases it is more efficient to switch to a non-numerical system like T-shirt sizes: XS, S, M, L, XL, and so on, with these sizes corresponding to the story size that the member estimates a story to have. This presents a simple yet accurate way to estimate testing efforts.
6. Test Case Point Analysis / Test case based estimation
This test case-based estimation technique is useful when test case writing is completed or a number of test cases and their complexity is known or estimated beforehand. In the Test Case Point (TCP) analysis, the test cases are used as input for estimating testing efforts. Test cases are classified in terms of complexity. Usually as low, medium, high, and very high. Then considering a test case of each complexity level, an effort value can be estimated for each level of complexity. This value can also be measured by running a test case each from the complexity levels and noting the time it took to run the test. Then this time is multiplied with the number of test cases of each category to come to final estimates of the complete test case set.
For eg, assume we have to estimate the testing effort of test cases set of 100 test cases
Step 1. Classify test cases on the complexity scale. Let's assume below is the output from our test case set of 100 test cases.
Step 2. Estimate the time it will take to run test cases for each complexity level.
Step 3. Calculate the total estimates for running all test cases using numbers from step 1 and step 2.
7. Estimation based Historical Data:
This method analyses and studies the historical data from past testing projects. This technique works on the rule that time taken for testing projects in the past will take similar efforts for similar complexity projects or functionality. This method is useful for estimating projects which have similar nature, technical stack, and test team members.
Conclusion
With the current ecosystem of reducing time to market and increasing customer needs, accurate estimation is the prime factor for achieving project success. Without accurate estimations, the projects will either fail or run out of budget and leave clients complaining. It is also important to determine which type of estimate will be the most suited for a particular project. There may also be cases where you need to use more than one technique or a combination of multiple techniques to bring about the most accurate estimation for testing. If the variation between different techniques is not large, it gives an assurance of accurate estimates.