Volume 12, Number 06, March 2022

Measurement of Software Development Effort Estimation Bias: Avoiding Biased Measures of Estimation Bias

  Authors

Magne Jørgensen, Simula Metropolitan Center for Digital Engineering, Norway

  Abstract

In this paper, we propose improvements in how estimation bias, e.g., the tendency towards under-estimating the effort, is measured. The proposed approach emphasizes the need to know what the estimates are meant to represent, i.e., the type of estimate we evaluate and the need for a match between the type of estimate given and the bias measure used. We show that even perfect estimates of the mean effort will not lead to an expectation of zero estimation bias when applying the frequently used bias measure: (actual effort – estimated effort)/actual effort. This measure will instead reward under-estimates of the mean effort. We also provide examples of bias measures that match estimates of the mean and the median effort, and argue that there are, in general, no practical bias measures for estimates of the most likely effort. The paper concludes with implications for the evaluation of bias of software development effort estimates.

  Keywords

Software development effort estimation, measurement of estimation overrun, proper measurement of bias.