Why it's hard to Estimate Time for Engineering Projects
By Tim Daycounter
I was recently listening to "Beyond the Goal" by Eliyahu Goldratt famously know for his work with TOC in manufacturing systems, and made the following observations about why it's difficult to predict how long an engineering project will take. He is an expert at identifying underlying assumptions which have either become obsolete due to changes in technology, or which were never valid at any time. In the area of project management, the underlying assumption is that a time estimate that an engineer will make has a normal probability density function (a bell curve). The symmetrical normal distribution has tails that extend to infinity around the mean. In the case of time we know that this density function doesn't apply because you can't have negative time, and thus the curve is not symmetrical, and has a longer or thicker tail on the longer side of the mean. That fact alone means that projects will tend to take longer than you would expect with a normal probability distribution because of the shifted uncertainty.
In the typical project the customer or the client comes to an engineer and asks him how long a given task will take. The novice engineer with an ego, will underestimate project, and then will miss his deadline and lose credibility, in the case of a solo contract engineer he'll also lose money, when he finds he's underestimated the project greatly. Engineering projects after all like onions, as you peal away a layer there's another problem layer, that you never anticipated.
The more experienced engineer will pad his bid with a large margin, if he finds that he's over estimated the time he will just add more busy work, so that his time estimate matches the actual time spent. This gives him credibility and protects his pocket book. Notice that in either case the length of the project is not minimized.The experienced engineer is creating a local optima by buffering his own estimate.
This concept also applies to budgets. At the beginning of a year managers will make budgets and receive funds accordingly. Only the foolish manager will come in under budget for the year, because this will mean that his funds next year will be cut.
This affect is even further amplified with larger projects, with multiple tasks and many individuals.
Goldratt's suggestion as usual is to try to do a global optimization of the system at the expense of local optima. Having each engineer estimate his task creates many local optima, but system wide inefficiencies. The solution that he proposes is to remove the local buffers, and to place them at different global project points. I wish he had elaborated further, as to the implementation of this, but I believe he's on to something here.