On Time Estimation of Software Tasks

A while ago, I was working on a project where I was the sole engineer. As part of planning for the project, I had created a task breakdown and gave an estimated story points number for each task. The total development effort required (in this case, the total story point count) came out to be more than what my manager and the product manager had expected. Since we wanted to ship it to production quickly, my manager suggested that we cut scope and simplify the requirements. In the end, we decided to cut down some functionality and ship the project out in phases.

While I understand that this is common industry practice to cut down scope from projects in order to meet timelines, it really makes me question:

If you know something is important enough to be built, why do you care how long it takes to build. If we make the decision of whether to build or not build based on how long it takes to build something, is that feature really important enough?

I was talking to Aditya Agarwal about it recently, and he said something that stuck with me:

Essentially, working in sprints is corner cutting

When I think back, I don't remember any instance where estimating the time it takes to build a feature has helped the feature ship any faster. The only thing it has really helped with is keeping the product teams up to date by how long their feature is delayed by.

And there's the other thing: estimation of software tasks is hard. Hard enough that Dropbox has a guide on how to estimate tasks. And yet, there is a whole section on why you should add buffer time because unexpected things are going to happen and things are going to go wrong. There is always unknown complexity when it comes to programming. While having some sort of timeline does help product teams get an understanding of when a feature is going to go live, it is rare that a project actually follows that timeline and ships on time without additional scope cuts.

I think this is also the reason why most products or features suck when they come out. Because they have been committed to a tight deadline and due to poor estimation of effort, a thorough testing has been lacking.

That being said, there are legitimate cases where an estimate is helpful. For example, if you are operating a factory line, you know exactly how much time it takes to produce a particular item but software engineering is not like that.