Imagine you're part of a software development team, charged with creating a groundbreaking new feature. Your team, diligent and thorough, estimates a three-month timeline for this project, carefully considering every phase: design, development, testing, and integration. This timeline is your team's commitment to quality and reliability.
Then comes a twist. The team product manager, driven by internal pressures, wants this feature to be the highlight of the next product launch - just a month away. To push the agenda, the PM mentions an offhand remark from an uninvolved engineer, who casually suggests any good engineer could build such a feature in just three weeks. Hell, they even claim they could do most of it in a weekend themselves.
This casual comment sets off a chain reaction. Higher-ups, lured by the prospect of an accelerated timeline, begin to scrutinize your team's three-month estimate. The pressure builds, and the meticulous, reasoned plan you initially provided starts to be seen as overly conservative, even obstructive.
Under mounting pressure, your team is compelled to condense the original timeline, forcing tough choices: shortened testing phases, a reduced feature set, and a heightened risk of bugs and system failures.
Fast forward to the launch. The feature, though operational, is far from what your team envisioned. The rushed timeline led to unaddressed bugs and missed refinement opportunities, culminating in customer dissatisfaction and a flurry of negative feedback. Worst of all, you agree with the feedback. Your original plan had time to dogfood, iterate, and improve - the new plan was a rushed shot in the dark. Your team shifts into damage control mode, issuing fixes for issues that a more realistic timeline could have prevented. What’s more, you’re already eating into the next scheduled project which will now need to be pushed back as well.
Have I touched a nerve yet? Many software engineers can relate. This story is as common as it is frustrating - but why? Assume everyone involved was well-intentioned and competent - it still happens. In this post, I’ll take the time to dig into why estimates get twisted, the common areas where eng and product conflict, and offer a few suggestions on where a good engineering leader should hold the line.
Note
Greg spends full workdays writing weekly deep dives on engineering practices and dev-tools. This is made possible because these articles help get the word out about Graphite. If you like this post, try Graphite today, and start shipping 30% faster!
Trying for accuracy
While leading engineering projects, I’m constantly required to estimate timelines. How long might it take us to build an async job queue? How long would it take to migrate our webapp to NextJS? To halve the load time on fresh page load?
The fundamental difficulty with any of these estimates is that it’s an attempt to predict the future (hard) in an environment rife with biases (even harder). The best thing I can do for my team is create as accurate of estimates as possible, and hold strong on them, despite outside forces trying to bend me. When these estimates change, these changes have cascading effects across the organization.
According to Andy Grove, former CEO of Intel, organizations can be perceived as systems of stocks and flows. Work progresses through a dependency graph - first an idea is formed, then you have road mapping, planning, design, research, engineering, testing, marketing, phased rollout, until finally completion. And it’s not just one feature passing through this flow at a time - it’s often a myriad of features crisscrossing from dependency to dependency, some bigger, some looping back to earlier steps. For an organization to act efficiently, it must sequence the work through this graph as smoothly as possible so there are no blockages in the flow.
Each approximation isn’t in a vacuum; it dictates a single step in a maze of other features that also all have estimates. For example, if I underestimate how long the new modal takes to build, then I may create a pileup on the engineering team, delaying both the modal work and other projects queuing behind it. That might lead the design team to pause and work on something unblocked but lower priority, and leave marketing in a full stall because there’s no new feature to promote. The traffic jam can cascade quickly, creating frustration, stress, and wasted capacity. Accurate estimates respect that engineering is just one step in a larger web of production.
The pressures on estimates
There are a myriad of forces trying to sway an engineering estimate. The best way to resist is to understand where the push is coming from, and then to tactfully defang it. In this section, I’ll walk through some of the common pushes I experience and recommendations on how to navigate.
Pressure from the side
Project managers, sales and marketing leads, and designers may all push on your proposed engineering estimate from time to time. Often this stems from them leaking their own pressure - a pressure for them to ship particular features and hit their goals on time. Engineering delivering more quickly is strictly better for cross-functional roles because it creates a buffer for non-technical delays and allows for extra features to be squeezed into product launches.
The best strategy I’ve found for navigating cross-functional pressure is to focus the conversation not on shortening the engineering estimate but rather on reducing the project scope. Cross-functional leads are often the exact folks with authority to make scope adjustments - take the time to walk through specs detail by detail, explaining how long each section would take to build and suggesting how much time could be saved if parts were cut. By focusing on scope, you make their pressure their problem to solve, and you maintain integrity in your estimates.
Pressure from above
The second force I regularly face is from non-technical business leaders who care less about a specific feature getting out the door faster, and more about the overall pace of the engineering org. These folks could include leads, executives, and sometimes even board members. I empathize with their desire - they want to keep the company in business, and one of the best ways to do that is to ship products at a blistering pace. They don’t understand engineering well enough to nitpick any specific estimate, but they do wonder if everyone is working hard enough given the realities of the market that they can see.
The best way to deal with pressure from above is an idea I heard suggested by ex-Uber engineering manager Will Larson. He mentioned that pressure from above is one of the few cases where engineering productivity metrics can actually be useful. Quantitative metrics are usually a poor tool for managing the actual execution of engineers, but they can be great for communicating the broad idea that “our team is well above industry averages when it comes to deployment velocity, PRs merged, review times, and more.” While you shouldn’t generally use these metrics to evaluate performance of the individual, quantitive data can be helpful to give larger picture reassurances to non-technical, politically powerful stakeholders.
If you check the metrics only to find that your engineering team is significantly lower than industry averages - for example, if you realize your CI is 5 times longer than other companies in your industry, you’ve found a great point to investigate and maybe unlock new efficiencies.
As a starting point, the book Accelerate recommends the following key metrics as a starting point:
Lead Time
Deployment Frequency
Mean Time to Restore (MTTR)
Change Failure Rate
Code completion vs delivery
Reiterating the example above, sometimes I really will hear a cocky engineer who’s uninvolved in my project say, “I could code that in half the time - maybe even over the weekend if I made the time.” Worse, sometimes a lead or executive will hear the same remark and anchor to it, which inevitably begs the question - well, could it really be done in half the time? Is the owning team fudging their estimates and not pushing hard enough, or are they mediocre engineers for not realizing the possible faster path? These questions can be grating and destructive to trust - seeded from sideline estimates.
Firstly, I’d recommend being very direct in curtailing any culture of giving estimates for projects one isn't responsible for - this can minimize folks anchoring to lower numbers. However, given that the phenomenon is natural, I’d recommend using this as a moment to break apart code-completion estimates from feature-delivery estimations.
It’s possible that the sideline engineer is half correct - maybe two days is a feasible estimate for a buggy code-complete implementation. But you’re not just responsible for a buggy code complete feature that never ships to users. You’re also responsible for debugging it, architecting it cleanly, adding tests to avoid immediate regressions, dogfooding it, adding monitors, performing a partial rollout, listening to user feedback and edge cases, polishing the edges, and finally rolling out to all users. The grander engineering process exists for a reason and needs to be included in the final estimate. By splitting apart a v0 code completion estimate from a longer delivery estimate, you create a framework where you can both validate the sideline engineer while still explaining why delivery of the feature will take longer.
Optimism bias & black swans
The previous three pressures have come from individuals surrounding your project. But there are dangerous pressures that come from within oneself, too. The first is optimism bias - the natural assumption that everything will go according to your diligent and disciplined plan. I can think of projects where I did everything right in planning - researching the codebase, building prototypes, measuring load, and getting thorough feedback on design docs, only for the lead engineer to get called into jury duty for two weeks and delay the project. There are always unpredictable events that can and do happen to disrupt a project’s estimate.
Black swan events sound unplannable by definition. But there are in fact a few strategies capable of mitigating them - as offered by the book “How Big Things Get Done.” The first obvious strategy is to build buffer into a project. This will always help avoid missing deadlines but it may be the most unpalatable for a stressed high-speed organization. Asking stakeholders for 50% extra time just in case may be too big of an ask.
The second strategy is to narrow the window of execution as much as possible. Black swan events follow a memoryless distribution - each day you're still building the project is a day that something unpredictable might happen, regardless of how early into the project it is or what happened yesterday. The best thing you can do to reduce the number of unforeseen delays is to minimize the number of days you're actually building the project. This can be done two ways:
First, you can shift more time into the planning phase. As authors Bent Flyvbjerg and Dan Gardner say, the worst thing that can happen in planning is a lost notion document or an erased whiteboard. Go slower in planning where delays are cheap, so you can go faster in execution where delays are expensive.
Second, you can break projects into small modular parts. Avoid taking on a three-month migration or redesign. Break the work into 2-4 week-long projects. A black swan event may still strike, but a 50% delay in a two-week project is much less than the equivalent in a three-month project. And due to the modular structure, you it becomes cheaper to replan or cut your losses when the unexpected strikes.
Black swan events are so tricky to deal with that there exists yet a third strategy in dealing with them - “reference class forecasting.” This technique is a fancy name for simply finding five, ten, or a hundred similar projects and averaging their delivery time. Because black swans are an unpredictable statistical occurrence, your best chance of accounting for them in estimates is to average their occurrence from as many similar projects as possible. You might not know all the ways things could go off the rails, but the last twenty projects certainly have the failures built in.
Want to estimate how long building a settings page will take? Find how long it took ten other teams to build their settings pages and average the durations. Even anchoring to one or two real examples can put you in a far better starting place than going off your optimistic imagination. A great place to look is similar engineering projects within the same company, whether from your own experience or those of other teams.
Pressure from ego
The last pressure I regularly experience is one of ego, identity, and self-worth. I believe myself to be a great engineer, and I care a lot about upholding that self-image on successive projects. I want to push myself to be better, and the best engineers build hard things fast. Given that everyone around me would love it if I could ship this project just a bit faster - why not give them what they want?
This mentality is tempting, but yet another psychological force that needs to be resisted. While you might be able to make up a short estimate through force of will, or worse, cutting corners - you’ll incur a cost on your teammates around you. Other folks will inevitably need to deal with your bugs and bad code - a feature is always part of a larger codebase. You may speed up a single project, but you slow down the broader org.
What's more, engineering estimates are not only for projects that you implement. As you gain responsibility within an organization, you’ll find yourself weighing in on estimates for projects that tens of other engineers need to execute. You’re no longer in control of their execution, and you need to estimate accurately given the average engineer on your team.
Lastly, firmly believe that an overly tight estimation is not conducive to getting folks to “work harder” or “more creatively.” Rather, working under too much stress reduces creativity, lowers product quality, and takes a debt on future work. If urgency is extremely high, I instead think the effort should be redirected into creatively adjusting scope, modularizing projects, and planning away edge case failures that could delay the delivery.
Closing reflections
With all these pressures swirling around project estimates, you might feel the best option is hiding from them. Give vague, noncommittal estimates. Upscale every estimate by 2x, and mix in uncertain error bounds beyond that. I know a few engineers who act this way, and I can firmly say it’s a bad habit. It may work to protect the stress, autonomy, and ego of the developer - but it hurts the organization. As stated above, accurate estimates are key for sequencing the entire orchestration that goes into making a tech product. Wishy-washy estimates will erode folks' trust in you and force them to make estimates on your behalf.
The better strategy is to strive for accuracy, reflect after each project, maximize your iterations, and build a streak of trust in what you say. This responsibly and corresponding skill is part of what distinguishes an experienced and mature engineer from others, and is worth striving for from a care for professionalism.
Recommended readings related to project estimations
For a deep dive on engineering projects going overbudget: How big things get done
For suggestions on providing engineering estimates as ranges: The clean coder
For thoughtful reasoning on why a little slack can be good: Elegant puzzle
For good research on engineering velocity metrics: Accelerate
