Continuing my introspection on the #noestimates discussion (yes, let me admit I am fascinated by the idea!), I am trying to explore why there is so much disconnect between developers and managers on estimates.
There are 2 points I would like to discuss in this post:
- Software development as art or science
Consider the first point – some of the comments to blog posts (Tobias Mayer, for example, on Woody Zuill’s post on the noestimates hashtag) point to the age-old discussion of Software development as art or science.
Many Developers believe it is art and hence every time a user story or requirement comes up, they don’t know what the final form the implementation would take. So internally, they answer, “I don’t know” to the question “What do you think of this story? How long do you think it reasonably would take to implement?”
Do you ask a painter how long it would take to paint?
However, they may answer with a number or range depending on the culture of the organization.
There are languages with precise rules, standardized services and tools, known design patterns, coding standards, reams of historical data and most importantly, enterprise apps have well-defined business workflows.
“We are not asking you to imagine something in the air and develop it. We are asking you to digitize business processes that we have been working on paper for the last decade!”
We have “software engineering” and “computer science” courses in Universities and most developers/testers are called Software engineers, analysts or other terms that indicate that this is a profession based on “method and logic”. In other engineering or scientific professions, an estimate of time and cost is an essential pre-requisite to starting work. If we continue to call ourselves software engineers, we should expect to be asked for numbers.
“Every science touches art at some points—every art has its scientific side; the worst man of science is he who is never an artist, and the worst artist is he who is never a man of science.” - Armand Trousseau, Lectures on Clinical Medicine (vol 2), The New Sydenham Society, 1869.
The second part is accountability. The development team wants to be held accountable only for the value they produce, value which is defined and managed by others. The cost of producing that value has to be borne and managed by those who ask for it. However, managers want to hold the team responsible for time (which translates to cost).
One of our better-run projects had a 2-day story mapping workshop, where all MVP business stories were discussed in detail. The team arrived at 16 weeks to complete with a team size of 11. At the end of 16 weeks, 85% of the MVP was completed with both developers and biz being satisfied. There was some cost overrun due to additional software tools, servers, DB licenses and services of an expert, but this could not have been foreseen anyway.
The key point here is at what level estimates are being considered – micro estimates which are based on a well-defined user story with clear acceptance criteria vs macro estimates which are based on Epics? Accuracy is often better in the short term than in the long term. As described in the paper, “An Economic Model of The Planning Fallacy“, incentives for accuracy help in getting estimates that are closer to actual time than a reward for faster completion!
These two fundamental differences have to be resolved, without which the discussion on #noestimates will never end.
Glen Alleman (@galleman) responded to my previous post with this paragraph, which resonated with my experiences with Execs and Managers:
“If most projects overrun their cost targets, then the business that allows that, is eroding its bottom line with every dollar that is above the ROI limit. Not a sustainable business strategy”
I think the fundamental difference in the two approaches is “On what basis are project funding decisions made.” In the government sector (with its regulations and standards), a budget based on cost estimations has to be submitted in advance. This would also be the case in many Enterprise SW development places where software intensive systems may be built, but the software is NOT the core competency of the organization.
However, where the software is the primary product of the organization, estimates may not drive the decision making – you already know that you HAVE to build it. So just build it!
As Matt Barcomb, VP of product development at Taxware, notes in the CIO magazine’s #NoEstimates session at AgileNashville 2013, Projects can be decided for investment based on value modeling and risk modeling rather than cost modeling. What value would an application give you in terms of market capture, lead time over competition or other business value meters (taking cost of the product out of the equation for a minute)? Also, if you did not fund this project or build it this way, what would you lose now, 2 years down the line, 10 years in the future?
I am not convinced that this paradigm can be extended to all other domains – at least in places where the head honcho of IT doesn’t even report to the CEO! As noted in my earlier posts, cost is a major driver in investment decisions.
It boils down to this – can the cost of building something be disassociated from the resulting value and probable risk of the product itself?
Please share your experiences, solutions that worked in your context and best practices.
Over the past few weeks, some of us have been debating on the #NoEstimates paradigm. To summarize, the two positions are:
|Estimations are necessary||No, they are not|
|When we spend other people’s money, we are contractually and morally obliged to provide them with cost information based on which they can take meaningful decisions. Without this cost estimate, ROI and hence a Business case cannot be justified||Projects need to be selected based on intrinsic value and importance to Business goals rather than vague notions of economics. Most projects anyway run over in terms of cost and therefore spending a lot of time on estimating is a waste of time|
|If you are spending too much time estimating, then you either don’t have expertise in estimating, have no historical data for estimates or have flawed estimation models.||We can arrive at cost in other ways, say by retaining a team for 6 months. Estimating individual size of features and then adding it up gives a flawed estimate, since the implementation for the SW can dramatically change the estimates. Further, the whole notion of agile is built upon accepting changes based on the PO’s definition of value|
|Estimation does not mean a large upfront design estimate. Iterative estimates are possible based on the overall capabilities + capabilities delivered every iteration||Another reason not to estimate is because in today’s biz scenarios, estimations are also used for so-called “controls.” Story points and velocity is used to “predict and measure” team performance. Any deviation becomes a reason for root cause analysis – the dysfunction here is that estimates are used for performance measurement of individuals. Teams spend a lot of time in refining estimates that could be spent in delivering SW|
|That is a management dysfunction, not a fault of the estimation models||Probably, but because the widespread management dysfunction causes development teams to focus more on non-essentials AND the fact that we could deliver software by simply doing it and delivering every two weeks without any impact the SW development process, we are suggesting #noestimates to avoid the dysfunction|
I think I captured the essence of the debate
- While #noestimates is certainly possible, a lot of things need to come together: self-less teams (teams that have only value delivery at heart rather than stretching out dev tim to earn more overtime pay), measuring progress by features developed against a roadmap (capabilities against dates), management participation in risk avoidance/risk mgmt rather than simply pushing for resolution, work-life balance etc
- However, real world business certainly needs to make “this project or that project” decisions based on cost vs value delivered in the shortest time possible. E.g., one customer I worked with had to decide whether to modernize a key legacy application in the current (but modern platform) vs porting the entire app to a cloud-based product with customization. In both cases, capability is the same, but costs are very difficult (dev cost + maintenance costs over the years). In summary, #noestimates can work on small projects, where a specific budget is set apart and development progresses till that budget is reached (after which there may be decision to terminate or continue the project)
- While there are multiple sources on how to estimate, there are no studies on the actual results of the corporate IT projects – what historical estimates are available, how do teams report actuals, how many changes have occurred, how were overruns handled and reported etc. Specifically, what consequences exist when estimation is incorrect (due to simple estimation bias) at individual levels
- The Cone of uncertainty principle shows that initial estimates can vary upto 400%, but as we go further in the lifecycle, estimates become more accurate. Scrum uses this intrinsically, so estimation at the micro level is built into the methodology that has been successfully used all over the world. But adding up story point estimates and deriving velocity to be used for measuring performance of the team does not seem appropriate. Also, as Naresh Jain mentions in his blog (3 years ago!), “Its a real big stretch to take something which was inherently not meant to be mathematical or statistical in nature and calculate velocity based on it”
- With limited budgets, organizations are looking for “guaranteed” capabilities within an approved budget (you said $500K for this feature set +/- some buffer for contingencies). Some change and risk management actions can be taken, but at least 75% of the scope needs to be delivered. This is NOT going away soon in Enterprise situations. On the other hand, Neil Killick argues that this is an inverted budgeting mechanism – you decide what you have in hand and then decide to spend in small increments
What, you ask, is my position? Here it is:
- Estimations are needed for 2 reasons – one, to help make Execs decide on VOI (Value on investment) and two, to track progress of achieving capabilities against a release date (needed for internal and market reasons). This will also help in making course corrections as needed
- However, the fundamental basis for #NoEstimates is that they are misused by teams and executives. This must stop. Use estimates as guide posts and realize that micro estimates are good only in the very short term
- Without historical data, it is impossible and even with it, you must budget for a minimum of 50-100% contingency OR understand that all needed capabilities may not be delivered
- Technical decisions will add cost – every line of code may not be functional in nature
It is possible that my understanding of the discussion and my position is flawed – please do let me know otherwise. For a lot more info, please visit the following blogs/posts
Too often, teams make the mistake of thinking that decision-making in Agile must always be consensus-based. While self-organization means adopting practices that help deliver value, signing up for tasks instead of allocation, being honest about problems, it does not mean that every decision has to be unanimously approved by everyone.
However, on an individual story level, there can be confusing, no-right-answer decisions. Which JS library to use, stored procedure or function etc. So, how do we decide, “Is this the best way”?
I often recommend having the person implementing the task decide how to implement it. Of course, this may involve talking with experts in and outside the team, but the decision remains with the person doing the task. For example, in film making [which I think is a close analogy to SW Dev], the Screenplay writer, the director and others may place constraints on the location/setting, but the cinematographer decides the camera/lighting positions, panning, zoom etc.
This also avoids a subtle and insidious process “smell” I often see in Agile teams – when all developers are not equal – competency, experience or maturity may widely vary, one or two people tend to dominate. As Mike Cohn elegantly puts it in his article “The role of leaders in a self-organizing team“, some team members may exhibit subtle control and make decisions for the entire team.
The Scrum Master should be watchful for such destabilizing factors – after all how decisions are made is important to the morale of the team while the results of the decisions determine success or failure.
When an Enterprise decides to “go Agile”, most execs do it because of peer pressure and what they understand about agile. Nothing wrong about it, but there is a next step. The Agile Evangelists now have to convince everyone, from the CxO to her cat, on adopting Agile.
Image by Ansobol
Another case is when a consultant is brought in to “establish an Agile framework and kick start it in the company” – whatever that means.
The worst way to drive Agile adoption?
“Risk in traditional PM is shelf-ware”, “Changes are painful in typical software development”, “People lie and cheat in our projects”, “No one knows about the real state of the project”, “there is too much scope creep ” are common complaints.
These may be true, but the problems certainly are the not the result of following plan-driven methods like the PMBOK or CMMI. Where does the PMBOK say that the PM should not look at the Risk plan once it is baselined? or that scope creep is allowed indefinitely?
To sell your product, you show its benefits – selling it by stretching the competition’s weak points to unreasonable extremes is not a good strategy. These methods have helped the companies survive so long (even if they are limping along…).
So how do you sell the transition to Agile methods?
Begin by articulating in business terms the benefits:
- Delay commitment from Business on what gives most value till the team is ready to start development
- Move risk of building the wrong product or building the right product wrong to early in the project stage, where the cost of throwing it out is very less
- Derive value from the start of the project – value is based on what biz functionality is working. If you have to terminate the project due to funding, you have a base from which to restart later
- Move the responsibility of project success from IT (Project Manager) to the Biz, who are the sponsors of projects (paymasters). This builds Biz-IT relationships like no other way
- Empower the team to become self-organized and take decisions – this reduces the need for oversight (lesser management levels). Risk of failure is higher, but the cost of failure is lower due to frequent feedback loops from the Biz
- No prescriptive rules means lesser overhead on “compliance” and cost of “non-compliance” to arbitrary rules
This is not a Top 10 benefits of Agile blog post, so will leave it to you to come up with other benefits.
Can the Enterprise transition to Agile?
Yes, but… there are growing pains.
- Org Structures and roles will substantially change, which means that some people will feel threatened. There are no easy answers – you may retrain, reskill and redeploy most of them, but some personality types cannot function in an Agile organization
- Cultural change will need to be driven by Execs – respect is the single most important attribute. If you have respect at all levels:
- Execs and Managers will not question/bully developers on estimates and capacity
- Developers will provide honest, no-slack estimates and will be willing to take up work in the interest of the team
- There are no superstars in the team
- Biz does not seek to take advantage of the team to achieve infinite Scope with limited time
- Late-night and late-weekend work is not a regular phenomenon and everyone understands Work-Life balance
- You have to understand there are no easy answers to certain problems. Not you, not the team, not the expert team you brought in to consult. you have try, fail, learn, and try again
- It will be expensive early on – hiring experienced Scrum Masters, training the teams, being patient while the team struggles to find a cadence etc
- Single biggest change would be loss of control [or even better, the illusion of control] and correspondingly, the sense of importance of being a senior executive! Carrot and Sticks don’t work well
- Your employee performance goals, assessments and reward systems will undergo drastic changes. This is a team game, not an individual one
There are definitely more, depending on your company culture.
Why is this so hard in Software?
I like to think it is because Software development involves both science as well as art – it deals with bits and bytes but depends solely on humans for making those bytes work. Methods focusing on one aspect are doomed to fail (and that brings up my pet peeve of using manufacturing/construction as metaphors for SW Development).