Dan North: Embracing uncertainty

István Soós September 11, 2012 5 min read

Dan North has been coaching, coding and consulting for over 20 years, he is the originator of Behaviour-Driven Development. He gave the following talk at the Norwegian Developers Conference in June 2012.

You can read the outline of the talk here.

Fear leads to Risk,
Risk leads to Process,
Process leads to Hate, Suffering and Gantt charts.

Risk - the business-friendly word for uncertainty

We are uncertain of the future, but that is not something you talk about on a business meeting, it doesn’t sound professional. Nevertheless, the business dictionary defines a phrase for it: we will be managing risk. In addition to that, we will create processes, as the ultimate tools to resist the uncertainty, with defined validations and checkpoints.

Waterfall software development is a good example how business imagined an ideal world: it is a gated model, and we have well-defined rules, with checkpoints when everyone knows and accepts that certain things are true. In reality, everyone is frightened to deliver a bad feedback or bad news, in the hope that eventually problems will disappear, or at least someone else will be blamed for it. Until the end, when everyone switches to a new project or goes to a different company.

The Agile Manifesto was created to uncover better ways of developing software. Yet, after ~10 years, we tend to defy its goals and principles:

It seems that we have a good idea here, but its interpretation becomes a dogma, our complex problems demand simplistic answers. We would rather be wrong than uncertain.

The risk analysis lie: likelihood is never zero, impact is never infinite

Analyzing risk in a one-dimensional model is easy, but risk is more a multi-dimensional variable, than a simple numerical value. However, to simplify this uncertainty, analysis tend to put events in the extremes, e.g. considering impact as infinite. After such assumption, the only way to mitigate the risk is to minimize the likelihood, preferably to zero. Everyone knows it is a lie, but at least we can find someone to blame if the event happens.

It will happen at some point, no matter what.

Embrace: uncertainty

Dan North gives a few examples how we can embrace the uncertainty and face our fears.

If we are uncertain of the scope: scrap the backlog, forget the 600+ items, instead do a rolling planning: every time only the next 2-3 biggish themes. With just that, we can have surprising amount of certainty in the delivery.

If we are uncertain of the technology: accept that you don’t know if something is going to be good. Contract 2-3 vendors in isolation (all paid), and after a point, stay with one.

If we are uncertain of effort: think of throughput and results instead of efforts. We are obsessed with activities, but we should measure and balance everything else too. Self-organizing teams are better, as the team sees the bigger vision, more efficient solutions might came out of nowhere.

If we are uncertain of structure: don’t work with specialists, instead put people in different roles in different times (e.g. developer, designer, tester, support).

Oh, that craving for certainty!

A good way to think about it to treat decisions like buying a financial option. Like options, any decision has value (opportunity factor, different cost model, capital expense vs operating expense), which changes over time, and eventually it expires. (Read more about real options.)

Early commitment signals our fear of uncertainty. If a sprint-planning closes down options, it means we will learn nothing in the next 4-6 weeks.

A good methodology allows us to defer decisions, let them be about scope, tooling or technologies.

Estimation as a process isn’t useful in itself. It becomes useful, because while we are doing it, we learn something. But these discoveries are accidental, and usually because some business guy happens to be in the same room. Having these "oh-crap" moments earlier plays a critical part of our success (read more about deliberate discovery).

Assume: some unexpected bad things will happen.
- some == nonzero
- unexpected == you cannot plan for them
- bad things == will affect the project, we cannot sidestep them

What can we do?

Don’t make lists (as the ultime source of truth). No matter how long list you make, things will be missing. Assume you are second order ignorant (lack of awareness): you don’t know what you don’t know. But you can act actively to reduce your ignorance.

Use group activities to detect and point out other’s blind spots.

Making something better with planning, deciding, doing, checking and measuring is good, but sometimes you need to step back and ask: is the cycle the right cycle?

Convincing others might be hard:

Expect the unexpectable.
Anticipate ignorance.
Embrace uncertainty - it’s inevitable.

updated: August 29, 2014
István Soós
software engineer, business advisor
Advocates for the maker-movement, self-directed learning and agile methods. His regular topics include: machine intelligence, data and risk analysis, distributed systems and knowledge management.