The only way to understand (innovation) success, is by measuring failure

Success is a scarcity, especially in innovation. In fact, one of the few certainties in life is that everything will fail at a certain moment. Few innovations and experiments succeed and even the biggest success stories end at one point. It’s just the tendency of our universe towards entropy and disorder. Could be after 70 years, like in the case of Toys R Us, but happen it will.

So, let’s say that about 5% (I’m using this exceedingly speculative statistic for a reason. Bear with me.) of all the endeavours of all the startups, scaleups, SMEs and humongous corporations in the world amount to a big hit. That’s not much, right?

So why do we try to understand success by investigating only this tiny portion of positive results in the world of organizations. That’s insane if you take the time to think about it.

The informational value of success is vastly overrated

Truth lies in numbers. Its only surfaces through the patterns in a very wide and seemingly chaotic sea of data. You would never base a delicate and far-reaching decision on a small test-group of 37 people, however relevant their profiles are, right? Yet that’s exactly what we are trying to do when we only measure “what works” in companies, or on a market level.

Thinking about this, reminded me of one of the most interesting mysteries of science: dark matter and dark energy. We only ‘know’ about 5% (the numbers vary, but everyone agrees that it’s a very small portion) of the knowable world. All the rest is dark matter (about 27%) and dark energy (about 68%), which are basically fancy and highly imaginative descriptions for “we have no f***ing clue what they are, but according to our measurements they are definitely there; otherwise we cannot possibly explain how the universe is expanding”.

Our failure to ‘see’ and understand dark matter, is probably why - even though science absolutely rocks, you guys - there are so many things we do not yet have a solid explanation for. The fact, for instance that we lack an approved theory that unites quantum mechanics (which explains how things work on a very small scale) and general relativity (which explains forces on a cosmic scale), is just one of the many examples. To explain the universe, we will need to understand and analyse all the matter, and all the energy.

And, just like that, to explain how organizations work, you need to have all the data: the 5% success as well as the 95% failures.

A new mind-set for failure: from shame to measurement

In order to start measuring failure, we need a change of behavior. We must stop being ashamed and trying to sweep it under the rug. True, there is a slowly changing attitude in the matter (Silicon Valley may be quite good at it, but we’re not there yet in good old Europe) but we still happily move on when our endeavors do not show the planned results and tend to communicate about it as little as possible. We’re learning to accept it, as the unavoidable part of experimenting, testing and innovating. But we still don’t like being confronted with it. We're biased.

It’s true, when our employees fail, they learn from it. But these lessons only tend to happen on an individual micro-level, at best on a team-based level. They rarely spread throughout the rest of the organizational system, because we’re still embarrassed.

It’s high time for a change in mindset. Yes, we can learn from it on a micro-level, but above all, failure is extremely valuable information on a macro-level. We should cherish it, instead of hiding it. It’s crucial we talk about it on an organizational level, and share what worked, what didn’t and what we learned from it. It’s also essential that the data is preserved in such a manner that it can be mined in order to uncover the patterns that drive the dynamics of both failure and success, especially in large corporations with longer lines of communication.

And when we zoom out even more, we really ought to measure the failed companies out there, as well as their context if we really want to get through the bottom of what makes successful organizations tick. Though this does happen, it is not nearly enough.

Does all of this this mean that I believe in some kind of secret ingredient of organizational success? Of course not. It all depends on your team, your culture, where you’re based, what the economic situation is… etc. But I do believe that we can go about this in a more fact-based manner, and that we tend to measure the wrong thing. Success stories are a powerful source of inspiration (it’s the reason why my company nexxworks exists), but if we truly want to understand success, we need all the data.

This piece first appeared in an adapted version in Dutch in Trends, and on the HYPE Innovation blog.