How well do you know your company? (my guess is not very well at all…)

In the 1970s, there was a series of academic studies which looked at managers’ perceptions of the “volatility” of their company’s business environments. These studies all found that different managers within the same organization usually had widely varying views on how volatile their business was: the correlation between different people’s assessments was virtually zero.

In addition, these studies found that there was hardly any relation between objective measures of business volatility and managers’ estimates of these measures (if anything, the correlations were negative; managers in stable environments thought their business was relatively turbulent and vice versa). Similar results were obtained for other variables. The researchers concluded that managers’ perceptions of their own businesses were usually plain wrong.

Two business school professors – John Mezias and Bill Starbuck; at the time at New York University – set out to examine this claim further (and published their results a few years ago) simply because they found it hard to believe; at least at first…

They thought “‘business volatility’, that’s a bit vague and abstract, let’s start with something simple” and they asked executives from a wide range of companies to tell them what their business unit’s sales were in the previous year. Then they looked up the units’ actual sales figures. The average answer was 475.7% wrong. That’s 475.5%...! Sales of their own business unit!!

Then John and Bill thought, “perhaps we should pick something they find really important”. So they approached a blue-chip company and asked them what the company’s absolute top-notch priority was: the CEO declared that the absolute top priority throughout the entire company was “quality improvement”.

And indeed, they had put their money where their mouth was: many managers attended quality improvement training courses, each division had a dedicated department focusing on quality performance and the company had developed various quality metrics. Furthermore, all managers received quarterly quality improvement reports, and 74% of them indicated in a survey that they expected to receive large increases in their personal rewards if their divisions managed to increase quality. Yep, “quality” was important to them!

Quality was measured in the company, following the specialist training techniques, in terms of “sigma” (a measurement of the error rate). When John and Bill asked the managers what the sigma of their department was, the average error in their answer was…(wait for it)... 715.1%. A whopping 715%! They really had no clue…

Note that these had even been managers brave enough to give any answer at all; 7 out of 10 managers, when asked, had refused to give any estimate, declaring “I don’t know”. It seems likely that they had realised they had no clue and rather than make a complete fool of themselves, they opted not to say anything.

Granted, when John and Bill finally asked the brave ones who dared to give an answer to express their unit’s error rate not in terms of the illustrious “sigma” but in the plain human terms of “what percentage of products have errors?” they did a lot better: almost 7 out of 10 managers managed to give an answer which was less than 50% off the mark.

Of course these people are not all fools. They are usually smart, well-trained and hard-working. It is just that they have no clue about the numbers describing their own business – and managers usually don’t. We spend a lot of time, money, effort and attention quantifying all sorts of aspects of our organisations but, at the end of the day, make decisions ignoring all these numbers, using our experience, qualitative assessment and gut instinct.

And that’s probably for the better; if we’d base decisions on our (alleged) knowledge of the numbers, we’d be prone to not only shoot ourselves in the foot, but also in the chin, the head, the back-side and the bodily parts of several of our neighbours.