Relevant statistics you can trust

nbr-relevant-statisticsPublished in The National Business Review (Auckland), 21 June 2013 (PDF)

If you have not noticed it yet, 2013 is the International Year of Statistics.

To celebrate this momentous occasion, Statistics New Zealand invited me to speak at a breakfast event on the trustworthiness of statistics.

Of course, it was tempting to open my remarks with an anecdote, in this case a quote attributed to Sir Winston Churchill. Allegedly, he once said he only trusted statistics that he had forged himself.

It’s a quote that is evidently only famous in Germany where everybody knows it. It’s the German equivalent of “lies, damned lies, and statistics.”

There is only one minor problem with the quote: It’s false.

When a former public servant of the statistics office of the state of Baden-Württemberg tried to verify its origins, he discovered that Churchill never said anything remotely similar about statistics.

In fact, all evidence suggests the quote was fabricated by Nazi minister Joseph Goebbels and his propaganda ministry.

There are two things to learn from this anecdote.

First, there are other categories of information far less trustworthy than statistics – information, for example, on the historical origins of famous quotes. And second, that the trustworthiness of any information always and everywhere depends on its source.

This has implications on how we deal with official statistics. A statistics agency is building up its
credibility with every set of statistics it publishes. It is vital for published statistics to be reliable and precise – not just in general but all of the time.

But when the reputation of an institution depends on such extreme accuracy, it can lead to too much risk aversion. For fear of not getting issues absolutely right, statistics may not get published or they may only be released after extensive verification and testing so that by the time they are out, they are almost historic.

In the end, it is a matter of balancing precision and quality with timeliness and relevance.

I often wish that statistics authorities could produce a wider variety of indices – going beyond the traditional statistics that are typically collected.

But in doing so, it is difficult to get things right and easy to get them wrong.

How on earth do we adequately measure “housing affordability,” “student achievement” or “infrastructure quality?”

A lot of these issues require value judgments and, as soon as you are forced to make these, you will be subject to criticism.

Ultimately, this could undermine the credibility of the statistics authority as an institution. Unless, of course, you keep the differentiation between value judgments and data completely transparent.

Which is, by the way why I personally have lost trust in official inflation measures, particularly in Europe and in the US. It is odd, to say the least, that the monetary base can be blown up without leaving a trace in official inflation records.

In the end, policymakers need statistics they can trust – if only because they do not even have the time to forge them all themselves.