Saturday, 28 February 2009

Risk models - incredibly precise and dangerously wrong

The following excerpt comes from Why Banks Failed The Stress Test which is a document authored by Andrew Haldane, Executive Director for Financial Stability at the Bank of England and echoes several of my own comments on the danger of the financial models that have been overly relied upon by risk managers.

Risk managers are of course known for their pessimistic streak. Back in August 2007, the Chief Financial Officer of Goldman Sachs, David Viniar, commented to the Financial Times: “We are seeing things that were 25-standard deviation moves, several days in a row” To provide some context, assuming a normal distribution, a 7.26-sigma daily loss would be expected to occur once every 13.7 billion or so years. That is roughly the estimated age of the universe.

A 25-sigma event would be expected to occur once every 6 x 10124 lives of the universe. That is quite a lot of human histories. When I tried to calculate the probability of a 25-sigma event occurring on several successive days, the lights visibly dimmed over London and, in a scene reminiscent of that Little Britain sketch, the computer said “No”. Suffice to say, time is very unlikely to tell whether Mr Viniar’s empirical observation proves correct.

Fortunately, there is a simpler explanation – the model was wrong. Of course, all models are wrong. The only model that is not wrong is reality and reality is not, by definition, a model. But risk management models have during this crisis proved themselves wrong in a more fundamental sense. They failed Keynes’ test – that it is better to be roughly right than precisely wrong. With hindsight, these models were both very precise and very wrong.

No comments:

Post a Comment