2009 podcast scholtes the art of asking the right question

The art of asking the right question, not the science of giving the right answer

11 February 2009

The article at a glance

Has risk modelling and risk management lost its way? Professor Stefan Scholtes, Dennis Gillings Professor of Health Management at Cambridge Judge Business …

Category: Podcasts Risk

The art of asking the right question, not the science of giving the right answerHas risk modelling and risk management lost its way? Professor Stefan Scholtes, Dennis Gillings Professor of Health Management at Cambridge Judge Business School, argues that contrary to popular belief, the risk management models in use today are not too simple – they are in fact far too complex. The goal of risk modelling, he says, is not and should never be to replicate a complex reality in a complex model. The fundamental flaw in using risk modelling in this way has been clearly revealed by the banking crisis. What we need, he argues, is a complementary balance between modelling and intuition; models that relate to and enforce our mental abilities, not replace them.

[soundcloud id=’36920726′]
“It is hard for us, without being flippant, to even see a scenario within any kind of realm of reason that would see us losing one dollar in any of those transactions,” said the head of AIG’s financial products division in August 2007, as quoted by the New York Times. A year later the company had lost billions on precisely those transactions and had to be bailed out by the US Fed with tax payers’ money to avoid a total collapse of the banking system.

“Risk management systems – and the models at their core – were supposed to guard against outsized losses. How did we go so wrong?” asked Alan Greenspan in the Financial Times in March 2008, and concluded “The essential problem is that our models – both risk models and econometric models – as complex as they have become, are still too simple to capture the full array of governing variables that drive global economic reality.”

I disagree – the models in use are not too simple, they are far too complex. The goal of risk modelling is not and should never be to replicate a complex reality in a complex model. The idea that we can use an engineering approach to risk modelling, the same approach that works to predict the air flow in a jet engine without having to build it, is a fundamental flaw that has been revealed by the banking crisis.

If the engineering approach to risk modelling doesn’t work, then what does? Do we have an alternative? Yes, we do – and the alternative is not to discard modelling and rely on gut-feel when decisions are to be made under significant uncertainty. Psychologists and behavioural economists, building on the Nobel Prize winning work of Kahneman and Tversky, have long established the shortcomings of intuition in decision making. Our intuition uses mental short-cuts which have evolved over millions of years when survival relied on quick reaction in fairly simple and repetitive environments. No wonder that our hard-wired mental heuristics can lead us astray when we are faced with the enormous complexities of today’s business environment.

The alternative to the two prevalent forms of decision-making under uncertainty – relying on gut-feel or pushing the button on a model – is obvious: We must integrate the two. The challenge is to build models that relate to and enforce our mental abilities, not replace them. This requires first and foremost that models are simple – not complex. When we have broken a leg, we use crutches to walk and not the latest electrical wheelchairs. This is not only because the crutches are cheaper but because we need to gain back our ability to walk and crutches force us to train our muscles and movements. This is what good risk models must do – they have to be crutches to our intuition, not wheelchairs.

In the words of an enlightened CEO of a bank that I had the good fortune to work with: A good model helps a manager test and improve their business intuition and communicate this intuition up and down the organisation. Model building becomes the art of asking the right question, not the science of giving the right answer. This is the new mantra for risk modellers. Shall we call it “behavioural modelling”?