Plenty of people, Paul Krugman among them, are pointing out that just like the polls (which, on average, had Obama ahead of Clinton by over 8 points), the prediction markets were plainly wrong in forecasting the outcome of the Democratic New Hampshire primary. They’ve got a point.
These are the daily closing prices on the Clinton and Obama contracts to win the New Hampshire primary from InTrade:
Up until Iowa, they were fairly steady at ~60% for Clinton and ~40% for Obama, but from the 3rd of January onwards, there was a clear movement towards Obama. On the day before the primary, the markets had Obama 97.8% likely to win. On the day, Clinton won with 39.07% of the vote, while Obama received 36.47%. So why did the market get it wrong?
Paul Krugman contends that the prediction markets were just reflecting the polls and talking heads, presumably because that was all the information there was to be had. This naturally raises the question of why they were wrong (e.g. did we just witness the Bradley effect in action?), but that is not the point here. A prediction market, according to the theory, is meant to be superior to the polls in predicting outcomes because it combines information contained in the polls with information from other sources. So perhaps Krugman is right. But if he is, why did the market go so far towards Obama?
My guesses:
- Perhaps Krugman is partially right, but the talking heads provided a positive feedback loop. The polls predicted Obama, which the markets saw. The talking heads saw the polls too (perhaps in more detail) and then spoke about it on television, but added effectively no extra information. The markets saw the talking heads and believed it to be extra information in support of the polls.
- Like any market, the prediction markets are susceptible to bubbles. Perhaps we saw one here in the days between Iowa and New Hampshire.
- A lack of “true” liquidity. There was plenty of nominal liquidity in these markets leading up to and during the counting, but how much of the trading was arbitrage, how much was momentum (i.e. bubble) trading and how much was “true,” changing-belief-based trading? As the counting occurred, I was watching both the leaked figures on the Drudge Report and the movement on InTrade. It seemed that the prediction market was moving steadily towards Clinton, but nowhere near as quickly as one would have expected. For example, at 9:40pm, with 46% of the vote counted, Clinton was leading 49,719 (40%) to 45,383 (36%), from which one would conclude with extremely high confidence that Clinton would win, but the market was still only putting her at 65%.
- Perhaps – and I’m by no means certain of this last point – in order for a prediction market to work perfectly, we also need people to set the size of their position in proportion to their confidence in that prediction. So perhaps there were traders who, looking at Drudge or some other source were extremely confident that Clinton would win from quite early in the counting, but since they did not take large enough positions, they did not move the market. In other words, liquidity requirements for a successful prediction market are not just on the number of trades, but on the volume traded.
Update: Justin Wolfers, a long-time researcher in prediction markets, has an article in the WSJ highlighting how surprising the result was given the market predictions.
We were led to this research by an age-old racetrack puzzle economists call the “favorite-long shot bias“: Horse bettors historically have overbet long shots, and they win less often than their odds suggest. Our research suggests that similar biases hold in political prediction markets.
As such, Sen. Clinton’s comeback is even more stunning, as political underdogs have historically won even less often than suggested by their prediction market odds.
Historical comparisons are already being drawn between the New Hampshire primary and the famous 1948 presidential race in which President Harry S. Truman beat Republican challenger Thomas Dewey, despite the infamous “Dewey Defeats Truman” headline in the Chicago Tribune.
Yet the magnitude of the Clinton surprise is arguably even greater. Indeed, historical research by Professors Paul Rhode of the University of Arizona and Koleman Strumpf of Kansas University has shown that in the Truman-Dewey race, prediction markets had seen hope for President Truman despite his dreadful polling numbers, and he was rated an 11% chance of winning the election by election-eve. Thus, Sen. Clinton’s victory on Tuesday was more surprising than President Truman’s in 1948.
Personally, I seem to be thinking of this the other way around. Assuming that prediction markets are generally better than other forms of forecasting, I find it surprising that they got it so wrong on this occasion. Rather than thinking of the result as the equivalent of a 6-sigma event given the prediction market, I wonder what was different this time that so disturbed the market’s ability to predict?
Update 2: Okay, okay a 3-sigma event 🙂 Justin in an email:
For the polls, this was about a 3-sigma event. For the market, which had Hillary priced at about a 7% chance [JB: Justin is referring to the WSJ market], it is about a 1.7 sigma event. They aren’t that unusual. Indeed, they probably happen about 7% of the time