On Prediction Markets and Self-Fulfilling Prophecies

Over on the Freakonomics blog, Ian Ayers writes:

One of the great unresolved questions of predictive analytics is trying to figure out when prediction markets will produce better predictions than good old-fashion mining of historic data. I think that there is fairly good evidence that either approach tends to beat the statistically unaided predictions of traditional experts.

But what is still unknown is whether prediction markets dominate statistical prediction.

In asking the "which is better" question, it is important to distinguish between two very different types of events for which prediction markets currently exist. Some events have a likelihood of occurrence that can safely be assumed to be independent of market predictions: they do not become more or less likely simply because beliefs about their likelihood change. Whether Justice Stevens will be next to depart the bench or snowfall in Central Park will exceed twenty inches this season are examples of such events (contracts on both are currently available on Intrade, and each is estimated to occur with 80% probability according to the price at last trade). Such events may be described as exogneous.

There is an entirely different class of events that may be termed endogenous: their likelihood of occurrence is sensitive to beliefs about this likelihood. Political campaigns, especially for party nominations in major elections, have this character. A candidate who is considered to be a prohibitive favorite will have a major fund-raising advantage, for instance if early donors believe that they will be rewarded with access, perks, or appointments. George W. Bush leveraged an aura of inevitability into a massive financial advantage in the contest for the Republican nomination in 2000, and Hillary Clinton attempted to do the same eight years later. By the same token, a campaign that is perceived to have little chance of success may never get off the ground at all, regardless of the strengths of the candidate in question. Hence managing expectations about the likelihood of success is often a major campaign priority.

Paradoxically, the very same market characteristics that serve to enhance predictive accuracy in the case of exogenous events could undermine accuracy in forecasting endogenous events. Accurate forecasting of exogneous events requires broad participation and high levels of market visibility and liquidity, so that decentralized information can be effectively aggregated. But in the case of endogenous events, the more reliable a market is perceived by the public to be, the greater the incentives to manipulate prices at the margin. The problem is especially severe when there is a positive feedback loop between subjective beliefs and objective probabilities, as in the case of contested elections. The costs of such manipulation are small when compared with the costs of prime time advertising, and the returns can be enormous if the viability of one's campaign (or that of a competitor) is at stake.

In an earlier post I discussed some of these issues in the context of a proposal by Robin Hanson arguing for the development of prediction markets for climate change (Nate Silver was supportive of the idea, while Matt Yglesias was skeptical). Would such markets be dealing with exogenous or endogenous events? At first glance, it might seem that the events are exogenous, as in the case of this season's snowfall. But when forecasting temperatures several decades into the future there is an important feedback loop to be considered. A credible prediction that temperatures will remain stable will have the effect of stalling efforts to curtail greenhouse gas emissions, and this in turn could affect the future course of climate change. Note, however, that in this case the feedback is negative rather than positive: an decrease in the perceived likelihood of warming will result in less aggressive curtailment of emissions, and hence an increase in the objective probability of warming. As a result, any attempt at market manipulation by those who stand to lose from abatement policies will become progressively more expensive as temperatures rise.

To put it another way, when the feedback between subjective beliefs and objective probabilities is positive, successful manipulation of prices can pay for itself by changing beliefs in a manner that becomes self-fulfilling. But when the feedback is negative, manipulation must eventually undermine its own success, since it results in beliefs that are systematically self-falsifying. For this reason I remain (cautiously) optimistic about the prospects for developing accurate prediction markets for climate change.