Suspension of Belief
In an earlier post I observed that if Joe Biden were to step aside as the presumptive nominee of the Democratic party, a number of statistical models built to forecast the election outcome would have to be taken offline and be substantially modified. Their predictions, having already lost reliability, would then also lose continuity.
This has now happened. The following message currently appears at FiveThirtyEight:
Similar notices were posted by the Economist and Silver Bulletin on July 21:
So prediction markets are the only high-frequency forecasting mechanism available to us for the time being.
It is worth looking at how market prices have evolved over the past few days, and thinking about whether these movements will be reflected in model forecasts once the latter are back online.
One rather clear shift has been in the likelihood of a victory by the former president:
The price of the Trump contract on PredictIt peaked at 69 on July 15, and has fallen to 59 since then, coinciding almost exactly with a sharp rise in the Harris contract.
What might account for this? There is not much reliable polling data as yet so I suspect that this is largely due to other factors, many of them qualitative.
One wild card is the choice of nominee for vice-president, where Josh Shapiro and Mark Kelly appear to be front-runners:
Either of these candidates could strengthen Democratic prospects in key swing states.
Beliefs may also be responding to a record fundraising haul, with $81 million in new donations to the Harris campaign within the first twenty-four hours of launch, involving more than a half million new donors. Perhaps the surprising amount of party unity behind the Harris candidacy is also playing a role.
Statistical models can accommodate fundraising totals and other metrics, but have a difficult time with more qualitative factors until they eventually make themselves felt in polling data.1
To see how model forecasts might look like when they return, consider what they were telling us prior to suspension. The Economist was the most bullish on a Trump victory, assigning a likelihood of 83 percent to the event. Silver Bulletin was not far behind, at 73 percent. FiveThirtyEight, which currently places more weight on fundamentals and less on polls, saw the race as a toss-up:
One thing I will be looking for when these models come back online is whether or not their updated forecasts reflect what we are seeing in markets. They may not do so right away—it might take time for opinion polls to reflect the effects of a vice-presidential choice for example.
In the meantime, should we be taking models more seriously or markets?
One way to think about whether models or markets are more reliable forecasting mechanisms is to the evaluate the performance of a virtual trader that acts as if it believes the model and buys and sells contracts based on what is sees as lucrative opportunities. This is the profitability test that I mentioned in an earlier post, and that is described and implemented a paper that was published just a few days ago.2
It will be interesting to see what the profitability test tells us when applied to the various models and markets once the major party nominees have been finalized. I suspect that markets at the moment are well ahead of the models, so the virtual traders would make a loss. But as we get closer to the election, it is entirely possible that market overreaction to various events could create profit opportunities for model-based trading.
We shall see. Either way, it will be difficult to draw general conclusions about the strengths and weaknesses of various forecasting mechanisms based on this bizarre and confounding election cycle.
Andrew Gelman has a number of very informative posts on modeling methodologies and the kind of adjustments that might be needed in response to a change in a major party nominee.
I am currently in Bogotá for the 2024 Complexity Global School, and will be discussing the paper and prediction markets more generally with participants this week. The school is a collaborative effort involving the Santa Fe Institute and the Universidad de los Andes.