The Invincible Markets Hypothesis

There has been a lot of impassioned debate over the efficient markets hypothesis recently, but some of the disagreement has been semantic rather than substantive, based on a failure to distinguish clearly between informational efficiency and allocative efficiency. Roughly speaking, informational efficiency states that active management strategies that seek to identify mispriced securities cannot succeed systematically, and that individuals should therefore adopt passive strategies such as investments in index funds. Allocative efficiency requires more than this, and is satisfied when the price of an asset accurately reflects the (appropriately discounted) stream of earnings that it is expected to yield over the course of its existence. If markets fail to satisfy this latter condition, then resource allocation decisions (such as residential construction or even career choices) that are based on price signals can result in significant economic inefficiencies.

Some of the earliest and most influential work on market efficiency was based on the (often implicit) assumption that informational efficiency implied allocative efficiency. Consider, for instance, the following passage from Eugene Fama's 1965 paper on random walks in stock market prices (emphasis added):

The assumption of the fundamental analysis approach is that at any point in time an individual security has an intrinsic value... which depends on the earning potential of the security. The earning potential of the security depends in turn on such fundamental factors as quality of management, outlook for the industry and the economy, etc...

In an efficient market, competition among the many intelligent participants leads to a situation where, at any point in time, actual prices of individual securities already reflect the effects of information based both on events that have already occurred and on events which, as of now, the market expects to take place in the future. In other words, in an efficient market at any point in time the actual price of a security will be a good estimate of its intrinsic value.

Or consider the opening paragraph of his enormously influential 1970 review of the theory and evidence for market efficiency:

The primary role of the capital market is allocation of ownership of the economy's capital stock. In general terms, the ideal is a market in which prices provide accurate signals for resource allocation: that is, a market in which firms can make production-investment decisions, and investors can choose among the securities that represent ownership of firms' activities under the assumption that security prices at any time “fully reflect” all available information. A market in which prices always “fully reflect” available information is called “efficient.”

The above passage is quoted by Justin Fox, who argues that proponents of the hypothesis have recently been defining efficiency down:

That leaves us with an efficient market hypothesis that merely claims, as John Cochrane puts it, that "nobody can tell where markets are going." This is an okay theory, and one that has held up reasonably well—although there are well-documented exceptions such as the value and momentum effects.

The most effective recent criticisms of the efficient markets hypothesis have not focused on these exceptions or anomalies, which for the most part are quite minor and impermanent. The critics concede that informational efficiency is a reasonable approximation, at least with respect to short-term price forecasts, but deny that prices consistently provide "accurate signals for resource allocation." This is the position taken by Richard Thaler in his recent interview with John Cassidy (h/t Mark Thoma):

I always stress that there are two components to the theory. One, the market price is always right. Two, there is no free lunch: you can’t beat the market without taking on more risk. The no-free-lunch component is still sturdy, and it was in no way shaken by recent events: in fact, it may have been strengthened. Some people thought that they could make a lot of money without taking more risk, and actually they couldn’t. So either you can’t beat the market, or beating the market is very difficult—everybody agrees with that...

The question of whether asset prices get things right is where there is a lot of dispute. Gene [Fama] doesn’t like to talk about that much, but it’s crucial from a policy point of view. We had two enormous bubbles in the last decade, with massive consequences for the allocation of resources.

The same point is made somewhat more tersely by The Economist:

Markets are efficient in the sense that it's hard to make an easy buck off of them, particularly when they're rushing maniacally up the skin of an inflating bubble. But are they efficient in the sense that prices are right? Tens of thousands of empty homes say no.

And again, by Jason Zweig, building on the ideas of Benjamin Graham:

Mr. Graham proposed that the price of every stock consists of two elements. One, "investment value," measures the worth of all the cash a company will generate now and in the future. The other, the "speculative element," is driven by sentiment and emotion: hope and greed and thrill-seeking in bull markets, fear and regret and revulsion in bear markets.

The market is quite efficient at processing the information that determines investment value. But predicting the shifting emotions of tens of millions of people is no easy task. So the speculative element in pricing is prone to huge and rapid swings that can swamp investment value.

Thus, it's important not to draw the wrong conclusions from the market's inefficiency... even after the crazy swings of the past decade, index funds still make the most sense for most investors. The market may be inefficient, but it remains close to invincible.

This passage illustrates very clearly the limited value of informational efficiency when allocative efficiency fails to hold. Prices may indeed contain "all relevant information" but this includes not just beliefs about earnings and discount rates, but also beliefs about "sentiment and emotion." These latter beliefs can change capriciously, and are notoriously difficult to track and predict. Prices therefore send messages that can be terribly garbled, and resource allocation decisions based on these prices can give rise to enormous (and avoidable) waste. Provided that major departures of prices from intrinsic values can be reliably identified, a case could be made for government intervention in affecting either the prices themselves, or at least the responses to the signals that they are sending.

Under these conditions it makes little sense to say that markets are efficient, even if they are essentially unpredictable in the short run. Lorenzo at Thinking Out Aloud suggests a different name:

...like other things in economics, such as rational expectations, EMH needs a better name. It is really something like the "all-information-is-incorporated hypothesis" just as rational expectations is really consistent expectations. If they had more descriptive names, people would not misconstrue them so easily and there would be less argument about them.

But a name that emphasizes informational efficiency is also misleading, because it does not adequately capture the range of non-fundamental information on market psychology that prices reflect. My own preference (following Jason Zweig) would be to simply call it the invincible markets hypothesis.

---
Update (2/16). Mark Thoma has more on the subject, as does Cyril Hédoin. Brad DeLong and Robert Waldmann have also linked here, which gives me an excuse to mention two papers of theirs (both written with Shleifer and Summers, and both published in 1990) that were among the first to try and grapple with the question of how rational arbitrageurs would adjust their behavior in response to the presence of noise traders. In a related article that I have discussed previously on this blog (here and here, for instance), Abreu and Brunnermeier have shown how the difficulty of coordinated attack can result in prolonged departures of prices from fundamentals.

---
Update (2/17). My purpose here was to characterize a hypothesis and not to endorse it. In a comment on the post (and also here), Rob Bennett makes the claim that market timing based on aggregate P/E ratios can be a far more effective strategy than passive investing over long horizons (ten years or more.) I am not in a position to evaluate this claim empirically but it is consistent with Shiller's analysis and I can see how it could be true. Over short horizons, however, attempts at market timing can be utterly disastrous, as I have discussed previously. This is what makes bubbles possible. In fact, I believe that market timing over short horizons is much riskier than it would be if markets satisfied allocative efficiency and the only risk came from changes in fundamentals and one's own valuation errors.

---
Update (2/20). Scott Sumner jumps into the fray, but grossly mischaracterizes my position:

Then there is talk (here and here) of a new type of inefficient markets; Rajiv Sethi calls it the invincible market hypothesis. I don’t buy it, nor do I think the more famous anti-EMH types would either. The claim is that markets are efficient, but they are also so irrational that there is no way for investors to take advantage of that fact. This implies that the gap between actual price and fundamental value doesn’t tend to close over time, but rather follows a sort of random walk, drifting off toward infinity.

This is obviously not what I claimed. There is absolutely no chance of the gap between prices and fundamentals "drifting off toward infinity." All bubbles are followed by crashes or bear markets, and prices do track fundamentals pretty well over long horizons. The problem lies in the fact that attempts to time the market over short horizons can be utterly disastrous, as I have discussed at length in a previous post; this is what makes asset bubbles possible in the first place. I used the term "invincible markets hypothesis" not as a "new type of inefficient markets hypothesis" but rather as a description of the claim that markets satisfy informational (but not allocative) efficiency. And I did not endorse this claim, except as a "reasonable approximation... with respect to short-term price forecasts."

Sumner continues as follows:

Sethi argues Shiller might be right in the long run, but may be wrong in the short run. I don’t buy that distinction. If Shiller’s right then the anti-EMH position has useful investment implications, even for short term investors...

This too is false. I believe that Shiller is right in the short run (since he argues that prices can depart significantly from fundamental values) and also right in the long run (since he believes that in the long run prices track fundamentals quite well). This does not have useful investment implications for short term investors because short run price movements are so unpredictable and taking short positions during a bubble is so risky. Over long horizons, however, Shiller's analysis does suggest that risk-adjusted returns will be greater if the P/E ratio is lower at the time of the initial investment. One could rationalize this with suitable assumptions about time-varying discount rates, but as Thaler points out, such rational choice models are incredibly flexible and lacking in discipline. Everyone acknowledges, however, that for most investors passive investing is far superior to short-term attempts at market timing. The disagreement is about whether prices can deviate significantly from fundamentals from time to time, resulting in severe economic dislocations and inefficiencies.

---
Update (2/24). For a sober assessment of why passive investing remains the best strategy for most investors despite modest violations of informational efficiency, see this post at Pop Economics.

Robert Waldmann's comments at Angry Bear are excellent, but need to be read with some care. His main point is this: there exist certain (standard but restrictive) general equilibrium models in which informational efficiency does imply allocative efficiency. But minor deviations from informational efficiency do not imply that deviations from allocative efficiency will also be minor.

Anomalies in risk adjusted returns on the order of 1% per year can't be detected. We can't be sure of exactly how to adjust for risk. However, they can make the difference between allocative efficiency and gross inefficiency.

For policy makers there is a huge huge difference between "markets are approximately informational efficient" and "markets are informational efficient." The second claim (plus standard false assumptions) implies that markets are allocatively efficient. The [first] implies nothing about allocative efficiency.

In other words, the link between informational efficiency and allocative efficiency is not robust. This is why the market can be hard to beat, and yet generate significant departures of prices from fundamentals from time to time.