Back in 1996 Yale economist Robert Shiller wrote:

Price Earnings Ratios as Forecasters of Returns: The theory that the stock market is approximately a random walk does not look right at all: Figure 1... show[s]... the ratio of the real Standard and Poor Index ten years later to the real index today (on the y axis) versus... the ratio of the real Standard and Poor Composite Index for the first year of the ten year interval, divided by a lagged thirty year moving average of real earnings.... If real stock prices were a random walk, they should be unforecastable, and there should really be no relation here between y and x. There certainly appears to be a distinct negative relation here. The January 1996 value for the ratio shown on the horizontal axis is 29.72, shown on the figure with a vertical line. Looking at the diagram, it is hard to come away without a feeling that the market is quite likely to decline substantially in value over the succeeding ten years; it appears that long run investors should stay out of the market for the next decade...

In 1996 Yale economist Robert Shiller looked around, considered the historical record on the performance of the stock market, and concluded that the American stock market was overvalued. Prices on the broad index of the S&P 500 stood at 29 times the average of the past decade's earnings. In the past, whenever price-earnings ratios were high future long-run stock returns would be low. On the basis of econometric regression studies carried out by him and by Harvard's John Campbell, Shiller predicted in 1996 that the S&P 500 would be a bad investment over the next decade. In the decade up to January 2006, he predicted, the real value of the S&P 500 would fall, and even including dividends his estimate of the likely real inflation-adjusted returns to be earned by investors holding the S&P 500 was zero--a far cry below the 6% per year or so real return that we have come to think typical of the American stock market.

Robert Shiller's arguments were convincing. They convinced Alan Greenspan enough so that in December of 1996 he gave his "irrational exuberance" speech to the American Enterprise Institute. They certainly convinced me.

But Robert Shiller's arguments were wrong--at least, wrong ex post. Unless the American stock market collapses before the end of January, the past decade has seen the stock market offer returns a little bit higher than the historical averages--much, much greater than zero. Those who invested and reinvested their money in America's stock market over the past decade have nearly doubled it, even after taking account of inflation.

Why was Shiller wrong? In an arithmetic sense, we can point to three factors, each of which can take roughly one-third the credit for real American stock returns of 6% per year over the past decade rather than zero:

• 2% per year because the acceleration of productivity growth produced by the high-tech revolutions behind the very real "new economy" has made American companies much more productive.
• 2% per year because of shifts in the distribution of income away from labor and toward capital that have boosted corporate profits as a share of production.
• 2% per year because the argument of Glasman and Hassett in Dow 36000 was one-twentieth correct: they argued that increasing risk tolerance on the part of stock market investors would raise long-run price-earnings ratios by 400%; it actually appears that increasing risk tolerance has raised long-run price-earnings ratios by 20% or so.

None of these three factors were obvious as of 1996 (although there were signs of the first and inklings of the third for those smart or lucky enough to read them). As of 1996, betting on Shiller's regression studies was a reasonable thing to do, perhaps an intelligent thing to do--but it was also an overhelmingly risky thing to do, as anybody who followed the portfolio strategy implicit in Shiller's analysis now painfully feels in his wallet or her purse.

Economists muse about just why it is that stock markets around the world are subject to fits of "irrational exuberance" and "excessive pessimism." Why don't rational and informed investors take more steps to bet heavily on fundamentals and against the enthusiasms of the uninformed crowd? The past decade gives us two reasons. First--if we grant that Shiller's regression analyses had correctly identified long-run fundamentals a decade ago--betting on fundamentals for the long term is overwhelmingly risky: lots of good news can happen over a decade, enough to bankrupt an even slightly leveraged bear when stocks look high; and lots of bad news can happen over a decade enough to bankrupt an even slightly leveraged bull when stocks look low. Thus even in extreme situations--like the peak of the dot-com bubble in late 1999 and early 2000--it is very difficult for even those who believe they know what fundamentals are to make large long-run bets on them. And it is even more difficult for those who claim they know what long-run fundamental values are and want to make large long-run contrarian bets to convince others to trust them with their money. As J.P. Morgan said when asked to predict what stocks would do: "They will fluctuate."

Perhaps this is how it should be: if it were easy to pierce the veils of time and ignorance and to assess long-run fundamental values with a high degree of confidence, it would be easy and safe to make large contrarian long-run bets on fundamentals. In this case the smart money would smooth out the enthusiasms--positive and negative--of the overenthusiastic crowd. And stocks would fluctuate less. And there wouldn't be teasing evidence at the edge of statistical significance of large-scale deviations of stock market prices from fundamental values.