Skip to content

From Me to You

This website is a personal outlet for information and opinions that don't fit into normal work routines. I hope you will find something useful here. I plan to contribute fresh material periodically, so please check back again if interested.


2015 April 20


Jarrod Wilcox

April 20, 2015



Robert Shiller’s book Irrational Exuberance  was recently published in its 3rd edition with updated comments and data.  Among many useful insights, it warns us that stock valuations are again at unusual highs.  Recall that he accurately forecast a negative 10 year real stock return in 2000.  He also forecast the housing bubble crash.  Although I greatly admire his work, not wanting to just take his word for it that we might be in another stock bubble, I set out to re-create an expected return for US stocks as a check and perhaps to improve the CAPE ratio a little by partnering it with other sources of information.  A better common expectation for U.S. stock index returns could be used by everyone in their personal asset allocation, and if I am correct, it would help to smooth out bubbles.

Given today’s super low interest rates and the resulting unattractive bond prices, it seems to me the first  task is to estimate the expected return advantage of stocks over cash, not bonds.   But analyzing the potential bond bubble and whether it is desirable result of Federal Reserve action would require going too far afield.  Instead, let’s stick to stocks versus cash.  To keep things a little simpler, we stay with U.S. stocks and investors.  We’ll start by estimating  a foundation long-term excess stock return, and then layer on a ten-year return differential  based on Shiller’s price/smoothed earnings ratio.


Bradford Cornell wrote a very interesting paper relating stock returns to the broad economy, published in 2010 in the Financial Analyst’s Journal.  Cornell notes that very long-term growth rates for business earnings must approximate the rate at which GDP grows.  Picture this economic growth in two main parts: productivity growth resulting in gains in real per capita GDP, and population growth.  Over long periods both these growth trends have been relatively stable, with the U.S. averaging a little less than 1.5% for productivity growth and 1% for population growth.  This gives an estimate for long-term real economic growth of 2.5%.  Cornell is a little more generous, and calls it 3%.  He shows a long history in which the ratio of business earnings to GDP began and ended at approximately the same place.  He then accepts a 2% estimate of dilutions in the profit available to existing shareholders after IPO’s and new firms entering the economy.  That leaves only about 1% as the real return available to existing shareholders from economic growth.  Cornell adds a dividend yield of 3% and gets a total of 4% net of inflation.  Call the average long-term inflation rate 2%, and you arrive at a 6% estimate for very long-term nominal stock returns, before shareholder taxes, assuming that over a long term, private and public businesses grow at the same rate.  Because safe short-term returns have averaged the rate of inflation, returns net of inflation are also returns in excess of cash (market proxies such as T-bills).

A complementary approach to estimating long-term returns was taken by Peter Bernstein, using data closer to home.   In 1997 he published an article in the Financial Analyst Journal that controlled for at least some of the historical fluctuations in stock prices.   He focused on sub-intervals in time in which  the beginning and end point showed practically the same stock valuation multiple, measured as dividend yield or price-earnings ratio.  Averaging the rate of return over the resulting 63 spliced sub-intervals, he found for 1871-1976:3 an average stock return of 5.7% after subtracting inflation.

Let’s take an average of Bernstein’s estimate of 5.7% and Cornwell’s GDP growth-based estimate at 4%, arriving at approximately 4.8% as a possible consensus estimate of the very long-term real return for US stocks, net of inflation or cash returns.

I wish we could objectively add another layer to our long-term return estimate based on factors that operate over several decades rather than the very long-term.  Cornell’s approach is very long term.  More important than changes in dividend yield, which merely involve the form of shareholder rewards, are factors underlying the ratio of business earnings to gross domestic product.  For example, the E/GDP ratio was on a downward trend for the 40 years until the mid 1980’s.  It then reversed direction and has climbed since, so that business owners have been capturing a greater share of the rewards of economic growth at the expense of employees.  Our 4.8% very long-term foundation forecast smoothes out these significant intermediate sub-trends.

Profits as a share of economic growth may remain high, and even grow further, based on continuing increases in business ability to use more competitive labor outside the U.S.  It could also increase if rapid substitution of machine intelligence for labor continues to worsen the bargaining position of employees. Alternatively, E/GDP may be ready to decline, as U.S. labor becomes more globally competitive in consequence of increases in Chinese and Indian wages.  Another reason for decline is that our government may react to increasing economic inequality by diverting net fund flows from business owners toward people who are not business owners.   That is, it can close business “tax loopholes” and impose expensive regulations meant to help the poor and middle class.  It can increase capital gain taxes and dividend taxes on shareholders and increase income taxes on wealthy people, most of whom derive wealth from business ownership.  Wait, it has already started this process!   Though each of these phenomena is clearly in play, I don’t have a basis for forecasting their net result quantitatively.   As they say, further research is needed.

I also wish we could use finance theory here.  Finance theory as we know it began with microeconomics and game theory in the 1940’s with a simple model based on utility curves as a function of wealth whose curvature represented risk aversion.  Later it was found that, under idealistic assumptions, these curves could be aggregated in a fictional representative investor to explain an equity risk premium over cash.  Over time, the simple theory was tortured with facts, particularly since Mehra and Prescott in 1988 could not use it to explain even a 1% risk premium.  In defending itself against attacks, it has evolved to a level of complexity that presents plausible explanations but seems not very useful for quantitative prediction of stock returns.


John Campbell (Harvard) and Robert Shiller (Yale) in 1996 told a  Federal Reserve panel that despite the efficient market hypothesis,  there was material predictability in market index stock returns over longer periods.  They showed it based on both dividend yield and a price-earnings ratio that smoothed cyclical fluctuations in earnings.  The latter matched current price adjusted for inflation against a ten-year average of prior inflation-adjusted earnings,.  This ratio, later dubbed the CAPE ratio, then forecast the average stock return outcome for the subsequent ten years.  There were big econometric problems caused by overlapping observations, but the authors of the study used Monte Carlo simulation to demonstrate that the effect was real.

Shiller’s book Irrational Exuberance was first published in 2000.  At that time, the CAPE ratio was over 40, higher than the peak before the 1929 crash.  His book came out just before the collapse of what we now call the Internet bubble.  This somewhat lucky coincidence made his reputation among investors, but behind it was very careful work.  His ratio’s 10 year implied forecast  (from his in-sample model) turned out to be a little too pessimistic, but in fact the realized annualized real return of the S&P 500 index for the subsequent 10 years was negative after netting for inflation.

Go back and look at chart of the CAPE history shown earlier.  The CAPE ratio, which Shiller makes available on his website ( , as of April 7, 2015  stood at a fairly high reading of about 27, the same as in April of 2007.  Though this is not as high as when it predicted actually negative returns in 2000, it is enough to ring alarm bells for Shiller.

Let’s see what could be recreated using total return data for the Standard & Poors 500 index (symbol ^GSPC on Yahoo Finance’s website) rather than Shiller’s somewhat massaged return data.  I used returns from 1950 forward through the first quarter of 2015, roughly the second half of the chart above.

A linear regression estimates an average reduction of 0.36% in annualized return over the 10 years subsequent to each 1 point higher in the level of the CAPE ratio.  Consequently, a current CAPE of 27 compared to a CAPE average of 18 in my sample would imply, if we believe the relationship,  a  9 times 0.36% reduction from whatever we use as our very long-term forecast.  (Note that a confidence interval requires additional work because of the hazard of overlapping variables.  We could approach it, though, by running 120 regressions over the 120 available non-overlapping observation sequences with six observations each.  Then we could look at the standard deviation of the 120 resulting slopes.)

Rather than use the average  return in the sample as a foundation, I take the 4.8% very long term estimate as a foundation from which to calculate a CAPE differential effect of about 3.2%.  This gives 4.8%- 3.2%, or a very low 1.6% real return.

This is a surprising and fairly grim outlook, but it seems reasonable if we cannot predict E/GDP deviations and if we have confidence in the modeling process.  We can investigate the latter and see if it can be improved.

The following chart shows a  scatterplot of the more than 600 overlapping observations of ten-year return for the S&P 500 index versus the preceding CAPE.


The chart shows observations plotted on a logarithmic axis for the CAPE ratio.  This gives a very slightly better relationship overall, and it is visually more linear, but the results were so close that for simplicity of interpretation the ordinary  non-logarithmic  relationship was used for the slope estimate of -0.36%.  The stringy structure obvious from the scatterplot is mostly an artifact of the fact that the ten-year intervals were highly overlapping.  The appearance of different slopes in different decades seems to be a mix of artifacts and some real differences.  For example, it is obvious that something different was going on in the 1990’s (yellow points on the chart), as returns stayed high despite the preceding increasing CAPE.  In the other 5 out of 6 decades, changes in Shiller’s CAPE were strongly negatively correlated with 10-year subsequent returns.  It looks like the 1990’s apparent anomaly may have been affected by delays caused by a long-lasting speculative bubble forming, so it could be argued that the effect was there but merely delayed.

Still concerned about overlapping observations, I plotted the correlation of CAPE ratios with each of the 120 component monthly returns in the next 10 years.  As the following chart shows, it is persistently negative all the way through the 10-year period and shows signs of usefulness for even longer periods.


Now let’s come back to the fact that we are looking at an in-sample model that implicitly assumes that the underlying probability distribution is stable and can be extended out of sample in practice.  As a direction for further improvement in our use of CAPE, consider rolling sub-sample models with no look-ahead to the end of the available sample.  The result in such cases is a tendency to reduce the magnitude of the strong negative slope we found earlier because in real economic life the landscape keeps changing, even if slowly.  Then instead of a final real return estimate of 1.6%, we might find the answer to be 2.5 or 3% because the estimated CAPE effect on subsequent returns would be shrunk.  That is, make a model on a warm-up sample, then forecast one month ahead, then expand the sample by one month and make a revised model.  Repeat until the test sample not included in the initial warm-up is exhausted.  Then compare the residuals of these more realistic no-look-ahead forecasts to the original return variation in the test sample.  As payback for my suffering long ago as a student at MIT, I will leave that exercise for the reader.


Shiller’s CAPE ratio as it stands does not take any account of changes in fundamental variables that might influence it.  Even if these changes are not predominant over very long time spans, they are worth reflecting.  For example, changes in company profitability measured as return on equity, and changes in inflation, and sometimes real interest rates (when the Federal Reserve is not too intrusive)  could be brought to bear.  In the following chart, return on equity (ROE) is calculated from aggregating  earnings and book values within an evergreen list of the 500 largest companies as ranked by book value among companies with calendar year-ends, taken from ValueLine’s historical database.


The 1990’s were a period of unusual behavior in the earlier scatterplot of returns versus CAPE signals. As an explanation, note in the chart immediately above that the gap between ROE and interest rates and inflation was rapidly widening during the 1990s up to 1999.  That is, inflation and interest rates were declining while profitability was maintained.  Is it surprising that stock valuation multiples increased?  Given the momentum investing tendencies of many investors, that valid source of increased valuation eventually sparked a full-blown bubble on top of it.  It took years before an erosion in profitability combined with an increase in inflation rates to let the air out.  One might give more weight to that part of CAPE which is a speculative residual of the “temporarily justified” CAPE that would be expected given fundamental input ingredients.


We began with a very long term excess return prediction of 4.8% from averaging Cornell and Bernstein to use as a foundation for applying CAPE.  The latter currently indicates a reduction of 3.2% using my reconstruction of the Shiller model since 1950, giving only a 1.6% excess return over cash and inflation.  We looked at the structure of CAPE in more detail and found that in-sample, at least, there was good evidence that transcended the technical issue of overlapping observations.  We might have gotten the slope wrong, but there is no denying a negative relationship.

I noted methods of improving the use of CAPE further that might soften the disappointment in the implications of high CAPE.  One method was moving from modeling based on a full-sample, which assumes an unchanging process, to using rolling no-look-ahead forecasts to characterize the distribution of likely errors in the future.  A second was modeling the effects on CAPE based on fundamental ingredients, so that pure speculative bubbles could be treated differently from cyclical changes in profits versus inflation and interest rates. This second step might help to increase expectations for the next year or two, although they probably would not change the 10-year outlook materially.

In order to improve on these readily achievable steps, I especially noted the need for researching the extent to which business owners capture of the fruits of economic growth that must be split with employees and the government (and foreign investors, if we want to split hairs).  We need to get at causality rather than just extrapolate either the present or the trends of the last 30 years.  Innovation (biotech?)and rapid cost-cutting (machine intelligence and new labor sources in emerging markets) will increase ownership share,  while government reaction to U.S. income inequality, and an increasingly competitive U.S. labor force because of wage increases in China and India, may reduce it.  These factors need to be quantified objectively if we are to use them as part of our common estimate.  Otherwise, we each have our own subjective impressions.

Nevertheless, what we have so far is useful.  It doesn’t mean we have to leave the party.  But it does strongly suggest we should be very careful of what we drink.