Predicting the Stock Market Is Easier Said Than Done

Based on research by professors Kerry Back and Kevin Crotty

• Scholars and finance professionals have spent decades trying to develop tools to predict the ups and downs of the stock market.
• Rice Business professors and a PhD student evaluated one of these tools, which has to do with option prices, and documented an important drawback.
• There is still potential to develop better predictive tools, but prediction will always be a risky endeavor. The stock market has always been hard, if not impossible, to forecast.

What do you think the Standard & Poor’s 500 index will do over the next year? 

When Rice Business finance professor Kevin Crotty asks his MBA students this question, the answers are all over the map. Some students expect the overall return on the stock market to be 10%, while others predict a loss of 20%. 

This guessing game is closer to real life than many people realize. Experienced investors, people who have watched the stock market ebb and flow for many years, know that making predictions is a risky business. “Many money managers are more confident choosing individual stocks than trying to time the market,” says finance professor Kevin Crotty. 

For most of the past century, academics have applied their power of analysis to understanding and predicting the stock market. Recently, some finance researchers have taken a closer look at option prices—the price paid for the right to buy or sell a security (like a stock or bond) at a specified price in the future. Combining economic theory with high-frequency options price data, they argued that they could estimate the expected return on the market in real-time, which would represent a tremendous development for finance practitioners and academics alike.

Crotty teamed up with Kerry Back, a fellow Rice Business professor, and Seyed Mohammad Kazempour, a finance Ph.D. student at the Jones Graduate School of Business, to evaluate whether the new predictors based on option prices really are a valuable forecasting tool. “Options are essentially a forward-looking contract, so it’s possible that they could be used to create a forward-looking measure of expected returns,” says Kazempour. 

Economic theory suggests that the new predictors might systematically underestimate expected returns.  The team set out to test if this may be the case, and if so, whether the predictors are useful as a forecasting tool.  In their paper, “Validity, Tightness, and Forecasting Power of Risk Premium Bounds,” the Rice Business researchers ran the predictors through a more rigorous set of statistical tests that provide more power to detect whether the predictors systematically underestimate expected returns.  The statistical tests used in previous research on the topic were less stringent, leading to conclusions that the predictors do not underestimate expected returns.  

In short, the new predictors didn’t pass the more stringent tests. The researchers found that forecasts built on stock options consistently underestimated market returns. Moreover, the predictors are enough of an underestimate that they are not very useful as forecasts of market returns. 

The results were somewhat anticlimatic, the researchers admit. If the option-based predictors had panned out, it could have become an innovative new tool for thinking about market timing for asset managers as well as investment decision-making for corporate finance projects. “Trying to estimate expected market returns is closely related to whether corporations decide to invest in projects,” notes Crotty. “The expected market return is an input in estimating the cost of capital when evaluating projects, and I explain in my MBA courses that we don’t have very precise estimates for this input.  During this research project, I kept thinking about how cool it would be if we really had a better estimate,” he says. 

Their research doesn’t end here. Crotty and Back have already begun brainstorming ways to potentially improve the option-based forecasting tool so that it can become more accurate. 

At best, though, using option prices as a forecasting tool will only be one ingredient out of many that investors use to make decisions. “This tool may inform money management, but it will never drive it,” says Back.

For now, at least, the Rice researchers believe that trying to predict the stock market is still a very risky game.  

Kerry Back is the J. Howard Creekmore Professor of Finance at Jones Graduate School of Business at Rice University. 

Kevin Crotty is an associate professor of finance at Jones Graduate School of Business at Rice University.

Seyed Mohammad Kazempour is a Ph. D. student at Jones Graduate School of Business at Rice University.

To learn more, please see: Back, K., Crotty, K., & Kazempour, S.M. Validity, Tightness, and Forecasting (2020). Power of Risk Premium Bounds. Journal of Financial Economics, vol. 144, issue 3, 732-760.