Essays Exploring Predictive Regressions (PhD)

AbstractCan stock market returns be forecast, and how would we know? This thesis looks at the predictive regression, the workhorse of the return-predictability literature, from three angles. The first essay asks whether industry portfolios help forecast the market, and finds that the stronger link runs the other way. The second shows that volatility dynamics left out of the model can make predictability appear where there is none. The third shows that testing predictors one at a time produces false discoveries, and that simple variable-selection methods do better.

Keywords: predictive regressions, stock return predictability, industry portfolios, conditional heteroskedasticity, variable selection, forecasting. Funded by the Economic and Social Research Council (ESRC).

The question

Can stock market returns be forecast? It matters well beyond trading: better return forecasts mean better portfolio decisions, better asset pricing models, and better inputs to option pricing and economic modelling. The question is also one of the oldest in finance. Cowles (1933) found that professional forecasters could not beat the market, and Fama (1970) argued that publicly available information should not predict returns.

By the early 2000s the mood had changed, and valuation ratios such as the dividend–price ratio were widely believed to forecast returns. Welch and Goyal (2008) then showed that most of these predictors would not have helped an investor timing the market in real time. Since then, research has looked for predictability that survives: robust econometric methods, economically motivated restrictions, regime changes, forecasting the direction of returns, and forecast combination.

The common tool in all of this work is the predictive regression: next period’s return regressed on a lagged predictor. It looks simple, but it is notoriously hard to get right. Predictors are persistent, their shocks are correlated with returns, and returns are mostly noise. This thesis asks what we can learn with that tool, and where it can mislead.

Industry portfolios and the market

Essay 1. Stock return predictability: what is the role of industry portfolios?

If news about the real economy reaches some sectors before others, as the gradual-diffusion story of Hong and Stein (1999) suggests, industry returns should help forecast the market. The first essay re-examines that idea using out-of-sample tests, longer forecast horizons, variable selection and forecast combination, with persistence-robust inference throughout.

Industry portfolios do carry some information about the market. The surprise is that the relationship is much stronger in the other direction: the market helps forecast industries. Macroeconomic predictors that cannot forecast the market can still forecast many industry portfolios, and industry-specific measures of valuation and volatility help too, often out of sample. One message of the essay is that industry returns may simply be easier to forecast than the market as a whole.

When volatility looks like predictability

Essay 2. Further evidence on the consequences of unmodelled conditional heteroskedasticity

Most robust methods for predictive regressions focus on one well-known problem: the Stambaugh (1999) bias created by persistent predictors. The second essay looks at a different one. When one series helps predict the volatility of another, standard tests of whether it predicts the mean can reject far too often, even when the data are well behaved in every other respect. This builds on earlier warnings by Vilasuso (2001).

Simulation evidence shows that the problem gets worse as the volatility dynamics become richer, and that the usual corrections only partly fix it. Two empirical examples bring the point home. A reported link from oil-market volatility to short-term stock-market volatility weakens once the volatility relationship is modelled jointly, and so does much of the daily evidence that industries lead the market.

Choosing the right predictors

Essay 3. Variable selection in predictive regressions

Researchers have dozens of candidate predictors, yet the literature mostly tests them one at a time. Which variables belong in a forecasting model is usually decided by theory or convention rather than tested. The third essay asks how often this approach gets the answer right.

In simulations where only one of several persistent predictors truly matters, testing each predictor on its own often “discovers” variables that have no real predictive power, and robust univariate methods are not immune. Simple variable-selection procedures, such as information criteria and general-to-specific searches combined with robust inference, recover the true predictor much more often. A real-time application with the Welch–Goyal predictors shows how much the selected model can change over time.

What ties it together

Forecasting the market is hard. At best we can explain a small share of the variation in returns, so small, honest gains are the realistic goal.
Where you look matters. Industry portfolios can be more predictable than the market they make up.
How you test matters as much as what you test. Unmodelled volatility dynamics and one-at-a-time testing can both manufacture predictability that is not there.
Selection is part of the model. Choosing predictors carefully, and re-choosing them in real time, is as important as estimating their effects.

Where the research went next

The industry theme continues in Industry Signals and the Dynamics of Stock Market Predictability (International Journal of Finance & Economics, 2026), which shows how industry information can be recovered once market returns are broken into their components. The focus on robust inference and on when predictors work carries through to Can financial uncertainty forecast aggregate stock market returns? (Financial Markets, Institutions & Instruments, 2024) and Can we forecast better in periods of low uncertainty? (Journal of Empirical Finance, 2023). The 14 technical indicators used in that paper are now updated every month, together with a trend uncertainty index built from them.

Key references

Ang, A., and Bekaert, G. (2007). Stock return predictability: Is it there? Review of Financial Studies, 20(3), 651–707.
Cowles, A. (1933). Can stock market forecasters forecast? Econometrica, 1(3), 309–324.
Fama, E. F. (1970). Efficient capital markets: A review of theory and empirical work. Journal of Finance, 25(2), 383–417.
Hong, H., and Stein, J. C. (1999). A unified theory of underreaction, momentum trading, and overreaction in asset markets. Journal of Finance, 54(6), 2143–2184.
Kostakis, A., Magdalinos, T., and Stamatogiannis, M. P. (2015). Robust econometric inference for stock return predictability. Review of Financial Studies, 28(5), 1506–1553.
Lettau, M., and Ludvigson, S. (2001). Consumption, aggregate wealth, and expected stock returns. Journal of Finance, 56(3), 815–849.
Stambaugh, R. F. (1999). Predictive regressions. Journal of Financial Economics, 54(3), 375–421.
Welch, I., and Goyal, A. (2008). A comprehensive look at the empirical performance of equity premium prediction. Review of Financial Studies, 21(4), 1455–1508.