Part 3 - Smart Beta, MPT, and Diversification

Some providers of financial products have sought to offer smart beta equity indices based on the mean-variance optimal portfolio construction pioneered by Harry Markowitz (1952).¹ However, this attempt to construct a mean-variance optimal portfolio is somewhat misguided.  The investors’ objectives are typically far more complex than achieving nice Sharpe ratios. For pension funds, what matters is pursuing an investment policy for surplus management that takes into account the desire to minimize funding cost as well as the probability of underfunding. For most investors, the standard asset allocation framework focuses on the beta exposure to various markets, macro risks and asset classes. Decisions are significantly driven by how to access these exposures effectively—capturing the associated premium while potentially earning alpha to boot. A mean-variance optimized equity portfolio is, frankly, of little relevance for addressing the needs of pension fund sponsors and other investors.

Still others have redefined optimal diversification in designing smart beta equity indices. Diversification is generally a poorly defined notion. Some people misuse the inverse Herfindahl score (effective N) to measure diversification. This is often inappropriate as the measure ignores the fact that different stocks have different correlations with each other. The modern multi-factor framework argues that there are only a handful of true economic exposures; raising the effective N of a portfolio does not necessarily improve the portfolio’s true risk exposure diversification. Holding an equally weighted portfolio of 100 technology stocks is not more diversified than holding two stocks from two distinct industries. And no amount of fancy optimization can change the fact that you can never truly have more exposures than what is made available by the ten industries. It is worth noting that the precise definition of diversification, in the investment literature, is reducing risk without reducing expected return—thus the claim that diversification is the investment world’s free lunch. So it is, definitionally, not possible to embark on improving diversification without having strong priors on expected returns for stocks.    

While I disagree with the theoretical foundations and the claims of optimal diversification or mean-variance efficiency associated with many smart beta products, I do believe in smart beta strategies’ reported outperformance against traditional cap-weighted indexes. Moving away from price-weighting is known to automatically benefit from the contrarian rebalancing effect, which exploits the mean-reversion in stock prices and which drives the documented value and small cap premium. Recent research finds that a large variety of nonsensical and whimsical non-price-based portfolio weighting schemes handily outperform cap-weighting—because they unintentionally execute on a contrarian rebalancing strategy to benefit from stock price mean-reversion. 

It is, indeed, accurate to claim that smart beta indices, such as low beta indices, RAFI™ indices, or other, more black-box and exotic varieties are generally more mean-variance efficient than cap-weighting since they either reduce volatility risk without reducing expected returns or improve expected returns without increasing volatility risk. The improvement in portfolio mean variance efficiency does not come from optimization or reinventing finance to produce the notion of “maximal diversification”; it comes simply from the fact that any equity portfolio which allocates risk away from 100% in the market beta to a more balanced distribution across other factors such as value, size, and BAB will have improved its equity premia capture. 

However, I would caution investors against pursuing the elusive dream of achieving optimality—as in paying up for the promise for optimal diversification or optimal mean-variance efficiency. There is a long literature on portfolio optimization which neatly demonstrates that complex optimization generally performs poorly relative to strategies as simple as equal-weighting. In the hands of the less skilled, optimization actually destroys information rather than extracts it and efficiently uses it. This is because standard optimization techniques are not robust to estimation error and as a result concentrate into assets with the largest positive estimation errors. In the hands of experienced investment professionals, the optimizer can only be tamed by a large number of ad hoc constraints, where the resulting portfolio is largely dominated by the user specified constraints rather than the inputs or the optimizing algorithm.

In a way, in smart beta land, the old adage that we should definitely avoid letting perfection stand in the way of good enough is truer than ever.