here is John Cochrane reporting from the NBER Asset Pricing conference:
Robert Novy-Marx presented Testing Strategies Based on Multiple Signals, discussed by Moto Yogo. We’re all familiar with the phenomenon that if you try 10 characteristics and pick the best few to forecast returns, t statistics are biased and performance falls out of sample.
Robert pointed out that if you put those best 3 in a portfolio, they diversify each other, reducing the in-sample variance of the portfolio, and boosting Sharpe ratios and t-statistics even further.
Many “smart beta” funds are doing this, so the fall-off in performance from backtest to real money is relevant beyond academia.
The extent of this bias is impressive. Here is the distribution of t statistics that results when you pick the best three of 20 completely useless signals, and put them in a portfolio. Critical values of 4 and 5 show up routinely in Robert’s calculations.
If you pick the three best of a bunch purely average performing stocks you will have a portfolio that looks amazing but is, in fact, crap. This is going to work for all kinds of variable asset or liability prices, including insurance.
Beware ststiatics, folks.
Not Unreasonable 