Data Science

The Netflix competition will probably go down as the event that gave birth to the Data Science Era. Like all iconic events there was absolutely nothing groundbreaking or new about it, it was just the firs time a few trends came together in a public way: large scale data, a public call for solutions, a prominent relatively recent startup disrupting an ‘evil empire’ kind of industry. And a bunch of money.

And the winner’s solution was never used:

If you followed the Prize competition, you might be wondering what happened with the final Grand Prize ensemble that won the $1M two years later. This is a truly impressive compilation and culmination of years of work, blending hundreds of predictive models to finally cross the finish line. We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment.

To me it makes the whole thing an even better story as a cautionary tale in the differences between academic indulgence and commercial needs.

Perfect is often the enemy of good.

Leave a Reply