Nate Silver and Small Data

Colby Cosh writes on prognosticator Nate Silver's move from the New York Times to ESPN

It is amusing to me, and I bet it is pretty amusing to him, that Nate Silver has become so venerated even as the popular understanding of what he does remains so impoverished. ... Silver went to the trouble of writing a whole book about what he does, but it is in the nature of such books to be bought more than they are read and read more than they are understood. The proof is that Silver’s name has become inextricably linked with one of the oppressive buzzwords of 2013: “big data”. 

People know “big data” has something to do with statistics, and, hey, who’s the most famous statistics guy on the planet? I’ll let you in on a bluffer’s secret: what “big data” denotes are massive realtime streams of ever-changing information, such as web traffic or the Twitter “firehose”, that can potentially be bent to commercial purposes using powerful and bleeding-edge computational techniques. Silver has always worked exclusively, at least in public, with what might be called “small data”: sets of a few hundred political polls or ballplayers’ statistical lines. He is living proof that there is money to be made applying 50- or 100-year old statistical nostrums and exploratory techniques to such small data. 

As for why Silver moved from a newspaper to a TV network, why wouldn't an ambitious young media figure want to be with a powerful TV network? What makes you money is getting your name and face recognized, and TV is a lot better for the latter.