When statistician and predictive analytics expert Nate Silver speaks, people pay attention.
After he successfully predicted the outcome of the presidential election in 49 of 50 states in 2008 and all 50 in 2012, his star has grown almost as fast the unread messages in his e-mail inbox -- more than 99,000 in a recent image he posted to Twitter.
Media outlets crowned Silver the real winner in November, with the triumphant success of his data-based predictive model in the face of critical go-by-the-gut pundits that he had claimed all along weren't much more accurate than a coin-flip.
Silver, an East Lansing, Mich., native, made the same assertion in his latest book, "The Signal and the Noise: Why so Many Predictions Fail – But Some Don't", which quickly became a New York Times bestseller after its September 2012 publication.
In it, he describes a range of forecasters, including enigmatic poker players and the supercomputer-based big-data models used by the National Oceanic and Atmospheric Administration that recently nailed the path of Hurricane Sandy five days before it struck land.
Somewhere along the way, Silver has helped make data cool. Political junkies hailed it, news outlets lauded it and some pundits weren't too happy about it. Yet data is everywhere – IBM says 2.5 quintillion bytes of it are created every day – even if corporations and federal agencies are still in the early stages of making sense of "big data," the massive data sets too large to process via traditional methods.
Hidden in these vast data sets are insights that could help agencies solve big problems, but in an interview with FCW, Silver said the "big data" era will only be successful if the government is willing to evolve with it.
