The problem with Big Data that all these consultants/proponents are not getting: you cannot separate the statistical problem from the researcher’s incentive (convex payoff, like an option). From the many angry responses, it seems that these big data people seem to be years, many years behind…
http://www.wired.com/opinion/2013/02/big-data-means-big-errors-people/
And speaking of genetics, why haven’t we found much of significance in the dozen or so years since we’ve decoded the human genome?
Well, if I generate by simulation a set of 200 variables — completely random and totally unrelated to each other — with about 1,000 data points for each, then it would be near impossible not to find in it a certain number of “significant” correlations of sorts. But these correlations would be entirely spurious. And while there are techniques to control the cherry-picking such as the Bonferroni adjustment, they don’t catch the culprits — much as regulation didn’t stop insiders from gaming the system. You can’t really police researchers, particularly when they are free agents toying with the large data available on the web.
I am not saying here that there is no information in big data. There is plenty of information. The problem — the central issue — is that the needle comes in an increasingly larger haystack.
via Beware the Big Errors of ‘Big Data’ | Wired Opinion | Wired.com.
In general, the older the technology, not only is it expected to last longer – but the more certainty I can attach to such a statement. Here’s the key principle: I am not saying that all technologies don’t age, only that those technologies that were prone to aging are already dead.
It is precisely because the world is getting more technological, that the old has a huge advantage over the new.
Now let’s take the idea beyond technology for a moment. If there’s something in the culture – say, a practice or a religion that you don’t understand – yet has been done for a long time – don’t call it “irrational.” And: Don’t expect the practice to discontinue.
Some things are opaque to us humans. Those things can only be revealed by time, which understands things we humans are unable to explain. But this method allows us to figure out how time and things work without quite getting inside the complexity of time’s mind. Time is scientifically equivalent to disorder, and things that gain from disorder are what this author calls “antifragile.”
via The Surprising Truth: Technology Is Aging in Reverse | Wired Opinion | Wired.com.