The problem with Big Data that all these consultants/proponents are not getting: you cannot separate the statistical problem from the researcher’s incentive (convex payoff, like an option). From the many angry responses, it seems that these big data people seem to be years, many years behind…
http://www.wired.com/opinion/2013/02/big-data-means-big-errors-people/
THE ANTIFRAGILITY of SAYING “F*** YOU” TO FATE
The classical, mostly Stoic, idea is that what matters isn’t the random event itself, but how one responds to it, how one acts when hitting a snag. This was believed by scholars to make people “robust”, that is immune from adversity –since we can control how we respond to events. But the point is, once again, misHarvardified: the classical man was vastly more antifragile than academic & library rats want him to be. He was not withdrawn from the world, but above it. His principal asset was in how much courage and fortitude he put in front of circumstances, how he could say “f*** you” to fate, how he defied reversals of fortune.
If that’s the case, then he is not robust, as academics want him to be, but antifragile, as he wants as much disorder, adversity, and volatility to show off, to say “f*** you” to circumstances. If so, he is long volatility.
The good news is that it takes a certain training. When a certain fellow failed his election bid for the Italian presidency (with an embarassingly low number of votes), as the results of the ballots were being announced, one of the senators was heard telling another: “now watch this man and learn from him how to lose”.
Nassim Taleb est devenu une célébrité avec son opus Black Swans, les cygnes noirs, ces événements rares, mais de plus en plus fréquents, qui sortent du cadre probabiliste classique et qui ont provoqué la crise financière et économique. Ancien trader, il a découvert en son temps que les traders n’avaient pas besoin de comprendre les théories complexes de l’évaluation des options pour les négocier, de même qu’il n’est pas nécessaire d’avoir un diplôme de physique pour monter à bicyclette.
via Le « Soviet Harvard » et la théorie « Antifragile » de Nassim Taleb | Le nouvel Economiste.
The more someone identifies with a profession or an “accomplishment” such as an award, the less human he will be in the classical sense. In virtue ethics, the only “excellence” worth attaining is that of “being human”, with all what it entails honor, courage, service, satisfaction of public & private duties, willingness to face death, etc.; “achievements” are reductions and alienations for lower forms of life.
IN ANCIENT ROME this was a privilege reserved for the patrician class. They were able to engage in professional activities without directly identifying with them: to write books, lead armies, farm land, or transact without being a writer, general, farmer, or merchant, but “a man vir rather than homo who” writes, commands, farms or transacts, as a side activity.
TODAY, as humanity got much, much richer, one would have thought that everyone would have access to the privilege. Instead, I only find it in minimum wage earners who just “make a living” and feel forced to separate their identity from their profession. The higher up in the social ladder, the more people derive their identity from their profession and “achievements”.
And speaking of genetics, why haven’t we found much of significance in the dozen or so years since we’ve decoded the human genome?
Well, if I generate by simulation a set of 200 variables — completely random and totally unrelated to each other — with about 1,000 data points for each, then it would be near impossible not to find in it a certain number of “significant” correlations of sorts. But these correlations would be entirely spurious. And while there are techniques to control the cherry-picking such as the Bonferroni adjustment, they don’t catch the culprits — much as regulation didn’t stop insiders from gaming the system. You can’t really police researchers, particularly when they are free agents toying with the large data available on the web.
I am not saying here that there is no information in big data. There is plenty of information. The problem — the central issue — is that the needle comes in an increasingly larger haystack.
via Beware the Big Errors of ‘Big Data’ | Wired Opinion | Wired.com.