I used to attend a health club in the middle of the day and chat with an interesting Eastern European fellow with two Ph.D. degrees, one in physics (statistical no less), the other in finance. He worked for a trading house and was obsessed with the anecdotal aspects of the markets. He once asked me doggedly what I thought the stock market would do that day. Clearly I gave him a social answer of the kind “I don’t know, perhaps lower”-quite possibly the opposite answer to what I would have given him had he asked me an hour earlier. The next day he showed great alarm upon seeing me. He went on and on discussing my credibility and wondering how I could be so wrong in my “predictions,” since the market went up subsequently. Now, if I went to the phone and called him and disguised my voice and said, “Hello, this is Doktorr Talebski from the Academy of Lodz and I have an interrresting prrroblem,” then presented the issue as a statistical puzzle, he would laugh at me. “Doktorr Talevski, did you get your degree in a fortune cookie?” Why is it so?
Clearly there are two problems. First, the quant did not use his statistical brain when making the inference, but a different one. Second, he made the mistake of overstating the importance of small samples (in this case just one single observation, the worst possible inferential mistake a person can make). Mathematicians tend to make egregious mathematical mistakes outside of their theoretical habitat. When Tversky and Kahneman sampled mathematical psychologists, some of whom were authors of statistical textbooks, they were puzzled by their errors. “Respondents put too much confidence in the result of small samples and their statistical judgments showed little sensitivity to sample size.” The puzzling aspect is that not only should they have known better, “they did know better.” And yet…
Taleb, Nassim Nicholas. The Black Swan: The Impact of the Highly Improbable. Random House, 2007. p. 194-5 (italics in original)
