Imagine that you employ a lot of people, each of whom works on a similar task. This task is not easy, and they each show different levels of productivity. You have a chart in front of you of each employee’s performance.
A question occurs to you. To improve performance, should you use incentives or punishments? The carrot or the stick? To test this hypothesis, you take your worst-performing employees and somehow punish them: withdraw some privileges, dock their pay or whatever. At the same time, employees from the top of the chart are rewarded: a cash bonus, better working conditions or keys to the executive washroom. You wait to see whether this changes anyone’s productivity.
Here are the results you get the next month. Almost all the employees who were punished improve their performance, some of them moving into the top half of the chart. The well-performing employees that you rewarded haven’t maintained their high performance, doing on average no better than the rest. The obvious conclusion is that punishment is more effective that reward, right? This has all kinds of implications about how authority figures can’t afford to be “nice” if they want to be effective, right? You might even say that rewarding an employee for doing well is counter-productive.
This would be a massive fallacy; specifically a failure to understand a phenomenon called Regression to the Mean, which is the source of the “hot hand” fallacy in sports (explained in Thomas Gilovich, R. Vallone and Amos Tversky (1985) “The Hot Hand in Basketball. On the Misperception of Random Sequences.” Cognitive Psychology 17: 295-314). Consider that the variations in performance might be random. If you like, imagine that the task is to throw dice a hundred times and add up the scores. The expected total score is 350, but of course any individual score may well be higher or lower. If someone gets a particularly high score, e.g. 370, their expected score in the next session is still 350, and they are as likely to get a below-average score as above-average.
For regression to the mean to apply, performance need not be totally random like the dice example. So long as there is some degree of randomness, punishments will have some spurious effectiveness and managers who rely on this sort of evaluation will be biased, to an extent, against reward.
An experiment almost exactly like this has been done: Schaffner, Paul E. (1985) “Specious learning about reward and punishment.” Journal of Personality and Social Psychology. Vol 48(6), Jun 1985, 1377-1386. DOI: 10.1037/0022-3514.48.6.1377 (found via Ben Goldacre’s book, Bad Science). In addition to the preference for punishment over reward, subjects had a bias towards thinking that both reward and punishment were effective, when neither were (since the performance was entirely random). This may be an example of Illusion of Control.
Bias and Belief 
#1 by Martin Poulter on January 9, 2009 - 12:36 pm
As a PS, an illustration from Daniel Kahneman’s autobiography where he quotes a flight instructor he was trying to teach about this illusion. “On many occasions I have praised flight cadets for clean execution of some aerobatic maneuver, and in general when they try it again, they do worse. On the other hand, I have often screamed at cadets for bad execution, and in general they do better the next time. So please don’t tell us that reinforcement works and punishment does not, because the opposite is the case.”