Review of Carol Tavris and Elliot Aronson (2007) Mistakes Were Made (but Not by Me): Why We Justify Foolish Beliefs, Bad Decisions and Hurtful Acts. ISBN 978-0151010981 (Hardback), 978-1905177219 (Paperback)

For clear, engaging explanations of psychological research, this is one of the best books you can get. Cognitive biases are like optical illusions, distorting our decisions, memories and judgement. This book focuses in particular on self-directed biases: the distortions of memory and explanation that make sure that each of us is the hero, not the villain, or our own life story.

When corrupt police frame innocent people, how do they justify to themselves what they are doing? When a couple divorce, how can two former lovers come to hate each other with such passion? When political or military mistakes lead to thousands of deaths, how do the decision-makers live with themselves? The authors take academic research (on cognitive dissonance, stereotypes, obedience and more) and apply it to a wide spectrum of issues from the White House to Mel Gibson’s racism.

It is eye-opening to read how malleable and unreliable memory is, and how easy it is to create feedback loops of increasing certainty from just a glimmer of evidence. An appalling example is the recovered memory craze of the 80s and 90s, which is discussed at length. The book isn’t entirely downbeat, even though it explains how prosecutions, marriages or therapy sessions can go terribly wrong. It shows how easy it is for good people to hurt others, but that we can avoid these traps with humility and self-questioning. They call science “a form of arrogance control”.

A theme running through the work of these two psychologists is how science can address real problems of human conflict. That warm, humane spirit pervades this book and I think anybody curious about the science or the solutions would benefit from reading it.

Noticing that there wasn’t an article about this concept on Wikipedia, I’ve written the following and donated it to start off an article. The GNU Free Documentation license applies. (Updated 2 June. 20 hours after its creation, the article is the number four hit for its title on Google UK!)

Attribute Substitution is a psychological process thought to underlie a number of cognitive biases and perceptual illusions. It occurs when an individual has to make a judgment (of a target attribute) that is computationally complex, and instead substitutes a more easily calculated heuristic attribute. This substitution is thought of as taking place in the automatic intuitive judgment system, rather than the more self-aware reflective system. This explains why biases are unconscious and persist even when the subject is made aware of them. It also explains why human judgments often fail to show regression toward the mean. Hence, when someone answers a difficult question, they may be answering a related but different question, without realising that a substitution has taken place.

Background

In a 1974 paper, psychologists Amos Tversky and Daniel Kahneman had argued that a broad family of biases (systematic errors in judgment and decision) were explainable in terms of a few heuristics (information-processing shortcuts), including availability and representativeness. In a 2002 revision of the theory, Kahneman and Shane Frederick proposed attribute substitution as a process underlying these and other effects.[1]

In 1975, psychologist Stanley Smith Stevens proposed that the strength of a stimulus (e.g. the brightness of a light, the severity of a crime) is encoded neurally in a way that is independent of modality. This idea was built on by Kahneman and Frederick in arguing that the target attribute and heuristic attribute could be very different in nature.[1]

Conditions

Kahneman and Frederick propose three conditions for attribute substitution:[1]

1. The target attribute is relatively inaccessible.Substitution is not expected to take place in answering factual questions that can be retrieved directly from memory (”What is your birthday?”) or about current experience (”Do you feel thirsty now?)
2. An associated attribute is highly accessible.This might be because it is evaluated automatically in normal perception or because it has been primed. For example, someone who has been thinking about their love life and is then asked how happy they are might substitute how happy they are with their love life rather than other areas.
3. The substitution is not detected and corrected by the reflective system.
   For example, when asked “A bat and a ball together cost $1.10. The bat costs $1 more than the ball. How much does the ball cost?” many subjects incorrectly answer $0.10. An explanation in terms of attribute substitution is that, rather than work out the sum, subjects parse the sum of $1.10 into a large amount and a small amount, which is easy to do. Whether they feel that is the right answer will depend on whether they check the calculation with their reflective system.

Examples

The Beautiful-is-Familiar effect

Psychologist Benoît Monin reports a series of experiments in which subjects, looking at photographs of faces, have to judge whether they have seen those faces before. It is repeatedly found that attractive faces are more likely to be mistakenly labeled as familiar^{. [3]} Monin interprets this result in terms of attribute substitution. The heuristic attribute in this case is a “warm glow”; a positive feeling towards someone that might either be due to their being familiar or being attractive.

Valuing insurance

When subjects are offered insurance against their own death in a terrorist attack while abroad, they are prepared to pay more for it than they would for insurance that covers death of any kind while abroad, even though the latter clearly includes the former. Kahneman suggests that the attribute of fear is being substituted for a calculation of the total risks of travel. Fear of terrorism is stronger than a general fear of dying on a foreign trip.

Stereotypes

Stereotypes can be a source of heuristic attributes. In a face-to-face conversation with a stranger, judging their intelligence is more computationally complex than judging the colour of their skin. So if the subject has a stereotype about the relative intelligence of whites, blacks and Asians, that racial attribute might substitute for the more intangible attribute of intelligence. The pre-conscious, intuitive nature of attribute substitution explains how subjects can be influenced by the stereotype while thinking that they have made an honest, unbiased evaluation of the other person’s intelligence.

In optical illusions

Attribute substitution would also explain the persistence of some illusions. For example, when subjects judge the size of two figures in a perspective picture, their apparent sizes can be distorted by the 3D context, making a convincing optical illusion. The theory states that the three-dimensional size of the figure (which is accessible because it is automatically computed by the visual system) is substituted for its two-dimensional size on the page. Experienced painters and photographers are less susceptible to this illusion, because the two-dimensional size is more accessible to their perception.

Notes

2. Kahneman, Daniel; Shane Frederick (2002). “Representativeness Revisited: Attribute Substitution in Intuitive Judgment”. in Thomas Gilovich, Dale Griffin, Daniel Kahneman. Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge: Cambridge University Press. pp. 49-81.

3. Monin, Benoît; Daniel M. Oppenheimer (2005). “Correlated Averages vs. Averaged Correlations: Demonstrating the Warm Glow Heuristic Beyond Aggregation“. Social Cognition 23 (3): 257-278. ISSN 0278-016X

References

• Daniel Kahneman (December 8, 2002) Maps of Bounded Rationality: A Perspective on Intuitive Judgement and Choice (Nobel Prize lecture)
• Daniel Kahneman (2007) Short Course in Thinking about Thinking (Edge.org)

Behavioural economists have been quick on the uptake in using video lectures to convey their message. Here is a short round-up focusing on quality rather than comprehensiveness.

The best example is Daniel Kahneman’s Nobel prize lecture “Maps of Bounded Rationality” (38 mins). It shows how cognitive biases resemble visual illusions and attacks rational-choice economics on a number of points.

Dan Gilbert, author of the wonderful Stumbling on Happiness, gives a run through some main topics of the book in “Our Mistaken Expectations” (34 mins, TED talk). He argues that we humans are hopeless at affective forecasting because we have trouble estimating both probability and utility. This is a useful introduction to the idea of heuristics.

Gilbert applies applies our poor understanding of probabilities to issues of global disaster in “The Human Brain’s problems with Global Warming” (15 minutes, free to download Creative Commons)

Dan Ariely, author of Predictably Irrational, has a number of videos online. In
“Are we in control of our decisions?” (17 minutes, TED talk) he discusses the effect on decisions of extending the choice set, and addresses the importance of defaults in a way that will be familiar to readers of Nudge. (This has also been posted on Fora.tv as “Tendencies of Irrational Behavior”)

In “Our Buggy Moral Code” (16 mins, TED talk) Ariely talks about his research on cheating, where the cheating behaviour seemed unaffected by the magnitude of the incentive, but depended greatly on priming effects and on whether the subject could get money or tokens that could be exchanged for money. He also has a lecture in the “Authors@Google” series (56 mins, Youtube) which is very similar to the “Are we in control…” video but at greater length.

Richard Thaler’s Google lecture (57 mins, Youtube) outlines the key ideas of his co-written book Nudge, including how the rational agents of conventional economics differ from real human beings.

The BBC television programme Horizon has an online extract showing the Hall and Johansson experiments on choice blindness (1′38″, online streaming) in which subjects confabulate their reasons for making a choice between two alternatives.

Review of Peter A. Ubel (2009) Free Market Madness: Why human nature is at odds with economics  – and why it matters. Harvard Business Press, ISBN:9781422126097

Despite the title, this book sings the praises of the free market. However, it soundly debunks a libertarian free-market fundamentalism that draws its legitimacy from the rational-choice assumptions of economics.

The author is a medical doctor and decision scientist, not to mention an accessible writer. The book is based on many important scientific studies, including the author’s own research, so there’s a high fact-to-opinion ratio. In his medical work, Ubel sees first-hand the obesity crisis, the stressful conditions in which we make medical decisions and the inefficiency of a market medical system. This in turn shows the danger of believing that people always make decisions in their own best interest.

Other writers have criticised free-market economics for ignoring market failures (such as pollution) or for ignoring the moral dimension of decision-making. Ubel instead shows that rational-choice economics has been refuted – no, not by the financial crisis – by decades of behavioural research, in laboratory and real-world experiments on decision making. The fact is that the brain has many decision-making modules. Maybe one of them wants you above all to get fit, eat right and avoid diabetes, but when you are out shopping it’s a different module that chooses to buy the doughnuts.

Human behaviour responds to incentives, but as this book shows, incentives are not the whole story. There are framing effects, comparison effects, social pressure, and the many human quirks which are exploited by marketers to make you buy stuff you don’t really want. To rational-choice economists, incentives are the whole story, hence their theory of “rational addiction”, and Ubel shows what a fallacy this is.

Thaler and Sunstein’s “Nudge” has brought the policy implications of behavioural economics to public awareness. However, that book reflects the political atmosphere of the US where it’s beyond the pale to challenge that the individual knows best. As a result, it reads as oddly right-wing to a European audience. This isn’t the case with “Free Market Madness”. Ubel believes in life, liberty and the pursuit of happiness but doesn’t want us to sacrifice our happiness on the altar of freedom, like so many of his patients have. Ariely’s “Predictably Irrational” is more wide-ranging, but Ubel is more specific about where orthodox economics has gone wrong. All three books are valuable contributions and deserve to be read with each other, with Ubel’s in particular being a serious wake-up call to our political discussion.

Despite its significance, I can’t give this the highest rating because after an excellent first few chapters, the argument becomes more diffuse. He gives examples of how we make decisions irrationally, and how the free-market fundamentalists are ignoring the science, but it’s not structured as tightly as it could have been. Perhaps that’s pedantic: the book is persuasive enough to show me that some of my own beliefs, however appealing, were wrong.

I wasn’t at the London meeting last night, but here is the New Scientist write-up about Simon Singh’s libel case with the British Chiropractic Association (previously featured). It seems that this case is capturing the mood of the nation’s skeptics to an unexpected extent, just as the Atheist Bus campaign and Godless People events succeeded beyond their wildest expectations (though I’m not saying that skeptic is the same as atheist).

ASKE colleagues have asked how to support Singh at a time when he might be about to launch a very expensive appeal.

One thing you can do to help Singh is to buy his book “Trick or Treatment: Alternative Medicine on Trial” (co-authored with Prof. Edzard Ernst), or if you have it already, give it some publicity. I’ve had it for a while and honestly only dipped into it so far, but it seems definitive and a large chunk of it deals with chiropractic. The end section is a set of very short assessments of the evidence relating to each of a long list of alternative practices: extremely helpful material for any skeptic.

It doesn’t help Singh’s legal case as directly as a cash donation, but it gets him some cash, raises the profile of the central arguments this case is about and adds a useful resource to your bookshelf. Reviewing or recommending the book on Amazon is a good idea as well: as expected, there are some alt-med supporters giving it one star reviews to bring its rating down, without really addressing the book’s content.

Along with a lot of other bloggers, I want to repost this quote from an article by Simon Singh:

The British Chiropractic Association claims that their members can help treat children with colic, sleeping and feeding problems, frequent ear infections, asthma and prolonged crying, even though there is not a jot of evidence. This organisation is the respectable face of the chiropractic profession and yet it happily promotes bogus treatments.

Singh, along with Prof. Edzard Ernst, is the author of “Trick or Treatment? : The undeniable facts about alternative medicine”, an excellent and definitive guide to the topic. I trust him on the issue of the status of chiropractic far more than I’d trust a lot of other people, including the British Chiropractic Association, who are suing Singh for libel. This weekend the court made a misguided preliminary ruling that favoured the BCA, causing a chilling effect for those who want to call them out for promoting quack remedies.

Let’s get the word out, help Singh out (he’s a entertaining, informative and bold writer and his books explain fascinating science in an accessible way) and make this a foot-bullet for the BCA.

* by “bogus” I here mean “inauthentic; not genuine; lacking in credible evidence”, your honour.

Update: David Allen Green, AKA Jack of Kent, writes about chilling effect of the case in the current New Scientist.

Update: A great round-up of reaction to the case from the God Knows What blog. Singh will announce his next steps at a meeting tomorrow (Monday 18th).

Review of Karen A. Cerulo (2006) Never Saw It Coming: Cultural Challenges to Envisioning the Worst. ISBN 9780226100333

This starts out so well, but wanders into such dubious and frankly mad territory that I can’t recommend it. The theme of the book is how we find it difficult to define or imagine the worst: in particular, worst outcomes such as business failure or loss of a child. Cerulo, a sociologist, argues that this asymmetry is part of our (Western, and USA in particular) culture.

This blindness to the worst has profound costs. In organisations, we have disasters like the Bay of Pigs, Hurricane Katrina or the 9/11 attacks. The banking crisis which occurred since the book was published may be the best illustration of all. At an individual level, people prepare inadequately for severe illness, death or other misfortune. Culturally, she claims, we resist anything that makes us think about the worst.

After a strong first few chapters, including some interesting material on relationship breakups (”I never imagined she might want a divorce”), Cerulo starts shoehorning the evidence into her idea, in a way that is often painful to watch. Despite the fact that World War I is called the “Great War”, it wasn’t actually great, she tells us. The Oscars, celebrating the best films, get far more attention than the Razzies, celebrating the worst. Radio host Art Bell has been ridiculed for interviewing end-of-the-world nuts, and to Cerulo this is an example of our resistance to imagining the worst. This is a ridiculously one-dimensional analysis of why we find such people annoying or risible. She could have made the reverse point: that despite all the ridicule and lack of evidence, fantasies about UFOs coming to end the world are still culturally embedded. Clearly, we often are culturally obsessed with the worst outcome (terrorism, the Y2K bug etc.), and Cerulo has to interpret this as the exception that proves the rule. Because she deals in interpretation rather than controlled experiment, she can make the evidence come out how she likes.

The book’s many references are useful, and from this material Malcolm Gladwell would have written a rip-roaring bestseller. Tavris and Aronson’s “Mistakes Were Made” is a much better introduction to this fascinating subject.

Via The Lay Scientist: Nauseating tabloid The Daily Mail publishes editions in both the UK and the Republic of Ireland. Like a lot of the UK press, it’s taking part in an irresponsible and anti-scientific campaign to deny people life-saving vaccines (the cervical cancer vaccine in this case), but the Irish edition is campaigning for the vaccine to be reintroduced. Unbelievable!

Humour site News Arse asks which women the Daily Mail wants to die, British or Irish?

The Halo Effect: How managers let themselves be deceived by Phil Rosenzweig. ISBN: 978-1-84739-336-4

Business academic Rosenzweig has written a definitive book about critical thinking in the context of business success. A lot of people claim to understand why businesses succeed or fail, whether in journalism such as Fortune magazine, in bestselling books such as In Search of Excellence or in academia. With admirable clarity, Rosenzweig sets out the scientific failings of these, boiling down the errors to a list of nine “delusions” which infect even some of the most prestigious business research.

For instance, business writers neglect the role of external factors in performance (similar to what Taleb’s Fooled by Randomness says about the financial sector). They mistake correlation for causality. They amass huge corpuses of data without addressing the known biases in that data. The Halo Effect of the title is an example of what is known in psychology as attribute substitution. Researchers want to measure a company’s customer focus (or strategic leadership, commitment to its people, etc.) to correlate against performance (such as profitability). However, customer focus is incredibly hard to measure, so in practice the estimates are based on the company’s profitability. It shouldn’t be surprising that this gives a strong positive correlation, as they are in effect correlating a variable against itself.

The book considers a succession of cases where companies were described as well-led, customer focused and innovative when their share price increased, but as soon as their fortunes changed suddenly “became” complacent, reckless or outdated in the eyes of commentators, or vice versa. It looks in detail at some of the books that claim to offer the secrets of guaranteed business success. Not only does he undermine the arguments of these books with straightforward science, but he shows that lasting business success, in the sense of staying ahead of the market for more than a generation, has never happened. He teaches you to recognise the delusions so you can apply the same critique to other claims.

The book examines some contrasting business science which is more rigorous but makes more modest claims, then concludes by discussing how to manage without pseudoscience. This requires that we acknowledge the role of chance and uncertainty, and stop looking for a list of “secrets” that will make success inevitable.

In a few generations, medical research has come from pseudoscience to some of the most rigorous controlled research, and for that change to happen it was necessary that people be unhappy with the status quo and complain that scientific-ish evidence just isn’t good enough. Rosenzweig has done the business and management field a big favour with this critique by setting out what needs to change for it to mature to a proper science. My reaction after reading The Halo Effect is to wonder when the other shoe will drop: which area of academic literature will next be elegantly demolished by an insider in this way?

The list of delusions:

1. The Halo Effect: see above. In statistical language, the independent variable is actually the dependent variable via an imperfect measure.
2. The Delusion of Correlation and Causality: A company is doing especially well and has high employee satisfaction. Some authors would conclude that employee satisfaction strongly benefits performance, but more likely is that people get satisfaction from knowing they are on the winning team.
3. The Delusion of Single Explanations: Arguments that factor X improves performance by 40% and factor Y improves by another 40%, so do both at once and you’ll get an 80% improvement. The fallacy is that X and Y might be very strongly correlated. E.g. X might improve performance by causing Y.
4. The Delusion of Connecting the Winning Dots: Looking only at successful companies, without comparing them against unsuccessful companies, is a useless way to gather evidence. You may as well conclude that buildings are the secrets of a company’s success, because every high-performing company has at least one building.
5. The Delusion of Rigorous Research: some authors boast of the amount of data that they have collected, as though that in itself made the conclusions of the research valid.
6. The Delusion of Lasting Success: the “secrets of success” books imply that lasting success is achievable, if only managers will follow their recommended approach. Rosenzweig shows that truly lasting success never happens in business (data presented in an appendix)
7. The Delusion of Absolute Performance: your market performance is down to what your competitors do as well as what you do. You can do everything right and yet still fall behind.
8. The Delusion of the Wrong End of the Stick: getting cause the wrong way round. Eg. successful companies have a Corporate Social Responsibility policy. Should we infer that CSR contributes to success, or that profitable companies have money to spend on CSR?
9. The Delusion of Organisational Physics: the idea that business performance is non-chaotically determined by discoverable factors, so that there are rules for success out there if only we can find them.

Update July 2009: I’ve created a Wikpedia page for the book, including some content from my post above, at The Halo Effect (business book).

Hooray for the mighty Wikileaks, preserving for us the New Scientist article “How to spot a hidden religious agenda” that the magazine itself has taken off its web site for legal reasons.

The piece is important and deserves a wider discussion. The “science” shelves of a high street bookshop put outright pseudo-science right alongside popularisations of genuine science. Why Us? by James le Fanu appears to me to be just such a book: misrepresenting the results of biology and psychology to make the reader think some bunk about immaterial essences has now been accepted by serious research. The proper science books mostly have quotes and recommendations from scientific authorities on the cover, which Why Us? conspicuously lacks, quoting instead a Daily Mail columnist.

To someone curious but lacking much of a scientific education, it must be hard to tell which is which. The scientific books and the fakes look the same (Why Us resembles the many new books celebrating Charles Darwin), and the publishers and booksellers are not going to make it any easier for them because they profit no matter which book you buy. If a book explores the origin of the human species or the workings of the brain, but from a pseudo-scientific point of view and using superficially scientific language, the reader might never realise that it’s been made up. Those of us who take an interest in the crackpot literature learn to spot warning signs, and that is what Amanda Gefter was writing about in her New Scientist piece.

To disagree with the article, I don’t just think these pseudoscience books are just a religious phenomenon. Crackpots can be motivated by pure egotism, affronted that some people (i.e. every qualified researcher) should dare to disagree with they feel they know. No wonder that they seek to persuade other people of their beliefs as validation. Many of the big results of science “dethrone” us in some way. They tell us we do not live in the center of the universe, were not created specially from the rest of the natural world, and have deeply unreliable perception, memory and even introspection. What’s more, lots of people seem to have trouble with the idea that they are made of meat. It is no wonder then that there is an economic demand for people who will tell us that science does not say these things.