Another excellent talk from TED.com about behavioural economics (to add to those already covered on this blog). Dan Pink, a former speech writer for Al Gore, explains how a lot of business practice still relies on extrinsic motivation which is known scientifically to be counter-productive (explained previously on this blog). He echoes Phil Rozenweig’s charge that the business world is lacking in scientific critical thinking, and offers Wikipedia versus Encarta as an example of how intrinsic motivation wins out. It’s not only an engaging talk, but another nail in the coffin of the concept of incentives which is core to rational-choice economics.

Reading today about illusory superiority to improve the Wikipedia article, I came across something tangential but intellectually delightful.

Most people have fewer friends than their friends (on average) have.

When I first read it, it sounded impossible, but it’s a practically inevitable fact.

It’s not specifically about friendship, but a mathematical fact about any relation which is symmetrical and which varies across a population. Consider a set of people such as the students in a college. There are some odd cases we have to ignore, for example:

• nobody knows anybody else (all have 0 friends)
• everybody is part of an insular couple (all have 1 friend each)
• everybody knows everybody else (all have the same large number of friends)

Instead we’re considering realistic situations: each person in the group knows some others but not necessarily all the others. People differ in popularity: some have more friends than average, some fewer. Oh, and it’s essential that friendship is a two-way thing: if A is B’s friend, then B is A’s friend.

In these circumstances, you can imagine counting how many friends someone has, and comparing it to the average of how many friends their friends have. Most of the time, that second number will be bigger.

I’m not sure I follow the rigorous mathematical proof, but you can get a grasp of it intuitively but drawing networks and counting the number of connections each point in the network has. The paper that made me aware of this includes one such diagram (see below).

Another intuition pump: imagine one student comes along who is amazingly popular and instantly makes friends with everybody. That new person (let’s just call him “Martin” ) knows lots of people who are less popular than him, but on the other hand, everybody else now knows someone more popular than they are. This might not be good for everyone else’s happiness if social comparison is important.

Another way to get the point is to imagine that you’re an averagely popular person. Consider the least popular people: the total loners or the insular couples. They have a smaller number of friends than you. As a sample of the population, the set of your friends is unlikely to include these people: it’s biased towards people who have friends. So even though your popularity is the same as the population average, the popularity of your friends, on average, is higher than that.

The irony is that when you actually do the surveys and get people to evaluate their own popularity, they (on the whole) describe themselves as having more friends than their friends have. The friendship paradox shows this is mathematically impossible, and so demonstrates that there’s a self-enhancement effect (an illusory superiority) in people’s estimates of how popular they are. It’s as though we find it hardly psychologically to handle the idea of being less popular than our friends, and the biases kick in to prevent us from acknowledging that.

Reference: Zuckerman, Ezra W.; John T. Jost (2001). “What Makes You Think You’re So Popular? Self Evaluation Maintenance and the Subjective Side of the “Friendship Paradox”“. Social Psychology Quarterly 64 (3): 207–223. doi:10.2307/3090112.

Confirmation bias is the bias to seek for, interpret and remember information in ways that confirm our existing beliefs rather than genuinely test them. In general, it’s an irrational preference for information that matches our expectations. This is one of the first biases I learned about, but recently I’ve been reading up on it in a more systematic way. I’m putting my notes direct into Wikipedia rather than improve my own site.

In what I’ve learned, there’s a massive irony that I’m surprised isn’t commented on. The term “confirmation bias” comes from the original pair of  experiments from the 1970s by Peter C. Wason. Since and because of them, it has become widely accepted that subjects seek to confirm their working hypotheses rather than subject them to falsification.

However, those experiments didn’t prove the existence of a confirmation bias. There were logical errors in the interpretation of the results, pointed out especially in a 1987 Psychological Review paper by Joshua Klayman and Young-Won Ha which is one of my all-time favourite academic papers (see the Wikipedia article for refs). Subsequent research has found genuine confirmation biases, but they’ve turned out to be specific to particular situations, rather than ubiquitous. When testing a hypothesis, people often seem to prefer a genuinely diagnostic strategy.

Despite this critique, there is still a lot of psychological writing that takes the Wason experiments as proving the reality of confirmation bias. Even Sutherland does so in “Irrationality”, his outstanding paperback round-up of bias research.

So why were these experiments accepted so easily as proof when, for a long time, the evidence was inadequate? Because it fit with expectations built up from informal observation: in other words, a clear case of confirmation bias.

Joyful news today on the BBC about a successful campaign for the charity Sense About Science. They asked the World Health Organisation to comment on the use of homeopathic treatment for diseases like HIV, malaria, TB and infant diarrhoea, and various WHO authorities have responded, stating in very clear terms that these conditions need to be treated with actual medicine and actual evidence. SAS have passed on these WHO statements in an open letter to the world’s health ministers (PDF link). Using “remedies” without any active ingredient to “treat” these horrible diseases, when effective alternatives are available, is an obscenity.

It’ll be interesting to see how much popular attention this gets (it was second-most-read on the BBC News site this morning): is the tide finally turning against alternative medicine?

This is a navel-gazing post about an exercise in vanity searching, mainly set down as a not to myself. There: you were warned.

I did some testing using the Wikipedia stats tool to investigate how many people read my contributions.

I assume that people rarely read entire WP articles, but usually skim them. So I can’t include every article I’ve contributed. I’ve looked up daily hits for articles I’ve created:

• Illusory superiority
• Introspection illusion
• Attribute substitution
• The Halo Effect (business book)
• Mistakes were made (but not by me)

and added articles that I totally or substantially rewrote:

• Scientology in the United Kingdom
• Purification Rundown
• Downtown Medical
• Disconnection
• Dianetics: The Modern Science of Mental Health (130 hits per day)
• Clear Body, Clear Mind
• Bare-faced Messiah
• A Piece of Blue Sky
• Cognitive bias (300 hits per day)
• Illusion of control

So far, these total 1000 hits per day. There are three more articles that I’ve made substantial, but not majority, contributions to, and I think it would be difficult even to skim them without reading content that I’ve written, so I feel justified in including them:

• Confirmation bias (600 hits per day)
• Xenu (1300 hits per day)
• Psychokinesis (2200 hits per day)

These take the total over 5000 daily reads.

(Introspection gets about 750 hits per day, but I haven’t included it because my contributions to it aren’t yet substantial enough).

These stats only give  a vague estimate, because:

• Web stats are unreliable for all sorts of technical issues.
• Wikipedia content is copied onto many other sites (and media) because of its open licensing.

but over the course of a year, an appropriate ballpark figure seems to be two million reads. Naturally, these are not hits on my own opinions or research but on my neutral summaries of existing published work. That’s still pretty gratifying to say the least (hence the title of this post). The point of my involvement isn’t and shouldn’t be to get “ratings”, as shown by the fact that it’s taken years to get around to thinking about this. The articles I personally think most important in this list are the ones that get the fewest hits.

Last week I gave a talk about happiness research. Here are some notes for posterity. I haven’t deliberately sought out happiness research, but bias research (my area of interest) overlaps with it a great deal.

First, a disclaimer. When we talk about one group being happier than another, we’re talking about the average of a large number of subjects. All the different life stories that arise from, say, having children, are boiled down to a single figure. I would prefer to see longitudinal studies of happiness displayed as a “heat map” rather than a line on a graph.

Second, a correction. Last week I hadn’t read Bella DePaulo’s Singled Out, which takes a close look at research on the effects of marriage. So when I said that marriage makes people happier by a wide margin, I was unaware of how much this apaprently “common-sense” finding was based on bad research which has been influenced by the “family values” lobby. Some of DePaulo’s findings:

1. The difference is tiny: on a ten-point scale, people expect that marriage will make a difference of four or five points, whereas the real difference between married subjects and subjects who never married is about 0.1.
2. The happiness effect of marriage seems to be temporary: about five years into the marriage, happiness returns to its pre-marriage level.
3. Longitudinal studies suggest that the causality may well be the other way. People who are happy in their marriage seem to be happier before marriage.
4. Note that the standard comparison is between married people and people who have never married. What about people who have divorced? They are less happy than both groups, even years after their divorce.  This crucial bit of data undermines the idea that getting married will make you happy.

So now, as promised, notes from the talk. The main sources were Daniel Gilbert’s excellent book Stumbling on Happiness and various papers and tidbits, for which PsyBlog : Psychology of Happiness is a handy starting point.

1. Human beings are really bad at predicting what will make them happy or sad. We imagine that severe disability would make us very unhappy, and winning the lottery would make us happy, but we ignore how we would adjust to those situations. The DePaulo experiment about perceptions and reality of marriage is an example.

2. The folk misconception about happiness is that it is determined to a large degree by life’s big decisions: where to live, what job to take, whether and whom to marry (see above) and so on. Which option we choose doesn’t matter much for happiness in the long run, but the decisions themselves lower our happiness. There is anxiety as we try to “keep our options open” as long as possible (Ariely writes about this in Predictably Irrational). Once we choose one option and it doesn’t go as well as we hoped, we can torture ourselves with regret that the other option would have been better (even if there’s no reason to think it would have been). Happiness seems to be more about day-to-day experiences than big decisions.

3. One reason we’re so bad at predicting our future mood is focalism. We visualise one change without visualising the whole context, and so we overestimate the effect on our lives of making a change or acquiring something new. Imagine if you had a yacht. Do you think that would that make you happy? When I imagine the yacht, I imagine sitting on the deck, sipping chilled wine while floating in the baking sun of a beautiful Mediterranean island. I don’t imagine cleaning, insuring, refuelling, docking or the other hassles which would affect my happiness if I really did own a yacht.

No doubt there are people who are made happier by owning a yacht, but this example illustrates how we can end up chasing a mirage of happiness because we don’t see the big picture. I suspect that we each have our own mental “yachts”: things, events, places or people that we pursue in the belief that they will “finally” make us happy.

4. Happiness seems to be based very much on comparison rather than objective welfare. It depends on counterfactuals; those stories we tell ourselves about what might have been. It’s vulnerable to framing effects, i.e. we respond differently to a possibility whether it’s described as a gain or as a loss, even if there’s no real difference. Imagine if you went on a TV game show and won £50,000: you’d probably be ecstatic. Now imagine you go on Deal Or No Deal and stay all the way down to two boxes. You choose one and open it: your winnings are £50,000. Then you find the other box – that you rejected – had £250,000. I expect people in this situation feel absolutely gutted, because for them it’s so easy to imagine the counterfactual where they get the bigger prize, so they experience a £200,000 loss rather than a gain.

This suggests that we can raise our happiness by altering our habits of mind. When you think about your job now, do you compare it to your first job, which really sucked, or your ideal job as a helicopter-flying vigilante porn-star? Do you look at someone unfortunate and feel grateful you’re not in the same situation, or do you compare your status to someone you know who’s very successful, nagging yourself that you, rather than they, should be in the porn helicopter?

5. Taleb’s loss-aversion argument from Fooled by Randomness: since we’re more affected by loss than by gain (Tversky and Kahneman’s famous 2.5:1 ratio), and since any complex situation will have ups and downs, it’s not wise to frequently monitor a situation you have no control over (e.g. regularly checking share values will make you more unhappy, even if the shares are generally increasing in value).

6. Talking about an emotion decreases the impact of that emotion. You can use this asymmetrically: when having a good time, don’t analyse why but when you have unpleasant feelings, put them into words. Do this by taking to a close friend or even just writing a diary.

7. Ruminating on an emotion doesn’t decrease its impact: going over an unpleasant event in your mind just means more unpleasantness.

8. The paradoxical role of variety: breaking up a life’s routine seems to make people happier, but Gilbert reports that people sometimes pursue variety for its own sake, rather than just choosing what they like best. In his experiment with meal choices, people with a “free choice” of meal on successive weeks were less happy with their meals that subjects who were restricted to eating their favourite meal.

9. The importance of control, even of minor things:

“[P]eople find it gratifying to exercise control – not just for the futures it buys them, but for the exercise itself. Being effective – changing things, influencing things, making things happen -is one of the fundamental needs with which our brains seem to be naturally endowed, and much of our behavior from infancy onward is simply an expression for this penchant for control.”  (Gilbert (2006), page 20)

10. The non-linear relationship between money and happiness: it sucks to lack money, but once you have “enough”, more money doesn’t seem to make people happier. While material wealth has increased hugely in the developed world over the last half century, happiness has stayed pretty constant. The two biggest determinants of happiness, according to DePaulo, are health and financial security. Note that it’s financial security, not wealth.

11. In case it’s not clear by now, happiness is incredibly hard to investigate scientifically. Our predictions of future happiness are incredibly unreliable. Our memories of how happy we were are distorted by all sorts of biases. All we can go on are present time self-ratings; answers to, “How happy are you now, on a scale from 1 to 10?” But even these answers are sensitive to whatever the person happened to be thinking about at the time. If you ask someone how many dates they have been on in the last six months, it affects the answers they subsequently give to the happiness question.

12. Beware the introspection illusion: you might feel you have a special insight into what makes you happy, because you know your own mind. Remember that although you’re an individual, you’re also a member of a species. It seems paradoxical, but you can learn more about what makes you happy by (non-credulously) observing what makes other people happy. This doesn’t mean I advocate copying other people’s lives.

13. Gilbert reports that over the course of an adult life, happiness takes a U-shaped course. This seems to be a journey from being young and carefree to being loaded with responsibilities (adolescent children especially), with happiness shooting up again as children grow up and leave home, and retirement comes along. Early retirement seems to be the happiest time of some people’s lives.

14. An interesting informal exploration of happiness: a science blog run by the New York Times asked people for ten most expensive purchases, and the ten purchases which had made them most happy.

• Some things that appear on both lists: houses, university education, travel, home entertainment systems
• Happy but not expensive: dinner parties or meals out, books, music, hobbies, pets, alcohol
• Expensive but not happy: marriage ceremonies (sorry!), children (sorry!), boats (including yachts, I presume!)

Update: Vox has some nice graphs of how happiness responds to major life events like marriage, divorce and being laid off.

Finally,  a note on the format of the session. I enjoy giving talks, but I enjoy the questions afterward even more. Having a discussion format like this was a way to get straight to the good bit, so thanks to everyone who turned up. Not much makes me happier than being in a group of people making each other laugh and making each other think!

Review of Carol Tavris and Elliot Aronson (2007) Mistakes Were Made (but Not by Me): Why We Justify Foolish Beliefs, Bad Decisions and Hurtful Acts. ISBN 978-0151010981 (Hardback), 978-1905177219 (Paperback)

For clear, engaging explanations of psychological research, this is one of the best books you can get. Cognitive biases are like optical illusions, distorting our decisions, memories and judgement. This book focuses in particular on self-directed biases: the distortions of memory and explanation that make sure that each of us is the hero, not the villain, or our own life story.

When corrupt police frame innocent people, how do they justify to themselves what they are doing? When a couple divorce, how can two former lovers come to hate each other with such passion? When political or military mistakes lead to thousands of deaths, how do the decision-makers live with themselves? The authors take academic research (on cognitive dissonance, stereotypes, obedience and more) and apply it to a wide spectrum of issues from the White House to Mel Gibson’s racism.

It is eye-opening to read how malleable and unreliable memory is, and how easy it is to create feedback loops of increasing certainty from just a glimmer of evidence. An appalling example is the recovered memory craze of the 80s and 90s, which is discussed at length. The book isn’t entirely downbeat, even though it explains how prosecutions, marriages or therapy sessions can go terribly wrong. It shows how easy it is for good people to hurt others, but that we can avoid these traps with humility and self-questioning. They call science “a form of arrogance control”.

A theme running through the work of these two psychologists is how science can address real problems of human conflict. That warm, humane spirit pervades this book and I think anybody curious about the science or the solutions would benefit from reading it.

Noticing that there wasn’t an article about this concept on Wikipedia, I’ve written the following and donated it to start off an article. The GNU Free Documentation license applies. (Updated 2 June. 20 hours after its creation, the article is the number four hit for its title on Google UK!)

Attribute Substitution is a psychological process thought to underlie a number of cognitive biases and perceptual illusions. It occurs when an individual has to make a judgment (of a target attribute) that is computationally complex, and instead substitutes a more easily calculated heuristic attribute. This substitution is thought of as taking place in the automatic intuitive judgment system, rather than the more self-aware reflective system. This explains why biases are unconscious and persist even when the subject is made aware of them. It also explains why human judgments often fail to show regression toward the mean. Hence, when someone answers a difficult question, they may be answering a related but different question, without realising that a substitution has taken place.

Background

In a 1974 paper, psychologists Amos Tversky and Daniel Kahneman had argued that a broad family of biases (systematic errors in judgment and decision) were explainable in terms of a few heuristics (information-processing shortcuts), including availability and representativeness. In a 2002 revision of the theory, Kahneman and Shane Frederick proposed attribute substitution as a process underlying these and other effects.[1]

In 1975, psychologist Stanley Smith Stevens proposed that the strength of a stimulus (e.g. the brightness of a light, the severity of a crime) is encoded neurally in a way that is independent of modality. This idea was built on by Kahneman and Frederick in arguing that the target attribute and heuristic attribute could be very different in nature.[1]

Conditions

Kahneman and Frederick propose three conditions for attribute substitution:[1]

1. The target attribute is relatively inaccessible.Substitution is not expected to take place in answering factual questions that can be retrieved directly from memory (“What is your birthday?”) or about current experience (“Do you feel thirsty now?)
2. An associated attribute is highly accessible.This might be because it is evaluated automatically in normal perception or because it has been primed. For example, someone who has been thinking about their love life and is then asked how happy they are might substitute how happy they are with their love life rather than other areas.
3. The substitution is not detected and corrected by the reflective system.
   For example, when asked “A bat and a ball together cost $1.10. The bat costs $1 more than the ball. How much does the ball cost?” many subjects incorrectly answer $0.10. An explanation in terms of attribute substitution is that, rather than work out the sum, subjects parse the sum of $1.10 into a large amount and a small amount, which is easy to do. Whether they feel that is the right answer will depend on whether they check the calculation with their reflective system.

Examples

The Beautiful-is-Familiar effect

Psychologist Benoît Monin reports a series of experiments in which subjects, looking at photographs of faces, have to judge whether they have seen those faces before. It is repeatedly found that attractive faces are more likely to be mistakenly labeled as familiar^{. [3]} Monin interprets this result in terms of attribute substitution. The heuristic attribute in this case is a “warm glow”; a positive feeling towards someone that might either be due to their being familiar or being attractive.

Valuing insurance

When subjects are offered insurance against their own death in a terrorist attack while abroad, they are prepared to pay more for it than they would for insurance that covers death of any kind while abroad, even though the latter clearly includes the former. Kahneman suggests that the attribute of fear is being substituted for a calculation of the total risks of travel. Fear of terrorism is stronger than a general fear of dying on a foreign trip.

Stereotypes

Stereotypes can be a source of heuristic attributes. In a face-to-face conversation with a stranger, judging their intelligence is more computationally complex than judging the colour of their skin. So if the subject has a stereotype about the relative intelligence of whites, blacks and Asians, that racial attribute might substitute for the more intangible attribute of intelligence. The pre-conscious, intuitive nature of attribute substitution explains how subjects can be influenced by the stereotype while thinking that they have made an honest, unbiased evaluation of the other person’s intelligence.

In optical illusions

Attribute substitution would also explain the persistence of some illusions. For example, when subjects judge the size of two figures in a perspective picture, their apparent sizes can be distorted by the 3D context, making a convincing optical illusion. The theory states that the three-dimensional size of the figure (which is accessible because it is automatically computed by the visual system) is substituted for its two-dimensional size on the page. Experienced painters and photographers are less susceptible to this illusion, because the two-dimensional size is more accessible to their perception.

Notes

2. Kahneman, Daniel; Shane Frederick (2002). “Representativeness Revisited: Attribute Substitution in Intuitive Judgment”. in Thomas Gilovich, Dale Griffin, Daniel Kahneman. Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge: Cambridge University Press. pp. 49-81.

3. Monin, Benoît; Daniel M. Oppenheimer (2005). “Correlated Averages vs. Averaged Correlations: Demonstrating the Warm Glow Heuristic Beyond Aggregation“. Social Cognition 23 (3): 257-278. ISSN 0278-016X

References

• Daniel Kahneman (December 8, 2002) Maps of Bounded Rationality: A Perspective on Intuitive Judgement and Choice (Nobel Prize lecture)
• Daniel Kahneman (2007) Short Course in Thinking about Thinking (Edge.org)

Behavioural economists have been quick on the uptake in using video lectures to convey their message. Here is a short round-up focusing on quality rather than comprehensiveness.

The best example is Daniel Kahneman’s Nobel prize lecture “Maps of Bounded Rationality” (38 mins). It shows how cognitive biases resemble visual illusions and attacks rational-choice economics on a number of points.

Dan Gilbert, author of the wonderful Stumbling on Happiness, gives a run through some main topics of the book in “Our Mistaken Expectations” (34 mins, TED talk). He argues that we humans are hopeless at affective forecasting because we have trouble estimating both probability and utility. This is a useful introduction to the idea of heuristics.

Gilbert applies applies our poor understanding of probabilities to issues of global disaster in “The Human Brain’s problems with Global Warming” (15 minutes, free to download Creative Commons)

Dan Ariely, author of Predictably Irrational, has a number of videos online. In
“Are we in control of our decisions?” (17 minutes, TED talk) he discusses the effect on decisions of extending the choice set, and addresses the importance of defaults in a way that will be familiar to readers of Nudge. (This has also been posted on Fora.tv as “Tendencies of Irrational Behavior”)

In “Our Buggy Moral Code” (16 mins, TED talk) Ariely talks about his research on cheating, where the cheating behaviour seemed unaffected by the magnitude of the incentive, but depended greatly on priming effects and on whether the subject could get money or tokens that could be exchanged for money. He also has a lecture in the “Authors@Google” series (56 mins, Youtube) which is very similar to the “Are we in control…” video but at greater length.

Richard Thaler’s Google lecture (57 mins, Youtube) outlines the key ideas of his co-written book Nudge, including how the rational agents of conventional economics differ from real human beings.

The BBC television programme Horizon has an online extract showing the Hall and Johansson experiments on choice blindness (1′38″, online streaming) in which subjects confabulate their reasons for making a choice between two alternatives.

Review of Peter A. Ubel (2009) Free Market Madness: Why human nature is at odds with economics  – and why it matters. Harvard Business Press, ISBN:9781422126097

Despite the title, this book sings the praises of the free market. However, it soundly debunks a libertarian free-market fundamentalism that draws its legitimacy from the rational-choice assumptions of economics.

The author is a medical doctor and decision scientist, not to mention an accessible writer. The book is based on many important scientific studies, including the author’s own research, so there’s a high fact-to-opinion ratio. In his medical work, Ubel sees first-hand the obesity crisis, the stressful conditions in which we make medical decisions and the inefficiency of a market medical system. This in turn shows the danger of believing that people always make decisions in their own best interest.

Other writers have criticised free-market economics for ignoring market failures (such as pollution) or for ignoring the moral dimension of decision-making. Ubel instead shows that rational-choice economics has been refuted – no, not by the financial crisis – by decades of behavioural research, in laboratory and real-world experiments on decision making. The fact is that the brain has many decision-making modules. Maybe one of them wants you above all to get fit, eat right and avoid diabetes, but when you are out shopping it’s a different module that chooses to buy the doughnuts.

Human behaviour responds to incentives, but as this book shows, incentives are not the whole story. There are framing effects, comparison effects, social pressure, and the many human quirks which are exploited by marketers to make you buy stuff you don’t really want. To rational-choice economists, incentives are the whole story, hence their theory of “rational addiction”, and Ubel shows what a fallacy this is.

Thaler and Sunstein’s “Nudge” has brought the policy implications of behavioural economics to public awareness. However, that book reflects the political atmosphere of the US where it’s beyond the pale to challenge that the individual knows best. As a result, it reads as oddly right-wing to a European audience. This isn’t the case with “Free Market Madness”. Ubel believes in life, liberty and the pursuit of happiness but doesn’t want us to sacrifice our happiness on the altar of freedom, like so many of his patients have. Ariely’s “Predictably Irrational” is more wide-ranging, but Ubel is more specific about where orthodox economics has gone wrong. All three books are valuable contributions and deserve to be read with each other, with Ubel’s in particular being a serious wake-up call to our political discussion.

Despite its significance, I can’t give this the highest rating because after an excellent first few chapters, the argument becomes more diffuse. He gives examples of how we make decisions irrationally, and how the free-market fundamentalists are ignoring the science, but it’s not structured as tightly as it could have been. Perhaps that’s pedantic: the book is persuasive enough to show me that some of my own beliefs, however appealing, were wrong.