Tuesday, April 26, 2016

Unskilled and unaware of it

I’ve just re-read the classic study “Unskilled and unaware of it” which established that when we’re incompetent at something we’re often so incompetent that we don’t realise that we’re incompetent. I had forgotten that it starts with a wonderful story about an inept bank robber.

In 1995, McArthur Wheeler walked into two Pittsburgh banks and robbed them in broad daylight, with no visible attempt at disguise. He was arrested later that night, less than an hour after videotapes of him taken from surveillance cameras were broadcast on the 11 o’clock news. When police later showed him the surveillance tapes, Mr. Wheeler stared in incredulity. “But I wore the juice” he mumbled. Apparently, Mr. Wheeler was under the impression that rubbing one’s face with lemon juice rendered it invisible to videotape cameras.

We bring up the unfortunate affairs of Mr. Wheeler to make three points. The first two are noncontroversial. First, in many domains in life, success and satisfaction depend on knowledge, wisdom, or savvy in knowing which rules to follow and which strategies to pursue. This is true not only for committing crimes, but also for many tasks in the social and intellectual domains, such as promoting effective leadership, raising children, constructing a solid logical argument, or designing a rigorous psychological study. Second, people differ widely in the knowledge and strategies they apply in these domains, with varying levels of success. Some of the knowledge and theories that people apply to their actions are sound and meet with favorable results. Others, like the lemon juice hypothesis of McArthur Wheeler, are imperfect at best and wrong-headed, incompetent, or dysfunctional at worst.

Perhaps more controversial is the third point, the one that is the focus of this article. We argue that when people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it. Instead, like Mr. Wheeler, they are left with the mistaken impression that they are doing just fine. As perceptively observed in the quote that opens this article [It is one of the essential features of incompetence that the person so afflicted is incapable of knowing that he is incompetent.  To have such knowledge would already be to remedy a good portion of the offense.] and as Charles Darwin sagely noted over a century ago, “ignorance more frequently begets confidence than does knowledge”.

This effect has since been named the Dunning-Kruger effect after the authors of the study.


Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in
recognizing one's own incompetence lead to inflated self-assessments. J Pers Soc
Psychol. 1999 Dec;77(6):1121-34.

People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

Courtesy of a colleague



  1. Psychologists have shown humans are poor judges of their own abilities, from sense of humour to grammar. Those worst at it are the worst judges of all.

    You’re pretty smart right? Clever, and funny too. Of course you are, just like me. But wouldn’t it be terrible if we were mistaken? Psychologists have shown that we are more likely to be blind to our own failings than perhaps we realise. This could explain why some incompetent people are so annoying, and also inject a healthy dose of humility into our own sense of self-regard...

    Kruger and Dunning were interested in testing another kind of laughing matter. They asked professional comedians to rate 30 jokes for funniness. Then, 65 undergraduates were asked to rate the jokes too, and then ranked according to how well their judgements matched those of the professionals. They were also asked how well they thought they had done compared to the average person.

    As you might expect, most people thought their ability to tell what was funny was above average. The results were, however, most interesting when split according to how well participants performed. Those slightly above average in their ability to rate jokes were highly accurate in their self-assessment, while those who actually did the best tended to think they were only slightly above average. Participants who were least able to judge what was funny (at least according to the professional comics) were also least able to accurately assess their own ability.

    This finding was not a quirk of trying to measure subjective sense of humour. The researchers repeated the experiment, only this time with tests of logical reasoning and grammar. These disciplines have defined answers, and in each case they found the same pattern: those people who performed the worst were also the worst in estimating their own aptitude. In all three studies, those whose performance put them in the lowest quarter massively overestimated their own abilities by rating themselves as above average.

    It didn’t even help the poor performers to be given a benchmark. In a later study, the most incompetent participants still failed to realise they were bottom of the pack even when given feedback on the performance of others.

    Kruger and Dunning’s interpretation is that accurately assessing skill level relies on some of the same core abilities as actually performing that skill, so the least competent suffer a double deficit. Not only are they incompetent, but they lack the mental tools to judge their own incompetence.

    In a key final test, Kruger and Dunning trained a group of poor performers in logical reasoning tasks. This improved participants’ self-assessments, suggesting that ability levels really did influence self-awareness.

    Other research has shown that this “unskilled and unaware of it” effect holds in real-life situations, not just in abstract laboratory tests. For example, hunters who know the least about firearms also have the most inaccurate view of their firearm knowledge, and doctors with the worst patient-interviewing skills are the least likely to recognize their inadequacies.

    What has become known as the Dunning-Kruger effect is an example of what psychologists call metacognition – thinking about thinking. It’s also something that should give us all pause for thought. The effect might just explain the apparently baffling self belief of some of your friends and colleagues. But before you start getting too smug, just remember one thing. As unlikely as you might think it is, you too could be walking around blissfully ignorant of your ignorance.


  2. On one fine morning in Pittsburgh (PA), in the year 1995, a man aged 44, known by the name McArthur Wheeler decided to rob a bank. Since he thought he knew a lot about a peculiar chemical property of lemon juice, he decided to smear the juice on his face before executing his plan to rob the bank.

    His logic – As lemon juice can be used to write invisible letters that become visible only when the letter is held close to a heat source, he thought, the same thing would work on his face too. By smearing lemon juice all over his face, he thought that his face would become invisible to the security cameras at the bank. He did not just think that, he was pretty confident about this. He even checked his “trick” by taking a selfie with a polaroid camera. I’m not sure if the film was defective, or the camera wasn’t operated properly, but the camera did give him a blank image. The blank image made him absolutely sure that this trick would work. Or he would not have ever dared to rob a bank with lemon juice on his face.

    That day, he went on and robbed not one, but two saving banks in Pittsburgh. A few hours after he had done his job, the police got their hands on the surveillance tape and decided to play it on the 11 O’Clock news. An hour later, an informant identified McArthur in the news video and contacted the police with the man’s name. McArthur got arrested on the same day. Ironically, the same surveillance cameras that he was confident would not be able to capture his face, got him behind the bars. During his interaction with the police, he was incredulous on how his ignorance had failed him.


  3. As journalist Michael A. Fuoco reported in the Pittsburgh Post-Gazette, he got the idea from two people he trusted, but he didn’t take their word for it. He performed an experiment to evaluate their claim. He bought a couple of lemons, rubbed the juice on his face, and snapped a Polaroid picture of himself and saw he was nowhere in the image. It was like a version of “Where’s Waldo?” with no Waldo, said one detective. A lawman who nabbed Wheeler offered three possibilities for his successful Polaroid test: the film was bad, he had not adjusted the camera correctly, or he had rotated the camera at the last moment and took a picture of the ceiling...

    There is an unbroken thread of near-comical mistakes criminals make. For instance, there’s the high-school student who vandalized his school by spray-painting his own initials on a wall. A Brit named Peter Addison went further, spray-painting “Peter Addison was here” on the side of a building. Peter Porter, a 66-year-old American lawbreaker, became irate and conspicuous when a supermarket cashier refused to make change when he gave her a one-million-dollar bill. Then there is the criminal-to-be who filled out an employment application at a fast-food restaurant providing his correct name, address and social security number — before he decided a couple of minutes later to rob the place.


  4. Gross incompetence that is not recognized by the incompetent individual has been parodied for years by the Darwin Awards, which honor Charles Darwin’s maxim of the survival of the fittest. Originating around 1985, they were formalized in 1993 by the creation of a website that has become widely known, along with a series of books starting in 2000. As editor Wendy Northcutt describes the criterion for the awards, “In the spirit of Charles Darwin, the Darwin Awards commemorate [posthumously] individuals who protect our gene pool by making the ultimate sacrifice of their own lives. Darwin Award winners eliminate themselves in an extraordinarily idiotic manner, thereby improving our species’ chances of long-term survival.” Some of the recipients include the following:

    2000: An Iraqi terrorist didn’t put enough postage on a letter bomb, and it came back marked “return to sender.” He opened the package and was blown away.

    2002: After being pulled over for erratic driving, a man chose to run from the stolen car while shooting at the police over his shoulder without looking back. He accidentally shot himself in the head.

    2003: During festivities in a park, an Australian man was rushed to the hospital after placing a lit firecracker between the cheeks of his buttocks before stumbling and falling on his behind. The emergency surgeon noted that it looked like “a war injury.”

    2004: After a few drinks, two Taiwanese university students attempted a jousting contest on motor scooters to impress a girl they were both interested in. They collided at 50 miles per hour, both dying on impact. The girl later stated that she was not interested in either man.

    2005: Two muggers in South Africa were fleeing after stealing a cell phone and a purse from a couple at knifepoint when one of them attempted to jump a fence to get away. After his ten-meter drop on the other side, one of them realized he had jumped into the tiger den at the Bloemfontein Zoo. He was fatally mauled.

    2006: An Englishman was found slumped over in his home with stab wounds to his chest. Though police initially thought he was attacked, his wife later told them that she remembered his wondering earlier that week whether or not his new jacket was “stab-proof.” The coroner ruled his death “accident by misadventure.”

    2009: In an attempt to disguise himself for a robbery, a man chose to spray-paint his face gold. He did not realize that the fumes from the paint were toxic and died from inhalation shortly after the burglary.

    2011: During a protest of motorcycle helmet laws, a New York resident was riding his bike without a helmet when he lost control, flipped over its handlebars and died on impact.

    2012: A man was at a friend’s apartment where he saw a salsa jar with an unknown fluid. He presumed it was liquor and took a sip before realizing it was gasoline and spitting it out. Shortly thereafter, he lit a cigarette, setting himself on fire.

    I’ve never liked the Darwin Awards. I find no humor in the death of people who make wrong choices. Moreover, the awards reek of condescension and superiority. Worse, they are a misrepresentation. Nearly all the recipients are adults who were well past puberty and into their reproductive years. Many no doubt had already reproduced before they managed to kill themselves. They had already contributed to the human gene pool before they accidentally removed themselves from it. In order to be consistent with their stated goal, the impresarios of the Darwin Awards should present them to deceased prepubescent, pre-reproductive children. In my view, the awards are mainly an indelicate opportunity to laugh at someone else’s misfortune, while posing as a champion of Darwinian theory.


  5. Kruger and Dunning noted historical observations that point in the same direction, including Confucius: “Real knowledge is to know the extent of one’s ignorance”; Bertrand Russell: “One of the painful things about our time is that those who feel certainty are stupid, and those with any imagination and understanding are filled with doubt and indecision”; Charles Darwin: “Ignorance more frequently begets confidence than does knowledge”; and Shakespeare in As You Like It: “The Foole doth thinke he is wise, but the wiseman knows himself to be a Foole.” Or as my grandmother used to say to my twin brother and me when we little tykes would do something abysmally stupid, “To be so smart, you boys don’t know very much.”


  6. In one such instance, I was invited to lecture on healing research at one of the leading medical schools in the country by a nationally known member of the medical faculty, who had performed a controlled experiment on the effects of distant healing intentions. His faculty colleagues were nervous about these directions, so he asked me to join him in a meeting with the research advisory committee. It was a strange experience. The advisors gingerly voiced their ambivalence toward the research in question. It was clear they were unlikely to approve any further investigations along these lines. I supported the healing researcher by citing foundational experiments in remote healing not just in humans, but also in other organisms including bacteria, animals, and plants. There was nothing new in the research on which they were passing judgment, I said; positive findings had been reported for decades in careful studies. The reaction of the advisors was a mixture of suffocating boredom, non-comprehension, and veiled hostility. Why were we wasting their time on something that “everyone knows” is impossible? No one was prepared to acknowledge the existence, let alone the possible validity, of any prior research that might have affirmed the respectability of the experimental approach in question. This was an incandescent example of the Dunning-Kruger effect. The ignorance of the advisors protected them from an awareness of their ignorance. They were blind to their blindness.

    It’s not just dumb criminals like McArthur Wheeler who are susceptible to the Dunning-Kruger effect. Super-intelligence is no guarantee of immunity to blindness, but may make one even more likely to be blindsided by one’s ignorance. As the eminent psychologist Hans Eysenck observed decades before the Dunning-Kruger effect surfaced, “Scientists, especially when they leave the particular field in which they have specialized, are just as ordinary, pig-headed and unreasonable as anybody else, and their unusually high intelligence only makes their prejudices all the more dangerous….”


  7. I acknowlege that the anosognosia sword cuts in all directions. We are all susceptible to blindness, unable to comprehend our ineptness and irrational beliefs. I often find myself wondering what personal blindness I don’t recognize. Will I ever know? Can I ever know? Still and all, we are not equally ignorant; some of Dunning and Kruger’s university students were better at grammar than others. The argument of equivalent stupidity in the form of the oft-heard rationalization that “everybody does it” fails. This argument says, “If I’m blind, then so are you, so stop criticizing,” as if across-the-board blindness cancels itself out. This ignores the plain fact that both psychological and physical blindness vary in degree. It can be partial or total, temporary or permanent, curable or incurable.


  8. Santa Ignorância
    (Padre Baloneiro)
    (Balloon Priest)
    Confirmed True by Darwin

    (20 April 2008, Atlantic Ocean) A Catholic priest recently ascended to heaven on a host of helium balloons, paying homage to Lawn Chair Larry's aerial adventure. In 1982, Lawnchair Larry attached 45 weather balloons to his lawnchair, packed a picnic lunch, and cut the tether--but instead of drifting above Los Angeles babescape as planned, he was rocketed into LAX air traffic lanes by the hefty lift of the balloons!

    Astoundingly, Larry survived the flight, inspiring the movies Up! and Deckchair Danny, and also inspiring Father Adelir Antonio, 51.

    This priest's audacious attempt to set a world record for clustered balloon flight was intended to publicize his plan to build spiritual rest stops for truckers. But as truckers know, sitting for 19 hours is not a trivial matter even in the comfort of your own lawn chair.

    The priest did take numerous precautions, including wearing a survival suit, flying a buoyant chair, packing a satellite phone and GPS. However, the late A.A. made a fatal mistake -- he did not know how to use the GPS.

    The winds changed, as winds do, and he was blown inexorably toward open sea. He could have parachuted to safety while over land but chose not to. When the voyager was perilously lost at sea, he finally phoned for help--but rescuers were unable to determine his location since he could not use his GPS. He struggled with the unit as the charge on the cellphone dwindled and died.

    Instead of a GPS, the Priest let God be his guide.

    Over the next few weeks, bits of balloons began appearing on mountains and beaches, indicating that God had guided him straight to heaven. Ultimately the priest's body surfaced, confirming that he had indeed paid a visit to his boss.


    Catholic priest Adelir Antonio attempted flight with a chair attached to 1000 balloons. Though he took GPS equipment with him, he did not know how to use it. He reached an altitude of 19,685 feet before the winds changed and he tried phoning for help, asking rescuers how to work his GPS system before he lost signal. Pieces of his body were found three months later.


  9. Another Darwin award: 2010: A wheelchair-bound man in South Korea began ramming his chair into the doors of a closed elevator, upset it departed without him. After three tries, the doors opened but the elevator was no longer there. He went in anyway, plunging to his death.


  10. As Dunning read through the article, a thought washed over him, an epiphany. If Wheeler was too stupid to be a bank robber, perhaps he was also too stupid to know that he was too stupid to be a bank robber — that is, his stupidity protected him from an awareness of his own stupidity...

    Dunning and Kruger argued in their paper, “When people are incompetent in the strategies they adopt to achieve success and satisfaction, they suffer a dual burden: Not only do they reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the ability to realize it. Instead, like Mr. Wheeler, they are left with the erroneous impression they are doing just fine.”...

    And I became very interested in judgments about the self, simply because, well, people tend to say things, whether it be in everyday life or in the lab, that just couldn’t possibly be true. And I became fascinated with that. Not just that people said these positive things about themselves, but they really, really believed them. Which led to my observation: if you’re incompetent, you can’t know you’re incompetent...

    If you knew it, you’d say, “Wait a minute. The decision I just made does not make much sense. I had better go and get some independent advice.” But when you’re incompetent, the skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is. In logical reasoning, in parenting, in management, problem solving, the skills you use to produce the right answer are exactly the same skills you use to evaluate the answer. And so we went on to see if this could possibly be true in many other areas. And to our astonishment, it was very, very true...

    Even if you are just the most honest, impartial person that you could be, you would still have a problem — namely, when your knowledge or expertise is imperfect, you really don’t know it. Left to your own devices, you just don’t know it. We’re not very good at knowing what we don’t know...(continued)

  11. (continued) It’s knowing that there are things you don’t know that you don’t know. Donald Rumsfeld gave this speech about “unknown unknowns.” It goes something like this: “There are things we know we know about terrorism. There are things we know we don’t know. And there are things that are unknown unknowns. We don’t know that we don’t know.” He got a lot of grief for that. And I thought, “That’s the smartest and most modest thing I’ve heard in a year.”...

    Of course, there are known unknowns. I don’t know the melting point of beryllium. And I know that I don’t know it. There are a zillion things I don’t know. And I know that I don’t know them. But what about the unknown unknowns? Are they like a scotoma, a blind spot in our field of vision that we are unaware of? I kept wondering if Rumsfeld’s real problem was with the unknown unknowns; or was it instead some variant of self-deception, thinking that you know something that you don’t know. A problem of hubris, not epistemology...

    The notion of unknown unknowns really does resonate with me, and perhaps the idea would resonate with other people if they knew that it originally came from the world of design and engineering rather than Rumsfeld.

    If I were given carte blanche to write about any topic I could, it would be about how much our ignorance, in general, shapes our lives in ways we do not know about. Put simply, people tend to do what they know and fail to do that which they have no conception of. In that way, ignorance profoundly channels the course we take in life. And unknown unknowns constitute a grand swath of everybody’s field of ignorance.(continued)

  12. (continued)To me, unknown unknowns enter at two different levels. The first is at the level of risk and problem. Many tasks in life contain uncertainties that are known — so-called “known unknowns.” These are potential problems for any venture, but they at least are problems that people can be vigilant about, prepare for, take insurance on, and often head off at the pass. Unknown unknown risks, on the other hand, are problems that people do not know they are vulnerable to.

    Unknown unknowns also exist at the level of solutions. People often come up with answers to problems that are o.k., but are not the best solutions. The reason they don’t come up with those solutions is that they are simply not aware of them. Stefan Fatsis, in his book “Word Freak,” talks about this when comparing everyday Scrabble players to professional ones. As he says: “In a way, the living-room player is lucky . . . He has no idea how miserably he fails with almost every turn, how many possible words or optimal plays slip by unnoticed. The idea of Scrabble greatness doesn’t exist for him.”...

    Unknown unknown solutions haunt the mediocre without their knowledge. The average detective does not realize the clues he or she neglects. The mediocre doctor is not aware of the diagnostic possibilities or treatments never considered. The run-of-the-mill lawyer fails to recognize the winning legal argument that is out there. People fail to reach their potential as professionals, lovers, parents and people simply because they are not aware of the possible. This is one of the reasons I often urge my student advisees to find out who the smart professors are, and to get themselves in front of those professors so they can see what smart looks like.

    So, yes, the idea resonates. I would write more, and there’s probably a lot more to write about, but I haven’t a clue what that all is.I can readily admit that the “everyday Scrabble player” has no idea how incompetent he is, but I don’t think that Scrabble provides an example of the unknown unknowns. An unknown unknown is not something like the word “ctenoid,” a difficult word by most accounts, or any other obscure, difficult word. Surely, the everyday Scrabble player knows that there are words he doesn’t know. Rumsfeld could have known about the gaps in his intelligence information. How are his unknown unknowns different from plain-old-vanilla unknowns? The fact that we don’t know something, or don’t bother to ask questions in an attempt to understand things better, does that constitute anything more than laziness on our part? A symptom of an underlying complacency rather than a confrontation with an unfathomable mystery?

    I found myself still puzzled by the unknown unknowns. Finally, I came up with an explanation. Using the expressions “known unknowns” and “unknown unknowns” is just a fancy — even pretentious — way of talking about questions and answers. A “known unknown” is a known question with an unknown answer. I can ask the question: what is the melting point of beryllium? I may not know the answer, but I can look it up. I can do some research. It may even be a question which no one knows the answer to. With an “unknown unknown,” I don’t even know what questions to ask, let alone how to answer those questions.

    But there is the deeper question. And I believe that Dunning and Kruger’s work speaks to this. Is an “unknown unknown” beyond anything I can imagine? Or am I confusing the “unknown unknowns” with the “unknowable unknowns?” Are we constituted in such a way that there are things we cannot know? Perhaps because we cannot even frame the questions we need to ask?


  13. People will often make the case, “We can’t be that stupid, or we would have been evolutionarily wiped out as a species a long time ago.” I don’t agree. I find myself saying, “Well, no. Gee, all you need to do is be far enough along to be able to get three square meals or to solve the calorie problem long enough so that you can reproduce. And then, that’s it. You don’t need a lot of smarts. You don’t have to do tensor calculus. You don’t have to do quantum physics to be able to survive to the point where you can reproduce.” One could argue that evolution suggests we’re not idiots, but I would say, “Well, no. Evolution just makes sure we’re not blithering idiots. But, we could be idiots in a lot of different ways and still make it through the day.”...

    [Another speaker]And I have an interview with the president of the Alcor Life Extension Foundation, a cryonics organization, on the 6 o’clock news in Riverside, California. One of the executives of the company had frozen his mother’s head for future resuscitation. (It’s called a “neuro,” as opposed to a “full-body” freezing.) The prosecutor claimed that they may not have waited for her to die. In answer to a reporter’s question, the president of the Alcor Life Extension Foundation said, “You know, we’re not stupid . . . ” And then corrected himself almost immediately, “We’re not that stupid that we would do something like that."...

    [Back to David Dunning]And in some sense we apply that to the human race. There’s some comfort in that. We may be stupid, but we’re not that stupid...

    Well, there’s no way we could be evolutionarily prepared for doing physics and doing our taxes at the end of the year. These are rather new in our evolutionary history. But solving social problems, getting along with other people, is something intrinsic to our survival as a species. You’d think we would know where our inabilities lie. But if we believe our data, we’re not necessarily very good at knowing what we’re lousy at with other people...

    Anosognosia is a condition in which a person who suffers from a disability seems unaware of or denies the existence of his or her disability...

    An anosognosic patient who is paralyzed simply does not know that he is paralyzed. If you put a pencil in front of them and ask them to pick up the pencil in front of their left hand they won’t do it. And you ask them why, and they’ll say, “Well, I’m tired,” or “I don’t need a pencil.” They literally aren’t alerted to their own paralysis. There is some monitoring system on the right side of the brain that has been damaged, as well as the damage that’s related to the paralysis on the left side. There is also something similar called “hemispatial neglect.” It has to do with a kind of brain damage where people literally cannot see or they can’t pay attention to one side of their environment. If they’re men, they literally only shave one half of their face. And they’re not aware about the other half. If you put food in front of them, they’ll eat half of what’s on the plate and then complain that there’s too little food. You could think of the Dunning-Kruger Effect as a psychological version of this physiological problem. If you have, for lack of a better term, damage to your expertise or imperfection in your knowledge or skill, you’re left literally not knowing that you have that damage. It was an analogy for us.