Continuing Competence: Unskilled and Unaware of it

One of my favorite research articles (although it made me a bit paranoid about my abilities) is Unskilled and Unaware of It: How difficulties in recognizing one’s own incompetence leads to inflated self-assessments. Authored in 1999 by Cornell university researchers Kruger and Dunning, this Nobel prize winning research compared actual and perceived performance in the areas of humor, logical reasoning, and grammar. Despite the domain studied, the results were similar. We all are inaccurate at self-assessing our actual performance. However, there is hope. If there are gaps in a specific domain, training can help close the perceived and actual reality gap; illustrating the old adage “the more I know the more I realize I know nothing.” This study does acknowledge they are not implying that people are always unaware of their incompetence.

Before I get into the meat of the article, a word about the word competent and its polar opposite incompetent. These terms have taken on a negative meaning because of articles like this – they are binary terms. In reality, there are many of shades of grey to the words competent and incompetent. In this article when I use the word incompetent, it is because I am paraphrasing the authors.

In the testing of humor, actual test scores were grouped into quartiles and plotted against individual’s ratings of perceived ability. For the most part all groups inaccurately perceived their test score. Most individuals overestimated their perceived ability. However, the top performers underestimated their ability.

Kruger and Dunning offered this up:

“First, although perceptions of ability were modestly correlated with actual ability, people tended to overestimate their ability relative to their peers. Second, and most important, those who performed particularly poorly relative to their peers were utterly unaware of this fact. Participants scoring in the bottom quartile on our humor test (Figure 1) not only overestimated their percentile ranking, but they overestimated it by 46 percentile points. To be sure, they had an inkling that they were not as talented in this domain as were participants in the top quartile, as evidenced by the significant correlation between perceived and actual ability. However, that suspicion failed to anticipate the magnitude of their shortcomings.”

The second study focused on logical reasoning (Figure 2). Perceived ability, perceived test scores and actual test scores were compared.

• For those in the bottom quartile, individual’s actual scores were in the 12-point percentile and they grossly overestimated their perceived abilities (68 percentile) and perceived test score (62 percentile).
• The top performers underestimated their perceived abilities (74th percentile) and perceived test scores (68th percentile) relative to their actual scores (86th percentile).
• Both top and bottom performers were inaccurate.

Study 2 replicated the primary results of Study 1 in a different domain. Participants in general overestimated their logical reasoning ability, and it was once again those in the bottom quartile who showed the greatest miscalibration. Participants in the bottom quartile also overestimated the number of test items they had gotten right by nearly 50%.

Researchers argued “Thus far, we have shown that people who lack the knowledge or wisdom to perform well are often unaware of this fact. We attribute this lack of awareness to a deficit in metacognitive skill (the capacity to distinguish accuracy from error). That is, the same incompetence that leads them to make wrong choices also deprives them of the savvy necessary to recognize competence, be it their own or anyone else's.”

A third study looked at whether observing others would assist incompetent individuals to gain insight into their own incompetence by observing the behavior of others. They gave each group their peers tests to grade. The researchers expected that bottom-quartile participants would have more trouble with this metacognitive task than would their top-quartile counterparts. The results showed exactly that. Bottom quartile participants failed to gain insight into their performance after seeing the more competent choices of their peers. Instead they raised their inflated self-estimates.

Top quartile participants reviewed their peers work, realized their peer’s performance was inferior to theirs and raised their own estimates of their own general ranking.

Miscalibration of the incompetent comes from an error about self, whereas miscalibration of the highly competent stems from error about others.

Study 4 looked at the impacts of a training packet on the grading of tests. In bottom performers, training helped the accuracy of test grading. While there were still inaccuracies in self-perceived and actual performance, the miscalibration was smaller. Training improved metacognitive skills in those who received training compared to those who did not.

While all individuals had an inherent distorted cognitive bias in their self–appraisal, it was the bottom quartile who overestimated their ability and the top quartile who underestimated theirs. When peer referencing, the bottom quartile had difficulty with social comparison which is why feedback may not be the best strategy for this bottom quartile group.

Relevance to physiotherapists and continuing competence

• Train and keep on learning. In each area of practice you must cross over a minimum threshold of knowledge, theory, or experience to reach a level of expertise which is deemed competent. Sometimes you won’t know you’ve reached that level of competent until after the fact. Once past that point you look back and say – Wow I never knew how little I knew.
• Learn in areas that you don’t like and do like. Competence is domain specific. There are seven domains to physiotherapist practice. We tend to pursue learning in areas we like and are good at and usually this is something to do with clinical care (i.e., the Physiotherapy Expertise competency domain). Thus, we address our strengths rather than our weakness. Furthermore, we are hard wired to fall in a mind trap of thinking that because I am in good in X, then I must be good in Y, when this may not be the case. It is essential to train in all areas of professional practice especially the ones we don’t like. Generalist training will ensure you meet the minimum threshold in all domains of physiotherapist practice. (I use Coursera to learn about areas I don’t “like” because its free and I can pace the course schedule to my time schedule.)
• Don’t guess – test or assess. All except the top performers overestimated actual vs perceived performance. Test your knowledge, assess your practice against external benchmarks such as standards, checklists and other quality improvement tools. Ask a peer to rate your practice against defined benchmarks. The purpose of assessment is formative with the intent that results inform your practice improvement.
• Find your benchmark. In the DO.LEARN.GROW. Self-Selected Activity, a variety of activities focusing on benchmarking and/or assessing one’s personal skill set, or patient and workplace practices are listed. Any of these activities can count for your self-selected activity. To support you in benchmarking practice, we have included links to practice review products that have been developed or curated such as Patient Care Reflection Tool, Competency Self-Assessment, Hazard Assessment and Management checklist.

Comments? Contact the Continuing Competence Program competence@cpta.ab.ca 780.438.0338, 1.800.291.2782

Page updated: 20/04/2022