Continuing Competence Theory: Impact of Self and Peer Assessment on Communication and Clinical Recordkeeping

Last month’s article looked at a pilot study examining the feasibility of its proposed quality improvement program (Quality in Motion). This month’s article highlights a follow-up study by KNGF (the Royal Dutch Society of Physical Therapy) that took a deeper dive into exploring the impacts of self and peer assessment and feedback of clinical performance (client communication and record-keeping) in a broader group of physiotherapists.  

Alberta physiotherapists interested in quality improvement initiatives, improving communication skills or record-keeping practices, and using feedback from self and peer assessments to inform practice improvement should find this study interesting.

The main questions

• How do physical therapists perceive interventions, based on performance feedback, aiming to advance the quality of physical therapy care?
• What is the impact of interventions, based on performance feedback, on learning and professional behavior change?

The study

The evaluation of physiotherapists’ clinical performance consisted of two cycles of self and peer assessment of:

1. Client communication: demonstrated through a video recording of a physiotherapist explaining to their patient the diagnosis, setting goals and outcome and establishing the treatment plan
2. Record-keeping: demonstrated through a clinical chart self-selected by the physiotherapist for assessment

The physiotherapist completed the self-assessment first, followed by the other four to six physiotherapists in their peer group completing assessments of the same items. The criteria they were rated on included domains of: 

• Client-centeredness
• Effectiveness including evidence-based practice
• Transparency

Once the assessments were complete the group met to discuss the results with a facilitator guiding the discussion. This discussion provided an opportunity for peers and physiotherapists to seek clarity on the quality indicators and explain their narrative comments and ratings. After cycle 1, physiotherapists developed personal learning goals which were to be completed in the intervening four to six weeks prior to cycle 2. Cycle 2 was a repeat of cycle 1 with the addition to the facilitated group discussion of a reflective discussion about the achievement of personal goals.

Commitment to change impacts

• Most physiotherapists partially or completely achieved the goals set out in the intervening period (four to six weeks) between cycle 1 and 2.

Performance improvement impacts

• Scores for recording keeping and communication were greater in cycle 2, indicating performance improved in these practice domains (Refer to article and supplement for results).
  • Client communication improvements were:
    • Self-assessment 11%
    • Peer assessment 8%
  • Record-keeping improvements were:
    • Self-assessment 7%
    • Peer assessment 4%

Self-assessment scores for both client communication and record-keeping were consistently lower than for peer assessment with the differences smaller in cycle 2. 

Discussion and conclusions

• The learning goals set after cycle 1 showed strong agreement with the quality indicators set out in the ratings forms for client communication and record-keeping. It appears that the review of the quality indictors for communication or record-keeping cued physiotherapists that routine performance related to an indicator may need to change, thus it became the target of a learning goal.
• Most personal learning goals (54%) related to communication. This was not surprising, as in the Netherlands record-keeping is routinely evaluated by physiotherapists and reviewed by insurers. KNFG ‘s Quality in Motion assessment of communication was novel, and at the time there weren’t a lot of supports available on how to “do” communication, as such physiotherapists felt that in the area of communication they may have the opportunity to learn and develop further.  
• Facilitated discussion of the video tape and clinical records promoted awareness of what other colleagues are doing in practice. It helped to identify physiotherapists who were strong performers and very adept at patient communication. The videotapes helped to identify role models and viewing their own performance may promote learning through social comparison. Moreover, the existence of “mirror neurons in the brain” may explain why the strategy of viewing peers interacting and communicating with patients helped to foster learning and provides ideas for improving their own performance.
• The differences in rating for self-assessment and peer assessment for both cycle 1 and 2 indicated that physiotherapists either underestimated themselves or were overestimated by their peers. In cycle 2 score differences were less because self-assessment scores increased. It is thought that the process encourages professionals to adopt a critical attitude to their own performance because they have been introduced to an assessor auditor perspective. For cycle 2, physiotherapists may have developed improved critical appraisal skills from cycle 1 and enhanced awareness of their true performance level and greater self-efficacy and trust in the process which motivated them to provide higher (more accurate) ratings of their performance.                                                                                                                                                               
• The improvements in the ratings for communication and record-keeping in both self-rating and peer ratings showed that KNFG quality improvement strategies worked. 
• In terms of long-term sustainability, the Quality in Motion program would have to address how much room there is for improvement when individuals are already high performers and introduce a system that facilitates development in both low and high performers. (KNFG Quality in Motion program description in Dutch)

Relevance to Alberta physiotherapist practice

The tools and strategies used in the KNFG initiative could be repurposed for use by Alberta-based physiotherapists interested in improving communication and record-keeping performance.

Any type of routine assessment against defined performance indicators is best practice for ensuring quality practice. In addition to the KNFG quality indicators, the College of Physiotherapists of Alberta’s standards of practice and competency profile are well suited to be repurposed to become assessment criteria for a rating tool. Using these to assess performance has the potential to facilitate goal setting on indicators expected for professional practice in Alberta

Consider whether using a peer group to rate the clinical performance of physiotherapists along with a discussion of the ratings would be a value-added quality improvement strategy in your facility. Peer assessment and feedback may be less threatening than assessment from other third parties. Feedback from trustworthy sources is key to individuals accepting data and acting upon it for performance improvement.

Consider whether videotaping of a patient interaction would be a value-added quality improvement strategy for your facility. Privacy and consent issues must be addressed as part of the feasibility analysis. Bear in mind, that in the feasibility pilot Dutch physiotherapists were initially reluctant to participate with videotaping but once they realized that the videotape showed what physiotherapists actually do and say, instead of what they say they do, it enhanced their learning. The videotape assessment and feedback process related to their core business and became the preferred choice (over role playing) to demonstrate communication performance. Moreover, videotaping performance has the potential to identify physiotherapists who are good role models which could be used for teaching others. Use of videotaping analysis in both face-to-face but also virtual telehealth practice contexts may be powerful strategy to understand whether individuals are strong performers in both practice context.

Comments and questions: Contact the Continuing Competence Program 1-800-291-2782 ext 349

Page updated: 20/04/2022