Continuing Competence Theory: Effectiveness of Peer Assessment for Implementing a Low Back Pain Guideline

Clinical practice guidelines are increasingly considered important vehicles to support professionals to deliver evidence-based, effective care. However, their uptake in physiotherapist professional practice remains a challenge.

Implementation of guidelines (new and revised) is challenging, especially when physiotherapists already have considerable experience in treating patients. Physiotherapists may be able to fine tune their behavior based on new research findings, which requires adopting small, but determining, differences. Finding the right implementation strategy is considered key to guideline adoption.

This fourth article in a series on self-assessment and feedback highlights research into peer assessment and feedback and whether it is more effective than case-based discussion to improve knowledge and guideline-consistent clinical reasoning on physical therapy guidelines for low back pain (LBP).1

The research study ¹ was co-funded by the Dutch Physical Therapy professional association (KNGF), Radboud University Medical Center, the Scientific Institute for Quality of Healthcare and the HAN University of applied sciences.

Rationale for the study

• Desire to improve the uptake of guidelines in physiotherapy, with a focus on improvement of knowledge, skills, attitude and awareness, and guideline adherence.
• Appreciation that small group education and peer review were widely used proven methods for guideline implementation.
  • Small group education is continuing education or skills training with a small group of peers.
  • Peer review is a “continuous, systematic, and critical reflection by a number of care providers, on their own and colleagues’ performance with the aim of achieving continuous improvement of the quality of care”. Peer review may include different methods, such as consensus development, evaluation of performance, practice visit or peer assessment. Peer assessment is a specific form of peer review with professionals assess (judge) the performance of their peers using relevant criteria and providing feedback.”²
• Interest in peer assessment as a training strategy, its impact on self-reflection, and improving awareness of actual performance.
  • It had been successfully used with physicians.
• Understanding that case-based discussion is widely used in post graduate education for implementing guidelines and stimulating reflection, integrating knowledge into clinical reasoning and decision-making.
• Desire to understand the impacts of two different pedagogical strategies: peer assessment and case-based discussion. The former focuses on what one does, that is performance, and the latter focuses what one says, that is discussion.

Study design

• Ten communities of physiotherapists were cluster-randomized into peer assessment and control groups N=90.
• Each group met for four two-hour educational sessions over a six-month period and used the same clinical cases and a variety of strategies to guide the learning.
  • Peer assessment reflected on and performed LBP management in different roles (patient, physiotherapists).
  • Control group used structured discussions.
• Changes in knowledge and guideline consistency were assessed at baseline and six months later after the sessions were completed.

Peer assessment procedure

The goal of peer assessment was to improve guideline-consistent knowledge and reasoning skills and performance.

• Participants received a peer assessment manual which included instruction about the peer assessment procedure, time schedule and instructions for providing constructive feedback.
• Participant’s clinical performance was directly observed and evaluated by peers in a simulated setting. Performance was assessed using a scoring sheet with defined criteria to be scored on a seven-point scale. Performance categories addressed diagnostic process and the intervention process.
• Participants adopted roles of physiotherapist, patient and assessor. They were mentored by an individual with expertise in assessment, low back pain and education.
• In the first two meetings, participants worked through three roles of physiotherapists, patient and assessor for two cases. Participants were not informed about which role they would adopt. When in the role of physiotherapist, participants were expected to reason aloud their choices for diagnosis and treatment skills. As an assessor, participants were expected to observe performance of their peer and provide them with oral and written feedback. As patients, participants simulated a clinical problem according to brief simulation guidelines.
• Following the second meeting, each participant developed a personal plan for improvement, including an action plan based on the feedback and assessment results from their colleagues and completed a strength, weaknesses, opportunities and threats (SWOT) analysis of their own performance which was brought forth for discussion with their peers in the third meeting. The plan was clarified and peers provided feedback.
• The fourth meeting was identical to the first two meetings with the exception that patient cases were adapted to meet the learning needs of participants.

Case-based discussion

The goal of case-based discussion was to improve  guideline-consistent knowledge and reasoning skills.

• Participants received a program manual with program schedule, case discussion procedure, case description for each meeting and time schedule.
• During meetings, assignments were given to guide and evaluate the case-discussion process involving
  • Supporting questions to unravel the problem
  • Supporting questions to establish a physiotherapy diagnosis
  • Writing a summary of discussion after each meeting.
• Following the meeting:
  • Learning results were evaluated by the group
  • Each participant would document an explanation of what was learned.
• Fourth meeting, 25 statements about low back pain management were discussed (anatomy, physiology, etiology, diagnosis and treatment).
• Following the fourth meeting participants answered true false statements and received feedback on answers from a research assistant.

Outcomes

1. Guideline Consistent Knowledge and Reasoning measured through:

• Completion of, pre- and post-intervention, a series of online questions related to four clinical vignettes representing four patient profiles.
  • Scoring for each vignette was dependent upon context of the vignette with one assigned to the answer corresponding to the guideline and zero to the answers contravening the guideline with a percentage assigned to each vignette.

1. Self-reflection measured by:

• Self Reflection Insight Scale, a 20-item closed question which is a validated measure of “the process of self-reflection and insight that is presumed as conditional for self-directed change.”¹

Results (n=78)

• Guideline-consistent knowledge and clinical reasoning
  • Peer assessment group increased 8.4%
  • Control group declined 0.1%
  • Differences were significant
• Self-reflection, no differences
  • Peer assessment group increased 2.5%, control group increased 0.5.
  • Differences were not significant

A tailored peer assessment and feedback strategy was more successful than case-based discussion to increase knowledge and clinical reasoning consistent with recommendations in a physiotherapy LBP guideline. The peer assessment and feedback strategy did not improve reflection.

The peer assessment strategy used multiple educational formats including pre-circulation, in-depth assessment anchored in a problem-solving format, assessment of performance with individualized and timely performance feedback, and individualized improvement plan.

“The strength of the peer assessment strategy is that participants performed different roles, which leads to reflection on the guideline from various perspectives. In the assessor role they had to reflect on professional qualities of their colleague using guidelines recommendations as gold standard. This facilitates the ability to improve clinical skills while comparing the observed performance of colleagues with their own performance level and the guideline. In the physiotherapist role, participants reflected on their own knowledge and performance level using the feedback of their peers. In the patient role they were able to reflect on the communication and perception of diagnosis and treatment from the patients’ perspective. This triangle of feedback might increase reflection and the awareness of individual shortcomings which are considered key factors in guideline implementation and improvement of professional practice.”¹

Relevance to Alberta physiotherapists

All organizations are challenged to keep up with evidence and introduce changes in practice. This study demonstrated that peer assessment and feedback and role playing were more effective than case-based discussion in enhancing knowledge and clinical reasoning on a new practice guideline.

Managers and practice leaders may want to try the small group peer assessment and feedback approaches described here when introducing guidelines and other performance-oriented changes in their facility.

Course instructors might want to include role playing as part of their teaching pedagogy.

Bear in mind that while behavior change toward consistency of practice between physiotherapists is desirable, it should take place with the understanding that changes in performance of physiotherapists should result in improved patient outcomes. Patient outcomes should also be monitored before and after the introduction of guidelines to understand the impacts of guidelines on practice.

Comments and questions - Contact the Continuing Competence Program 1-800-291-2782 etx 349

Page updated: 20/04/2022