Learning assessments are an important and integral part of competency-oriented training programs such as those advocated by Bronchoscopy International (BI) and The World Association for Bronchology and Interventional Pulmonology (WABIP). Assessment tools such as the BSTAT, EBUS-STAT, BSTAT-TBNA/TBLB and USG-STAT are already being used in training programs around the world to help measure progress along the learning curve, identify a learner’s strengths and weaknesses, and to objectively document technical skill as part of competency deliberations.
Contrary to high-stakes testing, where failing an assessment can have consequences on licensure or professional advancement, low-stakes assessments such as the STAT set serve a different purpose. Individual performance on these assessments, therefore, derives from different motivations; motivations all the more complicated because we are working with adult learners who suffer little or no adverse consequences if they perform poorly. In the next couple of paragraphs, I will discuss some of the challenges educators face while motivating adult learners, and why practice, such as that provided through our Train-the-Trainer programs, can helps teachers use assessment tools both efficiently and more effectively.
Students vary in the degree of effort they commit to taking a “test” without negative consequences. Some give it their best, in part because they sense this is their responsibility in becoming good doctors. Others might give less than their best, either because they resent test taking, do not believe in the usefulness of the assessment, do not believe the assessment tool accurately measures the skill being tested, or are unwilling to devote the time necessary to take the test correctly.
One-on-one time with a teacher is a unique opportunity to address these issues and remedy technical insufficiencies. Skills assessments also help positively reinforce good performance, providing encouragement and confidence. These elements are crucial to the
learner-centric educational model of our Train-the-Trainer and Introduction to Bronchoscopy courses.
Several studies demonstrate that as test-taking motivation decreases, so does test score validity. Trainers are challenged to encourage their students to try their best on any assessment, and several strategies might be used by program coordinators or department chairs to accomplish this task. One strategy is to incorporate an assessment tool such as the BSTAT (Bronchoscopy Skills and Tasks Assessment Tool) into a high-stakes competency assessment performed in both patients and models as part of the technical skill component of bronchoscopy training and even eventual certification in pulmonary and critical care medicine. By raising the stakes of testing, students will do their best with periodic assessment, which serve as practice and realistic measures of technical skill during training. Another strategy is to provide students with incentives to make their best effort. While financial rewards are unrealistic, creating a game-like situation where assessments are “fun, challenging, and collaborative” can change the testing dynamic in a positive way. A third strategy is to make the assessments more intrinsically motivating. This is done by teachers who are able to identify a “teaching moment” during each and every assessment, always finding at least one element that can be improved upon to make the student a better bronchoscopist. Providing feedback about test scores and itemized performance is a crucial element of learnercentric education, and must be done in a way that is interpretable and usable by the student. During our Train-the-Trainer programs, therefore, trainers work as students themselves, and practice different ways to provide feedback and encourage dialogue with learners.
The goal of this Colt’s Corner was to shed some light on adult motivation when it comes to the use of assessment tools in workshops and as part of competency-oriented training programs. These low-stakes assessments provide scores that are valid and beneficial, as long as trainers recognize the need for student motivation and the strategies that might be used to encourage productive trainer-learner dialogue.