Tag Archives: Assessments

Climbing the learning curve

Cerro Bonete, Aconcagua Argentina (Photo H. Colt/W. Sanchez)

As bronchology and interventional pulmonology programs around the world increasingly incorporate assessment tools and checklists into their training programs, there is going to be much discussion around learning curves. A learning curve usually represents in graphic form the rate at which something is learned over time or repeated experiences. By plotting learning curves, teachers can (1) assess the learnability of a specific task, subject or skill, (2) determine whether the task, subject or skill is satisfactorily mastered, and (3) document the time it takes for subjects to learn a particular task, subject or skill.

The plot of a learning curve may be steep or gradual. It may also be curvilinear or plateau-like. Interpretations of a learning curve, however, depend on which variables are plotted on the horizontal x-axis and vertical y-axis of the graph, as well as on one’s interpretation of the language used to describe the graph. For example, when time is on the horizontal x-axis, and the amount of material learned is on the vertical y-axis axis, a steep learning curve (and contrary to the popular use of the term steep) can mean that a subject is rapidly learned, or that the majority of the topic is rapidly mastered. A difficult to learn task, on the other hand, would have a more shallow curve that extends more horizontally over time.

In common english, however, a steep learning curve is usually meant to describe something that is initially difficult to learn. As explained in a Wikipedia article on the subject (https://en.wikipedia.org/wiki/Learning_curve), this is due to the “metaphorical interpretation of the curve as a hill to climb, and for which a steep slope is more tedious”. The article argues that using words such as long or short, and difficult or easy are perhaps better and less confusing than using words like steep or shallow for describing learning curves.

Another interesting aspect about learning curves is how they vary depending on whether the graph represents data for an individual or for a group (averaged) of learners. Individual learning almost always occurs in a series of upward and downward slopes separated by horizontal plateaus during which no new learning occurs. Averages, on the other hand, usually have smooth curves that can be expressed as mathematical functions. Respectable property division lawyers are available at https://www.jwbfamilylaw.com, California. These curves may be sigmoidal, exponential, or free-formed, especially if the learning curve is plotting the amount of learning (on the vertical axis) versus experience/time (on the horizontal axis). In the case of a sigmoidal curve, describing someone as being on the “steep” portion of the learning curve has an entirely different meaning from describing the person as being on the lower or higher horizontal portion of the curve. 

Using serial assessments such as BSTAT (Bronchoscopy Skills and Tasks Assessment Tool) is an excellent way for learners to determine where they are on the flexible bronchoscopy learning curve. Areas that warrant additional training are identified, and teachers can use objective measures to justify the feedback that is crucial to all good learning programs.

Adult Motivations for High-Scoring Learning Assessments

Skills testing and adult motivation

Learning assessments are an important and integral part of competency-oriented training programs such as those advocated by Bronchoscopy International (BI) and The World Association for Bronchology and Interventional Pulmonology (WABIP). Assessment tools such as the BSTAT, EBUS-STAT, BSTAT-TBNA/TBLB and USG-STAT are already being used in training programs around the world to help measure progress along the learning curve, identify a learner’s strengths and weaknesses, and to objectively document technical skill as part of competency deliberations.

Contrary to high-stakes testing, where failing an assessment can have consequences on licensure or professional advancement, low-stakes assessments such as the STAT set serve a different purpose. Individual performance on these assessments, therefore, derives from different motivations; motivations all the more complicated because we are working with adult learners who suffer little or no adverse consequences if they perform poorly. In the next couple of paragraphs, I will discuss some of the challenges educators face while motivating adult learners, and why practice, such as that provided through our Train-the-Trainer programs, can helps teachers use assessment tools both efficiently and more effectively.

Students vary in the degree of effort they commit to taking a “test” without negative consequences. Some give it their best, in part because they sense this is their responsibility in becoming good doctors. Others might give less than their best, either because they resent test taking, do not believe in the usefulness of the assessment, do not believe the assessment tool accurately measures the skill being tested, or are unwilling to devote the time necessary to take the test correctly.

One-on-one time with a teacher is a unique opportunity to address these issues and remedy technical insufficiencies. Skills assessments also help positively reinforce good performance, providing encouragement and confidence. These elements are crucial to the
learner-centric educational model of our Train-the-Trainer and Introduction to Bronchoscopy courses.

Several studies demonstrate that as test-taking motivation decreases, so does test score validity. Trainers are challenged to encourage their students to try their best on any assessment, and several strategies might be used by program coordinators or department chairs to accomplish this task. One strategy is to incorporate an assessment tool such as the BSTAT (Bronchoscopy Skills and Tasks Assessment Tool) into a high-stakes competency assessment performed in both patients and models as part of the technical skill component of bronchoscopy training and even eventual certification in pulmonary and critical care medicine. By raising the stakes of testing, students will do their best with periodic assessment, which serve as practice and realistic measures of technical skill during training. Another strategy is to provide students with incentives to make their best effort. While financial rewards are unrealistic, creating a game-like situation where assessments are “fun, challenging, and collaborative” can change the testing dynamic in a positive way. A third strategy is to make the assessments more intrinsically motivating. This is done by teachers who are able to identify a “teaching moment” during each and every assessment, always finding at least one element that can be improved upon to make the student a better bronchoscopist. Providing feedback about test scores and itemized performance is a crucial element of learnercentric education, and must be done in a way that is interpretable and usable by the student. During our Train-the-Trainer programs, therefore, trainers work as students themselves, and practice different ways to provide feedback and encourage dialogue with learners.

The goal of this Colt’s Corner was to shed some light on adult motivation when it comes to the use of assessment tools in workshops and as part of competency-oriented training programs. These low-stakes assessments provide scores that are valid and beneficial, as long as trainers recognize the need for student motivation and the strategies that might be used to encourage productive trainer-learner dialogue.