A History of Innovation in Performance-Based Assessment 

 

The need for authentic measures of clinical competency can be traced back at least as far as the Flexner Report. Often termed Problem-Based Assessments (PBAs), these measures focus on the ability to complete "real world" tasks that integrate multiple skills/abilities and are administered in a systematic standardized format that helps assure reproducibility. OMERAD and the College of Human Medicine have had an important role locally and nationally in the research and development of this form of assessment.

 

Certification in Emergency Medicine

 

One of the most notable areas in which OMERAD has played an important role is the certification of emergency medicine physicians. In the 1960's emergency medicine evolved into a defined medical specialty in large part due to the effort of a local Lansing physician and OMERAD clinical faculty member, John Wiegenstein. The American College of Emergency Physicians (ACEP) was formed in 1968 and in 1975 the AMA approved a formal section on emergency medicine. The American Board of Emergency Medicine (ABEM) was incorporated in 1976 and based in East Lansing. Over the next 15 years OMERAD maintained a close collaborative relationship with ABEM and was instrumental in developing ABEM's initial certification process.

 

 

Jack L. Maatsch OMERAD Professor 1971 to 1980.

Working closely with the ACEP and ABEM, OMERAD faculty under the direction of Jack Maatsch, PhD, pilot tested a certification exam in 1977 that included both innovative and traditional formats. This work resulted in a criterion-referenced certification process (this link requires MSU EZproxy authentication to access article) for emergency physicians that was one of the most innovative and well researched board certification processes at the time. Through the early 1990s, OMERAD and ABEM collaborated on the scoring and reporting of ABEM's certification examinations, a predictive validity study of the certification process and the piloting of a recertification examination for emergency physicians. 

 

The Development of Performance Assessment in the College of Human Medicine

 

During the 1990's the use of PBAs for assessing medical student clinical skills became more commonplace with many medical schools implementing Objective Structured Clinical Examinations (OSCEs). OSCEs consist of a series of stations that examinees rotate through to demonstrate specific focused clinical skills such as performing a physical examination maneuver or gathering focused historical information from a simulated patient. In 1988 CHM implemented an end-of-second year OSCE as a formative feedback experience.

 

By the mid-1990s the Liaison Committee on Medical Education (LCME) began requiring that all medical schools implement PBAs in their curriculum. With increasing pressure at a national level to implement high stakes PBAs, the College considered implementing a fourth year OSCE. These deliberations, chronicled in a special article in Academic Medicine, led to a half-day retreat in late 1996 to review how clinical competency was being assessed and how it might be improved. Instead of focusing on a comprehensive assessment of clinical skills such as an OSCE, the College chose to implement various forms of authentic assessment integrated into individual components of the curriculum. Examples included a neuromuscular PBA in the Block II curriculum and an oral examination in the OB/GYN clerkship.

 

In 2000 the Accreditation Council for Graduate Medical Education (ACGME) published a toolbox of assessment techniques highlighting the need for a variety of assessment methods to adequately evaluate the complex set of skills and abilities required of physicians. In addition during the late 1990's the United States Medical Licensure Examination (USMLE) began pilot work on a performance-based clinical skills component to be included as part of medical licensure. The result was the USMLE Step 2 CS that was implemented as part of the licensure process in 2005.

 

Throughout this time, advancements in technology made possible more authentic ways of assessing clinical competency, including the development of increasingly refined whole patient and partial-task simulators for teaching and assessing clinical skills. Along with the refinement of these simulators, comprehensive digital systems to manage PBAs and record examinee performance were being developed. Technology and simulation came together in the form of simulation centers; CHM along with the Colleges of Osteopathic Medicine, Nursing and Veterinary Medicine jointly sponsored the creation of a state-of-the-art Learning and Assessment Center (LAC) in 2006. The growth in the use of simulation in medical education spawned new professional societies, the Association of Standardized Patient Educators (ASPE) and the Society for Simulation in Healthcare (SSH). The SSH began accrediting simulation centers in 2010; the LAC was one of the first such centers to receive full accreditation.

Learning Assessment Center - Michigan State University

 

In 2005 the CHM Curriculum Committee charged OMERAD under the direction of Rebecca Henry to develop an organization and implementation plan for a PBA to be given at the end of Block III. The examination was pilot tested in the spring of 2006 and implemented in 2007 as an eight station Gateway Examination. In 2008, with the increasing sophistication and focus on the assessment of clinical competency, the College of Human Medicine established the Office for College Wide Assessment under direction of Dianne Wagner, MD, Associate Dean for College Wide Assessment. The Office of College Wide Assessment has created a systematic and organized system of authentic assessment methodology that is implemented across the entire curriculum.

 

In the fall of 2015 the College of Human Medicine will be implementing a new curriculum that will continue to focus on competency, but based on a progress testing approach. Progress testing involves repeated testing experiences using equivalent tests. These tests are calibrated on the performance expectations of learners who successfully complete the educational program. The results of these multiple assessments document the growth of biomedical knowledge and skills for each student over the course of their education, and provide reliable and valid information upon which to base progress decisions. These assessments are developed based on the end-competencies expected of graduates. The advantage of this approach is that it decreases stress among learners and obviates the "binge learning-binge forgetting" cycle common among medical students. Progress testing will be implemented as a suite of assessments involving tests of knowledge, such as the NBME Comprehensive Basic Science Exam and locally developed multiple choice questions, as well as clinical skills and problem solving tasks using standardized clinical encounters similar to the current Block III Care of Patients gateway exam described above. The progress suite also includes multisource learner feedback, portfolio documentation and personal learning plans. Information about a learner's strengths and weaknesses from this suite of assessments can inform their individualized learning plans.

 

Performance assessment has evolved over the 50-year history of the College of Human Medicine. It has been driven by research into how to better assess and train health care professionals, advances in technology and a changing conception of how performance assessment should be integrated into medical education. Faculty from OMERAD as well as other parts of the College have played a significant role in this evolution. 

 

B-CLR Showcase 

An interactive gallery featuring examples of CHM blended and online learning projects by B-CLR

 

CHM Educator 

A website with Information to help you teach, whether in a classroom or in a clinic 

 

DR-ED 

A medical education listserv maintained by OMERAD

 

Medical Education Online (MEO)  

Peer-reviewed international open access journal for disseminating information on the education and training of physicians and other health care professionals

 

Vital Signs Newsletter Archive

Click on the link to view past issues of our newsletter

 

Medical Education Scholarship Group

Meets first Wednesday of each month in room A116 East Fee Hall (East Lansing) and 451 Secchia Center (Grand Rapids) from 1:30 pm to 3:00 pm. The purpose is to develop and support scholarship related to our educational program. Got an idea?  Need an inspiration?  Looking for collaborators? Join us! For questions about this group contact Dr. Brian Mavis: mavis@msu.edu.

 

OMERAD Technology Center (OTC)  

Units within the College of Human Medicine interested in sponsoring faculty and/or staff development programs in support of education and scholarship may request use of the room.The OTC is located in A211 East Fee Hall. For questions about the OTC contact John Williamson: wmsonj@msu.edu