 |
 |
 |
 |
| Director's Message | 
Competencies, Milestones and CEPAERs, Oh My!
At the best of times, medical education is a hearty alphabet soup of acronyms that are often hard to digest; even career educators find themselves at a loss at times. In current conversations of learner assessment you are likely to hear mention of competencies and milestones, and more often now, entrustable professional activities (EPAs). Though related and seemingly interchangeable, they are distinct ingredients necessary for understanding the complex flavors of current thinking in assessment. Let me explain how we got here.
We had just survived Y2K when our alphabet soup became more complex, as medical education embraced competencies as the soup stock for long simmering concerns about learner assessment. As an observable ability that brought together multiple elements of knowledge, skills and attitudes, competencies had some intuitive appeal. It was not long before the Accreditation Council for Graduate Medical Education (ACGME) came out with the ultimate cookbook. Competencies became a fashionable if not obligatory appetizer for discussions of curriculum reform, and the Liaison Committee for Medical Education (LCME) added competencies to the menu of accreditation requirements. In response CHM cooked up SCRIPT to characterize the competencies underlying our educational program. Not long after, faculty-based competencies were served up for tasting. Fast-forward ten years. The Milestones Project, a joint effort of ACGME and the American Board of Medical Specialties (ABMS), was initiated to consider a more realistic longitudinal model of competency. Milestones give us significant competency-based developmental outcomes that combine knowledge, skills and/or attitudes. Residents demonstrate progressive proficiency in achieving competencies during their residency program; each resident is expected to achieve defined milestones for specific competencies prior to graduation. Much of the culinary creativity around competencies and milestones has come from GME. Around the same time, medical educators noted a continental influence in competency conversations: an unfamiliar seasoning called EPAs. EPAs are integrated tasks that trainees can be trusted to perform without supervision once they have demonstrated competence. EPAs are independently executable, observable, and measurable, which makes them appropriate for entrustment decisions. EPAs addressed concerns that competencies were often expressed in abstract terms, or as performance that was chopped too finely and inconsistent with meaningful approaches to learner assessment.
Now everyone is talking EPAs. The Association of American Medical Colleges released a consensus document for undergraduate medical education outlining thirteen core entrustable professional activities for entering residency (CEPAER). As illustrated in the diagram, EPAs integrate competencies, usually from two or more domains. For each competency, milestones can be identified and operationalized into descriptive narratives of expected behaviors for learners at pre-entrustable and entrustable stages of performance.
The AAMC document describes the 13 CEPAERs in terms of functions and expected behaviors; they represent professional activities that entering residents should be able to perform without direct supervision on the first day of their residency:
1) Gather a history and perform a physical examination
2) Prioritize a differential diagnosis following a clinical encounter
3) Recommend and interpret common diagnostic and screening tests
4) Enter and discuss orders/prescriptions
5) Document a clinical encounter in the patient record
6) Provide an oral presentation of a clinical encounter
7) Form clinical questions and retrieve evidence to advance patient care
8) Give or receive a patient handover to transition care responsibility
9) Collaborate as a member of an inter-professional team
10) Recognize a patient requiring urgent or emergent care, and initiate evaluation and management
11) Obtain informed consent for tests and/or procedures
12) Perform general procedures of a physician
13) Identify system failures and contribute to a culture of safety and improvement
These CEPAERs are something that medical educators will be chewing on for a long time. The good news is that CHM is one of ten schools participating in a pilot project to operationalize the assessment of EPAs. Even better news is that EPAs pair nicely with red wine, white wine or beer as we digest this newest ingredient.
Brian Mavis, PhD Associate Professor and Director
Office of Medical Education Research and Development
College of Human Medicine |
|
|
|
|
|
Announcements
|
- CHM's New Shared Discovery Curriculum Posters and Town Hall Presentation
The College of Human Medicine held a poster reception on January 28, 2015 to display various aspects of the shared discovery curriculum. All of the posters that were on display are available online. The recorded Town Hall presentation is available here.
- Lilly Teaching Seminars Series
MSU's Office of Faculty and Organizational Development has announced this semester's Lilly Teaching Seminar. The workshops are designed to provide approaches that participants can use in their own teaching. Resource materials are provided to assist with implementation. The format is interactive and intensive, with time for discussion. Please share with faculty and staff who may be interested in attending. See current topics at: http://fod.msu.edu/lilly-teaching-seminar-series
- Khan Academy Seeks Content Creators
Khan Academy Health and Medicine is looking for exceptional content creators to help us continue to make high quality videos in key topic areas. Thus far, our team has developed hundreds of videos to help students learn, and we want to expand and improve the learning experience. If you have a deep understanding of Health, Medicine, and Nursing and have a great intuition for how students like to learn, then we encourage you to apply!
|
 SPOTLIGHT
| |
OMERAD Medical Education Mini Grant Program
 For the second year, OMERAD is accepting proposals for its Medical Education Mini Grant Program. The mini grant awards are intended to promote collaboration and synergy among faculty of the Michigan State University College of Human Medicine, to encourage educational research relationships between departments within the institution, and to foster research and scholarship in undergraduate medical education. The goals of the mini grant program include:
- Expanding our base of nationally competitive undergraduate medical education research activity leading to the potential for increased extramural support for medical education research at Michigan State University.
- Elevating the visibility and reputation of the College of Human Medicine faculty for teaching excellence and scholarly accomplishment.
- Strengthening the medical school's educational offerings by providing medical students and junior faculty with opportunities to participate in educational research, scholarship, and creative activity.
- Stimulating the intellectual environment at the College of Human Medicine.
Enhancing Learner Engagement
This year, the theme for this grant solicitation is "enhancing learner engagement in undergraduate medical education." Grants addressing this theme will receive priority.
In this context, "learner engagement" refers to products or activities that enable learning by exploration and discovery. These products (presentations, digital interactive modules, games, simulations, blended courses, etc.) successfully integrate content with complementary "DO" activities to reduce the time that learners are physically or mentally passive.
Some examples of tested combinations of learner engagement activities include:
- Learners follow demonstrations of techniques or procedures with practice activities. Ask learners to apply what is being demonstrated. Learners may repeat demonstrations on their own or complete a coached simulation.
- Use discovery activities (virtual labs, instructor-led or virtual case studies, role-playing) to help learners notice principles and expand on these principles.
- Use games and simulations to provide meaningful productive play (e.g., software simulations, device simulations, quiz-show game that requires both recall and inference).
- Design presentation/content activities for social learning in communities of practice, learning communities, professional networks (e.g., embedded presentation in a blog, podcast, or forum) where discussions can naturally follow.
- Mobile learning, designed for the environment where the learning occurs.
Eligibility
All full-time faculty may submit proposals for the Medical Education Mini Grants program.See full application procedure details at: http://www.omerad.msu.edu/endowments/minigrant.html
DEADLINE for application is April 15, 2015 at 5:00 p.m.
|
| B-CLR |
Technologies That Will Change the Face of Education
 The New Media Consortium (NMC) has recently released their 2015 Technology Horizon Report: Higher Education Edition that identifies and describes emerging technologies likely to have an impact on learning, teaching, and creative inquiry in higher education in the next five years. The report is highly recommended as an essential read for educators, institution leaders, administrators, policymakers, and technologists who want to keep ahead of developments in education.The top six emerging technologies are identified and categorized under three different time-to-adoption horizons.
The report also highlights six key trends accelerating educational technology adoption, and significant challenges impeding it.
 | | - Six emerging technologies are identified with their respective time-to-adoption horizons. |
Key NMC Horizon report emerging technologies defined:
- Bring Your Own Device (BYOD) - The practice of people bringing their own laptops, tablets, smartphones, or other mobile devices to the learning environment.
- Flipped Classroom - A model of learning that rearranges how time is spent both in and out of class to shift the ownership of learning from the educators to the students.
- Makerspaces - Sometimes also referred to as hackerspaces, hackspaces, and fablabs, a physical location where people gather to share resources and knowledge, work on projects, network with others, and build.
- Wearable Technology - Devices that can be worn by users, taking the form of an accessory such as jewelry, sunglasses, a backpack, or even actual items of clothing such as shoes or a jacket.
- Adaptive Learning Technologies - Software and online platforms that adjust to individual students' needs as they learn.
- The Internet of Things (IoT) - Network-aware smart objects that connect the physical world with the world of information.
The research behind the NMC Horizon Report: 2015 Higher Education Edition is jointly conducted by the New Media Consortium (NMC) and the EDUCAUSE Learning Initiative (ELI), an EDUCAUSE Program. The full report is available for free: Download the 2015 NMC Higher Education Edition report (PDF). For more information about NMC, or to sign up for the monthly e-mail newsletter, go to http://www.nmc.org/ References:
Geraud Plantegenest, MA
Manager, Blended Curricular Learning Resources (B-CLR)
Office of Medical Education Research and Development
517-353-3455 | [email protected]
|
CHM Program Evaluation
|
A History of Innovation in Performance-Based Assessment
The need for authentic measures of clinical competency can be traced back at least as far as the Flexner Report. Often termed Problem-Based Assessments (PBAs), these measures focus on the ability to complete "real world" tasks that integrate multiple skills/abilities and are administered in a systematic standardized format that helps assure reproducibility. OMERAD and the College of Human Medicine have had an important role locally and nationally in the research and development of this form of assessment.
Certification in Emergency Medicine One of the most notable areas in which OMERAD has played an important role is the certification of emergency medicine physicians. In the 1960's emergency medicine evolved into a defined medical specialty in large part due to the effort of a local Lansing physician and OMERAD clinical faculty member, John Wiegenstein. The American College of Emergency Physicians (ACEP) was formed in 1968 and in 1975 the AMA approved a formal section on emergency medicine. The American Board of Emergency Medicine (ABEM) was incorporated in 1976 and based in East Lansing. Over the next 15 years OMERAD maintained a close collaborative relationship with ABEM and was instrumental in developing ABEM's initial certification process. 
 | |
Jack L. Maatsch
OMERAD Professor 1971 to 1980.
|
Working closely with the ACEP and ABEM, OMERAD faculty under the direction of Jack Maatsch, PhD, pilot tested a certification exam in 1977 that included both innovative and traditional formats. This work resulted in a criterion-referenced certification process (this link requires MSU EZproxy authentication to access article) for emergency physicians that was one of the most innovative and well researched board certification processes at the time. Through the early 1990s, OMERAD and ABEM collaborated on the scoring and reporting of ABEM's certification examinations, a predictive validity study of the certification process and the piloting of a recertification examination for emergency physicians. The Development of Performance Assessment in the College of Human Medicine During the 1990's the use of PBAs for assessing medical student clinical skills became more commonplace with many medical schools implementing Objective Structured Clinical Examinations (OSCEs). OSCEs consist of a series of stations that examinees rotate through to demonstrate specific focused clinical skills such as performing a physical examination maneuver or gathering focused historical information from a simulated patient. In 1988 CHM implemented an end-of-second year OSCE as a formative feedback experience. By the mid-1990s the Liaison Committee on Medical Education (LCME) began requiring that all medical schools implement PBAs in their curriculum. With increasing pressure at a national level to implement high stakes PBAs, the College considered implementing a fourth year OSCE. These deliberations, chronicled in a special article in Academic Medicine, led to a half-day retreat in late 1996 to review how clinical competency was being assessed and how it might be improved. Instead of focusing on a comprehensive assessment of clinical skills such as an OSCE, the College chose to implement various forms of authentic assessment integrated into individual components of the curriculum. Examples included a neuromuscular PBA in the Block II curriculum and an oral examination in the OB/GYN clerkship. In 2000 the Accreditation Council for Graduate Medical Education (ACGME) published a toolbox of assessment techniques highlighting the need for a variety of assessment methods to adequately evaluate the complex set of skills and abilities required of physicians. In addition during the late 1990's the United States Medical Licensure Examination (USMLE) began pilot work on a performance-based clinical skills component to be included as part of medical licensure. The result was the USMLE Step 2 CS that was implemented as part of the licensure process in 2005. Throughout this time, advancements in technology made possible more authentic ways of assessing clinical competency, including the development of increasingly refined whole patient and partial-task simulators for teaching and assessing clinical skills. Along with the refinement of these simulators, comprehensive digital systems to manage PBAs and record examinee performance were being developed. Technology and simulation came together in the form of simulation centers; CHM along with the Colleges of Osteopathic Medicine, Nursing and Veterinary Medicine jointly sponsored the creation of a state-of-the-art Learning and Assessment Center (LAC) in 2006. The growth in the use of simulation in medical education spawned new professional societies, the Association of Standardized Patient Educators (ASPE) and the Society for Simulation in Healthcare (SSH). The SSH began accrediting simulation centers in 2010; the LAC was one of the first such centers to receive full accreditation.  | | Learning Assessment Center - Michigan State University |
 In 2005 the CHM Curriculum Committee charged OMERAD under the direction of Rebecca Henry to develop an organization and implementation plan for a PBA to be given at the end of Block III. The examination was pilot tested in the spring of 2006 and implemented in 2007 as an eight station Gateway Examination. In 2008, with the increasing sophistication and focus on the assessment of clinical competency, the College of Human Medicine established the Office for College Wide Assessment under direction of Dianne Wagner, MD, Associate Dean for College Wide Assessment. The Office of College Wide Assessment has created a systematic and organized system of authentic assessment methodology that is implemented across the entire curriculum.
 In the fall of 2015 the College of Human Medicine will be implementing a new curriculum that will continue to focus on competency, but based on a progress testing approach. Progress testing involves repeated testing experiences using equivalent tests. These tests are calibrated on the performance expectations of learners who successfully complete the educational program. The results of these multiple assessments document the growth of biomedical knowledge and skills for each student over the course of their education, and provide reliable and valid information upon which to base progress decisions. These assessments are developed based on the end-competencies expected of graduates. The advantage of this approach is that it decreases stress among learners and obviates the "binge learning-binge forgetting" cycle common among medical students. Progress testing will be implemented as a suite of assessments involving tests of knowledge, such as the NBME Comprehensive Basic Science Exam and locally developed multiple choice questions, as well as clinical skills and problem solving tasks using standardized clinical encounters similar to the current Block III Care of Patients gateway exam described above. The progress suite also includes multisource learner feedback, portfolio documentation and personal learning plans. Information about a learner's strengths and weaknesses from this suite of assessments can inform their individualized learning plans.
Performance assessment has evolved over the 50-year history of the College of Human Medicine. It has been driven by research into how to better assess and train health care professionals, advances in technology and a changing conception of how performance assessment should be integrated into medical education. Faculty from OMERAD as well as other parts of the College have played a significant role in this evolution.
|
| Faculty Development |
OMERAD Seminar Series: Pilot Test of Blended Format
The OMERAD Seminar Series this year offered a five-week blended course on How to Develop a Curriculum. The blended format combines self-paced instruction with group meetings. The instructional tutorial was online and was accompanied by worksheets.
All meetings were held remotely via telephone or desktop videoconferencing. There were two participants from Flint, one from Grand Rapids, and four from East Lansing. The instructors were also in East Lansing.
We designed the course to be blended to make it more convenient for faculty schedules, and to provide communication and networking opportunities for faculty. We used distance technology so faculty would not have to travel to attend the course.
The seven participants worked on designing curricula for a wide range of learners--medical students, residents, practicing physicians. The topics were also diverse:
- Completing disability forms
- Patient-centered interviewing
- How to do research
- Designing a Procedures Day
- Quality improvement
- Using cognitive and functional assessment tools
The pilot test results were generally positive. Participants liked the components of the course (tutorial, worksheets, meetings) and felt that the course was useful and effective. Some wished the course had been longer, and some wanted more detail in the tutorial. All wanted more examples of the completed worksheets.
We plan on revising this course based on the pilot test results and will offer it again in the future. If you are interested in participating in the blended course on developing a curriculum, please contact Deborah Sleight at [email protected].
|
Hot Off the Press
| |
-Wagner DP. Trust Worthy. JAMA. 2014;312(24):2621-2622. http://doi.org/10.1001/jama.2014.10438
|
| OMERAD Resources |
|
|
The Office of Medical Education Research and Development is a unit within the College of Human Medicine at Michigan State University. Its mission is to improve medical education and related service programs through evaluation and research consultation, relevant instruction, and programs of faculty development.
Established in 1966, OMERAD is the oldest continuously operating office of medical education in the United States.
A-202 East Fee Hall
965 Fee Road
East Lansing, Michigan 48824
http://omerad.msu.edu
For questions about this newsletter, please contact us
@ VitalSignsNews
|
|
|
|
|
