Design Team 

• Dr. Churlsun Han, MD 
• Steven Roskos, MD 
• Geraud Plantegenest, MA

B-CLR offers free instructional design consultations to CHM faculty needing help with blended and online curricular materials. To find out more about B-CLR  services, visit  the B-CLR webpage. Contact Geraud Plantegenest, B-CLR Manager, at [email protected] for questions about this project.

Earlier this semester, there was a significant change in how we are handling obtaining consent from students for educational research for projects where the data are derived from our professional education programs. In 2001, we implemented a process for obtaining student consent to participate in educational scholarship. Matriculating students were asked to complete a consent form during orientation and then provided with a reminder each year thereafter as well as contact information should they want to change their consent status. This process was developed in collaboration with the MSU human research protection program (MSU-HRPP).

This summer at the request of MSU-HRPP we were asked to consider an alternative strategy that both enhances the protections for students and reduces administrative time. As a result, we have moved ahead and implemented an "honest broker" system. In terms of human research, the honest broker serves as an intermediary between the investigator and the research subjects, in this case the medical students and the data that they routinely generate in completing their educational programs such as test scores or progress through their educational program. The honest broker creates a de-identified dataset that protects the identity of the students and assures that the investigator is not interacting with the students nor recording any identifiable information about them. Identifiable student information (test scores, faculty ratings, etc.) from one or more sources are linked by the honest broker, who then replaces identifiers with a code. The code, along with the students' de-identified data, is provided to the investigator. Only the honest broker has access to the list that links the code number to the student's identity, allowing the broker the ability to add additional information about students on request from the investigator, without identifying research subjects to the investigators.

In this respect, the use of an Honest Broker has distinct advantages over traditionally exempt studies where the information is recorded by the investigator in such a manner that subjects cannot be identified, directly or through identifiers linked to the subjects. In this latter exempt case, the dataset is limited to what was originally collected by the investigator and IRB application for study must be submitted for the exempt determination to be made. The use of an honest broker allows for additional data to be added to the dataset as appropriate for the study. Further, an IRB application might not be necessary in many cases; this can be determined by sending an e-mail to the IRB outlining the project and the use of the honest broker to create a de-identified dataset.

Individuals serving as an honest broker must meet two requirements. First, they must have legitimate access to the data desired by the research investigator, such as a university or hospital employee with a right to access. Second, they must be completely independent of the research team; they cannot serve as a co-investigator or the faculty mentor.

A project that involves data collection specific to the research question and is not part of the regular curriculum does not qualify for this process and requires submission of an IRB application for review.

OMERAD will provide honest broker services to investigators using one or more educational datasets to provide maximum protection for students. Please contact Brian Mavis or Dave Solomon if you have any questions about this process.  

Do you need to write test questions and want some guidance? Do you have a test you would like to revise?

The Handbook of Learner Evaluation and Test Item Construction-formerly only provided to fellows in OMERAD's nationally known Primary Care Faculty Development Fellowship Program-is now being made available to CHM faculty and staff, for the low, low price of FREE!

This handbook has three sections. The first, titled Qualities of All Good Tests, defines the qualities of purposeful, valid, reliable, objective, comprehensive, differentiating, expected, instructive and useful. Two examples are presented for each quality. The reader is asked to identify the example that more closely exemplifies the quality. The correct answer is then identified and explained.

The second section, General Standards for Test Items, explains the standards for test items and provides examples. The standards are:

1. There are sufficient test items to cover all the important ideas.
2. All test items are on ideas related to instructional objectives.
3. Questions simulate the use of the test content as specified in the objective.
4. All questions are easy to read.
5. There is only one right answer.
6. Don't give the answer away.
7. The answer to one question doesn't affect the answer to another question.
8. Questions are grouped by type.
9. Questions refer to a source if needed.

The final section, Technical Specifications for Types of Test Items, explains how to write different kinds of test items, and includes explanations and clinical and non-clinical examples for each. The eight types of test items are:

• True or False
• Multiple Choice
• Matching
• Completion
• Short Answer
• Essay
• Production
• Procedures Checklist

The Appendix to the Handbook provides a worksheet for each type of test item you want to write. These worksheets show a summary of the specifications and provide space to write the related objective and test items, all on one page.

The Handbook is a PDF document formatted to be printed double-sided. It is available from the Tutorials section of the OMERAD website at: http://omerad.msu.edu/tutorials.php