The item-writing workshop is designed to help faculty members improve the quality of multiple-choice questions written for their examinations. Through informative presentations and discussions, faculty members are able to explore structural issues relating to constructing quality multiple-choice items for testing of basic science and/or clinical science content. An overview of item formats concentrating on the traditional one-best-answer item formats that require examinees to select the single best response is presented. One-best-answer (A-type) questions make explicit the number of options to be selected and are the most widely used multiple-choice-item format. Several examples of well-constructed A-type items are presented. Each consists of a focused item-stem (e.g., a clinical vignette) and a lead-in question, followed by a series of options with one correct answer for each item. 

Information regarding extended-matching (R-Type) items is also presented. Extended-matching items are organized into sets that use one list of options for all items in the set. Examinees are instructed to select the one-best answer. Examples of well-constructed matching sets were presented. Each R-type item includes four components: 

1. a theme
2. an option list
3. a lead-in statement
4. at least two item stems 

Participants are able to explore issues relating to constructing quality questions by recognizing and avoiding technical item flaws. The two types of technical item flaws described included testwiseness and irrelevant difficulty. The participants discuss flaws related to testwiseness that make it easier for some examinees to answer the question correctly, based on their test-taking skills alone. Flaws related to irrelevant difficulty that make the question difficult for reasons unrelated to the trait that is the focus of assessment are also discussed. Several examples of technical item flaws are presented and discussed during the session. 

The session also provides an opportunity for the participants to discuss general guidelines and basic rules for item construction for the basic sciences. Item stems illustrating the difference between a question assessing recall of an isolated fact and a question assessing application of knowledge are presented. Several examples of items using patient and laboratory vignettes to assess application of knowledge are presented and discussed.

Each participant is provided with a printed copy of the manual “Constructing Written Test Questions for the Basic and Clinical Sciences,” which is published by the National Board of Medical Examiners. The manual includes the material that is presented during the session and additional templates that can be used to assist item writers develop items for the basic and clinical sciences.

Speaker

Melissa Billings 
Director, Test Materials Development, National Board of Medical Examiners

Ms. Billings indicated she has no financial relationships to disclose relevant to the content of this CME activity.

Educational Grants

No commercial interest provided financial support for this continuing education activity.

Continuing Education Credit

• DPM: Des Moines University (DMU) is approved by the Council on Podiatric Medical Education as a provider of continuing education in podiatric medicine. DMU has approved this live activity for a maximum of 3.75 continuing education contact hour(s).
• MD: This activity has been planned and implemented in accordance with the accreditation requirements and policies of the Iowa Medical Society (IMS). Des Moines University (DMU) is accredited by the IMS to provide continuing medical education for physicians. DMU designates this live activity for 3.75 AMA PRA Category 1 Credit(s)™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
• DO: Des Moines University is accredited by the American Osteopathic Association (AOA) and approves this live activity for 3.75 AOA Category 2-A CME credit(s).
• ​Other: This live activity is designated for 3.75 AMA PRA Category 1 Credit(s)™.