Tuesday, October 23, 2012

The unsung virtues of multiple choice tests

Personally, I'm not a big fan of multiple choice tests. I'm not a great multiple choice test-taker. I'm not particularly good at the kinds of things most often measured by multiple choice tests--recall of specific facts and labels; rapid, accurate calculations--nor are these skills I particularly value. I often find more than one choice arguably correct, and I often wish there were space for justifying one's answers. Finally, as a teacher, I find it much more difficult to design a good multiple choice test than to write a good essay question.

However, given what often are highly problematic alternatives, I've come to view the multiple choice option as often the least bad. Multiple choice tests are much easier to score, and leave much less room for subjectivity in scoring, than essay tests and other open-ended tests. They're therefore easier to standardize or norm across large, diverse populations. Standardization, in turn, is the basis for predictive power. In the case of standardized, normed multiple tests tests like SATs or the Raven's Progressive Matrices IQ test, the test's predictive power is quite strong. Studies show the Raven's to be the best single predictor of performance on a wide variety of other intelligence tests (see reference here). And the SAT tests (as well as the various standardized IQ tests) are significant predictors of success in college.

As we all know, there is one part of the SATs that deviates from the multiple choice format, namely, the essay on the SAT writing test. But its objectivity and predictive power are notoriously weak--as we see, for example, when professional writers (for example Catherine Johnson of kitchentablemath, who has had excerpts of her best selling book showcased on the SAT reading test) earn significantly less than perfect scores.

As it turns out, there's at least one other advantage of multiple choice over other formats. An article in last weekend's Wall Street Journal on the cognitive benefits of test taking reports particular benefits for multiple choice tests:

Exams have long been known to facilitate later recall, and a student tested after initial study of a subject retains more learning than if he or she studies longer but isn't tested right away. Exams have long been known to facilitate later recall, and a student tested after initial study of a subject retains more learning than if he or she studies longer but isn't tested right away. In a new paper, a psychologist surveying relevant literature has found that testing had "robust benefits" when it came to students' ability to apply their learning in different contexts—presumably the point of school.
Another recent paper supports multiple-choice tests. In an experiment with undergraduates, a multiple-choice test improved recall more than a test without answer choices. Also, a multiple-choice test apparently can improve retrieval of facts used on the test as incorrect answers.
I have worried from time to time that the presence of these wrong choices (aka "distractors") in multiple choice tests could result in mislearning, but perhaps I was wrong.

4 comments:

John said...

Just to put it out there; another drawback of multiple choice tests are that for people like me who can pass nearly any multiple choice test as long as its in english regardless of subject matter, (and can do very well on tests on subjects I am only vaguely familiar with) it can give people a false sense of achievement.

I'd perhaps suggest that built in to "easier to grade" is a corresponding "harder to determine what the true error in measurement is"

Much of my schooling consisted of my teachers failing to demonstrate my ignorance, not me demonstrating mastery.

Auntie Ann said...

I had a physics teacher in high school who did something I've always thought was brilliant: he said "every Thursday is quiz day." He explained that he might not choose to give a quiz on any given Thursday, but that everyone in the class should be prepared to take one every single week. It forced everyone...or maybe just the students who cared...to keep up with the material and to study weekly. (Obviously, this worked better for 1st period classes, which could have no warning that he was giving a quiz. For students with class after lunch, they could always assume that they could cram at lunch if they had to.)

I can't remember if any were multiple choice, but the school was big on m.c. tests--we had a scanner that could quickly grade everyone's tests. I think in one class, we got a graded quiz back before the hour was over.

MagisterGreen said...

I have worried from time to time that the presence of these wrong choices (aka "distractors") in multiple choice tests could result in mislearning, but perhaps I was wrong.

As someone who A) likes multiple-choice tests and B) quizzes his students every day, I've come to appreciate the power of a well-put together m/c test. Especially with distractors...I always try to put one distractor in each question that could be correct but is subtly wrong. You can really force students to reflect on why they are picking one answer over another; it makes for great post-quiz discussion and has been a potent tool for review and practice in class. I'm a fan.

Nancy Bea Miller said...

I have always liked multiple choice tests, and although it never occurred to me that testing reinforced the learned material something in me says "That makes sense." Because I DO recall taking multiple choice tests and having a few moments of "Oh, yeah! THAT's what it was about, not this..." as I was taking the tests.