The Daily Pennsylvanian
The recent scoring mishap on the SAT has revealed one potential flaw in fill-in-the-bubble multiple-choice testing method, used at universities across the country, including Penn.
The College Board -- which administers the SAT Reasoning Test and Subject Tests -- admitted this month that about 4,000 tests from October were scored incorrectly. Changes in humidity altered the students' score sheets, making it difficult for the scoring machines -- similar to those used at Penn -- to grade accurately.
The incident revealed one of the vulnerabilities of multiple choice testing systems -- vulnerabilities that some experts say should make the practice have a limited role in higher education.
Robert Schaeffer, the public-education director of FairTest, a group that criticizes standardized testing, said that multiple-choice testing is appropriate when used to "measure limited factual recall and some procedural knowledge, but it's not suited to measure the complex higher-order thinking skills we want students to master."
Despite the controversy, many Penn professors have been using similar systems to test their students for years and show no signs of stopping.
But for Penn faculty teaching large lecture courses, open-ended forms of assessment -- like essay tests or short answer questions -- are not always possible.
In these classes, faculty often use multiple-choice testing, where students "bubble in" answers on "scantron" sheets similar to those used for the SAT.
The term "scantron" refers to a machine -- Penn has one on campus -- that grades the sheets by scanning the bubbled-in answers. Graded sheets are then returned to professors and teaching assistants for processing.
Psychology professor Andrew Shatte -- who has about 750 students currently taking his Psychology 001 course -- said that he relies solely on scantron sheets when he administers tests.
Shatte listed quicker grade return to students, fairness and efficiency among reasons he uses scantron sheets.
But Shatte also said that at times he feels "constrained by that exam technique" since it limits the amount of student knowledge that can be tested.
In order to lessen what Shatte called the "alienating" effect of scantron testing, he and his teaching assistants constantly review exams to make sure that the questions are as fair as possible and also "devote a lot of time for office hours" where students can discuss test results and course material.
Shatte's students echo both his praises and complaints regarding the scantron system.
Wharton senior Preethi Bettadapur -- who is currently in Shatte's class -- said that she likes the scantron system because she gets her test scores back quickly and "there had to be one right answer" on the tests.
Bettadapur was not concerned about incorrect scoring, but she agreed with Shatte's analysis that scantron testing "didn't really allow us to bring in outside knowledge and give other information that we had."
College of Arts and Sciences Dean Dennis DeTurck also uses multiple-choice exams in his mathematics courses but does not use scantron sheets so that students can show their work on the test questions.
DeTurck said that he likes the multiple-choice format since it allows him to give "immediate feedback" to students but said he also reviews exams and will alter an original score if necessary.
Overall, DeTurck said, the College does not encourage professors to use a certain kind of testing method; that assessment is usually "discipline-specific."
