Edutopia - April/May 2008 - (Page 37)

them comparable from year to year, could make it too impractical. Scary as it might sound, arti⇒cial intelligence is likely to play a big role in the scoring of such exams. If the technology becomes sophisticated enough to handle answers to trickier problems, it could make better assessment more affordable. The ETS’s Randy Bennett, on the other hand, believes the prospects of building an assessment system to match the demands of the twenty-⇒rst century are “pretty good.” The key is to convince states that it’s practical, affordable, and clearly better than today’s exams at providing meaningful information. At least one state, West Virginia, has begun asking the test makers it contracts to emphasize more modern problems and skills. Another hurdle will be for politicians to temper their devotion to multiplechoice questions and get comfortable with a little subjectivity. “For any assessment,” Schleicher says, “you have to make a tradeoff between objectivity and relevance.” Jennifer Simone, for one, is depending on forward-thinking test makers and policy makers to succeed—for the sake of her students, most of all. “That we are held accountable is a good thing. That we are doing something to measure the progress of our students is a good thing,” she says. “I just disagree with the way it’s being done.” e Command Performance While schools wait for innovation in accountability testing, some are taking matters into their own hands, creating performance assessments that guide and strengthen teaching and learning. Typically, these assessments come in the form of portfolios and presentations—tasks that bear something in common with the kind of work students may ultimately do in college or in a job. At Anzar High School, in San Juan Bautista, California, students must complete a series of exhibitions to graduate, each one including a research-based written piece and an oral presentation. The topics are of the students’ own choosing, fashioned (with guidance from a teacheradviser) to cover language arts, science, history, math, and service learning and postgraduate plans—areas typically combined into three crossdisciplinary exhibitions. Students work for a semester or more on each project, and a panel of jurists, including teachers, alumni, and community members, evaluates their performance. “If things are going as intended, students are really passionate about their issue, which means they’re getting to devote a whole class period to working on something they adore,” says Principal Charlene McKowen, whose school serves 420 students from San Juan Bautista and Aromas, rural communities south of San Jose. “It’s almost eerie once they get going. You just hear ‘Click-click-click,’ and it’s pretty quiet.” On exhibition day last spring, presentations covered such diverse topics as “Is Prison an Effective Rehabilitation for Latino Males?” “How Do Pets Affect Health and Education?” and “What Materials Will Be Used in the Future of Surfboard Manufacture?” Marisol Garcia, a junior who had researched the merits and failings of prisons, faced three panelists: a teacher, an alumnus now working at an international stafﬁng company, and a San Jose State University professor. She told them about her interview with a prison guard, and drew connections between the data she’d found and a memoir she’d read, Always Running: La Vida Loca: Gang Days in L.A., by Luis J. Rodriguez. The verdict: Garcia excelled in analyzing the book but needed more substance in her factual presentation, the jurists said. They gave her a 2 (“minimal pass”) for the history component and a 3 (“outstanding effort all around”) for language arts. Said Garcia, “You have to actually know what you’re talking about. It takes a lot of time and effort.” These assessments don’t take away the pain of state accountability tests, but they do steer instruction toward critical thinking and endow students with conﬁdence and useful skills. In other schools, says McKowen, “I would just see over and over again that students would go off to college and be afraid or feel like a fraud, because they’d learned how to play the game. We wanted to be sure that any student who graduated from here would know what they were capable of doing.” —GR Where Standardized Tests Fail Today’s standardized assessments can be useful for spotting big trends or gauging the effectiveness of state programs overall. However, when used in high-stakes accountability, as the sole indicator of an individual student’s achievement or the quality of a single school or school district, these tests can be imprecise. Creating and scoring such tests is complex. Here are some of the steps in the testing process where subjectivity prevails and inaccuracies arise: •Content selection: If the state sets too many standards, teachers won’t be able to cover them all and will have to guess which are on the test. If test makers include too few questions on any given skill, the results may not truly show how well a student can perform it. •Ambiguous questions: Particularly for multiple-choice questions, a child may be able to make a plausible, even creative, argument for choosing one of the “incorrect” answers, but the format doesn’t allow the child to explain. •Setting the difﬁculty level: This determination, typically based on educators’ and ofﬁcials’ opinions, is naturally subjective. To select ﬁnal questions, test makers often try them out on students, which works only insofar as the trial-run group accurately represents the students who will ultimately take the test. •Year-to-year comparison: To prevent cheating, states typically ask test makers to create new questions every year. Test makers must then perform the tricky business of trying to ensure that the exams are equally difﬁcult so that scores can be compared like apples to apples. •Test preparation: The teaching of test-taking strategies may favor some students and keep their scores from reﬂecting what they actually know. •Distractions: Whether internal or external, distractions such as test anxiety, personal problems, lack of sleep, a sick classmate, or a broken air conditioner can distort students’ scores. •Mechanical or human error: Mistakes may occur in setting the answer key, feeding answer sheets into scoring machines, marking answers right or wrong, or other steps in the process. •Cut scores: These cutoff points for passing and advanced scores are based partly on educators’ and ofﬁcials’ judgment, so they’re subjective. Also, given the natural imprecision of scores explained in this chart, a student’s score may fall below the cutoff point for failing even if she is knowledgeable enough to pass—and vice versa. —GR TEST MAKING TEST TEST TAKING SCORING EDUTOPIA.ORG EDUTOPIA 37 http://EDUTOPIA.ORG

Table of Contents Feed for the Digital Edition of Edutopia - April 2008

Edutopia - April 2008 Contents Up Front Feedback Dispatches Sage Advice Ask Ellen Head of the Class Cool Schools Design Reinventing the Big test The Daring Dozen Heart & Soul Pop Quiz: Jack Prelutsky

Edutopia - April 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.