Education Week - August 6, 2014 - 23

of the reviews.
To complicate the situation further,
federal education officials'
concept of validity has evolved to
emphasize predictive ability, experts
say. It's not enough anymore
for a state to show that a test is a
valid indicator of a middle school
student's math skills; it must show
that the test is a good predictor of
whether that student is "on track" to
be college-ready in a few more years.
The field also increasingly seeks
"more sophisticated" evidence of validity,
said Ellen Forte, who served
as a department peer reviewer and
now advises states on their assessment
systems as the president, ceo,
and chief scientist at edCount, a
Washington consulting firm.
"Now [the field is] working at a
much finer grain size, going deep
into the domain and its skills," said
Ms. Forte. She isn't convinced, she
added, that the peer-review process
demands the kinds of evidence that
form "the backbone of validity."
Similarly, peer reviews have often
sought to determine a test's alignment
to standards based on whether most
of the standards are found in the as"
States
have been going through this process
for a long time, but the temperature
has been turned way up now."
ANDY SMARICK
Bellwether Education Partners
sessment items. That's a lower level of
alignment than federal officials seem
to be seeking now when they describe
high-quality assessments as measuring
deeper, more nuanced levels of student
achievement, experts say.
"[Peer review] has looked at alignment
only on a superficial level,"
said the senior-level assessment
source. "Now, defining what alignment
means will be big deal. If a
test doesn't reflect the intended
depth of knowledge of the standards,
it will be found wanting."
Alignment in Question
Several people interviewed for this
story said that if the new criteria
don't require states' tests to reflect
the writing skills in the common core,
such as citing evidence from text to
support an argument, the federal
government will, in effect, be allowing
states to use tests that aren't aligned
to those standards.
"If kids aren't writing and drawing
evidence from text, then a test
on its face isn't aligned" to the common
core, Mr. Cohen of Achieve said.
"You don't need an elaborate set of
criteria to figure that out."
Some educators question the
value of peer review in part because
states rarely experience penalties
when their tests fall short of full approval.
And many do fall short: At
one point in 2002, only 19 states'
systems met federal criteria. Between
2010 and 2012, 15 to 20
states' systems had not obtained
even conditional approval.
And while some states have had
cerns in the field as it undertakes
revisions of the peer-review process.
Many state and testing-industry
officials who were interviewed for
this story said they'd like to see the
peer-review process evolve into an
ongoing relationship of technical
support. They'd also like to see it become
more open and collaborative.
During some periods of peer review,
state officials have been allowed to
speak directly with their reviewers.
But during other periods, no face-toface
communication was permitted.
States simply received decision letters
from the federal department.
Mr. Erpenbach said that when he
was allowed to sit down and talk with
state officials during the process, he
was often able to resolve many issues.
Some in the assessment world
have worried privately that the
Education Department's new criteria
might set forth requirements
that only its own grantees-parcc
and Smarter Balanced-could meet.
That would pose big problems, since
nearly half the states plan to use
other tests in 2014-15.
"It's really important in this process
that we stay open to other solutions
that could meet the criteria,"
said Chris Minnich, the executive
director of the ccsso, which co-led
the common-standards initiative.
Scan this tag with your
smartphone to read two
white papers that the
Education Department
considered in designing new criteria
for peer review of assessments.
www.edweek.org/links
EDUCATION WEEK | August 6, 2014 | www.edweek.org | 23
PROFESSIONAL DEVELOPMENT AT HARVARD
The Future of
Independent
Schools: Leading in
a Changing Context
OCTOBER 15-17
Making Caring
Common: Promoting
Caring and Respect
in Students, and
Preventing Bullying
and Behavior Problems
NOVEMBER 12-14
to submit to compliance agreements,
few have ever paid the ultimate
penalty for unapproved standards
or tests: forgoing a portion of their
federal Title I administrative funds.
Need For Expertise
In addition to concerns about a
lack of outside input in developing
the forthcoming criteria, many in
the assessment field are worried
that the Education Department
currently lacks the right kinds of
expertise to craft good criteria for
assessments. Key staff members
with backgrounds in measurement
or large-scale assessment, such as
Carlos Martinez and Sue Rigney,
who oversaw peer review in recent
years, have retired or changed jobs
within the department.
"The department is now in a place
where it's far less capable" of designing
the right criteria and supporting
states in building good testing systems,
said Ms. Forte, the edCount
executive.
Department officials did not respond
to a request for comment on
that question of capacity, or on conNot
a classroom experience;
a life experience.
The Leading Edge
of Early Childhood
Education: Linking
Science to Policy for
a New Generation
of Pre-K
DECEMBER 12
For a complete list of 2014 programs, visit
www.gse.harvard.edu/ppe/transform
http://www.edweek.org/go/21Trends http://www.edweek.org/go/21Trends http://www.gse.harvard.edu/ppe/transform http://www.gse.harvard.edu/ppe/transform http://www.edweek.org/links http://www.edweek.org
Education Week - August 6, 2014

Table of Contents for the Digital Edition of Education Week - August 6, 2014
