Washington Monthly - September/October 2021 - 96

A NOTE ON METHODOLOGY:
4-YEAR COLLEGES
AND UNIVERSITIES
T
o establish the set of colleges included in the rankings,
we started with the 1,550 colleges in the 50 states that
are listed in the U.S. Department of Education's Integrated
Postsecondary Education Data System (IPEDS) and have
a 2018 Carnegie basic classification of doctoral, master's, and
baccalaureate colleges, are not exclusively graduate colleges,
participate in federal financial aid programs, and had not announced
an impending closure as of June 15, 2021. We then
excluded 28 colleges with fewer than 100 undergraduate students
in any year they were open between fall 2017 and fall
2019 and an additional four colleges with fewer than 25 students
in the federal graduation rate cohort in 2018 and 2019.
Next, we decided to exclude the five federal military academies
(Air Force, Army, Coast Guard, Merchant Marine, and
Navy) because their unique missions make them difficult to
evaluate using our methodology. Our rankings are based in
part on the percentage of students receiving Pell Grants and
the percentage of students enrolled in the Reserve Officers'
Training Corps (ROTC), whereas the service academies provide
all students with free tuition (and thus no Pell Grants or student
loans) and commission graduates as officers in the armed
services (and thus not the ROTC program). Finally, we dropped
an additional 47 colleges for not having data on at least one
of our key social mobility outcomes (percent Pell, graduation
rate, net price, or the number of Pell recipients earning bachelor's
degrees). This resulted in a final sample of 1,466 colleges
and includes public, private nonprofit, and for-profit colleges.
In the face of changing data availability, we assembled a
metrics advisory group of seven higher education experts to
advise us on updating our main rankings. The board consisted
of Fenaba Addo of the University of North Carolina at Chapel
Hill; Beth Akers of the American Enterprise Institute; Michael
Itzkowitz of Third Way; Konrad Mugglestone and Eleanor Eckerson
Peters of the Institute for Higher Education Policy; and
Nicole Smith and Martin Van Der Werf of the Georgetown University
Center on Education and the Workforce. Our changes
to this year's rankings reflect these conversations and the advisory
group's focus on equity and value for all students.
Our rankings consist of three equally weighted portions:
social mobility, research, and community and national service.
This means that top-ranked colleges needed to be excellent
across the full breadth of our measures, rather than excelling
in just one measure. In order to ensure that each measurement
contributed equally to a college's score within any given category,
we standardized each data element so that each had a
96 September/October 2021
mean of zero and a standard deviation of one (unless noted).
Missing social mobility data (affecting less than 1 percent of all
observations) was imputed and noted with " N/A " in the rankings
tables. We adjusted data to account for statistical outliers
by allowing no college's performance in any single area to exceed
five standard deviations from the mean of the data set.
All measures (unless noted) use an average of the three most
recent years of data in an effort to get a better picture of a college's
performance rather than statistical noise.
The social mobility portion of the rankings also doubles
as our Best Bang for the Buck rankings, with the exception
that the main rankings are by Carnegie classification while the
Best Bang for the Buck rankings are by region (while predicted
rates are calculated by Carnegie classification). We used a
college's graduation rate over eight years for all students instead
of the first-time, full-time graduation rate that is typically
used but presents an incomplete picture of a college's success.
This graduation rate counted for 16.66 percent of the social
mobility score. Half of that score was determined by the
reported graduation rate and the other half came from comparing
the reported graduation rate to a predicted graduation
rate based on the percentage of Pell recipients, the percentage
of students receiving student loans, the admit rate, the racial/
ethnic and gender makeup of the student body, the number
of students (overall and full-time), and whether a college is primarily
residential. We estimated this predicted graduation rate
measure in a regression model separately for each classification
using average data from the past three years, imputing for
missing data when necessary. Colleges with graduation rates
that are higher than the " average " college with similar stats
score better than colleges that match or, worse, undershoot
the mark. A few colleges had predicted graduation rates over
100 percent, which we then trimmed back to 100 percent.
We used IPEDS data comparing graduation rates of Pell
and non-Pell students to develop a Pell graduation gap measure.
Colleges that had higher Pell than non-Pell graduation
rates received a positive score on this measure, which was
based on just the one year of available data and counted for
16.66 percent of a college's score. We included the number of
Pell recipients earning bachelor's degrees, which is designed to
reward colleges that successfully serve large numbers of students
from lower-income families. This measure, from IPEDS,
counts for 8.33 percent of the social mobility score.
We also used IPEDS data for the percentage of a college's
students receiving Pell Grants to get at colleges' commitments
to educating a diverse group of students. We had to drop our
previous first-generation student measure because the College
Scorecard has not updated the data in several years. Our
measure compared actual shares of Pell students to the predicted
share after controlling for ACT/SAT scores and the share
of families in a state with incomes below $35,000 and between
Washington Monthly - September/October 2021

Table of Contents for the Digital Edition of Washington Monthly - September/October 2021
