Washington Monthly - September/October 2020 - 106

A NOTE ON METHODOLOGY:
4-YEAR COLLEGES
AND UNIVERSITIES
o establish the set of colleges included in the rankings,
we started with colleges in the 50 states that are listed
in the U.S. Department of Education's Integrated Postsecondary Education Data System (IPEDS). Then, we winnowed
that list to include only the schools that have a 2018 Carnegie
basic classification of doctoral, master's, and baccalaureate colleges; that are not exclusively graduate colleges; that participate in federal financial aid programs; and that had not announced a fall 2020 closure as of July 1, 2020. That left us with
1,558 institutions. We then excluded 25 colleges with fewer
than 100 undergraduate students in any year they were open
between fall 2016 and fall 2018 and an additional four colleges
with fewer than 25 students in the federal graduation rate cohort in 2017 and 2018.

Next, we decided to exclude the five federal military academies (Air Force, Army, Coast Guard, Merchant Marine, and Navy)
because their unique missions make them difficult to evaluate
using our methodology. Our rankings are based in part on the
percentage of students receiving Pell Grants and the percentage of students enrolled in the Reserve Officers' Training Corps
(ROTC), whereas the service academies provide all students
with free tuition (and thus no Pell Grants or student loans) and
commission graduates as officers in the armed services (and
thus not the ROTC program). Finally, we dropped an additional
55 colleges for not having data on at least one of our key social
mobility outcomes (percent Pell, graduation rate, net price, or
the number of Pell recipients earning bachelor's degrees). This
resulted in a final sample of 1,469 colleges and includes public,
private nonprofit, and for-profit institutions.

Our rankings consist of three equally weighted portions:
social mobility, research, and community and national service.
This means that top-ranked colleges needed to be excellent
across the full breadth of our measures, rather than excelling
in just one. In order to ensure that each measurement contributed equally to a college's score within any given category, we
standardized each data element so that it had a mean of zero
and a standard deviation of one (unless noted). Missing social
mobility data (affecting less than 1 percent of all observations)
were imputed and noted with "N/A" in the rankings tables. We
adjusted data to account for statistical outliers by allowing no
college's performance in any single area to exceed five standard deviations from the mean of the data set. All measures
(unless noted) use an average of the three most recent years
of data in an effort to get a better picture of a college's performance rather than statistical noise.

106 September/October 2020

The social mobility portion of the national rankings also
doubles as our Best Bang for the Buck rankings, with the exception that the main rankings are grouped by Carnegie classification while the Best Bang for the Buck rankings are grouped by
region (while predicted rates are calculated by Carnegie classification). We again used a college's graduation rate over eight
years for all students instead of the first-time, full-time graduation rate that is typically used but which presents an incomplete picture of a college's success. We also created a predicted
graduation rate, based on student demographics, so we could
compare the school's graduation rate performance relative to
schools with similar student bodies. The predicted graduation
rate measure was based on the percentage of Pell recipients
and first-generation students, the percentage of students receiving student loans, the admit rate, the racial/ethnic and gender makeup of the student body, the number of students (overall and full-time), and whether a college is primarily residential.
We estimated this predicted graduation rate measure in a regression model separately for each classification using average
data from the past three years, imputing for missing data when
necessary. Colleges with graduation rates that are higher than
the "average" college with similar stats score better than colleges that match or, worse, undershoot the mark. A few colleges had predicted graduation rates over 100 percent, which we
then trimmed back to 100 percent.

The social mobility score takes these two metrics into account. The actual eight-year graduation rate accounts for 8.33
percent of the social mobility score, and the difference between the predicted versus the actual graduation rate counts
for another 8.33 percent. We used IPEDS data comparing graduation rates of Pell and non-Pell students to develop a Pell graduation gap measure. Colleges that had higher Pell than nonPell graduation rates received a positive score on this measure, which was based on just the one year of available data
and counted for 16.66 percent of a college's score. We also included the raw number of Pell recipients earning bachelor's degrees, which is designed to reward colleges that successfully
serve large numbers of students from lower-income families.
This measure, from IPEDS, counts for 5.56 percent of the social
mobility score.

To gauge a college's commitment to educating a diverse
group of students, we used IPEDS data to measure the percentage of students at each institution receiving Pell Grants,
and College Scorecard data to measure the percentage of
first-generation students at each school. Our measure compared actual shares of Pell and first-generation students to
the predicted share after controlling for ACT/SAT scores and
the share of families in a state with incomes below $35,000
and between $35,001 and $75,000 per year. The Pell enrollment performance measure counted for 5.56 percent of the
social mobility score, while the first-generation enrollment

Washington Monthly - September/October 2020

Table of Contents for the Digital Edition of Washington Monthly - September/October 2020