The world of education data analysis is generally pretty boring sedate. It doesn’t normally feature the kind of excitement associated with, say, the rolling out of a big-budget sequel to a Hollywood blockbuster. But there was a touch of that energy yesterday with the release of a new report from U.S. Department of Education researcher Clifford Adelman, titled Toolbox Revisited: Paths to Degree Completion from High School Through College.
The report is essentially a sequel to a 1999 Adelman report called Answers in the Tool Box, which analyzed a nationally representative sample of students tracked over more than a decade from high school through college. Tool Box I contained a huge amount of smart, interesting analysis, but one finding stood out: the intensity of a student’s high school curriculum, measured by the number and difficulty of courses taken, is the single most important pre-college factor in determining their likelihood of graduating from college. More than race, income, or even grades and test scores, curricular intensity—particularly in math—matters most.
Those findings proved to be hugely influential. A host of high-profile trends, analyses, and policy initiatives—from the rapid rise in AP and IB coursetaking and Newsweek’s annual AP-based list of 100 best high schools to the American Diploma Project and the growing number of states moving to establish college prep as the default high school curriculum—have used the Tool Box analysis for empirical support.
Toolbox Part Deux replicates and improves upon the first report, using a more recent longitudinal data set from the 1990s. Thankfully, it doesn’t suffer from the standard sequel letdown. Equally chock-full of insightful data tables and timely perspectives, it confirms the original findings: academic intensity in high school has the greatest impact on college success.
The report also has much to say about the college experience. Earning enough credits in the first year is crucial; students who earn less than 20 credits as freshmen are far less likely to graduate, even if they return for a second year. Students who repeat or withdraw without penalty from 20 percent or more of their courses have their likelihood of graduating cut in half. Delayed entry to college and part-time attendance are negatives and continuous enrollment matters a lot: students who leave for more than a short amount of time are at significant risk of never coming back.
Adelman uses his empirical findings as the basis for some arguable opions about policy: like many in higher education, he’s too quick to dismiss institutional graduation rates as a meaningful measure of success.
But overall the Tool Box series should serve as an object lesson for data geeks and policy analysts everywhere: if you combine a rock-solid data source with smart, sophisticated analysis and translate those findings into a clear message that connects with a timely policy problem, you can, through sheer force of evidence, change the world around you. Hopefully, further sequels are to come.
– Kevin Carey