The after-school research debate continues. A new study (pdf) making the rounds up on Capitol Hill, purports to show the wrongness of the 2005 Mathematica study (pdf) on federally funded after school programs. Ed Week’s Viadero reports here.
There is at once less and more here than meets the eye.In the less department: The basic punchline is that if you go out and find really high quality after-school programs, they’re good for kids. Wow. I also hear it gets awfully cold in Chicago in the wintertime. The effect sizes in the new study seem a little too good to be true but I don’t think you’ll find many people that dispute the underlying premise.
Problem is, not every program is a high quality one and the federal government spends almost a billion dollars a year on after-school programs. In fact, in most of the attention to extending learning time almost no one mentions that the feds are already in that game in a big way through the after-school program and also some Title I funds (via No Child Left Behind’s tutoring provisions per Sherman Dorn’s comment below).
That’s why the Mathematica study offered more analytic leverage than this new one, it was randomized (for students at the elementary school level and schools at the middle school level). In other words a random selection of programs, not a deliberate sample of good programs. Unfortunately, Viadero covered that more like a horse race than just explaining the issue to readers. But this is education…methods…we don’t need no stinkin’ methods!
Still, program quality problems are (a) hardly surprising considering how quickly these programs came on-line and (b) not a reason to just cut the funding as many congressional Republicans want. See this paper for more on all that.
So, in the more than meets the eye department, here’s a radical idea to address two (maybe three) problems: Why not convert the entire after-school funding stream into portable vouchers (call them “coupons” or some such thing if it goes down easier) that eligible families can use to purchase after-school services either via traditional after school programs or through the various supplemental services programs that have sprung up in the wake of No Child Left Behind? At the same time, eliminate the supplemental services (eg tutoring) provisions of Title I, which are something of a disaster anyway and distract from the intended purpose of that program. But, means-test the program so that it’s aimed at low-income kids.
This would be one step to (a) get on top of the quality problem in after-school by introducing some actual competition though you’d still need some sort of certification process for providers, something that’s been a joke under the current law (b) possibly turn the supplemental services idea into a more effective one by making it a genuine “wrap-around” for at-risk kids and (c) it might be enough of a choice program to ease the demand for vouchers and, as in 2001, put together enough of a coalition to pass a reauthorization of No Child Left Behind without vouchers becoming a flashpoint. What you’d basically have would be a billion dollar program through repurposed dollars that you could target to poor kids in struggling schools to get them extra help.
By empowering parents you’d get around the fox guarding the henhouse challenges of No Child Left Behind’s tutoring provisions but school districts could also compete to serve these kids or encourage parents to pool the dollars for various programs.
The burgeoning after school industry won’t like it, nor will the public school establishment (though taking supplemental services out of Title I might ease their pain a bit), and middle class parents will be pissed if this impacts their after-school programs, but it just might be good for poor kids…
In referring to after-school funding streams, do you mean the set-aside money in Title I for SES?
Back when Matt Miller made his liberal voucher proposal, and even before, I looked at the issue from every angle hoping I could find a vouchers proposal that made sense, but I couldn’t.
But I’m still intrigued about a vouchers/coupon program for after-school or summer school, or even in pre-school.
I’m still reading the study but two patterns emerge, programs for younger students are easier and that we have better success with the less difficult problems because it is much easier to help kids who are helping themselves. We’re still stumped by the harder challenges.
On one extreme, simplistic SES efforts and enthusiatic supporters of after-school prgrams can do a serious mis-servie by trying to de-professionalize teaching. The idea that you can chop knowledge up into measurable pieces and buy some teacher-proof system to replace classroom instruction is absurd. On the other hand, we need after-school programs that emphasize enrichment and travel, and the entire human mind and spirit.
To create an effective system of after-school and summer school programs that wrap around students – as is necessary – would be nearly as much of a challenge as reforming the education we provide during the school day. It can’t be done on the cheap, and it can’t be done in bits and pieces.
That’s why I’m open to your coupon suggestion. We need the incentives that would be required by someone, and I don’t care who, to build a system, and to coordinate that system with our current system. I don’t know how you do so much system building without an architect of some sort who has the capacity to lay a proper foundation and to create communication systems. Its an absolutely huge task, and educators clearly can’t do that in our spare time.
But its a challenge we need to tackle. I hope I’m not being naive in hoping that the private sector, or more likely the Not for Profit sectors could help.
John Thompson
I was surprised by how unimpressed you were with the gains that Vandell, Reisner and Pierce reported in their study of high-quality afterschool programs – as if we really know that good programs necessarily show good results. For the past 10 years, there have been lively disagreements among afterschool, child care, and youth development leaders about whether afterschool programs should even try to address academic goals, and if they did, whether they are likely to have any effects other than to drive kids out of the programs. So when the Mathematica study showed that mediocre programs don’t have effects, the question still remained about whether well-designed and implemented programs could. Two things happened then. First, the Department of Education started to stress the importance of program quality, and started offering training and online resources to that end. Second, evaluations started to shift to a “proof of concept” model, most notably very expensive RCT study that MDRC is doing for the Institute of Education Sciences. That study is examining whether carefully developed and implemented mathematics and reading afterschool programs can have measurable effects. (Apparently an interim report from that study will be released soon.) Similarly, Vandell et al focused on mature, well-implemented projects, and children who participated for at least two years, and they found large gains on a variety of outcomes – but not reading. It will be interesting to see whether the MDRC study also shows math, but not reading, effects.