Regression discontinuity analysis, or Why I Don’t Care What School My Baby Goes To

15 Mar

Last summer a paper by Abdulkadiroglu, Angrist, and Pathak (yes, now that’s a mouthful) demonstrated the effects of exam schools on test score outcomes. In NYC, exam schools are like Stuyvesant and Bronx Science; they are public, but students need to test into them. In Boston, Boston Latin in the oldest and most well-known exam school. They are all wildly competitive to get into.

What they wanted to know is, are higher scores at exam schools due to the schools being awesome, or are the higher scores due to the schools’ selectivity in admitting students. Are they awesome because they shepherd through already-awesome students, or are they awesome because they are awesome-making factories of awesome?

The authors used a regression-discontinuity method to investigate the effects of schools on test scores. That is, they looked at students who were just above the minimum score required for admission to an exam school, and compared them to students who were just below that score requirement. Since this is a continuous variable, we expect that people on just either side of that line should be quite similar. But since there is a large difference in outcomes for those tests (admission to, or rejection from, exam school), there is a discontinuity of ‘treatment.’

The upshot is that there is virtually no difference between those two groups of students. And if the awesome school had an effect on scores, we should see students who went to the exam schools doing better than those who did not.

So, this means a few things; and it suggests some other things. It demonstrates that for those who are marginally good enough to qualify for an exam school, you would do no better at an exam school than you would at a public school, measured by test scores as a proxy for educational achievement. The authors suggest that other data points to not giant differences between ‘marginal’ exam school students and the rest of the exam school students, but they don’t show this directly. And, the best students at exam schools test higher than the best students at public schools. We just don’t have any evidence that the school is responsible for those scores.

The work, in combination with other anecdotal data suggests that exam schools cater to their best students – to allow the best students to do great at things like the Intel Science and Engineering Fair and other competitions that matter for small numbers at the high end.

I would also strongly suggest that the ‘best’ schools, how do I say this delicately, allow your little one to mix with other type-A elite-seeky students and their families. I’m going to call this a mixed bag at the moment, since I know that in the back of their minds, people are thinking ‘networking!’ But there’s a healthy dose of fuck you in my assessment of the worth of that.

At the end of the day, as Felix Salmon notes, you would be better off spending the money you’d spend in school fees for more books, more trips to museums and interesting places. You could buy a Solar Stirling Engine! Or a high-grade chemistry set! Or a year’s membership to the MoMA!

But I am sorry, but I just can’t bring myself to get worked up about what school baby goes to. Maybe he’ll go to fancy school. Maybe he’ll go to schlumpy public school. But the dirty little truth is that for us, with our resources and educations and emphasis on education, it just doesn’t matter.

Advertisements

4 Responses to “Regression discontinuity analysis, or Why I Don’t Care What School My Baby Goes To”

  1. Alan March 17, 2012 at 12:31 pm #

    Thank you! It’s hard to believe that with a 16-month old, this debate has already started in my house. But it’s nice to have data. My lovely wife is very naturally getting sucked into some of the NYC-craziness around the importance of a good preschool…where my natural instinct is that our precious little explorer will do “awesomely” at whatever school she goes to. In fact, she’ll single-handedly make the school MORE awesome…

  2. Davin March 18, 2012 at 5:59 am #

    Hmmmm.
    So you wanna avoid Type-A elite-seeky pricks, eh? Who could blame you? Buncha jerks, anyway.
    The data suggests that you’re fine doing so, but unfortunately the process starts way before that Stuyvesant exam, as Alan appropriately points out. In fact, heavy as it makes my heart to point it out, you’re probably going to find it impossible to avoid in your current neighborhood. Sigh.

    If only there were some other, more pleasant and laid-back community in which your little bundle of hopes and dreams could spend these so very formative years. A relaxed place, conveniently accessible to Manhattan via express trains, with top rated public primary schools and more ethnic diversity than any other county in the US. Sigh. If only…

  3. Davin March 18, 2012 at 6:05 am #

    Oops, sorry – accidentally misstated my case a little there. We probably shouldn’t think of Queenz as the most ethnically diverse county in the US.
    In point of fact, Queenz is actually the most ethnically diverse urban area in the world.

    I apologize for the error.

  4. Ivy March 19, 2012 at 1:45 pm #

    Even if my husband is annoying, at least he’s amusing ;-).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s