« Consumer surplus is what you make of it | Main | Peccavi nimis cogitatione, verbo et opere: mea culpa, mea culpa, mea maxima culpa »

A stitch in time . . .

13 May 2008 07:09 pm

Ryan Avent responds on teacher's unions:

Things in New Orleans have improved. But they remain terrible. What’s more, too many institutional factors changed for us to have a good idea what generated the improvement. And statisticians out there might note that when tracking changes over time, it helps to keep the sample constant. For an economist to look at a city’s educational system, subtract a quarter of a million poor people, then look at it again and suggest that destroying the teachers’ unions made all the difference is…well it’s not exactly a rigorous analysis.

For my money, the best new research on this subject emphasizes the role of parental skill levels in achievement and the importance of investment in disadvantaged children when they’re young. Union busting is a waste of time; it’s like changing the oil on a car missing a wheel and hoping for huge performance improvements.

I agree that there's a sample problem, but it also seems that more kids in New Orleans now are qualifying for free lunch than did before, so I'm skeptical that this explains the change. Also, the test scores improved from 2007 to 2008. And the pattern of improvement--strongest in the younger grades--is what you'd expect if the school were the major factor rather than the demographics.

I'm familiar with the research on parental skills and early childhood intervention. I just don't know what to do with it. So far I have not seen a single successful early childhood intervention that is even arguably scaleable: you're talking about intensively monitored programs using top-notch personnel, all of whom are deeply committed to the project's goals and procedures. The longest data set we have is, AFAIK, the Perry Pre-School Project, which produced exceedingly modest gains for a pricetag of more than $20,000 per child per year in today's dollars. Again, that's with a small program of highly committed staff.

What we got instead was Head Start, which produces small gains that most evidence suggests disappear a few years after the kids exit the program. Even if we wanted to do Perry Pre-School, or something even better, nationwide, where would we find the staff? Pre-K sounds great, but it's very likely to be slightly glorified baby sitting outside of affluent school districts that don't need it in the first place.

Since we're not (I hope) going to take kids out of the disadvantaged homes they are born to, the schools are what we're stuck with. And there are programs that work--at least, better than what we have now. They just bore the hell out of the teachers.

I'm not against early childhood intervention, if it works. But it's not going to save us from having to teach the kids better in grades 1-12.

TrackBack

TrackBack URL for this entry:
http://meganmcardle.theatlantic.com/mt/mt-tb.cgi/18642

Comments (36)

And there are programs that work--at least, better than what we have now. They just bore the hell out of the teachers.

Again, do you have any evidence of these sweeping statements? You make one after another, all insulting.

What we got instead was Head Start, which produces small gains that most evidence suggests disappear a few years after the kids exit the program

This NYTimes article says that Head Start works, and has lasting impact. (Plus you do not address the additional services provided by Head Start such as medical care or parenting classes, whose effect aren't measured later.) After reading the article, which claims careful assessment proves Head Start does help, I assume you are looking at studies that don't closely follow the students afterwards, but how are we to know?

Pre-K sounds great, but it's very likely to be slightly glorified baby sitting outside of affluent school districts that don't need it in the first place.

On what information do you base this statement?

Again, you do not provide evidence that "programs that work" "bore the hell out of the teachers." You are saying teachers don't want to be successful--they don't want their students to learn, or want to get pay raises, or good performance reviews, or help kids learn. That's just insulting.

See: Freakonomics, Levitt & Fryer, Waldfoegel, the HHS follow up survey, for starters.

Freakanomics tells us teachers are too lazy to use effective methods?

I am all astonishment.

There is no subject about which you are more thoroughly and laughably incorrect than education, and a large part of your problem is this: you assume educational output=educational input. This casual assumption that a child's educational performance is totally or even significantly the product of who is teaching him is just flat wrong.

I posted this on the Marginal Revolution discussion thread, and its relevant here as well since it seems to undermine your assumption about the demographic changes after Katrina. There are serious sample selection issues, and I find it astonishing that bloggers grounded in positivist research are exploiting this story to fuel opposition against teachers' unions. Even though improvements occurred between 2007-2008, the counterfactual is modus irrealis--that is, would these improvements have occurred if Katrina had not led to the dissolution of the teacher's union. The removal of poor, underperforming under-city students will almost certainly accentuate gains in achievement scores, irrespective of system-level structural changes.

Here are some data from the 2007 Census for New Orleans.

http://www.brookings.edu/reports/2007/07katrinafreysinger.aspx

Takeaway:

"B. New Orleans’ 2006 post-storm population was smaller, older, more educated, less poor, with fewer renters, and fewer households with children than was recorded in Census 2000."

And:

"C. Compared with “stayers” in the city of New
Orleans, out-migrants were younger, poorer,
more likely to be black, and more likely to
have children."

There is some danger in extrapolating from aggregate characteristics to school enrollment patterns, but this appears to be the most comprehensive data available, since NO school district is not required to report demographic and achievement data for state and federal accountability purposes.

From the data, it appears as though the demographic makeup of the suburbs was largely undisturbed. This would provide a nice research design for analyzing the true effects of the variables extolled by school-choice advocates.

IMHO, it is absolutely impossible to do any sort of large comparisons between pre and post Katrina schools in New Orleans. Far too many things have changed so there are many, many variables.

For one thing, there are three schools systems now, instead of one. It is not just the teacher's union that is gone in two of the cases. Many, many things are gone and some things are new.

Perhaps the biggest change is akin to dreaded tracking. Motivated parents have gotten their motivated students into charter schools. The Recovery School District, at least, now has a motivated and presumably far less corrupt administration that OPPS pre-K. There is also, I believe, more order and discipline in the RSD schools -- fewer attacks on teachers and other students. That must help.

The remaining OPPS schools were actually the best performing of a poor lot pre-K. I think the unions are still represented there.

There has been some overall demographic shift also, as alluded to above, with many of the very poorest parents and students not returning.

The grand experimentation in the inner city schools is one of the most results of the storm.

Until we acknowledge that equal outcomes at the lowest common denominator level are not an acceptable result, the rest of the issues are merely a distraction.

"Union busting is a waste of time; it’s like changing the oil on a car missing a wheel and hoping for huge performance improvements."

Perhaps. But the constant calls of "more money! more money! more money!" are like pouring more and more gas into the tank and insisting that it will start moving any day now.

I don't see anyone can fail to realize the impact the unions are having. Firing a public school teacher is a long, time-consuming, money-draining process. Is anyone surprised that the quality suffers? It only takes one bad teacher to ruin the work of ten goods one, especially in disicplines like math and science where knowledge is built over many years.

Freddie and Susan --

Megan is talking about Direct Instruction. Use Google, and look up the research that was done on its effectiveness (particularly "Project Follow Through").

Megan doesn't have to reinvent the wheel in every post that mentions education; what a tedious blog it would be, if every single post had to assume that all readers needed to be led by the nose as to every fact ever mentioned.

There is one type of preschool intervention which is actually effective: parental. A small child has imprinted their parents and will do whatever they can to please mom and dad, but is poorly motivated when their teacher is anyone else. So you can teach your kid to read at about age three if you get rid of the tube, pay attention, and read to them. The trick is to use phonetic readers with carefully controlled vocabulary. You can't buy these at Borders or Barnes & Noble, so try googling 'phonics practice readers' on Amazon. Due to the age of the child, this will only work for a parent. I hypothesize that failure to use the parent in the role of teacher explains why programs such as Headstart have mixed results. It doesn't matter how much money you spend if you can't get the parents to do their job, which is to teach and to motivate their child by example. My wife and I have taught our three children to read using this method and I can attest to it's effectiveness. Armies of union controlled, highly trained, lavishly paid, government certified, committed teachers are, and always will be, ineffective without parental involvement in a teaching role.

If you can't get the parents to cooperate, you are likely wasting your time. It's too bad, but if wishes were horses then beggars would ride.

When my oldest boy was five, we visited one of California's state parks. He was bored and busied himself by reading aloud a poster, in front of the ranger's office, describing the park (he couldn't read silently at that age). A passerby, thinking his performance remarkable, asked me where he had learned to read, then added as subtext, 'probably not in a public school'. I replied 'no, not in a public school'. She kind of sighed and added 'I know, because I'm a teacher'.

Freddie and Susan --

Megan is referring to "Direct Instruction." Google that phrase, along with "Project Follow Through," and you'll see what she's talking about.

Megan isn't required to reinvent the wheel with every single post, you know. Bloggers are entitled to assume that readers are generally educated, and it would be extremely tedious if every post had to assume that readers had never learned anything from any previous discussions.

"This casual assumption that a child's educational performance is totally or even significantly the product of who is teaching him is just flat wrong."

So does that mean that there are no good teachers and bad teachers? If that is the case, it would seem like we are paying teachers too much, not too little, since your assertion is that it doesn't matter who is teaching kids.

On it's face, that seems rather absurd, and seems a counter argument to teachers that complain that they are working too much because they want to make a difference. In short, what you are saying is that teachers don't matter, and shouldn't bother putting in extra effort.

Here's a little selection from a book a NYC teacher wrote about teaching and the union here, 'twas enough for me.

"This casual assumption that a child's educational performance is totally or even significantly the product of who is teaching him is just flat wrong."

Bizarrely, this was exactly the argument the NY teachers union just made while successfully lobbying the state legislature to pass a law that makes NY the first and only state where it is now illegal for school administrators to consider the test scores of a teacher's students when deciding whether to grant the teacher tenure. (As to the rest of that tenure granting process, see the link in my prior comment).

Yes, they actually argued in public that teachers don't affect students' measurable performance, in getting this law passed over the howling objections of Bloomberg, the NYC Schools Chancellor and other school district heads.

Of course, that was after they negotiated a new conract giving senior teachers pay of over $100,000 for 180 work days a year (plus far better than private sector benefits, etc.) on the strength of how greatly important teachers are to student success.

Which illustrates an odd thing about teachers unions: they prefer to get what they want through legislation rather than collective bargaining. Especially when they can't get it through collective bargaining. (That way they don't have to worry about ever having to give it back in some dang future "negotiation".)

The NY teachers union is the top spending lobbying group in the state, and while its students of course are centered in NYC it has a beautiful headquarters office building 100+ miles away in Albany, right across from the state capital building. A no doubt wise investment of the money it gets from the taxpayers.

Probably, in fact, the single most effective investment in the entire NY school system.

Some anecdotal evidence from when I was helping out with church efforts right after Katrina. Most of the Louisianans who arrived in my town were at least in the middle class. They were the ones who had cars, credit cards to buy gas and enough foresight to drive out of town before the hurricane hit. Without exception all of the Louisianans with children were amazed by how much better the Texas schools were than what they had back home. This includes folks who had been sending their kids to Catholic schools in New Orleans who's kids were now in public schools in Texas.

The unscientific conclusion that I took from this admittedly limited sample was that schools in Louisiana must have really sucked.

Freddie: There is no subject about which you are more thoroughly and laughably incorrect than education, and a large part of your problem is this: you assume educational output=educational input. This casual assumption that a child's educational performance is totally or even significantly the product of who is teaching him is just flat wrong.

If you are correct in this assertion, that would seem to argue that it is financially irresponsible to pay teachers any more than minimum wage. The current system of requiring college degrees and education courses then increasing pay based on seniority and taking of additional courses seems to be a grotesque waste of tax dollars when the appropriate pool of job applicants would seem to be fast food workers who are bothered by the smell of fryer grease.

One of the few factors that professional educators organizations will admit to affecting educational outcome is classroom size. Since children's educational performance is not significantly the product of who is teaching them, we would seem to get better results if we had small classes of a dozen or fewer children and taught them with minimum wage former carpenters and mortgage loan officers fresh from the unemployment office.

I posted this on the Marginal Revolution discussion thread, and its relevant here as well since it seems to undermine your assumption about the demographic changes after Katrina.

Until you post direct evidence about public school attendees, your indirect evidence about New Orleans as a whole doesn't undermine direct evidence from federal free lunch statistics.

There are lots of people in New Orleans who don't use the public schools -- too old, no kids, rich enough to put their kids in private school.

We have direct evidence of the poverty of NOLA school attendees -- the federal free lunch program. Indirect evidence is much less useful.

Again, you do not provide evidence that "programs that work" "bore the hell out of the teachers." You are saying teachers don't want to be successful--they don't want their students to learn, or want to get pay raises, or good performance reviews, or help kids learn. That's just insulting.

You're right - 'programs that work' don't necessarily bore the hell out of teachers; they bore the hell out of administrators and professors of education. They then get blocked because they contradict the ideology taught at education schools.

There was, in fact, one controlled, long-term experiment educational experiment that examined which pedagogies worked, and it produced results which were significant, consistent, and replicable. The problem is that the results got buried by the educational bureaucracy.

Where are the data on free/reduced price lunch participation for New Orleans post-Katrina? As I caveated in my post, I'm aware of the ecological inference fallacy. However, I have found no data on NSL participation in New Orleans post-Katrina, since it appears as though they have been absolved from reporting accountability data.

Also, there is some evidence that an increase in school lunch participation is an artifact of Katrina, since they expanded the program to assist survivors. This may artificially inflate the number of participants, if its reported anywhere. These documents are a bit old, so its unclear whether the expansion has been discontinued.

http://www.fns.usda.gov/cnd/Governance/Policy-Memos/Reissued/2005-12-06-R.pdf#xml=http://65.216.150.153/texis/search/pdfhi.txt?query=katrina&pr=FNS&prox=&sufs=&order=r&mode=&opts=&cq=2&sr=&id=4592c355df

"This casual assumption that a child's educational performance is totally or even significantly the product of who is teaching him is just flat wrong."

So does that mean that there are no good teachers and bad teachers? If that is the case, it would seem like we are paying teachers too much, not too little, since your assertion is that it doesn't matter who is teaching kids.

On it's face, that seems rather absurd, and seems a counter argument to teachers that complain that they are working too much because they want to make a difference. In short, what you are saying is that teachers don't matter, and shouldn't bother putting in extra effort.

Posted by Jon N

Sigh. No, not at all, and the fact that you won't bother to come up with the trivially obvious reason says something about you. Ever hear of the aphorism that you can lead a horse to water, but you can't make him drink? That's the problem. I've given F's to very bright students, and A's to some rather dull ones. The difference was the amount of work each put into the course.

Now, if the students put the same amount of effort into the class, there is a clear difference between a good teacher and a bad one. But for any particular student? That's a different story.

If performance isn't "even significantly the product of who is teaching him" then student effort and motivation fall into that performance.

If this is the case, then teachers unions are merely rent-seekers exploiting their monopsony powers to extort money from the public.

So, Skullberg thinks that if the student doesn't do his homework, the parents don't get on their kids to do their homework, in fact, tell the teacher that they don't see why their kids have to learn this stuff, the teacher is not allowed to drop the kids who do this from their class, in fact, despite all this, and are under pressure to pass the kids even though they are failing - under pressure from the parents and the administration, mind you - then Skullberg thinks that this is somehow the teacher's fault.

What a loon. And dishonest as all get out. Again.

Did the Houston school district see a corresponding decrease in the educational gains of similar age cohorts? It's my understanding that the majority of the relocated families settled in Houston, so if the gains in New Orleans are due to the marginal learners leaving there should be an offsetting trend to be found in Houston.

Excellent question.

http://www.chron.com/CDA/archives/archive.mpl?id=2006_4177945

I can't seem to find the raw data, and comparisons are fraught with difficulties--accountability systems differ across states, the displacement trauma surely affects student performance--but this provides tentative evidence that those students displaced by Katrina, at least to Houston, were already struggling academically. Without student-level data, drawing sweeping conclusions would be foolish.

SoV,

You just attributed a lot of things to me that I've never claimed. I think performance is "significantly the product of who is teaching him" along with a host of other factors. The argument that we can't judge teachers because their contribution is noise in the system is laughable, and belittle to educators of all stripes.

I'm not the caricature you have in your head, no matter how much you hate that person. Family, culture, lifestyle, peer group, school administration, neighborhood, environment, and faculty all play significant roles in a child's education. There are tons of things wrong with the US educational system, and a ton of things wrong with society that worsen the problem.

As with welfare, there are people who just refuse to make difficult or intelligent choices (not all, but some) and will be dependent on the dole no matter what you do, that doesn't mean we don't work welfare to try and move the marginal cases into self-sufficiency. There are bad situations, bad parents and the like, but instead of giving up, we should attempt to make the educational experience maximally effective despite those outside influences.

No, that's exactly what you claimed:

If performance isn't "even significantly the product of who is teaching him" then student effort and motivation fall into that performance.

The top three(four predictors) of academic outcomes are 1)parental involvement, 2)income and profession of parent(s), 3)neighborhood composition.

Give two teachers kids whose parents are involved in their education, parents whose success is very much dependent on their level of education, and live in a community of like-minded people in a nice bit of suburbia, then there will be a noticeable difference between good and bad teachers.

Give an excellent teacher a child whose parents are uninvolved in their school activities, parents who are dismissive of education in general, parents who live in a burned-out inner-city area, then I can comfortably guarantee that this student will do much worse than a student who has a poor teacher but lives in the first scenario.

But somehow, according to you, this is _still_ on the teacher to succeed, and if they can't motivate their students to do well, with absolutely no authority to enforce anything, well, it's still on them.

If that's not what you were writing up above, then you've better explain what you meant by that comment. I don't know how else to take it.

Teachers are partly responsible for "student effort and motivation," that is exactly what I said and what I meant.

The false dichotomy you set up is laughable. I believe the following things have meaning impacts on student performance :

-Parental involvement
-Student intelligence
-Student behavior / personality
-Homelife / stability
-Teachers
-School Administrators
-Peer group

I think removing teacher's impact entirely is belittling to them, since I've seent he impact good and bad teachers have had on the same student.


You will also note, I've never discussed teacher's authority or disciplinary role. That falls on the teachers and administrators, but I never said I endorsed the status quo. Further, I never said I was happy with the current setup in any way.

Also, and this may back you away from the edge here, saying teacher's impact student performance DOES NOT mean teachers are the ONLY impact on student performance. Good or great teachers cannot overcome certain behaviors and bad teachers may not hold back the smart kids, but that isn't to say that good teachers don't help the marginal kid more than bad ones.

Using the bottom (third/quarter/fifth/tenth) of the class as a comparison isn't helpful, since the lack in other areas can a=outweigh other gains. Similarly, using the top group doesn't show much either. Using that middle group, teacher's can affect performance, and good ones can affect it more.

You have somehow made my position that all school performance is based on the teacher you get, I don't know where you got that from, but it has never been my position.

Teachers are partly responsible for "student effort and motivation," that is exactly what I said and what I meant.

Yeah. Right. You're saying that if the kid doesn't turn in his homework, it's on the teacher.

That's contemptible, and designed to put teachers in a no-win situation.

NO. It is not my responsibility to see that the kids do their homework. I assign it, I collect, I grade it, I hand it back, along with comments and examples on the board when I see enough people making the same type of error. That's it.

I don't remonstrate with them to do their homework, I don't beg them, I don't offer incentives, I don't coddle, swaddle, wheedle , plead or whine.

It's not my job. That's on them and their parents.

But you and yours want to push that off on me, make it my responsibility - even if I zero authority or power to compell them to do so. No, I don't think anyone wonders why I question your sincerity.

SoV,

But you could make your class so interesting the marginal kid decides to do his homework. We're not asking you to make the horse drink the water, but to lead the horse to a nice stream instead of brackish pool with slime on it.

You're saying that if the kid doesn't turn in his homework, it's on the teacher.

I AM NOT! Did you miss the "partly" in the middle of that sentence?

By your recount of your job, we could replace you with a website, syllabus, scantron machine and a semi-decent computer program.

Also, the authority issue is another one entirely, so bringing it up here is pointless. If you want more authority over discipline and motivation, take that up with your union rep. Don't wine that you don't have any, since your union agreed to it.

You really are way too close to this, you can't see objectively at all, and so you take every hint that somehow, maybe, teachers affect education as some sort of vile smear against you and a conspiracy to degrade and deride teachers. Chill out.

Aaron, I do try to make the material interesting. In fact, it is interesting. And I try to communicate this as much as possible to kids. 'Hey, polynomials can approximate _any_ good function arbitrarily closely at a given point.' 'And so, _any_ finite set that is closed with respect to a defined operation _must_ be a group.' And so on and so forth. I write up extra notes in pretty colored inks showing three-D demand and supply surfaces, printed up at my own expense (color costs extra.) I look for cool math applications in the business sections of the papers I read.

And I get excellent reviews. And students telling me I'm the best math teacher they've ever had.

But I wouldn't call that 'motivation'. Motivation to do what? Learn the material? And how do I know what they've learned? Homework. Quizzes. Grading. If they don't turn that stuff in - according to Skullberg - it's on me. I must not have 'motivated' them.

Iow, 'motivation' seems to be just another fuzzy buzz-word that in reality is just going to be used as a club to beat the teachers with. Ironically enough, 'motivation' is the sort of edu-speak that I would associate with 'the left'.

And so, Skullberg, if a kid only turns in a quarter of his homework, isn't there Friday mornings for quizzes, and misses one exam, how much of that is _my_ fault? Since I am 'partly' to blame, according to you?

I might add that this 'liberal teachers' jazz really ticks me off; I've met very, very few of them over the years. Every so often, over the summer I do this 10-week program for 'disadvantaged youths', marginal types who don't quite qualify to be admitted. But if they pass the program, they're put on provisional standing in fall, and they get a $2,000 scholarship. Sounds like a good deal, right? You'd think they'd be grateful, right? You'd think they'd dig in, to show that they can do the work, right? Wrong. Almost uniformly, every year, they are the surliest, rudest, laziest louts it has been my privilege to educate. And the excuses! These kids don't hand in their homework because _I'm_ 'prejudiced'. They don't turn in their homework because the people they're staying with will laugh at them if they actually crack a book(someone recently tried to claim this is a 'liberal' special pleading.) They don't turn in their homework because they've been 'too busy'.

And this has been true of this program for a long, long time. And the people who actually administer it have heard it all. Yes, they are dedicated people. Yes, they want to help 'institutionally disadvantaged' kids get a crack at the brass ring. And they've heard it all from the little darlin's over the years. And they are some of the leas 'liberal' people I know.

(I don't mean to poor-mouth this program, btw. There's always that 5% - 20% who really do want to use this opportunity to better themselves. So while there are a lot of nogoodnik slackers who get in, overall, it's worth it. I just wish there were ways to lower the percentage of dross.)

You really are way too close to this, you can't see objectively at all, and so you take every hint that somehow, maybe, teachers affect education as some sort of vile smear against you and a conspiracy to degrade and deride teachers. Chill out.

Posted by skullberg

Geeze. More extremely dishonest words being put in my mouth. No, I never said that, and you know it. I'm saying you're trying to hold teachers responsible for things they have no control over, and using that as an excuse to foist off your own kooky education schemes. Knowingly. And no, I'm not especially het up. Telling you exactly what you are doing, and describing someone who would deliberately adopt that as a strategy to push their own agenda is hardly being agitated.

SoV,

I'm going to stop responding here because you simply are too close to this subject to be worth talking to. Plenty of factors can swamp or amplify the effect a teacher has on a child's education. In the cases you cite, which, like I said before, aren't illustrative to the points being made, no amount of instruction, classroom or extracurricular help on your part could make up for the problems in the other areas.

It is never "on you" to force people to succeed. You willfully at this point misrepresent what I say, when I say teachers are more than scantrons and lectures, you interpret that as some sort of backhanded insult.

Riiiiight. That's why you refuse to say how much responsibility a teacher bears for a student not turning in their homework. Because I'm being mean. Not because that's a completely ridiculous standard to hold a teacher to.

What a piece of work.

Sorry to skip to the end without reading the comments, but I'm coming in late.

Here are Kathy G's comments, with cites from rather prestigious sources:
http://thegspot.typepad.com/blog/2008/05/here-we-go-agai.html#more

Megan, It would be great to hear a back and forth with you and Kathy G on these sources. I tend to be skeptical of claims for educational effects, and it certainly was not my understanding that anything like a consensus on the efficacy of early childhood intervention existed. That said, it's not an area I know much about, and the claims made by Heckman here are quite interesting.

Post a comment

By using this service you agree not to post material that is obscene, harassing, defamatory, or otherwise objectionable. Although The Atlantic does not monitor comments posted to this site (and has no obligation to), it reserves the right to delete, edit, or move any material that it deems to be in violation of this rule.


Copyright © 2008 by The Atlantic Monthly Group. All rights reserved.