Work Matters

Tag: schools

  • Charter Schools, California and New York City: $6000 vs. $13,500 Per Pupil

    Huggy Rao and I have been reading and talking about charter schools for our scaling-up excellence project.  Charter schools come in many forms, but the basic idea is that these often smaller and more focused schools are freed from many of the usual rules and constraints that other public schools face, and in exchange,  are held more accountable for student achievement – on measures like standardized test scores, graduation rates, and the percentage of students who go onto college.  

    There is much controversy and debate about these public schools:  Are they generally superior or inferior to other forms of public education?  Are they cheaper or more expensive?  Can the best ones be scaled-up without screwing-up the original excellence?  Which charter school models are best and worst? 

    There is so much ideology and self-interest running through such debates that, despite some decent research, it is hard to answer such questions objectively.  But one lesson is unfortunately becoming clear enough that there is growing agreement — that my home state of California is so poor that it is a lousy place to start a Charter school of any kind.  I first heard this a few weeks about from Anthony Bryk, a renowned educational researcher and the current President of the Carnegie Foundation for the Advancement of Teaching.  He was also directly involved in starting and running one (or perhaps more — I don't recall for sure) charter schools when he lived in Chicago. 

    Tony told me that California was providing such meager funding that — although much of the charter school movement started here, there are many charter schools here, and many of the organizations that start and run these schools (called "charter management organizations") are  here — the funding that California schools receive is so meager that they are increasingly hesitant to start schools in California because the schools are condemned to mediocrity or worse.

    I started digging into it, and what I am finding is distressing as both a Californian and an American.  I knew that our schools were suffering, but I did not realize how much.  For a glimpse, here is an interesting and detailed article on scaling-up charter schools in Education Week from last year.  As Tony warned me, the charter operators described in this article are struggling to sustain quality in California and are looking elsewhere. Here is an interesting excerpt:

    Aspire, in Oakland, has also focused so far only on California. It opened its first charter school in Stockton, Calif., in the 1999-2000 school year and has grown by several schools each year. The CMO operates 30 schools and has nearly doubled its enrollment, to 12,000, over the past three school years. James Willcox, the chief executive officer of Aspire, said the difficult budget climate in California is causing him and other Aspire leaders to think about opening schools outside the state. “It’s getting harder and harder to do quality schools in California,” he said, “because the funding is so painfully low, and charter schools get less per student than traditional public schools.”

    He isn't exaggerating. I was shocked to see, for example, that (according to the article) the State of California is currently providing less than $6000 per pupil each year; in contrast, New York City provides $13,500.  Ouch.   I know that government wastes lots of money, and certainly there are inefficiencies in education.  But can we afford to do this our kids and our future?  As Tony suggested, California has degenerated to the point where all they can do is support a teacher for every 30 kids or so, a tired old classroom and school, and little else.

    I knew it was bad, but I didn't know it was this bad.  There is plenty of blame to go around — we all have our own pet targets — but perhaps it is time to put our differences aside and do the right things.

  • New York City Halts Teacher Bonus Program: Another Blow to Evidence-Resistant Ideology

    The New York Times reports that the school system has abandoned their teacher bonus system because it is ineffective. I quote:

    A New York City program that distributed $56 million in performance bonuses to teachers and other school staff members over the last three years will be permanently discontinued, the city Department of Education said on Sunday. The decision was made in light of a study that found the bonuses had no positive effect on either student performance or teachers’ attitudes toward their jobs.

    The research appears to be quite careful and the RAND Corporation is highly respected:

    The study, commissioned by the city, is to be published Monday by the RAND Corporation, the public policy research institution. It compared the performance of the approximately 200 city schools that participated in the bonus program with that of a control group of schools. Weighing surveys, interviews and statistics, the study found that the bonus program had no effect on students’ test scores, on grades on the city’s controversial A to F school report cards, or on the way teachers did their jobs.  “We did not find improvements in student achievement at any of the grade levels,” said Julie A. Marsh, the report’s lead researcher and a visiting professor at the University of Southern California. “A lot of the principals and teachers saw the bonuses as a recognition and reward, as icing on the cake. But it’s not necessarily something that motivated them to change.”

    Are you surprised? I am not, and if the people running the New York City school system had actually read a large body of existing research, they would never have wasted all this money in the first place. In our opening chapter of Hard Facts, Dangerous Half-Truths, and Total Nonsense, Jeff Pfeffer and I reviewed the extensive literature on the links between incentives and teacher performance, and it turns out that although there always have been people with great faith in pay for performance systems for teachers — going back to at least 1918 — careful studies show over and over again that they do not improve student performance.  The New York Times article suggests that despite the ideology supporting pay for performance systems, there is growing evidence that the current round of incentive-based teacher pay isn't working — just as it never had worked:

    The results add to a growing body of evidence nationally that so-called pay-for-performance bonuses for teachers that consist only of financial incentives have no effect on student achievement, the researchers wrote. Even so, federal education policy champions the concept, and spending on performance-based pay for teachers grew to $439 million nationally last year from $99 million in 2006, the study said.

    To be clear, pay for performance schemes do appear to have some effects in schools — most of which are bad. One of the most well-documented (see this post on findings in Freakomomics and related research) is that some teachers and administrators start cheating when their pay is linked to performance on student's standardized tests.  Their are strong hints that this is exactly what happened in Washington, D.C. and other cities where financial incentives for teachers and administrators are linked to student test scores. 

    Note that I am not arguing against pay for performance systems in general.  They work in other settings –sports, sales, lots of other places,as we show in Hard Facts.  But they don't work for teachers for a host of reasons, perhaps paramount among them are that teachers rarely have enough control over key student behaviors before, during, and outside of class, over class composition (and when they do, they sometimes use it to cheat the tests.. such as by sending poor perfomers to special education classes), and over other resources they need to have a strong enough impact on student learning.   Also, giving students a test once a year probably isn't a very good way to measure what students are learning.  As The New York Times report argues, another problem with pay for performance schemes is that it turns teachers' attention away from intrinsic rewards (the reason most go into the profession in the first place) and toward extrinsic rewards (See Dan Pink's Drive to learn more about the trouble with extrinsic rewards). 

    To be clear, I am NOT a general supporter of the policies of teacher's unions.  Although I do think that way too much blame and way too little credit is given to teachers, I do have an evidence-based pet peeve against how vehemently teacher's union's defend the jobs of bad apples, the rotten and incompetent teachers.  This argument is consistent with the work on "Bad is stronger than good" that I've discussed here before… while it is tough for even a good teacher to overcome a lousy system and have strong positive impact on students, it is pretty clear that really lousy teachers can make a bad system worse, and dampen the positive effects of a good one.   I believe that if unions changed policy here and became even more vehement about reforming and removing bad teachers than their critics it would improve their reputations and the quality of education — and earn them political capital to battle lousy policies such as tying teacher pay to student test performance.  (See this great discussion and debate at The New York Times).

    To return the dismal record of pay for performance systems in schools, some years back, I had an interesting conversation with Tony Bryk, a prestigious educational researcher who is now heads the Carnegie Foundation.  We were were at a think tank, a place called the Center for Advanced Study in the Behavioral Sciences, and I asked Tony why — even though there is so much evidence against practices like pay for performance for teachers — they remain popular and come back in waves… until overwhelming evidence emerges again that in fact they are bad.  Tony suggested two reasons.  The first has to do with ideology — that people hold some assumptions so strongly (like economics and business minded folks who believe that incentives are the best answer to changing any kind of human behavior) that they refuse to accept any evidence that runs counter to their beliefs — no matter how strong those findings might be.  The second reason was what Tony called "collective amnesia."  He argued that, in the history of educational policy, the same bad ideas seem to come around every 10 or 20 years, and policy makers and their staffs either don't remember or make no effort to dig-up relevant research to guide their decisions, regardless of their ideologies.  In the case of pay for performance, it appears that both of these factors are operating. 

    Practicing evidence-based management isn't easy given our various human flaws.  But we sure could save a lot of money, a lot of heartache, and make people's lives a lot better if we all tried a lot harder to do it.  There are plenty of outcomes in life that are impossible to predict.  Unfortunately, what happened in New York was completely predictable, even if people with blind faith in linking test scores to financial rewards for schools and teachers remain unwilling to believe this well-established truth now. 

    P.S. A comment below suggests that some recent studies do show a positive effect of financial rewards on student performance.  My reading of the Rand Report suggests that for studies done in the U.S. there are a a few studies that show a positive impact, but the weight of the evidence supports the historical pattern of no effects or negative effects.  The more rigorous studies in particular find no ot weak effects on test scores, and little effect on teacher motivation, although there is some evidence that teachers devote more evidence to teaching to the test and less to teaching other things (not a surprise).  There is also some evidence that teacher cooperation goes down a bit and evidence that teachers game the system more to boost test scores.   A researcher from Chicago explained to me that in the schools she was studying (this was about 10 years ago) that scores were going up but she believed that it was not so much because incentives motivated teachers to work harder, but because it motivated them to get rid of their weakest students (often by sending them to special ed classes) and to refuse to "skip" gifted students because they pumped-up the average test scores in a class.  Finally, the most obvious effects of pressure on teachers and administrators to pump up test scores is cheating on the tests (by the teachers and administrators), as we have seen from evidence from Chicago, Philadelphia, and Atlanta, and just today, a probe started into cheating at schools in New Jersey.

    I am not rejecting the value of financial rewards as motivational tools for teachers outright, and it does appear that there are some special conditions under which they may be of some value. But the weight of the evidence suggests that most of the money spent on such incentives could have been to better use, that the ideological support for them is much stronger than than the evidence in support, and that one of the most consistent effects is bad — teachers and administrators cheat either to get the incentives or because they fear losing their jobs.

    This is my conclusion. You may reach a different one.  Here is a link to the 300 page Rand study of the failed New York program, which contains an excellent and very current review of the research.