Welcome everyone. Thank you in advance to Dominic, Michael, Rob, and Sandy for joining us for this discussion, and thank you for joining us. Please consider not only reading but being an active participant and submitting questions for the panelists. We’ve got four terrific panelists for this discussion and since three of the four are from other countries (Dominic and Sir Michael are British and Sandy is from Texas) we’ll get a broad perspective on this issue of positive incentives in education.

It’s safe to say that within the education community there is a general agreement that incentives matter but a great deal of debate about what kind of incentives would be most conducive to dramatically improved student outcomes relative to what we see today. And although the Elementary and Secondary Education Act (No Child Left Behind) includes provisions aimed at rewarding higher performance, the overwhelming emphasis of the law’s implementation has been focused on addressing the problem of low-achievement. Where you see positive incentives in education it's often token gestures, for instance recognition for schools and teachers. Larger monetary awards remain rare and truly creative incentives, for instance greater autonomy or the ability to replicate successful ideas, are almost unheard of.

So let me open by asking the panelists to respond to and debate three questions: First, in the context of American public education, with its decentralized organization and highly inequitable outcomes, what is the role of positive incentives in spurring better outcomes? Second, why has so little been done on the positive incentive side of the ledger in terms of national, state, and local accountability policies? Finally, what kinds of incentives should state and national policymakers be putting in place?

Posted by: Andrew Rotherham from Education Sector

Robert Manwaring from Education Sector responds:

Looking forward to the discussion. Here are some of my opening thoughts. I think it is important to start off by answering why positive incentives have not played a more prominent role in education policy in the past. There were attempts to use cash rewards dating back to the 1980's. But these programs suffered both technical and political problems and were generally short lived. On the technical side, the state assessments were often low quality, and data systems were not very sophisticated. As a result, it was unclear what was driving rewards—demographics or school effectiveness.

Schools would get rewarded one year, and not the next, but educators even at successful schools were uncertain why there was volatility. As a result educators may not have seen the connection of extra effort on their part and the school receiving an award. If this linkage is not there, then teachers and administrators may not be influenced by the incentives, and as such the rewards may not have motivated school improvement.

On the political side, unions have historically not been fans of school reward programs.

(Perhaps the best use of school reward funding would be to provide rewards to teachers and administrators at a schools that drove the school to be successful). Because unions have historically not wanted to differentiate funding across teachers within a district, teacher rewards, or merit pay programs have struggled to become a permanent part of teacher compensation. In addition, because schools could not count of the funding from year-to-year, when rewards were received, the money was viewed as truly one-time in nature and was often spent on low leverage investments. So when state budgets would get tight in the next business cycle, rewards programs were often early victims of state budget cuts.

How could it work better?

(1) Use better measures. To be perceived as fair, a rewards system must provide an opportunity for any school to get a reward regardless of the demographics of the students that it serves. Improvements in assessments and data systems that allow states to determine a value-added measure is an important step forward that can provide a more accurate measure of how elementary and middle schools are doing at educating their students regardless of where the students start academically. So value-added accountability combined with measures of proficiency provide a better tool to support a rewards program. Using multiple years of data also makes sense, and would lead to greater stability in identify successful schools (Successful schools do not materialize over night and then stop being successful the next year, the measures should be more stable). Broadening the accountability measures beyond current assessments in math and English would also improve the viability of a rewards program. For example, developing a school inspection system to monitor the effectiveness of schools. The inspection would measure common traits associated with successful schools like whether instruction is aligned with standards, whether data is being used effectively to inform school decisions whether schools plans align with school budgets. By having measures beyond test schools, incentive programs would be more sustainable.

(2)Make sure that reward program is forward funded. Incentives only work when people you are trying to influence know about the incentive. Often incentive programs are started with little warning to the educators about what the rules of the game will be. For example around 2000, California created and funded school and teacher reward programs in the late spring after that year's annual test had been administered. So, there was no connection between the reward program creating an incentive, educators responding to that incentive and changing their actions to improve the chances of getting a reward.

Currently, Race to the Top incentives may be suffering from a similar lack of time to respond. The Race to the Top funds will more than likely be distributed to states that have been working of school reform initiatives for the last 5 years or so. Such focused investment will support continued reform in the limited number of states that already have the capacity to implement meaningful reform. However, the funding may not create the incentive for states to make dramatic changes to their education policies because even now, the specific measures that will be used to distribute the funding are unknown. So the Race to the Top may not motivate reform in states that have been struggling. So educators must be given enough time to respond for incentives to be effective.

(3) Rewards should be for multiple years. To ensure that the funds are used effectively to support further improvements at the school site, rewards should be provided over a multiple year period. If there is the risk of the funding going away the following year, schools will be appropriately conservative in how they use the funds, and will spend the funds on one-time investments (that are often low impact investments). Longer grants will ensure that the funds can support multi-year reform efforts.

(4) Encourage rewards to be linked with teacher compensation reform. One of the best uses of school rewards is to link the reward funding to an overhaul of a district's teacher compensation system. Compensation reform can motivate teachers to be more effective, and linking rewards funding into such a system can provide the funding source to support differential compensation.

(5) Successful schools should be granted greater autonomy.

At the state and district level, budgets are going to be tight for at least the near term. This means that new investments in school reward programs are not likely for the next couple of years. It is difficult to create new programs at the same time that teachers are being laid off. This does not mean that positive incentives can not be implemented. Perhaps the most powerful positive incentive program would allow successful school sites to have greater autonomy from district policies and greater budgetary control. The common argument against school site management and school site budgeting is that school leaders (principals and senior teachers) do not have the training or skills to operate the school with greater independence. While this lack of capacity is likely the case in lower performing schools, high functioning schools are clearly up to the task and should be given greater autonomy. School inspection reports could verify that a school had the capacity to manage itself. Giving school leaders the opportunity to earn greater autonomy is likely to be a stronger motivator than rewards.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

Before we get into the very complex matter of financial incentives, I want to spend some time on a set of more customary and traditional incentives that matter a lot.

The incentives that matter most to Texas schools, I believe, are the ratings the state gives them in our accountability system. The labels of "recognized" and "exemplary" are highly coveted. It is not only a deep source of pride to get these ratings; it is also touted in the community, recognized by parents, local businesses, and, maybe most important, realtors!

We've not been strategic in the past about awarding this incentive. In Texas, as with most other states, we give it to schools that get the most kids to passing, or whatever states in the nclb era have dubbed "proficiency."

My big idea today is that we ought to use these labels to reward schools that get the most students (including the subgroups) to higher levels of achievement or greatest growth against higher standards.

In Texas, our legislature just this week passed a bill to do just that. The bill calls for the commissioner to set a new performance standards for 3-11 in math and reading as reflective of being on a path to college readiness. (Science and social studies come later.) Schools that have the greatest success in doing so or achieving significant growth to this goal will be eligible to receive the recognized and exemplary ratings.

I think, as do many of my fellow reformers, that this conversion of our top ratings from rewarding schools for getting more kids merely to passing at low standards to rewarding schools for getting kids to achieving at real world, higher standards may be the single most important advance in accountability in Texas since we began the system in 1993.

Think about this on a national level. In the ESEA reauthorization, we might consider creating financial incentives for states to do just what Texas has done. States could and should keep the proficiency bar for ayp purposes. This would keep pressure on systems to maintain minimum bars above which all students must jump. But states would be challenged to use higher bars for the carrots of their higher school ratings. These ratings would be reserved for those schools that significantly lifted the performance of their students to advanced levels of achievement, tied in some true way to being on the path to college/career readiness.

I've done a lot of work on financial incentives in Texas and elsewhere and will join in a discussion of issues around their use in later answers. But I felt compelled for my first comment to be around this idea of addressing the use of school ratings. I've done so because I truly believe that these are now and are likely to be for the foreseeable future the most powerful incentives available for rewarding school performance.

DOMINIC BREWER from University of Southern California responds:

This is a big set of issues. For most of us, thinking about incentives seems an obvious and necessary part of thinking about improving student performance, but it has typically been largely absent from the culture of schools and educators. When many of the EDD students I teach are challenged to think about the incentives, positive and negative, that they or their teachers or their students face, they realize they are pervasive. But it is a revelation! This has to do with the compliance mindset that is endemic in a highly regulated environment, where so many decisions are made at state and district levels. Creative use of incentives requires a mindset which is less risk averse than the norm, and it also is easier when there are good data and evaluation systems in place so that the effects of introducing new incentives can be understood. In all but a few places, the latter systems are still absent and policy is “done” to schools typically across the board, rather than in a planned roll-out or demonstration that would permit everyone to understand the effects.

A few framing things. It’s hard when focusing on “positive incentives” to not immediately jump to thinking about bonuses for schools or similar such devices. In fact non-monetary incentives are likely as important as monetary ones. Further, let’s remember that incentives occur at all levels—students, parents, teachers, schools, administrators. In the U.S. we tend to devote almost all our attention to schools when we are focused on accountability, but I would start with students. (As a product of both British and U.S. systems myself, I am still amazed at how we expect kids to work hard in school but in the absence of almost any stakes, until late in high school.) I will come back to this later. Finally, there are certainly both positive incentives- rewards- and negative incentives- sanctions. Although this discussion is going to focus on the former, it is certainly my belief that you cannot have one without the other. There are issues of which works most effectively (in general, positive preferred over negative), and at what points in time (or where in the improvement process) but just having positive incentives won’t work. You must have real sanctions. Real i.e. not as now in most states where failing schools might be in year 6 or 7 of program improvement with the same group of educators re-writing the same old improvement plan with the same old consultants, and knowing they really wont be shut down, or converted to a charter. So for me one can have all the most brilliant positive incentives one wants, but at some point, there has to be a real and tough sanction. The two, positive and negative, should work together to maximize the motivation of all.

Michael Barber from McKinsey & Company responds:

In answer to the original questions, here are some thoughts from me.

Positive incentives are a potentially powerful part of an overall approach to system improvement. There are a number of questions to consider about their introduction.

• Should they be individual (for teachers) or collective (for teams or schools)?
• Should they be financial or non-financial?
• Should they reward improvement or excellence or both?
• Should they reward progress or test results or a wider range of contributions to the success of schools?
• What evidence should they take into account?

Here are a few successful examples I've seen in education and related sectors in England.

Financial

• Financial rewards for hospitals that hit milestones on the way to reduced wait times for patients.
• Whole school cash bonuses for distribution among the staff of schools which improve significantly in value-added terms or achieve sustained excellence against a benchmark group or move rapidly and sustainably out of an underperforming category.
• Financial incentives to universities which reward course completion rather than enrolment.

Non-Financial

Offering successful school leaders the opportunity to become system leaders, playing a part in assisting improvement in other schools or indeed contributing to policy development. The National Leaders of Education programme and the National College of School Leadership in England involved over 200 demonstrably successful school principals. It trains them to offer mentoring and support to principals in schools which face severe challenges. More recently the law also allows these principals to literally take over other schools, in effect creating the beginnings of charter chains from within the system.

More later.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

Before responding to Andy's first questions and Rob's excellent first answers, I wanted to let Andy know that I did belatedly note his comment that I was from another country. It's true; I AM a Texan. Of course, my earlier answers referred to my Texas experiences as Sir Michael might discuss his experiences in Britain! In fact, I was recently bragging to a few Obama folks about how our pay for performance program in Texas is lvirtually as large as the federal program! So, another country? Proud? Obnoxious? I'll leave that to "y'all"!

We need to acknowledge early on that while incentive programs are common to most human endeavors, they have been rare in the education world. And there are reasons we must acknowledge.

Yes, unions have been hostile, sometimes for good reason. Many early versions were highly subjective and unfair.

Administrators have seldom promoted them, typically believing them too hard to do and generally not worth the bother.

All in all, the culture has been, "we don't do that; that's for the private sector; we don't respond to those kinds of things."

It hasn't helped that the tools made available through technology and data have been late to the education sector. These tools are essential to making incentive programs work well, and yet they're largely unused still today. Walmart can track everything in every way in its supply chain; yet we know little about the exact performance of students, how they've grown, why or why not, and even whether they've dropped out. I won't get into why this is so, but I will surmise it may be due to some of the same reticence educators have to the whole idea of differential treatment.

The evidence that incentive programs work is building. We saw moderately positive effects in Dallas with the school effectiveness program we administered in the mid-90s. We're seeing good effects in the early stages of our pay for performance programs in Texas, particularly with respect to the retention of teachers who get sizeable payments. I eagerly await the first results from Guthrie's major studies at Vanderbilt. I believe they will be encouraging.

Here are a few thoughts in response to Rob's answers on the common elements of good incentive programs.

First, the metrics must be fair and accurate. Indeed I've come to the view that growth, probably measured on a value add basis, is essential. If the players don't believe the system to be fair and accurate, it won't survive and shouldn't.

Second, there must be some local buy-in, and whether the unions are gung-ho or not, teachers and principals must understand and at least broadly support the program

Third, Rob is right: the program must be "as on-going" as possible. We have just made it through the third affirmation from our legislature. It's been a struggle each time. If folks believe this is a fad and not to be sustained, the program won't penetrate and thus there will be no lasting change.

Fourth, while the rules ought to be clear from the beginning with sustainable funding, the money shouldn't in my view flow to a school automatically for multiple years. Unless it depends heavily on continuing strong performance, it will become something other than "an incentive program."

Fifth, and Sir Michael might argue with this, the incentives ought not be based on inspections. The criteria should be about results, not inputs. I believe firmly that inspections have not been standardized enough and the the causality of measurable criteria to results is too uncertain for inspections to serve now as a basis for incentives. Again fairness is key, and basing incentives on results, however hard that is to do, is the fairest thing.

Sixth, these programs must have outcomes in mind from the beginning, and they must be reviewed and re-affirmed or revised by stakeholders on an ongoing basis. Further, the details must be right. Is this about student achievement? Growth? What subjects? How measured? How are non-tested goals measured? Is it about the school as a whole? Does it get to teacher effectiveness? Is it just about achievement? Or is it also about other objectives, such as getting effective teachers to low income areas? Or more math and science teachers? And, if so, how do we know the teachers we're incentivising are effective? If we don't know, why are we incentivising them?

Seventh, is there a solid strategy in place at the state or local level for implementation and adequate political support in place to survive the inevitable problems and pushback? We had a very good state program in Texas die because agency folks loaded up a simple school reward program with so many regulatory requirements that the schools no longer thought the redtape and bureaucratic burdens were worth it.

Finally, I would suggest that the easier and more defensible programs might be a good place for newcomers to begin. For example, school effectiveness incentives have a positive history with less controversy. I believe we must get to teacher effectiveness, but there are far greater challenges to getting it right. Since we have a long climb to change the culture, school awards, even with their flaws and limitations, might be the best way to begin for some.

DOMINIC BREWER from University of Southern California responds:

This certainly seems an excellent list to me. I agree that I'm not sure inspectations are that effective - my cursory knowledge of them in the UK or Zew Zealand is that they can focus too much on process and conformity. Maybe they have a role, but only with a clear outcomes-driven system of accountability.

One of Rob's interesting comments relates to granting autonomy as a positive incentive. It strikes me as a potentially powerful one—but only to some eductaors.  For many, the prospect of autonomy and the greater responsbility it brings is not welcome, not so much because of lack of capcity or tools to handle it (though that they be present) but more because it likely necessitates the breaking down of norms that make for a comfortable status quo. Some high performing schools, and some trailblazing schools that are lucky to have less risk averse teachers and administrators, will embrace it. But the tricky thing is convincing a much greater number that it is something that can make schools more rewarding and fun places to work. I think the evidence is that it is hard to do on a wide scale within the existing heavy regulation, teacher compensation/tenure/evaluation system, and school funding mechanisms.

Robert Manwaring from Education Sector responds:

I can understand Sandy's concern with using the results of a school inspection process as part of a reward structure. But I think that school inspections if done right, can broaden the measure of the quality of a school. Even if you don't use school inspection as part of a rewards program, using inspection results or at least an overhauled school accredition results as part of a process to decentralize decision making to the school level would still make sense.

DOMINIC BREWER from University of Southern California responds:

To the extent that we are caught in a catch 22 in many states - California for example - whereby legislators do not trust local districts and schools to make decisions, and hence invent new programs/categorical funding streams/regualtions, there needs to be a mechanism to break the logjam.  This is where I supposed an inspection system could be helpful in providing a smart assessment of school level capacity that would in turn then permit what kind of capacity building is needed, and when it is possible to grant autonomy etc.  They could play aan important role in helpding spread best practices more effectively. What seems to happen, however, is that inspectorates are viewed as yet more heavy-handed interference and/or become simply "going through the motions" whereby lots of paper is gathered but nothing behavioral changes.  So if they are part of the solution, there has to be mechanisms in place to avoid the new agency simply becoming another layer of bureacracy that makes more work for teachers and adminsitrators in schools, and which in the worse cases, helps enforce a confomity about what schools should be doing.

The debate around NCLB reauthorization is sure to include a discussion about teacher effectiveness. What role do teacher evaluation systems play in school improvement? How can federal policy encourage the development and use of better measures of teacher effectiveness?Posted by: Catherine Cullen from Hope Street Group

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

This is a great and profound question.

We must build far stronger systems of evaluating teachers based on effectiveness and responding with appropriate professional consequences. I could fill up lots of space about my thoughts about the necessary elements of such a system, but I'll spare you. I do believe, though, and will say since this whole notion involves the sensitive matter of pay, professional status, and the instructional performance of the school, these systems must involve teachers crucially in their construction and operation, and be fair.

But, to the point of our discussion, I don't see how we create incentive programs for schools if they don't fundamentally relate to teacher effectiveness. Teacher effectiveness is the single most important ingredient in an effective school. What else would we want to incentivize more?

Yet, teacher evaluation systems are generally so crude today, how can we properly structure incentives?

We did a school incentive program in Dallas back in the mid-90s when I was on the school board. It was based on a value add, regression analysis of school effectiveness each year. And it rewarded the school with a financial reward that essentially gave each teacher and each staff member in the winning schools a bonus that year. We did some serious evaluation of the program and found it modestly effective in improving student achievement across the district.

I think this model is a solid and conservative one to consider. It has the advantage of avoiding the difficult issues of attributing teacher effect and instead relying on the easier-to-prove school effect. But that's its disadvantage, too. There remains great variation within these schools. None of that is accounted for.

The future surely is in building models based on value add that fairly and accurately judge teacher effectiveness and using those evaluations for incentives. There is much promising work these days in this area, that of NCTQ, the New Teacher Project, TAP, et.al. We really must build on their work.

DOMINIC BREWER from University of Southern California responds:

I agree with Sandy. I don't see how one can have an effective incentive system unless teachers are part of the mix. The way teachers work lives are organized (Inlcuding pay, benefits, tenure, working conditions, evaluation, etc.) have profoundly perverse incentives. What first generation standards-based accountability has begun to run up against in many states is that it is hard to do in the context of a compliance oriented model with relatively few flexible resourecs at the school house and salary schedules tying up the bulk of your resources. Mechanisms for motivating teacehrs have to ibnclude evaluation and compensation—as well as more subtle (but important) thigns like how to encourage team work, sharing of information etc so that high expectations become the organization's norm.

Robert Manwaring from Education Sector responds:

Improving district teacher evaluation systems is the linchpin to improving the quality of education in this country. Evaluations are hard work, and the issue needs serious attention. Currently administrators invest little time in evaluating teachers, and generally give good evaluations because it is not worth the headaches of giving a bad evaluation because so few teachers are ever let go. There is a nice recent study by the New Teacher Project on teacher evaluation. The study shows how few teachers receive poor evaluations. While it is not surprising that they only had one district in their sample, Denver, that used evaluations in compensation decisions, it was surprising that evaluations results are not used to inform professional development decisions. Imagine that you are one of the few teachers that is told that you need to make improvement in say classroom management. But that year all of your professional development is focused on teaching reading to English learners. How is the district supporting you to improve your shortcomings? Unfortunately, often they are not. For teacher evaluations to become more meaningful, the results of the evaluation should have some influence on a teacher's future professional development and mentoring support.

The second part of the question is more difficult. A more rigorous evaluation process in the end is something that needs to be negotiated between a district and its teacher's union. At the same time the federal government can provide competitive grant support to districts willing to overhaul their evaluation systems. And this type of work needs to be rigorously evaluated to find best practice models that can be used by other districts. Currently federal teacher funding is spent on low impact uses like class size reduction and generic professional development. Redirecting some of that funding to support improvements to teacher evaluations would be a step in the right direction.

One of my great disappointments in the debate over incentives is the failure of most participants to read the existing literature in I/O psychology. (As I've written in several places, it's a shame that a historian is better read in this area than the policy wonks.) What research do you think is most relevant, and what lessons do you draw from the existing research in I/O psychology for performance-pay systems?Posted by: Sherman Dorn from University of South Florida

DOMINIC BREWER from University of Southern California responds:

I'm not a pyschologist by training but I think the research is pretty clear that motivation is often the cause of performance gaps rather than other explanations like lack of knowledge. In education, our first solution is often "more professional development" implying that if only we had better skills/knoweldge we would perform better. Clearly, there has been little conversation historically about how to motivate students and teachers—students must go to school, they are expected to be motivated by intrinsic love of learning or more commonly by the clear connection to subsequent economic success; teachers are "not in it for the money" and extrinisic rewards seem to run counter to the "culture" of schools; administrators at the school level have little other than "moralsuasion" to motivate teachers and stduents because resources are largely "given" to them, they cannot hire/fire and often can't even control work assignments. In this context, I think you are right that motivation is often dismissed as a possible underlying cause of a performance problem; to suggest a teacher isn't motivated is somehow to question their professionalism. I think there are likely many sepcific lessons from the study of incentives within non-educational organization vis a vis motivation—like setting clear but challenging goals, limiting use of negative emotion which tends to de-motivate, etc. A very basic practitioner introduction to these issues we use with our own EDD students is Richard Clark and Fred Estes' book "Turning Research into Results."

My question relates to the environmental conditions for sustainability of incentives in a time of scarcity.

For example, in Austin ISD, the innovative teacher incentive program has been recommended for cuts in the second year of the program. (We really don't have enough information on how well it works yet, but let's assume it does.) When a carefully crafted program that brings together the teacher's union, administration, business community, and stakeholders in general is threatened by scarcity of resources and a massive shortfall, it suggests to me that all incentives (and other innovations for that matter) are subject to dissolution prior to any measurable ROI. This makes future risk-taking less likely as institutions do not want to burn goodwill and have nothing to show for it.In other districts, innovations end as superintendents depart or as grant funding dries up.

So my question is, what are your thoughts on the staying power of incentives? What sort of expectations should we be establishing on the front end to ensure that these investments are seen through to their fruition?

Posted by: J J Baskin from Education Capital

DOMINIC BREWER from University of Southern California responds:

This is an excellent point. One of the things some of the innovations in teacher compensation that are currently being tried around the country (e.g. Denver, Houston, SF) do is try to build in a stable source of revenue from a formal budget set aside, or a separate revenue stream such as a parcel tax. Without commenting on the merits of any particular incentive plan, this clearly seems to increase the likelihood of sustainability. Using one-time foundation or federal funds for bonuses or other rewards for a couple of years, seems doomed to minimize effects—schools are just so used to endless reforms that come and go, that any positive effects are likely to be short lived.

The bigger point however is that existing resources are already generating a set of incentives, just more often than not, ones that arent that effective. The reward of educational credits in the salary scheule results in literally hundreds of millions of dollars tied up in providing an incentive that is not directly tied to subsequent teacher job performance or student performance. If we continue to view positive incentives/rewards as marginal add-ons the existing finance and revenue structure it seems to me we will get only very small results. If 99.5% of dollars are distributed as usual and we devote .5% to performance incnetives, what do you expect?

Michael Barber from McKinsey & Company responds:

I think your point about the sustainability of incentive systems is crucial. There are two points to my answer. One is that incentives need to be part of a whole-system reform, not a separate initiative (American educators are obsessed with the 'initiative', by the way—it should be banned!). In other words incentives should be incorporated into the whole approach to accountability, transparency and performance management. There need to be a combination of positive and negative incentives—and people in the system need to know what they have to achieve to benefit from the positive incentives and when the negative incentives will apply.

Clearly if the arrangements aren't sustained or are changed repeatedly they will not work since people in the system lose track of what they reward. In the case of financial incentives therefore they need to be budgeted for into the future. After four or five years there may be case for changing what is incentivised either because the original goals have been largely achieved or because the priorities have shifted—or because over time people have learned to game the system.

As one of the previous contributions points out very often systems emphasise the negative incentives at the expense of the positive. System leaders need to think hard about this, especially where the media are involved because they will inevitably accentuate the negative which means the system must work even harder at the positive.

One other point about sustainability; often incentive schemes are controversial at the beginning not least because in the first year of a scheme those NOT rewarded far outnumber those rewarded but over time this will change which helps build consent. System leaders who introduce even a well-designed schem therefore need to be ready to hold their nerve for a rough ride in the first year.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

I appreciate this hometown question from my friend, JJ Baskin. And it's a good one.

My experience has been quite simple: the business community must insist and continue to insist that the incentive program be instituted and maintained. Please understand that that is not all I'm saying. As in indicated in an earlier answer, I believe there are many essential elements to a good incentive program, including the participation of key stakeholders. But until "the roots are firmly established," the business community, which supports the bond campaigns and the tax increases, must insist frankly in return for its support that the culture change from one of "all things, the same" to "results matter, and good results should be incentivized."

Now that the Obama administraton is committed to pay for performance, things may change. But, for now, the law of nature, as I've witnessed it, is that the business community must put its muscle behind getting the program up and running for the first few years. Then, whether  it works and whether it draws broader community support will and should determine if it becomes part of the fabric of the life of the school community.

Robert Manwaring from Education Sector responds:

I agree with much of what the other have said, but will add a couple points. First a quick comment, did I read that an education professor suggested that incentives to reward teacher that get master degrees incentivizes the wrong thing? This could work a lot of education professors right out of jobs.

To build on Michael's point about not doing rewards as a stand alone initiative. Rewards should be linked to compensation reform which should be supported by other teacher related reforms. I am a fan of the Teacher Advancement Program (TAP) that has four components to it - rigorous evaluation, professional development support directly linked to a teacher's evaluation results, career ladders (including mentor and master teacher roles), and performance based pay. By linking these reforms together they are likely to have more staying power even in the difficult times.

Of course, it is difficult to start these types of programs in difficult fiscal times. And as Dominic pointed out, using one time federal funds as the sole source is not likely to lead to sustainability. I think that one of the more viable options currently would be linking greater autonomy to school outcomes. This is generally a cost neutral type of policy (although it may expose some significant inequities in the distribution of funding across schools in a district.) Once in place, this type of reform is likely to gain grassroots support of teachers and administrators that will lead to a greater chance of sustainability.

DOMINIC BREWER from University of Southern California responds:

Yes you read it right Rob!   I think many faculty believe we need to get much more serious about measuring the effectiveness of our programs.  One of the byproducts of "competitors" like TFA, New Leaders, NISL, and district-alternative certification, has been to at least gently introduce the notion that the University monopoly may not last forever.  Monopolies are bad in any context, be it school of education over certification or public school districts over provision of free schooling in a jurisdiction. It will certainly take some time to put in place effective program evaluation systems, that include crucially measures of how well the kids of the teachers and leaders we graduate with our degrees perform after the students have left the campus.  This loops back to one of your earlier points re: measurement.   It is still the case that teacher education faculty (as well as teachers and adminsitrators in schools) can, with some justification, say that the emasures we use for what we want schools to do are so weak and/or narrow they cannot be trusted.   Progress in many areas seems to me contingent on being able to put this argument to rest ultimately by having clear, thoughtful and well rounded multi-dimensiuonal assessments of kids.

 

Years of research repeatedly demonstrates that teaching improves—and with improved teaching student achievement improves—with considerable time for collaboration and job-embedded, continuing professional development. This is especially important in an high-stakes accountability environment, where it is central that all teachers focus their energy on ensuring student learning and achievement, rather than on test scores. What policy levers might re-focus investment in continuing teacher PD centered on meeting student academic needs rather on the end-goal of test scores?Posted by: Ellen Pechman from EMP Consulting

Michael Barber from McKinsey & Company responds:

Four quick points:

1. the state of Victoria in Australia is about to publish materials to go to all principals and ultimately teachers pulling together the research on effective pedagogy (the five 'E' model). they will back it up with a major professional development programme. Will be worth looking at - an attempt to take evidence-based pedagogical imrovement to scale.

2. The Task Force on Strategic Management of Human Capital has a great website with extensive evidence on the subject of incentives etc. at www.smhc-cpre.org

3. The TAP program has a very good trackrecord in this respect.

4. a strategic dilemma for systems is whether to try to regulate/shape incentives (and professional development etc) for individual teachers from the centre or to devolve power and responsibility to schools and make this a major responsibility for principals. NYC has moved from the first to the second as we did in England. there are risks both ways and much depends on how well the system is performing at the moment you start.

Michael Barber from McKinsey & Company responds:

One further point - good teaching which meets student needs and aspirations should help students do well in tests too. Its important to avoid false dichotomies in this debate.

 

DOMINIC BREWER from University of Southern California responds:

I was being cowardly in holding off on responding to this question, but now Michael has said it, I feel empowered....the question poses a false dichotomy in my view!   Yes the measures may suck but they are better than handwaiving or platitudinous statements about value of a broad education.  Given that standards-based accoutnability seems here to stay, the time has come to stop complaining about the narrowness of student assessments and develop multidimensional measures that capture what it is we want schools to achieve.  This is perhaps an area where federal investment can play a role.

Robert Manwaring from Education Sector responds:

 

On professional development, my understanding of the research is that good professional development like the type that you describe can be effective at improving the quality of teaching. The collaborative environment that you talk about is a common trait at our most successful schools. At the same time generic professional development has virtually no impact. What I can not figure out is why so much of professional development funding is invested in the latter instead of the former. Making professional development more relevant to the teacher either because it is linked to teacher evaluation or job-embedded or preferably both seems like a promising school improvement strategy. Can positive incentives lead schools to this model?  

I have also wondered why schools are not more effective at building collaboration time into the school week. There can be more win-win opportunities to provide students with enrichment opportunities that are often on the chopping block like art, music, foreign language, and PE while at the same time providing the opportunities for teachers to get the support that they need. But schools need to be intentional about creating this time instead of just giving teacher some release time to grade papers.

I have a two-fold question for anyone to answer. First, in the context of reauthorization, will the US Dept of Education and other policy impact organizations push for congress to address the need to establish national language development standards? Part 2, The effort to develop national language standards for both language development and English language acquisition would facilitate a common national definition of "limited English proficiency", will congress be encouraged to support a federally authorized panel to establish a common national definition of "limited English proficient" based on scientific research and well- developed language standards (including all linguistic aspects - (not just reading or TESOL) of language development and language acquisition for English language learners? thank youPosted by: kathleen leos from Global Institute for Language and Literacy Development

Robert Manwaring from Education Sector responds:

 

I am by no means on expert on English learner issues, but here are my thoughts from an outsiders perspective. My sense is that a lot more research is needed in this area before some of the ideas that you are suggesting could be successful. For example, I think that there are great benefits of states working collaboratively on EL standards because many states do not have the expertise to develop them independently. But there may be benefits from having multiple consortia working on this issue somewhat independently because there is still a lot of innovation needed in this area, and several consortia may over time help develop a richer understanding than putting all of our eggs in one national standard basket. Similarly on having a single definition of Limited English Proficient, and the best practices on how to serve student's language development needs. My sense is that there is not consensus on when is the appropriate time to reclassify a student. Some districts and states take the approach of reclassifying students as soon as possible with an underlying philosophy similar to special education's least restrictive environment approach. Get the student in the mainstream classroom and curriculum as quickly as possible. This is perhaps the tough love approach. While other districts believe that students need on-going support even after they have achieved fluency because otherwise student's achievement in English language arts tends to fall somewhat after reclassification (Patricia Gándara research has emphasized this issue). I think that a lot more research to answer these fundamental questions is needed before a single standard is adopted. And this research needs to be done using longitudinal data that track students through the various different support programs out there. Often a student will be served in numerous different ways over the course of their transition to fluency. When researcher focus on one snapshot in time, and conclude that one model is more effective than another, they miss the nuance that students migrate from one program to another over the years. For example, a student may start out in a bilingual setting, then move to a SDAIE program then a dual emersion program over the course of several years. If you just look at a snapshot, the language skills of students in bilingual classes may be lower, but that has nothing to do with the quality of the bilingual program, but more to do with the types of students that are placed into those settings. What is important is how effect these different approaches are at advancing a students language development. So the analysis must be more sophisticated. Given the advancements that states have made in implementing longitudinal data systems, I am hopeful that EL research will really take advantage. Until we know more it is difficult to impose one national standard.

Is there a role for teacher education to participate in preparing preservice teachers for an incentive environment?Posted by: Kimberly Shotwell

DOMINIC BREWER from University of Southern California responds:

What a great question!  Undoubtedly, a sustainable system of incentives, positive and negative, should include teacher pre-service education.  Although it's anecdotal, my sense is that even after several years of standards-based accountability, many University teacher education programs continue to ignore the reality of accountability, let alone embracing it and helping teachers understand what it means to be data driven, for example.  Undoubtedly there are some exceptions, but teacher education faculty are creatures of a pre-NCLB world, and very slow to change.  There isnt an evidence-based culture so that ideologically dirven views of what works tend to have long lives.  And it seems many have a hard time moving beyond the mantra that "measures are flawed" and therefore accoutnability should be rejected as a policy paradigm.  

Carefully considering what kinds of different or additional skills might be helpful in a positive-incentive-rich school is a fascinating question.  Let me give this some more thought.

Robert Manwaring from Education Sector responds:

Teacher surveys continually show that younger teacher (fresh out of preservice) are much more receptive to positive incentives, teacher compensation reform or teacher bonuses. For example see work by Susan Moore Johnson at Harvard on the differences of the next generation of teachers. Partly younger teacher may be more receptive to these reforms because they are so disadvantaged in the current salary structure, and are thus more willing to take risks. But also this generation often does not see teaching as their lifetime career. This again makes newer teachers more receptive. So, from my perspective  newer teachers are interested in these types of reforms, but are overruled by veteran teachers. This is not to say that there is not a role for schools of education. As discussed above, teacher evaluation policies are a linchpin to many of these reforms. I think that schools of education could play a critical role in improving the quality of teacher evaluation systems in two ways. First from an R&D perspective, education school faculty could support district efforts to improve the quality of teacher evaluation systems. In addition, schools of education train the principals as well. Improving the quality of the training that principals get on conducting rigorous evaluation would helpful. Ditto for master programs. Many more rigorous evaluation systems involve master teachers. Incorporating peer evaluation best practices into masters programs would improve the supports that these master teachers are able to provide more junior teachers. This collaborative approach is the type of support that the next generation of teachers say that they want.

Finally, schools of education should not be exempt from the incentive system. New data systems will allow us to link outcome data to teachers and the colleges that train those teachers. The colleges that do a better job will attract more teacher candidates because they will be better able to place those teacher candidates after their preservice programs. Since many teacher candidates are trained at state colleges, states have the ability to create incentives that expand the budgets of schools of education that train effective teachers, while reducing budget of those schools graduating ineffective teachers. These types of policies would like change the engagement level of education professors.

Michael Barber from McKinsey & Company responds:

This is important. It about culture change. Often people coming into teaching through TFA for example are prepared for a career in which incentives and other aspects of reform are part of their lives.

Teacher preparation progams do their students no service if they prepare them for a world long since gone.

A crucial underlying point here is this. There is not and has never been a system where incentives don't play a part/ All systems have incentives - the question is whether those incentives are explicit or implicit. The incentives in the old world are implicit and generally reward shutting the classroom door and teaching whatever you want to teach as long as you don't casue trouble...until you get your pension. Meanwhile on your behalf in order to make your life marginally easier someone will try to negotiate slight reducations in class size and steady incremental rises in salary. The incentives are therefore not to rock the boat, not to innovate, not to worry about evidence, not to develop yourself but to hang on in there. Of course many individual teachers went far beyond this for reasons of intrinsic motivation and commitment to the moral purpose.

The debate here - and for teacher preparation programs - should be about the incentives we want that support and encourage those whole have that intrinsic motivation so that those who use evidence, develop their colleagues and deliver results are rewarded and progress.

Let me begin by saying that I was born and bred in Texas and yes, it is a country unto itself :)

The issue of creating positive incentives for school improvement seems as if it has been made incredibly complex. The first question I would ask, is what is the goal? What outcome are we truly looking for? Are we measuring student success, or teacher skills? The second question would probalbly be, how are we going to get there? If we are truly looking at student success, why can't we start as simply as learning gains and then move on to a tiered system?

I am a Special Education teacher at the high school level. Many of my students read on a first grade level. If we are using an incentive system that only rewards teachers whose students achieve a certain level on a state's test, or who manage to go on to a two year or 4 year post school program, where is the incentive for me to work harder than I am? ( I say this tongue and cheek, because I am committed to helping my students gain in all areas of learning ). In addition, what do we do with a system that has a disproportionate number of low level learners, or teachers in an Alterntive school setting? I know from personal experience that when I taught at our alternative school ( a placement for students with pending felony or severe behavioral issues) that when the other schools recieved incentives for improvement, we did not.

Posted by: Jeffrey Horn from Walden University

DOMINIC BREWER from University of Southern California responds:

The starting point should be gains in student learning.  Then we recognize that not all students can have gains calculated for them, and there are measurement issues in generating good gain indicators etc.   I think progress is being made on these issues.   Some of the TIF grantees, and balanced aproach of the TAP program, show that it is possible to state clear objectives for students in the absence of state standardized assessments, and still measure progress at points in time.  So long as there are a set of measures, rather than a single measure, I think it is possible to measure what ultiamtely we care about which is student progress.

Robert Manwaring from Education Sector responds:

You are not alone. Value added growth measures based on standardized assessments may work well for measuring English and math for most students in some grades, but it is a limited number. That means elementary teachers [grades 4-6 because you need two years of data (3^rd grade and 4^th grade) to measure growth), and a limited number of middle and high school teachers.  But there are a lot of other subjects and students with special needs where the standard assessments are not appropriate. So while using these measures as part of a teacher incentive program may make sense for some it will not make sense for all or even most. This is why a rigorous teacher evaluation system is a critical element. Developing other local student outcome measure to incorporate into an evaluation system is a good idea but can only play a part of a system.    

You hit on another issue that in my opinion has not received enough attention namely alternative education schools. States have different approaches on what alternative schools are, but generally these are high schools focused on dropout recovery or disciplinary issues. Accountability for these schools is difficult. These schools often serve students for a short timeframe (1-6 months) that make standard accountability measures meaningless. Because the students attending an alternative school this year are completely different than the students that were there last year, common measures like the number of students that are proficient in a given year don't tell us a lot about the effectiveness of that school. They are more likely to reflect the random draw of students sent there in a given year. Worse, currently there is an incentive for mainstream high schools to dump students into the alternative high school around testing time because these students are often not likely to score well on the state tests. Often times when this occurs the student's scores do not count toward either school's AYPs because they were not tested at the mainstream high school, and had not been at the alternative school long enough. This is problematic. Finally, many students drop out in the process of moving between alternative schools and high schools.

Here are some possibilities to improve accountability for these schools. First, for accountability purposes, the scores of the students in alternative high schools should be linked back to the student's school of origin. These alternative schools are temporary in nature and usually one of the top goals of these programs is to get the students back on track in the mainstream high school. By linking the scores back to the mainstream high schools it will create the incentive for the high school to actively engage and coordinate with the alternative school to best support the student. Second, for purposes of holding alternative schools accountable, state should develop standard-aligned pre-post tests. Ideally, these assessments would be taken online using dynamic testing. The ability levels of students transferring to alternative high schools can vary a lot. Some students will be several grade levels behind while others will not. This is a challenge for a test to provide effective placement when the range of abilities varies so much. Dynamic assessments make the questions more difficult if a student answers early questions correctly and makes questions easier if the student misses the early questions. In the same time dynamic assessments can get a more accurate read of a student's ability over a broader range of skills. A quality pre/post test would help with figuring out how the alternative school can support a student's academic development. Then having the student take a post test would allow a value added measure to be determined. Using a pre/post test value added result is a better way to hold alternative schools accountable.

I think part of this discussion needs to be on incentives/sanctions that promote an air of transparency for districts/states and mitigate the "need for" states to game the system. We currently have states with very good (in a relative sense) interstate test scores, but terrible national scores (e.g., NAEP).Posted by: Andrew McEachin

Michael Barber from McKinsey & Company responds:

 This is a good point too. Much will depend on the development of national standards which is gathering pace. Earlier this week 46 states signed up in principle to adopting common national standards in English and Maths. Ultimately this would lead to the possibility of benchmarking states against each other on tests set against these standards - and then incentives for states could in theory be introduced.

Australia is some way down this track having introduced universal tesing in four subjects (Eng, Math, Science and Social Studies) a couple of years ago which has led to state performance being compared. Those who do less well certainly coe under some pressure as a result. 

In theory,  a system would align the incentives - both positive and negative - at each level in the system. this is what we did with some success in England in the late 90s; the incentives at school, district and national level were aligned around a specified target level of performance; later we allowed the clarity of what we introduced to be diluted. Learning from England and building on it, they have done a nice job of alignment in Ontario on literacy and numeracy standards in the last four years. they have not used financial incentives there but have made the school-by-school data transparent. 

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

When we first made our recommendations to the Congress as part of the NCLB process, the Administration actually proposed a rewards program for states that made the greatest gains on the NAEP. I thought this was a good idea at the time, and I believe the same today. It caused great consternation at the time among some conservatives about federal influence over standards. The idea might have greater currency today. Plus it would go some good distance toward addressing the concerns of this questionner.

Michael Barber from McKinsey & Company responds:

On the recurring theme of Texas one of the most surprising things I ever learnt is that one of Santa Ana's attempts to attack Texas was severly disrupted by snow. Now that really is a negative incentive. 

Also picking up the point made way back by Sandy about whether or not incentives should be based on inspections or test scores, my view is that financial rewards should be bsed on objective measures of progress, ideally a run of results not just a one year blip and ideally taking value-added into account. However, for interventions in underperforming schools - which might be triggered by test scores or some other trigge - an inspection by qualified inspectors is really helpful in understanding the state of affairs and helping to tailor the nature of the intervention. The problem with intervention s in many American states is their ineffectiveness, the result of which is many lists of underperforming schools but very little effective action to improve teaching, learning and outcomes.

DOMINIC BREWER from University of Southern California responds:

This is important and it looks like it is slowly happening.   It seems odd that MA and Ok can have such wildly differing expectations of students.   In a global economy this is not sustainable.   The rhetoric from Secretary Duncan has been very clear that states should not fool themeselves or the public into claiming high scores or significant progress, if NAEP doesnt refelct this too.  There have been a number of good benchmarking efforts comparing standards and assessments.   Incentives such as those Sandy references would undoubtedly speed the process of shared standards and expectations, and ultimately shared or very similar assessments. 

Robert Manwaring from Education Sector responds:

 

Getting states to agree to talk about common standards is the easy part of this battle. The real battles will come later when states are asked to walk away from their locally negotiated standards and adopting common standards or at least benchmarking against them. There will need to be a quid pro quo to encourage states to make this transition. In NCLB reauthorization there will be an opportunity to provide the positive incentives for states that are willing to adopt more rigorous standards. The federal government could provide states adopting rigorous standards with flexibility on the implementation of other portions of the law. So that states like Mass that adopt challenging standards get the added flexibility while states like OK or Tenn would not.  Effectively earned autonomy applied to the states.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

I want to clarify a point about inspections. I very much agree with Sir Michael about the potential value of well structured inspections. I also agree with him about the importance of solid implementation with fidelity. My concern is with using inspections as the basis for incentives.

 

As to the timing of incentives, I understand and agree with the point about the value of timing certain incentives over a period of time greater than a year. But improvement in results, however well measured, can be seen each year. There are other incentives that ought to provided to recognize and honor this annual improvement.

Early in the discussion, Rob linked financial rewards to performance pay. Do school rewards make any sense if a district does not allow differential compensation of teachers and staff? Can rewards work without overhauling how we compensate teachers?Posted by: Andrew Rotherham from Education Sector

DOMINIC BREWER from University of Southern California responds:

Hmmm, this is an intriguing question Andy.  Theoretically one could create very large school based financial incentives that might counteract the perverse incentive effects of existing teacher compensation.  But that seems awfully wasteful and not the best approach.   One could try some significant school based non-financial incentives within the current teacher compensation structure -- additional flexiblity over resources including allowing for (or encouraging) additional collaboration time, enhancements to working conditions and such.  This would require some $$ from somewhere, which is the dilemma.  If one isn't willing to "re-program" resources tied up in the salary schedule, they have to come from somewhere and that is tough - I suppose one could use marginal federal $ from Title I or save a bit on some state programmatic items, but that isn't where most of the resources are... We are left I think with a pretty compelling case that any incentives at the school level, be they monetary or non-monetary- will require resources taken from somewhere else.  It's hard to see that without a radical restructuring of teacher compensation.   But the case for such a restructuring is powerful absent this argument -- why accept incredibly powerful peverse incentives (to acquire additional education not linked to student achievement, for younger smart teachers to quit early, for risk averse individuals to enter and stay in a profession for years regardless of talent or passion, for experienced teachers to go to schools that need them least, for a big surplus of teachers in subjects they arent needed and shortages in key areas where they are.....) which tie up 60-70% of the dollars as a "given"?

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

Andy, this is exactly what we did in Dallas for several years. We honored the schools that made the greatest gains on our value-add measures. The winning schools also had to have a substantial number of their students over the bar of grade level proficiency. The schools were honored at a city-wide lunch and received flags and other honors. Each teacher in the winning schools received a "bonus" of $1500 or so, and staff received something less. This was in the 90s, and the payments were well received. School communities wanted to win.

 

The system was run by an accountability taskforce made up of key stakeholders, and, because it was growth-oriented, there was a lot of buy-in to the measurements.

 

Our R&E folks, Bill Webster and Bob Mendro, were tops, equal, I think, to Bill Sanders, and they ran a first-rate program. We found positive effects on achievement in the district.

 

This is clearly a good path for districts that are not ready to go to differentiated teacher comp.

 

The issue, as Dominic points out, is how does one sustain the funding over time, given the competing demands for money. It's not easy. There really must be a powerful constituency, usually the business community, if it is engaged and so inclined, that will insist on sustaining the program.

 

I believe that differentiated pay, though harder to start,  is probably easier to sustain. We've had three tough votes in the Texas legislature, but we've kept pay for performance alive. The good news: it seems to be building its own constituency, and it's seen as another form of compensation.  Thus, in the long term, it may be a better path than school rewards.

Does the panel see law makers building demand-side incentives into the reauthorizing of NCLB (beyond the choice options for student attending failing schools)? Fiscal and governance incentives for states/districts/schools/teachers bring an air of "marketization" into the education world, but what would that look like if we pushed that one step further to students and parents. The removal of catchment zones through the institutions of large-scale vouchers could provide another level of incentives for schools/districts/states to become more efficacious. I know the research is mixed on demand-side choice reforms, with some glaring trade-offs at times, but where do they fit in this discussion?Posted by: Andrew McEachin

Michael Barber from McKinsey & Company responds:

Market incentives could be an important part of this. While very few systems have adopted pure voucher models (Chile is one of those that has) increasing numbers have the money following the student within the public system—New Orleans would be an example. Where there is an excess of places this does have an incentive effect. Encouraging this was a key part of the Blair education reform here in England.

However, market incentives in school systems are not as powerful as in some other sectors for the simplereason that parents are rightly reluctant to keep changing where their children go to school. Whereas in health it is perfectly plausible to have knee operation in one hospital and a hip  operation somewhere else, ideally you want to choose a school for a child which will work out for a number of years. For this reason other incentives have always seemed to me necessary over and above market incentives.

This doesn't mean market incentives shouldn't apply; just that on their own they are not enough.

Incidentally, returning to the discussion on the balance between positive and negative incentives,  the evidence from psychology would certainly suggest that the positive should outweigh the negative in weight. the rule of thumb I arrived at some time ago was a ratio of 3 to 1. Interesingly (or not) I read an article in a Sunday paper supplement a few years ago that said that one of the most noticeable factors in successful marriages in the US was a praise-blame ratio between the partners of 3 to 1! (Have you tried achieving this by the way? Its a serious standard!) Whether this has any bearing on the incentives debate I'm not sure - but I do think many systems get this ratio out of kilter. In fact, in the reforms in England, in spite of the efforts we made to accentuate the positive, I'm sure most teachers would say that they heard the negative far more loudly than the positve. One of the benefits therefore of whole school bonuses, such as those introduced by Joel Klein in New York City, is that they give practical effect to the positive message - they put money where the mouth is, as it were. This is a significant argument for bonuses of this kind in my view.

Michael mentioned that the current focus on negative incentives often leads to the media accentuating the negative and that positive incentives could impact the public perception of school reform. Explain more about the role of positive incentive, the media and why it is important to the success of broader education reforms.Posted by: Andrew Rotherham from Education Sector

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

Positive incentives are very useful, Andy, and if implemented more broadly, could be even more so.

 

It was very helpful in Dallas, a big urban district, to put the spotlight on schools that were making great strides. This was particularly so for schools that served the poor as well as schools that were turning around sufficiently to win an award. In other words, it showed that "it can be done." This gave a morale boost to the community of the school that won; it helped the superintendent recruit solid folks to schools that had potential; and it gave the broader community more confidence in the district and, thus, more support for bonds, taxes, and the like. This confidence, by the way, was due not only to seeing schools that work but also to the fact that the district bothered to distinguish in terms of consequences between those that work and those that don't. 

 

As a member of the select committee that considered changes to our state accountability system, I noted with pleasure the panel of leaders from large, medium, and small districts who testified with exuberance about their varied ways of implementing pay for performance. The legislators were impressed, as was the public.

 

Andy, your question is an important one. I hope these responses help.

Michael Barber from McKinsey & Company responds:

I agree with Sandy's remarks. In order to accentuate the positive, on the day we published school results we would put out lists of the 100 best schools in England, the 100 most improved schools etc. This really did help shift the focus of the media onto success whereas if the results just go out, the media inevitably look underperformance.

We also introduced National Teacher Awards, invited successful principals to No10 Downing Street, ensured that the schools with the best inspection reports were named in the chief inspector's annual report.

 

Perhaps most significantly we introduced a 'beacon school' scheme which gave extra funding to schools with a proven trackrecord of success - the funding had to be used to provided demonstration lessons or other kinds of support to other schools. This worked well for a while. It generated controversy at the sart (why we're picking out some schools from the pack and rewarding them?) but this was exactly the controversy we wanted to have. the evaluation showed that the scheme brought improvement not just to the schools assisted but also the beacon schools themselves.

It seems that there is little attention paid in these debates on educational policy to the role of technology in helping to fix these issues. Should we not being looking at increasing the ability for children to access computer-based learning, be that in the form of immersive simulations, online synchronous or asynchronous student-teacher interactions, games-based learning, adaptive testing, forums etc.? This would also be far, far cheaper, bearing in mind that "labor costs are about 70 percent of a school district's costs."

It seems like a lot of this debate is focused on how to incent players in the historical educational structure of a teacher standing at the front of a classroom, which seems a little like saying: how should we approach merit pay for the corporate mail room, rather than suggesting email?

Posted by: Jonathan Dean

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

This is a good question.

 

I believe incentives ought to be used primarily to reward improved student results, however achieved. I would like to think that the more we reward better results, the greater incentive key players will have to structure the way they operate in more efficient, productive, and effective ways. So, if it's more effective to teach through a better use of technology, such practice would be far more powerfully encouraged because the better results from such practice would be rewarded.

 

If the basis for incentives shifts from proven results to different ways of doing business, I fear that politics will take over and reward favored inputs and faddish stratefies. This will not help. Indeed this concern, by the way,  should be kept in mind in the administration of Race to the Top.

 

Having said all that, I fully recognize the merit of the point the questioner is making. One possible approach that comes to mind is an idea that's been promoted by an education resources consulting group in Texas. They've recommended that ratings be based on a mix of student results and efficiencies in spending that schools and districts can effect. In other words, the highest rated schools and districts should be those that get the best student results with the most efficienctly spent funds. This group has developed some pretty sophisticated ways of measuring for this. I think it would not be a far stretch to find ways to reward such schools and districts.

DOMINIC BREWER from University of Southern California responds:

Long term major productivity improvements undoubtedly have to come from substitutin capital for labor. This doesn't mean we won't need adults in schools but I suspect the role will only in part be like traditional teachers.   One of the challenges on accountability systems is ensuring there is the autonomy, resources and capacity to innovate at teh school level so that there is a drive to search for new and better ways fo doing things.   This is easier said than done because legislators typically are reluctant to grant autonomy, provide suffieicnt resources or genuinely help build capacity for innovation.  Instead - and this is certainly true in CA - standards based accountability is layered on top of an input based regulatory framework and little is done to seed or spread innovation.  Even if you have brilliant incentives you still will need to create the space, provide the resources and help build the capcaity to respond to the incentives.

Robert Manwaring from Education Sector responds:

 

There are two different approaches to using technology in the education process, and the incentives will vary for each. First, technology can support the educational process in the traditional schools. Here, I would agree with Dominic's comment that we need to create the incentives for schools to be effective, and then provide the schools the autonomy to take the steps that meet expectations in a cost effective way. Schools can certainly do a better job of incorporating technology into the daily operations of schools. And, as new teachers enter the profession who grew up with technology playing such an important part of their lives, I think that this transition will happen.

A second approach is more of a school choice model of students taking on-line courses. The potential of these courses are especially powerful at the high school level. Students can take AP courses, HS unit recovery, or simply choose from a broad set of course offerings. On-line courses can be a cost effective approach to education. That said, the incentives are complicated, and ensuring the quality of on-line courses will be important.

For example, students with credit deficiencies are at a higher risk of dropping out of high school if they don't think they will have enough units to graduate on time. To address the chronic student drop out problem, it makes sense for students to be able to make up units by taking on-line courses. But, will a student shopping for on-line courses choose the highest quality course or the one that requires the least amount of work? Who gets to decide? Who should pay for the course? Who should ensure the quality of these courses?

Online courses provide a huge opportunity to expand choice as a replacement or substitute for classroom based instruction, but there are a lot of incentive issue that will need to be address to take full advantage of the potential this reform.

Since high schools are supposed to prepare students for employment as well as colleges shouldn't employers have a way to input their requirements into the incentive scheme? Also, since employers habitually include such things as responsibility, teamwork, and oral communication (also needed for college success) shouldn't these be behaviors and skills be included?Posted by: Arnold Packer from Western Carolina University

DOMINIC BREWER from University of Southern California responds:

Yes and yes!  The latter point returns us to the need for better, multi-dimensional assessments that assess not just academic content knowledge.

Robert Manwaring from Education Sector responds:

 While I agree with your assessment, the critical issue is one of measurement. My ES colleague, Elena Silva recent wrote a paper on the measurement of 21^st century skills (here).  Until we are better able to measure the 21^st century skills that you discuss, it is difficult to determine whether schools are doing a good job or bad job of teaching these skills. Of course until there is a broader consensus about what these skills are, they will be tough to measure. Basically, the education community has a lot of important work to do in this area.

Race to the Top funds are creating an unprecedented opportunity to reward states that are doing the right things. Will these incentives work? Why or why not? What if anything should Congress do in NCLB reauthorization to build the Race to the Top?Posted by: Andrew Rotherham from Education Sector

DOMINIC BREWER from University of Southern California responds:

It's hard to be optimistic.  Distribution of federal funds seems likely to devolve to spreading the money around and/or rewarding friends or fads or favorites.   We have hard time in education rewarding success - there is that continuous urge to throw resources at those who are underperforming in the belief that with just a little more help they will succeed.  We don't like making judgements of students, teachers or schools because we are not sure we can be fair and worry that judging negatively will cause harm.  

Reconceptualizing the federal role as one of an incubator of good ideas, the grower of profoundly radical innovation, and disseminator of good practice, seems appropriate.  It's not clear to my why one needs to go through states - or state departments of education at least - to play this role.   You are more likely to get a bigger bang for your buck by having open national competition, with a rigorous selection process.  I suspect fewer very big bets would be more effective than a very diffuse distribution of funds to lots of projects, which is what will happen.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

 Great question, Andy.

 

Above all else, performance on, say, NAEP ought to be a fundamental part of the criteria for receiving Race to the Top funds. If the Secretary is going to limit the funds to a relatively small number of states, why in the world would he give $$ to states that perform toward the bottom on NAEP, particularly in results for African American, Hispanic, and low income students?

 

I know these funds are designed to support initiatives to accomplish the four big goals in the statute. But why would states that have done little to lift the achievement of the very students the Act is intended to benefit be able to qualify among a small number of recipients? How credible would their plans be, however well written they may be? One darn good way to tell a good proposal from a bad, especially given the artful way grant writers do their job, is to see how the states have done at getting the results the Act itself is promoting.

 

I realize that past results will not and should not be the sole criteria. The Secretary should expect solid, strong plans under the requirements of the Act. But I hope he'll be careful in avoiding granting money to poor performers. That would not be a good "incentive" for reform!

Robert Manwaring from Education Sector responds:

 

As discussed above, incentives tend to work best when in this case states know about the incentive ahead of time, and have an opportunity to make a change in response to the incentive. In this sense, I don't think that Race to the Top (RTT) is structured very well. Likely the specific rules of the game will come out at the end of summer and the first round of funding will be awarded some time in the fall. Now this will not give state's much time to make changes that would give them a better chance of winning a RTT grant. So from an incentive perspective, the RTT incentive provides no time to work. Basically during the application time period, state legislatures will not be in session and nothing will change.

Now Sec. Duncan is doing the best job that he can to get out there and push reforms using the bully pulpit. He basically scolded California for its poor performance of late, especially the state's budget situation. He suggested that California has lost its way (here). And he has pushed strong for allowing charter schools and loosening of charter school caps and a push for common standards. This may be having some incentive effect as 46 states have committed to that process (adopting will be another story).  Another example, Maine is currently seriously considering passing charter school legislation. And as their local paper put it, "Maine stands to lose too much by resisting this reasonable reform effort" in reference to the Sec. threat.

The hope is not high for this funding to leverage significant change. For example, Washington state, the largest state without a charter school law, the issue was not even discussed in this legislative session which has already finished for the year. So, if as the Sec. recently suggested, states without reasonable charter school law will be disadvantaged in RTT funding, does this put WA out of the running.

To be clear, even if the incentive effect of RTT are not successful, it does not mean that the funding was wasted, as the funding will still go to some set of progressive states that hopefully will do good things with the funds to continue to drive reform in their states. But, it is not too late to create real incentives that could drive much more of the system. More time is the key. Ideally, in the near term, the Administration would push for Race to the Top (round 2) as a key component of NCLB reauthorization. Even putting that possibility out there may lead to state's taking round 1 more seriously. If the federal budget does not have room for another RTT, then the Administration should consider going back to Congress to extend the timeframe of the RTT funding, and then backload much of the funding. If for example, the rules of the RTT game were well established, and state's had an entire year to take action to best position themselves to as contenders for the funding, then you would likely create the incentive to get the best bang for the buck. If these changes can't happen, then at least the Sec. needs to play hardball in the first round of applications, and hope to pressure some states to make changes before the second round of funding is available.

In the larger context of educational incentives, how much attention is being given to scaling up the idea of attracting the top 15% of high school graduates (who also have the natural ability excite youngsters to learn) to enter the teaching profession by paying a substantial amount or all of their tuition with the expectation of a 3 - 5 year teaching contract?Posted by: Winsome Waite from Learning Point Associates

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

I'd rather see us move toward the sort of teacher evaluation systems suggested by Kate Walsh at NCTQ and others and implement the sort of differentiated pay programs that have been mentioned in this discussion.

 

You're right: we need to begin to attract the best and the brightest into teaching. But, I think, as Caroline Hoxby and others have so amply demonstrated, we will likely get to that goal better by having the sort of differentiated pay and working conditions that help attract the best in all professions.

Robert Manwaring from Education Sector responds:

 From someone who knew exactly what I wanted to be when I grew up as I was leaving high school, and then proceeded to change majors multiple times and add a couple of minors in my undergraduate career, this approach may be a poor return on investment. So, generally I would agree with Sandy on this one. That said, I think that there is opportunity to expose high achieving high school students and even community college students to consider teaching as a profession. I think that high achieved in both of these settings could be targeted to form a tutoring corp to assist younger students in reading and math. From my perception, cross age tutoring is underused. You could set up programs that trained these young rising stars in the basics of teaching as part of this process. For the younger students in need of additional assistance, you provide an opportunity to get supported by someone from the students local community who is interested in going to college and comes from a similar background. For the tutor, you expose them to the personal rewards of teaching and provide them with a minimum wage job that is much more rewarding than working as a bus boy or in an ice cream shop as many of us did in our youth. Such programs may lead to future educators, but even if they don't they would likely be cost effective programs.

Robert, thank you for your thoughtful answer. I just want to clarify a few items related to ELLs and NCLB for the panel. The foundation of language development principles are universal. It would be very helpful if Congress would commission a National Language Development Panel similar to the National Literacy Panel and the National Reading Panel with reseachers, linguisitcs, educators and teachers to review the research literature and studies and develop a common "set of language principles" that educators could use as the foundation of developing language proficiency standards. Since 2003 there has been a national effort of states working in consortia to develop language standards for ELLs, however, there is a basic lack of understanding of the science and research that form the underpinning of language development and langauge acquisition. Often the conversation quickly turns to program approach as opposed to the principles of language. It would facilitate ELL academic achievement and language acquisition if Congress would consider establishing such a panel. This could lead to a common understanding and definition of LEP and support state efforts in ELP standards development that inform academic achievement and could lead to states, districts, schools and teachers embracing incentives for instructing ELLs. Finally, I just want to clarify that ELL students are no longer allowed to be placed in separate learning environments- ie segregated instruction- while developing language. ELLs are not removed from regular classrooms during core instructional time. This is new for content teachers who now need the instructional strategies and tools to deliver language instruction and core academic content instruction simultaneously. These factors viewed positively may increase support for incentive rewards for teachers, schools and districts. Additional thoughts?Posted by: kathleen leos from Global Institute for Language and Literacy Development

Robert Manwaring from Education Sector responds:

 

Kathleen, thanks for following up. I am all for a national panel focused on improving our understanding and developing best practices. As I suggested earlier and you suggest as well, there is much room for improving the quality of the tools and supports that the system provides to teachers of English learners. I think that the lack of consensus in the field would make it more difficult to take the next steps of adopting common definitions for reclassification or national standards. I have some experience with the California system, where they are still a long way from getting a common definition of LEP across school districts.

On your last point, I think that it would be great to incorporate measures of language development in a school's AYP, rewards or sanctions, teacher evaluation processes ... We know that what gets measured is what gets done. And improving the focus on the quality of language development instruction would be a huge step forward. I am not sure that the education world is ready for higher stakes in this area until there is a broader consensus. This goes back to your recommendation of a national panel being a good place to start.

This question may be directed to Mr. Kress, but we foreigners rarely have time to discuss things. 1. The implication is that teacher aren't doing their best. Is this your take on the situation? 2. Teachers are at-will employees for 3 full years. If they are ineffective, then why don't administrators release them with our sincere gratitude (like the "contestants not appearing on stage" on The Price Is Right, who receive a toaster and a year's supply of Centrum vitamins)? 3. "Student Achievement" has been substituted for test scores throughout the discussion (with one exception I can find). Instead of discussing teacher "merit" or teacher "performance," why can't we just call it what it is—Pay for Test Scores—or PTS? Thank you ALL for engaging in this discussion.Posted by: Paul Henley from Texas State Teachers Association

DOMINIC BREWER from University of Southern California responds:

1) is a completely false characterization.  Are all teachers lazy?  No, clearly not.  Many work extremely hard and to the best of their ability.   In any organization, there are some employees who work harder than others!   There are some who are more skilled than others!   There are some that are brilliant performers in one task and lousy in another task!   Does saying it is possible to devise incentives so that some employees work harder imply that they all are lazy?  No of course not.   Can we make judgements about who performs better than others?  Yes, in every profession we can.  Is it hard?  Yes, performance isnt one dimension and measuring it fairly isnt easy, but clealry it can be done.  The fact that the current system of compensation treats all employees as i they are the same, is extremely inefficient - it assuredly does not get the best out of every employee and it assuredly wastes a lot of resources by rewarding traits that don't matter, overpaying lousy employees and underpaying exceptional ones.

 

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

1. I don't know of anyone who is doing his/her best. Kobe Bryant probably didn't think he did his best last night, though the Magic most certainly thought he did! The question is how can we do better. All professionals can do better, and the issue is how we can create an environment in which teachers are treated fully as professionals, including appropriate evaluation and response. I think that's the central purpose of this conversation - not to say teachers are or are not doing their best, but rather how teachers, as all other professionals, can do better, and get the help they need as well as the support and honor they deserve for the work and results they accomplish.

 

2. So, this is NOT about getting rid of teachers, and the remedy of relieving ineffective teachers from duty after three years in the classroom is really not the point. Again, the idea here is that the world of differentiated responsibilities, opportunities, and rewards is the world of professionals, and these ideas, including incentives, ought to characterize more and more the world of education.

 

3. Test scores are critical measures of success, yes, but by no means do I think they should be the only measures. Whether in the TAP program or in other such programs that are being developed around the country, teachers, administrators, and other stakeholders are imagining all sorts of ways of measuring success. As long as the metrics are fair, objective, and related to student success, I, for one, think they're all legitimate for consideration

Increasing autonomy for high performers has been a recurring theme in the discussion. At which level—school, district, state, federal—is increased autonomy most likely to pay off? How could/should autonomy as an incentive be aligned at each level? Also—Sir Michael, please elaborate on England's success in aligning incentives at the school, district, and national level, and how England later "allowed the clarity of what [was] introduced to be diluted"? What lessons might that hold for NCLB reauthorization? Thanks!Posted by: Kelly Scott from Margaret Spellings & Company

DOMINIC BREWER from University of Southern California responds:

This is a great question and not one easily answered because there isn't great empirical evidence to guide us.  Even if we try to use principles such as "those with the most knowledge should have the power to make decisions" we don't get a clear picture.   But a hard look at all sets of relationships both vertical (federal-state, state-district and district-school) AND horizontal (sate-state, district-district, school-school) asking what is most appropriately done by each level and how the relationships should be structured makes sense.   My own gut is that the biggest barriers are the state-district relationship through over-regulation and categorical funding and then district-school through collectively-bargained restrictive labor agreements.  But it isn't a clean picture (because for example, some districts actively seek waivers from regs, state policy sets the framework for bargaining, and so on) and varies a lot across states.  Resource allocation decisions are likely to be most efficient at the school level, given the right skillset of managers and tools for making decisions, but obviously the higher levels play key roles (as auditor, as capacity-builder, as source of economies of scale in purchasing, as convener of national expertise, etc.).  The conventional wisdom was that higher levels of authority were needed to ensure equity.  But is seems to me the reality of urban public schools belies this- clearly there isnt the will to be as aggressively re-distributive as may be needed to reduce inequities.  It's hard to see how one could do worse with more decentralized resource-decisions.

Sandy Kress from Akin, Gump, Strauss, Hauer & Feld LLP responds:

Kelly - it's truly nasty of you to pose this question toward the end of our discussion! This is a hard one.

 

I've promoted this idea of increased autonomy as a reward for the last 20 years. Yet, honestly, I'm not sure I've ever known what I really meant by it. I tried to foster it when I was on the Dallas school board. We've tried to do it in state policy in Texas over several re-writes of the Education Code. And we tried to "free up the states" in policy through all our deliverations over NCLB.

 

As the Chairman of the Senate Education Committee in our state asked in a recent hearing, "Now , as much as I want to let the winners loose, what exactly would that autonomy look like?" More control over spending? Maybe. More freedom over professional development decisions? Letting the school out of certain regulations? If so, which ones?

 

There's plenty of work to do here, and we'll have to do better in future discussions with greater thought and detail.

Robert Manwaring from Education Sector responds:

 

I think that there are opportunities at all levels, but I think as with most policies the biggest bang would be at the local level. For higher achieving schools that have the leadership capacity to operate more independently, disticts would provide the school resources and some minimal guidance, directing a higher portion of funding to the school site (because there is less central office support for these schools). In this approach, over time, a district central office would transition into serving two functions - supporting lower performing schools, and providing contract services to the higher performing schools. This type of model would also likely lead to greater differentiation of schools and greater parental choice. It would require a rethinking of district resource allocation, and decision making, and likely a dramatic reduction of central office support. Dollars would need to truly follow the student as they move from one school site to another. Now over time, school would likely realize that some things are better done centrally, for example payroll and legal. Also, there would need to be a process for a school that was high performing to be taken over by the district if the schools achievement started to fall.

State's could support this type of reform, by starting to deregulate the state process. The greater the state regulation, the harder it is for school districts to devolve decision making authority. For example, take Title I funding. Currently districts often operate and oversee Title I funding centrally. Because of the reporting requirements, and regulations of the uses, districts centralize the operations to ensure that the funding is used in a way that is consistent with the regulatory requirements. Thus, often the current central offices are a set of categorical silos that in large districts do not talk to each other.

Now as with any policy, this approach on its own is not a panacea. We know this because there are many districts with one or two schools in this country, and these schools are not dramatically outperforming other districts. But, this type of policy would support many of the other recommendations made in this discussion.

At the federal level with reauthorization approaching, I think that there is an opportunity to use earn autonomy as well at the state level. The reauthorization could provide states with greater flexibility with the implementation of their accountability model if the state took significant action to address the four assurance areas in the stimulus package - teacher quality, low performing schools, using quality data and quality standards and assessments. To get this flexibility, the key would be that the state was taking action in these areas which is a much higher bar than providing assurances.

Thanks Rob, Michael, Sandy, and Dominic for an engaging conversation. Any closing thoughts to share?Posted by: Andrew Rotherham from Education Sector

DOMINIC BREWER from University of Southern California responds:

Thanks to my fellow panelists, Rob and Andy and the questioners for a stimulating discussion.   It is interesting to see the directions the discussion went - towards teacher compensation issues, around outcomes measurement, and on how to grant autonomy.  One re-read of the discussion reveals relatively few concrete examples.  This tells me we are still very much in the early stages of figuring out how to structure a set of rewards and sanctions that gets the most out of students, parents, teachers and administrators - we are maybe at the stage where the Wright brothers have just gotten the plane off the ground, but a very long way from transoceanic flight.   The good news is that we are able to have a conversation about structuring incentives in the K-12 context; 10 years ago even the topic might have seemed alien.  In true policy wonk fashion, I'm inclined to believe it is possible to design better incentives, and they should be largely poisitive ones accompanied by some very firm "negative" ones (such as closing schools or converting them to charters).  We're gonna have to get more serious at focusing on student incentives.   And we are gonna have to consider true "high-stakes" for all players, which we do not have today. 

However, it is crucial that we have in place a clear plan for introducing different incentives in a planned way and monitoring and evaluating their impact.  This requries the ability to make policy more sensibly and to learn over time.  Some states have this, others (like my own) do not.   We need a much richer array of evidence on what can and cannot work under different contexts.   There is never going to be a perfect system of incentives, it will always be a work in progress, contiunously changing.  Much of our discussion is based on informed hunches.  That's better than nothing - and not having much evidence is no excuse for the status quo, where it seems the implicit incentive structures do not work.  But hopefully if we are asked back to one of these (or whatever technology has us doing) in a decade, we would have a more systematic way of predicting what would and wouldn't work.

Robert Manwaring from Education Sector responds:

 First of all, I would like to thank all of the panelist for the seriousness with which they approach this discussion. Their dedication has lead to my own thinking on this subject evolving over the last couple of days. I would also like to thank Tom Toch who suggested conducting this talk based on some informal conversations with Sandy and Michael. Finally thanks to the audience for pushing the conversation forward to address some difficult issues. What is clear from the audience questions is that no policy can operate in isolation, but instead these issues are interrelated. So, instead of ad-hoc rounds of reforms, education policy making needs to take a more integrated approach. Over the next several months our plan is to build upon the seeds planted in this conversation to develop a more formal paper on the roles of positive incentives in our education system.

Please feel free to email me directly (rmanwaring@educationsector.org) if you have addition comments that you would like to make as we continue to investigate this topic.