In the late 1990s, Oakland A's general manager Billy Beane revolutionized professional baseball by ignoring what his players looked like and focusing, objectively, on how they performed. Now New York City Schools Chancellor Joel Klein is trying to do the same thing for public education—if the teachers union doesn't stop him first.

True fans of the public schools should root for Klein to prevail.

By crunching numbers without prejudice, Beane discovered that certain statistics that really mattered on the field, like on-base percentage, were being hugely undervalued in the player job market. While scouts and other executives made decisions based on personal bias and flawed perceptions, Beane kept to the statistical bottom line.

Traditionalists complained, but it worked. The A's routinely outperformed teams with twice the payroll or more (including, sometimes, the pricey Mets and Yankees).

Klein's new initiative is based on the same fundamental insight: Teachers who may by all appearances seem similar—same age, experience, level of education and demeanor—can be hugely different in how well they help their students learn.

And so, Klein plans to start using something called "value-added" data to figure out differences among teachers. The data compare annual test score gains in a teacher's classroom to statistically predicted gains, given students' backgrounds, previous academic history and a range of other factors.

At first, school officials are going to measure the performance of some 2,500 teachers—a small fraction of the 80,000-plus in the system. There are no plans to attach huge rewards or penalties to the results, at least not yet.

But the promise of the program is clear. Down the line, teachers who make great progress with previously low-scoring students could get stellar ratings. Those who coast with previously high-performing students—or fail to bring struggling kids up to grade level—won't.

Unfortunately, when the initiative was reported last month, United Federation of Teachers President Randi Weingarten rejected the notion outright, saying, "Any real educator can know within five minutes of walking into a classroom if a teacher is effective."

In this, Weingarten is following in the tradition of the baseball scouts who ignored statistics in favor of "natural athletes," players with a "good body" or those who just plain looked like major leaguers.

Some will consider it heresy for anyone to liken the fine art of teaching to the simple game of playing with balls and bats. To be sure, it's easier to measure home runs than appreciation of great literature. Standardized tests have margins of error that need to be accounted for. And it's fair to caution that value-added statistics should never be the sole way teachers are judged. They should only be used for high-stakes decisions like tenure or pay when other factors, like peer and principal evaluations, are part of the mix.

But in both baseball stadiums and schools, the stakes are high and there are serious differences among professionals based on the kinds of results they can produce. We ignore those differences at our peril.

To those concerned about fairness, let me point out: Using value-added statistics to inform teacher evaluations is far fairer than judging teachers by their students' raw test scores. It is far fairer than judging teachers by their principals' like or dislike of their work. And, yes, it is far fairer than making no distinctions at all. If some teachers are consistently more effective than others, it's only fair to recognize those differences.

In the last few years, teams like the Boston Red Sox have rebuilt their teams based on the ideas Beane pioneered in Oakland. New Yorkers don't need to be reminded of how that worked out.

Let's hope that in education, Klein and Mayor Bloomberg put the nation's largest public school system on the front lines of this revolution.