Lessons for Online Learning

Originally published in Education Next

Also from ES | January 27, 2011

Advocates for virtual education say that it has the power to transform an archaic K–12 system of schooling. Instead of blackboards, schoolhouses, and a six-hour school day, interactive technology will personalize learning to meet each student’s needs, ensure all students have access to quality teaching, extend learning opportunities to all hours of the day and all days of the week, and innovate and improve over time. Indeed, virtual education has the potential not only to help solve many of the most pressing issues in K–12 education, but to do so in a cost-effective manner. More than 1 million public-education students now take online courses, and as more districts and states initiate and expand online offerings, the numbers continue to grow. But to date, there's little research or publicly available data on the outcomes from K–12 online learning. And even when data are publicly available, as is the case with virtual charter schools, analysts and education officials have paid scant attention to—and have few tools for analyzing—performance. Until policymakers, educators, and advocates pay as much attention to quality as they do to expansion, virtual education will not be ready for a lead role in education reform.

Virtual education is in a period of rapid growth, as school districts, for-profit providers, and nonprofit start-ups all move into the online learning world. (See sidebars for just a few examples.) But without rigorous oversight, a thousand flowers blooming will also yield a lot of weeds. Real accountability, including the means to identify and end ineffective practices and programs, must be constantly balanced with the time required to refine new, immature technologies and approaches to learning. Both virtual education advocates and education policymakers should learn from nearly two decades of experience with charter schooling, another reform movement predicated on innovation and change within public education. After nearly 20 years of practice, the charter school movement provides important lessons on how to ensure that improved student outcomes remain the top priority.

Focus on Outcomes

At present, virtual education lacks a firm understanding of what high performance looks like. The situation is not unlike that faced by the charter school movement just a few years ago. In 2005, after a decade of rapid growth in the charter school sector, the National Alliance for Public Charter Schools (NAPCS) was formed to increase the availability of high-quality charter schools. NAPCS soon published "Renewing the Compact," a statement by its Task Force on Charter School Quality and Accountability. "Renewing the Compact" came on the heels of an August 17, 2004, lead story in the New York Times, which highlighted findings from a simplistic, and controversial, study of charter school achievement sponsored by the American Federation of Teachers, "Charter School Achievement on the 2003 National Assessment of Educational Progress (NAEP)." According to the Times, in "virtually every instance, the charter students did worse than their counterparts in regular public schools." The NAPCS task force did not mince words about the need for a sharper focus on quality within the charter school movement. The report challenged the charter community to "fully 'own' the issue of how well its schools perform" and also challenged charter advocates “to embrace rigorous measures of quality and accountability for our own schools' success."

But the wide range of education options within charter schooling makes "owning" quality difficult, and the variety is even greater for virtual education. Virtual public education can be delivered by all types of providers, including charter schools, for-profit companies, universities, state entities, and school districts. Types of online schools and programs range from state-run programs like Florida Virtual School, where each year 100,000 students take one or two courses online as a supplement to traditional schools, to "blended" models, which allow schools to combine online and classroom-based instruction. The most controversial virtual schools are so-called "cyber" charter schools—fully online public schools that students "attend" on a full-time basis. Funded with public dollars but independently run, many of these cyber schools are managed by private, for-profit companies such as K12 and Connections Academy. John Watson, author of the annual "Keeping Pace" report on the status of K–12 virtual learning, notes that virtual education is "several times more complex than charter schooling."

Such diversity brings challenges. While the International Association for K-12 Online Learning (iNACOL) has published program quality standards, virtual education lacks a commonly accepted set of quality outcome measures. Quality can't be defined by the design of a school or by inputs alone; instead, it must focus primarily on outcomes. Traditional measures, such as attendance and instructional contact hours, do not fit the virtual model. And while federal and state accountability systems, which focus on school-level accountability, provide data on and oversight of the performance of full-time cyber schools, there's little data and few mechanisms for evaluating supplemental and blended programs, in which students take only a portion of their schooling online. Moreover, it's the supplemental and blended courses, increasingly offered by school districts, where growth is likely to be fastest.

Still, complexity can't be an excuse for inaction. Unless providers rise to this task, outside groups, whether supporters or opponents, will define success and the lack of it for them. Once again, the charter experience is worth noting. Less than two years after publishing "Renewing the Compact," the NAPCS, in partnership with the National Association of Charter School Authorizers, which was established in 2004 to push for more professionalism and higher standards among authorizers across the country, convened a working panel on charter school quality with the goal of establishing a "common set of basic quality expectations and performance measures" to assess charter school success. Without these measures, the panel noted, "it is no wonder that judgments about the performance of charter schools are so frequently ill-informed." The result of the working panel was "A Framework for Academic Quality," which provides a list of indicators, such as student achievement levels and growth measures, to which schools should be held accountable, metrics that can be used to assess school performance.

Take Charge of the Data

Even with outcome measures established, it's unwise to assume that providers, be they districts, charter schools, or private companies, will collect data and conduct research on their own. States and districts, through their rapidly evolving data systems, must gather information about virtual course enrollments, demographics, and performance, and encourage further research into determining successful programs and practices.

Paul Hill, director of the Center on Reinventing Public Education at the University of Washington and leader of the National Charter School Research Project, notes that the 2004 Times article caught charter school supporters "flat-footed." Before the article, Hill says, "the movement was not thinking about what a bad study would look like." The Times story spurred investments in high-quality research on charter school performance—research that goes beyond snapshot comparisons of average charter-school and average traditional public-school performance to examine variation in performance among charter schools, incorporate measures of growth in student outcomes, and employ appropriate controls for student background.

In comparison, research on K–12 virtual education has been limited. A 2010 meta-analysis of virtual education conducted by the U.S. Department of Education, drawn mostly from studies focused on higher education, concluded that "students in online learning conditions performed modestly better than those receiving face-to-face instruction." But the report also found an "unexpected…small number of rigorous studies." Studies that only compare virtual learning with traditional instruction, though, like those that compare charter schools with traditional public schools, mask many of the most interesting questions about virtual education. To be useful, research needs to be specific as to "what works for whom, what implementation practices matter, and why," says Marianne Bakia, senior education researcher at SRI International and one of the authors of the Department of Education study.

"Keeping Pace" author Watson agrees. He fears that with districts everywhere experimenting with multiple forms of virtual learning, from online credit recovery to blended learning classes, three years from now we'll still have little to no information as to what actually works in a systemic way. And even if we know that a program is successful, we may not know why.

Watson cautions, however, that reliance on a few time-consuming megastudies would be a mistake. Not only are there a limited number of questions that can be answered in this manner, but perhaps more importantly, the field is moving so quickly that the practices studied may already be outdated before the results are known. Instead, he notes, it's much more important to develop systems to track quality and collect existing information, such as course participation, grades, and assessment results, in a manner that can help monitor student outcomes and practices at the course level. And better data on the impact of curriculum, instructional materials, and teaching practices would benefit all of education, not just virtual learning. But current data, Bakia says, are extremely scarce: "in most places you can't even tell if a course is online."

Paul Hill adds that ongoing data collection and research is especially important for at-risk students and in areas like credit recovery, dropout prevention, and juvenile justice: "Almost every party involved with a poor kid who is about to drop out of school doesn't want to turn that rock over." Without outside pressure, programs for these students could be the most vulnerable to quality concerns.

Secure Independent Oversight

Once providers develop quality measures and relevant data exist, one or more independent entities must be charged with deciding which providers can enter the marketplace and holding them accountable for student outcomes. An independent entity, whether it's an authorizer, district, or state, needs to ensure that competition rewards high quality, not just low cost or easy access.

Markets can offer new choices to parents and students. And unlike place-based charter schools, virtual learning allows these choices to be unbound by geographic constraints. A student in rural Alabama can now look online for better instructional models, make up credits for missed or failed classes, or even access Mandarin Chinese courses.

But policymakers cannot rely solely on parent and student choice to ensure quality. Sixteen-year-olds do not always make the best decisions, and parents have many different motives for choosing a virtual provider. Some may want an accelerated curriculum for a gifted student, others may be looking for more scheduling flexibility, and still others may just be interested in getting course credit quickly, regardless of quality or rigor.

The parallel to the charter experience is striking. "Parent accountability as the only driver of accountability hasn't always worked out for charter schools," notes Todd Ziebarth of the NAPCS, "Parents sometimes keep sending their kids to schools that aren’t academically successful."

The first system of grading charter laws rewarded states for easy access to charters but put little emphasis on quality control over new and existing schools. Since then, things have changed dramatically. A report released in 2010 by the NAPCS graded states not just on the opportunities for charter schooling to expand in the state, but also on the quality of charter school authorizing supported by the state law.

But current charter-school authorizing methods, which focus on an entire school, are not adequate for supplemental or blended virtual-learning providers. Nor is accreditation (until the NCAA halted the practice, fully accredited BYU Independent Study was known to college sports fans as the place where football player Michael Oher, profiled in the movie The Blind Side, and others went to quickly raise grades to become eligible for college athletics). And even in the case of virtual charter schools, authorizers are just now beginning to understand the unique qualities of virtual schools that change the nature of oversight, including these schools' capacity to serve tens of thousands of students across wide geographic areas.

Virtual education has the potential to operate in a more nimble and responsive market than charter schools. Without the large up-front costs associated with brick-and-mortar schools and the long lag time for determining school success, the virtual education market may not need as many limits on new entrants. But for the market to yield innovation and high performance, providers need to be rewarded for successful student outcomes, not just enrollments, and an independent agency, whether it’s a charter-like authorizer, the school district, or the state, needs to be responsible for quickly shutting down low performers. ...Read more from this article in the spring 2011 issue of Education Next.