Student Achievment in Edison Schools: Mixed Results in an Ongoing Enterprise

Preface

The Edison Project is the creation of media entrepreneur Christopher
Whittle. Whittle's original plan for the Edison Project was to establish 1,000 for-profit private schools. A series of setbacks forced Whittle and his fellow investors to settle on a scaled-back plan that would use school district funds to operate a much smaller number of public schools. Whittle now hopes to open 200 public schools that will receive the same per-pupil spending as traditional public schools, yet still yield a profit.

Edison spent an estimated $40 million developing its program before its first school opened for business, and news accounts have reported that Edison has invested more than $100 million in its schools. Edison's pitch to prospective school districts makes several attractive claims: Its program will raise student achievement for the same per-pupil expenditure as other district schools; provide a longer school day and year; and supply a home computer for the family of nearly every student in an Edison school.

The AFT has followed the Edison Project since its inception, and AFT leaders and staff leadership have engaged in an ongoing dialogue with Edison for several years. Christopher Whittle has addressed the AFT Executive Council and policy councils. And AFT affiliates in two locations have negotiated contractual waivers with Edison, smoothing the way for Edison to operate schools in those districts. In fact, leaders of the United Teachers of Dade successfully lobbied for Edison when a new school board member attempted to terminate the Edison contract after one year of operation.

Not all of AFT affiliates' dealings with Edison have been harmonious, however. The Duluth Federation of Teachers has met with Edison officials and visited Dade-Edison, but was not successful in resolving union concerns. Edison opened two charter schools in Duluth in fall 1997.

In 1995, Edison opened four elementary schools: in Boston, Massachusetts; Wichita, Kansas; Mount Clemens, Michigan; and Sherman, Texas. In 1996, four more elementary schools opened: in Dade County, Florida; Colorado Springs, Colorado; Worcester, Massachusetts; and Lansing, Michigan. And this past fall, Edison opened elementary schools in Chula Vista, California; Detroit, Michigan; Duluth, Minnesota; Flint, Michigan; Southwest Independent School District -- a small district on the outskirts of San Antonio, Texas -- and an additional school in Wichita. Edison also operates middle schools in Wichita, Sherman, Mt. Clemens, Boston, Worcester, Lansing, and Duluth. The company currently operates 25 schools.

Overview of Edison Project Sites, 1997-98

Executive Summary

The Edison Project offers a mixture of appealing features, some backed by solid evidence of educational efficacy and others not. There is also evidence of some demographic and financial advantages that are not an acknowledged part of Edison's financial package but are nonetheless significant. All told, there are certain features of the Edison Project that would predict success in raising student achievement.

In contrast, there are certain features of the program, as implemented, that might undermine Edison's success. Chief among them are departures from the recommended implementation of Success for All, reliance on novice teachers, high teacher turnover, and large class size.

1. Potential advantages of the Edison program

• In its elementary schools, Edison relies heavily on Success for All (SFA), a successful and widely admired reading program designed to raise student achievement in low-performing schools with high concentrations of disadvantaged children. SFA was developed by researchers at Johns Hopkins University and is now used in approximately 500 schools throughout the U.S. It includes daily 90-minute reading periods in small groups of like proficiency, extensive one-on-one tutoring in the early grades, family support services, and a number of specific staffing and staff development components.

• Edison expands the time students spend in school with a longer school day and school year and a full-time kindergarten that has an academic program. Though these enhancements are attractive and popular, their educational effectiveness has not been solidly established. However, some studies have found benefits in a good, full-day kindergarten program, and if more time in school means more time on task, a longer school day and school year also might be expected to improve student achievement.

• Edison charter schools are, in effect, magnet schools. The parent and student choice that such schools entail is often a proxy for highly motivated parents; and that characteristic, in turn, can be expected to show up in improved student achievement. The same is true, to a lesser extent, of the so-called partnership schools, which Edison manages under contract with school districts. In those schools, some out-of-boundary students who wish to enroll are permitted to do so, thus creating a "neighborhood magnet."

  Magnet schools also can exercise some degree of selectivity, which probably explains the consistent decline in the number of free-lunch students in many of the Edison schools for which we have data.^fn1 Since there is strong correlation between student achievement and socioeconomic status, fewer poor children at Edison schools would probably increase the likelihood of higher test scores.

• Being able to enroll out-of-boundary students also allows Edison to fill every seat by limiting enrollment and admitting students from waiting lists. The company can thus make the most economical possible use of the school staff and facilities.

• The first Edison schools have not, in fact, had to make do with the same per-pupil expenditures as other public schools. Although Edison schools probably got approximately the same local and state funding as similar local schools, most received additional funds from Edison investors for technology, consultants and staff development, and, sometimes, other expenses. Some schools, such as the Boston charter school, have benefitted from substantial private donations. In fact, Edison has a person on staff to assist schools in raising contributions from outside sources.

2. Potential shortcomings of the Edison program

• The effectiveness of Success for All has been threatened by the company's failure to carry out the program as developed by SFA researchers. Though reading tutors are central to SFA's success, Edison has yet to provide the number of tutors that SFA calls for in high-poverty schools, and it spreads reading tutors over all grade levels, from kindergarten through grade five, instead of concentrating them in kindergarten through grade two where, SFA researchers have found, they are most effective. (That may account for the lackluster performance of the Dade County school in 1996-97 in comparison with that of the Edison school in Wichita, where SFA was fully implemented.) Also, except for Dade County, beginning in 1997, Edison does not employ the full-time SFA coordinator in each school, even though the program insists this is necessary in order to ensure quality control.

• Edison relies heavily on relatively inexperienced teachers. Typically, half of the teaching force has fewer than five years of experience, in comparison with the national average of 16 years.

• Teacher turnover rates in Edison schools are high. The company admits to a 23 percent turnover rate, which is twice the national average for urban public schools. At some of the Edison schools, turnover is 25 to 40 percent in a single year, as much as triple the public-school average nationwide.

• Class size also tends to be high. With the exception of the small reading groups required by Success for All, the average is in the neighborhood of 28 students per class. In some of the Edison schools, this is no higher than the school-district average. However, the national norm is 24 students, and most experts now agree that elementary school students, particularly poor children, benefit from a class size of 15 to 17.

3. Edison's Primary Reading Studies

Edison commissioned Robert Mislevy, a distinguished researcher, to do reading achievement studies of students in grades K-3. The design of these studies was modeled on the standard evaluation used with Success for All programs. Mislevy carried out his study in Edison elementary schools in Wichita, Kansas; Mt. Clemens, Michigan; Sherman, Texas; and Colorado Springs, Colorado. (He also conducted a reading study at Edison's Boston charter elementary school. However, it had no control group and so is not included in our analysis.)

On the whole, Edison results were mediocre. Kindergarten students were the most successful in comparison with control-group students. However, this is not surprising given the fact that Edison runs full-day kindergartens with an academic program and the schools attended by control-group schools did not.

Edison first graders did much less well. Students at Wichita performed somewhat better than students in the control groups -- the effect size was small to moderate. In Mt. Clemens, the differences were generally not statistically significant. Furthermore, since no initial achievement data were collected, it is not possible to compare Edison students with control group students, so test results are inconclusive anyway. At Sherman, the Edison Program showed no effect -- that is, there was no difference in test results between Edison students and students in the control group. At Colorado Springs, results were not good but should probably be considered inconclusive because of insufficient data and problems in matching control group students with children attending the Edison school.

Edison shows no signs of continuing the Primary Reading Studies at either Sherman or Colorado Springs, so we are unlikely to find out if the students in these schools would have gone on to match the achievement levels of youngsters at Wichita and Mt. Clemens.

4. Success for All results compared with Edison's

Mislevy's evaluations compared scores for Edison elementary school students in Wichita, Mt. Clemens, Sherman, and Colorado Springs with scores for control group students. However, if you also compare Edison scores with scores on a national SFA study carried out by the researchers who originated the program, Edison's results are much less impressive. Generally, they are not as good as the results for an average, fully-implemented SFA program. It should be noted that the great majority of SFA programs are located in public schools that serve poor, minority communities and that student populations in these schools are, on average, more disadvantaged than student populations in Edison schools.

5. Other elementary and middle school results

• The results for Wichita students in grades 3 to 5 are contradictory. Over a two-year period, student achievement, as measured by MAT/7, a standardized test of reading and writing administered by the school district, has improved dramatically and puts Edison's Wichita school well ahead of most comparable schools. In stark contrast, however, the Kansas State Assessment shows that Edison students have fallen behind students in comparable schools.

• After one year of operation, the middle school in Wichita, unlike the more successful Edison elementary school in Wichita, demonstrated no gains on the MAT/7 tests or the Kansas state assessments.

• Over two years, the Texas state assessments show that third- and fourth-grade students in Sherman have suffered substantial declines in their math scores and third graders in their reading scores.

• Results in the Sherman middle school are inconclusive because we lack the data to interpret them.

• First-year data for Mt. Clemens students in grades three to five show little overall improvement on state assessments, and after two years, standardized test scores improved only a small amount.

• In the Boston charter school, scores for students in grades 3, 4 and 5 have shown improvement on MAT/7 tests, though there were irregularities in the way Edison admininstered the tests. On the state assessments, third graders scored at the 45th percentile in reading, in contrast to the 60th percentile score the same students received on the MAT/7 the following fall. (It is unheard of for students to register an achievement gain like this over the summer.)

• In Dade County, an early evaluation conducted by the school district concluded that Edison students in grades 1 to 6 are doing as well in reading as comparable students in the district, but they have fallen behind in math.

• Early results in Colorado Springs indicate that students in grades 3 to 5 did not improve in the school district's standardized test of reading, math, and language as much as either other third to fifth graders in the district or third to fifth graders in prior years when the school was not run by Edison.

6. Recommendations

As one might expect at this early stage in the history of the Edison Project, the evidence on student achievement is mixed and inconclusive. However, two things are clear. There are discrepancies between the record of Edison schools, as measured by standard methods of educational evaluation, and the company's sales presentations and promotional materials. Edison has exaggerated test score gains and emphasized favorable comparisons in order to show Edison schools in the most positive light. In fact, if public schools were to use some of Edison's evaluation methods and modes of presenting data, they would look a lot better, too.

This is unacceptable. Edison should be expected to measure achievement in its schools using the same standards that apply to other schools in the districts where the company operates. Despite its claims to being a better alternative to regular public schools and a model for public education, Edison is obviously confronting the same difficulties in improving student achievement as regular public schools.

Although accountability for the outcomes of private management of a public institution, like a school, rests with the private provider, the ultimate responsibility is with public officials whose duty it is to protect the public interest. Private management of public schools requires good public oversight, both before entering into an agreement with a private provider and during it. To that end, there are a number of things school districts that are considering hiring the Edison Project should keep in mind:

• Improving student achievement is at the heart of all the many efforts to improve U.S. education, including Edison's. So school districts should pay at least as much attention to Edison's student achievement data as they do to impressions from site visits to Edison schools.

• School districts should have their own experts -- district officials in charge of program evaluation and student assessment -- review and verify all student achievement data in all of the Edison schools. This review should be made public and should include not only Edison-administered achievement tests but also state and local assessments.

• School districts considering hiring Edison should have a timeline that is unhurried enough so that everyone involved can give careful attention to the advantages and disadvantages of Edison before making a decision.

• If a school district decides to sign a contract with Edison, officials should either hire independent professional evaluators or conduct their own annual evaluation of program implementation and student achievement using qualified school-district employees.

In her article about Edison's Boston charter school, which appeared in the March 1998 issue of Phi Delta Kappan, Peggy Farber puts the Edison Project into its perspective as a business venture and as one struggling school reform project among many. She directs her caveats to members of the media, but they are equally applicable to school districts:

It is not reasonable to expect the Edison Project to present the public -- its potential customers -- with a truly comprehensive, objective picture of its schools. Edison officials are naturally eager to draw attention to signs of success, which clearly exist. But it is essential, especially now, when the idea of a single solution to complex social problems has such a strong grip on the American imagination, that reporters give an honest and thorough accounting of what it's like inside schools that operate beyond the reach of almost all local and state agencies. The evidence from Boston Renaissance suggests that the Edison Project is struggling -- succeeding in some areas, stumbling in others -- to improve schools for students. And the same can be said for countless reform efforts across the country.

To a large extent, Edison's popularity depends on the perception that public schools are dysfunctional and helpless to change and that anyone with a fix to offer must be able to do better. Before buying into this idea, public officials and the people who report and comment on education should look at some of the reform efforts that are demonstrably -- that is, using accepted measures -- changing public schools for the better. This report will comment on one of them -- Success for All -- and to point out that, thus far, Edison students who are following the SFA program are not doing as well as SFA students in non-Edison schools where SFA is fully implemented. This is true despite the fact that Edison students are generally more advantaged and have the benefit of the rest of the Edison program.

The point is that Edison is one reform effort among many. Its more polished public relations have ensured it a national reputation, and since it is a for-profit business, it also has more money behind it. Will the Edison Project be able to live up to its promises? We'll have to wait and see. In the meantime, there are unheralded programs that are already living up to their claims, and people who are looking for ideas -- or good news about education -- should be giving them the same kind of attention that Edison gets.

¹ The proportion of students eligible for free or reduced-price lunch in Edison elementary schools has fallen from 76 to 69 percent in Wichita, from 70 to 64 percent in Sherman, Texas, from 54 to 44 percent in Colorado Springs, from 64 to 50 percent in Boston, and from 59 to 48 percent in Mt. Clemens, Michigan.

To see the full report, visit the American Federation of Teachers' website.

• 187 Privatization