2009 Think Twice Reviews
Think Twice is one of the nation's first efforts to serve as a watchdog to review think tank research on public education issues and policies, ensuring that published work meets the quality and standards of university scholarship. As think tank research becomes increasingly important reference sources in public policy debates, media and other critics have called for increased scrutiny to ensure validity and objectivity (click here to see related stories).
The Think Twice project is funded by the Great Lakes Center which commissions the National Education Policy Center (NEPC) to conduct expert third party reviews of think tank publications for the project. NEPC is directed by University of Colorado at Boulder professor Kevin Welner. Its publications director is Arizona State University professor Alex Molnar.
Click Here To Read Our Legal Statement
Reports & Reviews for 2009
| Report Reviewed: | Everyone Wins: How Charter Schools Benefit All New York City Public School Students |
| Publisher/Think Tank: | Manhattan Institute |
| This report looks at the competition effects of New York City's charter schools and concludes that "students benefit academically when their public school is exposed to competition from a charter." | |
| Think Twice Review Date: | November 17, 2009 |
| Reviewer: | Patrick McEwan, Wellesley College |
| McEwan's review of the Manhattan Institute's report finds that the report's findings are minimal and may be explained by factors other than charter-school competition. | |
  |
|
| Report Reviewed: | How New York City's Charter Schools Affect Achievement |
| Publisher/Think Tank: | The New York City Charter Schools Evaluation Project |
| This report estimates the effects on student achievement of attending a New York City charter school rather than a traditional public school. A key finding compares the cumulative effect of attending a New York City charter school for 9 years to the magnitude of average test score differences between students in Harlem and the wealthy New York community of Scarsdale. | |
| Think Twice Review Date: | November 12, 2009 |
| Reviewer: | Sean Reardon, Stanford University |
| Reardon's review finds that the report "has the potential to add usefully to the growing body of evidence regarding the effectiveness of charter schools." However, it also notes that the report's key findings are grounded in an unsound analysis—an inappropriate set of statistical models—and that the report's authors never provide crucial information that would allow readers to more thoroughly evaluate "its methods, results, or generalizability." He warns that "Policymakers, educators, and parents should therefore not rely on these estimates until the bias issues have been fully investigated and the analysis has undergone rigorous peer review." | |
|
|
| Report Reviewed: | The Widget Effect: Our National Failure to Acknowledge and Act on Teacher Differences |
| Publisher/Think Tank: | The New Teacher Project |
| This report describes serious problems with teacher evaluation systems based on surveys conducted in 12 school districts in four states. The report urges changes to establish more effective evaluation systems that will be able to discern the highest and lowest quality teachers. | |
| Think Twice Review Date: | August 5, 2009 |
| Reviewer: | Raymond Pecheone and Ruth Chung Wei, Stanford University |
| The reviewers find that the Widget report is well grounded and praises the report's policy recommendations as sound and consistent with prior research. However, the review also notes that the report lacks important details about its own methods and fails to consider or to draw upon any promising teacher evaluation strategies currently in use. | |
|
|
| Report Reviewed: | Multiple Choice: Charter School Performance in 16 States |
| Publisher/Think Tank: | Center for Research on Education Outcomes (CREDO), Stanford University |
| This report involved a large scale analysis of 65-70% of the nation’s charter schools and found that average charter performance is equal to, or perhaps lower than, the performance of traditional public schools. | |
| Think Twice Review Date: | June 24, 2009 |
| Reviewer: | Gary Miron and Brooks Applegate, Western Michigan University |
| Miron’s and Applegate’s review of the report finds that the solid analytic approach employed and the comprehensive data set used make the report a valuable addition to the body of charter school research. | |
|
|
| Report Reviewed: | The Fiscal Impact of Tax-Credit Scholarships in Montana The Fiscal Impact of Tax-Credit Scholarships in Georgia The Fiscal Impact of a Corporate & Individual Tax-Credit Scholarship Program on the State of Indiana |
| Publisher/Think Tank: | Friedman Foundation for Educational Choice |
| The Friedman reports which focus on Georgia, Montana and Indiana purport to demonstrate that programs awarding tax-credits to donors funding private school vouchers will reduce government expenditures and make the finance system more efficient. | |
| Think Twice Review Date: | June 22, 2009 |
| Reviewer: | Luis Huerta, Teachers College, Columbia University. |
| In his review, Huerta finds the reports' conclusions highly suspect and sharply criticizes them for presenting unsubstantiated claims and failing to adequately consider short- and long-term costs of such tax-credit programs. | |
|
|
| Report Reviewed: | The MPCP Longitudinal Educational Growth Study Second Year Report |
| Publisher/Think Tank: | School Choice Demonstration Project; Department of Education Reform, University of Arkansas |
| This report on the second year of a five-year evaluation of the Milwaukee school voucher program finds no achievement growth differences between voucher schools and the Milwaukee Public Schools over a one-year period. | |
| Think Twice Review Date: | May 28, 2009 |
| Reviewer: | Casey Cobb |
| Cobb's review finds that the report's design and methods are sound, but its usefulness is limited by several factors, including a lack of clarity about the representativeness of the included voucher schools. Correction On June 24, 2009, the authors of the report under review brought to our attention an error in the original May 28, 2009 review of their work by Casey Cobb. Professor Cobb has revised his review accordingly. As revised, the review includes an important correction. In the initial review, Professor Cobb made an incorrect assumption about the sampling procedures used in the study. As a result, he drew the incorrect conclusion that the external validity of the findings was tempered by a non-random sample of MPCP students, and hence a non-representative sample of MPCP schools. As the study authors have pointed out, the MPCP sample was in fact drawn randomly. |
|
|
|
| Report Reviewed: | Charter Schools in Eight States: Effects on Achievement, Attainment, Integration and Competition |
| Publisher/Think Tank: | RAND Corporation |
| This report on charter schools in eight states generally finds that charters have had only modest or insignificant effects on such things as student achievement, student attainment, integration and public school competition. | |
| Think Twice Review Date: | May 27, 2009 |
| Reviewer: | Derek C. Briggs |
| Briggs’s review finds some weaknesses in the data regarding some of the findings, but overall concludes that the report is of high quality and makes an important contribution to the empirical literature on charter school effectiveness. | |
|
|
| Report Reviewed: | Evaluation of the DC Opportunity Scholarship Program: Impacts After Three Years |
| Publisher/Think Tank: | School Choice Demonstration Project; Institute of Educational Sciences, National Center for Educational Evaluation and Regional Assistance |
| The report finds that after three years, students in the D.C. voucher program showed modestly higher reading scores, but no significant difference in math scores when compared with a control group of students who didn’t get vouchers. | |
| Think Twice Review Date: | May 21, 2009 |
| Reviewer: | Martin Carnoy, Stanford University |
| Carnoy’s review points out that any change in scores is modest and provides little support for vouchers as a solution to D.C.’s educational problems. He suggests that the report could have done far better in analyzing the results and presenting them in a more nuanced fashion. | |
|
|
| Report Reviewed: | The Effect of Milwaukee's Parental Choice Program on Student Achievement in Milwaukee Public Schools |
| Publisher/Think Tank: | Choice Demonstration Project; Department of Education Reform, University of Arkansas |
| The report contends that competition from Milwaukee's private school voucher program has benefited Milwaukee public schools. | |
| Think Twice Review Date: | May 18, 2009 |
| Reviewer: | Gregory Camilli, Rutgers University |
| Camilli's review raises a number of questions about the statistical methods used in the report and concludes that any positive effect of competition is very small, if it exists at all. | |
|
|
| Report Reviewed: | Weighted Student Formula Yearbook 2009 |
| Think Tank: | Reason Foundation |
| The report examines 14 city school systems and one statewide one - Hawaii- that the report presents as reflecting "best practices" in implementing what it calls Weighted Student Funding reforms. The report is based on two underlying premises: (1) school budgets should be allocated directly to schools within a district, with the amount based on each child's needs; and (2) school principals should have full discretion on how to allocate those funds. | |
| Think Twice Review Date: | May 13, 2009 |
| Reviewers: | Bruce Baker, Rutgers University |
| The Baker review of the report finds that it cherry-picks evidence, lumps many different strategies under a single reform umbrella, ignores contradictory findings, and in one-third of its examples credits the reforms for outcomes that actually preceded the reforms themselves. Baker concludes that the report is reckless and irresponsible. | |
|
|
| Report Reviewed: | The Impact of Milwaukee Charter Schools on Student Achievement |
| Think Tank: | Brookings Institution |
| This report finds that there are no math achievement gains and small gains in reading in students who attend Milwaukee charter schools when compared to traditional public schools. The report also found that there is no evidence that the presence of charter schools influences that performance in traditional public schools. | |
| Think Twice Review Date: | May 5, 2009 |
| Reviewer: | Robert Bifulco, Syracuse University |
| Though Bifulco finds that the methods used in the study have important strengths, he identifies several questions about the internal and eternal validity of the reported estimates. He indicates that this report is most informative when considered in the context of the larger body of research on charter schools. He finds the report's conclusion that charter schools "should not be expected to be the silver bullet that some reformers seek" to be reasonable. | |
|
|
| Report Reviewed: | The Fiscal Impact of the Milwaukee Parental Choice Program: 2009 Update |
| Think Tank: | School Choice Demonstration Project; Department of Education Reform University of Arkansas |
| This report examines issues concerning the funding formula used for the Milwaukee Parental Choice Program and finds that it generates a net savings to taxpayers in Wisconsin but imposes a large fiscal burden on taxpayers in Milwaukee. | |
| Think Twice Review Date: | April 27, 2009 |
| Reviewer: | Christopher Lubienski, University of Illinois |
| Belfield finds that the report offers valuable insights on the fiscal impact of voucher programs. However, he questions the benefits and notes that the report omits some important factors which are essential in order for a more complete analysis to be done. | |
|
|
| Report Reviewed: | A Win-Win Solution: The Empirical Evidence on How Vouchers Affect Public Schools |
| Think Tank: | Friedman Foundation for Educational Choice |
| This report purports to present a summary of previous research about the competitive effect of vouchers on public schools. The report concludes that such competition improves public schools. | |
| Think Twice Review Date: | April 27, 2009 |
| Reviewer: | Christopher Lubienski, University of Illinois |
| Lubienski finds that the report misrepresents the research and selectively reads the evidence in some of the studies. He points out that the majority of the studies cited were produced by voucher advocacy organizations and almost none were peer reviewed. Lubienski concludes that the report is actually “an overview that seems designed to build a pro-voucher argument rather than an evenhanded presentation of research.” | |
|
|
| Report Reviewed: | When Private Schools Take Public Dollars: What’s the Place of Accountability in School Voucher Programs? |
| Think Tank: | Thomas B. Fordham Institute |
| This report examines the opinions of experts who are school choice advocates regarding accountability issues for private schools that accept public monies (vouchers). | |
| Think Twice Review Date: | April 15, 2009 |
| Reviewer: | Ernest House, University of Colorado at Boulder |
| House praises the report and says that it offers some useful insights. He credits the Fordham researchers with making an honest attempt to tackle the difficulties of holding private schools accountable. | |
|
|
| Report Reviewed: | An Evaluation of Teachers Trained Through Different Routes to Certification |
| Think Tank: | Mathematica Policy Research, Inc. |
| This report finds that students who were assigned an alternatively certified teacher did no worse on achievement tests than students whose teacher came through the traditional teacher-education route. Moreover, the report concludes that there is no association between greater amounts of teacher training coursework and effectiveness in the classroom. | |
| Think Twice Review Date: | March 10, 2009 |
| Reviewer: | Sean P. Corocoran, New York University and Jennifer Jennings |
| In their review of this report, the authors find that the conclusions drawn by the report are not supported by the report’s actual data. They contend that few, if any, valid conclusions about teacher certification policy can be drawn from the Mathematica report and that any conclusions that are drawn tend to favor traditional certification routes. | |
|
|
| Report Reviewed: | The High Cost of High School Dropouts in Ohio |
| Think Tank: | The Buckeye Institute for Public Policy Solutions |
| This report offers up “dropout recovery” charter schools as a solution to reduce Ohio’s dropout rates. | |
| Think Twice Review Date: | March 4, 2009 |
| Reviewer: | Sherman Dorn, University of South Florida |
| In his review of this report, Dorn finds that the data the report relies upon is grossly exaggerated when compared to Ohio state data regarding the number of students graduating from charter schools. In addition, Dorn points out that the report ignores a large body of useful charter school research. He recommends that policymakers seek out well-researched scholarship on this topic and not rely upon this report. | |
|
|
These articles and/or reports are copyrighted material, the use of which has not been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of educational issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the U.S. Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml. If you wish to use copyrighted material from this site for purposes of your own that go beyond fair use, you must obtain permission from the copyright owner.
Worth A Read
A weekly selection of thought-provoking research and commentary focused on
education reform.
Stay Informed
Enter your email address to receive the latest news and information from The Great Lakes Center. We value your privacy and do not share email addresses with any other organizations.
Recommended Books
The Obama Education Blueprint: Researchers Examine the Evidence
Think Tank Research Quality: Lessons for Policy Makers, the Media, and the Public
Collateral Damage: How High-Stakes Testing Corrupts America's Schools
NeoVouchers: The Emergence of Tuition Tax Credits for Private Schooling
