Great Lakes Center Logo

OCTOBER 12, 2023

Simplistic Analysis of School Voucher Studies Results in an Incomplete and Misleading Report

An NEPC Review funded by the Great Lakes Center

Key Takeaway: Report uses flawed methods to "prove" that school choice works.

GRAND RAPIDS, MI (October 12, 2023) - The 2023 (fifth) edition of a semi-regular EdChoice report about school-choice studies is billed as an updated overview of the varied and often contested research on outcomes in voucher-like programs that provide public funding for private schools. But like earlier editions, it uses flawed methods that tally certain voucher studies finding impacts on any subgroup, even if there was no effect on most students. There is little to no accounting for the studies’ sampling, quality, generalizability, or other important factors.

Christopher Lubienski of Indiana University reviewed The 123s of School Choice: What the Research Says About Private School Choice Programs in America, 2023 Edition and found fault in its study-selection issues, a mis-weighting of studies of varied value, and a simplistic and often misleading design.

The report examines research on student achievement, access, competitive effects, and other topics, purportedly to help policymakers and parents weigh the benefits and costs of these voucher programs. The report seeks to tally all the relevant studies meeting its criteria and summarize the findings in an easy-to-understand format.

Through this "vote-counting" exercise, EdChoice claims to show that school choice "works," based on finding more positive than negative studies. Yet, as Professor Lubienski explains, vote-counting is a simplistic, flawed approach that obscures important differences in studies and can create a misleading narrative about the research evidence.

Moreover, the report does not fully disclose its strategies for including studies, some of which are questionable. While it claims to present "the research" in a fair way, the report excludes or downplays research that is not as favorable to its agenda. In the end, this report is somewhat useful as a selective bibliography of studies, but the "vote-counting" methodology at the heart of the report seriously limits its use for evaluating these policies.

Find the review, by Christopher Lubienski, at:
https://www.greatlakescenter.org

Find The 123s of School Choice: What the Research Says About Private School Choice Programs in America, 2023 Edition, published by EdChoice, at:
https://www.edchoice.org/wp-content/uploads/2023/07/123s-of-School-Choice-WEB-07-10-23.pdf

NEPC Reviews (https://nepc.colorado.edu/reviews) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: https://www.greatlakescenter.org

Facebook Icon
X Icon
LinkedIn Icon
Threads Icon

The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high-quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.

Visit the Great Lakes Center website at GreatLakesCenter.org

Contacts

Michelle Renée Valladares
(720) 505-1958
michelle.valladares@colorado.edu

Christopher Lubienski
(812) 856-8389
clubiens@iu.edu