October 30, 2015

Contact: 
Martin Carnoy, (650) 725-1254, carnoy@stanford.edu
Daniel J. Quinn, (517) 203-2940, dquinn@greatlakescenter.org

A new policy brief out today underscores flaws of using international tests to drive educational policy

EAST LANSING, MI (Oct. 30, 2015) – A new policy brief out today explores policy analysis and claims around the Program for International Student Achievement (PISA), as well as a second prominent international test, the Trends in International Mathematics and Science Study (TIMSS).

Stanford education professor Martin Carnoy authored the brief, International Test Score Comparisons and Educational Policy: A Review of the Critiques, which is published by the National Education Policy Center with funding from the Great Lakes Center for Education Research and Practice. Carnoy is the Vida Jacks Professor of Education at Stanford University.

In the brief, Carnoy focuses on four critiques of analyses that use average PISA scores as a comparative measure of student achievement. According to Carnoy, the ranking is misleading because:

1. Students in different countries have different levels of family – not just school – academic resources;
2. The larger gains reported on the TIMSS, which is adjusted for different levels of family academic resources, raise questions about the validity of using PISA results for international comparisons;
3. PISA test score error terms—the difference between measured achievement and actual achievement—are considerably larger than the testing agencies acknowledge, making the country rankings unstable; and
4. The Shanghai educational system is held up as a model for the rest of the world on the basis of data on a subset of students that is not representative of the Shanghai student population as a whole.

The brief addresses several important criticisms of international testing. Notwithstanding, Carnoy concludes that these tests will not go away, nor will they stop being wrongly applied to shape educational policy.

Importantly, Carnoy resolves, "there are changes that could be made to reduce misuse." The brief includes five policy recommendations, which include reporting test results by family academic resource subgroups of students with different levels of resources.

Find International Test Score Comparisons and Educational Policy: A Review of the Critiqueson the web at: http://www.greatlakescenter.org

You can also find the brief on the NEPC webpage: http://nepc.colorado.edu

A separate companion report, also released today from the Economic Policy Institute (EPI) www.epi.org, provides detailed analyses explaining how and why comparisons using data at the level of U.S. states are more useful than comparing the U.S. with other countries. 

- ### -

The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform.

Visit the Great Lakes Center website at http://www.greatlakescenter.org/