< Back The School Staffing Surge: Decades of Employment Growth in America’s Public Schools, Part II Summary The report disaggregates trends in K-12 hiring for individual states, presenting ratios comparing the number of administrators and other non-teaching staff to the number of teachers or students. Publication Date March 25, 2013 Authors Joydeep Roy, Visiting Professor Teachers College – Columbia University & Senior Economist for the New York City Independent Budget Office Research Areas General / Needs Review Publication Type Think Twice Download the Policy Brief Press Release Second 'School Staffing Surge' Report Suffers From Same Flaws As Original EAST LANSING, Mich. (Mar. 26, 2013) – In a follow-up to an earlier report, the Friedman Foundation recently released a second report, describing a surge in school employment unaccompanied by progress in student achievement. Again, however, the report suffers from faulty premises and inaccurate data. Joydeep Roy reviewed the report, The School Staffing Surge, Part II , for the Think Twice think tank review project. Dr. Roy, a visiting professor at Teachers College – Columbia University and a senior economist for the New York City Independent Budget Office, prepared the review for the National Education Policy Center with funding from the Great Lakes Center for Education Research and Practice. Like the original report, Benjamin Scafidi wrote The School Staffing Surge, Part II for the Freidman Foundation for Educational Choice. Surge I reported that between 1992 and 2009, the number of employees grew at more than twice the rate that the population of students increased. It then claimed that despite the staffing surge and related spending increases, there had been no progress in student achievement or drop-out reductions. Surge II disaggregates trends in K-12 hiring for individual states, presenting ratios comparing the number of administrators and other non-teaching staff to the number of teachers or students. Both reports speculate about "savings" that might have been achieved if hiring had been held back. But neither report makes any serious attempt to link employment numbers to schools' needs or outcomes. Moreover, both reports "[fail] to acknowledge the fact that achievement scores and drop-out rates have steadily improved," Roy writes in his review. He adds, "Neither the old report or this new one ... explores the causes and consequences of faster employment growth, thus severely limiting the report's potential contribution." With no attempt to benchmark hiring against each state's needs and circumstances, the new report is once again unable to make any reasoned judgment about whether the added hiring was useful or wasteful. The result, Roy concludes, is that the new report, "much like the original one, is devoid of any serious policy implications." Find the review on the Great Lakes Center website: http://www.greatlakescenter.org Find The School Staffing Surge: Decades of Employment Growth in America's Public Schools, Part II by Benjamin Scafidi on the web: http://tinyurl.com/cj4xv9c Think Twice, a project of the National Education Policy Center, provides the public, policy makers and the press with timely, academically sound reviews of selected publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice. This review is also found on the NEPC website: http://nepc.colorado.edu/thinktank/review-school-staffing-surge-2 --###-- The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Find us on Facebook at: https://www.facebook.com/GreatLakesCenter. Contact: Joydeep Roy, (202) 821-2572, jr3137@columbia.edu Dan Quinn, (517) 203-2940, dquinn@greatlakescenter.org Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Due Diligence and the Evaluation of Teachers: A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by The Los Angeles Times Summary This report, by Richard Buddin, is a value-added analysis of elementary teachers in the LAUSD. Buddin asserts that teacher quality can validly be measured using student performance on the reading and math portions of the California Standardized Test. Buddin's analysis was the focus of an article in the LA Times which included a database of LAUSD teachers by name and their individual rankings which were based on Buddin's model. The reviewers found that Buddin's analysis is a deeply flawed and an inaccurate measure of teacher effectiveness. Publication Date February 7, 2011 Authors Derek Briggs and Ben Dominque, University of Colorado at Boulder Research Areas Assessment & Accountability; Teacher Workforce Publication Type Policy Brief Download the Policy Brief Press Release L.A. Times teacher rankings deeply flawed, new report finds February 8, 2011 "How Effective Are Los Angeles Elementary Teachers and Schools?" report is an inaccurate measure of teacher quality EAST LANSING, Mich. (Feb. 8, 2011) –— A widely reported analysis done of teachers in the Los Angeles Unified School District is a deeply flawed measure of teacher effectiveness, according to a report released today. On Aug. 14, 2010, the Los Angeles Times published a controversial article that highlighted the results of a study conducted by Richard Buddin, senior economist at the RAND Corporation. Buddin's analysis erroneously asserted that teacher quality can be validly measured using student performance on the math and reading portions of the California Standardized Test. In conjunction with the article, the L.A. Times published a database of LAUSD teachers by name and their individual effectiveness ratings, which were based on Buddin's flawed analysis. The review of Buddin's analysis was performed by Derek C. Briggs, chair of the Research and Evaluation Methodology Program at the University of Colorado at Boulder, and Colorado-Boulder doctoral student Ben Domingue. Key among their findings was that Buddin's analysis failed to consider several critical factors that impact student achievement, such as long-term achievement patterns and influence from their peers. Buddin's model led to overly broad and simplistic generalizations about teacher effectiveness, often resulting in negative consequences for many potentially effective teachers and their students, Briggs and Domingue found. Buddin's approach, known as "value-added modeling," can lead to a number of false positives and negatives – many average teachers may have been falsely categorized as either effective or ineffective. Briggs and Domingue developed a stronger alternative model that included students' long-term achievement patterns and peer influence. The researchers then compared the results of their model and Buddin's, and found that only 60.8 percent of teachers would retain the same effectiveness rating in both models for math outcomes, while only 46.4 percent of teachers would retain the same effectiveness rating for reading outcomes. Notably, Briggs and Domingue found that Buddin's model could not be independently replicated, thus casting doubts about the scientific validity of the study altogether. According to a Jan. 10 New York Times article, the New York City Board of Education is planning to follow Los Angeles' lead and publish ratings of 12,000 New York City teachers, based on an approach similar in nature to Buddin's. Briggs and Domingue's report, "Due Diligence and the Evaluation of Teachers: A Review of the Value-Added Analysis Underlying the Effectiveness Rankings of Los Angeles Unified School District Teachers by the Los Angeles Times," was produced by the National Education Policy Center (NEPC), housed at the University of Colorado at Boulder School of Education, with funding from the Great Lakes Center for Education Research and Practice. Find Briggs's and Domingue's complete report and a link to Buddin's analysis at: http://www.greatlakescenter.org . On Monday, Feb. 7, 2011, the LA Times, published a story about Briggs's and Domingue's report. That story included false statements and was generally misleading. Accordingly, along with the release of this report, a "Fact Sheet" about the Times' article has been published: http://nepc.colorado.edu/files/FactSheet_0.pdf Follow us on Twitter at http://twitter.com/greatlakescent . The brief is also available on the National Education Policy Center website at: http://nepc.colorado.edu/publication/due-diligence --###-- The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org Contact: Derek C. Briggs: (303) 492-6320, Derek.Briggs@colorado.edu Teri Battaglieri: (517) 203-2940, greatlakescenter@greatlakescenter.org Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Despite Serious Problems and a Lack of Research Support, Virtual Schools Continue to Spread as Policymakers Remain AWOL Summary Click the button below and signup to receive the latest news and information from The Great Lakes Center. We value your privacy and do not share email addresses with any other organizations. Publication Date May 28, 2019 Authors Alex Molnar, Series Editor University of Colorado, Boulder Research Areas Governance & Public Education; Technology & AI Publication Type Policy Brief Download the Policy Brief Press Release EAST LANSING, Mich. (May 28, 2019) — Virtual Schools in the U.S. 2019 , a three-part research brief released today by the National Education Policy Center, examines the claim that online curriculum can be tailored to individual students more effectively than curriculum in traditional classrooms. Proponents contend that this potential for individualization allows virtual schools to promote greater student achievement than can be realized in traditional brick-and-mortar schools. NEPC researchers found, however, that the research evidence does not support this claim. Yet the lack of research support has done little to dampen policymakers' enthusiasm, perhaps because virtual schools are marketed as promising lower operating costs—primarily via cutbacks in instructional personnel and facilities. Section I of the brief, Full-Time Virtual and Blended Schools: Enrollment, Student Characteristics, and Performance , provides straightforward analyses of the characteristics and performance of full-time, publicly funded K-12 virtual schools. The data reveal that full-time virtual and blended learning schools continue to perform poorly. Section II, What Virtual and Blended Education Research Reveals , points to a serious shortfall in the scholarly research. It reviews the relevant available studies related to virtual school practices and finds that much of this is atheoretical, methodologically questionable, contextually limited, and overgeneralized. As a result, the available research is of little value in guiding policy. Section III, Key Policy Issues in Virtual Schools: Finance and Governance, Instructional Quality, and Teacher Quality , provides an overview of recent state legislative efforts to craft policy regarding virtual schools. As in past years, bills to increase oversight of virtual schools continue to be introduced. Some legislative actions have been prompted by state audits and legal challenges, as exemplified by recent virtual school controversies in California and Ohio. As such, the bills have been aimed at addressing accountability and governance structures, as well as curbing the operation of for-profit virtual schools. However, there is little evidence that legislative actions are being informed by available research on the performance of virtual schools. The authors recommend that policymakers hit the pause button on further virtual school expansion until we understand how to address the poor performance of these schools. Find Virtual Schools in the U.S. 2019 , edited by Alex Molnar, at: http://www.greatlakescenter.org This research brief was made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu About The Great Lakes Center The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Find us on Facebook at: https://www.facebook.com/GreatLakesCenter. Friend on Facebook Follow on Twitter The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center website at http://www.greatlakescenter.org/ Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Review of Remote Learning Is Here to Stay Summary The RAND Corporation recently released a report based on a national survey of school district superintendents and charter management organization (CMO) directors (or their designees) about their experiences navigating the COVID-19 pandemic. Publication Date February 15, 2021 Authors Heather L. Schwartz, David Grant, Melissa Diliberti, Gerald P. Hunter, and Claude Messan Setodji; David Garcia, Arizona State University Research Areas Technology & AI Publication Type Think Twice Download the Policy Brief Press Release Misleading Report Fails to Provide Whole Picture of Survey Results An NEPC Review funded by the Great Lakes Center Key Takeaway: Policymakers looking for guidance about pandemic-induced challenges will have to look beyond the report's title and recommendations and read deeper to get a complete picture. EAST LANSING, MI (February 16, 2021) -The RAND Corporation recently released a report based on a national survey of school district superintendents and charter management organization (CMO) directors (or their designees) about their experiences navigating the COVID-19 pandemic. David Garcia of Arizona State University reviewed Remote Learning Is Here to Stay and found some issues with one of its two recommendations. Professor Garcia also found the report to be curiously titled, as the headline is unwarranted by the survey responses. In fact, the respondents expressed much higher concerns about three other areas: (1) "addressing students' Socio-Emotional Learning and mental health needs" (the area with the greatest need for additional resources), (2) "addressing disparities in student opportunities to learn that result from differences in supplemental supports provided by families" (the most anticipated challenge), and (3) inadequate funding (the top staffing challenge). Relative to these three concerns, remote learning is a minor consideration. While the report's first recommendation, for more funding to address inequities and socio-emotional learning, follows from the respondents' stated needs, the other recommendation, for more funding to support remote learning, does not appear to align with needs expressed by district leaders. Professor Garcia also points out that the report combines two different types of local education agencies (school districts and CMOs), and it does so in a way that prevents readers from determining whether a given result is driven by one group or the other. Thus, while the report suggests that its most important finding is that "about two in ten districts have already adopted, plan to adopt, or are considering adopting virtual schools as part of their district portfolio after the end of the COVID-19 pandemic," it is unclear how much of this result is driven by CMOs rather than school districts. For these reasons, readers are encouraged to go beyond the title and read deeper to get a complete picture of the challenges, needs, and future of education from district leaders' perspectives. Find the review, by David Garcia, at: https://www.greatlakescenter.org Find Remote Learning Is Here to Stay: Results from the First American School District Panel Survey, written by Heather L. Schwartz, David Grant, Melissa Diliberti, Gerald P. Hunter, and Claude Messan Setodji and published by the RAND Corporation, at: https://www.rand.org/pubs/research_reports/RRA956-1.html NEPC Reviews ( http://thinktankreview.org ) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: https://www.greatlakescenter.org The National Education Policy Center (NEPC), a university research center housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu/ About The Great Lakes Center The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: https://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Find us on Facebook at: https://www.facebook.com/GreatLakesCenter. Friend on Facebook Follow on Twitter The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center website at https://www.greatlakescenter.org/ Contact: Michelle Renée Valladares : (720) 505-1958, michelle.valladares@colorado.edu David R. Garcia: (480) 727-7423, david.garcia@asu.edu Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Transfer Incentives for High-Performing Teachers: Final Results from a Multisite Randomized Experiment Summary The Teacher Transfer Initiative (TTI) actively encouraged 1,500 "high-performing" elementary teachers in 10 urban school districts to transfer to low-performing schools in return for a total stipend of $20,000 distributed over two years. The teachers were all identified as having students who were in the top 20 percent in their districts, based on a value-added measure of standardized test scores. The assumption was that better teachers would get better results on standardized tests wherever they teach. Publication Date January 27, 2014 Authors Steven Glazerman, Ali Protik, Bing-ru Teh, Julie Bruch, and Jeffery Max; Donald Gratz, Curry College Research Areas Assessment & Accountability; Teacher Workforce Publication Type Think Twice Download the Policy Brief Press Release Review of Transfer Incentives for High-Performing Teachers Policymakers should look elsewhere for methods to improve student achievement EAST LANSING, Mich. (Jan. 28, 2014) – Access to quality teaching has been a concern for low-income schools, which have a larger percentage of lower-performing or less qualified teachers than schools in middle-class areas. Many believe that incentives are needed to encourage high-performing teachers to work in hard-to-staff schools. An academic review out today considers the evidence of a teacher transfer incentive, which promised to pay high-performing teachers more to work in lower-performing schools. Mathematica Policy Research (MPR) produced the Teacher Transfer Initiative (TTI) , funded by the U.S. Department of Education. The study tested whether high-performing teachers could be enticed by a $20,000 stipend over two years to transfer to and remain at a lower performing school, and measured the impact of the transfer on students' standardized test scores. The assumption was that better teachers would get better results on standardized tests wherever they teach. The results of this extensive study are not encouraging, writes reviewer Donald Gratz, who found several flaws that limit the report's usefulness for policy consideration. Gratz, Director of Graduate Programs in Education and Chair of the Education Department at Curry College, produced the review for the Think Twice think tank review project of the National Education Policy Center (NEPC), with funding from the Great Lakes Center for Education Research and Practice. In his review, Gratz identifies several flaws. Among these: (1) the study does not consider the potential impact of other relevant school factors that affect teacher success, such as school leadership; and (2) while it considers the overall cost of the program, it only compares that cost to class size reduction, a generally expensive program. Results of the study indicate that of the 1,500 teachers actively encouraged to transfer, only 5% actually did. In terms of retention, the review finds that 90% of the transfer teachers stayed the full two years required to receive their full stipend, however, only 60% planned to stay for a third year – the same rate as teachers not receiving the incentive. The review notes that elementary school students benefited modestly from these teachers, but middle-school students appear not to have benefited. The study is further limited because it uses only test scores as a measure of outcomes. The most significant problem, Gratz says, is that this two-year study was too short to draw conclusions either about teacher retention or student achievement, limiting its impact and usefulness. Finally, the review raises ethical concerns about a policy that moves good teachers from some students to others, a process that creates winners and losers. Find Donald Gratz's review on the Great Lakes Center website: http://www.greatlakescenter.org Find Transfer Incentives for High-Performing Teachers: Final Results from a Multisite Randomized Experiment , written by Steven Glazerman, Ali Protik, Bing-ru Teh, Julie Bruch, and Jeffrey Max, on the web at: http://www.mathematica-mpr.com/education/tti.asp Think Twice, a project of the National Education Policy Center, provides the public, policymakers and the press with timely, academically sound reviews of selected publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice. The review can also be found on the NEPC website: http://nepc.colorado.edu Don Gratz is currently Director of Graduate Programs in Education and Chair of the Education Department at Curry College. He has more than 35 years experience in both K-12 and college education. The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Find us on Facebook at: https://www.facebook.com/GreatLakesCenter. Contact: Donald Gratz, (617) 333-2130, dgratz@curry.edu Dan Quinn, (517) 203-2940, dquinn@greatlakescenter.org Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back 2018 Teacher Prep Review, National Council on Teacher Quality Summary The National Council on Teacher Quality (NCTQ) released its “2018 Teacher Prep Review,” examining whether U.S. teacher preparation programs are aligned with NCTQ’s standards. This alignment, the report insists, will produce teachers “ready to achieve individual successes, but also [ready] to start a broader movement toward increased student learning and proficiency.” Publication Date June 11, 2018 Authors Robert Rickenbrode, Graham Drake, Laura Pomerance, and Kate Walsh; Marilyn Cochran-Smith of Boston College, Elizabeth Stringer Keefe of Lesley University, Wen-Chia Chang of Boston College, and Molly Cummings Carney of Boston College Research Areas Early Childhood; Teacher Workforce Publication Type Think Twice Download the Policy Brief Press Release Review: 2018 Teacher Prep Review, National Council on Teacher Quality EAST LANSING, Mich. (June 12, 2018) — The National Council on Teacher Quality (NCTQ) recently released its 2018 Teacher Prep Review . The report examines whether U.S. teacher preparation programs are aligned with NCTQ's standards. This alignment, the report insists, will produce teachers "not only ready to achieve individual successes, but also [ready] to start a broader movement toward increased student learning and proficiency." Aimed at prospective "consumers" of teacher preparation programs, the report reviews 714 post-baccalaureate teacher preparation programs. These are labeled: "graduate" or "traditional" (n=567 programs); "alternative-route" or "internship" (n=129 programs); and, "residency" (n=18 programs). The report rated teacher preparation programs using internally developed input-based standards that were applied to syllabi and course documents. Marilyn Cochran-Smith of Boston College, Elizabeth Stringer Keefe of Lesley University, Wen-Chia Chang of Boston College, and Molly Cummings Carney of Boston College assessed the report for Think Twice Review, a project of the National Education Policy Center (NEPC) , funded by the Great Lakes Center for Education Research and Practice . Their key finding is that the NCTQ report does not distinguish differences between university programs and alternative routes by failing to account for broad shifts in the field of teacher education that are nuanced, hybridized, and dynamic. In addition, the reviewers found the report to have multiple logical, conceptual, and methodological flaws. The report ignores accumulating evidence that there is little relationship between the NCTQ's ratings of a program and its graduates' later classroom performance. Its methodology, which employs a highly questionable documents-only evaluation system to judge and rate preparation programs, is a maze of inconsistencies, ambiguities, and contradictions. The report perpetuates a simplistic dichotomy between university programs and alternative routes, ignoring that researchers have suggested for years there is as much or more variation within each of these categories as there is between them. In their conclusion , the authors contend that the latest NCTQ report on teacher prep programs provides another example of misleading, confusing analysis which "Ultimately . . . offers little guidance for consumers, policymakers, or practitioners." Find the Think Twice Review on the web: http://www.greatlakescenter.org Find the 2018 Teacher Prep Review, National Council on Teacher Quality here: https://www.nctq.org/dmsView/2018_Teacher_Prep_Review_733174 Think Twice, a project of the National Education Policy Center (NEPC), provides the public, policymakers and the press with timely, academically sound reviews of selected publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice. You can also find the review on the NEPC website: http://nepc.colorado.edu Friend on Facebook Follow on Twitter The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center website at http://www.greatlakescenter.org/ Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Review of Comparing Ed Reforms: Assessing the Experimental Research on Nine K-12 Education Reforms Summary A recent report from EdChoice, working with Hanover Research, identifies and reviews studies that use Randomized Control Trials (RCTs) to determine student achievement or educational attainment outcomes of nine broad "education reform" areas. Publication Date May 25, 2020 Authors Paul DiPerna; Bruce D. Baker, Rutgers University Research Areas Charter Schools; Equity & Civil Rights; Assessment & Accountability; Early Childhood; Governance & Public Education; School Choice & Vouchers Publication Type Think Twice Download the Policy Brief Press Release Report Makes a Small Contribution to Understanding Education Reform Studies An NEPC Review funded by the Great Lakes Center Key Takeaway: Readers must resist misinterpretations of the report's table of results. EAST LANSING, MI (May 26, 2020) - A recent report from EdChoice, working with Hanover Research, identifies and reviews studies that use Randomized Control Trials (RCTs) to determine student achievement or educational attainment outcomes of nine broad "education reform" areas. Bruce Baker of Rutgers University reviewed Comparing Ed Reforms: Assessing the Experimental Research on Nine K-12 Education Reforms . He concludes that the report accomplishes what it set out to do, but he points to several issues that minimize its usefulness in guiding policy, practice, or research. The report simply presents counts of studies with positive, negative, and neutral findings from RCT studies. RCTs are presented in the report as "gold standard" studies for determining effects of specific treatments on measured outcomes, and the approach has clear strengths. But Professor Baker explains that the approach also has notable limitations. In fact, the actual randomization in education studies generally applies to only part of natural experiment; the randomization operates in a largely non-random context. Similarly, the studies are not fully controlled. Each of the report's specific education-reform-area counts also has notable limitations. RCT studies of charter schools tend to be limited to specific contexts, models, programs and services. Private school voucher policies are similarly varied and difficult to classify as a single "treatment." Moreover, as the report notes, studies categorized under the reform "open enrollment" are actually magnet school programs. In fact, the report found no RCT studies that could be squeezed into three of the reform categories. All six of the reform areas that were in fact tallied-from smaller class sizes and schools to pre-k programs to school choice-score well, with positives outweighing negatives by large margins. But that truly tells us very little, Professor Baker explains, and the study authors are in fact cautious in explaining their modest goals-merely counting the quantity and superficial results of research done in those six specific areas. The main concern with this report, therefore, is that the casual reader will take the table presenting the tallies out of context and use it to argue that charter schools and vouchers for private schools are most important or worthwhile because they have been studied most and thus have the highest counts of positive effects. If, however, the report is not misused in such a way, it offers a limited contribution for readers wanting to get an initial feel for the RCT research in these areas. Find the review, by Bruce D. Baker, at: https://www.greatlakescenter.org/ Find Comparing Ed Reforms: Assessing the Experimental Research on Nine K-12 Education Reforms , written by Paul DiPerna, with reviews conducted by Hanover Research and published by EdChoice, at: https://www.edchoice.org/wp-content/uploads/2020/04/comparing-ed-reforms.pdf NEPC Reviews (http://thinktankreview.org ) provide the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC Reviews are made possible in part by support provided by the Great Lakes Center for Education Research and Practice: https://www.greatlakescenter.org The National Education Policy Center (NEPC), a university research center housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu/ About The Great Lakes Center research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Contact: William J. Mathis : (802) 383-0058, wmathis@sover.net Bruce D. Baker: (303) 492-8370, bruce.baker@gse.rutgers.edu Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Research-Based Options for Education Policymaking—Teacher Evaluation Summary Dr. William Mathis, managing director of the National Education Policy Center, summarizes research findings on the effects of teacher evaluation systems, including unintended as well as intended consequences. This two-page brief is part of Research-Based Options for Education Policymaking, a multipart brief that takes up a number of important policy issues and identifies policies supported by research. Publication Date September 19, 2012 Authors William Mathis, University of Colorado-Boulder Research Areas Teacher Workforce Publication Type Policy Brief Download the Policy Brief Press Release New Brief Offers Suggestions for Teacher Evaluation Design EAST LANSING, Mich. (Sept. 20, 2012) –The first in a new series of two-page briefs summarizing important education policies offers suggestions for policymakers designing teacher evaluation systems. The paper, written by Dr. William Mathis, managing director of the National Education Policy Center, summarizes research findings on the effects of teacher evaluation systems, including unintended as well as intended consequences. At a time when teacher evaluation controversies in New York City, Los Angeles, Chicago and other school districts have erupted—particularly over the issue of evaluations based in part on the growth of students' test scores—understanding the evidence about these issues has taken on new urgency. This paper suggests that lawmakers should be wary of approaches based in large part on test scores, because of three problems: The measurement error is large—which results in many teachers being incorrectly labeled as effective or ineffective; Given that only certain grade levels and subject areas are tested, relevant test scores are not available for most teachers; and The incentives created by the high-stakes use of test scores drive undesirable teaching practices such as curriculum narrowing and teaching to the test. Instead, Mathis advocates systems like peer assistance and review (PAR) that de-emphasize test scores. Such systems are more labor intensive but that have "far greater potential to enrich instruction and improve education." He also advocates balancing summative, high-stakes assessment systems "with formative approaches that identify strengths and weaknesses of teachers and directly focus on developing and improving their teaching." This two-page brief is part of Research-Based Options for Education Policymaking , a multipart brief that takes up a number of important policy issues and identifies policies supported by research. Each section focuses on a different issue, and its recommendations to policymakers are based on the latest scholarship. Find William Mathis's brief on the GLC website at: http://www.greatlakescenter.org This report is published by the National Education Policy Center, housed at the University of Colorado-Boulder, and is made possible in part by funding from the Great Lakes Center for Education Research and Practice. This brief is also found on the NEPC website at: http://nepc.colorado.edu/publication/options-teacher-evaluations The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Contact: William J. Mathis, (802) 383-0058, wmathis@sover.net Dan Quinn, (517) 203-2940, dquinn@greatlakescenter.org Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back An Evaluation of Teachers Trained Through Different Routes to Certification Summary This report finds that students who were assigned an alternatively certified teacher did no worse on achievement tests than students whose teacher came through the traditional teacher-education route. Moreover, the report concludes that there is no association between greater amounts of teacher training coursework and effectiveness in the classroom. Publication Date March 9, 2009 Authors Sean P. Corocoran, New York University and Jennifer Jennings Research Areas Teacher Workforce Publication Type Think Twice Download the Policy Brief Press Release They Note Methodological Limitations and Caution against Relying on the Report to Inform Teacher Certification Policy EAST LANSING, Mi., (March 10, 2009) – Despite the headlines announcing that a recent Mathematica Policy Research evaluation found alternative teacher certification to be as effective as traditional teacher preparation programs, a closer look by several education researchers shows that such conclusions are not supported by the study’s actual data. Commissioned by the federal Institute of Education Sciences, the Mathematica study reports that teachers from most alternative certification programs have similar impact on student achievement as teachers from traditional certification routes. But the report’s conclusions are based on a selective interpretation of results from a sample of schools and teachers that is dissimilar from the general population of schools in the United States, two new critiques have found. One review of the Mathematica report was prepared by Sean P. Corcoran, assistant professor of educational economics at New York University, and Jennifer L. Jennings, who will join the NYU faculty in the fall. They found significant methodological limitations that call into question the sweeping findings announced by Mathematica. They also found the Mathematica data themselves often suggest the opposite finding: that the alternatively certified teachers who were studied generally produced poorer outcomes than do the traditionally certified teachers in the sample. The full review is available at http://www.greatlakescenter.org as part of the Think Twice think tank review project funded by the Great Lakes Center for Education Research and Practice. Corcoran and Jennings point out that policymakers and other readers of the Mathematica report tend to look for simple answers to simple questions such as “Is alternative teacher certification a bad thing or a good thing, and if alternative certification does no harm, is traditional teacher certification even necessary?” “Notwithstanding suggestions to the contrary in the report’s press release or executive summary, this report is unable to provide a satisfactory or general answer to either of those questions,” they conclude. Specifically, the review found the Mathematica study: Did not fully report and acknowledge in its conclusions the many analyses from the study finding that traditionally trained teachers outperformed alternative route teachers in both math and reading. Has a research design that favors finding few significant differences between groups, most notably its small sample size, sampling methods, and failure to distinguish the “treatments” that alternative certification and traditional certification teachers provided (meaning that members of the two compared groups had substantially overlapping preparation experiences). Is relevant only to a very limited population of teachers in schools that hire many alternatively certified teachers, and is not generalizable to most states, districts, and schools that do not allow such programs and are more selective in their hiring. The review found that the study sample includes teachers with an average of three years experience and is heavily weighted towards grades K-2, which comprise 71 percent of the study’s teachers. The average school in the study is a high-poverty, high-minority urban school, and the traditionally certified teachers who are the comparison group in the study are only those employed by these disadvantaged schools. As a result, alternatively certified teachers are compared only with teachers that prior research has demonstrated are, on average, less qualified than other traditionally certified teachers. “Unfortunately, none of the results found in this study can be used to meaningfully inform the policy debate over alternative certification because they cannot be generalized to the larger population of schools and teachers,” Corcoran said. The review also took issue with Mathematica’s selected emphasis of results, highlighting those results suggesting no difference between alternatively and conventionally certified teachers while minimizing results showing alternatively certified teachers doing significantly less well than other teachers in many categories. For example, alternatively certified teachers still taking coursework were significantly less effective in teaching reading and mathematics than their traditionally certified counterparts; and alternatively certified teachers from California were significantly less effective in teaching mathematics than traditionally prepared teachers. “We want to be careful here not to overstate the findings—because in reality the findings are minimal,” wrote Corcoran and Jennings. “This is for good reason: if one set out to design a study that would find no statistically significant differences between the achievement of students taught by traditionally and alternatively certified teachers, this is precisely the study one would have designed.” Aaron Pallas, professor of education and sociology at Teachers College, Columbia University, concurred with the findings, noting that “The design of this study successfully precludes it from addressing the most salient policy questions about alternative teacher certification – but we get a pretty clean estimate of the relative effectiveness of pairs of traditional-route and alternate-route teachers that are not representative of any population of teacher education programs, teachers, or schools.” These concerns are also reinforced by a second review of the Mathematica research, conducted by Linda Darling-Hammond, the Charles Ducommun Professor of Education at Stanford University, as part of a policy brief on teacher preparation programs generally. In her analysis, Darling-Hammond pointed to a number of similar methodological flaws in the Mathematica study. She also noted that, when students’ fall-to-spring scores are examined, Mathematica’s data show that alternatively certified teachers from “low-coursework” programs – most of them from Texas schools – actually lowered student achievement noticeably – hardly a standard to which other schools should aspire. Students of “high-coursework” alternatively certified teachers did only slightly better. In all cases, the students of traditionally certified teachers gained more or declined less than the alternatively certified teachers, but none of these already low-achieving students did well enough to begin to close the achievement gap. Characterizing the outcomes of these programs as a “race to the bottom,” Darling-Hammond also reviewed the results of studies showing substantial gains for students of teachers with stronger qualifications and much higher-quality training. “If we truly want our children to engage in a race to the top, we must create substantially higher levels of teacher effectiveness, particularly in those classrooms that have been left the furthest behind,” Darling-Hammond said. “The poor overall outcomes reported by Mathematica for these overwhelmingly disadvantaged schools show that these children are being abandoned by a system that has not invested in the quality of teachers needed to substantially improve their learning. This race to the bottom cannot be halted through alternative certification programs that simply seek to ‘do no more harm’ – or at least not much more harm – than other relatively poor training. Our students deserve far better than that. We need to invest in replicating the high-quality programs that achieve better results.” Darling-Hammond’s review, “Educational Opportunity and Alternative Certification: New Evidence and New Questions,” can be found on the Stanford Center for Opportunity Policy in Education (SCOPE) website, http://edpolicy.stanford.edu/ . About The Think Twice Project The Think Twice project provides the public, policy makers and the press with timely, academically sound reviews of selected think tank publications. It is a collaboration of the Education Policy Studies Laboratory at Arizona State University and the Education and the Public Interest Center at the University of Colorado at Boulder and is funded by the Great Lakes Center for Education Research and Practice. ### The mission of the Great Lakes Center is to improve public education for all students in the Great Lakes region through the support and dissemination of high quality, academically sound research on education policy and practices. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org Contact: Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back The Science of Reading Movement: The Never-Ending Debate and the Need for a Different Approach to Reading Instruction Summary How students learn to read and how reading is best taught are often the focus of media, public, and political criticism. In a new NEPC policy brief, The Science of Reading Movement: The Never-Ending Debate and the Need for a Different Approach to Reading Instruction, Paul Thomas of Furman University explores the controversial history of the reading reform movement. Publication Date September 12, 2022 Authors Paul Thomas, Furman University Research Areas Curriculum & Instruction Publication Type Policy Brief Download the Policy Brief Press Release Closing the Gap Between Research and Practice in the Science of Reading Key Takeaway : Some research claims of the "science of reading" movement are overly simplistic, so policymakers should seek different approaches to legislating reading. EAST LANSING, MI (September 13, 2022) - How students learn to read and how reading is best taught are often the focus of media, public, and political criticism. In a new NEPC policy brief, The Science of Reading Movement: The Never-Ending Debate and the Need for a Different Approach to Reading Instruction , Paul Thomas of Furman University explores the controversial history of the reading reform movement. Throughout the decades, a striking amount of attention has sporadically been focused on how teachers teach reading-typically with a specific concern for phonics instruction. This attention has then spread to standardized test scores (including international comparisons) and a changing list of hypothetical causes for disappointing test scores (including progressivism, whole language, and balanced literacy). Disappointing reading achievement has been sometimes attributed to how reading is taught, sometimes to social influences on students (such as technology and media), and sometimes to both. Widespread and ongoing criticism over the last 80 years has targeted a wide array of culprits: State and federal reading policy; The quality of teacher education and teacher professional development; Theories of learning to read and reading instruction; The role of phonics and other reading skills in teaching reading; and The persistent gaps among classroom practices, reading policy, and the nature or application of science and research. These discussions have not been evidence-free. In fact, scholars and literacy educators have over this time conducted extensive research into these and other issues. But the research has only limited impact on policy and practice. Specifically, in contrast to much of the public debate and policymaking, these researchers have found reading instruction and learning to be complex, complicating the design of effective policy and classroom practice. Overall, this robust research base supports policies and approaches that acknowledge a range of individual student needs and that argue against "one-size-fits-all" prescriptions. Among literacy educators and scholars, then, important reading debates continue but do so without any identified silver-bullet solutions. The current public debate is different. Since 2018, the phrase "science of reading" has been popularized as loosely defined shorthand for the broad and complex research base characterizing how children learn to read and how best to teach reading. Simplifying the issue for the public and for political readers, and failing to acknowledge the full complement of research findings, prominent members of the education media have used the term when framing the contemporary debate-often as pro-phonics versus no phonics. Various types of vendors have also found the shorthand term "science of reading" highly useful in branding and marketing specific phonics-oriented reading and literacy programs. As the "science of reading" movement has grown, scholars have cautioned that advocates and commercial vendors often exaggerate and oversimplify both the problems and solutions around reading achievement and instruction. Yet these advocates have been extremely effective in lobbying for revised and new phonics-heavy reading legislation across most states in the U.S., producing rigid and ultimately harmful policy and practices. Still, in pursuing reform to address identified challenges, the movement does provide an opportunity for policymakers to investigate different approaches to reading instruction and to develop more nuanced policy. Accordingly, Professor Thomas provides recommendations for state and local policymakers to provide teachers the flexibility and support necessary to adapt their teaching strategies to specific students’ needs. Find The Science of Reading Movement: The Never-Ending Debate and the Need for a Different Approach to Reading Instruction, by Paul Thomas, at: https://www.greatlakescenter.org/ . This policy brief was made possible in part by the support of the Great Lakes Center for Education Research and Practice ( greatlakescenter.org ). The National Education Policy Center (NEPC), a university research center housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu About The Great Lakes Center research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Contact: Alex Molnar : (480) 797-7261, nepc.molnar@gmail.com Faith Boninger : (480) 390-6736, fboninger@gmail.com Paul Thomas : (864) 294-3386, paul.thomas@furman.edu Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back Mayoral Governance and Student Achievement: How Mayor-Led Districts are Improving School and Student Performance Summary A recent report from the Center for American Progress claims that mayor-led districts may use resources more strategically and that mayor-controlled districts have seen increases in student achievement. The report also states that moving to a mayoral-controlled district can also help spur innovation and advancement. Publication Date April 22, 2013 Authors Katrina E. Bulkley, Montclair State University Research Areas Governance & Public Education; Assessment & Accountability Publication Type Think Twice Download the Policy Brief Press Release Mayor-Led Schools Report Problematic, Academic Review Finds EAST LANSING, Mich. (Apr. 23, 2013) – Mayoral governance – where a city's mayor replaces an elected school board – is in use in several major American cities, including New York City and Chicago. A recent report from the Center for American Progress claims that "mayoral-led" districts improve school and student performance. A new review questions whether mayoral control is appropriately credited with the claimed improvements. The report, Mayoral Governance and Student Achievement: How Mayor-Led Districts are Improving School and Student Performance , was authored by Kenneth K. Wong and Francis X. Shen. Wong and Shen studied fiscal and student achievement data in an effort expand the discussion around mayoral control of schools. Katrina Bulkley, Professor of Education Leadership at Montclair State University, reviewed the report for the Think Twice think tank review project. The review was produced by the National Education Policy Center (NEPC), with funding from the Great Lakes Center for Education Research and Practice. Professor Bulkley's review finds that the fiscal analysis of mayoral-led cities is "problematic due to inappropriate comparisons and a lack of reliable and valid evidence." Specifically, mayoral-led districts are claimed to have more resources as a result of the governance change but no evidence is provided that shows this is true. Regarding student achievement claims, she found the report highlights selected positive findings in a few districts, but does not address mayor-led cities where such gains were not found nor cities in the country that saw strong gains without mayoral control. The report's use of research literature is also called into question, as no peer-reviewed journals appear in the endnotes and there is a lack of citations to research that examines finances or student achievement. Furthermore, the review finds that the report "surprisingly" includes Philadelphia and Baltimore in the sections on student achievement but not the section on finances. This is surprising because these two cities are cited in the report as "mixed" models – with a combination of state and mayoral control. "Without their inclusion, only three out of the nine analyzed districts would have shown improvement [in student achievement] by the report's own definitions." On a positive note, Bulkley states, "This report offers useful information about the context for shifts to mayoral control in different cities and the challenges that may arise in such governance changes." However, Bulkley concludes that the limitations presented in the review prevent the report from being used for serious policy decisions. Find Katrina Bulkley's review on the Great Lakes Center website: http://www.greatlakescenter.org Find Mayoral Governance and Student Achievement: How Mayor-Led Districts are Improving School and Student Performance on the web: http://www.americanprogress.org/wp-content/uploads/2013/03/MayoralControl-6.pdf Think Twice, a project of the National Education Policy Center, provides the public, policymakers and the press timely, academically sound reviews of selected publications. The project is made possible by the support of the Great Lakes Center for Education Research and Practice. The review is also available on the NEPC website: http://nepc.colorado.edu --###-- The mission of the Great Lakes Center for Education Research and Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center Web Site at: http://www.greatlakescenter.org . Follow us on Twitter at: http://twitter.com/greatlakescent . Find us on Facebook at: https://www.facebook.com/GreatLakesCenter. Contact: Katrina Bulkley, (973) 655-5189, bulkleyk@mail.montclair.edu Dan Quinn, (517) 203-2940, dquinn@greatlakescenter.org Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More

< Back A New Agenda: Research to Build a Better Teacher Preparation Program Summary Bellwether Education Partners (BEP) released a report calling into question the effectiveness of teacher preparation programs in the U.S. The report proposed that improving teacher quality could be accomplished through teacher preparation program redesign. Further, it asserted that teacher preparation programs are “blindly swinging from one popular reform to the next” and decades of input- and outcome-based research have failed to improve teacher education. Publication Date March 27, 2017 Authors Ashley LiBetti Mitchel & Melissa Steel King; Marilyn Cochran-Smith, Stephani Burton, Molly Cummings Carney, Juan Gabriel Sánchez, and Andrew F. Miller; Boston College Research Areas Teacher Workforce Publication Type Think Twice Download the Policy Brief Press Release Teacher education report represents a missed opportunity, review finds EAST LANSING, Mich. (Mar. 28, 2017) — In October, Bellwether Education Partners (BEP) released a report calling into question the effectiveness of teacher preparation programs in the U.S. The report proposed that improving teacher quality could be accomplished through teacher preparation program redesign. However, an academic review of the report finds that it ignores past research as well as the present context of teacher education. The report, A New Agenda: Research to Build a Better Teacher Preparation Program , was reviewed by Dr. Marilyn Cochran-Smith, Boston College, and a team of scholars and practitioners. The review was produced for the Think Twice think tank review project, a project of the National Education Policy Center. Think Twice is funded in part by the Great Lakes Center for Education Research and Practice. A New Agenda asserted that teacher preparation programs are "blindly swinging from one popular reform to the next" and decades of input- and outcome-based research have failed to improve teacher education. Despite a reasonable assumption that teacher quality can be improved through redesigned teacher preparation programs, the reviewers find that the report includes mischaracterizations of teacher education research and omits relevant literature. The reviewers identified three primary problems of the report: It leaves practical questions unanswered; Muddies the waters about promising research avenues; and Ignores important bodies of literature in teacher education. In conclusion, the reviewers find that the report "represents a missed opportunity to offer guidance to [either] policymakers or institutions." Find the report on the GLC website: http://www.greatlakescenter.org Find the original report on the web: http://bellwethereducation.org/publication/new-agenda-research-build-better-teacher-preparation-program Think Twice, a project of the National Education Policy Center, provides the public, policymakers and the press with timely, academically sound reviews of selected publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice. The review can also be found on the NEPC website: http://nepc.colorado.edu The reviewers were part of Project TEER (Teacher Education and Education Reform), which is a collaborative research group interested in the intersection of teacher education policy and practice, on one hand, and neoliberal education reform, on the other. Reviewers included: Stephani Burton, Molly Cummings Carney, Juan Gabriel Sánchez, and Andrew F. Miller. Dr. Cochran-Smith is the Cawthorne Professor of Teacher Education for Urban Schools and Director of the Doctoral Program in Curriculum and Instruction at the Lynch School of Education, Boston College. Friend on Facebook Follow on Twitter The mission of the Great Lakes Center for Education Research & Practice is to support and disseminate high quality research and reviews of research for the purpose of informing education policy and to develop research-based resources for use by those who advocate for education reform. Visit the Great Lakes Center website at http://www.greatlakescenter.org/ Previous Next sign up for emails Get the latest K-12 education policy research delivered straight to your inbox! Read More