So, About those International Test Comparisons

2 minutes read

For sure, you have heard that the U.S. is way behind other industrialized nations based on international tests in literacy and math. There are many footnotes that should accompany such claims, but usually don’t. Stanford researchers broke down precisely how one of those footnotes–economic inequality–affects where the nation stands.

Based on their analysis, the co-authors found that average U.S. scores in reading and math on the PISA are low partly because a disproportionately greater share of U.S. students comes from disadvantaged social class groups, whose performance is relatively low in every country.

As part of the study, Carnoy and Rothstein calculated how international rankings on the most recent PISA might change if the United States had a social class composition similar to that of top-ranking nations: U.S. rankings would rise to sixth from 14th in reading and to 13th from 25th in math. The gap between U.S. students and those from the highest-achieving countries would be cut in half in reading and by at least a third in math.

“You can’t compare nations’ test scores without looking at the social class characteristics of students who take the test in different countries,” said Carnoy. “Nations with more lower social class students will have lower overall scores, because these students don’t perform as well academically, even in good schools. Policymakers should understand how our lower and higher social class students perform in comparison to similar students in other countries before recommending sweeping school reforms.”

The report also found:

– There is an achievement gap between more and less disadvantaged students in every country; surprisingly, that gap is smaller in the United States than in similar post-industrial countries, and not much larger than in the very highest scoring countries.

– Achievement of U.S. disadvantaged students has been rising rapidly over time, while achievement of disadvantaged students in countries to which the United States is frequently unfavorably compared – Canada, Finland and Korea, for example – has been falling rapidly.

– But the highest social class students in United States do worse than their peers in other nations, and this gap widened from 2000 to 2009 on the PISA.

– U.S. PISA scores are depressed partly because of a sampling flaw resulting in a disproportionate number of students from high-poverty schools among the test-takers. About 40 percent of the PISA sample in the United States was drawn from schools where half or more of the students are eligible for the free lunch program, though only 32 percent of students nationwide attend such schools.

With each release of international test scores, many education leaders assert that American students are unprepared to compete in the new global economy, largely because of U.S. schools’ shortcomings in educating disadvantaged students.

“Such conclusions are oversimplified, frequently exaggerated and misleading,” said Rothstein, who is also senior fellow at the Chief Justice Earl Warren Institute of Law and Social Policy at the University of California – Berkeley School of Law. “They ignore the complexity of test results and may lead policymakers to pursue inappropriate and even harmful reforms.”