Thursday, December 05, 2013

On PISA and Poverty

This is cross-posted at Harry's Place.

The newest PISA scores have been released, and surprise, the US and Britain are trailing most of their Asian and European counterparts in both reading and mathematics. When these scores are released every three years, the same regrettable headlines are brought to head and we are reminded that our education systems sputter along the path of mediocrity. For the last two decades, this meme sat perfectly with the rise of China and Korea economically and the perceived faltering of Western dominance on the world stage, and thus produced countless reforms that were meant to put our students back in a competitive advantage.

Yet, with the growth of the charter school movement, merit pay, and rigorous testing protocols, American students have been stuck right about where they were 20 years ago. There is a reason for this, but it is rarely addressed in the media when education policy is discussed.

That central variable is poverty.

I can see many eyes rolling already. “Yet another excuse for why we can’t compete internationally from some lefty bleeding heart” is likely already filling the comments section below by incensed detractors. Some of the criticisms made of our current education model are sound: training qualified teachers and firing the bad ones while increasing the rigor of the curriculum should be paramount to reformers of all stripes.  Nor am I opposed to testing as a tool to measure student and school success. There is a lot we can learn from the highest performing PISA states, and my years teaching in Korea have given me a window into just why many Asian countries excel in this regard. However, if we are to use assessment tools like PISA to make broad judgments about our education systems, we need to look closely at what those results truly reflect.

Let’s deal with the most glaring headline garnered form recent PISA data: Chinese students are doing far better than their American and British counterparts. David Stout was quick to remind people that China was the only nation to be represented by a single city, and not their entire populace. He argues:
Shanghainese and Hong Kong students are much better educated than those elsewhere in China. Slate quoted the Brookings Institution’s Tom Loveless as saying that  “About 84 percent of Shanghai high school graduates go to college, compared to 24 percent nationally.” In addition, Loveless points out that affluent Shanghainese parents will spend large sums on extra tuition for the children — paying fees that far exceed what an average worker makes in a year.
China has simply cherry picked their most affluent students for testing, and completely omitted the majority of their people. In fact, Chinese rural education policy is geared almost entirely towards attendance and not testing. Poorer students from the countryside that happen to live in Shanghai, but are designated by the government as “non-residents,” are routinely denied access to the city’s high schools as not to tarnish the city’s PISA ranking.
Finland’s education system has also been championed as a model for the US to follow. Unlike their Asian counterparts, Finland does not have mandatory testing until late into a student’s schooling, practices loose curriculum management, and provides a generous welfare state making it a highly egalitarian state. Concurrently, Finland routinely comes in near the top of the PISA rankings.

However, comparing Finish test scores to American and British ones is a bit like comparing apples to oranges. Rafael Irizarry at Simply Statistics graphed data provided by the NASSP to show just how well affluent American students compare to their like-minded Finish counterparts.

Rafael explains:
“The plot on the left shows PISA scores versus the percent of students living in poverty for several countries. There is a pattern suggesting that higher poverty rates are associated with lower PISA scores. In the plot on the right, US schools are stratified by % poverty (orange points). The regression line is the same. Some countries are added (purple) for comparative purposes (the post does not provide their poverty rates).   Note that US school with poverty rates comparable to Finland's (below 10%) outperform Finland and schools in the 10-24% range aren't far behind. So why should these schools change what they are doing? Schools with poverty rates above 25% are another story.”
This glaring fact is rarely addressed in coverage related to the PISA, and adds much needed perspective to the problems facing our education systems.
None of this excuses bad schools and teachers from responsibility. Yet it does demonstrate that when variables like wealth and access are equal, Americans generally perform as well as their counterparts elsewhere. In both the US and Britain, we face cultural and social problems that have helped exacerbate our educational problems. Conservative critics of our public education systems are right to criticize waste and “feel-good” teaching policies that have only worsened our schools. But poverty matters, and as long as it remains unaddressed by the “school reform” movement, poor students will generally lag behind their more affluent brethren around the globe.


TNC said...

I saw your link to this on fb a while back but I am sorry to say I passed it by because I had no clue what PISA is. Thought it was some IR term or something. :-)

My comment, and I can see you received similar ones at HP, is you missed out on another key variable and that is race. Do Chinese or Korean tests use "race" as a category? How about European countries? I am not talking about national origin. We have that as a category too. My guess is among the poor and high performing students the correlation is strongest with race, national origin/ethnicity, and then income. Income is an important factor, especially since we can easily quantify it, but the other two are important as well.

Look at the percentage of Indian, or Chinese or Jewish kids who graduate from high school across various social classes and compare them to other groups. African-American, Latino, and Native-American students have the greatest difficulty graduating.

The term race might be a bit misleading as, interestingly, African immigrants and Afro-Caribbean immigrants do not have the same difficulty graduating as African-Americans. And Latino is also somewhat misleading. It is more accurate to break things down by ethnic/national group. Cubans and Argentinians tend to graduate at higher rates than Puerto Ricans and Mexicans. Again, none of this dismisses income as a variable, just trying to suggest some other explanations.

One last comment regards language. We have a lot of immigrants who speak and use English as a second language in our urban schools and those along the southern border. Of course most people think of Spanish, but there are also Mandarin and Cantonese speakers, the languages of the Indian subcontinent, Africa, and across the world. Is this taken into consideration?

TNC said...

In case I was not clear, I am not arguing in favor of biological determinism, that Europeans are smarter than Africans, or anything of the sort. I am trying to point out the complexity of race both as a term of analysis and self-identity. And that racial and ethno-national diversity likely accounts for more statistical variation on these PISA tests than income.

Roland Dodds said...

There is surely a cultural dimension. It should be noted that in countries like Korea, these tests are stressed as a way to demonstrate national and individual pride. Other states (like the US), see them as something you get through but little else. It is thus not surprising that students coming from nation's like Korea do better on said tests when they come to the US.

Furthermore, I am inclined to note that culture matters (as you specified). Not the broad "culture" we often use in the US (European, Mexican-American, etc.), but specific aims and goals of a local community. It would be interesting to see if Caribbean-American students who are first generation come from what would constitute "middle class" in their parent's home country. I don't know the answer to this.

TNC said...

Good point re: national pride/nationalism. Another thing is the cultural diversity in the US compared to South Korea, Japan, or the Nordic countries. The pressure you mention seems to be more effective within a particular cultural/ethnic group than across these divides.

Re: Caribbean immigrants, there is no common class background, at least not in my limited teaching experience. In terms of islands, mostly PR, DR, Trinidad/Tobago, Haiti, and Guyana. Not many Cubans or Jamaicans.