By John Sener
[Note: This is the first in a three-part series, “PISA Days Are Here Again.” See part 2 and 3.]
I’ve already written about comparative achievement tests.1 Now that “PISA Days” are here again, thus begins another round of national self-flagellation about the supposedly sorry state of American education based on dubious interpretations of international test scores of questionable credibility and limited value.2 I’m in the midst of a lot of work-related catching up, but I suspect I’ll be pulling together my thoughts about this latest round of PISA-induced moaning and gnashing of teeth over the next week or so.
I don’t think that the NBCNEWS article3 is strong at all — it’s as formulaic as all the others that are sprouting up as predictably as flowers after a desert rain. It quotes sources (e.g., Eric Hanushek) with well-known biases on which they base their living and professional careers. It has the predictable counterargument buried in the middle and ultimately dismissed without much substance, and it concludes that Things Are Bad and Slowly Getting Worse. I’ve already read several of these articles, and I will be surprised to find one for which I don’t already know how the plot ends.
BTW, here’s an angle on such articles that just popped up for me: last night, for unrelated reasons I was re-reading parts of Daniel Goleman’s book Emotional Intelligence (1995), in particular the chapter about “The Hallmark of the Emotional Mind.” Goleman apparently posits that we have an emotional mind as distinct from our analytical one. I haven’t done the mapping yet, but I have a strong suspicion that the current spate of PISA doom articles are based almost entirely on emotional arguments. No one writing these articles is doing any actual analysis of the facts, and if they did, the arguments would fall apart like a house of cards.
Part of me wants to bemoan the fact that so many professionals, including some educators, want to embrace the PISA results so uncritically. But I think the more interesting angle is the fact that we seem to respond, collectively and individually, to emotional arguments. Winning the analytical argument is not enough; one has to win the emotional argument somehow. I’m still figuring out how that happens…
Or, here is one antidote to this madness: Diane Ravitch’s article, which includes an analysis of Keith Baker’s work (which unfortunately costs $$ to access) through which he concludes that “standings in the league tables of international tests are worthless.” I’m in the process of collecting other more rational responses to this latest release of PISA results, from which I hope will emerge a reasoned, analytical summary of what the PISA results mean (or don’t mean). But I remain more intrigued by the question lurking in the background: why do people respond so emotionally and not rationally when it comes to international test score results? More on that in a later post…
__________
1 John Sener, “Standardized Tests and Foul Shooting: Look Out, Michael Jordan!“, ETCJ, 3/12/11.
2 John Sener, The Seven Futures of American Education: Improving Learning and Teaching in a Screen-Captured World, 2012, p. 78.
3 Daniel Arkin, “US Teens Lag in Global Education Rankings As Asian Countries Rise to the Top,” NBCNEWS/AP, 12/3/13.
Filed under: Uncategorized |
John Mark Walker: "If educational communities can continue to push platform integration and content portability, in the future, students may be able to design their own personalized degrees from smaller, modular chunks that cross institutional barriers" (
Richard Koubek
Judith McDaniel
Tim Fraser-Bumatay: "Although the format leaves us far-removed physically, the online forum has its own sense of intimacy" (Judith McDaniel, "
Ryan Kelly: "For me to be able to work with people clear across the country for an extended period of time opened me up to new things" (Judith McDaniel, "
Daniel Herrera: "As a Mexican American, I know that words of identity are powerful; so to discuss white privilege with my professor and classmates in a face-to-face class would have been terrifying and impossible" (Judith McDaniel, "
Cathy Gunn: "Traditional methods for effecting change at my institution aren’t getting us even to a trickle yet, let alone to thinking about or planning for a wave!" (
Educational Technology and Change Journal 
Just as some medical measurements should not be taken alone but looked at over a period of time, so it is with PISA. Differences in schooling practices among nations make direct comparisons difficult as do the nature of testing itself. However, we have enough data over many years to see if things are going up or down. It appears that we’re staying the same in terms of score and dropping relative to other nations. The first result would not be so bad if the scores were higher. The second is a cause for concern.
Let’s consider these data with other data that we have, including NAEP. At best, we’re not improving. I can put all sorts of reasons forth for this situation but cannot guarantee that one is the culprit. I also can suggest fixes but few can be implemented in our culture.
I would agree with your comment if PISA scores had anywhere near the utility of those medical measurements to which you refer.
The core issue is whether PISA tests in any way reliably measure anything useful. If not, then fluctuation of results don’t matter over time. I would argue that they don’t mean much (that argument is forthcoming), and rankings mean even less (Baker’s work cited above covers the latter ground, I believe).
Even if PISA tests do measure something useful, numerous question remain: for example, why should these results apply to all students? Why is it important that an individual has the ability, for example, to calculate square footage to figure out the cost of carpet (one of the PISA math questions) if that individual can go to the web and have an app do the calculations for them? (or a friend or a spouse) How important is this ability when real world experience constantly demonstrates that millions of adults function quite well without having this capability?
I’ve long suspected that some of the answers to these questions ultimately relate not to learning or even education, but to equity — more specificially, to a society that still has no clue about how to create an equitable society. Seeking to install a uniform body of content in each child is simply one manifestation of a cultural inability to deal with the issue of societal equity…
that should be “numerous questions” above, not “question”…
All right, we may have slightly different takes on this issue.
First off, I’ll suggest that that carpet cost question requires more than just calculating square footage. Sure, it’s just a two-step problem, but too many graduates of our mathematics classes can only do one-step problems. Apps tend not to handle multi-step problems well except in unusual cases. We should be testing the ability of students to solve problems that require more than one step, and preferably more than two, of reasoning. You cannot replace real thinking with an app.
Also, I may have failed with my analogy. I was attempting to illustrate the fact that a series of measurements can have more value than a single snapshot. Most of the hue and cry over PISA has to do with the latter. Oh my! We’re only at (whatever the number is) ranking internationally. We’re doomed.
Deeper thinking is required. PISA can have use if used well. Why toss out data that might be mined? The decline in our standing shows that other countries are improving their education systems more than we are. Even if the instrument is flawed, there’s still a message.
However, the most striking observation is the one you mention, the extreme inequity in education across our country. No one can doubt these figures. They’re far beyond any error bars.
Yet, I do not agree that we are “a society that has no clue about how to create an equitable society.” We have plenty of clues but have chosen not to do so — to our eternal shame. Remember busing in the 1960s? That was an attempt and was fought every inch of the way even though it did not create education equity but only moved in that direction.
Imagine a more extreme set of measures that include taking money from wealthy areas and pouring it into poor areas without allowing the wealthy to make up the difference through private foundations. We do have clues about how to make our education system more equitable, but we choose only to do the little things that make only small differences. Were we to try more, we’d be accused of socialism and fascism at the same time.
Where to start? How about shutting down those radio demagogues that spew hatred daily? Oops. Can’t curtail free speech, but our free speech amendment was written for a time when soap boxes and print were the only means of communication and hate mongers could not make ten million dollars a year.
The saddest part of all of this is that better education for all would create a citizenry less susceptible to demagogues.
I’ll add that, apropos of this journal, technology is beginning to have an impact on learning for the poorer schools. It’s just a start, and poor schools are technology-poor as well.
Today, I can walk a few blocks and pick up a $70 Android tablet. It’s slow and has a small display, pixel-wise, but it’s an indication of what may be coming.
I have been working hard and sacrificing a comfortable lifestyle for many years to contribute to the promise of technology for the poor schools. I am happy to see my work now beginning to be used by middle schools in poor neighborhoods in NYC and by the California State University, the largest university system in the country with over 400,000 students.
We can do better. Technology can help if the purveyors of snake oil technology would get out of the way and if poor schools could get some real support for Internet-based technology.