What Do the PISA Test Results Really Mean?

In the aftermath of the recent PISA test reports, the common belief is (1) the scores bode ill for the US’s future in the global economy and (2) the answer to better scores is innovative instructional technology. What do our ETCJ readers think?

Marion Herbert, in “The United States Receives Average Rankings from PISA“* (District Administration, 15 Dec. 2010), underscores the relationship between PISA test results and America’s future in the global economy. She echoes U.S. Secretary of Education Arne Duncan, who says:

Being average in reading and science — and below average in math — is not nearly good enough in a knowledge economy where scientific and technological literacy is so central to sustaining innovation and international competitiveness. The results are especially troubling because PISA assesses applied knowledge and the higher-order thinking skills critical to success in the information age. (7 Dec. 2010)

Central to the popular response to the test results is the belief that instructional technology will need to play a key role in America’s comeback. Herbert says, “Reactions to the PISA results have generally spouted the same message — the United States needs to raise the bar with education innovation to create college-and-career-ready students who can better compete in a global economy.”

__________
* Webcitation alternate link.

2 Responses

  1. I’ll make a controversial statement here. Science scores, especially with well designed assessments, are better summaries of student ability and better predictors of future performance. They also forecast the future of their countries. Reading and math scores are useful adjunct measures.

    Why?

    It’s because success in a good science assessment requires excellent reading skills and good mathematics ability. Those separate scores can be used to tease apart the reasons for science scores and decide whether students did well or poorly due to language arts or mathematics.

    In addition, science requires more advanced thinking skills than most language or math assessments — if the test has been designed well.

    Science is not about why the sky is blue or the phases of the Moon or of mitosis. It’s about a way of viewing the world. Rather than press on with a diatribe on the value of a science education, I’ll just finish off with a quote from a famous scientist: John Herschel. He was the son of William Herschel who discovered Uranus and upset the then-current views of the cosmos.

    “To the natural philosopher there is no natural object unimportant or trifling.. A mind that has once imbibed a taste for scientific enquiry has within itself an inexhaustible source of pure and exciting contemplation. One would think that Shakespeare had such a mind in view when he describes a contemplative man finding

    Tongues in trees — books in the running brooks
    Sermons in stones — and good in everything

    “Where the uninformed and unenquiring eye perceives neither novelty not beauty, **he** walks in the midst of wonders.”

    (The **he** was in italics in the original.)

  2. The PISA report draws a lot of inferences, but the problem is that people who really ought to know better are taking them as fact.

    There seems to be a correlation between high PISA scores and rank in lists of global economies. Thus, based on its scores, the U.S. appears to be headed for economic mediocrity.

    Yet, based on other indicators, e.g., The Global Competitiveness Report 2010–2011 (page 14), the U.S. remains within the top 5 economies. We’ve dropped from 2nd to 4th since the last report, and that’s definitely reason for concern, but the causes for this decline aren’t clear.

    The danger of correlations is the tendency to read causal connections into them. For example, if declining scores seem to be related to our declining economy, then raising the scores will improve our economy.

    The problem is that we don’t have conclusive proof that one causes the other. In fact, we may be oversimplifying the issue. Dropping test scores may be indicative of one of the causes, but they may not be the only one or even the most significant.

    The upshot is that the nation’s schools — and, more specifically, teachers — are, once again, the convenient scapegoats.

    Questions remain: What are the most valid measures or dependent variables for economic decline, and, more importantly, what are the critical causal independent variables?

    Our educational systems play an important role, but it’s more than likely that the causes are complex, and schools and colleges play a part in but are not completely responsible for our perceived economic woes. -Jim S

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s