By John Sener
Upon further consideration, I’ve decided to join the crowd and embrace the common wisdom about PISA, SAT, and other standardized test results. What changed my mind was when I realized how awesomely powerful the principles that drive the acceptance of these results really are. You see, simply by applying them, I can prove to you that I am a better basketball player than Michael Jordan.
Don’t laugh. I have the test results. Some time ago I saw something online somewhere which showed MJ participating in a friendly free throw shooting contest. He made 16 out of 20. Pretty good, but I figured I could do better. So I went out to my local gym and practiced and practiced and practiced until, finally, I achieved my aim: I made 18 out of 20! There was a witness who could vouch for me. I’ll send you the video if you like. (Or you could do what most people do with PISA scores and simply take my word for it.)
It’s not a fair comparison, you say? But that’s what’s so great about this — all you have to do is apply the same rules we apply to judging PISA scores, and it’s perfectly fair:
- You say it needs to be a head-to-head competition? But PISA’s not a head-to-head competition. The students take the tests at different times in different places under different conditions. Heck, they take the reading test in different languages.
- You say I took it far more seriously? What makes you think that American students take PISA tests or scores seriously? When I tested my teenage son’s knowledge of the PISA exam, he looked at me quizzically — not surprising since he’d never heard of it. Now, the SAT test we take very seriously. In fact, I just shelled out a sizable chunk of change for an SAT prep class. But that test has actual personal consequences attached to it, however tangential or dubious they might be. So seriousness matters.
- I had an unfair amount of practice, you say? Do you really believe that every student who takes the PISA has the same amount of practice? I earned my superior results through a laser-like, focused effort on the defined task at hand — free throw excellence.
- You say I’m not capable of replicating my results? No one asks PISA test takers to replicate their test results; they simply accept them and the rankings they imply. So if the US is #23 or #45 or whatever it is in the PISA rankings, then I’m above MJ in the basketball rankings.
You may question my results based on the notion that there’s a lot more to being a basketball player than shooting free throws. That’s the beauty of it — there’s a lot more to learning and education, too, but it doesn’t matter. So what if MJ can dribble, pass, shoot, rebound, defend, penetrate and dish, run the floor, or do lots of other skills so much better? Standardized tests don’t measure most skills either, yet opinion leaders and policymakers constantly tell us how America’s education is going down the toilet based on those scores. So how important could those other skills really be?
You may question my superiority based on the fact that MJ can do creative feats on the court that I can’t even dream of. But there is no place in standardized tests for creativity so it doesn’t matter. You may be curious about whether or not I actually followed the rules to achieve my reported results, or question whether or not the free throw competition I described even exists. You would be wise to ask these questions even though standardized tests don’t care about curiosity either.
But whatever you do, don’t question the value of my rankings — because then you might have to question the value of other rankings like PISA, and goodness knows it’d be foolish to think that MJ could possibly be a better player than me after considering the definitive, quantitative evidence that I’ve given you. After all, free throws are the perfect measure of basketball attainment — they’re rigorous, objective, and easy to score. They’re the same for everyone regardless of gender, race, or nationality. They can be used to compare individuals, teams, schools, states, even nations. (I figure my free throw shooting percentage puts me in the top five of developed nations worldwide.) Most of all, they often make the difference between winning and losing — and we want to be winners in the ever-more fiercely competitive global arena, don’t we?
So the next time the U.S. basketball team fails to win an Olympic gold medal or world championship, instead of doing silly things like finding the right coach or more dedicated players, I have a much better idea. Let’s launch GAFSP — the Great American Foul Shooting Program. Every 4th, 8th, and 12th grader will be required to practice free throw shooting daily until we know through continual assessment that our basketball superiority is forever secure. We’ll pattern it on No Child Left Behind — you know the drill. No allowing for “pushouts” this time around, though. Our reputation as the world’s Greatest Basketball Power is too precious to squander by failing to fix this problem.
And don’t be sidetracked by sideshows such as dunking phenom Jacob Tucker with his 50″ vertical leap. Everybody knows that dunking is not a good measure of overall basketball ability since it depends on genetic good fortune and individual talent brought to fruition by hard work. How could his dunking prowess possibly compare to the value demonstrated by superior performance on a standardized test?
Of course, my ultimate goal is to have the world accept my superior basketball prowess uncritically and unquestioningly so I can use this to my economic advantage. I might even start my own spin-off company to teach others how to attain these lofty results using the latest technology-enabled methods — instructional videos, online materials and support, maybe even a little scripted curriculum thrown in for good measure. Can’t be too careful about achieving consistent results, you know. So if you’ll excuse me, I’m off to spread my “I Rule Hoops” meme as far and as wide as I can — Tucker’s YouTube video has gone viral, and I have a bit of catching up to do…
Filed under: Uncategorized |
John Mark Walker: "If educational communities can continue to push platform integration and content portability, in the future, students may be able to design their own personalized degrees from smaller, modular chunks that cross institutional barriers" (
Richard Koubek
Judith McDaniel
Tim Fraser-Bumatay: "Although the format leaves us far-removed physically, the online forum has its own sense of intimacy" (Judith McDaniel, "
Ryan Kelly: "For me to be able to work with people clear across the country for an extended period of time opened me up to new things" (Judith McDaniel, "
Daniel Herrera: "As a Mexican American, I know that words of identity are powerful; so to discuss white privilege with my professor and classmates in a face-to-face class would have been terrifying and impossible" (Judith McDaniel, "
Cathy Gunn: "Traditional methods for effecting change at my institution aren’t getting us even to a trickle yet, let alone to thinking about or planning for a wave!" (
Educational Technology and Change Journal 
John, funny how much we can learn when we’re able to let our guard down and just laugh — at ourselves for buying so completely into standardized tests that we allow them to rule our common sense.
Yes, exactly! What do the scores really mean? Are they valid? Do they really measure what they’re supposed to? Or are they measuring variables that have little or nothing to do with the skills and knowledge that we value?
Your example of free-throws as a measure of overall basketball ability is a good one. As you “say,” they’re one measure — but there are many, many others with vastly higher predictive value. But we don’t test these because they’re harder to quantify and we haven’t found cost-effective means to do so.
Actually it’s not so funny that we’ve developed an educational system that selects a few key, easily measurable variables to decide who gets the best education, who get the second best, etc.
And as you say, if we selected our Olympic basketball team the same way, we’d end up with a team full of great free throw shooters with average or even poor overall basketball skills.
When we look at our nation’s colleges and complain about the deterioration of students’ critical thinking skills, we really don’t need to look very far for at least one of the answers. Standardized tests are cost-effective, but the corners they cut are the very ones that make the difference between Michael Jordan and the rest of the human race.
Thanks for a humorous and illuminating article. -Jim S
Back in my staff developer days, when I talked about assessment, I invented this phrase: When we can’t measure what is important, we make important what we can measure.
John’s article is a good example of what I meant by that. We look for any objective measure we can find, no matter how peripheral it may be to what really counts, make a test out of it, and before long that is the only things that matters.
John A, I like your wisdom in a nutshell: “When we can’t measure what is important, we make important what we can measure.”
It’s a perfect prompt for writing. I’ve “written” at least three different articles in my head in the last few minutes. Each met a dead end, and I’ve had to double back to the prompt and start anew.
I’m not sure if I’ll find the article I’m looking for. If I do, I’ll post it here as a comment. -Jim S
Jim,
Your last comment reminds me of an old Peter Drucker chestnut, “what gets measured gets done.” In education, that tends to translate into “what gets graded gets done.” In my experience, we do that because it’s the easy(ier) path to take…
John S, I like that chestnut, too, and the logical counterpart, “what’s not tested isn’t taught or learned.”
In our private email exchange, I mentioned spending a lot of my college days with dorm-mates playing pick-up basketball at the nearby playground and campus gym. It wasn’t part of the credited, tested, and graded program so the university completely ignored this part of my education.
Nevertheless, I learned a lot of life skills there. And perhaps the most important is that all players understand the rules, procedures, and “net”-iquette without referees or any official or formal oversight. If profs or staff were participating, they did so as players like everyone else.
We came and left when we felt like it. If we came alone, we waited at courtside for someone to leave a game. If more than one wanted in, we shot free-throws to see who’d play. If we came as a group of three or four, we’d pick up 1 or 2 players waiting to get into a game and claim “next” to play the team, on the court, that won. The winning team stayed on the floor and played all comers.
I never played pick-up ball in other states, but my guess is the rules are the same wherever you go. As basketball has spread throughout the world, I’m guessing that the same unwritten playground rules are applied universally.
The lesson is that learning doesn’t have to be formally organized or sanctioned. We use the word “informal” to distinguish this type of learning, and in this way we separate it from formal learning that’s based on tests, scores, grades, etc.
I learned that this “open” and democratic form of play is fun and rewarding and that players learn how to collaborate, communicate, value and rely on one another, form temporary as well as lifelong bonds in a game that comes close to anytime and anywhere.
It provides insights into people and effective real-world social practices, and it’s based on individual ability, effort, and attitudes. Equality. Race, sex, age, socio-economic status, GPA, nationality, language — none of this matters. To survive and thrive in this environment, you have to know the rules, respect others, collaborate, carry your share of the load — and play fair.
Perhaps one of the most important aspects of pick-up ball is the universal understanding that you call your own fouls. If you foul someone in the act of shooting, you call it on your self and turn the ball over to the other team. If the shooter scores despite the foul, the score stands. If you’re dishonest, you quickly lose the respect of the other players on both teams, and this is enough of a sanction to keep 99% of the players honest.
Pick-up ball doesn’t appear on my transcripts or on any of my formal school records, but I gained an invaluable education there that has influenced who I am personally and as an educator. As editor of ETCJ, I now realize that I’m applying the same pick-up rules: anyone is free to join us as a writer as long as s/he is willing to respect others, contribute, collaborate, etc., and I’ll sometimes invite (“pick up”) people to join ETCJ as writers. -Jim S
As a member of the staff of a much larger forum, I have frequently advocated the kind of philosophy that you describe here when it comes to moderation. If handled properly, a forum can become self-moderated this way. In my arguments I differentiate between what you referred to as “rules” and the concept of “norms.”
In 1887 Ferdinand Tonnies wrote about the difference between gemeinschaft (community) and gesellschaft (society). A community operates by community values, or norms of behavior, such as a pickup basketball game. A society operates by contractual rules, which re often codified. A community polices itself, but a society usually needs some sort of enforcement mechanism. As an organization moves closer and closer to a rule-based society, it loses its norm-based nature. The community stops policing itself because that is not its job–it waits for the identified enforcer to step in and act.
I thus always advocate that we strive as much as possible to keep a forum like this as community-oriented as possible so that a norm-based behavior will be encouraged. Moderating according to contractual rules should be reserved only for the hard cases that resist such pressure.
The roles change primarily because of consequences. You said that 99% of the time it works. In a pickup basketball game, the lone time that it doesn’t work causes nothing worse than hard feelings. When the stakes are raised, the individual who is committing the transgression will be more likely to endure the taunts and peer pressure to gain the intended advantage. A burglar is not held back by the knowledge that a majority of the world’s population does not approve of his profession.
John A, you raise the issue of how best to teach ethical behavior, and the large, impersonal educational systems that we’ve developed may not be the best approach since they’re based on institutional rather than peer enforcement. Thus, in a sense, we may be increasing rather than decreasing poor behavior. In the “norm-based” context the rule seems to be “unethical behavior is OK as long you don’t get caught by the authorities.” Peers turn a blind eye to transgressions because they don’t want to get involved. The attitude seems to be, “Let the authorities deal with it.” In fact, if they see that others are getting away with infractions, then they’re also likely to join them.
Re rewards and learning — in the ideal school or college environment, students would learn to develop cohort-based ethical systems, and these systems would be formally recognized and rewarded by the institution.
The concept isn’t new. The honor system has worked in places, and the military academies come to mind. But to really work, the reward and punishment need to be totally peer managed. The question is how to guide students toward an ethical system that they “own” that also reflects institutional and social standards. -Jim S
[…] Standardized Tests and Foul Shooting: Look Out, Michael Jordan! « Educational Technology and Change… (tags: education reform standardized testing foul shooting) […]
[…] Standardized Tests and Foul Shooting: Look Out, Michael Jordan! is by John Sener. […]
[…] Standardized Tests and Foul Shooting: Look Out, Michael Jordan! is by John Sener. I’m adding it to The Best Sites For Getting Some Perspective On International Test Comparison Demagoguery. addthis_url = 'http%3A%2F%2Flarryferlazzo.edublogs.org%2F2011%2F03%2F21%2Ftodays-collection-of-good-school-reform-articles%2F'; addthis_title = 'Today%26%238217%3Bs+Collection+Of+Good+School+Reform+Articles'; addthis_pub = ''; […]
So, then I am probably better than Emeril! I just gotta burn enough omelettes and I am golden.
…at least Mom appreciates my cooking :-)
Great commentary! Eloquent reminder of why we need to be exposing the Publishing Industries efforts to capitalize on my children I teach everyday. When my students fail their tests, they can sell my district the Practice Exams for next year. Ho hum.
Looking forward to more gems from you, Sener.
Adam Heenan
Chicago
@ClassroomSoothe
[…] Also this Thursday, we are joined by John Sener, an expert in technology-enabled learning methods, and founder & CEO of Sener Learning, to discuss his take on the future of cyber-education. We’ll also discuss John’s take on the role of standardized testing in the US and global educational systems as published in the Washington Post as well as in the Educational Testing & Change blog. […]