In reference to his unsuccessful run for California governor, Upton Sinclair used to say “It is difficult to get a man to understand something, when his salary depends upon his not understanding it!” Similarly, when bureaucrats’ or politicians’ livelihoods or national prestige are on the line, do not underestimate the ability of bureaucrats to torture statistical findings to produce the desired results.
More than fifty years ago there was a best-selling book, “What Ivan Knows That Johnny Doesn’t”, a comparison of Soviet and American curricula, that engendered a furor here in the States and a push for curriculum reform, shortlived as these pushes invariably are.
That’s the context in which I see the periodic panics about how the test scores of American kids on the PISA exams compare with the scores of kids in other countries, notably China. As reported by Adam Minter at Bloomberg View, China is experiencing a little panic of its own:
In 2009, Shanghai students did so well — beating the world in math, science and reading — that President Barack Obama declared it a “Sputnik moment,” requiring immediate action. A similar panic broke out in 2012. But this year proved to be a surprise. The results from the 2015 tests, released this month, showed Chinese students ranked sixth in math, 10th in science and 27th in reading. What happened?
On one hand, the answer is simple. Instead of merely testing Shanghai’s elite, the 2015 exams included a broader selection of students across China, which dragged down scores. But the results also highlighted an important problem: China’s much-lauded education system remains riven by inequality, with far-reaching consequences for schools, students and, ultimately, the economy.
There is a well-known, widespread, cheap and easy way to improve a performance of a country’s students on international standardized tests: jigger the statistics. Limit who takes the test. Engage in fraud. Teach the test. After all, who will know?
Shenanigans in reporting educational statistics is by no means limited to China. Here in Chicago our city fathers have perfected it.
There is no global, cross-cultural definition of literacy. Each country defines it in its own way and measures it on its own. I do not know that it is still the case but China used to use different definitions of literacy depending on what was referred to as the individual’s “station in life”. A farmer could be deemed literate if he could recognize his name when he saw it. The standards were higher for a factor worker and higher yet for, say, a government worker.
That’s why when I see all of those 99% literacy rates reported by UNESCO, I take them with a grain of salt, especially when they’ve improved dramatically over a very short period of time. One possible reality check is to compare the reported literacy rate with the number of books published annually in a country. In the United States that’s about 304,192, roughly 92 per 100K population. There are more books published in Spain (population 48 million) every year than in the entire Arab world (population 366 million).
