Sturgeon’s Law is as true of scientific research as everything else. “Sturgeon’s Law” is usually stated as “90% of everything is crap”.
At Stat Ivan Oransky and David Marcus point out that the real problem afflicting science may not be outright fraud but just plain shoddy workmanship:
If a burst pipe in your house is flooding your basement, you’re probably going to be more worried about that than the couple termites you previously spotted. But multiply those termites times a thousand and suddenly the bigger threat to your house might be, well, the little things.
The same holds true for science. Science fraud draws urgent attention whenever it comes to light, the equivalent of a busted pipe emergency. But it turns out, most scientists think it’s a far lesser threat to their field than the small, but legion, instances of underreporting of negative findings and scientists’ use of shoddy methodology.
I think there’s an even greater problem: the creation of a scientific establishment funded through government grants. It lends itself to cronyism, corruption, and waste. There are far, far too many examples of studies with irreproducible results being published by the “right people”. I’ll give your bogus study a good peer review if you’ll give mine one.
Maybe we should start worrying that there’s a sort of Gresham’s Law with respect to scientific research in which bad research drives out good research.
IMO we should begin changing our strategy for funding research to one based on results. What might that be? I don’t know. A greatly expanded prize system as opposed to the present grant system?
Here’s a part of the problem. Start with the tweet linked and the whole thread, or at least those tweets by the original tweeter.
I have a proposal to change the entirety of US higher education. Change the bankruptcy laws so that seven years after completing a degree, a college student CAN discharge student loan debt in bankruptcy. No sane person is going to pass on seven years of productive living if they can do it, so that’s enough time to keep sane people from gaming the system, if they can.
The catch is that then the institution that granted the degree is on the hook unless they can easily prove it has nothing to do with their work. (For example the person was crippled in an accident and was unable to work.) But if not, the college has to pay the remaining debt plus a penalty. I suggest the penalty be 100% of the degree costs. It’ll take 20 years or so, but that will clear up a lot of nonsense with the current system. It’ll probably mean fewer barristas with PhDs in grievance studies down the line, too. Consider that an added benefit.
It’s possible I read that idea somewhere else, once. Honestly I read so much I have no idea which thoughts, if any, are my own anymore.
It’s definitely a systemic problem, the incentives practically guarantee there will be a lot of junk science.
Here’s the thing. It’s cheaper and easier to do bad science than it is to do good science. Who will get the grants? As I suggested, the bad drives out the good.
Dave,
Your analysis has a distinguished pedigree:
“Akin to, and largely responsible for the sweeping changes in our industrial-military posture, has been the technological revolution during recent decades.
In this revolution, research has become central; it also becomes more formalized, complex, and costly. A steadily increasing share is conducted for, by, or at the direction of, the Federal government.
Today, the solitary inventor, tinkering in his shop, has been overshadowed by task forces of scientists in laboratories and testing fields. In the same fashion, the free university, historically the fountainhead of free ideas and scientific discovery, has experienced a revolution in the conduct of research. Partly because of the huge costs involved, a government contract becomes virtually a substitute for intellectual curiosity. For every old blackboard there are now hundreds of new electronic computers.
The prospect of domination of the nation’s scholars by Federal employment, project allocations, and the power of money is ever present
and is gravely to be regarded. Yet, in holding scientific research and discovery in respect, as we should, we must also be alert to the equal and opposite danger that public policy could itself become the captive of a scientific technological elite.”
Dwight Eienhower, Farewell Address, 1961