What if 80% of all medical studies were confabulated poppycock? That’s what the work of John Ioannidis and his colleagues suggests:
Ioannidis was shocked at the range and reach of the reversals he was seeing in everyday medical research. “Randomized controlled trials,” which compare how one group responds to a treatment against how an identical group fares without the treatment, had long been considered nearly unshakable evidence, but they, too, ended up being wrong some of the time. “I realized even our gold-standard research had a lot of problems,” he says. Baffled, he started looking for the specific ways in which studies were going wrong. And before long he discovered that the range of errors being committed was astonishing: from what questions researchers posed, to how they set up the studies, to which patients they recruited for the studies, to which measurements they took, to how they analyzed the data, to how they presented their results, to how particular studies came to be published in medical journals.
This array suggested a bigger, underlying dysfunction, and Ioannidis thought he knew what it was. “The studies were biased,” he says. “Sometimes they were overtly biased. Sometimes it was difficult to see the bias, but it was there.” Researchers headed into their studies wanting certain results—and, lo and behold, they were getting them. We think of the scientific process as being objective, rigorous, and even ruthless in separating out what is true from what we merely wish to be true, but in fact it’s easy to manipulate results, even unintentionally or unconsciously. “At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” says Ioannidis. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.”
Perhaps only a minority of researchers were succumbing to this bias, but their distorted findings were having an outsize effect on published research. To get funding and tenured positions, and often merely to stay afloat, researchers have to get their work published in well-regarded journals, where rejection rates can climb above 90 percent. Not surprisingly, the studies that tend to make the grade are those with eye-catching findings. But while coming up with eye-catching theories is relatively easy, getting reality to bear them out is another matter. The great majority collapse under the weight of contradictory data when studied rigorously. Imagine, though, that five different research teams test an interesting theory that’s making the rounds, and four of the groups correctly prove the idea false, while the one less cautious group incorrectly “proves” it true through some combination of error, fluke, and clever selection of data. Guess whose findings your doctor ends up reading about in the journal, and you end up hearing about on the evening news? Researchers can sometimes win attention by refuting a prominent finding, which can help to at least raise doubts about results, but in general it is far more rewarding to add a new insight or exciting-sounding twist to existing research than to retest its basic premises—after all, simply re-proving someone else’s results is unlikely to get you published, and attempting to undermine the work of respected colleagues can have ugly professional repercussions.
Lest anyone think that medical research is unique in this regard the problem appears to be pandemic in the sciences, in everything from physics to economics.
I don’t believe that the solution to this particular problem is to fund more research. Quite the opposite, I think it’s to fund less research. The incentives have got to change.
{ 71 comments… read them below or add one }
← Previous Comments
Sam, I am tired of being lied to and lied about. I am especially tired of having someone like you shit on me and call me a liar (by implication, not even having the guts to state what you mean clearly in a fucking anonymous forum, you sniveling coward) in one sentence and then tell me I should just smile and be happy about it in the next.
There are two ways to be free of bonds in this country: to be independently wealthy and to be independently poor. Through luck and fortune I am now independently poor. I am not willing to let the bullshit get so high that it blocks out the late morning sun without at least saying “Stop piling up that bullshit and blocking out the sun!”
Republicans pulled a lot of crap in their run in power in the early part of this millennium. They were crooked and cynical and stupid, and for their sins some of the lesser lights of the party were kicked out of office. But not the leaders (other than Bush, who was term-limited out) and so they deserve no forgiveness.
That put the Democrats in charge. They have been even worse than the Republicans, which was a low bar to get under.
Most of those that see the Democratic corruption and incompetence for what it is are completely unwilling to do the same for their party of choice. I am not one of them. The tea partiers can delude themselves that they are voting for the good guys all they want. But it ain’t so.
But the idea that the Democrats are anything but worse is revolting. It is to the point now where Reynolds is even disavowing any kind of knowledge in economics in order to forgive total ineptness. (Clue for the clueless, the micro- stuff is pretty good.) And now you are passing off this shit that the President didn’t say what he said the other day, as though I can’ t look up a citation. The tea partiers are delusional now, but you Democratic apparatchiks are just revoltingly incompetent at this point. Did you actually not follow Obama’s speech, or did you just assume I would be cowed by your language and wouldn’t look it up? Or were you just so lazy that you didn’t bother reading or listening to it yourself?
No, someone tries to pull this crap, they deserve contempt, not approbation. The idea that everyone should just lie there and enjoy it? Riiiiiggghhhhtttt. I am sure Barry Half-White would love that, as would Reynolds.
Even more pernicious is the idea that if someone is offended about something it dominates their life. Hardly so. This is a forum that discusses public policy, and perforce discussions of those who initiate and oversee those policies. Schuler occasionally comments about dog racing, or his kitchen remodeling, or cooking, or what have you. Other than those about movies and/or TV you won’t find me commenting on those, because that’s not why I read this site. You will find me on posts about public policy. And here there is everything to be outraged about. To NOT be outraged, or to meekly accept things, is wrong.
But that doesn’t actually rule my whole life. I am getting a break right now because my daughter is taking a very long nap, and because of a minor health issue she is having right now I am happy about that. It will do her good. So I have time to comment. Last night I had some unexpected free time too.
I have also in recent weeks been reading up on zombie attacks, commenting on dreams, taking care of a mess of kittens, cooking, cleaning, getting my teeth cleaned, following the continuing Dwight Howard fiasco (with a certain level of schadenfreude), playing with my daughter, teaching my daughter (although the last two are synonymous at her age), going on little day trips with the family, contesting an estate, fighting with lawyers, following various chess tournaments when time and energy allowed, and on and on. Some of that has made me angry, some of it has made me very happy, and most if it has just been stuff that needed to be done. (Cleaning the toilets doesn’t warrant much of a response other than I wish someone else were doing it.)
So spare me the CONCERN TROLL about my mental health, ’cause this ain’t my whole life. And try looking up some proper citations next time you want to call me a liar, fuck-face. Better still, look them up AND READ THEM first.
As I said, the sickness unto death.
Yes, Sam, because I should be happy about being called a racist by Reynolds every other Tuesday, and getting called a liar by you when you didn’t even bother to READ THE SPEECH is something else I should be happy go lucky about. You don’t even have the decency to admit that you just got caught doing it. What a pair of turds.
All I can say is I spent an entire year teaching myself the basic science underpinning global warming theory, and I’m convinced.
A year? Really? You do know then that the models are giant simulations of our atmosphere and that they depend on lots and lots of variables. If you are going to complain about models with lots of variables…there you go. They make macro-econometric models and macro simulations look like simplistic kids toys.
Icepick
I don’t always agree with you. And, sometimes don’t even understand some of the rationales for your statements. However, I think you are brutally honest, and an expressive writer, running circles around the esteemed and well-heeled author who posts here. Also, whether you do little or big things in your life, what matters is that you do something. From the list you posted, I would say that you are involved with a variety of basic, daily activities/obligations/chores/past times that I can totally relate to, including the cleaning and arguing with lawyers about estates.
Anyway, bottom line is that I take an interest reading your views, whether they are mine or not. Kudos to you!
Also, playing with your daughter is important. They grow up, and then you miss the good old days of the 2 year old monster walk, carrying them on your hip, playing games or reading to or with them.
They grow up, and then you miss the good old days of the 2 year old monster walk….
Lordy, how they grow! We miss our little baby! But she’s all grown up, and now she’s a “dinosaur rex” (her phrasing for tyranosauri) digging for treasure with her claws on our bed! Which is a surprise because most of the time she is a monkey swinging from the ceiling fans. (To DCS: That last part is a joke. Mostly.) I’m thinking she can win Ninja Warrior/Sasuke around the age of five or six at this rate.
Everyone tells you how fast they grow, but unless you’ve been through it I don’t know that you can believe it. Really, just a couple of months ago she was this tiny little baby that couldn’t even role over, and now she scampers up ladders more easily than I do. (This at the play ground – I’m careful to not leave any ladders in her range at the house.)
The most amazing part, though, is how brilliant she is. I’m not making any claims that she is more brilliant than any other child her age, as I have no way to really compare them (observations at the playground and library and such are inconclusive – plus I’m focused on my own little charmer/beastling) and wouldn’t care to if I could. (I’m not a cognitive development researcher.) But dang, it is hard, demanding work being a baby and then a toddler. I really didn’t appreciate how hard the little ones work just getting through the day, all while their bodies make tremendous investments in physical growth. Wow….
“Cheaper energy will help the economy more than the jobs created building the pipeline. ”
Oil is fungible. The profits from the pipeline go to TransCanada. We dont really benefit from the pipeline much more than any other country, other than the few hundred jobs it does create. If we are going to take the risk, we should minimize it.
Steve
In other words, you want to outsource what jobs we could get from it, including the value we could get from having it flow through our country and our refineries? (I’ll also note that the Chinese seem to be hot and heavy to do a deal with the Canadians. Perhaps they have a different feeling about the fungiblity of oil than you do.)
Also, when did you Dems become such big fans of energy dependence for the US? Surely it is better to be dependent on Canada, easily invaded by tourists in a pinch, than to be dependent on the Chavezs and oil sheiks of the world.
And Sam, I am waiting for your answers to my questions about inviolable property rights. Not that I expect any forthcoming – too busy trying to psychoanalyze me, I guess.
@Icepick
RE: Libya
The President got dragged into Libya by the Europeans. The Europeans were worried about Gaddafi destroying the oil field infrastructure similar to Saddam. If Syria had oil, Europe would have dragged the President into it.
RE: Property Rights
Eminent Domain should never be used to transfer private property to another private entity. If the property is needed for the “greater good”, it should be owned by the government. I am not familiar with particulars of the Keystone pipeline issue, but they can route around any reluctant property owners.
A new highway would be an appropriate use of Eminent Domain, but as you noted, it does affect more property indirectly. I would expect the politicians to purchase cheap land, and sell it for the highway.
RE: Bullshit
I agree. I am tired of hearing my betters spout crap about things they know nothing about. I think that Romney will be a little better than Obama, but the Republicans are not going to roll-back most of what Obama has implemented. To politicians, bipartisanship means splitting money and power 60-40%.
RE: Comment Reactions
The emotional reactions to your comments is due to your writing style. The structure creates a cadence which transmits passion, and in an effort to calm you down, condensation results.
“Proud Mary keep on burnin’”
- CCR
Icepick
You sound like a caring and receptive father. Your daughter is a lucky little girl! I agree about the baby years. Their bodies are on a fast track to get big and capable enough to explore the universe. Reading your comments had me relive some of our own parenting years when our son was a little guy on the go, 24/7. I think kids are much more intelligent than we give them credit for too.
You’re not the only unemployed guy in the country, they don’t all turn into rage-o-holics.
How many of them do you know personally? I meet with a few dozen of them every month. There’s a lot of rage out there, and you’ve got no clue how that works living in Marin County drinking expensive swill and gazing at the Bay. The rage masks the fear for some of them, for others they’re just burned out and don’t have anything else left. The lucky few have gotten old enough to start collecting SS and Medicare and don’t care enough to get outraged anymore.
I’m in a great spot, actually. I am truly independently poor. I OWN my house. No mortgage. I own my cars. My wife got a new job several months ago and we can now cover our expenses. (She’s still my wife! – divorce followed LTUE for a lot of people in my tribe. People married 30 years and they couldn’t handle the strain of financial ruin.) Life isn’t easy (mostly because we live in fear of any kind of emergency, which will wipe us out), but life is much improved. It would be good if I could find a part-time job with exactly the right hours, but it is more important now that I be home to engage with the little one. (Besides, my experience and that of all the LTUEs I know is that employers just don’t want to hire LTUEs – it isn’t worth the risk from their perspective. And I don’t blame them.)
But you wouldn’t understand these concerns, and you clearly don’t understand why I find the incompetence of your party and Drew’s party so maddening. People are being wasted in their generations, en mass, and you think it is a fun little game for your bullshit cries of “Racist!” and “Homophobe!” I’ll at least give Drew this tiny bit of credit: He does seem to understand that there’s a heep-big load of misery out here, and not just for the hair dresser who can’t marry his latest boyfriend.
But please, tell me something else about something you don’t know anything about. It will be just like when you were explaining to me about how insurance companies work, having no idea how any of that worked either. Paying a premium gives you no insight into the business workings. Having had to look for work 30 years ago doesn’t tell you a goddamned thing about being one of millions of people out of work for YEARS in this day and age. You’ve got no clue, and you are too stupid to listen to what those with experience have to tell you.
As for food stamps – my discontent on that front is obviously above your comprehension level. I’ve made it clear that I don’t blame people for committing fraud to get SSDI. Some of them are just crooks (see Quartavious Davis), but most of them are simply desperate to keep body and soul together. (Some, including two friends, shockingly are actually disabled!) There’s probably more fraud with food stamps, but again, people are desperate. (See above regarding all the UE people I meet every month.) My outrage is with a President and a party that thinks this is an acceptable result for the economy. (“The private sector is doing fine.” “Recovery summer!” et cetera.) These programs are soul crushing for free people, and they erode self-worth*. This is not acceptable, and the fact that people like you think it is as long as your team wins is outrageous. (Clue for the clueless: Thus the outrage!)
This President has no plan to get us out of this mess, and seems to want to keep us in it. Romney has a few good ideas (mostly of a de minimis nature), but mostly that is because he is looking to help the investor class. I haven’t heard much from him that makes me believe he is really going to push for meaningful tax reform, regulatory reform, entitlement reform, healthcare reform, et cetera. Mostly I am hearing campaign pablum, not any real commitment. If he were serious about these issues he would be pushing his own party’s agenda hard, so as to force the hands of people like Boehner and McConnell. He isn’t doing that. He’s playing to win for winning’s sake. A pox on his house too.
As for what I believe in – I’ve covered that before. Not that you would bother to read it. Not that you wouldn’t simply call me a racist again. You only believe in spending as much as possible to make up for your guilt about being a dick to Bob-in-a-Bag all those years ago. And you’ve got no clue what it is like to live amongst the ruin of this economy.
* I will note that every month when I meet with the other LTUEs and UEs, the topic of food banks and private charities come up. Lots of praise for these groups. Pretty much nothing but contempt for the government programs even when needed and utilized. This is because the government programs treat people like dirt. That is mostly a state-level failing blameable on the Republicans running this state, but I don’t hear anyone making great noises about the federal government either.
I’m glad I’m not the only one who sees it that way. Meanwhile, both the uproar in Mali and the Russians’ and the Chinese reactions to Syria are foreseeable consequences of the Libyan intervention.
I think that Romney will be a little better than Obama….
I agree, and that is a pathetically low bar to clear.
And great, I can calm down and be a condensate! (Also: Subtle!)
Hey, I already get some benefit from that – condensates, I mean. I am a petrochemical mogulhettino! I’ve got some mineral rights from the old family homestead back in the mountains. I get about $30 a year off that, and only have to pay about $2.50 back in taxes! Woohoo, score one for petro-riches! Not that it makes up for the extra time I spend filing out tax forms. Hey, I wonder if I’ll get more if they start fracking the Hell out of that old mountain. Maybe I should invest that extra $27.50 in a lobbyist….
The President should have let the Euros do their own damned dirty work. Isn’t that why the French have a Foreign Legion?
Icepick
Being a ‘candidate’ is like being on an obstacle course, swerving around the posts and traps in the road, in order to make it to the end of the course.
When Romney (a political novice) ran for the Senate in the 90’s he talked more about his policy agenda, which was set upon and destroyed by Kennedy and the dems. He said he has learned not to be too open too early about these things. It’s the same reasoning he’s giving about his tax returns, as the dems are like locusts, ready to swarm on any tidbit, distort it, expand on it, anything to divert attention away from a poor record and discredit their opponents. Character assassination is what their major objective is, not having an honest engagement of relevant issues. Basically, Romney is playing the game that has been set in cement by previous contenders.
I do think, though, that Romney will show more discernment, better business judgement and courage to make appropriate changes to amp up the economy. I think he will come up with a better replacement HC reform, loosen up inhibiting business/energy regulations, open up energy exploration and production, tinker with tax reform, probably adding fees and closing loopholes rather than raise taxes, per se.
IMO, just getting out of the Obama anti-business syndrome will be a healthy start (like airing out a room), causing business to breathe a little easier, which will be reflected in better prospects for job creation.
One other aspect about Romney is how many really smart people he has around him. He seems to be able to cull and collaborate with brains, not ideologues, in any industry. Even this Rob Portman, who has been bantered around as a possible VP pick, has spiked my interest. I heard him being interviewed today, and was impressed with his sharp and concise answers. He also has people like Jindal (who has experience in health care), Ayotte, Rick Snyder (who has positively impacted MI), Sandoval, Christie, McDonnell, Sean Duffy and Scott Walker in WI, Paul Ryan — the list of seriously competent R governors, senators and even some House members who have proven their mettle in their various states, much more so than most dems. I think the democratic party has gotten old, corrupt and bereft of innovative ideas. They are the same-old, same-old party, who spout muscle in being socially cool/open, plying additional votes by divisive politics, promising more entitlement programs, and expanding government dependency. For me there is no question, at the moment, in which party to put one’s faith in when the goal is to pull out of all this economic malaise we’re in.
@jan
… Romney (a political novice) …
Romney = CEO
CEO = hustler (operator, player, insider, scammer, etc.)
politician = hustler
Romney = CEO = hustler = politician
“Don’t bogart that joint, my friend”
tastybits
I think politics is a completely different game from being a CEO. There are simply more moving parts in politics to be aware of and counter.
Also, from accounts I’ve read about Romney (from people who worked with him) his big thing was practicing ‘integrity’ in business matters, expecting it to be used by his employees as well.
This trait kind of jives with his adherance to religious values, as well as numerous anecdotal stories that have floated around about below the radar acts of kindness.
@Michael Reynolds
In fact, I have a brand-new conspiracy theory on Romney’s taxes. Here it is: He’s hiding how much he made from the Mormon church. He owes them 10%, and as a church official he’s undoubtedly been involved in muscling other members for money over the years. So what if he’d been lying to his own church?
Yeah: no shred of evidence. But it’s the kind of thing that evil minds will contemplate when denied the facts.
It may not apply to Romney, but I do not find it hard to believe. The number of people who will willingly part with their money is small.
“Never get between a mother fu*@er and his money.”
– TastyBits
Jan, Romney might get elected playing that game, but if he doesn’t articulate a clear vision of what he wants to accomplish he will not accomplish anything of substance. Big reform requires the buy-in (ugh – jargon) of lots of people, starting with the buy-in of a lot of voters, so that they can push their reps into actually acting out of something other than simple partisan gain. If Romney doesn’t push hard for that NOW he won’t have the political muscle to do anything important LATER.
Obama did one thing right, and that was he pushed the idea of HC reform when he was running – it created space in which to operate. It wasn’t as important as the “Hope and Change” crap that was his tent-pole, but it was something that was there.
When was the last time I heard Romney or one of his ads make a sustained case for tax reform, as opposed to dropping in a bullet point that taxes should be reformed? One is a consistent and powerful message for change, the other an electioneering strategy. And that is one of several things we need done. We need leadership, not gamesmanship.
You may wonder how he can push several things at once. The idea is that when he speaks about a topic he needs to focus on THAT TOPIC ALONE. Make it all about the economy. Explain that many things are broken. And then focus on ONE THING AT A TIME. A drib of this and a drab of that all the time will lack any sort of punch. But it does allow for shifting electioneer tactics. Big deal if he can’t do anything once he gets there….
@jan
A hustler is one who tries to change your understanding of reality. This is not necessarily illegal, unethical, or immoral, but it can be. We are all hustling, but some of us are a lot better. You can substitute illusionist if you prefer.
Romney at one time was a “novice politician”, but that time has long passed. He has been in several political campaigns, and he was the Governor of Massachusetts. If he is still a novice, he should be sitting at the kids table.
Trying to convince someone that he is a novice politician is a hustler move. He is trying to create the illusion that he is Andy Griffith and The Beaver.
As to his personal character, great, but I do not want a nice guy leading the country. I want someone who is going to leave me alone, and who has a spine and a set of balls. I want a President who will look Putin in the eye, and tell him to piss off. Putin has turned out two Presidents. Bush “looked into his soul” and fell in love. I expected Bush to start calling Putin “Daddy”. Obama got bitch slapped, and I have no doubt he was on his knees.
Mitt Romney is a big government Republican. Much of the structure President Obama has put in place will remain. It will be changed to achieve Romney and big government Republican goals. “Meet the new boss, same as the old boss.”
Tastybits
I’m the one who called Romney a novice. That was not something I took from his own self-description. Perhaps he isn’t. But, he seems ‘nicer’ than Obama on the stump. However, I’ve never met the man. All I know is what I hear him say, read about him, the comments of others about him, the company he keeps and consults with, and it all seems more experienced, smarter and higher minded than his opponent.
I like Romney’s business brain, and so many of our problems seem fiscally oriented. If he manages to become POTUS, and the R’s take the congress I think there is a good chance of more reform taking place. A big reservation I have, though, are the current republican leaders in Congress — Boehner and McConnell — who I am not impressed with, at all! They seem stilted and establishment-oriented.
Icepick
From what I hear, Romney’s policies will be honed in on once he gets through the Convention process and beyond. He is in a transitional stage at the moment. He doesn’t have the bully pulpit of the encumbancy, and really is only the presumptive nominee until the August convention. After that, he can use all his GE funding, and be the legitimate voicebox of the opposition. But, you, like everyone else, are looking for ’specifics,’ which is a valid request. That was also a catchy phrase about ‘needing leadership versus gamemanship.’
I’m tired…good night.
Jan, I am looking for broad, bold strokes. Details will have to be worked out later, but if he made the point, over and over again, that he believes in broad-based tax reform (I’m picking this topic as an example) that achieves X, Y and Z, then he is creating space in which to act later. But it has to be a consistent message, and he and his surrogates must hammer away at it all the time. He’s not spending much on ads now? Okay. But he is putting ads out there, he has a web-portal. He’s giving stump speeches all the time, he and his surrogates appear on TV a lot, et cetera. He’s not doing that.
← Previous Comments