What’s a credible source?

Note: This post should in no way, shape, or form be construed as a criticism of Pundita, one of my favorite bloggers and correspondents. This post are my musings on the path I took after reading one of her many fine posts.


I tend to follow links. You can learn a lot by doing that and find yourself in places you might not otherwise discover. I also read footnotes. I can think of at least one book that can be read for the footnotes alone: Sir Richard Francis Burton’s remarkable translation of the Thousand Nights and a Night.

In a recent post, “Once upon a midnight dreary quoth the Raven, ‘The vodka is good but the meat is rotten.’”, Pundita linked to an article from The Epoch Times. The article purports to be a transcript of a speech given by Chinese Minister of Defense Chi Haotian. I found the speech completely horrifying. A mixture of mythology, fantasy, poppycock, racism, nationalism, strategic planning, and monstrosity. Here’s a snippet:

Comrade Mao Zedong taught us that we must have a resolute and correct political orientation. What is our key, correct orientation? It is to solve the issue of America.

This appears to be shocking, but the logic is actually very simple.

Comrade He Xin put forward a very fundamental judgment that is very reasonable. He asserted in his report to the Party Central Committee: The renaissance of China is in fundamental conflict with the western strategic interest, and therefore will inevitably be obstructed by the western countries doing everything they can. So, only by breaking the blockade formed by the western countries headed by the United States can China grow and move towards the world!

Would the United States allow us to go out to gain new living space? First, if the United States is firm in blocking us, it is hard for us to do anything significant to Taiwan and some other countries! Second, even if we could snatch some land from Taiwan, Vietnam, India, or even Japan, how much more living space can we get? Very trivial! Only countries like the United States, Canada and Australia have the vast land to serve our need for mass colonization.

[…]

To resolve the issue of America we must be able to transcend conventions and restrictions. In history, when a country defeated another country or occupied another country, it could not kill all the people in the conquered land, because back then you could not kill people effectively with sabers or long spears, or even with rifles or machine guns. Therefore, it was impossible to gain a stretch of land without keeping the people on that land. However, if we conquered America in this fashion, we would not be able to make many people migrate there.

Only by using special means to “clean up” America will we be able to lead the Chinese people there. This is the only choice left for us. This is not a matter of whether we are willing to do it or not. What kind of special means is there available for us to “clean up” America? Conventional weapons such as fighters, canons, missiles and battleships won’t do; neither will highly destructive weapons such as nuclear weapons. We are not as foolish as to want to perish together with America by using nuclear weapons, despite the fact that we have been exclaiming that we will have the Taiwan issue resolved at whatever cost. Only by using non-destructive weapons that can kill many people will we be able to reserve America for ourselves. There has been rapid development of modern biological technology, and new bio weapons have been invented one after another. Of course we have not been idle; in the past years we have seized the opportunity to master weapons of this kind. We are capable of achieving our purpose of “cleaning up” America all of a sudden. When Comrade Xiaoping was still with us, the Party Central Committee had the perspicacity to make the right decision not to develop aircraft carrier groups and focus instead on developing lethal weapons that can eliminate mass populations of the enemy country.

You may read the whole thing, for what it is worth.

My initial reaction to the article was anger. My second reaction was curiosity. Could this possibly be true? I read everything I could put my hands on on Chi Haotian particularly transcripts of speeches. I checked the source of the speech. I read everything I could find about the source of the translation and the source of the original.

My Mandarin isn’t good enough to read the Chinese language version of the speech (or any of the general’s speeches). It’s pretty difficult to determine authoricity based on translations.

The Epoch Times is a questionable source. The Times is an anti-Chinese Communist Party newspaper and has been characterized as having a pro-Falun Gong bias.

The source of the Chinese language version of the speech was Boxun.com. Boxun (“Abundant News”) is a US-based Chinese language dissident news site. Here’s what EastSouthWestNorth has to say about it (from the unlinkable South China Morning Post):

Boxun’s founder, who goes by the pseudonym of Wei Shi and describes himself as a businessman, said from the US that he could not verify the web-posted stories from Qinghai that Boxun had run. Nor could he vouch for the alleged whistleblower’s credentials. All Boxun’s non-secondary source reports are posted anonymously. But he said he hoped that by putting the stories in the public domain, somebody would prove them true or false.

There is no second independent source for the speech.

So let’s recap. We have a speech that was allegedly given by an actual Chinese official. It’s not possible to establish the authoricity of the speech by textual analysis. There’s only a single source for the speech and the publishers of both the English language and Chinese versions of the speech may have axes to grind. To further complicate things it’s not possible to establish the reliability of several of the critics of these sources e.g. Wikipedia and EastSouthWestNorth. The final critic, the South China Morning News, is a legitimate newspaper and, presumably, can be considered to have at least a little reliability. Unfortunately, the quotation attributed to the SCMP isn’t actually linked and can’t be verified.

It seems reasonable to treat all of this with at least a grain of salt.

That brought me to a larger question: how do you evaluate the credibility of sources, generally? I fired off a number of emails to relatives and friends who do this sort of thing for a living: reference librarians, journalists, etc.

The sad truth is that there is no gold standard for establishing the credibility of a source. Journalism, particularly, is seat-of-the-pants on the issue. I have, however, compiled quite a nice little library of tips for establishing the credibility of sources on the Internet and generally. Here are a few that you might find useful:

University of Vermont bibliography on evaluating web information A great site with lots of resources (online and otherwise) for evaluating credibility. Forms, checklists, guidelines, etc.
University of California, Berkeley Library Guide to Evaluating web resources An excellent, detailed, highly explanatory set of tips for evaluation.
Cornell University Library Guide to Critically Analyzing Information Sources A fantastic, detailed authoritative guide to the subject.

It appears to me that it should be possible to at least make a start at a formal tool for measuring the credibility of sources using the approaches suggested in these resources and coming up with a quantitative measure of reliability. I’m thinking of something using Likert scales, that would save prior results in a database, and so on. Ideas?

UPDATE: One thing I’d meant to mention but somehow lost track of in formulating the post is that simply because a source isn’t credible or reliable doesn’t necessarily mean it’s wrong. That’s called the Genetic Fallacy. There are other considerations for measuring the truth or falsehood of a proposition than the source of the assertion.

18 comments… add one
  • Barnabus Link

    Sounds bogus to me.

    On the general idea of using biological weapons to “clear” a country, it may sound easy but it is quite difficult. You need to engineer something that only attacks “them” and not “us.” Simply spreading a lethal virus around will almost certainly result in an outbreak in your own community. Designing something new (or using say smallpox), where you can immunize your population and then unleash it on a susceptible population has the result of leaving your calling card. An attack of that sort on the U.S. would invite a nuclear response.

  • Precisely, Barnabus. At any given time I’m pretty sure we have enough submarines within striking distance of China and sufficient nuclear-weaponry to render the population centers of China uninhabited and uninhabitable. And the Chinese themselves have done enough of a job on their own countryside that life there after nuclear holocaust would no doubt be brutish and short. A biological weapon that could eliminate the population of the United States, spread to the submarines, not be detected in the meantime, and not spread to the Chinese themselves would be a formidable weapon, indeed.

  • eswn Link

    on the question of establishing what is true or not with respect to epoch times and its sister pubications, pls read:
    http://www.zonaeuropa.com/20050710_1.htm

    on the question of whether it is ‘rational’ chinese general to think this way, who knows? i’m thinking about the movie Dr. Strangelove.

  • Another sharp post.

    When I write for ( or read) academic listservs or journnals I abide by scholarly standards of credibility which are pretty high and I tend to qualify my remarks far more than I would on my blog. I too read the footnotes/endnotes and sometimes I see things that aren’t quite right or conversely, they lead me to very interesting material.

    In the old, old, old days, an eminent professor who had earned his spurs by becoming an unimpeachable expert in their field would be allowed by the convention of the time, to publish without using footnotes. Not too many people rate that kind of blank check today IMHO. When I read ” known” figures in the MSM or the blogosphere I tend to view them in three categories:

    a) Highly reliable ex. The Economist

    b) Useful as a raw file, interesting but often dubious/alarmist/highly speculative ex. Newsmax, DEbkaFile

    c) Junk ex. Democratic Underground, Pat Robertson’s outlets and media vehicles of known crackpots

  • Mark, your a,b,c categories are useful but I think there are some others as well. For example, another category is “highly credible in some areas; not particularly credible in other areas”. That’s the direction that it seems to me that mainstream journalism is headed in.

  • “highly credible in some areas; not particularly credible in other areas”.

    That strikes me as particularly appropriate category for pundity and blogging.

    We all get out of our areas of relative expertise sometimes – when we generate horizontal insights from doing that it’s great, when we end up just blowing smoke, not so much.

  • “That brought me to a larger question: how do you evaluate the credibility of sources, generally?”

    I’m not sure how to answer that comprehensively, since I do it so automatically, for the most part. Certainly anything questionable from a single source I generally would not go with. Pretty much the rule of thumb is that the more unlikely something sounds, the more credible sources I need to consider it worth passing on. Some sites enjoy picking up wild rumors and running with stuff from, say, Debka, or any number of sites with immense political biases; I don’t, though.

  • "liz" Link

    re using a Lifert scale for quantifiable analysis — see my guest post at Pundita —
    http://pundita.blogspot.com/2005/08/ignorance-knowledge-and-three-strikes.html

    enjoy checking in over here … in my copiuos free time.

  • Good credible sources are very hard to find on the internet.

  • The Chinese are known to speak cryptically and in this sense a “biological weapon” could be referring to a computer VIRUS. This would make sense if the reports of them attacking our power grid were accurate. Theoretically you could wipe out a substantial number of Americans if our power grid were disabled. Everything we do from growing food to transporting and refrigerating it relies on it, not to mention the pumping of gas for transportation, the control of inmates at penitentiaries, the ability to transmit data quickly and effectively (communications).
    A major strike like that would leave most Americans blind and disorganized, unable to mount a coordinated defensive effort against a half billion person army should they decide to storm our mainland as was claimed in the supposed speech. I would say treat it like a boy scout and be prepared.

  • Phil Link

    I want to say something about this speech. I’m chinese and I watched the ENTIRE video of this speech. I saw this back in 2006 on youtube. Yes, it was on youtube but for a very short time. Chinese in China cannot access youtube but i’m in north america.

    Ever since, I have not been able to hunt this speech down. I’ve tried for years looking for it. It’s gone. I remember his speech because it is purely Hitlerian and horrifying. Now, on youtube you get translations but not the actual speech.

    The average Chinese have no clue what the communist party is setting the Chinese people up for (WW3).

    I’ll keep looking for it but I really doubt i’ll find it.

  • fiveyearslater Link

    Five years ago I saw a UFO.. i’ve been searching for it for years.. I found an old gold mine in a secret underground passageway behind a toilet in the train station.. I opened it accidentally and it was the remains of a gold mine operation.. i only saw the craft briefly as it caused a small hole in the ceiling open and it flew up into the sky with what i assume was a ton of earth’s gold.. i climbed out eventually.. the next day all evidence of what i saw was gone….. you believe me right?

  • 2Cents Link

    Why don’t you just fly to China…. and ASK HIM?!!!

  • 2Cents Link

    Phil: “I want to say something about this speech. I’m chinese and I watched the ENTIRE video of this speech. I saw this back in 2006 on youtube. Yes, it was on youtube but for a very short time. Chinese in China cannot access youtube but i’m in north america.”

    A Chinese guy named “phil” that saw an UFUTUBEO. Well, Phil…. if a Video ever did grace youtube… then there’s a chance they still have a copy of it on a back-up server somewhere… try contacting youtube.

    And if I were truly into finding the Truth… along with a trip to China… (I’m brazen like that… I would just take a copy of the original and go hunt the old man down… or at least walk around China close to where he might be and ask folks) I’d be looking for you… to see what your true nationality, affiliations, intentions and agenda might be. If we could nail this speech down to a Russian guy, say…. might be enlightening!!!! If I were a Rusky… I’d try to wage a war between my two greatest enemies!!! While I sat back and drank coffee!!!

    But then… we also have the Hidden Hand NWO Nazi’s working behind the scenes… “Weaving Spiders” they are…. except at Bohemian Grove!!!

Leave a Comment