Google Behaving Badly?

Jack Lewis notes Google apparently disregarding robots.txt – in fact using robots.txt to actively hunt out information. (hat tip: Little Green Footballs) I question the conclusions so rapidly jumped to, but the behavior is interesting and worth investigation. Just out of curiosity, I set up a honey pot robots.txt on my home domain, so see what pops up. I’ll let you know.

UPDATE (2/18): So far, Google has been honoring the file.

1 comment

Surprise: I’m a Centrist

I just took yet another test that tries to place people in the political spectrum. As one might expect, I’m a Centrist—very slightly socially liberal, very slightly economically conservative.

3 comments

Authors I’ve read a lot of

I’ve seen several blogs whose proprietors have mentioned authors whose books they’ve read ten (or more) of. Here are a few of mine. I’ll add to the list over time since I tend to read all of the works of authors I discover that I like. In no particular order:

  • L. Frank Baum
  • Charles Dickens
  • Jules Verne
  • Robert Heinlein
  • Fyodor Dostoyevsky
  • Victor Hugo
  • Isaac Asimov
  • Dorothy Sayers
  • G. K. Chesterton
  • Edgar Rice Burroughs
  • Piers Anthony
  • Steven Jay Gould
  • Mark Twain (Sam Clemens)
  • Agatha Christie
  • Louis L’Amour
  • P. G. Wodehouse
  • Erma Bombeck
  • O. Henry (collections of short stories)
  • C. S. Lewis
  • H. P. Lovecraft (mostly collections of short stories)
  • H. Rider Haggard
  • Rudyard Kipling
  • Walter Scott
  • Robert Louis Stevenson

Lots and lots more especially science fiction, fantasy, and mystery writers.

3 comments

Catching my eye: morning A through Z

Here’s what’s caught my eye this morning:

  • The Skeptical Optimist comments on the “National Security Disinvestment” (hat tip: Econopundit).
  • I missed it yesterday since real life prevented me from posting a CME feature: Go Fug Yourself
    has post-Grammy coverage, mocking every outfit in sight. Just keep scrolling.
  • Juan Cole of Informed Comment
    has a good post on the ongoing political machinations in Iraq (along with the daily casualty report). I
    do have a question, though. Does it make a difference whether he opens the post (as he did):

    Against the backdrop of political horse trading, the violence continued in Iraq on Tuesday.

    or

    Against the backdrop of ongoing violence, political horse-trading continued in Iraq on Tuesday.

    Which characterizes his post better?

  • Phil Carter of Intel Dump
    reports on creative destruction in the U. S. Army force structure. I’m wondering if this doesn’t
    also presage a transfer of forces from Germany and South Korea.
  • James Joyner of Outside the Beltway has a round-up of
    media reporting on the explosion near an Iranian nuclear plant this morning.
  • The Carnival of Education is up on The Education Wonks.
  • There’s an amusing post from The Poor Man
    on a crisis in the right blogosphere.

That’s the lot.

0 comments

Glenn’s retirement plan: live forever

Glenn Reynold’s recent Tech Central Station article contains his proposal for real social security reform:

While people are debating Social Security reform, I want to make a more ambitious proposal: Let’s get rid of Social Security. And let’s do it by making sure that people will live so long that the idea of “retirement” will become largely meaningless.

He continues by conflating Social Security retirement, Medicare, and Medicaid, but that’s a subject for a different post. I think that his post makes a good companion piece with this one:

BOSTON (AP) – A new study from the Harvard Medical School suggests that older doctors generally know less, provide lower quality care and expose patients to greater risks than physicians recently out of medical school.

The Harvard team analyzed nearly 40 years of research into factors that shape health care quality and found older doctors were less likely to know or follow current treatment standards on everything from surgery to treating children’s fevers.

One study found that heart attack patients were 10 percent more likely to die in the care of a doctor 20 years out of medical school, compared with a recent graduate.

Now that’s a prospect that’s full of hope: a doctor who lives forever and does a worse job the longer he lives.

To go on I recognize that it’s anecdotal but anybody in any technical field will tell you that it’s progressively harder to hold onto your job with every year past 45 and that much harder to get a new job in your field as well. And technology is a river you just can’t step into twice: once completely out of a field substantial re-education is needed to regain your place and the prejudices against you are nearly insurmountable.

Glenn, is there any reason to believe that older lawyers are more capable or more hireable than young lawyers (unlike all other skilled professions)?

Living longer is less likely to mean that you’ll have that much more time to enjoy life as you’ll be forced to work longer (presumably forever as well) in decreasingly skilled and responsible positions.

I suppose there’s nothing like ten years of unremitting pain as a result of an untreatable chronic pain condition that prevents you from leading a normal active life to make the idea of eternal life sound less appealing but I find the idea excruciatingly dystopic. So here are my predictions for the social developments that eternal life would bring:

  1. If eternal life became a real possibility it would be construed as a right. Remember “life, liberty, and the pursuit of happiness”? I know that’s not in the Constitution but it’s part of the fabric of our social thought. The rich will not be allowed to separate themselves from the common lot of the rest of us.
  2. If life expectancy goes up dramatically, scarcity (particularly of land) will re-emerge as a real issue. Consequently prices of scarce items and particularly housing will rise dramatically. Nearly everyone will be compelled to work all of their lives (and, as I suggested above, in decreasingly skilled and responsible positions). These are words we’d better all learn: “Would you like fries with that?”
  3. Prejudices against the old will endure long past their social utility and the ability of technology to prolong life.
6 comments

What Happens and What Should Happen

Yesterday, Dave had an extensive post on Gammon’s Law. He referenced an older post on Gammon’s Law that made some interesting observations about why government programs fail.

Why do so many government programs fail? We’ve seen it time and time again. A need is identified, a program is formulated and put into place, everything starts out well enough, and then, perhaps over time, something happens. The program doesn’t achieve its goals. Or the amount of resources needed for it to achieve its goals are vastly more than expected.

We’ve seen this in Social Security, Medicare, the Great Society programs, and the public school system. Is it waste, fraud, and abuse (those favorite whipping-boys of legislators)? Welfare cheats? Incompetence? Just needs a little fine tuning? We’re not spending enough (no matter how much we seem to be spending)?

[snip]

Why does this [rising inputs, such as increased funding and increased staff, leading to decreased outputs, however measured] happen? Does it have to happen? The short answer is yes, it does. Unfortunately for those who contemplate grand solutions to the geniune problems in the world. In a modern society the implementation of the kinds of plans we’re talking about here requires a bureaucracy. And Gammon’s Law is an intrinsic feature of bureaucracies.

[snip]

Note that bureaucracies are not about outputs. They are about process. And it’s been known since Weber’s time that bureaucracies take on lives of their own. They’re like one-celled organisms. Their only objective is survival. And survival in a bureaucracy is not about output but about process.

There’s a kind of entropy in a bureaucracy: it becomes more and more organized and less and less work gets done. There are fewer outputs.

[snip]

There are only two known organizing principles in modern societies: bureaucracy and the unpredictable large scale group behaviors of complex systems known as emergent phenomena. Reliance on emergent phenomena to solve the great problems requires an enormous amount of faith and hope.

Why then, if we know that bureaucratic programs will always grow in cost and invasiveness while shrinking in positive effect, do we continue to look for centralized – thus bureaucratic – solutions to societal problems? The key is found in Dave’s earlier post on emergent phenomena: “Quite a few of the things that are absolutely the most important to us are emergent phenomena: life, consciousness, history, the Market (Adam Smith’s Invisible Hand), and the workings of a free and democratic society are all emergent phenomena and, as such, are highly distasteful to those who look for a simple, tidy, elegant, and orderly universe.”

There are in the world more people who believe that the world can indeed be “simple, tidy, elegant, and orderly”, that we can exactly predict the future, and that we can control conditions to bring about an ideal future. The fact that every attempt to do so has resulted in failure – sometimes catastrophic failure – does not dissuade such a True Believer. It’s a case of the difference between what happens and what “should” happen. If only we spent more money on education, or forced people to stop black market work and only contribute their labor to the State, or prayed harder, everything would work out just fine.

But these people are not insane: there are places where centralized control works quite well. For example, a family is generally a centralized entity, with critical decisions taken by the parents (or, in some cases, by only one of the parents). Aircraft crews and ship crews, some types of military units and some kinds of small businesses work quite well on a centralized command and response model.

The critical factors that determine whether a centralized command and response model can work are the number of components in the system, the reliability of each system component, the number of rule state changes in the system, and the latency between command and response. Each of these critical factors acts on a system in a different fashion to shape its response to stimulus.

As the number of components in a system grows, the amount of effort needed to simply monitor the components grows as well. The effect is not linear, but geometric. In other words, doubling the number of components will more than double the amount of effort necessary to control them. The reason for this is simply that the controllers must also be controlled. As a result, adding components adds both monitors, and the controllers of the monitors, and eventually the supervisors of the controllers of the monitors, and so on. This is what resulted in the decreased safety margins in US nuclear plants after the TMI incident: fear of accidents resulted in an increase in safety systems, and now the safety systems have safety systems, which themselves have safety systems, which themselves have safety systems. If the safety of the safety of the safety fails, the plant is still shut down or degraded. This doesn’t make the plant safer, after a point, just less reliable.

The reliability of each component within the system has a different impact. As the reliability drops from theoretically perfect (never, in practice, attained), several different effects emerge. These include inability to trust inputs or outputs, faulty analysis causing improper response to stimulus, and degraded response times. For example, take an assembly line which has an automated parts counter. When the parts inventory gets to a certain level, new parts are ordered. But if the parts counter (human or machine) is faulty, the system can incur excessive costs (from ordering too quickly) or suffer degraded output from not having parts on hand (from not ordering quickly enough). At best, the system will degrade; at worst it will fail. To prevent the degradation, critical components are given some kind of monitoring in most systems, which of course increases the number of components and the possibility for failure.

Rule state changes cause yet another kind of challenge for command and response systems. Simply put, it takes a non-zero amount of effort to evaluate a rule. As the rules grow in number and/or complexity, the amount of time needed to evaluate the rules grows. And depending on the system’s inherent ability to resolve rules contradictions or situations outside of the rules, any fault might not be self-correcting. This is why computer software is buggy: it simply is not possible in any reasonably-complex piece of software to predict all of the possible program states within anything approaching a reasonable time and cost budget. For really critical systems, like Space Shuttle guidance computers, the software is incredibly expensive, because the time and effort must be expended to make them nearly defect-free, or people will die. Consumer operating systems, not so much. And of course, as the number and complexity of features in software grows, the number of possible states also grows, and at a faster pace.

The last critical factor in controlling a complex system is the time required to notice a stimulus, analyze it, determine an appropriate response, and command the response. If the latency in the system is sufficiently high, the response may be too little, too late. One really good example of this kind of failure is shown by the OODA loop, and what happens when a military gets inside the enemy’s decision cycle. Take a look, for example, at how the US military in Iraq dismembered the Republican Guards around Baghdad. The IRGC fought, but ineffectively. This is because we deprived them of sensors (and in some cases commanders), degraded and channeled their ability to respond, and moved very quickly. As a result, IRGC units would attempt to defend against US units that had already moved beyond the point where the defense was supposed to be set up. This resulted in the IRGC command system rapidly losing the ability to even locate its own forces, never mind do anything useful with them.

Combine these effects, and it’s easy to see where centralized systems fail: where the number of independent components to be controlled is large, the components are unreliable or inconsistent, rules are complex and numerous (and possibly contradictory, and frequently over-specialized), and the time between a stimulus and a response is large. In such circumstances, a bureaucratic system will either fail or seriously degrade.

For example, let’s look at public education. There are some 94000 public schools, 3 million teachers and 47 million students in this system. The number of components is large.

There are 50 states (plus DC) and some 17000 school districts or other educational agencies contributing rules, often conflicting. The rules are complex, numerous, contradictory, and frequently very specialized.

Not only do individuals in the system, and in the regulatory bodies overseeing the system, frequently act arbitrarily or unpredictably, but the schools have infrastructure problems and weather can cause the schedule to slip and sometimes there are media circuses and students transfer between schools and teachers quit in the middle of the year for personal reasons and the state of knowledge is constantly changing…. The components of the system are not reliable and predictable.

The lag time in public education (any education, actually) is stunning: it can take years or decades for the results of system changes to become known. Sometimes, there is no way to determine if the system changed because of commanded changes or because other circumstances forced changes that simply weren’t accounted for. Even over the course of a school year, the time between teaching, testing, and the teacher absorbing the test results and accordingly adjusting their techniques (one hopes) is months long – and that’s at the lowest level of the system.

In other words, it is simply not mathematically possible for a centralized public school system to effectively produce output (educated students) consistently over time. There are simply too many possible points of failure. Sure, there will be some successes – even a great number of them in absolute terms. But overall, the system’s efficiency is terribly low.

It should be kept in mind, though, that the alternative – using emergent behaviors – is not suitable in every case, either (a trap libertarians often fall into). Emergent systems can fall apart if the components of the system don’t agree on the rules of behavior at interfaces. (That is how civil wars and computer viruses happen.) Emergent systems can fail if they are faced with a challenge that can be handled by the system globally, but not by the individual elements of the system that are in direct contact with the challenge. (That is why we have national armies instead of just militias, and why we have health insurance.) Emergent systems can also fail because of insufficient damping, where a behavior once started causes larger and larger oscillations until the system can no longer function. (I sometimes fear that political debate in the US, particularly over executive appointments to office, is in such a series of oscillations, and that eventually it will simply be impossible to get any judge or cabinet member confirmed.)

But emergent systems do handle a great many problems very well, particularly the very large and very complex problems that the bureaucrats and statists tend to want to control centrally. And this is anathema to the bureaucrats and statists: in an emergent system, if something goes wrong, it is someone’s fault, and that person can be identified and punished. Bureaucracy and statism are all about avoiding personal responsibility by shifting the responsibility to “the system” or “the process” or “anyone but me”.

0 comments

Increasing supply and reducing demand won’t help health care

I can see that neither Matthew Yglesias nor Steve Verdon is reading The Glittering Eye. If they had, they’d have read my recent post on Gammon’s Law and they’d know why neither increasing the supply of health care providers nor decreasing the demand for health care will have much effect on the rising cost of health care in the United States.

2 comments

Iraqi health care: another example of American soft power

Have you noticed that a disproportionate number of Iraqi bloggers seem to be medical doctors, dentists, medical students, and dental students? I have. I attributed it to attainment of higher educational levels, higher disposable income, and possible greater access to the Internet (through affiliation with universities and hospitals). These may all be true but it doesn’t seem to be the whole story.

When I was preparing the Carnival of the Liberated this morning, I stumbled across this post from Truth Teller of A Citizen of Mosul who is a medical doctor (I probably should have included it in the Carnival):

I have told you before that I am a medical doctor, I graduated from Iraqi Medical College in 1972, the study in the medical college is six years. I have my post graduate study here in Iraq too.

The study of medicine is in English language for both pre and post graduate. After graduation, we use to have 12 months residency as junior doctor in one of the general hospital, we have to rotate in all branches of medicine to practice clinical application of medicine, now they change the rules and the training period extend to 24 months.

If that’s true all over Iraq (I suspect it is) it suggests another reason for the number of bloggers in Iraq who are health care professionals: they read and write English so they’re able to have an English-language blog.

People commonly point to industries like aerospace and computer technology (for which English is the universal language) as proliferators of American soft power. But health care may be yet another area in which the English language unites us with countries in which English is not the official language.

UPDATED: Submitted to the Beltway Traffic Jam.

2 comments

Carnival of the Liberated

The Carnival of the Liberated, a sampler of some of the best posts from Iraqi bloggers, is now available on Dean’s World. This week we’ve got reactions to the election results, comments on assassination attempts, history lessons on the Imam Ali, and setting us straight.

0 comments

Making plans, health care costs, and bureaucracies

With the Becker-Posner Blog’s observations on health care that I linked to this morning and the recent related posts from Coyote Blog and Different River both of which suggest that the rise in health care costs can be explained solely (or mostly) by excess demand, I can see it’s time to remind everybody about Gammon’s Law. I wrote about this back in September in my post The best laid plans (you might want to go back and read it—it’s one of my best):

Why do so many government programs fail? We’ve seen it time and time again. A need is identified, a program is formulated and put into place, everything starts out well enough, and then, perhaps over time, something happens. The program doesn’t achieve its goals. Or the amount of resources needed for it to achieve its goals are vastly more than expected.

We’ve seen this in Social Security, Medicare, the Great Society programs, and the public school system. Is it waste, fraud, and abuse (those favorite whipping-boys of legislators)? Welfare cheats? Incompetence? Just needs a little fine tuning? We’re not spending enough (no matter how much we seem to be spending)?

Back in the 1960’s a clever British physician named Dr. Max Gammon noticed that in health care although inputs were rising sharply (it was getting much more expensive) outputs (however they were measured) were actually falling. As an aside to those who are convinced that the problems in our health care system are due to patients demanding too much care that means that the health care system (as a whole) is getting paid a lot more for doing the same (or less) work.

Dr. Gammon has recently made additional comments about his findings:

The destructive process involves the progressive displacement of productive activity by non-productive and often counterproductive bureaucratic activity.

Before presenting my evidence I should like to correct a widespread and very damaging misconception: bureaucracy is not, I repeat is not synonymous with administration. By bureaucracy I mean a rigid system of human organisation governed by fixed rules and tending to exclude individual initiative. By administration I mean the guidance and facilitation of an enterprise. And this should be the very opposite of bureaucratic. The tragedy of the NHS is that it is an inherently bureaucratic organisation which imposes the bureaucratic mode of operation on all who work in it.

Two short stories will show what I mean by bureaucratic displacement. Some years ago I worked in a small hospital just outside London. The quality of the nursing on one particular ward was outstanding. I returned a few years later to find the ward in chaos. When I asked for the Sister who had previously run the ward so superbly I was told that she had been moved into administration. I found her in an office and she wept as she told me that she had been forced to leave her ward and become an administrator since her salary would have been reduced and fixed at a low level if she had remained in her clinical post. The focus of nursing had been changed from the ward and the patient to the office and the seminar room. The intimate care of patients had come to be regarded and rewarded as a menial task, a preliminary stage in a nursing career which was now seen to be essentially that of management as my second story shows.

Walking along a wide corridor in a London Teaching Hospital with which I was unfamiliar I asked a person in nursing uniform the way to a certain ward. “Oh, I don’t know where the wards are, I am in administration” she replied as though I had insulted her. She turned and walked through some glazed doors which opened on to what had been a long Nightingale ward. It was now divided into a multitude of open-plan offices. Meanwhile in the A&E Department on the floor below patients were lying on trolleys for up to 48 hours and others were having their operations postponed owing to shortage of beds, Two examples of bureaucratic displacement. Politicians dismiss such stories as ‘anecdotal’ – by that they mean that they relate to real people and real events rather than to statistics. So let us take a look at the statistics.

When the NHS was established in 1948 we had 480,000 hospital beds. By the year 2000 the number had fallen to 186,000. This represents a fall from 10 beds for every thousand of the population in 1948 to 3.7 in the year 2000. It means that we often now have insufficient hospital capacity for prompt investigation and treatment even of first class emergencies. According to the official statistics we have just under one million patients waiting for hospital admission. There is no margin for handling epidemics and admissions for elective surgery are frequently cancelled owing to lack of beds. In an attempt to deal with this state of constant crisis patients are now being sent to France and Germany for their operations. And still the number of NHS beds is falling.

As for staff, the number employed by the NHS has more than doubled from 350,000 in 1948 to 882,000 in 2002. The greatest percentage increase has been among designated administrative staff. Between 1997 and 2002 Senior Managers and Managers increased by no less than 47.6% compared to an overall increase in the workforce of 16% (nurses increased by 1.8%) But these figures reveal only the tip of the bureaucratic iceberg. For example large numbers of nurses are now wholly engaged in management but are still counted as nurses. Of even greater significance is the proliferation of bureaucratic procedures involving all staff, progressively displacing their productive activity. And now, here we come to the heart of the matter.

Nearly thirty years ago I discovered a close correlation between the increase in the numbers of NHS administrative staff and the fall in numbers of NHS hospital beds that had occurred over the preceding nine years. For statisticians: linear regression analysis showed a correlation coefficient of -0.99. For non-statisticians I should explain that this figure represents an almost perfect correlation between the growth in numbers of administrators and the fall in numbers of beds.

A statistical correlation, no matter how close, is not necessarily significant. However I suggested that this correlation could have an important explanatory value if the number of designated NHS administrators was proportional to the bureaucratic activity of the NHS workforce as a whole. The correlation of the growth in numbers of administrators with the fall in the number of beds would then follow from a progressive displacement of productive activity of all NHS staff by the proliferation of useless and often counterproductive bureaucratic activities throughout the whole organisation. In this way, an expanding workforce and increased spending would be matched by a fall in production; the more that was put into the system the less would come out of it, a process I likened to the implosion of a black hole.

So bureaucracy itself is the culprit. And, as I wrote in The best laid plans:

There are only two known organizing principles in modern societies: bureaucracy and the unpredictable large scale group behaviors of complex systems known as emergent phenomena. Reliance on emergent phenomena to solve the great problems requires an enormous amount of faith and hope.

Government is not the only source of bureaucracy in our health care system in the United States. Actually, there are three:

  1. The government
  2. Insurance companies
  3. Private medical bureaucracies including hospitals and HMO’s.

If Gammon’s Law is false, there’s a simple way for advocates of the demand theory for rising health care costs to disprove it: show that outputs are rising (and that inputs per output are falling).

But if, as Dr. Friedman suggests, Gammon’s Law is true, neither a reduction in consumer demand nor an increase in producer supply (doctors, hospitals, etc.) , nor the combination of reduced demand and increased supply will stem the rise in costs per output. So, what do we do about the ailing health care system? As I see it there are several alternatives: going on as we are now, a pure market system and a ban on large-scale medical bureaucracies at all levels, and a single-payer system (and attendant elimination of private large-scale medical bureaucracies)

If we maintain the status quo, health care experts are predicting a melt-down of the system in the next 3 to 5 years, see here and here. That’s not much time to plan and execute the next step but I’d suggest we’d better start doing that or the system that will emerge from the wreckage could be much, much worse.

The pure market and attendant ban on large-scale medical bureaucracies approach would include the following:

  1. Eliminate all federal and state government health care programs including Medicare, Medicaid, and the Veteran’s Administration health care system (which is itself the size of British Public Health).
  2. Eliminate all private insurance plans.
  3. Eliminate national and regional HMO’s and hospital chains.

For those who advocate such a plan I have a number of questions:

  1. How do you plan to pass such a plan through the Congress?
  2. What Constitutional changes will be required for such a plan? How
    will you pass the necessary changes through the state legislatures?
  3. How do you manage the transition in a humane and ethical manner that’s sound from a public health standpoint?
  4. Will this create the necessary market environment or will we need to abolish the FDA, the Patent Office, and medical licensing as well?

The final option is a single-payer system. The challenge here is to implement a plan that has the low administrative overhead of the Social Security sytem rather than the high overhead of the current system. As the old Magic 8-Ball might say “Prospects doubtful”.

It’s not surprising that this whole situation is a nettle that no one has been willing or able to grasp for a long time now. But remember: we simply cannot solve the problem and leave the bureaucracies in place. I’ll give Dr. Gammon the last word:

Sensitivity to external stimuli transmitted throughout the system, rather than centrally prescribed rules and directives, is what ultimately drives and governs individual performance in the non-bureaucratic organisation. By contrast labyrinthine systems of ‘monitoring’ and the setting of targets – with penalties and rewards – is characteristic of bureaucratic organisations attempting to improve performance. As was notoriously demonstrated in the Soviet Union and as we are now finding in the NHS, centrally imposed targets cause systemic distortions and rigidities that further impair performance and also, inevitably, lead to falsified statistics and the coercion of those tempted to reveal the truth. My friends, the National Health Service is an experiment which has failed. The cost of this failure to patients, to the medical profession, to the nursing profession and the to profession of Hospital Administration is beyond calculation. But the experiment will not have been in vain if others learn from its example not to repeat it.

10 comments