What Can You Expect From a Half Billion Dollar Web Site?

The technical criticisms of the Healthcare.gov web site are becoming more strident:

WASHINGTON — The federal health care exchange was built using 10-year-old technology that may require constant fixes and updates for the next six months and the eventual overhaul of the entire system, technology experts told USA TODAY.

“The application could be fundamentally flawed,” said Jeff Kim, president of CDNetworks, a content-delivery network. “They may be using 1990s technology in 2.0 world.”

Recent changes have made the exchanges easier to use, but they still require clearing the computer’s cache several times, stopping a pop-up blocker, talking to people via Web chat who suggest waiting until the server is not busy, opening links in new windows and clicking on every available possibility on a page in the hopes of not receiving an error message. With those changes, it took one hour to navigate the HealthCare.gov enrollment process Wednesday.

Those steps shouldn’t be necessary, experts said.

“I have never seen a website — in the last five years — require you to delete the cache in an effort to resolve errors,” said Dan Schuyler, a director at Leavitt Partners, a health care group by former Health and Human Services secretary Mike Leavitt. “This is a very early Web 1.0 type of fix.”

Multiple problems are noted and they go on to suggest that the site may need a complete do-over.

It’s well known that design teams need to be small and cohesive and, frankly, it’s hard to have a small, cohesive design team with a web site with as many moving parts and as many players involved in the development as Healthcare.gov.

An additional problem. In a sense every computer program is a map of the designers’ and/or developers’ minds. The more complicated the problem that the program attempts to solve, the smarter the designer must be to see the inherent simplicity in the problem. There are some problems so complex that nobody is that smart.

29 comments… add one
  • Andy Link

    Those are significant problems. Even more worrying could be that the back end is messed up, as some have alleged, which is feeding false cost/benefit information between insurers and people using the site. In short, are the price quotes accurate even after someone does get through the lousy web interface?

  • Mike S Link

    Wow. Who could have seen this coming?? Actually, I think it’s fair to say that not even Obamacare’s most fiercest critics could have possibly imagined or predicted this monumental level of incompetence and disaster. It’s simply mindboggling. A train wreck would be too modest to describe it and an insult to train wrecks. All very predictable though for anyone who bothered to actually inform themselves, especially with regards to how government actually functions in the real world and not make-believe liberal land…

    But hey, liberals care. They care so much they’re willing to bankrupt the country to prove it. “Forward”

    This is the Democrats’ Iraq War (probably worse cause it WILL affect millions of Americans DIRECTLY)…and they so deserve it

  • jan Link

    Actually, I think it’s fair to say that not even Obamacare’s most fiercest critics could have possibly imagined or predicted this monumental level of incompetence and disaster.

    And, once the technical issues are resolved, how will the actual delivery of HC be, regarding quality and timeliness? Also, what about that much cited bending of the cost curve. If you are spending upwards of $600 million to create a sign-in nightmare, just how are costs going going to be decreased during a similar chaotic implementation of the rest of Obamacare?

    If this were a Republican HC plan, forced down people’s throats in the same way Obama did, the MSM and libs would be protesting in the streets. But, because it’s the dream legislation of social progressives, all you hear are excuses, rationalizations and a distracting humming noise from the WH, in their attempts to ignore the raucous complaints of the public in the background.

    Oh yes, let’s go back to democratic programming excoriating those awful republicans, and how they are destroying the country.

  • michael reynolds Link

    Well, I could point out that this half billion is 10% of what was spent for the latest Army camouflage uniform, which was abandoned after 8 years. . . http://www.thedailybeast.com/the-hero-project/articles/2013/10/14/the-army-s-5-billion-new-uniform-already-being-replaced.html

    . . . But I don’t get this any more than anyone else does. It ain’t that complex. This isn’t 1% of what Amazon handles flawlessly. It’s not 1990, it’s 2013. They don’t need to re-invent the wheel at this point.

  • Roy Lofquist Link

    Mike S October 17, 2013 at 4:11 pm
    Wow. Who could have seen this coming?? Actually, I think it’s fair to say that not even Obamacare’s most fiercest critics could have possibly imagined or predicted this monumental level of incompetence and disaster.

    Mike,

    This outcome was apparent to me, and many others, three years ago. As Dave said:

    “There are some problems so complex that nobody is that smart.”

    Or, in other words, non-computable, chaotic, np-complete (fancy mathematical term). I was in the business of systems design and development for more than 40 years. I have so many arrows in my back that the dog thinks I’m a porcupine. I wouldn’t touch this system with a ten foot Lithuanian – not for all the Maui-wowie in Petaluma.

    If there is any good to come of this there will be much immediate suffering and a necessary cauterizing of the wounds inflicted. Pain can be salutary – think about touching hot stoves.

  • Modulo Myself Link

    Roy,
    Well then three years ago you were wrong. The states that set up their exchanges have working websites. The 35 states that decided not to are the ones who are in trouble.

    So clearly the plan to have states run their exchanges was doable.

  • michael reynolds Link

    MM:

    Exactly. If New York can do it and California can do it (although there apparently have been some issues with the CA site despite my not encountering any) then we can do it at the national level. If Amazon, eBay, and any number of insurance sites can do it, it’s obviously not impossible.

  • PD Shaw Link

    I’m surprised in that I assumed that all of the exchanges would be based at the state level, just in some cases the exchange would be administered by the state itself, others by a state/federal partnership, and others by the feds. IOW, there would be 50+ exchanges, and probably some of them would have problems. Illinois is a partnership state and I’m surprised that it appears little different in this respect than those who defaulted to the feds.

    Good article here on what the regulators are saying privately about what happened:

    http://www.nationalreview.com/corner/361577/assessing-exchanges-yuval-levin

    I know, its the National Review, but it looks like actual reporting.

  • michael reynolds Link

    PD:

    I believe the technical term for this is “clusterfuck.”

  • jan Link

    It’s more than just the presence or even the absence of technical glitches, like people tooting horns in CA and NY are anecdotally reporting. It’s how this HC glob can be functionally pieced together, in all states, realistically giving better, less expensive care to more people. Wasn’t that the PPACA’s premise? “If these problems persist longer — weeks, months, a whole year — the entire Obamacare project falls apart, It’s a holy shit moment.”

    Many health insurance products currently on the market don’t meet Obamacare’s benefit standards and consumer protections so they are being discontinued. Consumers with these plans are the most likely to see rate increases next year, especially if they earn too much to get tax credits. “They’ve got to convert to a new policy — no ifs, ands or buts about it,” Laszewski said.

    And while people who currently pay for their own health insurance are likely to do whatever they can to remain covered, buying a plan directly from an insurance carrier or using a private online broker isn’t what Obamacare promised, and tax-credit subsidies aren’t available without the federal system. Moreover, these private companies aren’t prepared to deal with millions of customers who were supposed to be using the government marketplace, Laszewski said.

    Under these circumstances, the lion’s share of the people who do whatever is necessary to sign up through HealthCare.gov are likely to be the sickest and most expensive to cover because they have the greatest need, Laszewski said. That would make the pool of people covered very costly, causing health insurers to lose money and likely rethink whether they want to participate in the exchanges, he said. “The fundamental threat to Obamacare is we don’t get enough healthy people in the pool to keep the rates reasonable, and they are in grave danger of that problem,” he said.

    There is a massive logistics problem revealing itself here. Big government, though, is staying the course in offering itself as the centralized HC guru. However, prevailing logic continues to say such HC responsibilities are better handled within the states themselves, or by the encouragement of private sector competition, with free market innovation guiding and/or refreshing better approaches to healthcare gaps and/or inequities.

    Time will tell who is right. The only problem is that big government never admits it’s failings or wrongdoing in legislative matters or entitlement program experiments. They only pridefully throw more taxpayer money to fix a flawed outcome. And, if that doesn’t work out, they will introduce an even more onerous, outrageous program as it’s replacement.

  • sam Link

    That’s a good story, PD. And I’d never register a caveat for quoting Yuval Levin. I don’t agree with the guy most of the time, but I’ve never read anything of his that wasn’t a first-rate (and fair to the opposition) statement of his position. (I sometimes wonder what he’d doing at NRO. Same for Reihan Salem.)

  • TastyBits Link

    @Modulo Myself

    For each state, there are a number of interfaces that need to be developed for data transfer. Many of the existing systems were designed years ago, and there is no easy way to get the data. There could be multiple interfaces per state, and all of these would need to function correctly.

    In addition, badly written code can cause memory leaks, and these will will crash the system. If the data writes are not done properly, the data will become corrupt, and it can leave the database unstable.

    @michael reynolds

    The successful Internet based companies have been working on their code for years, but @Dave Schuler has gone over that with you. One big difference is that they do not need to interface with legacy systems. Amazon databases belong to Amazon, and if they decide to change the data format, they do it.

    A better example would be one company purchasing another. In many instances, it will be years before the two companies are using the same system.

  • jan Link

    PD, that was the best, most detailed written description of the Obamacare roll out I’ve read to date. It paints such a clear, but abject, picture of how unintended consequences flower under government programs created and administered by mediocre hires and overly ambitious bureaucrats.

  • In many instances, it will be years before the two companies are using the same system.

    And sometimes never. DuPont acquired Pioneer Hi-Bred back in 1999. They’ve been struggling to integrate their computer operations for almost 15 years now.

  • TastyBits Link

    @jan

    These are not limited to the government. Large scale software projects with multiple data sources are always a problem. Most companies outsource most of it because they do not have the in-house experience.

    There have been spectacular failures in the private sector, and there are several companies that no longer exist because of the failures.

  • jan Link

    Tasty,

    Some IT pros are estimately 3-5 years before all these data bases can be appropriately integrated with each other. It’s mind-boggling to me, how the complexity involved keeps gettting more and more revealed. I just don’t see why some of these issues couldn’t have been anticipated earlier, and worked on accordingly, in a time-frame befitting the work load.

  • Modulo Myself Link

    Then what did the states that ran their exchanges do right? Was it simply a matter of scale?

  • Modulo Myself Link

    Sorry, that question was directed to TastyBits.

  • TastyBits Link

    From @PD Shaw’s link:

    … My gut sense after listening to these insiders, for what little it’s worth, is that it’s not likely that the situation will prove to be much worse than it now seems, and it’s more likely that it will prove to be less bad than it now seems.

    More than likely the opposite is true. Most of the problems cascade, and the new problems cascade.

    They need to get an outside company with software project management expertise (the real kind), and they can begin sorting out this mess.

  • TastyBits Link

    @Modulo Myself

    The complexity can be an order of magnitude greater. Each interface can create a problem or problems. The numbers add up quickly, and they begin to impact each other.

    About 15 years ago, my company had an application that used a MS Access database, and it worked with few problems. One of our customers wanted to use an Oracle database instead, and we agreed. We used planned to use an ODBC connection and swap out anything Access specific. Many people know where this is going.

    Oracle and Access use SQL which is how you interact with a database. Both used the same standard – SQL-89, but each had caveats. Special code had to be written for the places with the caveats.

    There were multiple issues, and as one issue got resolved, another one would emerge. It took about 5 years before the problems were mostly worked out.

    A lot of the issues arose because the initial application used a database located on the user’s computer. When we offered an Oracle option, the original design was no longer applicable. Working in a networked environment using common servers is an order of magnitude greater.

  • PD Shaw Link

    The states are having problems, not just as bad. From the National Review link:

    “All of the CMS people I spoke with thought the state-run exchanges are in far better shape than the federal system under their purview. But the insurers do not seem that much happier with many of those state exchanges. Back-end data issues seem to be a problem everywhere, and some of the early enrollment figures being released by the states are not matching up with insurance company data about enrollments in those states, which suggests a breakdown in communication that is only beginning to be understood. The insurers believe that only Nevada, Colorado, Washington state, and Kentucky have what could reasonably be described as working systems at this point. Still, there is no question that on the whole the states with state-run exchanges are in better shape than those with federal ones.”

    I wonder if the medical and insurance sectors in the states with “working systems” are more consolidated than in the other states. While the listed states are large, they do tend to have populations consolidated in a certain part of the state.

  • Red Barchetta Link

    “A better example would be one company purchasing another. In many instances, it will be years before the two companies are using the same system.”

    Exactly. Along with integrating cultures, the two biggest problem in company integrations.

    I do have to say its interesting to see those defending certain states, as dubious as those reports are, have not tumbled to the fact that a ginormous national system was a doomed by its inherent design, including discovery of best practices and firewalls in case of problems.

    Hell, my club is putting in a new irrigation system and had the basic sense to have each branch pipe and pump system cover no more than 3 holes apiece (of 18) to avoid a total system meltdown in the case of a problem. Design 101.

  • All of the CMS people I spoke with thought the state-run exchanges are in far better shape than the federal system

    The operative word in that sentence is “thought”. In the absence of a uniform empirical standard for measuring how well their web sites are operating I’m reluctant to venture an opinion one way or another. We know that Healthcare.gov has problems because so many people continue to have such problems accessing, navigating, and getting results from it.

    What underlying problems any of these sites may have will appear in due course. However, if I were managing any of these developments I would have a standard of operational success a bit more stringent than “well, it looks like it works”. I’m reminded of a famous sentence from The Mythical Man Month: “If builders built buildings the way programmers write programs, the first woodpecker to come along would destroy civilization.”

  • ... Link

    A train wreck would be too modest to describe it and an insult to train wrecks.

    That’s hardly fair to the exchanges, now, is it? How many trains have millions of cars loaded with fuel, no locomotives, no engineers, and are still going into the curve at 120 mph? Admit it, that would be a much more interesting train wreck than what you’ve seen in the past.

  • ... Link

    If Amazon, eBay, and any number of insurance sites can do it, it’s obviously not impossible.

    And yet the Administration, stocked with thousands of the most brilliant minds, with huge resources at their beck and call, have completely fucked it up, given almost three and a half years lead time.

    That’s almost as much time as it took for the US to beat the Japanese after the bombing of Pearl Harbor, which was itself a massive undertaking in logistics and management (among many other things), and that was ancillary to the US war efforts in Europe. (German scientists scared the leaders of the Atlantic Allies, and the Japanese scientists didn’t. Thus the focus on the European theater.)

    So, why could THAT government, led by a man with a second rate intellect, do much more than this government led by a first rate intellect? Remember, this is all within the bounds of the executive branch, so talk of Republican interference, intransigence and hostility doesn’t cut it.

  • Red Barchetta Link

    Now, now. Don’t bother them with facts and perspective. After all, the have a hero to slavishly worship……..no matter the obvious incompetence. You must be a racist or something.

  • Red Barchetta Link

    Because I’m generally always in a sarcastic, mocking, poke people in the ribs mode I feel the need these days to identify when I’m being serious.

    I’m being serious. steve – I see this and wonder what your reaction is. I recently cited a Cleveland Clinic doc who claimed his organization expects to have to reduce expenses by some 30% because of government reimbursement issues. You recently cited “belly surgeries” and the like as huge expense drivers. I have no reason to dispute that; its your field of expertise, not mine. The Cleveland Clinic guy cites chronic diseases such as diabetes or smoking and obesity related chronic disease (yet the government champions “hunger” and churns toward ever more food stamps…..but I digress) as huge cost drivers.

    Thoughts?

    http://hotair.com/archives/2013/10/19/doctors-prepare-for-obamacare-race-to-the-bottom/

  • jan Link

    Drew,

    I’ve often thought about the cognitive dissonance of pouring more food stamps and free breakfast/lunch programs into a population often criticized by even the FLOTUS for their dietary excesses and poor nutritional choices. Aren’t we one of the most obese countries in the world? And, don’t we also have some of the richest poor people in the world as well? One thing for sure is that we are becoming one of the dumbest civilized countries in the world, having an ever decreasing influence on the global stage because of our mediocre standards and leading-from-behind behavior, making us irrelevant in almost everything we do and say. Now we are going down the road of becoming a statist-driven health care provider, and everyone on the left is cheering and turning a blind eye to all the red chaotic flags waving in the wind.

    It’s amazing, simply amazing…..

  • steve Link

    Drew- None of the insurers in our state have asked to change reimbursement rates. I have not heard of it happening elsewhere, but the people I know well enough to ask about that represent about 5 states. I was just at an IT conference in Denver last month and no one there, people from about 16 states I think, had seen this happen. In general, commercial carriers paying less than Medicaid is uncommon.

    On the other issue, yes, the big cost drivers are procedures and chronic illness. I am an acute care guy so I concentrate more on procedures, and i bet he is an internist, focusing more on chronic care. That said, if you look at where the growth is in medical care, it is much more in the outpatient procedural area than any other.

    Steve

Leave a Comment