What Not to Do With Mathematical Models

For some reason or other I’m seeing a lot about mathematical models these days. Maybe it’s just me. For example, much of the scientific argument about global warming and climate change is about mathematical models. The political argument on the other hand has gotten way ahead of the science.

The models of the spread of the H1N1 virus have been stunningly wrong:

In the waning days of April, as federal officials were declaring a public health emergency and the world seemed gripped by swine flu panic, two rival supercomputer teams made projections about the epidemic that were surprisingly similar — and surprisingly reassuring. By the end of May, they said, there would be only 2,000 to 2,500 cases in the United States.

May’s over. They were a bit off.

On May 15, the Centers for Disease Control and Prevention estimated that there were “upwards of 100,000” cases in the country, even though only 7,415 had been confirmed at that point.

The agency declines to update that estimate just yet. But Tim Germann, a computational scientist who worked on a 2006 flu forecast model at Los Alamos National Laboratory, said he imagined there were now “a few hundred thousand” cases. (At their peaks, epidemics are thought to double in as little as three days, which could drive the number into the millions, but Dr. Germann said he would not use such a rapid doubling rate unless it was a cold November and no countermeasures, like closing schools, were being taken.)

What went wrong?

The leaders of both the Northwestern University and Indiana University teams seemed a bit abashed when they were asked that last week.

The rationale behind the stimulus package that was passed earlier this year and its predicted effects on unemployment were based at least in part on a mathematical model and that has gone wrong, too, viz.:

Hat tip: Innocent Bystanders via QandO

Oddly enough this is a subject on which I can comment with at least a little authority. In all likelihood I was creating mathematical models and programming computer models in Fortran and Simula before most of the people creating mathematical models were born or at least before they were able to read. When you create a mathematical model you start with observation, then you collect data, you make some assumptions about the processes that may have lead to the phenomena being modeled, using the assumptions you create a model that explains that data, and only then and with some skepticism can you use that model to extrapolate into future behavior.

The Keynesian multiplier on which the stimulus was hypothesized is a mathematical model and back in the ancient days when I took economics Keynes was king. I have never been skeptical that government spending might produce the stimulus effect. Over the years I have wondered about the empirical basis of the theory, whether the effect was actually observed in the real world or whether secondary effects overwhelmed the stimulus effect.

When a model fails to predict you’ve only got a few directions in which to turn. If you’ve made a flat-out error, you need to identify the error, correct it, and ensure that the error doesn’t recur. It’s possible that the data were improperly gathered or incomplete. In that case you gather new or more data, adjust your assumptions if necessary and correct your model to produce results consistent with the revised data. Or it may be that you need to make different assumptions.

That’s the problem that I have with the reactions I’ve seen so far to the failure of the models to predict outcomes. If the data are correct, they need to change their assumptions.

That’s why I find Steve Green’s critique of the Obama Administration’s economic policy a little Luddite. I won’t rule out the possibility that government spending might produce a stimulus effect.

But I do think that Steve is right in being concerned and warning of the secondary effects. I’m also concerned that the Obama Administration is not revisiting its assumptions. In the absence of strong empirical evidence for it, deciding that all you need to do to produce the effects you’re looking for is to do more of whatever you’re doing is unempirical.

I’ve seen that attitude in economists, in managers, and in physicians. I think it must be a universal human characteristic. In physicians it generally takes the form of prescribing a higher dose of something when the lower dose hasn’t produced results despite the fact that studies demonstrating the efficacy of whatever it is showed a therapeutic dose within a range with no greater effects shown above the therapeutic dose. Scientists aren’t always scientific.

25 comments… add one
  • I’m not sure how you get from “they need to change their assumptions” to “That’s why I find Steve Green’s critique of the Obama Administration’s economic policy a little Luddite.” I think his critique was dead on: businesses are refusing to commit to growing the business because the legal and regulatory environment has become uncertain and unpredictable. There’s nothing backwards-looking about that; it’s simply a note that the administration’s beliefs must be wrong because their predictions are utterly off base, and a diagnosis (from psychology and history, rather than modeling) of why their predictions are so off base. I don’t think you have to argue against a model with a different model, nor within the framework of the model being argued against.

    The thing is, we often miss the point of the phrase “the rule of law.” It is meant to be contrasted to “the rule of men.” When rules are unknowable and cannot be fully understood, then whether, when and how the rules are enforced is up to the arbitrary will of those in power. And whether those in power were put there by elections or by inheritance or by the naked exercise of power, the arbitrary exercise of authority is tyranny. For a long time now, our laws have been becoming less and less knowable by sheer effect of their volume, and increasingly they have become so technical as to be beyond the understanding of experts in the field, never mind the lay public. And thus the exercise of the law has been increasingly more and more arbitrary, and has trended more and more towards “what we can get away with that is to our advantage or our liking,” rather than towards what is right and just and good for society. The political class has become a group of cultural and economic looters, and Obama’s recent exertions have simply increased the visibility of this trend.

    There are few options, frankly, for return to a true rule of law. There are even fewer that don’t involve violence. We are among the first few generations of Americans that are substantially less free than our fathers, and I suspect it will get worse rather than better.

  • Roy Lofquist Link

    The problem with mathematical modeling is the failure to recognize that many systems are, in Roger Penrose’s words, “not computable”.

  • Jeff, at least judging from that post, Steve rejects the idea of the Keynesian multiplier a priori. I think that’s excessive and, as I wrote, Luddite since I can envision how it might work under specific circumstances. The pertinent question is are these the right circumstances?

    I have my doubts. For one thing I see no reason that the nth dollar of stimulus spending should have the same effects as the n+1st and we’ve been engaged in fiscal stimulus via spending for much of the last 70 years. But, again, that’s no reason to reject the theory out of hand.

  • OK, but he does give a reason to reject the theory out of hand: the theory’s claimed results do not mesh with the actual results. Now, it could be that that is a question of zone of applicability, that we are beyond the point (or as Krugman would have it, not yet to the point) where the effect would be seen. But that is an argument for the theory’s supporters, not its opponents, to make. Under what conditions would it work, and why would it work? Then demonstrate it.

    Moreover, maybe the difference is just that I know enough history to realize that the Luddites were not about reasoned opposition to progress, but about sabotaging any attempt at progress. Even taking it as a given that Keynes is progress (and I do not take that as a given), and that progress is good (and while I do not take that as a given, I think that it’s likely), Green is being anything but a Luddite. I suspect that you know that history as well; hence my confusion as to what you are getting at.

    Note that, in all of this, I am not making a partisan argument. The Republicans buy the Keynesian premises as surely as do the Democrats, and McCain made very clear that he intended to do much of the same that Obama is doing.

  • I believe Steve is rejecting the technology out-of-hand because he doesn’t understand the technology. I think a more nuanced argument is required.

    Note that I think the model is goofy. That’s how I explain what we’re seeing. And I’m worried that President Obama’s economic advisors are busy sacrificing goats to ensure the fertility of the crops.

  • As a Pagan, I only wish Obama had that level of sophistication. 🙂

    Seriously, though, I don’t get that from Steve. I think that Keynes has become received wisdom among the political class because it tells them they can do what they want to do anyway (centralize power on themselves) and the limousine liberals (feel good about themselves for their generosity) without any sacrifices on their parts, and with a patina of moral rightness from having done good. But just because it is attractive does not make it right. I can think of many, many reasons to argue against Keynesian economic policies, and a large number of those arguments must be made from a frame of reference outside of Keynes’ models. Steve is making such an argument: the predictions of the model are wrong, therefore the model is wrong.

    That is a very strong argument, one of the strongest that can be made against a proposal. The way for supporters of the Keynes model to counter the argument is to show that there are very good reasons for why the model’s predictions were off in this case, and those cases in which it works and does not work, and to correct the predictions accordingly. They have not done any of this. In fact, as far as I can tell, Obama’s administration generally does not argue in good faith. This is not a surprise, as I cannot remember any administration that consistently has done so, and only a few (Bush I and II, Eisenhower, Truman) since FDR appear to have even tried to argue in good faith.

    By arguing in good faith, I mean that those making the argument have a willingness to find the best public policy, instead of just a desire to “win” the argument, by shutting down or ignoring opposition if necessary. The ends of argument in the public sphere were originally to arrive at the best public policy. We have largely abandoned those ends along with the abandonment of classical Western culture generally.

    Now, all of that said, it seems to me that you are not addressing Steve’s argument: the models do not predict correctly. Instead, you are expecting him to make a different argument. I cannot see how he has the obligation to do so, or how doing so indicates that he does not understand Keynes. Admittedly, I don’t read VodkaPundit that often any more, so he may have given evidence of that elsewhere, but I see no evidence in the post under discussion that he misunderstands what he is arguing against.

  • PD Shaw Link

    Oddly, I was graded on this issue in high school. Senior year, we were assigned to write a critical paper and I had somehow gotten a study that challenged the notion that defense spending had a greater multiplier than domestic spending. Without really understanding the statistical principles, I was able to identify the assumptions and challenge them. I only got a B. Why? I concluded not only that the study was wrong, but that spending on guns had a greater multiplier than butter. Disproving a study is not the same as proof.

    (BTW/ this was a rhetoric class, not economics, so whether I had the economics right was not relevant)

  • PD Shaw Link

    Jeff: some where in those unemployment figures is the effect of the administration hiring IIRC 60,000 people to work on the next census. That’s certainly government spending that increases jobs and at least some marginal demands in the economy. How much? Probably not much, but I can’t believe that government spending has a zero effect on the larger economy.

  • I get your point, Jeff.

    I think that Keynes’s arguments are compelling but misapplied. I guess I fit into your second camp. And, importantly, I take the fundamental soundness of the argument which, simply stated, is that depressions are caused by not enough consumption, as a given, almost as sure as the Third Law of Thermodynamics.

    I’d think that rejecting the Third Law of Thermodyamics was pretty anti-technological, too.

  • I would not argue that government spending has a zero effect on the larger economy. I would argue that government spending does not have a larger positive effect on the economy (and likely has a smaller positive effect) than equivalent private spending. Clearly, government jobs are jobs, and some of them produce wealth (though not many, and not census jobs). Many more of them safeguard wealth. And a large number of them simply consume or transfer wealth without creating or safeguarding anything.

  • PD, that brings us full circle to a point I made some time ago about the size of the multiplier. The way I use the term a multiplier of 1 means that every dollar spent by the government is a dollar spent, a multiplier of zero means every dollar spent by the government is a dollar removed from the economy, and a multiplier of 1.5 means that for every dollar spent by the government another half dollar is being created.

    I think the empirical evidence is quite strong that the multiplier approaches 1. That doesn’t necessarily mean that it’s a bad idea from a policy standpoint, only one that must be used cautiously. The reason it’s not necessarily bad is that you’re moving spending from some time in the future to now and it may be very seriously needed now.

    As I understand it the Obama Administration economic advisors are using a model of something like 1.4 to 1.7 in their calculations. I think the empirical case for that is quite weak, especially when the way they’re spending the money is taken into account.

  • That’s not the argument, Jeff. The argument is that there’s not enough private spending. If government were a prudent steward (it isn’t), we would say that the government is the consumer of last resort.

  • I’m sorry; I was unclear in my antecedents. I was merely responding to PD Shaw’s “I can’t believe that government spending has a zero effect on the larger economy.”

  • Drew Link

    Musings…..

    1. Jeff’s assertion that the business environment had degraded (because it is more uncertain) is simply flat damned…….correct, and that investment and future economic activity will be muted is flat damned……correct.

    No doubt, a portion of this is the general environment. But that is the case in any recession. New to this recession is Obama policy. Choosing winners in an anti-capital way. Higher taxing as a philosophical issue……… crazy spending that will mean higher taxing as a practical issue. Increasing the costs of employing people.

    It is unclear to me from reading comments here what the various professional backgrounds are. But I get the feeling I am the only resident professional investor in small to middle market businesses. As such, (for 19 years) I think I have a decent grasp on the mindset of the entrepreneur/small business owner. Anyone who doesn’t understand that small business owners are hunkering down into no growth/contraction mode right now because of fears about Obama costs to employing/tax rates or flat out ignoring the law or rights of capital providers in favor of his political faves is just in denial. This is as real as a heart attack. Employing less; investing less. Dave, you have criticized this. But we can skip the morality. As an empirical fact its happening and it will continue if Obama continues down his path.

    2. Dave – if you know physicians who beat their head on the wall as you cite, I would kindly advise you not continue with them. That’s a bad doc. BTW – I do not know such physicians.

    3. The problem with stimulus and multipliers is that when withdrawn……….you are back in the soup. Oh, and there are costs down the road. Obama has announced he’s “not satisfied” and wants another stimulus. Think it will be the last? Can you say Japan??

  • Drew, I do know such physicians. I’ve had them, friends have had them, members of my family have.

    I think that it’s virtually a done deal that the Obama Administration will be asking for an additional stimulus package within the next month or despite having spent, what, less than 20% of the original appropriation? I think it’s near-perfect example of doing the same thing and looking for a different result.

    As to whether the Congress will go along, on that I’m not so sure.

    Note: I opposed the stimulus package. I know too much about the project process in state and local governments to believe that they could use the money effectively and in the timeframe required.

    Professional background? I’ve worked in management positions for small, medium, and large businesses. I’ve been a small business owner for many years. Over the years I’ve had clients who’ve been small, medium, and large businesses including a half dozen Fortune 500 clients. I’ve had local, state, and federal government clients. I’ve had many, many professionals as clients: lawyers, docs, dentists. I know a lot about the business of law, the business of the practice of medicine, and the business of dental practices. I’ve had several insurance companies and banks as clients including Mutual, Bankers, Household Finance, and Bank of New York. I know more about the Big 3 auto makers than is good for one.

  • Re “depressions are caused by not enough consumption,” does anyone seriously contest this point? My understanding was that most of the argument is around Keynes’ prescriptions for what to do about it. And more than that, the argument I would enter into is that even if Keynes is right as to his prescriptions, his theory has been so misapplied (deficit spending in a non-recessionary cycle) as to render ineffective or worse an attempt to restore economic health by shifting consumption into the present, because part of the current problem is an outgrowth of past deficits and regulations creating artificially high and unsustainable demand in several sectors of the economy, which can only be corrected by letting the pain happen now.

    Even given that, there are three problems with the administration’s economic policies. First, the government is not substantially bringing consumption forwards: most of the money is spent in future years, not the present. Second, the arbitrary changing of the rules (for example, on bankruptcy or executive compensation) and, as Green puts it, “the printing of the money, the driving up of the interest rates, and the impending raising of the taxes,” have contrived to make business more, not less, skittish about spending right now. Third, the likely effects of current policy, in future losses of currency value and increased taxes and interest, will act to depress future economic recovery, thus prolonging, rather than shortening, the period of recession (and probably deepening it as well).

    All of this can be argued without arguing against Keynes, and in fact Green does not even mention Keynes. Even though I am convinced the laws of thermodynamics are the best available explanation for the evidence, and even though their predictive power has proven (so far) quite accurate and utile, I would not accept at face value an argument that the laws of thermodynamics compel us to try to build a perpetual motion machine. And trying to build a perpetually growing economy (which itself violates a number of economic theories far more solidly grounded than Keynes’) is exactly what the government has been engaging in for the past seventy years.

  • PD Shaw Link

    Dave: “The way I use the term a multiplier of 1 means that every dollar spent by the government is a dollar spent, a multiplier of zero means every dollar spent by the government is a dollar removed from the economy, and a multiplier of 1.5 means that for every dollar spent by the government another half dollar is being created.”

    This is my understanding as well. Plus the multiplier assumes a a dollar in hand. If the government has to borrow a dollar to spend a dollar, then the government needs to make some kind of return to pay back the borrowing costs.

    I too assume it’s close to 1.

  • Drew Link

    “I know more about the Big 3 auto makers than is good for one.”

    LOL, I understand.

    Re: Docs. If you know such physicians, it argues against opening the floodgates. Either the students are bad, or the training is bad.

    Re: Stimulus. In Latin: Bangus headus on wallus. No-us usus.

    Re: Background. So with that understanding of small business owners, you must know they manage to the amount of money in the cash register on Friday. That is, they will not sit back and just eat tax increases or employment cost increases. Right or wrong. The vast majority just simply don’t. The left never gets that. Just this weekend on the driving range I was speaking with people about this. (We have a lot of small business owners.) They are fed up, nervous and just hunkering down.

    For every dollar of stimulus, I bet Obama’s policies have a negative $2 effect. See, I built this mathematical model………alpha’s, beta’s, cosines, polynomials……………

  • Drew Link

    Jeff –

    Except perhaps for Paul Krugman, fiscal policy as a recession fighter has widely been discounted in the profession until recently.

    Why? In a word, it comes with costs. Stimulate today, but when withdrawn, now what? That’s just kicking the can down the road. Second, financing the stimulus means higher taxes (destimulating) down the road, or higher financing interest rates. Screw your kids. Lastly, its viewed as mostly wasteful or ill timed.

    Japan did it serially with no effect. Team Obama seems hell bent on doing the same.

    Japan ended up with debt to GDP of 180%.

  • I don’t have an economic model, but I did ask my 8 year old son to draw a picture. It has dinosaurs, and space aliens attacking in a big laser battle, and a pirate with a rather gruesomely bloody sword, and Harry Potter. Hmm – extinction, aliens, piracy and magic … maybe I have a model for the administration, at least.

    🙂

  • PD Shaw Link

    I don’t necessarily disagree with the “rule of law” issues raised by Jeff and Drew, but it ain’t part of the stimulus package. That part is a failure of leadership on the part of Obama, as was the failure to direct the stimulus package to areas that would best have a chance of a high multiplier.

    Specifically, the holdout secureds of Chrysler need not have been called terrorists and told that the administration doesn’t negotiate. The positions of the parties seem close enough given the scope of spending to be negotiable.

    When the administration had approved past, but limited future bonuses at AIG, it should have stuck to its guns, called for national unity and promised future financial regulation to be a top priority in the coming years. This was a squabble over penies.

  • The point of Green’s essay was not the stimulus package narrowly, but Obama’s economic policies broadly. But it is clear in many ways that this administration looks at its opponents as actual enemies, and is willing to do anything against them (far more than they would, in fact, against foreign enemies, who are simply not close enough to them to matter).

  • So with that understanding of small business owners, you must know they manage to the amount of money in the cash register on Friday.

    Absolutely. Not only do I know that, I’ve examined the books of small business owners and advised them to do it. Nothing else made any sense for them.

Leave a Comment