Obamacare: 500M Lines of Code, $500M, 60% Complete, No Surprise

Obamacare: 500M Lines of Code, $500M, 60% Complete, No Surprise

on Sun, 1 Dec 2013

The historic IT disaster known as the Obamacare web site has been of intense interest to me, since I've been a Senior Software Engineer for decades, and I have personally participated in, witnessed, and reported on (as a technology reporter) a number of IT disasters. But even so, the size of the Obamacare web site catastrophe on October 1 still takes my breath away.

When I first heard, shortly after October 1, that there was 500 million lines of code inHealthcare.gov, I quickly rejected that figure, because it's impossible. 

Going back to the development of IBM's System/360 operation system, as described in Fred Brooks' classic book, The Mythical Man-Month, the average programmer on the project wrote six lines of code per day. Of course, every programmer writes a lot more than that on SOME days, but on other days he writes zero lines of code, since he's doing testing or debugging or rewriting or documenting. So for System/360, it all averaged out to six lines per day per programmer. 

So let's say that the Obamacare programmers were much better than that, and wrote 100 lines of code per day average. Let's say that there were 1000 programmers. And let's say that, over the three year period, there were about 660 business days. Then, with those generous assumptions, you get 100*1000*660 = 66,000,000 lines of code. It's simply impossible to reach 500,000,000. 

And yet, the 500 million figure is apparently true. I've heard it dozens of times in the last month, and no one is denying it. Healthcare.govapparently really does have 500 million lines of code. How is that possible? 

I get a picture in my mind of 1,000 people sitting a computers typing code, without worrying about whether or not it works. Given the size of the catastrophe, some variation of that must have happened.

More important than that, a code base that size is unsupportable. Health services is a rapidly changing field, and every time there's some kind of process or rule change, it will take an army of programmers to make all the necessary changes in the code base. And that assumes that all the bugs have been fixed, which is far from true.Healthcare.gov will not be fully functional at any time in the foreseeable future, if ever. 

On October 1, Healthcare.gov had 500 million lines of code, and could handle a handful of simultaneous users. Facebook.com has 20 million lines of code, and handles millions of simultaneous users.

Then there's the cost. Healthcare.gov should have cost $5-10 million to implement. Take into account government corruption and incompetence, it should have cost $10-25 million. Instead, it cost $300-600 million -- let's say $500 million. How do we get to that figure? Well, assume 1000 programmers are paid an average of $100 per hour ($200,000 per year) for 8 hours per day for 660 business days: $100*1000*8*660 = $528 million. So at least that figure makes sense -- as long as you understand that the Obama administration poured half a billion dollars into the pockets of his cronies and supporters, and got exactly what he deserved with Healthcare.gov. 

How could President Obama have been so wrong?

There have been numerous reports that the Obama administration had been informed many times, including by McKinsey & Co. in March, that Healthcare.gov was in serious trouble. And yet, just two weeks ago, on November 19,President Barack Obama said: 

I was not informed directly that the Web site would not be working the way it was supposed to. Had I been informed, I wouldn't be going out saying, boy, this is going to be great. You know -- I'm accused of a lot of things, but I don't think I'm stupid enough to go around saying this is going to be like shopping on Amazon or Travelocity a week before the Web site opens if I thought that it wasn't going to work. So, clearly, we and I did not have enough awareness about the problems in the Web site. And so, since he posed the question of how he couldn't have been that stupid, let's try to find an answer.

And so, since he posed the question of how he couldn't have been that stupid, let's try to find an answer. 

Once again, I have a number of personal experiences that relate to this, and the main one I'd like to relate is the most bizarre day of my professional life. 

In 1985, I was doing contract programming for Northrop Corp., developing embedded software for munitions guidance systems. The project manager had to be reassigned, and I became acting manager. After being in this position for 2-3 weeks, it was clear to me that the whole project was in trouble, would slip at least three months. I told this to the program manager, and he nearly freaked out. 

I was then pulled into one meeting after another, and met high-level managers that I never knew existed. Their conclusion apparently was that I was full of crap, and they decided to fire me, but they weren't sure, so they decided to let me stay on until the release date, and then they would fire me. 

About a week before the release date, there was a meeting in the lab, where the lead programmer was to demonstrate the embedded system to the Corporate VP. I attended this demo, but I was ordered just to stand there and keep my mouth shut. So I stood back, leaned against the wall, and just watched the proceedings. 

The lead programmer gave his demo, and the VP ooohed and ahhhhed. He then asked, "And this will be ready for release next Monday?" The lead programmer said, "Yes, it will be ready on Monday." The other managers in the room also said, "Monday." 

I was holding my breath through all this. I couldn't believe what I was hearing. Either I was crazy or all of them were crazy. That's why I call this the most bizarre day of my life. 

Well anyway, to make a long story short, the project was not ready for release the following Monday. It slipped six months. I wasn't fired because one Silent generation manager went around and told everyone that "Xenakis was right, so he shouldn't be fired." So I wasn't fired. 

I've had several other experiences like that, except that usually I was fired, even though I was always right. In 1992, I was working as a contract programmer for Fidelity. I did a little unit testing, and wrote a memo to my manager, listing numerous problems why the project was in trouble, saying that it would slip at least six months. The manager was so furious that he had smoke coming out of his ears. He fired me. The project crashed completely a couple of months later. I was right, and he was wrong, though I paid the price. 

In a previous article ( "14-Oct-13 World View -- HealthCare.gov IT systems a continuing disaster") I related a 2005 experience where I was fired for telling my management that the $10 million project currently under development would slip 6-12 months. I was fired and out of a salary, while the dozens of incompetent engineers kept collecting salaries for over another year, at which time the disastrous project was canceled completely. 

So there's really nothing about theHealthcare.gov disaster that's a surprise to me (except the 500 million lines of code). I've been a Senior Software Engineer for a long time, and I've worked on over 100 projects, so I've seen disasters. The only thing that's different aboutHealthcare.gov is the breathtaking size of criminality of the Obama administration in wasting so much money on cronies and corruption, and the breathtaking size of the resulting well-deserved disaster. 

It's now December 1, the scheduled date of theHealthcare.gov re-launch. Based on news reports, the following has been done: 

• The screens will be changed to be prettier and more fun.
• All the small business functionality has been removed and postponed for a year.
• All the functionality for online payments has been removed, which means that customers will have to be billed by mail by the insurance companies.
• The security issues, including the exposure to identity theft, have not been dealt with at all.
• According to some reports, the web site is only 60% completed, even after 500 million lines of code and $500 million. The cost of the web site could run into the billions.
• Users will be encouraged to go online at "off-peak hours," to keep the web site from being overloaded.
• The administration is licensing other web sites to share the load. GoHealth.com andTheHealthSherpa.com have been named in news reports.

With regard to the billing being done by mail, as I understand it, you have to miss three payments in a row to be dropped off your Obamacare plan. If that turns out to be true, then what you can do, Dear Reader, is sign up for an Obamacare plan, not pay your bill, and then sign up with a different insurance company in three months. However, your doctor won't be happy to see you, since the insurance company won't reimburse him for treating you. 

The Future of Obamacare

Long-time readers are aware that from the day it was first proposed in 2009, I've referred to President Barack Obama's health care plan as a proposal of economic insanity, because it's a repeat of President Richard Nixon's wage-price controls, which were an utter, total disaster for the economy. 

In an article on July 5 ( "5-Jul-13 World View -- Eurozone and Obamacare continue their parallel economic collapse"), I explained in greater detail why Obamacare would fail for the same reasons as Nixon's wage-price controls and Mao Zedong's Great Leap Forward in the 1950s. 

A couple of people criticized me for making the comparison to Mao's Great Leap Forward. However, Mao's Great Leap Forward contained exactly the same elements as Nixon's wage-price controls and Obamacare -- attempts to control a huge market by passing a law, which makes just as much sense as trying to control the weather by passing a law. 

But if the elements are the same, there are major differences in degree. Mao carried those elements much farther, and the results were commensurately more disastrous, with the deaths of tens of millions of people. Nothing like that happened with Nixon's controls, or is expected to happen with Obamacare, where the "only" result is a disastrous economic and medical crisis, with the destruction of much of the medical services marketplace. 

Still, there's one more lesson to be learned from Mao's Great Leap Forward. When things started going wrong in early 1959, Mao was warned by numerous experts. Instead of heeding those warnings immediately, which might have saved ten or twenty million lives, Mao had the experts fired or executed. We're now at a very dangerous time when the economic calamities of Obamacare are obvious to almost everyone, and are being pointed out by many people. But instead of heeding the warnings, President Obama is ignoring them, and threatening political opponents with such things as further IRS audits. The longer President Obama continues in this way, the more damage he does. Obamacare will not cause the tens of millions of deaths that Mao's Great Leap Forward did, but it's now quite possible that Obamacare will be the direct cause of hundreds or even thousands of deaths. Investors Business Daily