SoftwareMarketSolution: Joel, what, in your opinion, is the single greatest development sin a software company can commit? Joel Spolsky: Deciding to completely rewrite your product from scratch, on the theory that all your code is messy and bug-prone and is bloated and needs to be completely rethought and rebuilt from ground zero.
SMS: Uh, what's wrong with that?
JS: Because it's almost never true. It's not like code rusts if it's not used. The idea that new code is better than old is patently absurd. Old code has been used. It has been tested. Lots of bugs have been found, and they've been fixed. There's nothing wrong with it. SMS: Well, why do programmers constantly go charging into management's offices claiming the existing code base is junk and has to be replaced?
JS: My theory is that this happens because it's harder to read code than to write it. A programmer will whine about a function that he thinks is messy. It's supposed to be a simple function to display a window or something, but for some reason it takes up two pages and has all these ugly little hairs and stuff on it and nobody knows why. OK. I'll tell you why. Those are bug fixes. One of them fixes that bug that Jill had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes a bug that occurs in low-memory conditions. Another one fixes some bug that occurred when the file is on a floppy disk and the user yanks out the diskette in the middle. That LoadLibrary call is sure ugly, but it makes the code work on old versions of Windows 95.
When you throw that function away and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.
SMS: Well, let's assume some of your top programmers walked in the door and said, "We absolutely have to rewrite this thing from scratch, top to bottom." What's the right response? JS: What I learned from Charles Ferguson's great book (High [email protected], No Prisoners [Crown, 1999]) is that you need to hire programmers who can understand the business goals. People who can answer questions like "What does it really cost the company if we rewrite?" "How many months will it delay shipping the product?" "Will we sell enough marginal copies to justify the lost time and market share?" If your programmers insist on a rewrite, they probably don't understand the financials of the company, or the competitive situation. Explain this to them. Then get an honest estimate for the rewrite effort and insist on a financial spreadsheet showing a detailed cost/benefit analysis for the rewrite.
SMS: Yeah, great, but, believe it or not, programmers have been known to, uh, "shave the truth" when it comes to such matters.
JS: What you're seeing is the famous programmer tactic: All features that I want take 1 hour, all features that I don't want take 99 years. If you suspect you are being lied to, just drill down. Get a schedule with granularity measured in hours, not months. Insist that each task have an estimate that is 2 days or less. If it's longer than that, you need to break it down into subtasks or the schedule can't be realistic.
SMS: Are there any circumstances where a complete code rewrite is justified?
JS: Probably not. The most extreme circumstance I can think of would be if you are simultaneously moving to a new platform and changing the architecture of the code dramatically. Even in this case you are probably better off looking at the old code as you develop the new code.
SMS: Hmm. Let's take a look at your theory and compare it to some real-world software meltdowns. For instance, what happened at Netscape?
JS: Way back in April 2000, I wrote on my website that Netscape made the single worst strategic mistake that any software company can make by deciding to rewrite their code from scratch. Lou Montulli, one of the five programming superstars who did the original version of Navigator, e-mailed me to say, "I agree completely; it's one of the major reasons I resigned from Netscape." This one decision cost Netscape 4 years. That's 3 years they spent with their prize aircraft carrier in 200,000 pieces in dry dock. They couldn't add new features, couldn't respond to the competitive threats from IE, and had to sit on their hands while Microsoft completely ate their lunch.
SMS: OK, how about Borland? Another famous meltdown. Any ideas?
JS: Borland also got into the habit of throwing away perfectly good code and starting from scratch. Even after the purchase of AshtonTate, Borland bought Arago and tried to make that into dBASE for Windows, a doomed project that took so long that Microsoft Access ate their lunch. With Paradox, they jumped into a huge rewrite effort with C++ and took forever to release the Windows version of the product. And it was buggy and slow where Paradox for DOS was solid and fast. Then they did it all over again with Quattro Pro, rewriting it from scratch and astonishing the world with how little new functionality it had. SMS: Yeah, and their pricing strategy didn't help.
JS: While I was on the Excel team, Borland cut the MSRP on Quattro Pro from around $500.00 to around $100.00. Clueless newbie that I was, I thought this was the beginning of a bloody price war. Lewis Levin,1 Excel BUM (business unit manager) was ecstatic. "Don't you see, Joel, once they have to cut prices, they've lost." He had no plan to respond to the lower price. And he didn't need to.
SMS: Having worked at Ashton-Tate, I have to tell you the dBASE IV code base was no thing of beauty. But, I take your point. Actually, I saw this syndrome at work in Ashton-Tate's word-processing division. After they bought MultiMate, they spent about 2 years planning a complete rewrite of the product and wasted months evaluating new "engines" for the next version. Nothing ever happened. When a
1 Lewis Levin got his start in the industry as the product manager for MicroPro's PlanStar.
new version of the product was released, it was based on the same "clunky" engine everyone had been moaning about. Of course, in those 2 years WordPerfect and Microsoft ate Ashton-Tate's word-processing lunch.
JS: Ashton-Tate had a word processor?
SMS: Yes, but nothing as good as WordStar, mind you!
JS: Hmm. That reminds me that Microsoft learned the "no rewrite" lesson the hard way. They tried to rewrite Word for Windows from scratch in a doomed project called "Pyramid," which was shut down, thrown away, and swept under the rug. Fortunately for Microsoft, they did this with parallel teams and had never stopped working on the old code base, so they had something to ship, making it merely a financial disaster, not a strategic one. SMS: OK, Lotus?
JS: Too many MBAs at all levels and not enough people with a technical understanding of what could and needed to be built.
SMS: And I suppose building a brand-new product called "Jazz"2 instead of getting 1-2-3 over to the Mac as quickly as possible, thus staking Microsoft to a 2-year lead with Excel, is an example of the same thing?
JS: Actually, they made a worse mistake: They spent something like 18 months trying to squeeze 1-2-3/3.0 into 640KB. By the time the 18 months were up, they hadn't succeeded, and in the meantime, everybody bought 386s with 4 megs of ram. Microsoft always figured that it's better to let the hardware catch up with the software rather than spending time writing code for old computers owned by people who aren't buying much software any more. SMS: WordPerfect?
JS: That's an interesting case and leads to another development sin software companies often make: using the wrong-level tools for the job. At WordPerfect, everything, including everything, had to be written in assembler. Company policy. If a programmer needed a little one-off utility, it had to be hand-coded and hand-optimized in assembler. They were the only people on Earth writing all-assembler
2 Jazz was intended to be the Macintosh equivalent of Symphony for the PC. Like most of the integrated products, it managed to do too much while not doing anything particularly well.
apps for Windows. Insane. It's like making your ballerinas wear balls and chains and taping their arms to their sides. SMS: What should they have been coding in? JS: In those days? C. Or maybe Pascal. Programmers should only use lower-level tools for those parts of the product where they are adding the most value. For example, if you're writing a game where the 3D effects are your major selling point, you can't use an off-the-shelf 3D engine; you have to roll your own. But if the major selling point of your game is the story, don't waste time getting great 3D graphics—just use a library. But WordPerfect was writing UI code that operates in "user time" and doesn't need to be particularly fast. Hand-coded assembler is insane and adds no value. SMS: Yes, but isn't such code tight and small? Don't products built this way avoid the dreaded "bloatware" label? JS: Don't get me started! If you're a software company, there are lots of great business reasons to love bloatware. For one, if programmers don't have to worry about how large their code is, they can ship it sooner. And that means you get more features, and features make users' lives better (if they use them) and don't usually hurt (if they don't). As a user, if your software vendor stops, before shipping, and spends 2 months squeezing the code down to make it 50 percent smaller, the net benefit to you is going to be imperceptible, but you went for 2 months without new features that you needed, and that hurt.
SMS: Could this possibly account for the fact that no one uses WordStar version 3.3 anymore despite the fact it can fit on one 1.4 meg floppy?
JS: That and Control-K. But seriously, Moore's law makes much of the whining about bloatware ridiculous. In 1993, Microsoft Excel 5.0 took up about $36.00 worth of hard drive space. In 2000, Microsoft Excel 2000 takes up about $1.03 in hard drive space. All adjusted for inflation. So stop whining about how bloated it is.
SMS: Well, we've had much personal experience with the press slamming a product we were managing. For example, for years reviewers gave MicroPro hell over the fact it didn't support columns and tables. Somehow the fact that the product would fit on a 360KB floppy just didn't seem to mean as much as the idea that the reviewer couldn't use our product to write his or her resume.
JS: There's a famous fallacy that people learn in business school called the 80/20 rule. It's false, but it seduces a lot of dumb software start-ups. It seems to make sense. Eighty percent of the people use 20 percent of the features. So you convince yourself that you only need to implement 20 percent of the features, and you can still sell 80 percent as many copies. The trouble here, of course, is that it's never the same 20 percent. Everybody uses a different set of features. When you start marketing your "lite" product and you tell people, "Hey, it's lite, only 1MB," they tend to be very happy, then they ask you if it has word counts, or spell checking, or little rubber feet, or whatever obscure thing they can't live without, and it doesn't, so they don't buy your product.
SMS: Let's talk about product marketing and development at Microsoft. How did these two groups work together?
JS: Well, in theory, the marketing group (called "product management") was supposed to give the development team feedback on what customers wanted. Features requests from the field. That kind of stuff. In reality, they never did. SMS: Really?
JS: Really. Yes, we listened to customers, but not through product management—they were never very good at channeling this information. So the program management (design) teams just went out and talked to customers ourselves. One thing I noticed pretty quickly is that you don't actually learn all that much from asking customers what features they want. Sure, they'll tell you, but it's all stuff you knew anyway.
SMS: You paint a picture of the programmer almost as a semideity. But in my experience, I've seen powerful technical personalities take down major companies. For instance, in The Product Marketing Handbook for Software (Aegis Resources, 2006), I describe how the MicroPro development staff's refusal to add the aforementioned columns and table features to WordStar badly hurt the product's sales.3 How do you manage situations like these?
3 Over time, the programming staff noted that requests for this feature from users were dropping. This was absolutely true, because people who wanted this capability in a word processor bought other products.
JS: This is a hard problem. I've seen plenty of companies with prima donna programmers who literally drive their companies into the ground. If the management of the company is technical (think Bill Gates), management isn't afraid to argue with them and win—or fire the programmer and get someone new in. If the management of the company is not technical enough (think John Sculley), they act like scared rabbits, strangely believing that this one person is the only person on the planet who can write code, and it's not a long way from there to the failure of the company.
If you're a nontechnical CEO with programmers who aren't getting with the program, you have to bite the bullet and fire them. This is your only hope. And it means you're going to have to find new technical talent, so your chances aren't great. That's why I don't think technology companies that don't have engineers at the very top have much of a chance. SMS: Joel, thank you very much.
Was this article helpful?
I refined my information, added notes as things came up and took out old or outdated information. Each time I discovered a new little tip or trick as I was making money with EBay, I added that information. Today, I bring home anywhere from 500 to 2000 each week with EBay. Thats not too shabby for someone who was just a year ago afraid to even purchase something from the site.