“Organization doesn’t really accomplish anything. Plans don’t accomplish anything, either. Theories of management don’t much matter. Endeavors succeed or fail because of the people involved. Only by attracting the best people will you accomplish great deeds.”
Software engineers love rewriting old code. I’m a software engineer, and whenever I look at my own older code (as in more than a month or so) I get an instant itch to rewrite it. I’m a bit embarrassed by it, and I think that for various reasons, I can do it better now. It happens even more when I look at somebody else’s code, because I also assume they must not be nearly as clever as I. When something’s WORKING we still want to rewrite it, so when a product or component is causing maintenance or performance issues, there is usually a very strong case being made that it needs a ground-up rewrite. We say things like “we won’t be able to improve the graphic performance unless we completely rework it with a different set of libraries” or “this should have never been developed in Perl, we can’t scale unless it gets redone in C++” (direct quote, by me several years ago). Or just “this program/class is an incoherent mess, I’m writing a replacement”
Don’t do it. This will probably kill your company.
Even if it has issues, your production code is the result of say two years of development, of which something like 20% was the fun part of writing shiny new code to solve a problem, and the other 80% was a long slog of tweaks and bug fixes. Guess what, when you replace that ugly code with your new shiny code you just gave up hundreds if not thousands of man-hours spent improving things. You re-introduced old bugs and, worse, added new ones that you don’t know about yet.
And the more strategically horrible part of this, is say you spend 2 months writing the new version. Don’t forget, it will take you twice as long as you think. During those two months you continue running the old code, but you stop maintaining it because it’s going to be replaced by something amazing “any day now”. In my experience, you end up with a legacy system that has fallen way behind where it should be, thanks to lack of attention, and a new system that can’t be successfully rolled out because in a production environment it’s not performing as expected. You now need to invest another significant chunk of time finding and fixing the bugs in it, which you had already done on the old version.
Now you just wasted huge amounts of your most precious resources – time and momentum. Your customers have wandered off and your competitors just ate your lunch.
The other trigger for this, where there is a legitimate need to do a rewrite, is usually for performance issues. Such as replacing the Perl code with C++ (bad Jeff).
What should be done instead? Both situations will arise…
When you have legacy classes/components/scripts that are so full of special cases that they can’t be maintained without causing stability problems you DO NOT REWRITE THEM, you pull out functional parts bit by bit, and you name variables and functions bit by bit until it’s clear and well-organized. And at no point did you add or remove a single line of code that was not there previously. The code is unchanged, it’s just laid out much better and you can now work in it with confidence. All of the little fixes and special cases are still there and, perhaps most importantly, there was NO point in time when the code being edited could not be used in production.
Similarly, when you have to replace a program or component in its entirety, you approach it with a refactoring mindset but it’s a bit more complex and has a bit more risk. In this case, sooner or later you’re going to replace the component with a new one, and it might be in a completely different language, so you can’t use the classic code refactoring techniques. Instead, you get as close to that as you can. First, you work to lock down the real functionality of the legacy code with an exhaustive test suite. This will likely take you longer to do than actually writing the new one, but it’s the only way you can ensure you stay true to the current behavior. Look at the code as you set up the tests, ensuring that all the funny special cases and outliers are tested. Generate tests from three months of live usage, ensuring that you handle everything that users/environments actually try. Next, keep in the refactoring mindset by making the minimal changes possible. For example, if I were moving a program from Perl to C++ I would actually have the legacy Perl program call a new C++ program and use its output. This way I can refactor my code from Perl to C++ line by line and function by function, until eventually I still have a Perl program that actually calls a C++ program to perform ALL of its functionality. All the Perl program does is invoke the new version and return its output. Now you can take the final step of replacing the tiny Perl with the new C++ and everything should continue as before.
This process allows you to accomplish BIG architectural changes without ever having code that could not be in live use. If something unrelated breaks, you can fix it on the latest version and push it out, and you’re not maintaining “production” and “future” versions separately.
If I were undertaking this right now, I would also like to pair it with continuous deployment, so that as I made each tiny, safe, change it went live and any errors would be detected immediately. Also, as mentioned, refactoring is best done with the safety net of a great unit and integration testing framework.
If you want to learn more about refactoring, most learn from The Bible: “Refactoring: Improving the Design of Existing Code” by Fowler, Beck, and (apparently) a dozen others. I’ve also learned some great techniques on using this practice to improve architecture from “Refactoring to Patterns” by Josh Kerievsky. Also, Martin Fowler has a website about refactoring at refactoring.com that appears to have some good resources. Let me know of any other favorite resources in the comments. I follow Kent Beck at @kentbeck.
Joel Spolsky (@spolsky) wrote a fantastic article on these problems TEN YEARS AGO and I still see this happening. I think people need reminded of it regularly.
If you write software or manage people who do, you need to go read his old posts now. It’ll make you look smart 🙂
Startup Reading is an ongoing reading list of articles and resources that I think will be of great value to startups and entrepreneurs.
full PDF $6.50 from HBR
There are three common patterns of stressful conversations in this article, and I’ve had the pleasure of experiencing each of them, almost exactly as they play out in the examples. Stressful conversations aren’t something that can be avoided, but in fact are probably some of the most important conversations you will ever have with your co-workers, partners or employees. The times when emotions are running high and important information needs to be heard are critical make-or-break moments for your relationships and, over time, your company.
Luckily, there are a few key findings from behavioral research that give us tools to use when preparing for a stressful conversation or managing one that’s gone seriously sideways. The situations and tactics are well described in the article, but a few brief extractions are:
– Disarm by acknowledging responsibility for your part of a problem, even if it’s in the form of “I feel like I’ve let you down by not bringing this up in the past, because I value our relationship and your contributions here, but we need to rectify this issue…”
– Disarm by restating your intentions – often people hear something completely different from your intentions, so it’s not necessary to give ground but instead work to clarify what you really mean: “I can see how you took that from what I said. That wasn’t what I meant though, so let’s go through this again.” – Don’t argue with them about their perceptions, instead take the responsibility for aligning your words with your intentions.
– Fight tactics, not people. Some people use aggressive “thwarting” tactics that prevent you from making your points. The best way to neutralize a tactic like this is to name it, as people are generally not comfortable raising the bar and continuing to be aggressive once it’s out in the open.
A worthwhile read, that pretty much anyone can use to improve their conversational abilities where it’ll do the most good.
I was having a beer with a friend yesterday, discussing how I’ve crammed my life close to the bewaking point with a toddler, a new baby, an MBA program, a job search, some mentoring and networking, and he asked me what pearls of wisdom I could bestow on him for when he finds himself in a similarly hectic spot. And I came up blank. It’s not that I haven’t found shortcuts and processes that let me handle this without losing my head, it’s that I haven’t stepped out of the flow, gone to a 10,000 foot view and checked out what’s going on. This reminded me of something one of our profs said when discussing innovation and creativity: there can be no innovation without organizational slack. If you (I) don’t stop fighting fires or attacking your task list, you’ll (I’ll) never improve our abilities/capacity to deal with the situation. No matter how busy you are, if you don’t stop to breathe and evaluate your activities and formulate some strategy, you’re going to get demolished by something you didn’t see coming. Keep your head up!
So today’s advice / resolution is to create time for slack. Even with my schedule being crammed to 30-second intervals (I’m working out, doing dishes and debating preschool with my wife while I write this – partially kidding) I figure I can make the time to sit alone at a coffee shop or my front stoop for an hour every week and let myself think about bigger pictures than my todo list. In fact, if you’re busy like me, I think it’s required that you put it in the schedule. That’s what they tell us about workouts and it applies here – put it in the calendar, make an appointment to do it.
Sometimes though there just isn’t time. And I think when that happens, in a lot of cases, you can move towards it incrementally. If you’re fighting fires 18 hours a day, and your organization or family is always in crisis mode, there’s probably something wrong. You might not be able to go ponder what that is without seeing something else blow up, so just ask five questions when you fix the problem. Address the immediate fire, sure, but also use this technique to tease out the root causes and commit to making a corrective action at each level of the analysis. This way you slowly, incrementally improve your processes and behaviors, instead of just dousing a single flame. Over a few iterations you will start to see the number of fires decreasing, and you can pop up to 10,000 feet for a few seconds for a clear view.
In a recent HBR article (What Really Motivates Workers) it was revealed that workers are motivated less by the usual suspects (incentives, recognition, …) than we thought and that in fact progress in their work is the key driver of daily satisfaction. How can we take advantage of this in software development to crank up real and perceived progress, and keep workers happier on a daily basis?
First, it seems we’re an industry uniquely situated to take advantage of creating perceived progress. I don’t mean false progress, but with the real time metrics available such as compile errors, bug databases, unit tests passed or test runtime, we have the tools to demonstrate progress in several ways depending on the project. The trick may be to identify the ones that are most valuable to your product, and to keep some perspective on how much you emphasize them. If you keep cheering up your team by pointing to the diminishing bug list, you’ll have a hard time rallying them when a large dump of bugs comes in from new testing. Keep it focused on customer satisfaction and percent of code under testing.
Second, there are methodologies and tools for software engineering that support constant progress and prevent the kind of backslide that kills morale and motivation. Specifically, I’m thinking of agile methods that cut out time spent on exhaustive planning and requirement specifications that doesn’t feel like progress to the person writing it, and creates the opportunity for big negative progress when it turns out they’re misguided. Then tack on test driven development to allow constant, incremental additions while locking out the possibility of backsliding again, and continuous integration and deployment to keep your work rolling forward instead of halting and going back.
These are very pragmatic, and well tested methods that, used together, will hugely increase the number of days your team can go without a big backslide, adding incremental improvement or capability day in and day out. And, according to the article, this is a key driver of motivation. So in addition to the benefits we already knew about (faster results, less wasted time, better tested code, no ‘ocean boiling’ integration and release cycle) we can now say that agile methods keep your team happy and motivated.
What other techniques support constant progress?