Many of us work on systems which have grown over the years to become significantly complex. Most likely, in many of these systems, there are a lot of messes to clean up—inconsistent design themes, “spaghetti” code, questionable database schema elements, and so on. What do you do if you’re hired to maintain and enhance a tangled mess of code, most of which is still delivering significant value to the client, but which is rough around the edges, broken in some places, unstable in others, and so on?
I spend much of my development time on one such system—used by a small company of ~20-30 users all day during the working week. What follows are some lessons learned from working with this beast.
Stability is Key
Every decision you make needs to be based on how it will affect active users. Make sure your basic infrastructure (servers, etc.) are more than beefy enough to handle your usual user load. When building new functionality, don’t take risks that might cause instability or system-wide crashes. If you need to build intensive routines that churn through your data, try to keep them cordoned off in an area where their intensity won’t affect the day-to-day operations of the system—like in a data warehouse or another separate set of data tables.
Keep extensive logs, implement major changes during light traffic, and have scheduled downtime for major changes.
Don’t Be Afraid to Delete Stuff (Carefully…)
I find it well worth my time to spend an hour a week or so carefully determining what’s no longer used—schema elements, scripting/code, and the data itself—and then actually deleting (and/or archiving) this stuff, so it isn’t “there” in the live system anymore. Later on, I thank myself when I need to ask questions like, “Where else is this field used?”
Fighting Band-aids & Technical Debt
In one of his brilliant presentations, some years ago, design and usability expert Don Levan introduced me to the concept of Technical Debt. It comes down to the question of “do I band-aid this problem, or do I really fix it the right way?” The idea is that, every time you band-aid, quick-and-dirty-fix a problem, you are kicking the can down the road—accruing “debt”… some day you are going to have to fix the problem the right way, and by that time, it may be much worse. (Debt accrues interest.) This is acutely true in large, complex systems, where band-aids build up quickly and make future programming more difficult, inefficient, and confusing for new developers. Keep these considerations in mind pretty much at all times, even if you might not have the resources/time to do a given job or solve a given problem “the right way” at a given moment in time.
One of the ways I try to fight band-aids is: as I gather user requests and so forth, I try to clump them together into larger jobs. The larger job is a non-band-aid solution to what might be many small band-aid-like problems. It might take me longer to work on, but I feel good about telling the client that “yes, the solution to that problem is coming as part of this bigger thing I am going to build.” And it always feels good when I get to finishing that larger project, satisfying the client’s needs, and doing it in a way that lays the groundwork properly for future work and extensions, should they be needed.
There are a lot of technical tricks to maintaining complex systems but these overall guidelines have helped me stay focused.