Over time, large systems come to resemble human civilizations that have evolved over centuries or millennia. Cities are built upon the remains of older cities; cultural norms are handed down from generation to generation; revolutions come and go; architectural styles and tastes change; modernization happens.

As the developers learn and grow, they adopt new design philosophies, techniques, and approaches.

New developers join the project, bringing in new ideas.

Old developers leave the project, reducing the tribal knowledge of the team, but also making it less likely to fall back on the trite excuse of “we’ve always done it this way.”

New tools, languages, and frameworks arise and are adopted.

Consider how a Rails app may evolve over time:

  • The application starts out as a typical Rails app, following the default conventions.

  • The controllers start getting fat, brittle, and hard to test, so we start moving behavior into the models following the “skinny controllers; fat models” advice.

  • The models start getting fat and the tests start running too slow because everything touches the database, so we start moving behavior into service objects.

  • We want to add more dynamic behavior in the browser, so we start augmenting the application with “a sprinkle of JavaScript.”

  • The amount of JavaScript increases over time and begins to get unwieldy, so we move it into its own subtree of the application, adding in jQuery to get a handle on some of it.

  • The jQuery starts getting messy, so we adopt a lightweight front-end framework such as Backbone.

  • We want more and more dynamic behavior, so we migrate to a full-featured front-end framework like Ember, Angular, or React.

  • The back-end keeps growing, so we start splitting out separate services into a service-oriented architecture.

The entire time this evolution is happening, the application keeps growing because we need to constantly add new features. In addition, the team has learned better techniques for architecture, design, and testing.

After such an evolutionary path, our system has become like an archaeological dig site. It has “strata” representing the different eras in its history.

If our team is like most teams, we didn’t have the time and money to refactor the entire codebase every time we adopted a new tool, framework, or technique. New code is written using our latest, best ideas, but the old code is as it was when we initially wrote it. It is not cheap or fast to go back and port all of our ERB-based Rails views into React components or rewrite our JavaScript into CoffeeScript.

The study of human civilizations is called archaeology. Google defines archaeology as:

The study of human history and prehistory through the excavation of sites and the analysis of artifacts and other physical remains.

When new developers come onto the team, they don’t know this history. They don’t know which structures are from which generation of the system. It’s hard to know which patterns to follow and which to migrate away from. They don’t know which parts of the application need to be modernized as they are touched.

If we’re not careful, our developers will only be able to find their way around using “excavation of sites and the analysis of artifacts and other physical remains.” This is not a speedy process.

The more strata in our system and the more outdated our tools and codebase, the slower the team will move when adding new features. New developers will take longer to onboard. It will become harder to attract and keep good developers.

What is the solution to this problem? Should we just keep using our original tools and not adopt anything new? Should we stop the world every time we decide to move forward so we can bring all of the old code up to date with our new ideas? Neither of these seem like a good choice.

Staying in one place is a form of technical debt. The further out of date our tools get, the harder they become to support and maintain.

Moving our tools forward without bringing the old code up to date is also a form of technical debt. Every time we touch some of the old code, we have to remember how we used to do things; or we have to decide that we now need to modernize it, making our current feature that much more expensive to add.

Stopping the world to modernize has a huge opportunity cost. The time spent modernizing is time spent not adding the next feature that will attract new customers.

There needs to be a balance, and the tradeoffs should be made with careful consideration. It isn’t wise to go chasing after every new fad that comes out. But it’s also important to keep your eyes open to see what’s coming next and to make strategic decisions to adopt new technology when there’s a clear benefit to it.

After deciding to adopt something new, the next careful consideration is how much time and effort to put into bringing the old code forward.

I suggest the following approach any time you’ve decided to adopt a new technique, tool, or framework:

  • Make it very clear to everyone on the team what the history of the system is. What are the different eras the system has come through? Which parts of the system belong to which era? What parts of the system are modern? Where are the best patterns to re-use? What code is considered exemplary? This is especially important information for new developers, so it must be continually socialized through the team on an ongoing basis.

  • Write all new code the new way.

  • Immediately modernize strategic parts of the system. There are likely a few features that would really benefit from the new way; they’re probably the ones that tipped the scales in favor of making the change. Take advantage of that. Establish a beachhead for the new way to give it traction and to provide more examples of the new right way to do things.

  • Any significant changes to older parts of the system should include the effort to modernize. You get to decide what “significant changes” mean in your world, but it’s probably smaller than you think it should be. The bar for modernizing needs to be low.

  • Maintain an ongoing effort to bring the old code forward. Perhaps a track that always has a fraction of your team assigned, a background task that people work on a bit at a time, a half-day or day per week for everyone, or a once-a-quarter hack week. Focus on modernizing. Again, you probably need to invest more in this effort than you think you should. Work to minimize the number of archaeological strata in your system by modernizing the oldest parts of the application first.

Using this approach will keep you on a healthy, balanced path that will allow your application to live long and prosper (RIP Leonard Nimoy).