Valley of Refactoring Death – How to avoid it?

Building software systems is hard. In the beginning, everyone is excited and new functionality builds up rapidly. When the project goes further, this speed starts to decline. To compensate, one adds new developers to the team. After some time, velocity starts to ramp up again – until it goes down. 

There comes a time when you cannot simply add more people to the team since they do not improve the situation anymore. Then the product owner hears the dreaded words from the lead developer: “We need to stop and refactor our codebase.”

New ways are needed to tackle technical debt

What happened? Why did the team velocity go down the drain? The situation escalated suddenly: all new functionality development must stop for an extended period to clean up the mess. Is the team simply incompetent? Why didn’t they do it correctly from the beginning? 

This is technical debt in play: The shortcuts done during development to meet the deadline. The wrong decisions done based on lacking knowledge which are hard to pull back later. The missing automated tests which aren’t doing regression testing tirelessly and now it is done by the development team. Bugs slipping to production due to the above mentioned reasons.

Technical debt is the plague of software development, the one boogie monster that all developers dread, and they fight it each day. It is debt just like taking a loan from a bank: it accumulates interest, and both the debt and interest have to be paid off. With technical debt, this accumulating interest can be seen as the symptoms described earlier in this article.

To go fast, one must go slow

But why do seemingly all non-trivial software development projects end up in this same situation? What is the path that leads to the Valley of Refactoring Death? As it usually is, the road to Hell is paved with good intentions.

When a new project starts, there is no technical debt. It’s a clean table, and the team can proceed with agility and speed. Thus, they set off at full speed and produce new functionality rapidly. At this point, they set the expected level for their velocity to the project management. 

Stakeholders are happy and expect to see this level of progression at all points of the project. The problem is, however, that the team can not maintain this level of speed. To extend the same velocity level, they need to keep on making those shortcuts. Not writing tests, for example, accumulates technical debt at an ever-increasing pace. But at some point, the velocity will plummet, hard.

To go fast, one must go slow. If a software project will last longer than a few months, it really doesn’t matter how quickly the team produces new features in the beginning. The only thing that really matters is how much value the delivered solution can produce over a longer period of time.

Experimenting and building the final product are two very different things

There is, however, value in the fast initial burst of features. It allows stakeholders to get a better understanding of the problem at hand. It inspires people. The issue lies in how the team creates this value.

To get the initial inspiration and boost understanding, one doesn’t have to start creating the final solution. That’s where prototyping tools, such as Figma or InVision, step in. They allow the creation of clickable prototypes very quickly. This empowers quick prototyping and innovation. If one needs a heavier approach, then doing a Proof of Concept – a quickly built throw-away solution to prove that solution is viable – can help give the confidence needed to commit the organization to a full-scale software project. 

Do not make the mistake of taking a Proof of Concept implementation and start building on top of that. You’ll make the PoC phase slower, and the foundations of your final solution are not built properly. It will lead to remarkable initial technical debt, which drags down development for a long time.

Refactoring from day one

Both prototyping approaches differ from the actual development phase and should be treated accordingly. When the team starts actual development, the key factor to remember is that one should start refactoring from day one. 

From my experience, I would say a good rule of thumb is to use 60% of the time to implement new functionality and 40% of the time to improve the existing system in different ways.

But here lies the final trap that leads to the Valley of Refactoring. In agile projects, teams typically manage work in a Product Backlog or some similar prioritization-based approach. The Product Owner – who is strictly a business-oriented person – manages the backlog, and rightly so: delivering business value should be the number one priority for the whole development team. After all, who would be better to guide towards this goal than the person whose primary job is within this domain? 

This person, however, is not a technical person. Thus, they lack an understanding of technical issues at a deep level.

Split the backlog

To balance these aspects, many organizations have formed the roles of two product owners: business and technical. Sometimes this approach works, sometimes not. It is completely up to the people who have been assigned to these roles and how well they can work together.

What I’m proposing next is not to split the role of the Product Owner but to split the backlog.

After all, if the Product Owner knows the business problems best, then who else than the development team would know the technical problems best? But how can one prioritize a technical task over a business-oriented one? The simple answer is: you don’t

To solve this problem, one should add a Technical Backlog adjacent to the Product Backlog. Both are prioritized independently of each other. Where the Product Owner has the main responsibility of prioritizing the Product Backlog, then the main responsibility to prioritize the Technical Backlog goes to the lead developer or architect.

When picking tasks from these backlogs, one should follow the simple rule of thumb: roughly 60% of tasks are picked from the Product Backlog and 40% are picked from the Technical Backlog.

Keep your backlogs lean, and keep them concise

Now, obviously, tasks might be of different sizes but along the way they will be balanced. There might be cases where technical tasks are vastly larger than product tasks, or vice versa, but especially over time, when a team finds its own ways of working, this too will even out. 

And even if this happens, the business value is still delivered constantly. The team ensures this by picking 60% of tasks from the Product Backlog. Simultaneously, since 40% of tasks are picked from Technical Backlog, it ensures the system keeps on improving and technical debt is paid off promptly which prevents it from accumulating exponential interest.

This ensures constant velocity – a thing that also provides predictability. That is definitely something that business needs.

Furthermore, this 60-40 rule isn’t that strict. There are times when you pick more tasks from one backlog. I’m also not saying this split is perfect for all teams and scenarios. The main thing to take from here is that the team needs to provide business value constantly, and they also need to keep on improving the technical foundations, metrics collection, monitoring, and several other things all the time. 

Start from a 60-40 split and adjust accordingly. But be careful not to start doing only tasks from one backlog!

Quality over quantity

When splitting the backlog into two, one must be even more aware of the inherent problem with the whole backlog approach in general. Backlogs which have 100 items in them are not useful.

It is another type of debt in play. As a person responsible for maintaining the backlog, your main job is to prioritize the backlog. If you have 100+ items in your backlog, then it is almost certain that your backlog is not prioritized well enough. Remember, just-in-time approach is great here too: concentrate on quality, not on quantity. 

Items which have been added to either backlog four months ago are most likely not relevant anymore, or at least the world has changed underneath. Your job is to ensure the team is doing things that deliver the maximum value, not to fill backlogs all day long. Keep your backlogs lean, and keep them concise.

So to avoid the Valley of Refactoring Death, one needs to refactor all the time. Split your backlogs, prioritize them separately, keep them lean and pick tasks from both – all the time. 

You’ll end up with a better quality system, constant velocity, a happier development team, and predictability for the business.