How to use hot spots to prioritize your technical debt

In this blog post I will describe an approach to prioritize technical debt in an Enterprise application which is under continuous development. But why should you prioritize it at all?

Let me start by explaining a little about technical debt. Over time the technical debt is built up and it can actually be much larger than you would imagine. Maybe the debt is caused by a mistaken code practice which might not be the right fit. This for example, could be a switch case over an enum where you do not have a default, or it might be using X509Certificate2 (.Net System.Security.Cryptography.X509Certificates) without calling the Reset method afterwards. As a product owner, or other stakeholder, you may suddenly be made aware of the technical debt. This could for example, be due to a transition from 'convention' (agreements on how to code) to 'structure' (addition of static code analysis tools, such as Resharper or SonarQube).

It is necessary to start prioritizing the technical debt because you rarely would have the time to deal with all the debt at once. Even if you had the time, it is relevant to decide whether your goal is to completely eradicate all the debt or not - perhaps you consider that the time is still better spent on further development on the application itself.

Let me introduce an example application. It has many hundreds of thousands of code lines and has been developed over several years by a large group of skilled developers who have been working by the test driven method, with great code principles and code reviews ('convention'). By letting SonarQube do the static code analysis, the focus was suddenly put on technical debt. where 31,000 issues were identified. They were distributed on severity like this:

The time dimension

First of all, we must have at look at the time dimension. In the following, the 'old code' is defined as code that has not been edited for a long time and at the same time has been in production by your customer for a long period of time. The 'New code' is defined as code that has been written or edited recently and never, or only recently, has come into production by your customer. A 'real problem' is defined as an issue experienced by the user of the application.

'Old code' can have technical debt with high severity, while 'new code' can have technical debt with lower severity. The latter is most important to address, as a 'real problem' in 'old code' would have been reported back to you from a (dissatisfied) user.

By analyzing the commit history in your code repository (for example GIT), it is possible to identify your 'old code' and the 'new code'. The analysis is made step by step. First at (code) project level. Then at file level - you could even analyze at the line level if you wish.

The distribution of 'commits over time' between two releases divided per project, in the example application is distributed as follows:

The analysis will ignore the 'old code' which is found at the right of the graph. Instead, the next step is to look at the projects that have the most commits - those are the projects in the red rectangle. Now the files with the most commits are found.

In the example application here, the files in the red rectangle have the most edits, in the projects with the most commits. Therefore, they are our identified hot spots.

The human dimension

Next, we must look at the human dimension. Think of two teams, 'Team Rock Solid' and 'Team Rookies'. Team Rock Solid consists of experienced and skilled developers, while Team Rookies consists of less experienced developers, some of whom have been replaced over time.

By analyzing commit history - again - it is possible to identify which teams have made the most changes in a project and in a file.

In the example application below, eight teams have committed code to the three projects with the most commits since the last release. This is how the distribution of commits per team looks:

Project A has a lot of commits by Team Rookies. It is a potential risk and therefore a hot spot. Project B looks better as Team Rock Solid clearly has the most commits in this part of the code, but it is a potential hot spot because there are so many different teams that have changed code here. Project C has commits from very few teams and is not a hot spot.

Combine the time dimension, the human dimension and static code analysis

With the time dimension and the human dimension in mind, it is easier to prioritize. Choose a hot spot and get rid of the technical debt in that code piece. You will need to know your code, your history, and your developers well enough to select the most important hot spots first. Then within the individual hot spot, the problems are solved in severity order.

In the example application, the distribution of technical debt for a selected hot spot was as follows (this made it much more manageable to deal with):

A hot spot analysis can be used for more than just prioritizing technical debt

The method of finding hot spots can also be used for other analyzes.

One example is finding hot spots that need refactoring. This might be the case if a project is continuously edited by several teams. It may be a symptom of not following the 'Domain Driven Design' principles. (Domain Driven Design is a concept where the structure of code is divided by business domain. A Lego brick has, for example, a serial number and a weight in the production hall, but in the children's room it is described by its shape and color).

Another example is finding hot spots with technical debt, where there is a single team who did most of the 'new code'. It may be worth considering strengthening the team with training or an experienced developer.

So, how do you and your team get started?

You might think that your organization is in control of your technical debt - and if you really are, it's absolutely fantastic! But to actually know this for real, you have to start by doing a static code analysis on a single application and thereby become aware of the extent of your technical debt. I would choose the largest and / or oldest application you have, to begin with. This is where you are most likely to find the greatest amount of technical debt.

After the previous steps, you can automate the static code analysis so that it runs at each commit or at least once a day.

At the same time, make a clear baseline and agree that no new technical debt may be introduced from now on.

It is also a good idea to spread the static code analysis to other newer and smaller applications, to prevent technical debt before it becomes unmanageable. It is better to know the technical debt and be able to make decisions on an informed basis, than to live in the blind.

Therefore: Address the technical debt BEFORE it becomes a critical issue. This will benefit both your application and your business. Today, for example, could be a great day to start - and I can only recommend you use the described attack-method to prioritize the debt. This way you 1) get the overview and 2) get the riskiest part of the technical debt identified and removed firstly.

Enjoy!