Controlling technical debt in a software driven world

This change requires companies to add software development as a core competency to stay ahead of the competition. Our software driven world seems to have an unquenchable appetite for new software applications, and these applications are growing in size and complexity so fast that development processes are struggling to keep pace. Our inability to build high quality software efficiently has resulted in the accrual of huge amounts of technical debt, a metaphorical term for latent defects introduced during system architecture, design and development.

Software by nature is as easy to change and as a result easy to break. It is common for the code change to support new features and bug fixes to break existing features - additionally, the constant churn of code results in ugly and hard to maintain applications, which further reduce development efficiency. Historically, when applications had been patched to the point where they were unmaintainable, organisations would declare ‘bankruptcy’, and embark on a complete re-write of the application. As applications become larger and more complex, a complete re-write is no longer a realistic option. The only option is to address the technical debt in their legacy code bases to extend their useful life.

Technical debt is caused by four main problems (shown by the Technical Debt Quadrant below) - poor code architecture, overly complex code, inadequate testing, and the lack of documentation. Debt increases over time primarily because of the time to market pressure to get new features and applications to customers as quickly as possible. What organisations don’t realise is that the time or cost saved by taking development shortcuts will negatively affect the cost of maintaining that application over its lifecycle. Over a nominal 15 year application lifecycle, the initial development costs average just eight percent of the maintenance costs. So the apparent time and cost savings achieved during development by incurring technical debt are dwarfed by the lifecycle costs.

Some amount of technical debt is unavoidable and benign, but other types of technical debt is dangerous. ‘Prudent’ debt refers to debt that is deliberate and visible, for example incomplete low level requirements might be ok early in the development phase as a proof of concept is developed, and multiple architecture approaches are investigated. This deliberate technical debt can be managed for example, by ensuring that the design documents are complete once the architecture is decided. ‘Inadvertent’ debt such as lack of test cases for critical parts of an application has the potential to lead to unhappy customers, brand damage or, in the case of safety related applications, loss of life.

With all industries under pressure to produce more with less, how can the technical debt time bomb be defused? The first step for any organisation is to understand the problem, by capturing data. Interestingly, three of the top four causes of technical debt can be easily understood by simply installing software quality metrics gathering and reporting tools. The key metrics to capture are Cyclomatic Complexity which measures the total number of paths for each function, Code Coverage, which measures the percentage of the source code that has been tested, and Comment Density, which measures the documentation of a function or source file. By capturing simple metrics and trends you will be able to easily identify the worst offenders in your code base, for example highly complex sections that are poorly tested.

Once you identify where the problems are, quality improvement can begin using a variety of strategies, such as ‘Shift Left Testing’ - performing testing earlier in the lifecycle and ‘DevOps’ - performing continuous testing and delivery. Both of these strategies require a test automation platform to allow all stakeholders to collaborate on test activities. Once testing is automated, an additional benefit is a reduction in total test times by integrating a continuous test strategy with your existing continuous build tools so that the correct sub-set of tests is run as each source change is made.

All of the above techniques rely on some baseline of tests that can be measured and extended to improve quality, but what happens when a legacy code base has few, if any tests. This is more common than you might realise. In fact groups that have not previously measured code coverage are often appalled at the low level of coverage achieved by their existing tests. Often these legacy applications have been proven by use, they have been in the field being used by customers, but there are very limited tests to formalise what the applications actually do. In this case, baseline or characterisation testing to capture existing behaviour of the software can be a valuable approach to prove that updates and modifications to the code don’t break existing functionality. Using Automatic Test Generation technology to build tests backwards from the code is a key technology to solve this problem. While this approach would not be useful for new code bases it can be a critical component for refactoring legacy applications with confidence.

In conclusion, groups that are tasked with the maintenance of legacy code bases should be proactive with identifying and managing their technical debt. Investments in a collaborative, process-driven, and automated software test strategy will reduce maintenance costs, reduce time to market, and improve customer brand loyalty.