Are you capable of watching your technical debt?
Bob McIlree wrote an interesting blog at the start of October about watching the amount of 'technical debt'. The term is one that I had not previously encountered. Ward Cunningham first defined it back in 1992 and Martin Fowler provided an additional and perhaps slightly clearer perspective in 2003.
"...You have a piece of functionality that you need to add to your system. You see two ways to do it, one is quick to do but is messy - you are sure that it will make further changes harder in the future. The other results in a cleaner design, but will take longer to put in place.
Technical Debt is a wonderful metaphor developed by Ward Cunningham to help us think about this problem. In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt. Like a financial debt, the technical debt incurs interest payments, which come in the form of the extra effort that we have to do in future development because of the quick and dirty design choice. We can choose to continue paying the interest, or we can pay down the principal by refactoring the quick and dirty design into the better design. Although it costs to pay down the principal, we gain by reduced interest payments in the future.
The metaphor also explains why it may be sensible to do the quick and dirty approach. Just as a business incurs some debt to take advantage of a market opportunity developers may incur technical debt to hit an important deadline. The all too common problem is that development organizations let their debt get out of control and spend most of their future development effort paying crippling interest payments."Technical debt is something that we should always bear in mind irrespective of the economic climate (although it is obviously particularly important at present!). Given the common trend to apply quick and dirty fixes (making the minimum monthly payment!); I have little confidence that many IT organisations take a blind bit of notice. The fundamental problem comes down to corporate DNA and the working culture of the IT organisation.
Let us explore the financial debt analogy and equate the practice of paying off technical debt with clearing credit card debts at the end of every month. Typically, people who follow this practice tend to be fairly organised and disciplined when it comes to financial management and it wouldn't be at all surprising to learn that they exhibit similar traits in other aspects of their lives. Similarly, organisations that are capable of managing technical debt well probably exhibit a disciplined and mature approach to the management of other aspects of their business too. Therefore in the same way that an individual manages their credit card is probably a fair indicator of the efficiency to which they manage their general finances, so an organisation's management of its technical debt is likely also to be a good indicator of its approach to IT management as a whole.
Furthermore, just as many of us don't actually manage our credit card debts I doubt that there are very many organisations that are effective when it comes to the management of technical debt either. Given the current (and historical) performance of many IT projects in many organisations it's highly likely that they have amassed a significant technical debt that is unwieldy and poorly managed.
This is more than a hunch: there is evidence to suggest that technical debt is building. For example,
Original Software recently released the results of a survey that showed that "40% of CIOs admit corporate indifference to their own software quality". This suggests that effective management of any technical debt is unlikely to be a concern to a sizeable percentage of organisations. The stark truth is that most businesses are simply too lax and lack incentives to monitor and, more importantly, manage their technical indebtedness. There is already ample evidence to demonstrate that all manner of cost savings and productivity benefits can be achieved if organizations just did things right with the support of appropriate tools. But we still see significant IT failures and IT teams still end up fire fighting for any number of reasons, with some even attaching a certain amount of macho cachet to their ability to fighting fires rather than preventing them in the first place. Clearly, conditions are ripe for building up an unhealthy technical debt.
That said, I definitely agree with
Bob McIlree when he states that there are lots of valid reasons for incurring technical debt. But I would add that in doing so, organisations need to apply a healthy dose of risk analysis and be their own 'bank manager', working out the likelihood of the debt being paid off in an acceptable timeframe. This will require organisations to look at the history of IT successes and failures and to assess the assets they have at their disposal to ensure that the technical debt is paid off. For example:
- take a cold hard look at the effectiveness of IT processes
- ascertain the discipline and strength of character of the IT organisation to communicate sensible and pragmatic solutions to demands made by the business
- determine what tools are in place to prevent the debt spiralling out of control
The other factor to consider is where the technical debt is being incurred. How critical is the solution and what are the implications if the debt isn't paid off, or worse still, if it increases? Is it likely to cost you your 'business'? The examples of toxic technical debt highlighted by Bob together with the actions outlined above highlight the dangers and provide a good guide as to the likelihood of an IT organisation being prone to mismanaging their technical debt.
Ultimately, it always comes back to maturity and discipline in every sense of both words. If your organisation is one that has invested soundly to put in place the right tools and policies to effectively manage your software delivery process (even if it is outsourced); and if takes calm measured steps and thinks long term, then it is likely to be able to effectively manage technical debt.
If your organisation isn't like that then the following need to be in place:
- quality management facilities that are integral to all phases of the software delivery process;
- a robust intelligence, monitoring and analytics frameworks that is tied to key KPIs, SLAs and other relevant metrics;
- effective collaboration amongst the software delivery team and with business units; an integrated framework of (most likely heterogeneous) tools;
- an end-to-end software delivery process, support by appropriate tools, that helps to clearly identify the business requirements that are driving the software development.
It perhaps comes as no surprise, given the above, that Application Lifecycle Management (ALM) should help to alleviate technical debt (or at least smooth out the payments).
I recently spoke with
Clive Howard of
Howard Baines who, like me was unaware of "technical debt". He wishes he had been as he's had debt management conversation with clients on many occasions.
"None of the stats surprise me. I think the situation gets worse as development becomes "easier" and faster due to mainly better tools (anyone can build an application with development tools like Visual Studio 2008, it just wouldn't necessarily be a good application).
The nuances of development can be completely lost on many clients. In some cases made worse by clients thinking that they know better and opting for the cheaper path. It is always frustrating the way that well architected and coded applications deteriorate over time due to sloppy decisions later when shortest timescales and lowest cost become the immediate priority with no thought to the additional time and cost later."If organisations actually consider upfront capital expenditure together with long-term operational expenditure, perhaps they will have a better handle on understanding the true nature of technical debt.
Labels: ALM, development, technical debt
