The book 97 Things Every Software Architect Should Know is a useful collection of personal recommendations from experienced software practitioners around the world, and it contains some excellent advice for any thinking person engaged in software systems engineering.

However, it’s clear that even as recently as 2009 (when the book was published), there was very little focus amongst those who identify as software architects on making sure the software we design and built is operable by the operations teams on the “front line”. In fact, the only contributors who directly touch on software operabililty (aside from Michael Nygard, naturally) are  Rebecca Parsons,  Mncedisi Kasper, and Dave Anderson; four contributions out of 97.

Traceability

Dave Anderson of Liberty IT asks a telling question (p.130) of our software:

Can Ernie the Emergency Bug Fixer – who has never seen the code before – jump into production, diagnose a fault, and put in a fix?

A core feature of operable software is its traceability – the ability of system operators to trace the system’s execution via log messages, metrics, and other application output.

Undertake Technical Testing Early On

Rebecca Parsons (in It’s Never Too Early to Think About Performance, p.26) highlights the need to bring forward performance testing (and by extension, I’d argue, all kinds of technical and “non-functional” testing) much earlier in the software development cycle than most organisations typically do.

If performance is going to be an important architectural and design criterion, then performance testing should begin as soon as possible.

Leaving  performance testing to the end of the timeline means that we may have no time to correct major performance problems. If we measure the performance of our software at the latest by half-way through the project timeline, we still have the remaining half of the time to completely change the implementation if needed.

Likewise, if we leave measures of operability until very late in the project, we may have no time or budget to re-work the software in order to address the concerns of the operations team.

In my experience, the best way to ensure that software is operable is to have the product development team write the first draft of the System Operation Manual (aka Run Book). That’s right: the software team should demonstrate that their software is going to be operable by supplying a draft operation manual to the operations team, rather than leaving the operational details to “those guys in the ops department”.

Do Not Sacrifice Operability to Satisfy Budgets

Michael Nygard (whose book Release It! is essential for anyone concerned with software operability) makes an interesting point in You’re Often Negotiating More Often Than You Think (p.18): project sponsors typically come to a budget meeting expecting to achieve a trade-off, whatever that trade-off is – the details are not interesting to them. So, when the sponsor asks “do we really need that second server”,

[t]he correct way to answer this is “Yes. We do.”

The mind-sets of engineers and budget-holders are different: collaborative for engineers, and win-lose for bean-counters. Poor software operability can cost the organisation many times the amount saved by not designing and building for good operability, so insisting on operability as a first-class concern is part of our job.

Focus on Application Support and Maintainability

The contribution in 97 Things which most clearly addresses the concerns of software operability is the piece by Mncedisi Kasper, called Focus on Application Support and Maintainability (p.114):

Since over 80% of an application’s lifecycle is spent in maintenance, you should pay a lot of attention to the problems of support and maintenance when you’re designing.

Mncedisi highlights the mismatch between the skills and focus of a typical software developer (and software architect) and those of a typical application support person; software is too often designed with the needs of the developer in mind, rather than those of the support engineer.

For software to be properly operable, the needs of the operations and support teams should be understood and valued as key inputs to the software development process; Mncedisi recommends bringing the software support lead into the software team as a core member early on. This kind of cross-team collaboration is becoming more common as DevOps thinking takes hold, but this might require a change in the organisation’s budgeting or project process to achieve.