Software Failures

Having been involved in software development for 40 years in support, engineering, project management and as CEO, I can profess to being confounded that the electronics industry is still being undermined by software failures. By electronics industry I know that I am using an out-of-date description, but how better to describe the wide range of services which are either directly or indirectly dependent on electronics with software running in it. Back in the 1980s these products were fairly specialised but now it can apply to the wideness of cloud computing to the narrowness of the control and payment systems in your local coffee shop’s barista machine. And in most cases communicating with the outside world.

I defy any of you reading this to give me a case of a seamless electronics product development. One that hasn’t been stressful as a result of the software problems, whether cost and time overruns or unexpected support costs as a result of failure. Certainly Jaguar Land Rover (Evoke recall in the USA) or Lockheed Martin (F35 incorrect target detection) would find it difficult. Through the 100 plus Aerospace and Defence projects I’ve been involved with, I cannot think of one which was free of software issues. In fact I can put my premature grey hair down directly to the Merlin Flight Control System (frame time extension) in 1992.

So why in these advanced times has the problem not gone away. In fact if anything the problem has gotten worse. If you are tempted to disagree then have a look at last year’s copy of Computerworld magazine which lists 33 of the top software failures in recent history creating havoc at banks, airlines and the NHS, doing billions of pounds of damage and devastating disruption. In future blogs it might be worth a look at some of these in a bit more detail.  

Part of the reason is obvious. Things are getting more complex and interconnected so the software is getting bigger. Developments are now measured in tens of gigabytes when the early Airbus Flight Management System would barely scrape a few hundred K. Unfortunately what hasn’t changed is the developer’s inability to make the software development agile. (easy to change and verify). As a programme manager, the biggest fight I had was with those who wanted to change the system requirements, usually after we had generated the software code and probably tested and certified it on target hardware. Unfortunately this was usually the customer (always right) or the marketing department (always get their way).

Oh for a way of automatically driving such changes through to the code and hardware without having to go through the laborious, time consuming and costly re-test and re-verification. Who on earth wants to repeat all that specialist test and (flawed) manual review stuff? Believe me no-one, as the process is repetitive and soul destroying and inevitably corners will be cut. It can also constitutes around 40% of the project cost if the system has to comply with high regulatory standards. FD’s and governance risk managers of avionic companies know this better than most when confronted with the reality of the cost of change.

So what joy when working as a consultant with QinetiQ in 2010 that I came across a small team nestled in their Aerospace Group in Malvern who were being funded to develop a way of verifying the behaviours of the Eurofighter critical electronics systems software, other than by using traditional test methods (as used by the main contractor). They had been working on the problem for over ten years and had developed a tool which could automatically verify that the source code could only ever deliver the software requirements, period. This team of computer scientists and mathematicians had taken the concept of Formal Methods mathematical proof as conjectured by Alan Turing in the late 1940s, re-invented in the 1960s in academia, and applied it to software design. It worked. However, the tool was extremely specialist in application and interpretation and only applicable to a defence software language (Ada). Unfortunately, with no future UK military aircraft on the horizon, funding was dropped and the team was scheduled to be disbanded losing all the know how built up over the previous 15 years.

What if this technology could be commercialised, made applicable to commercial software languages, made simple for the software developer, and could automate the verification of the design all the way from system requirements through to object code to the highest regulatory standards? In 2012 D-RisQ Ltd. was born to make this happen.

David Sheppard 2021