Why We Will Continue to See Catastrophic Software Failures

This article does a good job summarizing why catastrophic software failures are inevitable given our current development practices. There’s an old engineering maxim that once a system becomes sufficiently complex, it will eventually fail no matter how much testing you do. This is why critical systems, such as medical equipment, are designed to “fail safe”.

Unfortunately, almost every application today depends on libraries developed by programmers who may be halfway across the world, which makes designing fail-safe software extremely challenging. Until we address this coordination problem, catastrophic failures like those mentioned in the article continue to be a real danger.

The Coming Software Apocalypse

Machine Learning: Is “Off-the-Shelf” Software Enough?

As more and more businesses use machine learning to inform their decisions, I’m often asked whether”off-the-shelf” software can really take the place of custom solutions.

To answer this, it’s important to distinguish between software that analyzes the data (e.g. estimates a decision tree, or other machine learning model) and software that attempts to interpret the data. In most cases, you don’t need a custom solution to implement the most common algorithms: software such as Talend’s enterprise product can do that with minimal hassle. For those committed to open-source solutions, the R statistical program does more than you’ll ever need it to, albeit with a rather high learning curve (you can see what R makes possible, and the technical skill needed to use it, on this page).

However, while there’s no need to program your own Bayesian classifier from scratch, you’re still likely going to need an experienced data scientist to interpret the data. The reason is that computer programs are ultimately just lists of instructions: they can’t actually “think”. Because of this, they don’t cope well with the unusual, or the unexpected.

For example, I built a model to estimate house prices from property data such as square footage, age, and condition. Overall, it was a good fit, explaining about 85% of the variation in the data: however, there were certain properties that were assigned negative prices, so something was clearly wrong.

After drilling down into the data, I found that these properties were listed as having zero square footage. This was because they were condominiums, and in this data set, the square footage for all units was assigned to the overall building, not the individual units. There wasn’t any way to fix this, so I ended up excluding those cases and building a separate model for condominiums. If I hadn’t understood how the model worked, I wouldn’t have known where to look to find the source of the problem, and I wouldn’t have known to use a different type of model for the other cases.

To summarize, you’ll get the most return on your investment by paying for an experienced analyst to work with one of the off-the-shelf software packages. This gives you the best of both worlds: a proven modelling engine to crunch the numbers, and a skilled professional to interpret them.