This is usually (though not always) the third most important thing about a piece of software. So what does it mean, in ACRUMEN terms, for a piece of software to be Robust?
The short definition is that it’s hard to make it malfunction, or even seem to. But what does that mean?
Most of what I mean is covered by a basic concept from information security, the CIA Triad. No, it’s nothing to do with spies and gangsters, but the combination of Confidentiality, Integrity, and Availability. So, mainly, robust software should not reveal information when it’s not supposed to, alter information when it’s not supposed to, or become unavailable when it’s not supposed to. (Some people would add that it should not become available when it’s not supposed to, but so long as it doesn’t significantly raise the risk of violate either of the other two, it’s usually not a big deal.)
There’s also the question of seeming to malfunction. By this I mean mainly showing abstruse error messages, that might lead the user to think something has gone wrong, even if it hasn’t. (Yes, this overlaps greatly with Usability.) For instance, if the system wants the user to type a filename, the user could type it wrong, or type the name of a file they don’t have access to. The system should not say ENOENT (short for “Error: No [Such Directory] Entry”, meaning “File Not Found”), or “HTTP ERROR 500” (meaning “something went wrong on the server”), let alone abort the whole program and show a stack trace! (Which is also information they are probably not supposed to have, and which could probably be very useful to an attacker.) Instead, it should show a user-friendly error message, to the effect of “Sorry, I can’t open that file”, and let the user try again! (In case they are an attacker, there are many ways to deal with that.)
So how do we make sure that our software is Robust?
Again, we can hire the experts, and in this case that would be penetration testers. However, they’re usually rather expensive, and disruptive, because they need to test the production system, not a testing or staging environment.
But we can get pretty far by using some of their tools, such as:
- static analyzers, which simulate the execution of our program;
- fuzzers, which test our program’s reaction to invalid inputs; and
- scanners and probes, which test our systems’ defenses against specific known attacks.
Many of these are available as Open Source.
But even without their tools, we can go a long way with just their mindset. The main part of this is to always look for what can go wrong. This includes not just innocent mistakes like a typo in a filename, or external circumstantial mishaps like losing a network connection, but also what an attacker can make go wrong in such a way as to get closer to their goal. For instance, in what unusual ways is it possible to get information out of our system — or into it? (Don’t forget about not altering information!)
Once we’ve brainstormed and figured out lots of stuff that could go wrong (perhaps with the aid of the security technique of Threat Modeling), we can figure out appropriate responses. This isn’t always to completely prevent it, though; some alternatives include:
- preventing the situation in which the mistake/attack is even possible,
- mitigating the negative effects,
- recovering from the negative effects, perhaps with the help of insurance, or
- just accepting it, if the negative effects are mild enough.