In the last few years, we’ve continuously been hearing that we should automate, automate, automate. So it might be weird to hear that manual verification still matters. Jeroen Willemsen explains to us why we still need to perform manual checks.
The Application in Its RuntimeMost applications consist of application code and external libraries that are executed by a runtime:
You’ll potentially have issues in these places:
The first thing to do is to check your dependencies manually. It’s important to learn how to verify if an issue is actually a problem and how to fix it. Fixing can mean:
After that, you can start automating this process. But there are caveats. For example, not all ecosystems have proper dependency checkers. And even if they do, not finding a registered vulnerability doesn’t mean it’s not there.
A great starting point for securing your application code is to use a static analysis tool, or SAST (Static Application Security Testing). The tooling here varies greatly among languages and frameworks.
When you decide to suppress warnings or errors, make sure you explain why you’re doing so. This will make it clear to anyone seeing the suppression once you’re no longer on the team.
If you’re looking at a commercial tool, test before you buy. What works for one team, might not work for another.
But these tools will never find everything. For example, a static analysis tool won’t tell you if authorization rules are set up correctly or if you’re disclosing information in errors.
If you want to know how your backend is behaving in general, you can use a DAST tool (Dynamic Application Security Testing). This will help you:
It will, of course, create false positives (issues that can be ignored) or false negatives (things that the tool thinks are okay but are in fact a problem).
And when you do find a vulnerability, through manual or automatic tests, you can write your own security test. You can even add “evil user stories” in your backlog. These tests will only run successfully if your application isn’t vulnerable to the attack.
Basically, this is what your application in a container looks like:
It’s a bunch of layers built upon a base OS layer. At the top is your application. The layers below your application might have vulnerabilities or might be wrongly configured leading to vulnerabilities.
Some tools that can help are:
As with the previous sections, each of these blocks can have security vulnerabilities. For example, your Docker daemon might be configured in an insecure way. Or the host operating system could have security issues.
Use tools like:
In this part, immutability is king. If your systems are dynamic, you need to scan all your systems. But if you’re using a fixed and immutable image, you can scan only that image. Also, the smaller the host, the smaller the attack surface.
There are some general issues as well.
One issue is that secrets may leak. You’ll have to:
These are all places that might contain hard-code secrets in plain text. Be sure to also validate the process, because maybe not everyone knows how to work with secrets correctly.
For the network, it’s best to deny access by default. If you’re in the cloud, most providers offer tooling. If you’re not, use tools like nmap and checkssl.sh.
And don’t forget that manual verification may still be necessary to identify unexpected infiltration or exfiltration paths.
IAM issues can manifest themselves at different levels:
Your cloud provider should have a decent set of tools to get started. Then move up the stack. It will still require manual reviews: are the rules too broad, or too fine grained? Are our rules compliant with legal obligations?
Cloud providers have their own tooling ecosystem, so use those. But something like Cloud Custodian is “cross-cloud.” If you’re using Terraform, you can look at terraform-compliance.
Does this make you feel like you need to automate more? Good! Automate as much as you can. But realize that we can’t catch every mistake with automation.
Also, take risk based actions. Assess the risk of what you see first, otherwise you might drown in a pool of to-dos.
You can check out Jeroen Willemsen’s talk “Why Manual Verification Still Matters” here.