This is part 3 of 5 in the Managing Your Penetration Test Report Series
If you’re in the security industry, or you’ve dealt with security tools in recent years, you may have heard of the term “security theater.” This term, first coined by Bruce Schneier, describes security measures which make us feel like we’re doing more for our security than we actually are. Bruce’s inspiration came from the United State’s Transport Agency Services (TSA). As described in his essay, what appeared to be a complete network of security measures in airports still did not prevent a terrorist attack from occurring.
While Bruce’s example looks at the extreme end of risk that comes from security measures that aren’t all that secure, companies should also consider how security theater may affect their own applications or infrastructures. Understandably, this can be a big stressor for CTOs and CISOs, who ‘don’t even know what they don’t even know.’
The aim of this article is to identify areas where security theater often occurs and prepare the reader to critically examine their own security tools and processes for possible security theater.
Knowing why we need to look for security theater in an application is a critical step towards prioritizing and accurately implementing security in an organization. Here are three key ways in which security theater can impact an application’s security:
Many times, security theater can come from within. Examples include:
Something we’ve often heard from past clients is that they’ve received inaccurate results from past penetration testing providers. In these reports, it was not that the results themselves were false, as the issue identified usually does actually exist. The inaccuracy lies in mis-assigning a severity level to a vulnerability.
Some firms believe it's a good business practice to always provide critical issues, to show that their business is valuable at minimizing risk. However, mis-assigning issues is a huge red flag, and can leave your application or infrastructure more vulnerable or inefficient than before the penetration test.
As another case scenario, a disreputable penetration testing firm may not spend enough time understanding the application’s unique business logic to correctly identify the exact risk of the attack surface.
Knowing what the industry standards are for correctly assigned severity levels helps minimize your chances of being a victim of security theater. The breakdown below will help you to better identify if your pentest report is of a high quality, and if the results are accurately assigned. Keep in mind that we suggest standard service-level agreements for each severity level, but this may change based on your risk acceptance policies or vendor relationships at your organization.
Most firms break down vulnerabilities into 5 categories. These categories are based on unique combinations of risk and impact, and are based on the two models above, as well as OWASP’s Risk Rating Methodology. Vulnerabilities can be categorized on a multitude of factors such as impact, report confidence, likelihood of occurring, and complexity in remediation, among others.
These vulnerabilities have high impact, high risk, and are easily exploitable. Stop shipping and address these issues immediately. At minimum, a temporary “band-aid” solution should be implemented to prevent exploitation. Typically, the exposure is easily identified and successful exploitation is trivial. Fix before release or within 2 weeks.
These vulnerabilities have high impact and are relatively easy to exploit. Like critical vulnerabilities, a temporary “band-aid” solution should be implemented right away to prevent exploitation. Successful exploitation may require either user-level access to the system or application, or exploits may not be publicly available. Fix within 14-30 days.
These vulnerabilities have moderate impact and are relatively easy to exploit. Issues in this level are typically leveraged in conjunction with one or more security issues to compromise the system or application. Fix within 90-180 days.
These vulnerabilities have low impact and are more difficult to exploit. Items can be addressed at the discretion of system owners, depending on company or client risk-acceptance policies. Fix within 180-270 days.
These vulnerabilities have very little impact and are difficult to exploit successfully. Fix at developer’s discretion or by customer requirement only.
A reputable firm will always provide unbiased risk. This means that sometimes you may receive a report that just contains a series of informational risks, for example. This doesn’t necessarily mean the firm did a poor job at finding critical vulnerabilities. It just means that your development team already did a good job at securing your code, so there are less vulnerabilities to find. The reputable firm just provides you with the confidence that your code is secure.
A trustworthy firm will also provide a detailed report that explains how they found the issue, with substantial evidence to backup their findings. You should be able to question any of their findings, and they should have a confident response as to the "why" they chose that severity level. At Software Secured, we also include screenshots and a step-by-step guide to help you replicate the issues yourself, to fully understand what we've found.
Having the knowledge of what security theater looks like and the confidence to critically examine your pentest results helps you earn the best results possible.