Vulnerability scanning aims to reveal security weaknesses in an application by using automated tools to assess its code, design, and functionality. Design flaws which lead to vulnerabilities like Cross Site Scripting (XSS), SQL Injection, path disclosure, and other vulnerabilities found in the OWASP Top 10.

The Vulnerability Scanner Landscape

Understanding what vulnerabilities exist and identifying those relevant to your application will be the first step in implementing vulnerability scanning practices. The OWASP Top 10 is an excellent resource that will call your attention to a short list of established threats. Integrating additional lists like the CWE/SANS Top 25 will help fill gaps and provide a complete vulnerability mitigation strategy.

OWASP Top 10 2021

• A01 Broken Access Control
• A02 Cryptographic Failures
• A3 Sensitive Data Exposure
• A03 Injection
• A04 Insecure Design
• A05 Security Misconfiguration
• A06 Vulnerable and Outdated Components
• A07 Identification and Authentication Failures
• A08 Software and Data Integrity Failures
• A09 Security Logging and Monitoring Failures
• A10 Server Side Request Forgery

Source: OWASP Top 10 2021

Organizational Considerations

The vulnerability scanner selection process begins by identifying organizational requirements which can be divided into four broad categories: cost, usability, update frequency, and support.

1. Cost: A vulnerability scanner’s cost can be subdivided divided into initial and operational costs. Initial costs include the cost of the software and additional hardware, training, personnel, or resources that its implementation might entail. Ongoing costs would include operational expenses such as licensing fees, and ongoing training.
2. Usability: The amount of effort, training, and skill a tool requires to be used effectively will describe its usability. For example, a small agile project might not have the skilled personnel available to properly employ a complex vulnerability scanner and would be better served by a more accessible option.
3. Update Frequency: The quantity and quality of updates to the application and the rulesets it uses to identify the most recent vulnerabilities are an important considerations when choosing a scanner. For example, if a vulnerability scanner is no longer receiving timely updates, it would be advantageous to know this before investing heavily into its implementation.
4. Support: The state of a scanner’s documentation and the channels through which support is available should also be considered. Discussion forums, email, and phone support become especially important as the complexity of a scanner increases.

Vulnerability Scanner Reviews

Qualys | On-premise

Rapid 7 Nexpose | On-premise

Nessus | On-premise

Acunetix WVS | On-premise

BurpSuite | On-premise  & SaaS

AppScan| On-premise

Contrast | On-premise  & SaaS

Technical Considerations

Technical requirements will eventually come to influence the selection process. The Web Application Security Scanner Evaluation Criteria’s (WASSEC) objective is to create vendor-neutral guidelines focusing on technical considerations to help security professionals choose the best scanner and also compliance requirements like those of the Payment Card Industry Data Security Standard (PCI-DSS). WASSEC has published a detailed document to this end describing an ideal scanner evaluation which outlines eight categories, listed below.

WASSEC Scanner Evaluation Categories

1. Protocol Support
2. Authentication
3. Session Management
4. Crawling
5. Parsing
6. Testing
7. Command + Control
8. Reporting

Black Box Testing

Black box scanners evaluate an application’s security through automated language agnostic functionality assessments. The following list outlines code design techniques black box testing evaluates.

Black Box Testing Techniques

1. Decision Table Testing: “Decision ... tables associate conditions with actions to perform”
2. All-pairs testing: “a combinatorial method of software testing that, for each pair of input parameters to a system (typically, a software algorithm, tests all possible discrete combinations of those parameters.”
3. Equivalence Partitioning: “a software testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived.”
4. Boundary value analysis: “a software testing technique in which tests are designed to include representatives of boundary values in a range.”
5. Cause-effect graph: “a directed graph that maps a set of causes to a set of effects.”
6. Error Guessing: “a test method in which test cases used to find bugs in programs are established based on experience in prior testing.”
7. State Transition Table: “a table showing what state (or states in the case of a nondeterministic finite automaton) a finite semiautomaton or finite state machine will move to, based on the current state and other inputs."
8. Use Case Testing: “a list of actions or event steps, typically defining the interactions between a role … and a system, to achieve a goal.”
9. User Story Testing: “an informal, natural language description of one or more features of a software system.
10. Domain Analysis: “the process of analyzing related software systems in a domain to find their common and variable parts.”

Source: Black Box Testing (https://en.wikipedia.org/wiki/Black-box_testing)

Considerations

The implementation of vulnerability scanning processes will require a strategic approach. To start, scanning tools should be calibrated once a list of relevant vulnerabilities has been compiled. Calibration will ensure your scanning tool is capable of detecting the vulnerabilities being sought within your application and its environment.

Employing multiple vulnerability scanners and creating multiple scanning profiles for each will help minimize blind spots, establishing stronger security practices. Furthermore, documenting scanner attributes like software versions, configurations, and environments will increase the value of reports generated by scanning practices. Lastly, rating the performance of each scanner used by your organization will improve future scanning efforts, allowing you to quickly select and employ the best scanner for an application and its specific environment.

In all cases, it should be noted that attackers will have access to the same scanners you use to analyze your applications giving you an opportunity to locate vulnerabilities before an attacker has the opportunity to exploit them. However, relying entirely on publicly available vulnerability lists, tools, and their default configurations will leave your application open to attack from threats not popular enough to make lists like the OWASP’s Top 10 or detected by default configurations creating a blind spot in your security coverage. Furthermore, the perpetual existence of unknown vulnerabilities ensures that a blind spot will be a constant fixture within your application security landscape. Consequently, employing a strategy which uses custom rules, integrates application specific plugins, and remains vigilant for new threats will give you a strong chance of mitigating not only the threats you know exist but those you aren’t yet aware of.

Final Thoughts

Choosing the right scanner will require identifying project objectives, scanner requirements, and also organizational requirements such as price, usability, and support. Vulnerability scanning is an essential component of application security efforts and its ability to analyze an application's functionality, code, and structure with the help of both white and black box testing will give application security teams a unique perspective by which security can be improved.

Vulnerability Scanning Resources

• OWASP Vulnerability Scanning Tools - Primarily Dynamic Application Security Testing (DAST) Tools.
  https://www.owasp.org/index.php/Category:Vulnerability_Scanning_Tool