Vulnerability scanning aims to reveal security weaknesses in an application by using automated tools to assess its code, design, and functionality. Design flaws which lead to vulnerabilities like Cross Site Scripting (XSS), SQL Injection, path disclosure, and other vulnerabilities found in the OWASP Top 10.
The Vulnerability Scanner Landscape
Understanding what vulnerabilities exist and identifying those relevant to your application will be the first step in implementing vulnerability scanning practices. The OWASP Top 10 is an excellent resource that will call your attention to a short list of established threats. Integrating additional lists like the CWE/SANS Top 25 will help fill gaps and provide a complete vulnerability mitigation strategy.
OWASP Top 10 2021
- A01 Broken Access Control
- A02 Cryptographic Failures
- A3 Sensitive Data Exposure
- A03 Injection
- A04 Insecure Design
- A05 Security Misconfiguration
- A06 Vulnerable and Outdated Components
- A07 Identification and Authentication Failures
- A08 Software and Data Integrity Failures
- A09 Security Logging and Monitoring Failures
- A10 Server Side Request Forgery
Source: OWASP Top 10 2021
Organizational Considerations
The vulnerability scanner selection process begins by identifying organizational requirements which can be divided into four broad categories: cost, usability, update frequency, and support.
- Cost: A vulnerability scanner’s cost can be subdivided divided into initial and operational costs. Initial costs include the cost of the software and additional hardware, training, personnel, or resources that its implementation might entail. Ongoing costs would include operational expenses such as licensing fees, and ongoing training.
- Usability: The amount of effort, training, and skill a tool requires to be used effectively will describe its usability. For example, a small agile project might not have the skilled personnel available to properly employ a complex vulnerability scanner and would be better served by a more accessible option.
- Update Frequency: The quantity and quality of updates to the application and the rulesets it uses to identify the most recent vulnerabilities are an important considerations when choosing a scanner. For example, if a vulnerability scanner is no longer receiving timely updates, it would be advantageous to know this before investing heavily into its implementation.
- Support: The state of a scanner’s documentation and the channels through which support is available should also be considered. Discussion forums, email, and phone support become especially important as the complexity of a scanner increases.
Vulnerability Scanner Reviews
Qualys | On-premise
Strengths |
Dislikes |
- Protects whole IT system
- Can easily understand vulnerabilities
- Nice reports
- User friendly
- Training goes a long way
- Powerful tool to keep track of all types of web systems
|
- doesn't read inside Docker containers
- Web-App Vulnerability Scanner had problems logging in with one-page JS applications
- Not well suited for modern technology
- Quality of security posture
- False positives
- Could be automated
- Can take a long time to complete a scan
- Cloud Agents work sometimes, multiple issues with data not processing or getting stuck in queue
|
Strengths |
Dislikes |
- Easy to install
- Intuitive UI
- Full range of basic penetration testing phases
- Reports are well presented with relevant information
- If a vulnerability is announced, Tenable releases a
- plugin for it within hours
- Great customer support
- Multiplatform scan
|
- Need to be rather technical to get up and running
- Price is rather high
- Customer support is a bit slow
|
Nessus | On-premise
Strengths |
Dislikes |
- Good for compliance checks
- Good plugin based checks
- Support for scanning several devices ( routers, switches, firewalls, Endpoints, etc.)
- Great UI
- 24/7 support
|
- Reports could be better
- Scanning through firewall creates a few false positives
- Price is high
- Not a good option for a web application scanner
- Reports lack data such as how to detect and prevent some incidents
|
Strengths |
Dislikes |
- User interface is easy to use
- Reports are informative
- Scanning process is fairly easy to set up
- Coverage of vulnerabilities is great
|
- Technical support team was not helpful
- Scans take a while since it explores all parts
- Tends to consume a lot of hardware resources
|
BurpSuite | On-premise & SaaS
Strengths |
Dislikes |
- Fast and easy to set up
- It is enough to test almost all security related vulnerabilities
- Great tool for pentesting work, with customization
- It allows Spidering the website: both manually and automatically
- can act as Man in the Middle (MITM) and help you change the GET and POST requests
|
- Often gives false positives
- User interface is not great
- Hard to link app with system web browser apps
|
Strengths |
Dislikes |
- Real time agent status monitoring
- Cost effective for its performance and features
- Generates accurate results based on inputs
- Has advanced configuration options for testing a broad range of cases
- It is easy to configure
- Broad range of testing
- Alerts of possible threats are good
|
- IBM AppScan Standard doesn’t offer SCA, it is limited to Enterprises only
- No support for Oracle fusion middleware stack scanning
- It is not tailored to different frameworks
- Enterprise management requires the purchase of additional AppScan products
- Sometimes gives fewer results when the number of tests performed increases
|
Contrast | On-premise & SaaS
Strengths |
Dislikes |
- Delivers fast results
- Easy to automate and integrate security testing into CI/CD
- Also works with open source/3rd party framework
- Easy for developers to run scans
- Security dashboard with real time metrics
- Low false positive ratio
|
- Dependent on tech stack (Java, Node js, Python & .Net.)
- Price is high for enterprises
- Missing web layer vulnerabilities detection
|
Technical Considerations
Technical requirements will eventually come to influence the selection process. The Web Application Security Scanner Evaluation Criteria’s (WASSEC) objective is to create vendor-neutral guidelines focusing on technical considerations to help security professionals choose the best scanner and also compliance requirements like those of the Payment Card Industry Data Security Standard (PCI-DSS). WASSEC has published a detailed document to this end describing an ideal scanner evaluation which outlines eight categories, listed below.
WASSEC Scanner Evaluation Categories
- Protocol Support
- Authentication
- Session Management
- Crawling
- Parsing
- Testing
- Command + Control
- Reporting
Black Box Testing
Black box scanners evaluate an application’s security through automated language agnostic functionality assessments. The following list outlines code design techniques black box testing evaluates.
Black Box Testing Techniques
- Decision Table Testing: “Decision ... tables associate conditions with actions to perform”
- All-pairs testing: “a combinatorial method of software testing that, for each pair of input parameters to a system (typically, a software algorithm, tests all possible discrete combinations of those parameters.”
- Equivalence Partitioning: “a software testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived.”
- Boundary value analysis: “a software testing technique in which tests are designed to include representatives of boundary values in a range.”
- Cause-effect graph: “a directed graph that maps a set of causes to a set of effects.”
- Error Guessing: “a test method in which test cases used to find bugs in programs are established based on experience in prior testing.”
- State Transition Table: “a table showing what state (or states in the case of a nondeterministic finite automaton) a finite semiautomaton or finite state machine will move to, based on the current state and other inputs."
- Use Case Testing: “a list of actions or event steps, typically defining the interactions between a role … and a system, to achieve a goal.”
- User Story Testing: “an informal, natural language description of one or more features of a software system.
- Domain Analysis: “the process of analyzing related software systems in a domain to find their common and variable parts.”
Source: Black Box Testing (https://en.wikipedia.org/wiki/Black-box_testing)
Considerations
The implementation of vulnerability scanning processes will require a strategic approach. To start, scanning tools should be calibrated once a list of relevant vulnerabilities has been compiled. Calibration will ensure your scanning tool is capable of detecting the vulnerabilities being sought within your application and its environment.
Employing multiple vulnerability scanners and creating multiple scanning profiles for each will help minimize blind spots, establishing stronger security practices. Furthermore, documenting scanner attributes like software versions, configurations, and environments will increase the value of reports generated by scanning practices. Lastly, rating the performance of each scanner used by your organization will improve future scanning efforts, allowing you to quickly select and employ the best scanner for an application and its specific environment.
In all cases, it should be noted that attackers will have access to the same scanners you use to analyze your applications giving you an opportunity to locate vulnerabilities before an attacker has the opportunity to exploit them. However, relying entirely on publicly available vulnerability lists, tools, and their default configurations will leave your application open to attack from threats not popular enough to make lists like the OWASP’s Top 10 or detected by default configurations creating a blind spot in your security coverage. Furthermore, the perpetual existence of unknown vulnerabilities ensures that a blind spot will be a constant fixture within your application security landscape. Consequently, employing a strategy which uses custom rules, integrates application specific plugins, and remains vigilant for new threats will give you a strong chance of mitigating not only the threats you know exist but those you aren’t yet aware of.
Final Thoughts
Choosing the right scanner will require identifying project objectives, scanner requirements, and also organizational requirements such as price, usability, and support. Vulnerability scanning is an essential component of application security efforts and its ability to analyze an application's functionality, code, and structure with the help of both white and black box testing will give application security teams a unique perspective by which security can be improved.
Vulnerability Scanning Resources
Let us know if you liked the post. That’s the only way we can improve.