This is part 2 of 5 in the Managing Your Penetration Test Report Series
The first thing you should do when receiving a penetration test report is to read it in entirety. You need to understand the key findings, how the testing was performed and get an idea of what remediation work needs to be done.
Most penetration tests come with an executive summary which can be very useful for getting a high level understanding of the test. You can also use this opportunity to evaluate how close your organization is to its final state. For example if the test was related to a compliance standard then the report should let you know if you have the proper controls in place or what controls you are lacking.
Now that you have investigated the findings and have identified what work needs to be done, the next step is to develop a plan and implement. This plan should outline the order in which vulnerabilities should be fixed, who will be responsible for doing the fixes, the overall timeline for completion of all vulnerabilities and final validation.
Once you have an overall understanding of the findings of the penetration test the next step is to assign resources. These resources will be needed to start working on remediating the issues identified in the report. This should be a combination of technical staff as well as management that will be needed to help drive the actions to completion and provide authorization to make required changes.
Once you have the resources in place you can start investigating the findings outlined in the report. Typically, this should be done in order of severity with the highest severity issues being investigated and remediated first. In addition to finding and fixing issues there should be attention given to identifying the root cause of the vulnerability and addressing that. For example rather than simply fixing a misconfiguration in a system you may want to look at the SOPs associated with configuring those systems and ensuring that the documented instructions specify secure configurations. Doing this extra step helps to reduce the likelihood of future vulnerabilities being introduced to the environment.
The final step is to perform a retest of your environment. This is critical to ensure that the fixes you implemented remediate the vulnerability and don’t leave any workarounds that can be exploited by a hacker. This retest should be done by professional penetration testers, not internal devs or staff.
Make sure you understand the finding: If you don’t think a finding is accurate the first thing you should do is make sure you understand it correctly. Read the details of how the finding was discovered in the report and work with your internal technical staff to understand how this finding may be affecting your business. If you still disagree with the finding then you should contact the testers and speak with them to ensure that it is a vulnerability and not a false positive. The goal is to understand the full content of the finding, this includes the technical issue, sensitivity of data involved and potential business impact.
Take it as a learning opportunity: Any negative penetration test can be demoralizing, but any penetration test that finds a vulnerability benefits the organization by allowing it to improve its overall security posture. A negative penetration test should be seen as a learning experience, not a failure.
If you have evaluated the context, severity, sensitivity of data involved and the other measures, you can use the following steps to help further determine the potential impact of any finding in your pen test report:
As mentioned above you shouldn’t hesitate to speak with your penetration testing provider if you have concerns about the report you were given. To do this first you should book a meeting with the penetration testing team. In this meeting you can explain that you do not agree with the finding(s) provided in the report. Typically, penetration testing reports include the exact steps the testers used and how you can reproduce the same results. While this is included in the report, if you have been unable to reproduce the results yourself the testers can walk you through the steps during this meeting and help you troubleshoot why you were unable to reproduce these results yourself during your initial validation. They can also walk through the steps as to why they may have scored this vulnerability as a certain ranking. There are many factors that can affect the severity ranking of a vulnerability, many of which may not be apparent from the client’s point of view. Sometimes with context, a pair of low or medium vulnerabilities can become a high or even critical depending on the scenario and how they are used. This may be a reason why something that appears to be a small risk to the client may be given a high ranking because of how it can be used or combined with other vulnerabilities. If they walk through the entire story/scoring/reproducibility and you still do not agree, there are more steps you can take to verify the impact.
Rather than focusing on vulnerability itself you may want to look at the company’s risk rating system. Verify if your pentester uses multiple risk rating systems, and how they come up with their scoring, if they are only using one risk rating system this might be an issue. Ideally, the pen testing provider should have more than one rating system and they should have a method of scoring vulnerabilities that is objective and not done by feeling. If they use 2 that are well known in the industry and do not score by feeling, there is more merit to their rating. You should have gotten a good picture of their scoring system already from the report itself, their follow up call, or during the penetration testing process. If you were not privy to this information the entire time, this is a large red flag.
If you have done these previous two steps, and still are not happy with the outcome, it might be useful to ask a third party to consult on the matter. This does not mean that you need to get a second penetration test, but another security consultant should be able to go through the report along with the pentesters notes, reproducibility, and have your applications context in order to make a determination on whether the penetration test and its findings are credible. Depending on the feedback you get from the third party you can decide whether or not a second penetration test is a good idea or not.
If you're not happy with your penetration test report, there are a few things you can do. First, try to understand where the testers were coming from. It's possible that they are considering different contexts and scenarios. Second, reach out to the testing company and ask for clarification. Let them walk you through their process and hear their explanation for their findings and rankings. Finally, if you still don't agree, you can always request a second opinion from a qualified third party.