Road to Better AppSec: Comparison of Top Testing Options vs. PTaaS

Application security isn’t new, but it is growing rapidly in demand, complexity, and depth. As cybercrime increased nearly 600% since the start of the pandemic, more SaaS businesses have been scrambling to secure their applications. Even those running up-to-date endpoint protection faced major vulnerabilities in their system.

And it really raises the question: “Even if you have these protection measures in place, how can you be sure that you’re not susceptible to attack?”

The answer lies in an application security testing solution that can actively test your code for errors, critical vulnerabilities, and areas for general improvement. With all the new options available today, it can be difficult to simultaneously compare each option. In this article, we’re going to look at four of the top testing options.

• One-Time Penetration Testing
• Penetration Testing as a Service (PTaaS)
• Hiring Your Own Internal Security Testers or Consultants
• Deploying Automated Tools like SAST, DAST, RAST & IAST

When comparing the top testing tools, some things to keep in mind are:

• Depth and breadth of testing
• Deployment frequency
• Level of manual effort involved
• Cost
• Ease of implementation
• Ease of maintenance
• Applicability to your business logic

Introduction to Penetration Testing

Penetration testing, also called a ‘pentest’ or ‘ethical hacking,’ is an authorized exercise to test the security resilience of a software or network system. As one of the top testing options, penetration testing usually involves an experienced security pentester who manually performs the test according to a predefined set of security test plans.

Penetration testing gained a lot of popularity in the last decade as a prerequisite for numerous compliance standards and regulations such as SOC2, PCI-DSS, HIPAA, ISO 27011, and NIST.

Comparing PTaaS to Traditional Penetration Testing

Penetration testing as a service (PTaaS) is a more integrated and comprehensive version of the traditional one-time penetration testing service. As the software development industry is moving faster than ever, some organizations have started deploying code even hundreds of times per day. Yearly penetration testing is way too slow for agile teams to ensure their software is being tested against vulnerabilities.

PTaaS typically provides more testing time per functionality of the application. For applications built on newer frameworks, there are less issues with typical vulnerabilities such as SQL injection, cross-site scripting (XSS), and XML External Entity Injection (XXE). Additionally, PTaaS can handle more issues with authentication and identification, multi-tenancy, and other more complicated business logic.

As each feature gets more testing time, more attack combinations and different perspectives from multiple engineers, we find on average twice as many bugs per application with PTaaS as compared to one-time testing.

Some businesses engage in annual or bi-annual traditional penetration tests in an attempt to create this more comprehensive testing program. In doing so, there are two schools of thoughts:

1. Maintain a relationship with one penetration testing vendor. The proponents of this school of thought mention that working with one vendor avoids a lot of paperwork and the vendor develops a deeper understanding of the system overtime. Together, these benefits ultimately lead to a more in-depth, comprehensive test with a better understanding of how issues affect the application.
2. Regularly switch penetration testing vendors. The proponents of this school firmly believe that a fresh set of eyes bring fresh perspective and hence a fresh set of bugs that might not have been uncovered yet. The time and energy for research, paperwork, and onboarding are believed to be worthwhile challenges in the mission for better test results.

Here is a quick summary comparison between these two approaches:

Creating The Best of Both Worlds

PTaaS combines the best of the two traditional approaches. While working with the same vendor keeps the administration and onboarding to a minimum, maintains the relationship, builds on the existing system knowledge and establishes a more intimate relationship with the application. Done well, it will also provide a fresh perspective through team rotation. With team rotation, clients have the opportunity for a close team of security engineers to take turns running each pentest. This grants the opportunity to bring in a new set of eyes without losing any of the customer relationship or existing system knowledge.

Now, let’s look at the overall comparison between PTaaS and regular penetration testing:

Hiring Internal Staff for Security Testing

Hiring internal staff to perform continuous penetration testing against the application is an idea that seems very enticing. In theory, it sounds like a great first step towards building a security function inside the organization. Possibly it could provide someone to manage the security tasks the CTO was usually responsible for, or someone to answer vendor questionnaires, champion internal efforts to build better security culture and perhaps start that long-awaited SOCII compliance.

In practice, this does not happen. Often, the internal team is pulled into so many directions from compliance and internal meetings, to design reviews, and handling daily incidents. Internal staff rarely have the time to do the proactive testing required to ensure the applications and the infrastructure are as resilient as required.

Regardless of the capacity of the internal staff to run comprehensive penetration testing, this approach is still a top testing option considered by many businesses. Given that most organizations don’t have the budget to hire a full-time penetration tester in-house, CTOs usually opt to hire a security generalist instead. This person would touch on all the above aspects such as security strategy, compliance initiatives, and championing internal security policies. While it makes sense to have more touchpoints covered, it also makes it hard for just one person to deeply engage in all the necessary skills required. Most systems right now contain many moving pieces from front-end technology, back end technology, cloud technologies, infrastructure technologies and the specific attack scenarios against each of these.

Compared to PTaaS, hiring internal staff can be more expensive, but has the potential to cover more territory within security. However, in-house penetration testing is not as in-depth as PTaaS, nor does it provide any fresh perspectives.

Use of Automated Security Tools

Some companies opt to use scanning tools as an alternative to pentesting or as an alternative to hiring internal staff. Sometimes it is perceived to be more scalable, cheaper, and sometimes it is perceived as the easiest way to just ‘check’ the security box.

Static Analysis Security Testing (SAST), Dynamic Analysis Security Testing (DAST), Interactive Analysis Security Testing (IAST) and Runtime Application Security Protection (RASP) are all different security testing tools. While the use of these tools is an important part of a full application security program, it does not negate the use of more comprehensive or manual testing options like penetration testing.

The use of these tools provides you necessary scale and immediate feedback. For example, if you have hundreds of applications, these tools can provide high-level testing coverage for all your applications quicker than manual testing. Another example for the use of these tools is if you have a requirement for basic security checks per PullRequest push.

Complete Comparison Chart of Top Testing Options

Conclusion

Compliance and vendor requirements remain the main drivers of application security in the B2B world. CTOs have a wide array of new and traditional options to choose from as they seek to satisfy the growing need for proof of application security. Considering the unique goals of your business, your budget, scale and number of applications, you can successfully use the above guide to determine the correct application testing service for your organization.