Vulnerability Testing

What is vulnerability testing?

Vulnerability testing is an assessment used to evaluate application security by identifying, diagnosing, and triaging application vulnerabilities. The entire process requires application security (AppSec) teams to plan vulnerability tests and analyze results. The goal is to better understand an application’s behavior and identify vulnerabilities that can be exploited by malicious attacks.

Vulnerability testing is helpful for the identification of risks but should not be the only application security tool a team uses in development. Effective analysis of an application’s vulnerabilities requires frequent testing and a team of highly skilled professionals to peruse results and make decisions as to which ones deserve attention and which do not.

Vulnerability testing throughout the SDLC  

The first step in the process of vulnerability management is the identification of exploitable application vulnerabilities using automated vulnerability management software tools, penetration testing, and/or manual tests. Automated tools like vulnerability scans are used at different stages in the software development life cycle (SDLC) to give security teams and developers a snapshot of potential vulnerabilities.

Planned application attacks with penetration testing

A penetration test is a planned attack that tests an application's vulnerabilities. AppSec teams take the same approach as a hacker, targeting weaknesses and launching application attacks that successfully exploit them, such as a brute-force attack, SQL injection attack, or cross-site scripting (XSS) attack. Specialized AppSec professionals typically plan and launch penetration tests during the testing phase of the SDLC—toward the end of deployment—using results to configure application defenses like web application firewalls (WAFs). 

Vulnerability scanning early in development with SAST

But as organizations embrace Agile and DevOps for greater digital transformation acceleration, penetration testing approaches simply do not scale to the speed required. Halting development cycles slows business acceleration, and identifying vulnerabilities during testing is highly inefficient.

To test earlier in the SDLC, developers use static application security testing (SAST) and dynamic application security testing (DAST). SAST works by analyzing code line by line, taking a signature-based approach. SAST’s inside-out testing of an application is configured based on the programming language and framework used to develop the application. Developers use results to find vulnerabilities that could lead to an application attack, fixing them before moving forward in development. Further, legacy SAST is only one piece of the vulnerability management puzzle, paired with other application security testing further into the development cycle.

Vulnerability testing in deployment with DAST

Developers not only need an idea of how an application will perform in a testing environment but also in deployment and throughout the operation. This requires a different form of vulnerability testing that tests from the outside in. Using DAST, developers can compare application programming interfaces (APIs) against a library of known application attacks. When application vulnerabilities are triggered, developers check for responses to observe application behavior to measure the risk of an application attack.

Limitations caused by legacy vulnerability scanning tools

When vulnerability testing software and scanning tools are well-managed, they provide development teams with useful information to help them reduce application vulnerabilities. The problem is that managing them is difficult, as each of them is known to produce inaccuracies. These inaccuracies waste developers’ time and prevent them from keeping up with the speed demanded by modern software development. In addition, static and dynamic scans take significant time to complete, upwards of eight hours or more in many instances, which doesn’t even account for analysis and remediation efforts, which can take hours or days to resolve.

Legacy security scanning and vulnerability testing software produces false positives

Starting with SAST, developers are met with numerous false positives. These are caused by limitations in configurations that lack context to identify vulnerabilities. SAST lacks visibility into libraries and cannot identify “dead code” that is never reached or executed. Thus, when legacy SAST tools are used on their own, they cannot detect unknown potential application attacks. Though SAST is a cost-effective option for catching vulnerabilities early in development, In addition, static and dynamic scanning is very time-consuming and requires knowledgeable experts to configure, execute, and analyze results—taking, on average, one hour per alert.

Vulnerability testing later in the SDLC

DAST is known to produce false negatives, checking code only against known threats. As with SAST, unknown or zero-day attacks are free to get past DAST controls, resulting in missed vulnerabilities that could result in an exploit. As DAST tools cannot be used when code is being written but rather require a running application, application security cannot shift complete left. This means vulnerabilities are more difficult to triage, diagnose, and remediate—all of which incur more time.

Limitations in the analysis of detected application vulnerabilities

With a long list of potential vulnerabilities through which to comb, highly skilled application security specialists are left to analyze the results. The analysis of application vulnerabilities is time-consuming, beginning with the need to triage and diagnose triggered alerts. SAST can easily produce hundreds of alerts, many of which turn out to be false. The analysis begins with AppSec specialists sorting through vulnerabilities, attempting to identify any false positives. It continues with ranking each vulnerability on a scale of the risk imposed if left untreated. Once vulnerability remediation is prioritized, developers are left to identify their location, determine their cause, implement a fix, and verify the success of the remediation.

Frequent testing piles up issues

In order to effective manage vulnerabilities in development, ongoing scanning and testing is required. While regular vulnerability testing helps in the prevention and detection of application vulnerabilities, it creates a cycle of ongoing issues for security teams. Each time a vulnerability test is run, they face the same issues of triggering numerous false alarms. Without the right controls in place and well-staffed AppSec team, security debt can quickly ensue. Further, each time a vulnerability test takes place, AppSec specialists must go through the entire triage and diagnosis process again, sorting through results, pinpointing true vulnerabilities, and prioritizing remediation based on risk.

Vulnerability testing with automated remediation

Legacy application security and vulnerability testing make application security a more difficult undertaking, weighing AppSec teams down with false positives, false negatives, and other numerous inefficiencies such as manual triage, diagnosis, prioritization of remediation, and more. Instead of relying on decades-old processes that raise additional issues that must later be addressed, developers need a reliable way of continuous security that removes time-consuming work by the AppSec team on triage and diagnosis.

Embedding continuous application security and protection into applications using instrumentation enables developers to get alerts about vulnerabilities as they are writing code. They no longer need to spend valuable time retracing their steps to locate vulnerabilities and determine their cause, as they can do so while still working the continuous integration/continuous deployment (CI/CD) pipeline. This reduces the time needed to find and test vulnerabilities. Through automated vulnerability detection, there is no longer a need to stop development for testing purposes or guess which triggers are false positives. Instead, code is analyzed as it is written in real time. Plus, integrating security testing into the development process is proven to reduce the costs as a result of a data breach by $10.55 per compromised record.

Instrumentation also enables organizations to extend application security directly from development into production. Known as runtime application self-protection (RASP), application behavior is continuously monitored and attacks are pinpointed in real time. And because RASP is embedded within running software, it virtually eliminates the false positives that occur with web application firewalls (WAFs).