Third category of application security tools beyond "static" and "dynamic"?

Recently, Clark Coleman asked a very logical question about application security tools.

• Clark Coleman Can you explain the difference between DAST (Dynamic Application Security Testing) and IAST (Interactive Application Security Testing)? To a novice like me, it would seem that you either analyze a program while it is not running (static) or while it is running (dynamic). Logically, (A) and (not A) cover the possible universe with no room for a third category.

The problem, as usual, is naming. What we call DAST means that an application is scanned with HTTP requests in a quest to reveal vulnerabilities based the HTTP responses generated.  What we call SAST means analyzing application code. Putting the names aside, these two techniques don't nearly cover the possible universe of ways to analyze an application's security.

An alternative way of thinking about the problem...

I wrote an article in 2014 to explain why the terms "static" and "dynamic" are misleading.  I proposed an alternative way to organize your thinking about applications security tools.  Instead of  thinking of tools as static or dynamic, let's classify them by what information they have available to make decisions.

1. Code - The code has interesting information about a lot of security issues. But it's very hard to extract without understanding exactly how it's all put together and how it actually behaves when it runs. For some problems, like hardcoded passwords and use of weak algorithms, direct code analysis is highly accurate and effective. (This is where SAST lives)
   
   
2. HTTP Request - You can collect a lot of good information from HTTP requests about authentication, sessions, use of SSL, certain headers, CSRF tokens, and sitemap information. If you monitor requests passively, you can test in just about any environment.  If you use requests to carry out security tests, you'll need an expert to configure and run the scan so it can actually get to any interesting pages.
   
   
3. HTTP Response - The HTTP responses also provide a wealth of security information. Security headers, CSRF tokens, SSL, autocomplete, access control, and session management all are easily revealed by analyzing responses. (This is where DAST tools live)
   
   
4. Data Flow - Data flow is the holy grail of security analysis. Most of the interesting injection problems can only be revealed with a strong data flow engine. Using static analysis tools used to be the only way to really analyze this, but they were difficult to use and suffered from massive false alarms. Newer instrumentation based approaches can perform highly accurate "whole application" data flow analysis including frameworks, libraries, dynamic classloading, and more.
   
   
5. Control Flow - Several types of vulnerabilities can be revealed through control flow analysis. Did the proper hashing occur during the authentication process? Is there an access control check in every Struts action. Control flow tells us whether the code executes the right steps in the right order.
   
   
6. Libraries and Components - We've written a lot about this, including here, here, and here, but modern applications are built from libraries, so analyzing them is of critical importance. We need to know if the libraries have known vulnerabilities.  But, more importantly, we need to know if the library has unknown vulnerabilities or hazards that an unsuspecting developer used improperly.
   
   
7. Frameworks - All modern web applications and web services use a framework, and many frameworks have a set of security controls built in. Were these controls used properly? Did the developers invoke them in all the right places? Understanding the framework pays off immensely in providing useful context-sensitive guidance to developers.
   
   
8. Application Server - The application server, like a framework, also has numerous security controls. Is the application using them properly? Tools that understand the application server in use and what protection it provides can do better analysis.
   
   
9. Configuration Data - Most security controls can be configured on or off. Many of them have detailed configuration that allows their behavior to be tailored. Configuration data can be in XML files, property files, stored in databases, etc... This is a direct and very accurate way to measure application protection.
   
   
10. Backend Connections - Understanding where sensitive data is stored is key to being able to perform great security analysis. Tools that can understand backend connections are much more likely to be able to see critical vulnerabilities and describe them to developers.

Interactive (IAST) tools, like Contrast, have access to all the information listed above because they work from inside the application itself. This information advantage is why they boast much greater accuracy than SAST or DAST tools.

It's important to remember that you need all this information during the analysis. You can't get the same results by running a bunch of 1-dimensional tools separately.  I call this failed approach "tool soup."

The path to effective application security analysis is using all this information together in a unified analysis. Contrast's CSRF rule, for example, takes advantage of HTTP headers, control flow analysis, and backend connection information. You can find out more in an article I wrote about this combined analysis technique.