What is a vulnerability?

Hi. My name is Larry Maccherone. I'm a DevSecOps transformation architect at Contrast Security, and I'm here to introduce you to a different definition of a vulnerability than the one you might be used to. And this definition, I think, is more actionable. And the reason I want you to have this definition is that with this accurate definition, with this more actionable definition, you're much more likely to be able to find and remediate the vulnerabilities.

Before we get into that, a little bit about my background. Some logos of sort of where I've been come up on the screen. Hopefully, some of you recognize some of them and cheer for them. But there's really two things that I think are important for you to understand about where I'm coming from.

The first is that I was the head of application security for Comcast for five years. And during that time, I took it from being a traditional application security program, one where you had the products essentially go for evaluation to a security specialist group and is gatekeeping and auditing and policing kind of approach. And I changed it to be more of a shift left DevSecOps approach. And now I'm at Contrast, and my role at Contrast is essentially helping other companies do this as well.

Large enterprises like Comcast or smaller companies as well. So the second thing I think it's important for you to understand about me is that I'm an active developer. I write code almost every day. I'm the primary author of a dozen open source projects, one of which gets a million downloads a month and is considered critical infrastructure by the US government.

They write me a check every year to make sure I don't allow the product to be a supply chain attack vector.

It's used by every cloud vendor, Azure, GCP, and AWS. It's used by every cryptocurrency exchange and wallet. So high security is important for those projects. There's lots of contributors to them, twenty two for the one that I mentioned, I described, in more detail.

And I run those projects the way I recommend you do application security at scale at a large enterprise. And so I'm not just preaching.

I'm practicing what I preach, and it's coming from the space of being an active developer developing products that require high security.

So let's start with sort of the generic definition of a vulnerability. You may have heard this. When I was at the Software engineering institute at Carnegie Mellon, I launched this thing called Build Security there, and I was the founding director of Scilab there. We actually came up with a definition of a vulnerability for the purposes of those projects.

And this isn't the exact wording we use, but it's pretty close to what we use. It's a defect in the software that allows you to compromise either the confidentiality or the integrity of data or the availability of the intended functionality of that software. And so this is a very broad definition.

It's also pretty accurate. It pretty much captures any sort of kind of vulnerability you can think of, at least software vulnerability.

Network vulnerability as well, it sort of captures as well. Maybe not the people side of it as well. So those are not defects, you know, so much as sort of the way people are.

But it may not be the most actionable or the most useful.

And I'm using the word model here. And and and the reason for that is that, a definition is essentially a model. It causes you to think about the problem a certain way. And so all models are wrong to some degree, imperfect, incomplete.

Some models are useful or more useful than other models for various purposes.

So when you talk about a network diagram of a system, it's gonna simplify things. And but as long as it has enough detail to do the analysis that you wanna do, it's useful. And so that's what I mean by all models are wrong, some models are useful. And so I'm gonna try to give you a more useful model for taking action and preventing vulnerabilities and removing them than you probably well, then it's up on the screen right here now and that you probably used before.

So I'm gonna focus down on a category of vulnerabilities. It's not a very precise definition.

I'm gonna just call them gnarly. Gnarly vulnerabilities are ones that are particularly difficult to detect and also have high impact. And the vast majority of vulnerabilities you think of are the OS top ten or the SANS top twenty five. Maybe eighty, ninety percent of them are these gnarly kinds of vulnerabilities.

But it's not all vulnerabilities. So for instance, this definition that I'm gonna give you doesn't work for embedded secrets, for instance. So, there's a different way you would sort of think about that that would be useful to detecting it and removing it and preventing it. But the ones that are like injection attacks, SQL injection or command injection or any remote code execution kind of attack, this is the definition that I think will be most actionable.

Okay. So the first concept I'm gonna introduce you to here is the source.

So the source is the source of data in a data flow, and the definition of vulnerability that we're gonna focus on today is a data flow oriented definition.

And so we're looking for data that comes from untrusted sources, like an input field, the user input field, or an API parameter.

If that source gets all the way to a dangerous sync like an SQL statement or a command spawn, then we say that this is a vulnerability.

So this is when in this mode, the data, as it comes in, is immediately considered tainted. And if it makes it all the way to the sink and it stays tainted, then we talk about taint propagation or or tainted data.

It's possible to sanitize the data, and then that's a good recommendation. In fact, a lot of vulnerabilities can be prevented by sanitizing.

And one sort of, you know, good rule of thumb is that anytime you have untrusted input, immediately sanitize it the second it comes in. Now, of course, if you start developing that way and you follow that guideline, that's great. But if you get a million lines of code in before you realize that you should have followed that design pattern, it becomes impossible to then inject that after the fact. And so you really want the ability to detect whether or not the data flow goes through a sanitizer, a trusted sanitizer or not. When it goes through the sanitizer, then the data becomes untainted. And so I've got this orange color until it gets sanitized, and then and then it becomes untainted here. And so this is not a vulnerability. 

And so to sum up what this slide is essentially saying is a vulnerability, the definition of a vulnerability, this definition of a subset of all vulnerabilities in reality is data that comes from an untrusted source and makes it to a dangerous sync without proper sanitization.

And so that's the definition we're gonna use here today. So it's not quite as simple as those first two diagrams I put up there.

You have data that isn't just raw pasted very quickly from source to sync or from source to sanitizer to sync. The data's manipulated. It's assigned to a new variable name. It's merged with other data, concatenated.

It's gone over, with loops and things like that. And so we call these things that just sort of have one place to come in and another place to go out, like an assignment statement or a concatenation statement. We call these propagators in the terminology sort of the analysis that you have to do in order to determine if something is a vulnerability or not.

So this is still pretty linear and still pretty easy to understand. But in reality, it's not that simple in code. There's loops and if statements, and it gets very complicated. And so what you have is you have this tool, like a SAST tool or an IAST tool that's trying to follow the data through the code and try to see if it gets sanitized before it gets to a dangerous sync. And it's gotta look at this messy code that you see here with all its loops. And every time it sees an if statement and it branches, that creates essentially another copy of the metaverse.

So think of the con concept of the superhero metaverse where you have every decision you make, the butterfly's wings flapping, creating another instance of the universe that's only slightly different from that one. But these decisions happen a lot, and so there's essentially an infinite number of possibilities. And so every if statement or every time through a loop creates another universe. And so the analysis gets very big, very fast, consumes a lot of memory, a lot of CPU if you try to do it statically.

And so that is what SAST tools, static application security testing tools, do is they basically look at the source code, and they try to trace it like this diagram here. And when it gets to the point where they've got three if statements or three times through the loop, they basically say, now this is gonna take too long to explore every possibility. We're gonna just simplify it. And that's where all the error comes into SAST tools.

You hear about SAST tools being inaccurate. They have high false positive rates and and and false negative rates. Basically, most of these come from the fact that once the analysis gets somewhat complicated, it gives up on trying to be precise. It's making this trade off between speed and accuracy.

And so, good SAST tools do this with statistics and dynamically figure it out and make pretty good decisions. Their simplifications are pretty accurate, but they're still never better than, say, seventy percent accurate. You know, there's always some error in there, and most of them are fifty percent inaccurate. So you have false positive rates of seventy to, from thirty to fifty percent in SAST tools. So Contrast, we do it differently.

We avoid this state space explosion problem.

We cheat. We basically instrument the application. And as the data comes in, we're watching when it comes from an untrusted source, and we're marking it as tainted. We're watching every time it's assigned. We're watching what it looks like after it's manipulated. We're watching what it looks like after it goes through an if statement, and we record the conditions under which the if statement and we attach it to that branch of the data as we're tracing it tracing it through. We're watching actual data flow through an actual running application.

So our data modeling, our data flow modeling, sorry, is never wrong. Now it can be incomplete if you don't exercise the app enough, but your customers and your testing should do a pretty good job of covering that. And it doesn't need to cover a hundred percent to do better than a SaaS tool, which is making approximations very quickly. So I would say you're probably better off using an IS tool when you have twenty five percent test coverage, and you're certainly gonna find more true pulse false positives when you get up to about sixty percent, test test coverage.

And we avoid this state space explosion problem. We're not making a trade off between speed and accuracy.

But so let me show you, sort of how this works with a little bit of code. And I know I know some of you watching are not code people, so that's fine. This is not complicated code. Basically, anyone who can read English should be able to sort of follow this simplified code.

And this isn't this isn't sort of the exact code that would actually be running, but it's it's it's approximation that gets the point across. As I said, all models are wrong. Some models are useful. This slide is a model of the way our system works, and it illustrates it's a useful model for illustrating how you can actually solve things like SQL injection forever, permanently.

So this is a snippet of typical code that would be in a program, making a SQL execute query executing an SQL query statement. You pass in the SQL statement, and then it passes it on to something that, would execute the query.

So we instrument at runtime. We modify the code as it loads, and we insert trust boundaries around these dangerous functions, these dangerous syncs like SQL execute SQL statements. And so this trust boundary, it says enforce SQL trust boundary, and it passes in the SQL statement and and this context of whatever is executing. And this is not the first part of our code, instrumentation is running.

We've actually and I'm not showing it here. We've actually injected code to watch every time code comes in through a dangerous source and then follow it throughout as it flows through the system. So we insert this one line essentially into your execute query function, and this could be in a library even, a third party library. So we don't care whether you have your own SQL or whether it's a third party library.

We're gonna we're gonna wrap the wrap the the dangerous sync with a trust boundary.

And so this calls enforce SQL trust boundary takes in this this string, this query string, this SQL query string. And we're gonna do three things when this happens. First of all, we're gonna look at the context of where this came from, and we're gonna see that the library that you're calling is version x y z, and we're gonna determine whether or not there's a a publicly reported vulnerability against that library. So we're doing what a traditional software composition analysis tool is doing.

But even if there's an unreported vulnerability in the library, we can actually still identify the vulnerability and block it. And that's something no SCA tool does. And most SAS tools don't do whole code analysis. There's a couple that I know of.

One in particular I know, intimately, SAST tool. But we do it. So we will find not only known vulnerabilities in third party libraries, but unknown ones. Just a month or so ago, we found a vulnerability in a project called Netflix Genie that was a CVSS nine point nine.

It's used by tens of thousands of companies.

And so our product is finding these all the time. We don't necessarily report all of them to the NVD, but when they're serious enough and the library is popular enough, we do. And so that's an example of us finding unknown vulnerabilities.

So we will detect and report SQL mistakes by the developer here. So if the data from the query if the SQL statement is called SQL here, it's called query here, if we're gonna look at that. And if that's tainted at this point, then we say, hey. This is a vulnerability, and we report that to the developer. We put it into the system as a vulnerability. You can triage it and do whatever you want in your vulnerability management.

But we go one step further than any SAS tool could possibly go because the SAS tool is running pre production.

We can be running at runtime and in production.

We will actually detect if this query itself has content that it is an attack and will allow non attack queries to go through, we won't block those. But if there's an actual attack in the SQL query, we will throw an error, and we will block the attack. So not only are we doing what a traditional SAST or DAST tool is doing, we're actually sort of doing what a WAF tool is doing, except with much higher accuracy and more context and and more able to to block things like zero days, beforehand. I I call our product negative day protection for zero days because the the, Log4Shell attacks that happened a few years ago, the three year old version of our product at that point, so I guess that's five and a half years ago now, that version of our product blocked every flavor of Log4Shell attacks.

And so you didn't have to have it be reported like a WAF had to have it be reported that there was an attack. And then and then they had to create a signature, and then they had to distribute the signature, and then they had to get installed, and then it had to get tested before you put it in blocking mode. Hamina hamina hamina WAFs don't provide a lot of instant protection, and a lot of folks don't trust it to be turned on, and and takes too much work to test it to turn it on. And so WAFs doesn't actually provide a whole lot of protection here.

We do, and it does it very accurately.

And in run time, very low overhead, about the same sort of latency it adds as a WAF would add, and and consumes less system resources. Now a WAF might be on a separate system. We're on the main system that the app's running, so that is a consideration. But less resources cumulatively than the WAF itself, might use. So I I talk about replacing SAST and software composition analysis SCA and DAST and strengthening your WAF. And the reason I don't say replace your WAF is that WAFs also do some things that that our product doesn't do.

They do things like the affinity of an IP address, and you can block a particular IP address. And we don't have that capability. We're not on the network. We're in the app.

So we don't really have all the details and information to do that. We could possibly do a little of that, but we don't even really try to do that because we know that a WAF could do that. And things like a distributed denial of service attack, WAFs are better at that. Although I would argue you have to be using a cloud based WAF in order to really have good DDoS protection.

If you actually bring it onto your network, then you're consuming all that network bandwidth anyway, before you block it. So I don't know how effective that is. So we pair really nicely with cloud based WAFs like Cloudflare's WAF, and we complement that very well. They're strong on the net IP address affinity and DDoS kind of stuff.

We're strong on the actual vulnerabilities, blocking attacks against the actual vulnerabilities.

So in summary of our product, we wrap dangerous functions with three of these capabilities. I'm adding a fourth here. The first of those capabilities is runtime application security testing, I or IAST, and that can replace your SAST or your DAST with much greater accuracy, much faster, and much more context to allow you to figure out how to fix it.

Runtime application software composition analysis.

So this not only finds known vulnerabilities, it finds unknown vulnerabilities, and it also shows you the vulnerabilities that are actually in the running application. A lot of the dependencies that are listed in the manifest of a typical software project never get deployed to production. And so we'll filter all those out and save you all that work and know which ones are really sort of difficult.

Runtime application security observability.

So this is the functionality I didn't talk about before now. But while we're watching the application execute and we're seeing attacks against it, and we're seeing data flow through it, and we're seeing what, the connection strings for SQL statements, we can build essentially a a map, a runtime map, security relevant runtime map of your actual running application. We call this observability, in our product, and it complements traditional observability products. And then the fourth capability, which I did talk about earlier, this ability to block attacks in progress, it is sometimes called runtime application self protection. There's a new term that's been introduced recently called application detection and response. If you've heard of XDR or NDR, this is sort of riffing off of that, application detection and response. And, basically, it's a way to block attacks like a WAF would do, and this is vulnerability oriented attacks.

So that's all I have for you here today. Please connect with me on LinkedIn. I'd be glad to answer any questions there, or connect you with somebody who can do a demo for you. And I'm sure that when we launch this video, there'll be more content to allow you to sign up for a demo and a free trial. And, we'll add that, make sure you have that for you going forward. Thank you.