Blowing up DevOps bottlenecks with pull requests

It’s the perfect time, Larry Maccherone said during his DevOps Connect knowledge-sharing session at RSA 2022: The code is fresh in devs’ minds, and they’re hungry for quality feedback. 

SAN FRANCISCO — If you could improve only one thing in your software development, what would it be: 1. Your ability to more completely detect vulnerabilities, or 2. your ability to more rapidly resolve vulnerabilities once detected?

It’s a trick question. If RSA 2022 attendees’ response to that poll holds true for the whole lot of you, 80 to 90 percent of respondents will choose option 2. That’s what happened at a DevOps Connect session presented by Contrast Security DevSecOps Transformation Architect Larry Maccherone, “The 3 Ways of DevOps as the Keys to Developer-First Security,” on Tuesday, the first (full) day of the finally! we’re seeing you in the flesh! post-pandemic security show. Hands shot up overwhelmingly in favor of fixing the glitches you find vs. finding them all. 

It’s always the same, whether he’s posing the question to an RSA crowd or to his LinkedIn followers (where it was two to one in favor of I want to thoroughly fix those suckers vs. let’s find vulnerabilities — ALL of ‘em, even if we can’t get around to fixing that massive pigpile!

The Three Keys to the DevOps Kingdom

Maccherone was there to share the three application security practices whose adoption most lowers risk. Clearly, the biggie isn’t finding all the software anomalies. Rather, the biggest issue is bottlenecks that gum up the secure flow of code. 

Larry Maccherone, Contrast Security DevSecOps Transformation Architect,
presenting at RSA 2022’s DevOps Connect. 

In order to unplug those bottlenecks, get secure code flowing and transform an organization's culture to developer-first security, Maccherone asserted, you have to build the approach around Gene Kim's Three Ways of DevOps: namely, flow, feedback and learning. 

Devs shouldn’t be yanked out of flow

In Kim’s seminal Three Ways, flow describes how an entire system performs, as opposed to a specific silo of work or a department. Flow can refer to division on the macro level, such as Development or IT Operations, or it can get as granular as an individual contributor — such as a  developer or system administrator — with the focus being on business-value streams enabled by IT. 

Flow begins from the get-go when requirements are identified — be it by the organization or by IT — are built during development and then move into operations to be delivered to customers in the form of a service. The point of Kim’s practice of flow entails never allowing defects to flow downstream. 

DevOps coaches whom Maccherone has trained in the Three Ways of DevOps.

But when you consider the individual developer, there are also psychological factors involved in flow, Maccherone explained, and they lead to a sweet spot where you can maintain the zen of developer flow while also plugging into the essential second key to developer-first security: namely, the pull request. 

The perfectly timed, non-flow-blowing pull request

With regards to flow, the first key to the kingdom, Maccherone described it as “this idea that you center everything around this workflow thing that software developers are using, almost completely for everything they do nowadays” — e.g., the pull request. 

The pull request is “where you get a chance to do peer review of your code,” Maccherone said. But what many people don't realize, especially in the security world, is that you can add any number of checks to the pull request, Maccherone noted. 

Such checks are perfectly timed in the pull request: They “provide feedback at the right moment in time for the developer to stay in flow,” Maccherone said. “If you're doing it while they're coding, and they're trying to get the functionality to work, and you provide security feedback, it'll take them out of flow; it'll take them out of the mindset of ‘get the functionality working.’” 

Basking in the Three Ways Flow.

But after devs make a commit and update the pull request, they’re eager to see if their beautiful code passes all the security checks in automated testing. It’s the perfect time to provide quality feedback, when they’re not trying to stitch new features into the application but are instead getting their code ready for their peers to take a look at. 

The pull request is the mechanism by which a developer asks for that feedback. Providing that feedback right there, at that point, is “certainly the key,” Maccherone stressed. 

The power of code pride

“There's … a psychological motivation that causes the developer to want that feedback at this point in time,” he said. “Developers … [are] proud of the code they wrote this morning, they want their peers to like it. That's what the pull request gets: It gets a peer review and other stuff. If these automated checks say, ‘Hey, your code has this problem,’ they're going to sit up, they're going to pay attention, they're going to care about that.” 

Also, at that point, the code is fresh in their mind: They understand how it works, and the feedback will immediately make sense to them, because they just wrote the code that morning. Thus, they can quickly resolve issues. 

“If you wait, and you triage the vulnerabilities, and it takes three weeks for that triage to occur, then it goes on to a product backlog …  And it's two or three months later, before someone on the team … gets to work on it, it's completely out of flow,” Maccherone went on. 

Plus, by that point, it might not even be the developer who wrote the original code. If it is, s/he might have forgotten what that code was. 

Death to bloated SLAs

That, in fact, is why Maccherone spurns any service-level agreement (SLA) that’s  greater than a day. In fact, they’re actually “harmful,” he asserted. 

The same day you wrote the code is the quickest time you can fix it, he said. 

“If you build up a pile of these things. It takes 10x more energy to fix it. That pile will just keep growing. You never get out of that mode,” he said

So, to reiterate the original poll question: What do you want to spend your time doing? 1. hunting for more and more and more code deviations that will never get addressed, instead merely getting glommed on top of the crud pile, or 2. resolving the bugs you’ve found, quickly and efficiently?

It’s clear to see why, hands-down, hands go up for no. 2. 

Power to the pull request!