Vegas sign

Photo by Kalle Schmitz on Unsplash

Cybersecurity folks flocked to Las Vegas this month for a confluence of conferences.

Vegas is also very, very hot in August – about 113 degrees on the Fahrenheit scale.

In Celsius, that’s roughly 45.

In CVSS, that’s basically a 10.

On the CVE scale, more than a few hundred.

On the people scale, tens of thousands.

On the token spend scale, at least several million.

And then there’s the scale of LLM-written LinkedIn posts, the value of which is absolute zero.

Prompting for Patches That Fix Vulns Without Adding New Ones (ep. 394)

In the last few months, LLM-based techniques have increased the volume of security flaws. And before that, orgs were already turning to LLMs to write code. So, what happens when orgs lean on LLMs to create patches for those security flaws? Keith Hoodlet gives an exclusive early look at his team's recent research into the success, quality, and failures of LLM-generated security patches.

The results revealed outcomes that ranged from robust, effective patches to patches that incorrectly changed the software's behavior to patches that introduced new vulns to patches that failed to fix the original vuln while also introducing a new one.

The research considered factors like quality and correctness of prompts, complexity of the target software, programming language, and expertise required to understand what a robust patch should look like. If you're going to spend tokens on fixing security flaws, you want a feedback loop that fixes them correctly – not a perpetual patching machine that introduces new flaws with every LLM iteration.

Prompting for Patches That Fix Vulns Without Adding New Ones (ep. 395)

Finding flaws has always been a focus of appsec. And with so many open source projects and open weight models, orgs have the opportunity to modernize code reviews and pentests. Rishi Sharma describes the motivation behind creating a platform of LLM-driven security tools and the effective ways to keep the tools in scope, on budget, and targeted to engineering teams. We talk about how prompts influence LLM activity, as well as using external constraints to keep the LLMs on task.

Even if finding flaws is a major focus of appsec, its ultimate goal should be delivering secure software and systems. We touch on some of the ways to keep bugs from creeping back into software and why it's more important to care about vuln classes than vuln counts.

Augmenting Threat Intel Analysis with Agents (ep. 396)

All sorts of cybersecurity disciplines are adopting agents to help humans save time and automate routine activities. Sai Kiran Uppu describes his work on creating a platform for agents to analyze external threat intel, examine internal systems, and present triage decisions to operators. This type of work is especially useful to orgs that deal with petabytes of data and thousands of systems. As Kiran notes, it's important to keep that scale from blowing up your budget or turning triage into a procession of false positives.

Ideally, the kind of threat intel that's paying attention to attack trends and searching internal systems for evidence of compromise turns into proactive defenses. We talk about some of the ways to engage developers to improve security visibility into their services and harden their designs against common attacks.

Applying Zero Trust Principles to Agents (ep. 397)

Sandboxing, least privilege, and monitoring are well-known controls in terms of the defenses they provide against unexpected and unauthorized actions. But being well-known in theory doesn't mean they translate to success in practice. Kieran Human describes some of the properties that a good sandbox should have and how monitoring creates a feedback loop to refine allow lists and access controls.

In practice, the potentially unpredictable behavior of an agent barely differs from malware. We talk through some of the ways orgs can deploy agents securely without unnecessarily increasing their attack surface.

Fixing Software Weaknesses Rather Than Just Finding More Flaws (ep. 398)

AppSec has always emphasized techniques and tools for discovering vulns, along with taxonomies and lists for describing them. But just piling up more CVEs into a prioritized patching queue has never been an effective strategy. Nidhi Aggarwal talks about some of the economics and decisions that orgs evaluate when figuring out how to improve and protect their software.

LLMs might be effective assistants for generating code, but only when they have the context of secure patterns to follow. We also talk about what some of the bug bounty data reveals in terms of successful researchers finding more impactful vulns and why the combination of domain expertise and curiosity remains profitable.

Subscribe to catch these episodes and more! Then go check out the previous recap.