Cybersecurity loves its mottos. It has fatalistic ones like, "Users are the weakest link," that shift failure onto users and absolve security teams from their bad UX, absent controls, and inadequate processes. It has generic ones like, "Shift left," that repackage old concepts like improving software quality at the design and implementation stages.

And now there's the emphatic modifier of "machine speed" that appears in dramatic discussions of agents and LLMs. Machine speed is trotted out as an apparently novel concern for cybersecurity in the age of AI, which is unfair to all the while loops, shell scripts, and automation that preceded this era of excitement.

Artificial Activities vs. Optimal Outcomes

Machine speed feels like it misdirects attention to tactical improvements at the expense of better outcomes. Meaningful improvements, to be sure, as measured by speed of discovering flaws. Yet it’s unclear if they’re more economically efficient improvements, let alone security ones. There’s little data on the rate of flaws found or events analyzed per unit of time per dollar. And you can't neglect the cost of developing and confirming the quality of patches for those flaws.

That eternal cycle of find flaw, fix flaw, find, fix, find, fix again is boringly narrow. It's artifical activity in the sense of being a false strategy. It promises easy metrics like CVE counts and days to patch. (Or events analyzed in the world of SOCs.) Sure, those are observable. But they're really just measuring how busy security and developer teams are with short-term tasks. It doesn't matter if agents augment those teams; that just adds agents burning tokens to teams burning time in the cost equation. Cybersecurity should have observable outcomes. I prefer proactive ones that reduce or eliminate flaws altogether.

One way to reduce flaws is to increase sensitivity, as in refining the threshold for which a flaw or suspicious event requires action. This is hardly an insightful recommendation. It’s an old, well-established technique. My disfavor for it is that it doesn’t improve the creation of code. It's reactive. It merely adjusts the baseline quality at which flaws are considered impactful or events considered supicious.

Another way to achieve a favorable outcome is by reducing attack surface and reinforcing security boundaries. This is the domain of proactive design choices. It's another old concept and one that feels like a fleeting wish whenever I whisper its possibility. It seems to have far less uptake than refining filters.

A Completely Wishful Enumeration

React and related frameworks are great examples of secure by design principles. They make XSS scarce by addressing the flaw at its root cause — maintaining a strict distinction between the structure of HTML nodes and the content that lands in those nodes. The frameworks render arbitrary, untrusted data such that the semantics of a page are unaffected by the syntax of that data. The frameworks foster design patterns that minimize security mistakes and make it easier for linters and IDEs to emit warnings when those mistakes happen.

CSRF is similarly diminished. Cloud providers have changed APIs and default settings to make flaws like SSRF and accidental exposure of resources less common. Sadly, it took several years for those defaults to take hold and become more common than their insecure predecessors. If there's a machine speed to strive for, it's in establishing more secure defaults in existing tech stacks.

C and C++ code is getting better through compiler feedback, CPU design, and standards. Rust is also making inroads into these areas. The Linux kernel won’t replace its millions of lines of C with Rust, but it will use Rust in notoriously insecure areas like device drivers. Seeing memory safety flaws disappear from device drivers is a favorable outcome. It's nice to make the patching process faster, but having to deal with fewer patches in the first place is far more appealing.

I suspect that some orgs are reluctant to embrace an approach that reduces their attack surface because it requires more engineering effort and, cynically, because that effort is more difficult to measure. Refactoring doesn't have the familiar monthly trends and charts of CVE counts and days to patch that vulnerability management programs produce.

Saying that 100 patches were applied within their however-many day deadline sounds informative. We could further tweak it by making it less passive (which app owners met or missed deadlines), more precise (slicing by distribution of severity), or more comparative (percent change from previous timeframe). Those aren’t bad refinements, they just reinforce a retreat to numbers that are easier to gather and report on.

Saying you introduced a design change that eradicates a vuln class doesn’t fit into that vuln management framing of short-term trends that track how much the cycle of security patches keeps people busy. But improving a design to avoid bugs or improve observability is still a good thing to do.

In terms of metrics, there should be a step change where the count of a vuln class goes to zero. (Or something to that effect, such as flaws per LOC.) But it wouldn’t be meaningful to count the lines of code that changed as an indicator of security, especially when pitching this as a proactive exercise. You’d have to estimate the engineering effort, then make the case for how that upfront expense saved time by reducing bug bounty payouts, limiting unpredictable patching, and avoiding potential incidents later on.

The goal of getting rid of CWEs instead of CVEs still falls more into fiction than real-world fixing. If there's an acceleration to find for appsec, it's in reducing the prevalence of CWEs, not chasing the patches for CVEs.

Smashing the Stack for a Fast Profit

So, what about the “attack faster” associated with this ambiguously defined, yet implicitly menacing, machine speed? The assertion that hackers are hacking faster implies that orgs need to change their strategies for defense. But those implications are couched in the pseudo-profundity of phrases like, “The game has changed” or “The AI era demands” or some phrase that mentions evolution.

Ask for an explanation of machine speed and the conversation too often continues into metaphor or vapid abstraction. It becomes a superficial equation — a faster attacker necessitates a faster defender. Just be faster.

I don’t think LLM-generated packets traverse a firewall faster than those from my browser. I don’t think post-compromise shell scripts execute faster when they have the artisanal touch of artificial agentry. There's clearly a speed to automation, whether from shell scripts or cascades of prompts, in chains of events and sequences of activity in compromises. That's a far more interesting problem to ponder instead of simply tracking days to patch.

Where are the indicators that machine speed is forcing fundamental changes to security controls? What proxy variables might we look at? What differences in attacker behavior would we look for?

How has machine speed affected cyberinsurance rates? What CISA revelations do they inspire? This would have been a wonderful topic for a CSRB to investigate.

At the top of my questions about whether “machine speed” means anything beyond a trite shorthand for automation by agents is what the phrase means, or at least implies, for defenders to take action on. There have always been examples of attacker automation, from the venerable Metasploit to commercial tools like Cobalt Strike to tons of open source tools from the bug bounty crowd.

There’s admittedly a faster pace of expanding privileges or pivoting across environments from automation. Yet those actions should still trigger security events and, even better, can be blocked by decidedly non-machine speed controls like network isolation and restricted access.

Time and Relative Dimension in Security

Even if so-called machine speed highlights the need for more controls, an org can apply those controls regardless of whether an army of agents is attacking them. Most controls work regardless of how fast or slow automation affects an attack. (One exception might be process-based controls.)

To reach for an awkward metaphor, it’s like mixing the dimensions of time and space, where machine speed speaks to attacks happening more quickly in time, whereas defenses are more effective when spread across space, as in reducing an attack surface or establishing boundaries.

The conversations around machine speed echo those from decades earlier about zero-days (here defined as a vuln known to attackers for which a patch is not available). One defense against a zero-day is to have the ability to obtain and apply a patch quickly. But, again, that's a very reactive stance. There are proactive steps like isolating resources, blocking egress traffic, monitoring traffic flows, setting granular access controls, and so on.

In other words, the defensive focus is on the larger system and how controls can make exploitation more difficult or post-exploitation activities easier to constrain and detect.

If there's another motto that cybersecurity loves, it's "Attackers only need to be right once, defenders need to be right all the time." Reducing your attack surface reduces where those opportunities are for an attacker to be right. Adding observability increases those situations where attackers have to still be right in order to avoid detection.

Cybersecurity folks flocked to Las Vegas this month for a confluence of conferences.

Vegas is also very, very hot in August – about 113 degrees on the Fahrenheit scale.

In Celsius, that’s roughly 45.

In CVSS, that’s basically a 10.

On the CVE scale, more than a few hundred.

On the people scale, tens of thousands.

On the token spend scale, at least several million.

And then there’s the scale of LLM-written LinkedIn posts, the value of which is absolute zero.

Prompting for Patches That Fix Vulns Without Adding New Ones (ep. 394)

In the last few months, LLM-based techniques have increased the volume of security flaws. And before that, orgs were already turning to LLMs to write code. So, what happens when orgs lean on LLMs to create patches for those security flaws? Keith Hoodlet gives an exclusive early look at his team's recent research into the success, quality, and failures of LLM-generated security patches.

The results revealed outcomes that ranged from robust, effective patches to patches that incorrectly changed the software's behavior to patches that introduced new vulns to patches that failed to fix the original vuln while also introducing a new one.

The research considered factors like quality and correctness of prompts, complexity of the target software, programming language, and expertise required to understand what a robust patch should look like. If you're going to spend tokens on fixing security flaws, you want a feedback loop that fixes them correctly – not a perpetual patching machine that introduces new flaws with every LLM iteration.

Prompting for Patches That Fix Vulns Without Adding New Ones (ep. 395)

Finding flaws has always been a focus of appsec. And with so many open source projects and open weight models, orgs have the opportunity to modernize code reviews and pentests. Rishi Sharma describes the motivation behind creating a platform of LLM-driven security tools and the effective ways to keep the tools in scope, on budget, and targeted to engineering teams. We talk about how prompts influence LLM activity, as well as using external constraints to keep the LLMs on task.

Even if finding flaws is a major focus of appsec, its ultimate goal should be delivering secure software and systems. We touch on some of the ways to keep bugs from creeping back into software and why it's more important to care about vuln classes than vuln counts.

Augmenting Threat Intel Analysis with Agents (ep. 396)

All sorts of cybersecurity disciplines are adopting agents to help humans save time and automate routine activities. Sai Kiran Uppu describes his work on creating a platform for agents to analyze external threat intel, examine internal systems, and present triage decisions to operators. This type of work is especially useful to orgs that deal with petabytes of data and thousands of systems. As Kiran notes, it's important to keep that scale from blowing up your budget or turning triage into a procession of false positives.

Ideally, the kind of threat intel that's paying attention to attack trends and searching internal systems for evidence of compromise turns into proactive defenses. We talk about some of the ways to engage developers to improve security visibility into their services and harden their designs against common attacks.

Applying Zero Trust Principles to Agents (ep. 397)

Sandboxing, least privilege, and monitoring are well-known controls in terms of the defenses they provide against unexpected and unauthorized actions. But being well-known in theory doesn't mean they translate to success in practice. Kieran Human describes some of the properties that a good sandbox should have and how monitoring creates a feedback loop to refine allow lists and access controls.

In practice, the potentially unpredictable behavior of an agent barely differs from malware. We talk through some of the ways orgs can deploy agents securely without unnecessarily increasing their attack surface.

Fixing Software Weaknesses Rather Than Just Finding More Flaws (ep. 398)

AppSec has always emphasized techniques and tools for discovering vulns, along with taxonomies and lists for describing them. But just piling up more CVEs into a prioritized patching queue has never been an effective strategy. Nidhi Aggarwal talks about some of the economics and decisions that orgs evaluate when figuring out how to improve and protect their software.

LLMs might be effective assistants for generating code, but only when they have the context of secure patterns to follow. We also talk about what some of the bug bounty data reveals in terms of successful researchers finding more impactful vulns and why the combination of domain expertise and curiosity remains profitable.

I loathe LLM-generated writing, especially when someone launders it as their own work, thoughts, or insights. LinkedIn is a prime example of posts laden with lazy tokenized text. The glut of fabricated text crowds out human voices, which compounds the insult of how text and image generation are derivative of copyrighted material collected without consent or compensation.

I lament how the emdash has become a superficial signal of synthetic thought. I haven't changed my writing style because of that. I embraced the emdash long ago and enjoy the rhythm and visual distinction it provides. I don’t and won’t use LLMs in my writing process. I enjoy the process of thinking — and struggling — my way through a sequence of sentences.

I also see it as respect for the audience.

Readers don’t want to waste their time on something thoughtlessly generated. And so they search for tics of tokenization.

They point to syntactic tells of artificial expression, from that glorious emdash to an abundance of bullet points. Others point to stylistic decorations, from the poker-like tell assumed in a word like "delve" to overused formulations like, “It’s not X. It’s Y,” and throw-away phrases that serve as word count filler between mundane or generic assertions.

They even turn to services that purport to identify LLM-generated text, albeit with questionable accuracy disguised by decimal points and methodologies that vary between grep with a few regexes and guessing with probabilities.

Relying on some particular punctuation is insufficient. It misses the weaknesses in structure, sense, and style that make LLM writing so unappealing. Unfortunately, we don't always have obvious shortcuts to identify and therefore avoid the cruft that comes from prompts.

That's the trap of these time-wasting tokens when their nature isn't immediately apparent. There's a great betrayal in the asymmetry of thought spent reading something when there was zero thought spent writing it.

Write, Revise, Refine, and Read

Bad writing and poor conference presentations predate people spending money to trickle their thoughts through a random number generator. Humans on LinkedIn have generated vapid business-speak since its inception. But I can accept many of those human efforts as people experimenting with their own first drafts. I’ve gone through, and continue to go through, that perpetual process of draft, revision, draft.

Writing is a craft that requires exercise. There’s nothing wrong with scattered thoughts and early drafts. Learning and improvement takes patience and practice. Taking deliberate decisions to iterate through drafts is far more beneficial than re-running prompts. There's no self-improvement or gainful exercise in relegating writing to a machine.

Reading is also a craft. Applying critical thinking to a spec, design, or idea is crucial to cybersecurity. Threat modeling is a loosely structured application of critical thinking. That’s another reason why reading text spewed from an LLM is so annoying – its boring patterns and extraneous phrases offer nothing valuable in terms of intellectual exercise. Cybersecurity doesn't use noise to train spam filters and ML-based detections. Reading skills hardly benefit from the emptiness of LLM-based writing.

Write Often, Read Often

Cybersecurity relies on writing, from explaining vuln research to describing secure designs to sharing knowledge to the occasional blog post about taking security seriously. There's a whole discipline of security awareness predicated on applying communication skills to change user behavior.

Cybersecurity likes patterns and lists. It talks about programming patterns and regex patterns. It draws on TTPs as patterns to vaguely define threat actors. It likes creating lists of things not to do. But if you take away all the exhortations of what not to do, that leaves a void of recommendations about what to do.

If we limit an analysis of writing to a list of “don’ts”, we risk framing writing as an absence of annoying characters rather than building examples of how to be convincing, entertaining, concise, or whatever a particular topic or situation calls for.

Those are the patterns to look for – the patterns that engage an audience. I'd rather seek out writing that makes me think, that makes a topic entertaining, or that captures my attention. It might be a single sentence, a blog post, or a novel. That context and experience informs my own ideas of the kind of writing I want to find more of or to create myself.

That’s why it’s so important to read and write on a regular basis. There’s no single, uniform reader to appeal to. Nor a single writing style to adopt. But there are outcomes to strive for.

To use cybersecurity as an example again, an outcome might be precision and insight into events when documenting a postmortem, being persuasive with trade-offs in a design decision, or educating someone on how to secure their systems.

Write about the steps you took to find a flaw. Write about the steps that failed to find a flaw. Write something you would enjoy reading, that comes from your thoughts and through your voice.

"You hear that Mr. Anderson? That is the sound of inevitability. It is the sound of your token-spend."

– Agent Smith, The Matrix (approx.)

How inevitable that a trade known for celebrating efficiency and optimization would eventually shift from spending as much as possible on tokens to tightening that token budget.

That’s not to say that there’s no place for experimentation and learning. Many developers were driven as much by curiosity about LLMs as they were by a mandate to consume them. But experimentation must at some point turn away from novelty that produces lines of code to more meaningful and measurable outcomes.

And what better measure than the efficiency of token spend. The instigating institutions, self-interested model providers, extolled optimal coding in terms of vibes and volume. And while that blossomed into substantial spending, it left behind a cruft of software that ignored the craft of software.

So what would be some good programming language properties for LLM budgets? What languages have a brevity of syntax and semantics such that they would translate to tinier token creation?

Very Verbose vs. Sufficiently Succinct

TypeScript is an easy example of an expensive language. It adds so much decoration that disappears when transpiled to JavaScript. All those annotations do is help humans understand the context and constraints around the project's data and functions, or give the compiler something to complain about when types don't match. But they’re tedious for LLMs to consume and create. Why bother with so much text that gets thrown away for production? Humans might need those reminders to jog their memory. But LLM coding prompts just need a concise closing clause like, “...and make it secure.”

That’s not to say that JavaScript doesn’t helpfully try to cut corners. Unfortunately, cutting things down like only having 53 bits for integers instead of 64 doesn’t translate to token efficiency. It just occasionally surprises humans. Humans don't look at the world through the lens of floating point numbers. LLMs do.

JavaScript can also be manipulated into minimizing its character set, which we'd hope would translate to tinier tokens. We’ve seen general guidance around this where APIs are encouraged to return sparse markdown instead of HTML in order to avoid spending money on all those angle brackets, tags, and attributes. LLMs don't care about the size or color of a font. They faithfully adhere to the security motto of, "If you see something, say something."¹

Thus, a smaller and shorter input range of characters seems economically favorable. A heading indicated by a leading # ... in markdown is more token efficient than wrapping it within <h1>...</h1>.

JavaScript's object and type model has some quirks that almost help here. For example, rather than use a literal string like alert, you can craft the same string through clever manipulation of objects and arrays like, (![]+[])[1] + (![]+[])[2] + (![]+[])[4] + (!![]+[])[1] +(!![]+[])[0]. This reduces the set of characters to tokenize. Yet sadly, it explodes the amount of tokens to consume. Such is the curse of premature optimization.

Still, punctuation feels like a fruitful path to explore for efficient languages. After all, if we can imbue an ancient telegram token like “STOP” into a period, that’s a good optimization. If we can imbue the semi-colon that signifies the termination of a line of code into a newline character, that’s another. Go removed the semi-colon burden from our humans brains by making the compiler toolchain silently add them in wherever the specs says they should go².

Lisp and its ilk have plenty of punctuation. Their parenthetical policy encourages simplified and condensed source, for example (Parse (:this (kind (:of :thing)))).

(Lisp even has mutable dynamic variables denoted by *earmuffs* and constants between +plus-signs+. That's some powerful punctuation!)

LLMs fly through a cascade of closing characters like that. Lisp's style guide also explains the hierarcy of comments denoted by four, three, or two semi-colons. Again, LLMs just chew through those single characters quickly.

But if we’re going to rely on syntax to shorten our token spend, we can do better.

Probably Effective and Reasonable Language

We can use Perl. Perl must surely be the ideal LLM programming language. It was created for – and excels at – text processing.

Perl, who's pedigree dates back to the late 80s, predicted and practiced the probabilistic nature of LLMs with its early design slogan, “There’s more than one way to do it.”

Seeing three different Perl programmers come up with seven different solutions for the same problem is just the small-scale version of running the same prompt through an LLM more than once.

Like LLMs, Perl is unburdened by expectations of predictability or a dependable nature. Like LLMs, Perl has limitations inherent to context. Take Perl’s list context. It's a stochastic minefield for operators and functions, which perlfunc officially explains as, “In general, they do what you want, unless you want consistency.”

That sentiment sounds like a cruel reformulation of the quote about technology and magic where, “Any sufficiently advanced list operator is indistinguishable from its context.”³

So, we have parallels in unpredictability and comparable challenges in explainability. What about efficiency in taming tokens?

One of the best ways to save on tokens is to not spend them in the first place. In the context of writing code, that means not writing code. Once again, Perl excels in this space by providing a native implementation for unimplemented code with the “yada yada” operator.

It gets better. That operator is otherwise known as the three-dot ellipsis (...), but sometimes three dots is just the same as two dots for a range operator. Ambiguity and surprise are what make LLMs so powerful as text creation systems!

Speaking of operators, you’ll never waste lines of code on no-op expressions – Perl doesn’t have them. Except it does if you really need one. This design flexibility reinforces the slogan’s subtext of, “There’s more than one way to spend tokens.”

Integers in Perl are superior to JavaScript. You get a full 64-bit representation rather than JavaScript’s paltry 53. And don't worry about those extra bits adding bloat. You can famously implement RSA encryption and decryption in three lines of Perl.⁴

Notably, in the 90s those three lines of code were briefly considered a munition restricted by U.S. export controls. Perl was abetting government sanctions well before modern models got their cybersecurity capabilities on a gradient descent into danger.

And where TypeScript wastes text on semantic hints like whether a string represents a name or an email or something else that humans have to remember when writing code, Perl eschews those descriptors.

In fact, one of the emergent properties of a slogan that encourages, “...more than one way to do it,” is that reading someone else's Perl code ranges from frustrating to near-impossible for humans. But not for an LLM.

One Token Over the Line of Code

LLMs don’t need to explain their work, they don’t need to produce human-readable code when the reviewers and consumers of that code are merely other LLMs. All those abstractions of types and decorators and comments are just crutches for human brains that can’t handle the precision of floating point numbers.

Humans long ago realized that measuring their success in creating software was poorly correlated to lines of code. There were so-called "code golf" competitions where they used their knowledge of language quirks to implement a function in the fewest characters possible. But that was an exercise driven by intellectual curiosity that had nothing to do with cost-savings.

But what if all that syntax isn't as important as what the code actually does? What if the semantics of symbols meant compilers could apply better security to data flows, control flows, and memory manipulation? Maybe TypeScript makes collaboration and maintenance easier. Maybe Rust makes heap allocation mistakes harder.

Maybe it was our natual written and spoken language all along that made better software. Perhaps not completely natural, as betrayed by the stilted and formal language of RFCs and similar specs. But it's close.

Even the emphasis of capital letters in words like SHOULD, MAY, and MUST mostly works on humans about as well as LLMs. But it's that mostly qualifier that leads to security flaws. And it's the ambiguity of language that leads to disparities between designs and implementations. And those disparities impact software quality.

So maybe we need a better language for humans and for programs. Maybe math, where operators operate predictably and statements can carry proofs that guarantee behavior. Those kind of concrete properties could be a boon to real-world security. No human mistakes, no LLM hallucinations. Surely there's nothing irrational or imaginary to such a choice.

Meanwhile, I'll continue to enjoy typing text in whatever language by hand. Ultimately, the only tokens I care about are the ones that represent creatures on maps in role-playing games.

We’ve just crossed through the halfway point of the year.

Is your vuln backlog half empty or half full?

Do you have a better strategy than half-heartedly applying patches?

What if we turned half-measures based on checklists into full-fledged designs rooted in secure principles?

After all –

I don’t know half of this year’s CVEs half as well as I should, and I like less than half of vuln patching emphasis half as well as it deserves.

Defense-in-Depth Strategies for Securing Mobile Applications (ep. 390)

Mobile apps need resilient design strategies to ensure their business logic and workflows remain secure. We dive into a defense-in-depth discussion with Ryan Lloyd about the threats that mobile apps face, the threat models devs should reason through, and the unsurprising reasons why mobile development requires server-side controls.

I particularly like talking about flaws in business logic and workflows because their discovery requires creative thinking about potential misuse and abuse. After all, the attack payloads are almost always valid and technically correct traffic. They just happen to be hitting weaknesses in a state machine, edge cases in expectations, or a surprising sequence of steps that developers didn't anticipate.

This interview was sponsored by GuardSquare.

Discovering & Securing Your AI Agent Attack Surface (ep. 391)

While LLMs and agents are new to appsec and everyone else, a lot of AI security requirements translate to well-known API security requirements. Jeremy Snyder frames the OWASP LLM Top 10 into five layers in order to understand and prioritize their attack surface.

Similar to securing APIs, having a solid foundation of observability and identity makes it easier to avoid unexpected activity from agents and to set boundaries on their behavior. That's especially important for constraining prompt injection and excessive agency.

Prompt injection is the unexpected activity that most orgs are likely to get an unpleasant surprise from. But unbounded consumption runs a close second. And that unbounded consumption can come from surprising areas. We talked a bit about using LLMs to evaluate intent and output handling, but burning more tokens to check security outcomes from burning other tokens isn't always the most reliable or efficient solution. And that's where we're looking for more interesting and useful appsec approaches.

MacOS Security Design Features, Flaws, And Futures (ep. 392)

Appsec often frames usability and security requirements as at odds with each other. Apple's software has famously emphasized the importance of usability while also establishing a solid security foundation.

Patrick Wardle talks about how he's seen malware shift from Windows to macOS, how Apple's aggressive stance on deprecation benefits security, and the areas of the OS where he sees plenty of opportunity for more security research. In fact, you'll be able to catch more of his research at this year's Black Hat and DEF CON!

We discuss how developers make defensible design choices and a few security principles that any app developer should keep in mind regardless of their programming language or operating system.

Inside the OWASP Agent Security Regression Harness Project (ep. 393)

Orgs need to be able to use agents, MCPs, and LLMs in ways that don't lead to unexpected actions and undesirable outcomes. The OWASP Agent Security Regression Harness project is an approach for defining customizable scenarios and testing whether those systems fail against known security threats.

Mert Saltimaz talks about the background of the project, how orgs can use it as they bring more LLMs into their environment, and how the project intends to grow. Importantly, we also talk about the security controls and designs that orgs can build around the systems and data that models interact with in addition to evaluating the security of the agents and agent harnesses themselves.