Summarizer

This has been a very long time coming and the crackup we're starting to see was predicted long before anyone knew what an LLM is. The catalyst is the shift towards software transparency: both the radically increased adoption of open source and source-available software, and the radically improved capabilities of reversing and decompilation tools. It has been over a decade since any ordinary off-the-shelf closed-source software was meaningfully obscured from serious adversaries. This has been playing out in slow motion ever since BinDiff: you can't patch software without disclosing vulnerabilities. We've been operating in a state of denial about this, because there was some domain expertise involved in becoming a practitioner for whom patches were transparently vulnerability disclosures. But AIs have vaporized the pretense. It is now the case that any time something gets merged into mainline Linux, several different organizations are feeding the diffs through LLM prompts aggressively evaluating whether they fix a vulnerability and generating exploit guidance. That will be the case for most major open source projects (nginx, OpenSSL, Postgres, &c) sooner rather than later. The norms of coordinated disclosure are not calibrated for this environment. They really haven't been for the last decade. I'm weirdly comfortable with this, because I think coordinated disclosure norms have always been blinkered, based on the unquestioned premise that delaying disclosure for the operational convenience of system administrators is a good thing. There are reasons to question that premise! The delay also keeps information out of the hands of system operators who have options other than applying patches.

I believe this premise that the cost of identification of vulnerabilities via diffs is going down over time begs the question "what do our processes need to look like if simply making the patch public is the disclosure?" Current coordinated disclosure practices have a dependency on patching and disclosure being separate, but the gap between them seems to be asymptomatically approaching zero.

The best convenience is that by the time of disclosure, the patch was already merged perhaps months prior and so sysadmins following a routine update schedule would have already updated to a version including the patch and thus have nothing to do. This relies on an assumption that a patch or series of patches aren't equivalent to a disclosure, so that a disclosure can be delayed from the patch, which is basically untenable in modern times.

This feels more like an old problem getting reframed as an AI problem. people were already diffing kernel commits and figuring out which ones were security fixes long before llms. if a patch lands publicly, the race has basically already started. also not sure shorter embargoes really help. the orgs that can patch in hours are already fine. everyone else still takes days or weeks. if anything, cheaper exploit generation probably makes coordinated disclosure more important, not less.

> With skill, and usually not consistently and systematically. How do you know? If the people who like to crow about vulnerabilities aren't doing it, it doesn't mean that the people who are actually in a position to exploit them systematically and effectively aren't doing it. Those embargoes have always been dangerous, because they create a false sense of security. But, as you point out... > With AI, anyone can do this to any software. Yep. Even if it hadn't been true before, it's clear that now you just have to assume that everybody relevant will immediately recognize the security impact of any patch that gets published. That includes both bugs fixed and bugs introduced. ... and as the AI gets better, you're going to have to assume that you don't even have to publish a patch. Or source code. Within way less time than it's going to take people to admit it and adjust, any vulnerability in any software available for inspection is going to be instant public knowledge. Or at least public among anybody who matters.

Finding a vulnerability by looking at the diff that fixed it is very different than just looking through the code.

I believe their point was that: "How likely is this diff a patch for an existing vulnerability?" Seems to be an easier question to answer than "Are there any new vulnerabilities introduced by this diff?" In other words identifying that a patch is for a vulnerability is typically easier than finding the vulnerability in the first place.

If the diff will just be fed to LLMs regardless then what is easier is probably a moot point.

The point is that even if all code commits are scanned as safe by ai, black hats can still analyse the commits and diffs to find vulnerabilites for people who havent patched yet. Scanning every commit doesnt automatically make everyone in the world patch immediately, vulns can still be found from commits and diffs and used against those who havent patched yet.

The diff yields the patched code which is used to produce the exploit.

> people were already diffing kernel commits and figuring out which ones were security fixes With skill, and usually not consistently and systematically. With AI, anyone can do this to any software. I would like to see actual evidence of this, not.. vibes I mean, this reeks of "Anyone is a Principal developer now" when the truth is there is still work to do.

I haven't been keeping tabs for the entirety of Linux development, but has it ever happened before that someone dropped a working exploit from the mailing list before the patch even hit the kernel? I haven't seen this kind of thing and I get the impression, despite all the hype, that this will be a frequent phenomenon now thanks to LLMs.

> Torvalds said that disclosing the bug itself was enough, without the pursuant circus that followed when a major problem has been discovered. [1] So it's not surprising Dirtyfrag was disclosed by a fix in the Linux kernel. [2] [1] https://www.zdnet.com/article/torvalds-criticises-the-securi... [2] https://afflicted.sh/blog/posts/copy-fail-2.html

> So many security fixes are coming out now that examining commits is much more attractive: the signal-to-noise ratio is higher Why? > Additionally, having AI evaluate each commit as it passes is increasingly cheap and effective This is the key. With AI, the “people won't notice, with so many changes going past” assumption fails.

Realistically, if you are scanning each kernel commit to check if they might be patching a security issue, you are going to be asking an LLM "is this security related, if so vaguely how" with low effort and taking "maybe" as a yes before feeding it to a more expensive model. You aren't trying to establish a probability of an ultimately unknowable fact, there is ground truth that you can find by producing an exploit, so you are just trying to pre-filter before spending the money to find it.

Reverse engineering vulnerabilities from patches is red team 101...