Press Space to continue
Curated tech Twitter/X posts on AI, startups, and product development—signal without the doomscroll, updated on weekdays.
Press Space to continue
Press Space to continue
Project Eleven CEO @apruden08 warns AI will put everything exposed to the internet under constant attack, just as crypto has been for years: "No one cares about security until they do. What people are coming to appreciate about security in the AI era is that everything that’s exposed to the web will be under constant attack." "The cost of attacking this infrastructure, if you’re a hacker, has dropped to basically zero. You can spin up any amount of agents and start pressure testing…
Assumes three things: (1) that the state of the art models are widely available/open-source. That is contrary to what my expectations are w.r.t. incentives. Whether for safety or revenue, OpenAI and Anthropic have reasons to gate their best cyber models and make you pay up for an AI-audit basically. That doesn't slow down sophisticated attackers much: nation-states will pay or build their own anyway. What it does is make AI-assisted defense something only large organizations can afford, leaving the long tail, where most of the easy targets are, exposed. (goes without saying that open source libraries maintained by volunteers are probably toast no matter what) (2) that the security vulnerabilities themselves are "shallow" and easily detectable. If bugs are deep, finding them takes a lot of tokens per line of code, and that's where the asymmetry bites: because defenders have to secure the whole surface area of their codebase against attack, dollar for dollar they will not be able to spend as much in tokens to *harden* every LOC as the attackers will be able to spend to *break* one specific point. A few reasons to doubt (2): a. The PHP vulnerability was a very deep bug that many people overlooked. b. It's been shown that just spending more tokens can get better results. (e.g. the Claude Riemann-Zeta result, where the engineer basically just kept prompting the model to "keep trying") c. ecdsa dot fail, the more tokens get thrown at the problem, the more progress gets made. Implies that spending more on tokens surfaces more insights as you get deeper into the problem. On their own, b. and c. cut both ways: if more tokens produce better results, defenders get more tokens too. They favor the attacker because of the asymmetry: the defender has to spend tokens hardening every line, while the attacker only has to spend them on one. (3) that the pace of software development slows down to the pace of security engineering. If anything, AI is vastly *accelerating* the product/engineering cycles, and exponentially *more code* will be produced in the coming years. Hard to imagine that security tools will be able to keep pace (security is always ignored by everyone until it's impossible not to). Defenders do have home-field advantages: source access, and the chance to scan code before it ships. But that only helps the orgs that actually do it, which brings us back to (1) and (3). So it's not obvious to me that AI favors the defense. In fact, at the limit things will settle to roughly where they stand today: attacker has slight advantage based on the asymmetry of resources they can bring to bear against one, carefully selected target. AI tools can certainly help improve security, but not enough to overcome this fundamental asymmetry and flip the advantage to the defender, imho.