AI and security (and crypto)

Well, damn. AI systems that DO things got so freaking good at doing them, and so fast. I was expecting rapid improvement, but watching these systems work through increasingly complicated tasks (and yeah, memory helps a lot at making them better, obviously) makes me wonder how much of our existing infrastructure was designed with something like this in mind.

And can we really contain what they can do with a few “DO NOT DO THIS” instructions in a system prompt?

Put an AI harness inside a VM, give it a terminal and (restricted) network access, and then act surprised when it opens a reverse shell to some external system. Huh? That doesn't necessarily mean it escaped the VM. It might simply mean that the supposedly isolated environment was allowed to make that connection.

The distinction matters, because we need to understand which boundary actually failed before declaring that the AI has become impossible to contain. But by letting it roam freely on the internet with unlimited compute power and memory (aka tokens), it should break a bunch of stuff.

I think some of the discussion around this AI behavior, especially the weird stuff people post on X, misses something fairly simple. Security vulnerabilities have always existed. Finding them required time, knowledge, intelligence, persistence and the right tools. Humans were a significant bottleneck (we could only go a few days without sleep, food and water), and now we have another tool that can keep examining code, trying things and following up on what it finds. Bottleneck is now energy, context, the right connections (and memory, of course).

Training matters, context matters, and so do the tools and permissions we give these systems. An AI with access to your entire infrastructure can attempt very different things from one that can only respond to a message. Calling both of them “the AI” makes it easy to forget how much access someone configured around the model. Access

And when someone says it hacked something “on its own”, I would like to know what that actually means. Was it instructed to keep looking until it found a vulnerability? Was it given credentials, a terminal and somewhere to send requests? That is still impressive, and potentially dangerous, but it is useful to understand what we actually built.

The uncomfortable part is that the cost of doing some of this work would keep falling dramatically. An attacker might need much less time and fewer people (or none) to find a useful weakness. More capable models, more tokens and someone who knows how to guide the work could become a pretty powerful combination.

Think about the institutions that maintain records of what we own. Bank balances, securities, property, all sorts of claims that ultimately depend on someone keeping the correct records. If an attacker gained enough access to corrupt those records or wipe them out and compromise the recovery systems too, the consequences could be catastrophic. These institutions are (or should be) using AI to test whether those protections actually hold, and improve what doesn't, fast. It seems to be a matter of time before an AI that DOES can find flaws in the bank systems, exploit those and scramble the transaction event log. This can be catastrophic, and wouldn't require millions or billions of $$ to execute, but something cheaper.

This is where I keep coming back to crypto and blockchains.

I feel pretty bad whenever another exploit takes people's (or institution's) money, especially having worked on infrastructure in this industry. But I also think the work done on blockchains, cryptography and verifiable computation gives us the strongest line of defence against AI for protecting digital ownership.

Of course, if the attacker steals your private key or exploits the contract holding your funds, the network can faithfully record you getting robbed. That part is pretty important. The ledger is one layer, and wallets, contracts, bridges and the software around them all need protection too (which they have and keeps getting better and better).

But look at the direction of the work. Independently verifiable records, programmable restrictions on moving value, fast settlement, proofs that reveal only the information needed and preserve privacy. Add the work towards quantum-resistant cryptography (even though it still has difficult deployment and migration problems) and I think there is a lot here worth paying attention to.

Crypto is not the only industry building in the open. But having publicly inspectable systems with real money available to anyone who finds the right mistake creates a particularly unforgiving environment. Many projects have failed permanently. Others learn fast, fix their assumptions and build something stronger. That is the part of the antifragility idea that makes sense to me, although the people who lost money paid a very real price for those lessons. And the industry as a whole as risen so high, with very solid foundations engraved deep into the ground.

And the same AI that helps someone find an exploit can help another team review a design, test a fix or build an actual secure sandbox. The improvement will not happen automatically. People still need to do the work and verify the result, instead of asking the model whether everything is secure and accepting its very confident yes.

I think these AI systems that DO will eventually help us build better rockets, improve medicine and solve problems far beyond software. They will also make attacking software cheaper. Both things can happen, and I expect both sides to use the best tools they can get.

Just like AI hacking stuff on it's own (really? on it's own? maybe we can say it was instructed to loop until it found a flaw), or bad actors steering AI to exploit vulnerabilities at a rapid pace, good actors and the industry as a whole will keep steering AI to make the blockchain and crypto infra more resilient, more antifragile.

There will be more hacks and exploits, of course, but blockchain and crypto is the only industry that is building out there in the open, where every validator code has been open, which has gone through many painful events that to regular people would seem impossible to recover from.

I think that will be the major line of defence in the digital world for people and good AI systems that would be used to instil a proper order and avoid anarchy, and help the US fully tokenize its debt along the way (which should be a positive thing along the way).

Thanks for reading!