BLOG
Blog
What we are researching, and what we learn along the way.
Models read code. They don’t know how it runs.
Put the same code on a different chip and its speed, memory use and execution path all change. We want models to learn that.
A binary-analysis agent has to show its evidence
On 72 binary-analysis tasks BinReX succeeded 83.3% of the time. Codex with IDA, on the same GPT-5, managed 43.1%. The difference comes down to looking up similar functions first, and backing every step with evidence.
How do you know when a deployed model starts getting it wrong?
Models for compiler optimization and bug detection work at launch, then quietly get worse. PROM uses a statistical test to flag the predictions it can’t trust, and relabeling under 5% of samples brings performance back.
Can LLMs fix memory bugs?
We tested ten LLMs with SecureMind. The best at repair, o1, fixed 37.1%, and none fixed a bug that spans several functions. Detection was easily thrown off by a variable name.
Letting a vulnerability detector see how code runs
Reading code as text, a model will flag a division by zero that can never run. Concoction adds run-time information from symbolic execution on a few chosen paths, and earned 37 new CVE IDs across 20 open-source projects.
Before RL can tune a compiler, someone has to design the RL
How to represent state, what reward to use, which algorithm: people used to try these one by one. SuperSonic leaves it to search, and against eight existing methods on four compiler tasks it usually comes out ahead.
Next step