Blog

Models read code. They don’t know how it runs.

Put the same code on a different chip and its speed, memory use and execution path all change. We want models to learn that.

Start with about the most ordinary code there is:

for (int i = 0; i < n; i++)
    sum += a[i];

Any coding model will tell you what it does: it adds up an array. Ask how long the loop takes on an x86 server, a GPU and a Chinese-made accelerator, whether it is bound by compute or memory bandwidth, or whether a different compiler flag would help, and the model is mostly guessing.

The reason is simple. Today’s code models learn almost entirely from code as text. They have seen enormous amounts of source, commits and discussion, and very little of what code does when it runs: how long it took, how much memory it used, which branches it took, which line it sat on waiting for data. None of that is in the text.

Where it starts to matter

For everyday application code the gap is easy to miss. It shows up when you move a system onto a different chip, or need to know why some code is slow or crashes now and then. Reading the text is not enough.

Most of what we see at customers is this kind of problem. The model is deployed and it runs, but slowly. The new accelerator lacks an operator the old code relied on. The scanner reports a pile of vulnerabilities, half of them false, and engineers spend longer triaging than fixing.

A computational world model

Our approach is to let the model watch code run. We run the same code for real across architectures, devices and inputs, record its performance, memory and resource use and execution paths, and train on that. We call the result a computational world model: give it some code and a target environment, and it estimates how the code will behave there.

  • Code in: the same code, with different scenarios and inputs.
  • Run and observe: real runs on CPUs, GPUs, Chinese-made chips and other accelerators, with their behavior recorded.
  • Learn: model the code’s performance, memory and execution paths on each kind of hardware.

Last year Meta released a Code World Model with a very similar name. It is about what code computes: it simulates variables and state step by step in a Python environment. We care about how the same code runs on different chips. Both are useful; they answer different questions.

What it is for

For now we use it for three things.

Security. If you know which paths a program actually takes when it runs, vulnerability detection no longer has to rely on whether the code looks like buggy code.

Performance. Knowing where the bottleneck is gives optimization a direction, and tells you whether a change really made things faster or just happened to on one machine.

Migration. When you move a model and its toolchain to another chip, you can see in advance what has to change and whether the result will be slower.

Each of these rests on papers we have published, and the posts that follow take them one at a time.

What is still missing

Run-time data is expensive. Every chip we add means more machines, another environment and a lot of run time. Over the next year or so our focus is collecting run data on more hardware, getting the model to predict performance, memory and execution paths directly, and mining the cases it gets wrong to feed back into training.

If you have code that needs to move to a new chip, or code that is slow for reasons nobody can quite pin down, we would like to hear about it.

BLOG

More posts

Next step

AI that understands how code runs.

Contact