TL;DR
- MIT asked OpenAI's GPT-6 Astra to fix a small AI model by editing its weights directly. It changed 0.016% of them and the maths score went from 7.9% to 48.1%.
- Much of that came from stopping the model going round in circles. On a model that didn't loop, the same approach gained less than a point.
- Our read is that model files now need the same care as code. A known base, a diff and someone who signs off.
A very small edit
On Monday, four researchers at MIT asked OpenAI's GPT-6 Astra to repair a small AI model without retraining it. Astra opened the model up, found what made it go round in circles and changed 131,072 of its 800 million weights. That's 0.016%, a few plugs moved on a switchboard with millions of them.
The model's maths score went from 7.9% to 48.1%.
The authors call this weight-space programming. A general AI model reads and edits another model's weights directly (the numbers that decide how it behaves). There was no training run and no new training data. Matthew Berman called it "the most important paper in AI right now" on Wednesday, and it hit almost 3,000 points on Hacker News.
We think the work is real. But much of that 48% is a broken model being unblocked. And if one model can now change another, somebody in the company has to check that change before it goes live.
What the 48% is made of
The first tests used very small networks. In one, a small AI drives a car in a simple physics game. To get over a ramp, the car has to back up first to build speed. The AI was trained for a billion steps and never learned that. It drove straight at the ramp and got stuck, 256 times out of 256.
Astra got the network's numbers, a description of the scene and a recording of one failed attempt. Without running anything, it explained why the car got stuck and rewrote 3.5% of the numbers, in one reply. After that, the car cleared the ramp 1,024 times out of 1,024.
In another test, it wrote a small network that plays Breakout (the old game where you bounce a ball into bricks) starting from nothing. Its network scored 517. One trained the usual way, for 500 million steps, scored 25 (Astra had the game's source code, to be fair).
The 48% comes from Qwen3.5-0.8B, a small open model from Alibaba. It tends to think in loops, repeating itself until it runs out of space. Qwen's own model card warns about it. In the MIT setup, 93% of its maths answers hit the length limit before reaching an answer.
Astra traced the loop to one direction in two layers and removed it. After that, 33% hit the limit. Much of the jump from 7.9% to 48.1% is the model finishing answers it could already work out.
The authors say so themselves. They tried the same thing on Google's Gemma 3 270M, which rarely loops, and got from 6.8% to at best 7.7%. Their words: editing "could remove an obstacle that hid an ability, but it could not add an ability the model lacked."
The authors published their full logs. In them, each version of the test was run only once. The winning edit took 20.3 hours. Only that edit is the 0.016% one (an earlier attempt touched almost every number in 30 matrices). Commenters on Hacker News say the same model scores in the 40s with thinking switched off (unverified comments, and the post doesn't report that baseline). On the plus side, an engineer at Hugging Face rebuilt the edited model from the logs and got 51.0%.
A known trick
The sharpest critics, Matthew Honnibal and Sean Goedecke, say there's nothing magic here. Removing one direction from a model's weights is a known technique (a 2024 paper used it to remove the safety refusals of open models). For the Qwen run, Astra wrote Python and ran the model 50 times, so "just by looking at the raw weights" doesn't describe it. Goedecke's verdict is that this is "exactly what a competent human researcher could have and would have done."
Fair enough. Still, Astra did it on request, on a model it had never seen, in under a day. That puts this kind of debugging within reach of teams that don't have such a researcher.
The idea isn't new to Switzerland either. In 2022 a team at IDSIA in Lugano published a paper that opens with "The weight matrix of a neural network is its program." One of its authors, Imanol Schlag (ETH), now co-leads Apertus, the Swiss open model, with Antoine Bosselut (EPFL), who co-wrote an early method for editing a model after training. What's new at MIT is that a general model, used as it comes, did the work.
Harder to read than to fix
The authors hope that one day a model can be checked just by reading its weights, for example to find a hidden backdoor without having to test it. They tried, and it doesn't work yet. They took two copies of a small Gemma model and trained one of them to always agree with the user. From the weights alone, Astra couldn't say which one it was. Its answers were no better than a coin flip. It only found the flatterer once it was allowed to watch both models run.
The same access that makes a clean, targeted change can also make a quiet one (the paper didn't test that, and nothing in the method prevents it).
Tampering with model files is already a known attack. In 2024 Trail of Bits showed how malicious code hidden in a model file can alter the model while it loads. In May a fake OpenAI repository on Hugging Face carried malware and showed around 244,000 downloads (the number may be inflated by bots). Those were malware, and a scanner can catch malware. A small edit to the weights is a different thing. The model would answer normally every day, except on one topic or for one kind of request, and nothing in the file would look wrong.
Where this goes
Most teams receive a model as a finished product. If it has a fault, they live with it, work around it with prompts, or pay to retrain it. Only the company that trained it can really change it.
Weight-space programming would let a model get a patch for one fault, the way software does, without a new training run. Vendors could ship fixes between versions. And anyone who has the weights of an open model, Apertus included, could have a frontier model repair it for them, without going back to the people who built it.
The authors' Gemma test also shows where it stops. A small, cheap model stays a small, cheap model. You only get back what a fault was hiding.
When anyone can change a model, a model you downloaded is only as trustworthy as the last person who touched it. For banks, FINMA already asks for checks "before productive use, in the event of changes", and a 0.016% edit is a change, whoever or whatever made it.
On recursive self-improvement, Anthropic's own read is that "we are not there yet". A frontier model debugging a much smaller one is a long way from a model improving itself. The authors are now building a benchmark for this. We'll watch whether it works on models of the size companies actually run, from 8 billion parameters up.