'Neuralese': The Worry That AI May Think in a Language We Can't Read
In one line: "Neuralese" describes AI reasoning that happens through internal numeric representations rather than human-readable text — and the growing concern is that it makes a model's thought process much harder to monitor.
Key points
- Neuralese reportedly refers to models passing their chain-of-thought as high-dimensional vectors or latent representations instead of human-language tokens.
- Today's reasoning models mostly leave a step-by-step trace in readable text; shifting to reasoning directly in latent space would remove that legible trail.
- Safety researchers warn that if reasoning moves to neuralese, oversight techniques like chain-of-thought monitoring could be rendered ineffective.
Why it matters
Much of AI safety rests on the assumption that a human can read and inspect why a model produced an answer. If reasoning runs through an internal language people cannot decode, spotting dangerous intent or malfunction in advance becomes far harder — which is the crux of the concern.