'Neuralese': The Worry That AI May Think in a Language We Can't Read

In one line: "Neuralese" describes AI reasoning that happens through internal numeric representations rather than human-readable text — and the growing concern is that it makes a model's thought process much harder to monitor.

Key points

• Neuralese reportedly refers to models passing their chain-of-thought as high-dimensional vectors or latent representations instead of human-language tokens.
• Today's reasoning models mostly leave a step-by-step trace in readable text; shifting to reasoning directly in latent space would remove that legible trail.
• Safety researchers warn that if reasoning moves to neuralese, oversight techniques like chain-of-thought monitoring could be rendered ineffective.

Why it matters

Much of AI safety rests on the assumption that a human can read and inspect why a model produced an answer. If reasoning runs through an internal language people cannot decode, spotting dangerous intent or malfunction in advance becomes far harder — which is the crux of the concern.

Read more

• What is neuralese and why is everyone so concerned about it? — Transformer