追踪大型语言模型的思维

Language models like Claude aren't programmed directly by humans—instead, they‘re trained on large amounts of data. During that training process, they learn their own strategies to solve problems. These strategies are encoded in the billions of computations a model performs for every word it writes. They arrive inscrutable to us, the model’s developers. This means that we don’t understand how models do most of the things they do.

像Claude这样的语言模型并不是直接由人类编程的——相反，它们是在大量数据上进行训练的。在训练过程中，它们学习自己解决问题的策略。这些策略被编码在模型为每个写出的单词执行的数十亿次计算中。对于我们这些模型的开发者来说，这些策略是不可理解的。这意味着我们并不理解模型是如何完成大多数任务的。

Knowing how models like Claude think would allow us to have a better understanding of their abilities, as well as help us ensure that they’re doing what we intend them to. For example:

了解像Claude这样的模型是如何思考的，将使我们更好地理解它们的能力，并帮助我们确保它们在执行我们所期望的任务。例如：

• Claude can speak dozens of languages. What language, if any, is it using "in its head"?
• Claude可以说数十种语言。它“在脑海中”使用的是什么语言（如果有的话）？
• Claude writes text one word at a time. Is it only focusing on predicting the next word or does it ever plan ahead?
• Claude 每次写一个单词。它是否只专注于预测下一个单词，还是有时会提前计划？
• Claude can write out its reasoning step-by-step. Does this explanation represent the actual steps it took to get to an answer, or is it sometimes fabricating a plausible argument for a foregone conclusion?
• Claude可以逐步写出其推理过程。这个解释是否代表了它得出答案所采取的实际步骤，还是有时在为一个既定结论编造一个合理的论点？

We take inspiration from the field of neuroscience, which has long studied the messy insides of thinking organisms, and try to build a kind of AI microscope that will let us identify patterns of activity and flows of information. There are limits to what you can learn just by talking to an AI model—after all, humans (even neuroscientists) don't know all the details of how our own brains work. So we look inside.

我们从神经科学领域获得灵感，该领域长期以来研究思维生物体的复杂内部，并试图构建一种AI显微镜，让我们识别活动模式和信息流。仅通过与AI模型对话，你能学到的东西是有限的——毕竟，人类（甚至神经科学家）并不知道我们自己大脑运作的所有细节。因此，我们深入内部。

Today, we're sharing two new papers that represent progress on the development of the "microscope", and the application of it t...