更好的模型：更差的工具

A very strange Pi issue sent me down a rabbit hole over the last two days. The short version is that newer Claude models sometimes call Pi’s edit tool with extra, invented fields in the nested edits[] array. And not Haiku or some small model: Opus 4.8. The edit itself is usually correct but the arguments do not match the schema as the model invents made-up keys and Pi thus rejects the tool call and asks to try again.

过去两天里，一个非常奇怪的 Pi issue 让我陷入了一个兔子洞。简而言之，较新的 Claude 模型有时会在调用 Pi 的编辑工具时，在嵌套的 edits[] 数组中包含额外的、虚构的字段。而且不是 Haiku 或某个小模型：是 Opus 4.8。编辑本身通常是正确的，但参数与 schema 不匹配，因为模型编造了虚构的键，因此 Pi 拒绝了该工具调用并要求重试。

That alone is not too surprising as models emit malformed tool calls sometimes. Particularly small ones. What surprised me is that this is getting worse with newer Anthropic models as both Opus 4.8 and Sonnet 5 show it but none of the older models. In other words, the SOTA models of the family are worse at this specific tool schema than their older siblings.

单凭这一点并不太令人惊讶，因为模型有时会发出格式错误的工具调用。尤其是较小的模型。让我惊讶的是，这在较新的Anthropic模型中变得越来越糟，因为Opus 4.8和Sonnet 5都出现了这个问题，但旧模型都没有。换句话说，该系列中SOTA模型在这种特定工具模式上的表现比它们的旧型号更差。

In case you are curious about Fable: I intentionally did not test it because I was not sure if the classifiers they are running might downgrade me to Opus silently.

如果你对Fable感到好奇：我故意没有测试它，因为我不确定他们运行的分类器是否会悄悄将我降级到Opus。

Tool Calls Are Text

工具调用即文本

If you have not spent too much time looking at LLM tool calling internals, the important thing to understand is that tool calls are not magic and use some rather crude in-band signalling. The model receives a transcript, a system prompt and a list of available tools. The server munches that into a large prompt with special marker tokens. Because the model was trained and reinforced on examples of that format, at some point during generation it emits something that the API or client interprets as “call this tool with these arguments”.

如果你没有花太多时间研究 LLM tool calling 的内部机制，需要理解的重要一点是，tool calls 并不是魔法，而是使用了一些相当粗糙的带内信令。模型接收一个 transcript、一个 system prompt 和一个可用工具列表。服务器将其处理成一个带有特殊标记 token...