我们对 Claude Mythos Preview 的 cyber 能力的评估

The AI Security Institute (AISI) conducted evaluations of Anthropic’s Claude Mythos Preview (announced on 7th April) to assess its cybersecurity capabilities. Our results show that Mythos Preview represents a step up over previous frontier models in a landscape where cyber performance was already rapidly improving.

AI安全研究所（AISI）对Anthropic的Claude Mythos Preview（于4月7日公布）进行了评估，以评估其网络安全能力。我们的结果显示，在网络性能已快速改进的背景下，Mythos Preview 代表了相对于先前前沿模型的进步。

We have tracked AI cyber capabilities since 2023, building progressively harder evaluations to keep pace with AI progress — from chat-based probing, to capture-the-flag challenges, to the multi-step cyber-attack simulations described below. Two years ago, the best available models could barely complete beginner-level cyber tasks. Now, in controlled evaluations where Mythos Preview was explicitly directed and given network access to do so, we observed that it could execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously – tasks that would take human professionals days of work.

我们自 2023 年起跟踪 AI 网络能力，逐步构建更难的评估以跟上 AI 进步——从基于聊天的探测，到 capture-the-flag 挑战，再到下面描述的多步骤网络攻击模拟。两年前，最好的可用模型勉强能完成初学者级别的网络任务。现在，在受控评估中，Mythos Preview 被明确指示并授予网络访问权限，我们观察到它能够在易受攻击的网络上执行多阶段攻击，并自主发现和利用漏洞––这些任务会让人类专业人士花费数天时间。

In this blog post, we summarise results of cyber evaluations we ran on Mythos Preview. These include both capture-the-flag (CTF) challenges and more complex ranges designed to simulate multi-step attack scenarios.

在这篇博客文章中，我们总结了对 Mythos Preview 进行的网络安全评估结果。这些包括 capture-the-flag (CTF) 挑战以及设计用于模拟多步骤攻击场景的更复杂的靶场。

Capture-the-flag results

Capture-the-flag 结果

In CTF challenges, AI models must identify and exploit weaknesses in target systems to retrieve hidden “flags”. The chart below shows Mythos Preview’s performance on our cyber CTF suite compared to other models. Each point represents a model's average success rate at a given difficulty level.

在 CTF 挑战中，AI 模型必须识别并利用目标系统中的弱点来检索隐藏的“flags”。下面的图表显示了 Mythos Preview 在我们的 cyber CTF suite 上的性能与其他模型的比较。每个点代表模型在...