如何在不触碰模型权重的情况下击败GRPO
[
[
](https://x.com/akshay_pachaar)
](https://x.com/akshay_pachaar)
How to Beat GRPO Without Touching Model Weights
如何在不修改模型权重的情况下击败 GRPO
A team at Berkeley beat GRPO by 10 points with 35× fewer rollouts and no GPU training, and why most teams are still doing prompt optimization the wrong way.
伯克利的一支团队以少35倍的rollout且无需GPU训练的优势，比GRPO高出10分，并解释了为什么大多数团队仍然以错误的方式进行提示优化。
You spent two weeks fine-tuning an 8B model with GRPO. Burned $40,000 in compute. Got a 7-point bump on your benchmark.
你花了两周时间使用 GRPO 微调一个 8B 模型。烧掉了 40,000 美元的计算资源。在基准测试中获得了 7 分的提升。
A team at Berkeley ran a different optimizer on the same setup. Spent under $1,000. Beat your score by 10 points.
伯克利的一支团队在相同的设置上运行了不同的优化器。花费不到1,000美元。比你的得分高出10分。
No weight updates. No GPUs. Just prompts.
没有权重更新。没有 GPU。只有提示词。
That's GEPA. The paper dropped in July 2025. Nine months later, it was accepted as an ICLR 2026. DSPy made it a first-class optimizer. Hugging Face and OpenAI shipped cookbooks. Shopify's CEO publicly said it's underhyped.
这就是GEPA。该论文于2025年7月发布。九个月后，它被ICLR 2026接收。DSPy将其作为一等优化器。Hugging Face和OpenAI发布了cookbooks。Shopify的CEO公开表示它的宣传还远远不够。
If you skipped GEPA when it came out, this is your second chance.
如果你在 GEPA 刚发布时错过了它，这是你的第二次机会。
Let's break down why it works and how to use it in DSPy.
让我们详细解析一下它为什么有效以及如何在 DSPy 中使用它。
The dirty secret of RL on LLMs
LLMs上RL的隐秘真相
Here's the thing nobody tells you about reinforcement learning on language models.
关于在语言模型上进行强化学习，这里有一件没人告诉你的事。
Every rollout your agent produces is a 5,000-token document, containing:
您的智能体生成的每一次rollout都是一个包含5,000个token的文档，其中包括：
- Reasoning steps
- 推理步骤
- Tool calls
- 工具调用
- Self-corrections
- 自我纠正
- Compiler errors
- 编译器错误
- Judge rationales
- 评判理由
Rich, structured, full of useful information.
丰富、结构化、充满有用信息。
GRPO takes all of that and reduces it to a single number.
GRPO将所有这些信息简化为一个单一的数字。
Was this trajectory better or worse than the group average? +1 or -1. That's your learning signal.
这条轨迹是优于还是劣于组平均值？+1 或 -1。这就是你的学习信号。
You just threw away thousands of bits of structured information so you could backprop on one b...