AlphaCodium的最先进的代码生成-从提示工程到流程工程

State-of-the-art Code Generation with AlphaCodium

Code generation problems differ from common natural language problems – they require matching the exact syntax of the target language, identifying happy paths and edge cases, paying attention to numerous small details in the problem spec, and addressing other code-specific issues and requirements. Hence, many of the optimizations and tricks that have been successful in natural language generation may not be effective for code tasks.

代码生成问题与常见的自然语言问题不同 - 它们需要匹配目标语言的确切语法，识别正常路径和边缘情况，注意问题规范中的许多细节，并解决其他与代码相关的问题和要求。因此，许多在自然语言生成中成功的优化和技巧可能对代码任务无效。

In this work, we propose a new approach to code generation by LLMs, which we call AlphaCodium – a test-based, multi-stage, code-oriented iterative flow, that improves the performances of LLMs on code problems.

在这项工作中，我们提出了一种新的LLMs代码生成方法，我们称之为AlphaCodium - 一种基于测试的、多阶段的、以代码为导向的迭代流程，提高了LLMs在代码问题上的性能。

We tested AlphaCodium on a challenging code generation dataset called CodeContests, which includes competitive programming problems from platforms such as Codeforces. The proposed flow consistently and significantly improves results.

我们在一个名为CodeContests的具有挑战性的代码生成数据集上测试了AlphaCodium，该数据集包括来自Codeforces等平台的竞赛编程问题。所提出的流程始终显著改善结果。

On the validation set, for example, GPT-4 accuracy (pass@5) increased from 19% with a single well-designed direct prompt to 44% with the AlphaCodium flow. AlphaCodium also outperforms previous works, such as AlphaCode, while having a significantly smaller computational budget.

例如，在验证集上，通过单个精心设计的直接提示，GPT-4的准确率（pass@5）从19%提高到了使用AlphaCodium流程的44%。AlphaCodium还优于以前的作品，如AlphaCode，同时具有显著较小的计算预算。

Many of the principles and best practices acquired in this work, we believe, are broadly applicable to general code generation tasks.
In our very new open-source on AlphaCodium****690 we share our AlphaCodium solution to CodeContests, along with a complete reproducible dataset evaluation and benchmarking scripts, to encourage further research in this area.

我们相信，在这项工作中获得的许多原则和最佳实践在一般的代码生成任务中都是适用的。
在我们非常新的开源项目 AlphaCodium****690 上，我们分享了我们对 CodeContests 的 AlphaCodium 解决方案，以及完整可复现的...