Agent Skills
› benchflow-ai/skillsbench
› attention-variants-from-papers
attention-variants-from-papers
GitHub用于将论文中定义的注意力机制变体实现为兼容 Transformer 的模块。涵盖提取不变量、构建形状账本、选择机制模式、保持接口一致性、数值验证及集成，确保新模块可无缝替换原有注意力块。
Trigger Scenarios
需要实现论文提出的新型注意力机制
修改注意力分数计算或分支结构但需保持接口兼容
Install
npx skills add benchflow-ai/skillsbench --skill attention-variants-from-papers -g -y
SKILL.md
Frontmatter
{
"name": "attention-variants-from-papers",
"description": "Implement paper-defined attention variants by extracting mechanism invariants, mapping tensor shapes, preserving module interfaces, validating numerical behavior, and integrating the custom attention into an existing transformer stack."
}
Attention Variants from Papers
Use this skill when a paper changes how attention scores, branches, normalization, or head sharing work, but the module still needs to behave like a drop-in transformer attention block.
Workflow
- Read the paper for invariants, not names. Use paper-to-implementation.md to extract the external contract, the changed computation, and the training-time constraints.
- Build a shape ledger before coding. Use shape-ledger.md to track projections, head grouping, branch count, and output width.
- Choose the mechanism pattern. Use mechanism-patterns.md for subtractive attention, branch mixing, learned gates, and extra normalization.
- Preserve the module boundary. Keep the same input and output shape, mask semantics, positional encoding flow, and cache behavior unless the task explicitly changes them.
- Validate in layers. Start with random-tensor smoke tests, then compare against a baseline attention path. Use stability-and-validation.md.
- Integrate into the stack last. Swap the new module into one transformer block, verify the residual path, then roll it through the full model. Use transformer-integration.md.
Checklist
- extract the paper's invariants before writing code
- account for every reshape, branch, and repeat in a shape ledger
- preserve output width at concatenation or output projection
- apply masks and positional terms at the intended stage
- confirm random smoke tests stay finite
- compare unchanged behaviors against a baseline attention implementation
Reference Map
- paper-to-implementation.md: turn paper text into module invariants and coding decisions
- shape-ledger.md: keep dimensions consistent while branch structure changes
- mechanism-patterns.md: reusable patterns for nonstandard score composition and mixing
- stability-and-validation.md: numerical checks, smoke tests, and baseline comparisons
- transformer-integration.md: wire the custom module into an existing transformer block
Version History
- 9a1f4dd Current 2026-07-24 16:36
