Agent Skills
› benchflow-ai/skillsbench
› attention-variants-from-papers
attention-variants-from-papers
GitHub用于将论文中定义的注意力机制变体实现为兼容现有Transformer架构的模块。通过提取机制不变量、管理张量形状、保留接口并验证数值稳定性，确保新模块可无缝集成到现有堆栈中。
Trigger Scenarios
需要复现论文中的非标准注意力机制
修改注意力分数计算或分支结构
在保持接口兼容前提下替换默认注意力模块
Install
npx skills add benchflow-ai/skillsbench --skill attention-variants-from-papers -g -y
SKILL.md
Frontmatter
{
"name": "attention-variants-from-papers",
"description": "Implement paper-defined attention variants by extracting mechanism invariants, mapping tensor shapes, preserving module interfaces, validating numerical behavior, and integrating the custom attention into an existing transformer stack."
}
Attention Variants from Papers
Use this skill when a paper changes how attention scores, branches, normalization, or head sharing work, but the module still needs to behave like a drop-in transformer attention block.
Workflow
- Read the paper for invariants, not names. Use paper-to-implementation.md to extract the external contract, the changed computation, and the training-time constraints.
- Build a shape ledger before coding. Use shape-ledger.md to track projections, head grouping, branch count, and output width.
- Choose the mechanism pattern. Use mechanism-patterns.md for subtractive attention, branch mixing, learned gates, and extra normalization.
- Preserve the module boundary. Keep the same input and output shape, mask semantics, positional encoding flow, and cache behavior unless the task explicitly changes them.
- Validate in layers. Start with random-tensor smoke tests, then compare against a baseline attention path. Use stability-and-validation.md.
- Integrate into the stack last. Swap the new module into one transformer block, verify the residual path, then roll it through the full model. Use transformer-integration.md.
Checklist
- extract the paper's invariants before writing code
- account for every reshape, branch, and repeat in a shape ledger
- preserve output width at concatenation or output projection
- apply masks and positional terms at the intended stage
- confirm random smoke tests stay finite
- compare unchanged behaviors against a baseline attention implementation
Reference Map
- paper-to-implementation.md: turn paper text into module invariants and coding decisions
- shape-ledger.md: keep dimensions consistent while branch structure changes
- mechanism-patterns.md: reusable patterns for nonstandard score composition and mixing
- stability-and-validation.md: numerical checks, smoke tests, and baseline comparisons
- transformer-integration.md: wire the custom module into an existing transformer block
Version History
- 9a1f4dd Current 2026-07-24 16:36
