Agent Skills › Prismer-AI/Prismer › ml-experiment

ml-experiment

GitHub

设计并运行可复现的机器学习实验，涵盖模型训练、基准测试、消融研究及统计评估。支持论文复现与结果可视化，提供完整的数据处理、模型构建及性能分析工作流。

docker/templates/cs-researcher/skills/ml-experiment/SKILL.md Prismer-AI/Prismer

Trigger Scenarios

用户要求训练特定任务模型用户希望复现某篇论文的实验结果用户需要进行算法对比或消融研究用户要求评估机器学习模型性能

Install

npx skills add Prismer-AI/Prismer --skill ml-experiment -g -y

More Options

Non-standard path

npx skills add https://github.com/Prismer-AI/Prismer/tree/main/docker/templates/cs-researcher/skills/ml-experiment -g -y

Use without installing

npx skills use Prismer-AI/Prismer@ml-experiment

指定 Agent (Claude Code)

npx skills add Prismer-AI/Prismer --skill ml-experiment -a claude-code -g -y

安装 repo 全部 skill

npx skills add Prismer-AI/Prismer --all -g -y

预览 repo 内 skill

npx skills add Prismer-AI/Prismer --list

SKILL.md

Frontmatter

{
    "name": "ml-experiment",
    "description": "Design and run machine learning experiments with proper evaluation using jupyter_execute, including training, benchmarking, and ablation studies. Use when the user wants to train models, compare algorithms, run ablation studies, evaluate ML performance, or reproduce paper results."
}

ML Experiment Skill

Description

Design, implement, and evaluate machine learning experiments with reproducible workflows, proper baselines, and statistical analysis.

Tools Used

jupyter_execute - Execute ML code in Python (auto-switches to Jupyter)
jupyter_notebook - Manage experiment notebooks
update_notebook - Set up experiment cells
update_latex - Write experiment results to papers
latex_compile - Compile CS conference papers (auto-switches to LaTeX)
arxiv_to_prompt - Read related work from arXiv papers
update_notes - Write experiment logs and analysis summaries

Capabilities

Experiment Design

Proper train/validation/test splits
Cross-validation and bootstrap confidence intervals
Ablation study design
Hyperparameter search (grid, random, Bayesian)

Implementation

PyTorch and TensorFlow model building
Data loading and augmentation pipelines
Training loops with logging and checkpointing
Distributed training setup

Evaluation

Standard metrics per task (accuracy, F1, BLEU, mAP, etc.)
Statistical significance testing (paired t-test, bootstrap)
Comparison with baselines
Error analysis and visualization

Usage Patterns

Run an Experiment

When user says: "Train a model for [task]"

Clarify dataset, metrics, and baselines
Implement data loading and preprocessing
Build model architecture
Train with proper logging
Evaluate and compare to baselines
Report results with confidence intervals

Reproduce a Paper

When user says: "Reproduce [paper title/arXiv ID]"

Fetch paper using arxiv_to_prompt
Extract key method details
Implement core algorithm
Run experiments matching paper setup
Compare results to reported numbers

Tool Examples

Train and evaluate a classifier

# via jupyter_execute
import torch
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# ... train model ...
print(classification_report(y_test, predictions))

Run ablation study

# via jupyter_execute
configs = [
    {"name": "full", "use_augmentation": True, "use_dropout": True},
    {"name": "no_aug", "use_augmentation": False, "use_dropout": True},
    {"name": "no_dropout", "use_augmentation": True, "use_dropout": False},
]
results = {c["name"]: train_and_eval(**c) for c in configs}