Agent Skills
› aiming-lab/AutoResearchClaw
› data-loading
data-loading
GitHub提供PyTorch DataLoader高效加载最佳实践，优化数据预处理与I/O性能，防止GPU空闲。适用于设置DataLoader或进行数据预处理的场景。
Trigger Scenarios
设置DataLoader参数
数据预处理优化
解决GPU饥饿问题
Install
npx skills add aiming-lab/AutoResearchClaw --skill data-loading -g -y
SKILL.md
Frontmatter
{
"name": "data-loading",
"metadata": {
"author": "researchclaw",
"version": "1.0",
"category": "tooling",
"priority": "6",
"references": "PyTorch Data Loading Tutorial, pytorch.org",
"trigger-keywords": "data,loading,dataloader,dataset,preprocessing,augmentation",
"applicable-stages": "10"
},
"description": "Optimize data loading pipeline to prevent GPU starvation. Use when setting up DataLoader or data preprocessing."
}
Efficient Data Loading Best Practice
- Use num_workers = min(8, os.cpu_count()) for DataLoader
- Enable pin_memory=True when using GPU
- Use persistent_workers=True to avoid re-spawning
- Pre-compute and cache transformations when possible
- For image data: use torchvision.transforms.v2 (faster)
- For large datasets: consider memory-mapped files or WebDataset
- Profile with torch.utils.bottleneck to find I/O bottlenecks
Version History
- e2e23c9 Current 2026-07-25 07:48
