iros-reproducibility
GitHub指导IROS论文强化可复现性，区分可运行与可审计声明，提供硬件账本、校准参数披露及诚实代码数据可用性说明，确保在正文篇幅内完成所有证据呈现。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill iros-reproducibility -g -y
SKILL.md
Frontmatter
{
"name": "iros-reproducibility",
"description": "Use when strengthening IROS reproducibility evidence — splitting rerunnable claims from auditable-only real-robot claims, the hardware specification ledger, calibration and parameter disclosure, log capture discipline, honest code\/data availability statements, and sim-to-real accounting, all inside a page budget with no supplementary PDF."
}
IROS Reproducibility
Use this before submission and again before camera-ready. Robotics reproducibility is not the same as ML reproducibility: a reader usually cannot rerun your robot, so the goal is to make every claim either rerunnable (code/sim someone can execute) or auditable (logs and specs detailed enough to check), and to be honest about which is which.
Evidence map
- Split each claim into rerunnable or auditable. A simulation result should be rerunnable; a real-robot reliability number is auditable through logs and a stated protocol, not rerunnable by a reader.
- For the rerunnable part: pin dependencies, seeds, configs, and an entry point; state the hardware it assumes.
- For the auditable part: give the hardware ledger, calibration, parameters, trial protocol, resets, and representative logs.
- Report the sim-to-real gap explicitly; never let simulation numbers stand in for real ones.
- Write an honest availability statement: what code and data are released, what is withheld and why (proprietary hardware, safety, licensing), and how a reader could approximate reproduction.
- Keep it all in the body — there is no supplementary PDF to park a reproducibility appendix in.
Rerunnable-versus-auditable table
| Claim | Rerunnable evidence | Auditable evidence |
|---|---|---|
| Simulation ablation | Seeded config + script a reader can run | Logged metrics per run |
| Onboard timing | Benchmark script for the compute stack | Power/rate measurement method + logs |
| Real-robot success rate | Not rerunnable by a reader | Trial log, criterion, resets, failure counts |
| Sim-to-real transfer | Sim side rerunnable | Paired real logs + the stated gap |
Marking a real-robot claim as if a reader could "just rerun it" is a recognizable IROS overstatement; name it auditable and make the audit trail complete instead.
The hardware specification ledger
A result that depends on a specific robot must specify it: platform and revision, sensor models and firmware, actuator limits, onboard computer, and power budget. Two labs with "the same" robot but different firmware or calibration can see different behavior, so the ledger is part of the claim, not metadata.
Vignette: a manipulation-reliability paper
Consider a paper reporting grasp success on unseen objects. Its reproducibility spine: the object set and where to get it, the calibration and gripper parameters, the success criterion and reset procedure, the per-object trial logs, and one honest sentence about the objects where the method fails and why — plus a rerunnable simulation of the grasp planner so the algorithmic core is executable even if the robot is not.
Availability statement skeleton:
Code: <released? which parts? link at acceptance>
Data/objects: <provenance + license, or access route>
Logs: <representative bags for the reported trials>
Withheld: <what + why: proprietary/safety/licensing>
Reproduce-in-principle: <what a reader without the robot can still run>
Output format
[Claim inventory] <claim -> rerunnable | auditable + location>
[Hardware ledger] complete / gaps: <list>
[Sim-to-real] gap stated? / implied-zero risk?
[Availability] honest / overpromised / missing
[Body fixes] <what must appear in the paper body>
Version History
- 9f86f09 Current 2026-07-19 16:05
