icra-reproducibility
GitHub用于增强ICRA论文可复现性，明确软硬件规格与版本，规范rosbag日志记录，区分代码仿真（可重跑）与硬件实验（可审计），并撰写诚实的可用性声明。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill icra-reproducibility -g -y
SKILL.md
Frontmatter
{
"name": "icra-reproducibility",
"description": "Use when hardening an ICRA paper's reproducibility — specifying robot platform, firmware, ROS and driver versions, control rates, and sensor calibration; logging rosbags and seeds; separating what others can rerun (code, sim) from what they can only audit (your hardware trials); and writing honest availability statements."
}
ICRA Reproducibility
Reproducibility at a robotics conference means something different than at an ML venue: no reader can rerun your hardware trials without your robot, your room, and your objects. The achievable standard is therefore two-tier — rerunnable (code, simulation, analysis) and auditable (hardware evidence logged well enough that a skeptic can verify you did what you claim). ICRA papers earn trust by being explicit about which tier each result sits in.
The specification ledger
A robotics result is unreproducible if the platform is underspecified. The paper (body, since there is no appendix escape) plus released materials should pin:
| Layer | Must specify | Example of "enough" |
|---|---|---|
| Robot | model, DOF, payload, firmware version | "UR5e, PolyScope 5.11" |
| End effector | model, mods, wear state if relevant | "Robotiq 2F-85, stock pads" |
| Sensors | model, resolution, rate, mounting frame | "D435i @ 640×480, 30 Hz, wrist" |
| Compute | on-board vs off-board, GPU, latency path | "off-board 3080, 5G-free wired" |
| Middleware | ROS distro, key package versions | "ROS 2 Humble, MoveIt 2.5.4" |
| Control | loop rates, gains or their source | "impedance @ 500 Hz, gains in repo" |
| Calibration | method + residual error | "hand-eye via [ref], 1.8 mm RMS" |
| Objects/scenes | identity, source, dimensions | "YCB subset listed in Tab. II" |
Standard object sets (YCB and similar) exist precisely so that "graspable household objects" can be a checkable claim; use them or publish the object specs.
Logging: the rosbag is the lab notebook
- Record every hardware session — including failed ones — with topic-complete rosbags (or vendor-equivalent logs) plus an untouched camera view.
- One log per trial, named by protocol:
trial_<cond>_<idx>_<date>.bag; the analysis pipeline should consume these logs, so paper numbers regenerate from raw data with one command. - Keep a session sheet: date, operator, firmware state, anomalies. Two months later, this distinguishes "the gripper was recalibrated between Tables II and III" from unexplainable drift.
- The video attachment should be cut from these recorded sessions, which guarantees footage corresponds to logged, reported trials.
Determinism where it exists, honesty where it doesn't
Hardware trials are not seed-reproducible — contact, cables, lighting, and thermal drift see to that. Split the claims accordingly:
- Deterministic tier: training runs, simulation experiments, and analysis scripts get pinned seeds, pinned dependency versions, and containerized environments. State seeds and repetition counts.
- Stochastic tier: hardware trials get protocol reproducibility — the documented procedure another lab could follow — plus dispersion reporting.
- Never write "results are fully reproducible" over a hardware table; write "code and simulation reproduce Tables I-II; hardware protocol and logs for Tables III-IV are released for audit."
Release scaffold
repo/
├── README.md # claim → command map (Table III = run_hw_analysis.sh)
├── env/ # Dockerfile or rosdep/lockfiles, pinned
├── sim/ # simulation experiments, seeds fixed
├── control/ # controllers/planners as ROS packages
├── hw_protocol.md # step-by-step trial procedure, success criterion,
│ # reset procedure, safety notes
├── logs/ # rosbags or download script (size!), session sheets
├── analysis/ # regenerates every figure/table from logs/
└── video/ # raw clips ↔ trial IDs mapping
Anonymize the repository for review under the double-anonymous policy (2026
cycle onward): anonymized hosting or an artifact stripped of names — an org URL
is an identity leak (see icra-submission).
Compute and latency reporting
Robotics adds a real-time dimension absent from ML reproducibility norms:
- Report where each computation ran (on-board CPU/embedded GPU vs off-board workstation) and the link between them; an off-board 3090 over gigabit wire is a materially different system than on-board inference.
- State end-to-end latency per pipeline stage (sensing → perception → decision → actuation) at least once; "runs in real time" is unverifiable without it.
- Training compute (GPU-hours, hardware) belongs in the paper for learned components — cheap to report, increasingly expected by review forms.
Availability statement patterns
Honest, specific statements outperform boilerplate promises:
- "Code, simulation environments, and trial logs: (anonymized for review; public on acceptance)."
- "The gripper design files are released; the arm is commercial (UR5e)."
- "Raw rosbags total 1.2 TB; per-trial extracted features and a 40 GB sample are hosted, full logs on request."
- Avoid: "code will be released upon publication" with nothing at review time — reviewers discount unverifiable promises, and some cycles ask availability questions directly on the review form (verify the current form).
Pre-submission reproducibility pass
- Specification ledger complete in-paper (table above, compressed)?
- Every reported number regenerates from logs via one documented command?
- Seeds/versions pinned for the deterministic tier; container builds clean?
- Hardware protocol written so a stranger could run trial 1 tomorrow?
- Release scaffold anonymized; links leak-checked?
- Availability statement matches what actually exists today?
Output format
[Tier split] rerunnable: <tables/figs> | auditable: <tables/figs>
[Spec ledger] missing rows: <list or none>
[Log coverage] trials logged <n>/<n>, analysis regenerates: y/n
[Determinism] seeds/versions pinned for sim+training: y/n
[Release] scaffold complete? anonymized? size plan?
[Statement] honest-specific / boilerplate-risk — rewrite: <line>
Version History
- 9f86f09 Current 2026-07-19 15:58
