hri-reproducibility
GitHub用于构建ACM/IEEE HRI论文的开放性与可复现性叙事，指导共享研究材料、机器人行为规范、去标识化数据及分析代码。明确界定可复现与不可复现内容，涵盖Wizard-of-Oz披露、预注册及视频去标识化，确保实验流程透明且符合伦理。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill hri-reproducibility -g -y
SKILL.md
Frontmatter
{
"name": "hri-reproducibility",
"description": "Use when building the openness and replicability story for an ACM\/IEEE HRI full paper — sharing study materials, robot-behavior specifications, de-identified data, and analysis code; reporting Wizard-of-Oz and pre-registration; and stating honestly what can and cannot be reproduced in an embodied human-subjects study."
}
HRI Reproducibility
Reproducibility means something specific — and harder — at HRI than at a benchmark-driven venue. You
cannot "re-run" a study of an embodied robot and real people the way you re-run a model on a test
set: the participants, the room, the exact robot, and the moment are gone. So HRI reproducibility is
about making the procedure, stimulus, and analysis transparent enough that another team could
replicate the method and readers can audit your inferences. This skill builds that story; it
pairs with the shared kit in ../../resources/code/README.md.
What "reproducible" can and cannot mean here
Be honest about the limits, and design the artifact around what is reproducible:
- Reproducible (aim for this): the study protocol, the robot behavior/stimulus, the measures, the analysis code, and the de-identified data — enough to re-analyze your results and to run the same study elsewhere.
- Not reproducible (say so): the exact human responses. A replication is a new sample; framing it otherwise overclaims. State this plainly rather than let a reviewer catch the overclaim.
The replication package for an HRI study
Assemble, and reference from the paper:
- Study materials — consent-form summary, instructions/scripts, questionnaires with exact items and scoring, interview guides, and coding schemes for qualitative work.
- Robot-behavior specification — the behaviors/conditions the robot exhibited, at enough
granularity to reproduce the stimulus: for autonomous behavior, the software version and
parameters; for Wizard-of-Oz, the wizard's action space, protocol/script, and error rates
(see
hri-experiments). - De-identified data — participant-level data with identifiers removed, plus a codebook. Scrub free-text and metadata that could re-identify participants.
- Analysis code — scripts that reproduce every reported statistic, figure, and table from the shared data, with fixed seeds and pinned dependencies.
- Pre-registration — a link/ID if you pre-registered the hypotheses and analysis, with any deviations documented.
Report the things that make a study re-runnable
- Sample and procedure in reproducible detail: recruitment, inclusion criteria, N and how it was determined, session flow, timing, compensation.
- Apparatus: the robot platform and version, sensors, environment, and anything that would change the interaction if swapped.
- Wizard-of-Oz disclosure and constraints, so the stimulus is not an unlogged human performance.
- Analysis decisions: exclusions and why, transformations, the exact tests, and the confirmatory/exploratory split.
De-identification and the video
HRI artifacts contain people, which raises obligations a code-only artifact does not:
- De-identify data before sharing; get consent for data sharing at study time — retrofitting it is usually impossible.
- The video figure and any recordings need consent for the use you are making (review-time
sharing, public archival, or conference presentation are different permissions). For review, blur
faces and mute identifying audio if consent for public use was not obtained (see
hri-supplementary). - Vulnerable populations (children, older adults, clinical) demand extra care in what is shared at all.
Openness at HRI (calibrated to the venue)
HRI's open-science and artifact-badge culture is less formalized than SIGSOFT's — there is not a long-standing, uniformly enforced ACM badge track for every edition (待核实 per cycle). So treat openness as a credibility and community-norm move, not a checkbox:
- Share what ethics allows via a DOI-issuing archive (e.g., OSF, Zenodo, figshare); avoid a personal homepage link that both rots and de-anonymizes.
- Where confidentiality or human-subjects constraints prevent full sharing, say what and why — an honest limitation reads far better than silence or "available on request."
- If the edition does run an artifact/reproducibility evaluation, follow it and see
hri-artifact-evaluation.
Anonymization for double-blind
Everything above must be anonymized at review time: strip author/institution names from
materials and code, host the archive behind an anonymizing view, and scrub the video and data of
identity leaks. See hri-submission for the mechanical sweep.
Anti-patterns HRI reviewers flag
- Overclaiming reproducibility — implying a human study can be exactly re-run.
- Unlogged Wizard-of-Oz — no way to reproduce the stimulus.
- "Data available on request" — treated as not available.
- Identifiable data or video shared without consent for that use.
- Analysis not reproducible from the shared data — figures that no script regenerates.
Output format
[Repro scope] procedure/stimulus/analysis reproducible? human responses correctly framed as not?
[Package] materials · robot-behavior spec (incl. WoZ) · de-identified data + codebook · analysis code
[Pre-registration] link/ID · deviations documented?
[De-identification] data scrubbed · video/audio consent for the use made?
[Openness] DOI archive · honest statement of what cannot be shared and why
[Anonymization] materials/code/archive/video anonymized for review?
[Fix queue] <ordered>
Version History
- 9f86f09 Current 2026-07-19 15:50
