Agent Skills
› brycewang-stanford/Awesome-Journal-Skills
› hri-artifact-evaluation
hri-artifact-evaluation
GitHub用于打包已接受的ACM/IEEE HRI论文的可开放研究材料（代码、机器人行为、去标识数据等）。强调诚实说明具身人类受试者研究的复现限制，构建可重分析且方法可复用的工件，遵循ACM徽章标准或通用开放规范。
触发场景
需要为HRI论文准备开源工件
申请ACM Artifact Badge
整理去标识化数据和预注册信息
文档化机器人行为和实验协议
安装
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill hri-artifact-evaluation -g -y
SKILL.md
Frontmatter
{
"name": "hri-artifact-evaluation",
"description": "Use when packaging the artifacts of an accepted ACM\/IEEE HRI paper — code, robot behaviors, de-identified data, and study materials — for openness and any ACM badges the edition offers, while being honest about the limits of reproducing an embodied human-subjects study."
}
HRI Artifact Evaluation
HRI's artifact and reproducibility culture is less formalized than software-engineering venues':
there is not a long-standing, uniformly enforced ACM artifact-badge track at every HRI edition, and
whether a given year runs a formal reproducibility/artifact evaluation is 待核实 per cycle. So
treat artifact work at HRI as a credibility and openness move first, and follow a formal badge
process only if this edition offers one. Either way, the artifact of an HRI paper is unusual: it is
the study, not a runnable system that reproduces a number. This skill packages it honestly. It
builds on hri-reproducibility and the shared kit in
../../resources/code/README.md.
First: does this edition run a formal evaluation?
- Check the current call for a reproducibility/artifact track, its badges, and its own deadline (待核实). If it exists, follow its instructions exactly.
- If ACM badges are offered, they follow the ACM scheme (Artifacts Available / Evaluated - Functional / Reusable / Results Reproduced). Map your package to whichever the edition supports.
- If no formal track runs, still release an open, well-documented artifact — reviewers and readers reward it, and it is the community norm even without a badge.
What an HRI artifact actually contains
Because the core evidence is a human study of an embodied robot, the artifact is rarely a one-command reproduction. Assemble:
- Study materials — questionnaires (items + scoring), interview guides, task scripts, consent-form summary, and coding schemes.
- Robot-behavior specification — the conditions/behaviors the robot exhibited: for autonomous
behavior, the software version and parameters; for Wizard-of-Oz, the wizard's action space,
protocol, and error rates (see
hri-experiments). - De-identified data — participant-level data with identifiers removed, plus a codebook.
- Analysis code — scripts that regenerate every reported statistic, figure, and table from the shared data, with pinned dependencies and fixed seeds.
- Pre-registration pointer and any deviation notes.
Make it reusable, within honest limits
- Reproducible = re-analyzable + re-runnable-as-a-method. An evaluator should be able to regenerate your results from the shared data and to understand the procedure well enough to run the study again — not to recreate your exact human responses. State this scope plainly.
- Document the environment. For analysis code, a
READMEwith dependency versions and a single entry point; a container helps if the toolchain is heavy. - License it (an OSI-approved code license; a documented data-use/consent basis for the data).
- Archive with a DOI (OSF, Zenodo, figshare, or Software Heritage) rather than a personal homepage, so the link is permanent and citable.
Human-subjects constraints on the artifact
An HRI artifact contains people, which limits what can be shared:
- Share only data/media the participants consented to share for that use; if consent for public data or video was not obtained, share what you can and explain the gap honestly.
- De-identify thoroughly — free-text responses and video are the usual re-identification risks.
- Vulnerable-population studies (children, older adults, clinical) may permit little sharing; say so rather than force an unethical release.
- A partial, honest artifact with a clear rationale beats an over-shared one that breaches consent.
Camera-ready badge mechanics (if applicable)
- Badges, when a paper earns them, are displayed on the published version — coordinate with
hri-camera-readyso any badge and the archive DOI appear correctly. - Keep the anonymized review-time archive and the de-anonymized public archive separate; the public one carries author names, the review one must not.
Anti-patterns
- Overclaiming a "reproducible" human study as if it re-runs like a benchmark.
- "Data available on request" — treated as unavailable.
- Unlogged Wizard-of-Oz — the stimulus cannot be reproduced.
- Identifiable data or video released without consent for that use.
- Chasing a badge that does not exist this cycle instead of simply releasing a good artifact.
Output format
[Formal track?] does this edition run artifact/reproducibility evaluation + badges? (verify)
[Artifact contents] materials · robot-behavior spec (incl. WoZ) · de-identified data + codebook · analysis code
[Reproducibility scope] re-analysis + method replication; human responses framed as not re-runnable
[Openness] DOI archive · license · honest statement of consent-limited gaps
[Human-subjects] de-identified · shared only what consent allows
[Badge readiness] (if offered) which ACM badge(s) targeted · public vs review archive separated
[Fix queue] <ordered>
版本历史
- 9f86f09 当前 2026-07-19 15:50
