ecai-reproducibility
GitHub用于构建ECAI论文的可复现性故事。支持理论/知识表示工作（提供完整证明和参考实现）及实证/ML工作（固定种子、缓存输出、锁定依赖）。强调双盲审查下的匿名化补充材料，确保在7页正文限制内满足严格评审要求。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill ecai-reproducibility -g -y
SKILL.md
Frontmatter
{
"name": "ecai-reproducibility",
"description": "Use when building the reproducibility story for an ECAI paper — a complete proof appendix for theory\/KR work, a seeded and cached package for empirical\/ML work, provenance pinning for datasets and models, and an anonymized supplement that satisfies double-blind review inside ECAI's tight 7-page body with no separate artifact track."
}
ECAI Reproducibility
ECAI reproducibility is in-band: there is no separate artifact-evaluation track, so the same reviewers who judge the paper judge whether the results and proofs are believable, from the 7-page body plus an anonymized supplement. Build the reproducibility story to survive that read — one pass, double-blind, in a short window — not a badge committee.
Because ECAI is a general-AI venue, "reproducible" means different things across its breadth. Pick the mode that matches your contribution.
Mode 1 — Theory / KR / argumentation: proofs are the artifact
- The body sketches; the supplement carries every full proof. A theorem stated without a checkable proof is a claim, not a result.
- State all assumptions explicitly (finiteness, admissibility, monotonicity, language fragment). The most common reject-driving misreading is a reviewer assuming a hidden condition.
- If the theory has a computational side (a solver, an encoding, complexity results), include a reference implementation or the exact encoding so a reviewer can re-run a small instance.
- Define objects once, precisely; ECAI's symbolic-AI reviewers check definitions against lemmas.
Mode 2 — Empirical / ML / planning: seed, cache, pin
- Fix and report seeds; report central tendency and spread across seeds, not a single lucky
run (
ecai-experiments). - Cache raw outputs (model predictions, planner traces, API responses) so results reproduce without live calls — a package that re-queries an API re-samples rather than reproduces.
- Pin provenance: dataset name and version/date, preprocessing scripts, model identifiers with dates, hardware where it affects timing.
- Provide a claim→file map: each reported table/number points to the script that regenerates it.
Provenance pinning (both modes, where applicable)
[ ] Dataset: name, version/DOI, download date, license, preprocessing script committed
[ ] Splits: exact train/val/test (or instance sets) fixed and included or scripted
[ ] Models: identifiers + dates (for hosted/LLM components); prompts/configs committed
[ ] Seeds: fixed and reported; number of runs stated
[ ] Environment: dependency versions pinned (lockfile / environment.yml / requirements)
[ ] Outputs: raw results cached so re-run does not depend on a live service
Double-blind, in the supplement too
The supplement is read under double-blind review. Anonymize it as carefully as the PDF:
# Sweep the staged supplement before zipping
grep -rniE 'university|@[a-z0-9.]+\.(edu|ac\.[a-z]+)|acknowledg|funded by|grant (no|number)' supplement/ | head
unzip -l supplement.zip | grep -Ei '\.git/|/home/|/Users/|\.DS_Store' | head
Strip repository owners, institution names, funding lines, and any system named after your group. A de-anonymizing supplement can trigger a summary reject before the science is even read.
Honesty over completeness
- If data cannot be shared (privacy, licensing, industrial confidentiality — common in PAIS applications), say so and why, and share what you can (code, a synthetic sample, the protocol). A silent gap reads worse than a stated, justified limitation.
- Do not claim reproducibility you have not tested. Run the package from a clean checkout yourself before submitting.
Fit the 7-page body
Reproducibility content that a reviewer needs to judge the paper (the core proof idea, the
evaluation protocol, the key numbers) belongs in the body; full proofs, extra tables, and code
belong in the supplement. Nothing decision-critical may live only outside the 7 pages
(ecai-supplementary).
Post-acceptance
Convert the anonymized supplement into a permanent, open release — DOI-issuing archive, open
license, de-anonymized owners — and link it from the open-access camera-ready
(ecai-camera-ready).
Output format
[Mode] theory (proof appendix) / empirical (seeded+cached) / mixed
[Proof completeness] every theorem has a full checkable proof + explicit assumptions? yes/no
[Provenance] datasets/models/seeds/env pinned? gaps: <list>
[Claim map] each table/number -> regenerating file
[Anonymity] supplement clean / leaks: <where>
[Body/supplement split] nothing decision-critical outside the 7-page body
[Post-acceptance] DOI + open license + de-anonymized link planned
Version History
- 9f86f09 Current 2026-07-19 15:15
