iccv-experiments
GitHub针对ICCV截稿日期的实验设计指南。涵盖基准漂移审计、基础模型时代的公平性对比、隔离机制的消融实验及失败案例证据，确保在截止日前完成关键结果验证。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill iccv-experiments -g -y
SKILL.md
Frontmatter
{
"name": "iccv-experiments",
"description": "Use when designing the experimental program for an ICCV paper against the early-March deadline, covering benchmark-drift audits across the two-year gap, baseline fairness in the foundation-model era, ablations that isolate mechanisms, qualitative and failure evidence, and sequencing runs so the decisive result lands before the deadline, not after."
}
ICCV Experiments
An ICCV experimental program is built under a fixed, unrepeatable date: the early-March deadline of an odd year (March 7 in 2025, with supplement due the same day). The program design problem is therefore sequencing under a deadline, on top of the usual question of what evidence convinces a vision reviewer. Both are handled here.
Start with the benchmark-drift audit
Because the venue is biennial, a project conceived after one ICCV and submitted to the next spans two years of field motion. Before designing experiments, audit what moved:
drift audit (fill once, in the autumn before the deadline)
benchmarks: which datasets did the last two years of CVPR/ECCV/ICCV papers
in this area actually evaluate on? any new canonical benchmark?
baselines: leaderboard top-3 today vs when the project started
→ any baseline in your draft older than ~18 months is a red flag
backbones: what does current SOTA initialize from? (matching this defines
"fair" for your comparisons)
metrics: any metric revision or new evaluation server since last cycle?
protocols: resolution/prompt/eval-harness conventions that changed
Papers rejected for "outdated comparisons" are usually not lazy — they froze their experiment matrix at project start and never re-based. Re-run the audit in January; two months before an ICCV deadline is exactly when the previous November's CVPR-cycle preprints flood arXiv.
The fairness ledger
Vision reviewers' most reliable objection is compute-and-pretraining confounds dressed as method wins. Make fairness auditable with a ledger column per comparison:
| Axis | Your method | Each baseline | Mismatch handling |
|---|---|---|---|
| Backbone + init checkpoint | Match, or add a matched row | ||
| Pretraining data exposure | Disclose; beware test-adjacent leakage in web-scale corpora | ||
| Input resolution / tokens | Match or tabulate both | ||
| Training schedule + budget | Report epochs and GPU-hours side by side | ||
| Number quoted vs re-run | Mark re-runs; footnote protocol deltas |
The foundation-model twist: when everything builds on the same giant checkpoint, data exposure replaces architecture as the confound reviewers hunt. If your improvement could plausibly come from the pretrain having seen the test domain, run the decontamination or cross-domain check before a reviewer asks for it in a window when you have one rebuttal page to respond.
Ablations that isolate, not decorate
Structure the ablation grid so every row flips one switch, and include the two rows that distinguish a mechanism from a lucky configuration: the transplant (your module inserted into a baseline — does the gain travel?) and the sensitivity sweep (is the headline number a plateau or a spike?). Rows argued from in the text belong in the body; the full grid goes to the same-day supplement. If the core ablation shows the mechanism is not doing the work, that is an October discovery you want in November — which is why it runs first (see sequencing below).
Qualitative evidence with stated rules
At a venue that reviews with its eyes, image and video evidence carries real
weight and attracts real skepticism. Three requirements: a declared selection
rule on every grid ("first N val images", "random seed 0" — curation without a
rule is what reviewers assume by default); side-by-sides against the two
strongest baselines on identical inputs; and a failure-mode section with a
taxonomy, previewed in the body and cataloged in the supplement. For temporal or
3D claims, the supplement video is the primary exhibit — packaging in
iccv-supplementary.
Sequencing runs toward March
The scarce resource is calendar, not GPUs. Order the program by decision value per week:
- Falsifiers first (autumn): the core ablation and the strongest baseline
comparison. If the idea dies, it dies while retargeting to CVPR-November is
still possible (
iccv-workflow's autumn fork). - Headline table (winter): full benchmark suite, matched settings, seeds on the cheap rows.
- Breadth pass (January): second domain, transfer, robustness suite — whatever supports the claim's outer scope.
- Freeze margin (mid-February): the main table freezes ~2–3 weeks out; the deadline is same-day for the supplement, so late results have no legal landing zone except the rebuttal — and reviewers may not request major new experiments anyway.
- Rebuttal reserve: hold 10–20% of compute for May; the most common
winnable rebuttal item is a small matched re-run reported as a mini-table
(
iccv-author-response).
# deadline_math.py — sanity-check the plan against the calendar
runs = {"core_ablation": 6, "main_table": 21, "breadth": 10} # GPU-days each
gpu = 8; days_left = (deadline - today).days - 18 # freeze margin
assert sum(runs.values()) / gpu <= days_left, "cut scope now, not in February"
The four questions any ICCV review silently asks
Does it work (main table, matched)? Why does it work (isolating ablations +
transplant)? Where does it break (failure taxonomy, honest transfer results)?
What does it cost (params, latency on named hardware, training GPU-hours —
volunteered, since no form mandates it; see iccv-reproducibility)? Draft the
experiments section as answers to these four, in this order, and the reviewer's
checklist fills itself.
Reverify each cycle
- The 2027 deadline chain — sequencing above is calendar-shaped and the calendar is 待核实 until posted.
- Whether supplements stay same-day (changes step 4).
- Evaluation-server rules and submission budgets on your benchmarks.
- Any new ethics/human-data documentation the 2027 forms may require.
Output format
[Drift audit] run on <date>; stale baselines found: <list>
[Fairness ledger] axes matched or disclosed per comparison: n/m
[Ablation] one-switch rows: <n>; transplant + sensitivity present: yes/no
[Qualitative] selection rules stated · failure taxonomy drafted
[Sequencing] falsifiers scheduled before <date>; rebuttal reserve: <GPU-days>
[Cut candidates] <lowest decision-value runs if the calendar slips>
Version History
- 9f86f09 Current 2026-07-19 15:54
