icdm-experiments
GitHub用于设计或审核ICDM论文的实验评估，涵盖任务定义、基线公平调优、机制隔离消融、可扩展性验证及发现有效性检查，确保结果可信且符合会议规范。
Trigger Scenarios
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill icdm-experiments -g -y
SKILL.md
Frontmatter
{
"name": "icdm-experiments",
"description": "Use when designing or auditing the empirical evaluation for an ICDM (IEEE International Conference on Data Mining) paper - mining-task definition, strong and fairly-tuned baselines, ablations that isolate the named mechanism, scalability curves that test scale claims, and discovery-validity checks that separate real findings from evaluation artifacts."
}
ICDM Experiments
Design the evaluation an ICDM reviewer will trust: a defined mining task, baselines tuned as carefully as your method, ablations that isolate the mechanism, a measured scale story, and a discovery-validity argument. ICDM's data-centric reviewers punish leaderboard-only wins and un-checkable discovery claims, and the whole evaluation must fit inside the 10-page all-inclusive cap.
Define the mining task before the metric
- State the task operationally: inputs, outputs, and what a correct answer is. "Anomaly detection" is a genre; "rank edges by anomalousness in a one-pass stream, evaluated against injected ground truth" is a task.
- Fix the evaluation protocol — splits, negatives, thresholds, ranking cutoffs — before running anything, and describe it precisely enough to reproduce inside the page cap.
The four evidence axes
| Axis | Question it answers | Typical evidence |
|---|---|---|
| Quality | Is the mining result good on the task? | Ranking/accuracy vs baselines with variance |
| Scale | Does the scale claim hold? | Latency/memory curves across data sizes |
| Mechanism | Is the named mechanism the reason? | Ablations toggling exactly that component |
| Validity | Is the finding real, not an artifact? | Controlled injections, known-truth checks |
A strong ICDM paper touches all four; missing "mechanism" or "validity" is the usual reason a methodologically fine paper reads as thin.
Baselines and tuning symmetry
- Compare against current strong baselines, and tune them with the same budget you gave your method; an under-tuned baseline is the fastest way to lose reviewer trust.
- Include the obvious simple baseline. If a cheap method nearly matches you, say so and argue the regime where your mechanism pays off.
- Report every number with variance over seeded runs; a single-run table invites the "is this noise?" review.
Ablations that isolate the mechanism
The mechanism-attached novelty of icdm-writing-style must be demonstrated, not asserted.
Mechanism: single-pass isolation sketch with m random partitions.
Ablation grid:
- remove the sketch, keep full storage -> isolates the streaming contribution
- vary m (partition count) -> maps the accuracy/memory knob
- swap the hash family -> tests sensitivity to the mechanism's core
- replace isolation score with density score -> isolates the isolation principle
Each row answers "was THIS the reason it worked?"
Test the scale claim, do not assert it
- If you claim scalability, plot behavior across at least an order of magnitude of data size, and report the cost model (linear, sub-linear memory, amortized constant update).
- Separate wall-clock from asymptotic claims; hardware-dependent speedups need the hardware stated and, ideally, an operation count that is not hardware-dependent.
Discovery validity: the ICDM instinct
- Where truth is unknown, build a setting where it is: injected anomalies, planted patterns, synthetic graphs with known structure — so you can show the method recovers known signal.
- Guard against leakage: temporal tasks need time-respecting splits; graph tasks need to avoid train/test edge overlap. State the guard explicitly.
- Do not overclaim when differences are within variance; an honest "matches at lower cost" is stronger here than a fragile "outperforms."
Vignette: an ablation that saved the claim
A team reports strong stream-anomaly numbers but reviewers cannot tell whether the sketch or the underlying isolation criterion did the work. Adding two ablation rows — full-storage isolation (isolating the streaming contribution) and a density-score swap (isolating the isolation principle) — showed the sketch preserved batch quality while the isolation principle drove detection. The claim survived because the mechanism was shown, not stated, and both rows fit in the appendix inside the 10-page cap.
Output format
[Task] <operational task definition>
[Axes covered] quality / scale / mechanism / validity - list gaps
[Baselines] strong + tuned symmetrically: yes / no
[Ablation] isolates the named mechanism: yes / no
[Validity] known-truth or leakage-guarded: yes / no
[Top evidence gap] <single most important missing experiment>
Version History
- 9f86f09 Current 2026-07-19 15:55
