Agent Skills
› brycewang-stanford/Awesome-Journal-Skills
› conext-experiments
conext-experiments
GitHub指导ACM CoNEXT论文实验设计，强调基于真实测试床或部署进行系统级评估。要求证据与主张匹配，使用诚实且调优的基线，报告测量统计与不确定性，提供追踪配置溯源，并对ML网络工作执行防污染消融分析。
Trigger Scenarios
设计ACM CoNEXT论文的评估方案
审计CoNEXT论文的实验数据与真实性
准备网络系统类论文的实证部分
Install
npx skills add brycewang-stanford/Awesome-Journal-Skills --skill conext-experiments -g -y
SKILL.md
Frontmatter
{
"name": "conext-experiments",
"description": "Use when designing or auditing the evaluation of an ACM CoNEXT paper — matching evidence to claim shape with real testbeds and deployments, honest and tuned baselines, measurement statistics and uncertainty, trace and config provenance, and contamination-aware ablations for ML-for-networking work."
}
CoNEXT Experiments
Build the evaluation a networking reviewer will actually interrogate. CoNEXT's evidence culture is systems-and-measurement: claims are backed on the real target platform — a testbed, deployment, or trace — with honest baselines and reported uncertainty, not simulation standing in for hardware or a single number with no variance. Because a one-shot major revision is decided on a list of minimum necessary changes, an evaluation gap you leave now often becomes a mandatory fix under a tight window later.
Match evidence to claim shape
| Claim shape | Evidence CoNEXT expects |
|---|---|
| A mechanism is faster/cheaper on real hardware | A run on the real target (switch, NIC, kernel, testbed) under its real constraints, vs. a tuned baseline, with effect sizes |
| A phenomenon exists in the wild | A measurement campaign with documented vantage points, capture dates, and a reproducible extraction methodology |
| An architecture scales | Scalability evidence (real deployment or faithful emulation) across the relevant range, not a point claim |
| An operator intervention helps | Evidence at operationally relevant scale, with the counterfactual measured or bounded |
| A learned component adds value | An ablation isolating the learned part from the mechanism, plus a contamination check |
Real testbeds and deployments over simulation
- Evaluate on the platform the paper is about. If the claim is about switch behavior, run it on the switch (or a faithful hardware testbed); if about a protocol on real paths, use a testbed or trace-driven replay. Simulation-only systems claims are a classic CoNEXT revision risk.
- Describe the testbed so it can be rebuilt: topology, hardware models, firmware/OS versions, link rates, buffer depths, and background traffic. A reviewer who cannot picture the setup cannot trust the numbers.
- State what is emulated vs. real, and why the emulation is faithful for the claim.
Honest, tuned baselines
- Compare against the strongest reasonable baseline, tuned with an equal, documented budget — not a strawman default. "The baseline is not tuned/fair" is one of the most common CoNEXT push-backs.
- Report the baseline's configuration alongside yours so the comparison is auditable.
- Where a fair baseline does not exist, say so and justify the comparison you made.
Measurement statistics and uncertainty
- Report multiple runs and variability (confidence intervals, percentiles/CDFs for latency, not just means). Tail behavior is often the point in networking.
- Use corrected comparisons when testing many conditions; a measurement reviewer notices uncorrected multiple testing.
- Prefer effect sizes and distributions over a single bar; a CDF that shows the whole distribution pre-empts questions a mean hides.
Trace and config provenance (pin it now)
- Record capture vantage points and dates, the extraction/anonymization methodology, and the exact configs used. These cannot be reconstructed after the fact — pin them at collection time.
- For anything derived from operator or third-party data, document the access terms and what can be released (this interacts with double-anonymity and reproducibility).
- Keep the raw-to-result pipeline scripted so the numbers in the paper regenerate from the trace.
Contamination-aware ML-for-networking ablations
If the paper uses a learner or LLM on networking data:
- Isolate the learned component with an ablation (e.g., replace it with a heuristic) so the reviewer sees its marginal value over the mechanism.
- Guard against contamination/leakage: ensure test paths/traces/time windows are disjoint from training, and that a temporal split reflects real deployment.
- Pin model identifiers and dates and cache raw model outputs; a package that needs live API
calls re-samples rather than reproduces (see
conext-reproducibility). - Apply the model-swap test: if the networking lesson does not survive swapping the model, the
paper may belong at an ML venue (see
conext-topic-selection).
Pre-submission evaluation audit
[Claim coverage] every claim has a matching measurement on the real target? yes/no
[Platform realism] real hardware/testbed/trace, or simulation standing in? note each
[Baselines] strongest reasonable, equally tuned, config reported? yes/no
[Uncertainty] multiple runs, CIs/CDFs, corrected comparisons? yes/no
[Provenance] vantage points, dates, configs, firmware/OS pinned? yes/no
[ML checks] ablation + contamination guard + model-swap survives? n/a or yes/no
Output format
[Evaluation status] solid / gaps
[Claim-evidence matrix] <claim -> measurement + platform + baseline + uncertainty>
[Platform] real target used? emulation justified?
[Provenance] traces/configs/firmware pinned for reproducibility
[Revision risk] <the gap most likely to become a minimum-necessary change>
Version History
- 9f86f09 Current 2026-07-19 14:56
