NDSS Artifact Evaluation

NDSS added artifact evaluation to the symposium in recent editions (running on the community secartifacts platform, verified for 2024-2026), and it is opt-in after conditional acceptance — a separate track, not a submission requirement. Papers that pass carry badges on the first page and add a 2-page artifact appendix (Call for Artifacts, checked 2026-07-08). Decide early whether you are engineering for it, because the packaging choices are cheap during the project and expensive afterward.

The three badges

Badges are independent — you can earn Available and Functional without Reproduced when the world (live targets, proprietary data) genuinely prevents re-execution. Claim only what you can defend; an evaluator who cannot reproduce a "Reproduced" claim writes that in the report.

From claims ledger to artifact

The claims.yml ledger built in ndss-reproducibility is the artifact's spine. Convert it into an evaluator-facing package:

artifact/
  README.md          # what this is, badges sought, hardware/time needs, safety notes
  INSTALL.md         # clean-machine setup; pinned deps / container image
  claims/            # one entry per headline result:
    fig4/  run.sh    # → expected_output/, runtime + resource note
    tab2/  run.sh
  data/              # snapshots or synthetic substitutes (never raw victim traffic)
  src/               # the code, with the dangerous parts gated (below)
  ETHICS.md          # disclosure status, gating rationale, usage boundaries

Give evaluators the shortest true path to each result. A "kick-the-tires" phase means the first five minutes decide the evaluator's mood — make the smoke test one command.

Packaging dangerous material responsibly

NDSS artifacts routinely include working exploits, attack tooling, or malware-adjacent code. This is allowed; unbounded release is not.

• Gate the weaponized parts. Ship the analysis and measurement pipeline openly; put live exploit modules behind documented access conditions or a defanged demonstration mode consistent with your Ethics Considerations section.
• Align with the disclosure clock. If a vulnerability is under a coordinated-disclosure embargo, the artifact's release timing must respect it — state the date the full module becomes available.
• Sandbox by default. Anything that touches a network should target a supplied local testbed, not the Internet, out of the box. Document the blast radius.
• No live victim data. Replace with prefix-preserved or synthetic traces carrying the statistics the result needs (see ndss-reproducibility).

When re-execution needs your lab

Hardware-bound results (SDRs, specific switches, IoT devices) cannot be reproduced by a committee at their desks. Bridge the gap: provide recorded traces plus a replay harness so the analysis is reproducible even when the capture is not, and say plainly in the README which results are trace-replayable and which need the physical setup. This commonly yields Functional + Reproduced-on-recorded-data rather than a failed full Reproduced.

The 2-page artifact appendix

Written after evaluation for the camera-ready. It states the badges earned, the artifact's scope, access instructions (the DOI), hardware/software requirements, and the mapping from paper claims to artifact components. Keep it factual and navigational — it is a map for future readers of the open-access paper, not a second results section.

Timing

The opt-in comes shortly after (conditional) acceptance and runs on its own schedule (2027 dates 待核实 — confirm on the Call for Artifacts). It overlaps the camera-ready window (Jan 6, 2027), so a team that starts packaging at acceptance, not at the AE deadline, avoids a two-front crunch.

Output format

[Badges targeted] Available / Functional / Reproduced — with feasibility per badge
[Package status] README / INSTALL / per-claim runners / data substitutes — done or gaps
[Danger handling] gated modules, sandbox default, disclosure-aligned release date
[Hardware gap] results needing physical setup + trace-replay bridge
[Appendix] 2-page artifact appendix drafted? claim→component map complete?