FAST Artifact Evaluation

Use this for the artifact track. FAST follows the USENIX artifact-evaluation model, and evaluation is a separate, post-acceptance process with its own timeline and an Artifact Evaluation Committee (AEC). Two things to internalize: badges are earned by evaluators actually using your package, and the review-time artifact (anonymized, for the paper's reviewers) is not the same deliverable as the badge artifact (de-anonymized, permanently archived, with an artifact appendix).

The three USENIX badges (verify the current set and names)

Available is a low-cost, high-value badge (archive the package). Functional and Reproduced require the evaluator's own run to succeed — and for a storage paper that run may need specific hardware, which shapes everything below.

The storage-artifact problem: hardware dependence

A storage result often depends on a particular SSD/NVM device, firmware, or a large trace. Design the package so an AEC can get as far as possible without your exact drive:

[Tier the claims]  which results are hardware-agnostic (analysis, plots from logs) vs. device-bound
                   (measured WA, tail latency)? Make the agnostic ones turnkey.
[Ship the logs]    include the raw device-counter logs and the scripts that turn them into figures,
                   so "Reproduced" is achievable from data even when the device is unavailable
[Document devices] state exact models + firmware; where feasible, offer a smaller commodity-SSD path
                   the AEC can run to see the trend, and say the factor is device-specific
[Bound the runtime] long endurance/aging runs: provide a short demo + the full slow protocol, and
                   label which results need the full run

What a storage AEC opens first

Assume an evaluator gives your package a bounded time budget. Design for the first ten minutes (build

• a small demo + a figure-from-logs) to succeed.

Packaging plan

[Container]   ship a Dockerfile or a pinned environment; avoid "install these 40 things by hand"
[README]      one screen: what it is, build, run the demo, reproduce each claim, expected runtime/outputs
[Mapping]     an explicit table: paper claim -> script -> expected result (mark device-bound ones)
[Data]        the archived traces and raw device-counter logs, not just names
[Provenance]  device models + firmware, host/kernel, mkfs/mount options, seeds
[License]     an OSI-approved license so the artifact can be evaluated toward Functional/Reproduced
[Archive]     deposit in a DOI-issuing repository for the Available badge
[Appendix]    write the artifact appendix; add the earned badges to the camera-ready appendix after AE

Anonymized review artifact vs. badge artifact

• At submission: anonymized for the paper's reviewers — no owner strings, hostnames, cluster paths, serials in device dumps, lab names, or identity-revealing trace hosts.
• After acceptance: replace anonymized placeholders with the public, licensed, DOI-issuing archive; write the artifact appendix; this is the version the AEC badges and the camera-ready cites, with badges shown on the first page.

Worked vignette: packaging a KV-store + measurement study

A paper contributes an SSD-conscious KV store and a write-amplification study. To target Reproduced: ship a Docker image with the store pre-built; a run_demo.sh that runs a short YCSB load on a commodity SSD (or a loopback file) in minutes; a reproduce/ directory whose scripts regenerate each figure from the shipped device-counter logs so the headline numbers reproduce even without the exact datacenter SSD; a claim-to-script table marking which results are device-bound; the archived trace with replay scripts; and an MIT/Apache license. State honestly which numbers are the specific drive's and which are the trend.

Calibration

• The artifact timeline is after acceptance and independent of the camera-ready deadline; do not conflate them.
• Badge names, the exact set offered, and whether AE is single- or double-anonymous vary by cycle (待核实 for the precise FAST '27 AEC anonymity model) — confirm on the current Call for Artifacts.

Output format

[Target badges] Available / Functional / Reproduced
[Artifact role] anonymized review artifact / public badge artifact
[Contents] <system/traces/device-logs/scripts/provenance/license>
[Hardware tiering] <which claims turnkey-from-logs vs. device-bound; demo path present?>
[Ten-minute test] does build + demo + a figure-from-logs succeed on a clean machine? yes/no
[Fixes before upload] <ordered list>