ACM SIGCOMM 2024 Call for Artifacts

The SIGCOMM 2024 conference’s artifact evaluation process is open to accepted papers at the SIGCOMM 2024 conference, to facilitate the public distribution of data, hardware, software, survey results, proofs, models, test suites, benchmarks, and other artifacts associated with a paper available for other researchers’ use. Artifact evaluation also intends to publicly recognize authors’ efforts in preparing artifacts, as well as to help improve those artifacts before their broader distribution.

Important Dates

Organizers

Committee Co-Chairs
Francis Y. Yan Microsoft Research
Guyue Liu Peking University
Yang Zhou Harvard University
Committee
TBA

Artifact Review and Badging

At artifact submission time, the authors will choose the criteria by which their artifacts will be evaluated. Based on ACM’s guidelines (https://www.acm.org/publications/policies/artifact-review-and-badging-current), the SIGCOMM 2024 conference will award two separate badges to a paper. An artifact can meet the criteria of one or both of the following badges:

When the AEC judges that an artifact meets the criteria for one or both of the badges listed above, those badges will appear on the final version of the associated paper. In addition, the authors of the paper will be encouraged to add an Artifact Appendix of up to two pages to their publication. The goal of the appendix is to describe and document the artifact in a standard format.

Review and Anonymity

Artifact evaluation is "single-blind." The identities of artifact authors will be known to members of the AEC, but authors will not know which members of the AEC have reviewed their artifacts.

To maintain the anonymity of artifact evaluators, the authors of artifacts should not embed any analytics or other tracking in the websites for their artifacts for the duration of the artifact-evaluation period. This is important to maintain the confidentiality of the evaluators. In cases where tracing is unavoidable, authors should notify the AEC chairs in advance so that AEC members can take adequate safeguards.

At the same time, though the authors are visible to AEC, the submission of an artifact does not give the AEC permission to make its content public. AEC members may not publicize any part of your artifact during or after completing the evaluation, nor may they retain any part of it after evaluation. Thus, you are free to include models, data files, proprietary binaries, etc. in your artifact. Participating in artifact evaluation does not require you to later publish your artifacts (although it is encouraged).

Submission Instructions

The SIGCOMM 2024 conference’s artifact evaluation welcomes submissions from all accepted papers. Authors of the accepted paper can submit their artifact via the artifact SIGCOMM24AE submission site https://sigcomm24ae.hotcrp.com/.

Packaging Artifacts

The AEC will accept any kind of digital artifact that authors wish to submit: software, data sets, survey results, test suites, mechanized proofs, etc. Physical objects, e.g., computer hardware, cannot be accepted due to the difficulty of making the objects available to members of the AEC. (If your artifact requires special hardware, consider if/how you can make it available to evaluators online.)

A complete artifact package must contain:

For the submitted artifact, we recommend the authors to consider (one, or multiple, but not limited to) the following methods to package their artifacts:

Artifact Metadata and Conflicts

We define conflict of interest with an AEC member using the following principles:

  1. You are currently employed at the same institution, have been previously employed at the same institution within the last 12 months, or are going to begin employment at the same institution.
  2. You have a professional partnership as follows:
    1. Past or present association as thesis advisor or advisee.
    2. Collaboration on a project, publication, or grant proposal within the past 2 years (i.e., 2022 or later).

The AEC chairs and members will review conflicts to ensure the integrity of the reviewing process, adding or removing conflicts where necessary and sanity-checking cases where conflicts do not appear justified. Improperly identifying AEC members as a conflict to avoid individual reviewers may lead to your artifact submission being rejected. If you have concerns, please contact the AEC chairs.

Artifact with Malicious Operations

Some artifacts may attempt to perform malicious or destructive operations by design. These cases should be boldly and explicitly flagged in detail in the README so the AEC can take appropriate precautions before installing and running these artifacts. Please contact AEC co-chairs if you believe that your artifacts fall into this category.

Further Advice

There are several sources of good advice about preparing artifacts for evaluation. These two are particularly noteworthy:

If you have any questions about how best to package your artifact, contact the artifact evaluation committee co-chairs.