(open to authors of papers at ACM Multimedia 2018 or 2017)
Authors of main-conference papers at ACM Multimedia 2017 and 2018 are invited to make a submission to the new ACM Multimedia Reproducibility track at ACM Multimedia 2019. The track implements the ACM Artifact and Review and Badging process.
In 2019, we will focus on the “Results Replicated” badge. ACM defines “Results Replicated” as “The main results of the paper have been obtained in a subsequent study by a person or team other than the authors, using, in part, artifacts provided by the author.” (from https://www.acm.org/publications/policies/artifact-review-badging)
A submission to the Reproducibility track is composed of two parts:
- Companion paper: The companion paper is 2–4 pages in length. It contains a high-level description of the experiments carried out in the original paper and that are implemented in the archive.
- Archive: Contains the artifacts (e.g., code, scripts, data sets, protocols), which are cleanly packaged, ready for download and use to replicate the results from the original paper. It contains detailed readme file(s), examples, and all information needed to successfully carry out the experiments.
For additional information, please see the Call for Reproducibility Papers posted on the ACM Multimedia Reproducibility website: https://project.inria.fr/acmmmreproducibility/call-2019/
More about reproducibility in general, and its implementation in the ACM Multimedia conference series is here: https://project.inria.fr/acmmmreproducibility
The companion paper and the associated artifacts will undergo a reproducibility review, which will result in an accept or a reject decision. Rejected papers are not disclosed.
If accepted, a “Results Replicated” ACM badge is added to the original paper and to the companion paper, which are both stored in the ACM Digital Library, together with the artifacts. The authors of the companion papers are both the authors of the original paper and the reproducibility reviewers. The badged companion paper will appear in the ACM Multimedia 2019 proceedings and will be presented as a poster in the ACM Multimedia 2019 Reproducibility poster session.
We are proud to introduce this Reproducibility Track in 2019, which will implement ACM Artifact Review and Badging for the first time at ACM Multimedia. As described above, the track has been designed to promote reproducibility, which in turn raises the visibility of papers that you have previously published at ACM Multimedia. Should you have any questions about the track or the process, please contact the ACM Multimedia 2019 Reproducibility Chairs.
Deadline: 1 April 2019
ACM Multimedia 2019 Reproducibility Chairs
Laurent Amsaleg, CNRS-IRISA, France
Björn Þór Jónsson, IT University of Copenhagen, Denmark