Call for Reproducibility Initiative

The Reproducibility Initiative is part of the 18th International Semantic Web Conference (ISWC 2019) – “Knowledge Graphs, Linked Data, Linked Schemas and AI on the Web” that will take place in Auckland, New Zealand on 26-30 October, 2019.

Venue: The University of Auckland, Auckland, New Zealand

Highlights

The Reproducibility Initiative is running for the first time and will evaluate submissions from the accepted papers of the ISWC Research Track.

Description

Many papers submitted to ISWC provide a very strong evaluation. Other researchers willing to build on top of these results often have a hard time to continue the work because the results in the paper are difficult to reproduce. Hence, as an author of a research track paper, we would like to invite you to submit your contribution to the Reproducibility Initiative.

The ISWC Reproducibility Initiative has the following goals:

• To enable easy sharing of code and experimentation set-ups (take a paper and reuse it).
• To make more code and data available.
• To highlight the impact and increase the credibility of the Semantic Web research.
• To facilitate the dissemination of research results.
• We would like to build a culture where sharing results, code, and scripts is the norm rather than an exception.

The challenge is to do this efficiently, which means building technical expertise on how to do better research via creating repeatable and shareable methods and results.

Considering reproducibility as a goal, the assessment of papers will consider a spectrum varying from:

• Basic availability of source code and data
• Fully reproducible results given that the data, source code, and documentation are made available.

For the reproducibility assessment, a member of the Programme Committee will interact with the authors to check the availability of the data, source code, documentation, configuration requirements and reproduce the most important results of the paper.

Scope

The Reproducibility Initiative is open to the accepted papers of the ISWC Research Track that have some significant experimental results that are prone to be reproducible.