Aims and Scope

The goal of CENTRE@CLEF 2018 is to run a joint CLEF/NTCIR/TREC task on challenging participants to reproduce best results of the most interesting systems in previous editions of CLEF/NTCIR/TREC and to contribute back to the community the additional components and resources developed to reproduce the results.
The CENTRE@CLEF 2018 lab will offer three pilot tasks:

  • Task 1 - Replicability: the task will focus on the replicability of selected methods on the same experimental collections;
  • Task 2 - Reproducibility: the task will focus on the reproducibility of selected methods on the different experimental collections;
  • Task 3 - Re-reproducibility: Re-reproducibility: the task will focus on using the components developed in T1 and T2 and made available by the other participants to replicate/reproduce their results.
Since CENTRE is a joint CLEF/NTCIR/TREC activity, participants in Tasks 1 and 2 will be challenged to reproduce methods and systems developed in the other two evaluation activities, i.e. CLEF will challenge against NTCIR and TREC results. Participants in Task 3 will be challenged to re-reproduce the results using the components developed in any of the three evaluation activities.

Sign Up

Important Dates

Task 1 runs due from participants: TBA

Task 2 runs due from participants: TBA

CLEF 2018 conference: September 10-14, 2018

Submission Guidelines



Nicola Ferro, University of Padua, Italy

Maria Maistro, University of Padua, Italy

Tetsuya Sakai, Waseda University, Japan

Ian Soboroff, National Institute of Standards and Technology (NIST), US