![]() |
ISCApad Archive » 2023 » ISCApad #305 » Events » Other Events » (2024-09-09) Cf Labs Proposals @CLEF 2024, Grenoble, France |
ISCApad #305 |
Friday, November 10, 2023 by Chris Wellekens |
Call for Labs Proposals @CLEF 2024At its 25th edition, the Conference and Labs of the Evaluation Forum (CLEF) is a continuation of the very successful series of evaluation campaigns of the Cross Language Evaluation Forum (CLEF) which ran between 2000 and 2009, and established a framework of systematic evaluation of information access systems, primarily through experimentation on shared tasks. As a leading annual international conference, CLEF uniquely combines evaluation laboratories and workshops with research presentations, panels, posters and demo sessions. In 2024, CLEF takes place in September, 9-12 at the University of Grenoble Alpes, France. Researchers and practitioners from all areas of information access and related communities are invited to submit proposals for running evaluation labs as part of CLEF 2024. Proposals will be reviewed by a lab selection committee, composed of researchers with extensive experience in evaluating information retrieval and extraction systems. Organisers of selected proposals will be invited to include their lab in the CLEF 2024 labs programme, possibly subject to suggested modifications to their proposal to better suit the CLEF lab workflow or timeline. Background The CLEF Initiative (http://www.clef-initiative.eu/) is a self-organised body whose main mission is to promote research, innovation, and development of information access systems with an emphasis on multilingual information in different modalities - including text and multimedia - with various levels of structure. CLEF promotes research and development by providing an infrastructure for:
Scope of CLEF Labs We invite submission of proposals for two types of labs:
We highly recommend organisers new to the CLEF format of shared task evaluation campaigns to first consider organising a lab workshop to discuss the format of their proposed task, the problem space and practicalities of the shared task. The CLEF 2024 programme will reserve about half of the conference schedule for lab sessions. During the conference, the lab organisers will present their overall results in overview presentations during the plenary scientific paper sessions to give non-participants insights into where the research frontiers are moving. During the conference, lab organisers are expected to organise separate sessions for their lab with ample time for general discussion and engagement with all participants - not just those presenting campaign results and papers. Organisers should plan time in their sessions for activities such as panels, demos, poster sessions, etc. as appropriate. CLEF is always interested in receiving and facilitating innovative lab proposals. Potential task proposers unsure of the suitability of their task proposal or its format for inclusion at CLEF are encouraged to contact the CLEF 2024 Lab Organizing Committee Chairs to discuss its suitability or design at an early stage. Proposal Submission Lab proposals must provide sufficient information to judge the relevance, timeliness, scientific quality, benefits for the research community, and the competence of the proposers to coordinate the lab. Each lab proposal should identify one or more organisers as responsible for ensuring the timely execution of the lab. Proposals should be 3 to 4 pages long and should provide the following information:
Lab proposals must be submitted at the following address: https://easychair.org/conferences/?conf=clef2024 choosing the “CLEF 2024 Lab Proposals” track. Reviewing Process Each submitted proposal will be reviewed by the CLEF 2024 Lab Organizing Committee. The acceptance decision will be sent by email to the responsible organiser by 28 July 2023. The final length of the lab session at the conference will be determined based on the overall organisation of the conference and the number of participant submissions received by a lab.
Advertising Labs at CLEF 2023 and ECIR 2024 Organisers of accepted labs are expected to advertise their labs at both CLEF 2023 (18-21 September 2023, Thessaloniki, Greece) and ECIR 2024 (24-28 March 2024, Glasgow, Scotland). So, at least one lab representative should attend these events. Advertising at CLEF 2023 will consist of displaying a poster describing the new lab, running a break-out session to discuss the lab with prospective participants, and advertising/announcing it during the closing session. Advertising at ECIR 2024 will consist of submitting a lab description to be included in ECIR 2024 proceedings (11 October 2023) and advertising the lab in a booster session during ECIR 2024. Mentorship Program for Lab Proposals from newcomers CLEF 2019 introduced a mentorship program to support the preparation of lab proposals for newcomers to CLEF. The program will be continued at CLEF 2024 and we encourage newcomers to refer to Friedberg et al. (2015) for initial guidance on preparing their proposal: Friedberg I, Wass MN, Mooney SD, Radivojac P. Ten simple rules for a community computational challenge. PLoS Comput Biol. 2015 Apr 23;11(4):e1004150. The CLEF newcomers mentoring program offers help, guidance, and feedback on the writing of your draft lab proposal by assigning a mentor to you, who help you in preparing and maturing the lab proposal for submission. If your lab proposal falls into the scope of an already existing CLEF lab, the mentor will help you to get in touch with those lab organisers and team up forces. Lab proposals for mentorship must be submitted at the following address: https://easychair.org/conferences/?conf=clef2024 choosing the “CLEF 2024 Lab Mentorship” track. Important Dates
CLEF 2024 Lab Chairs
CLEF 2024 Lab Mentorship Chair
|
![]() | Back | ![]() | Top |