The Lifelong learning Speaker Diarization Challenge 2020
https://lium.univ-lemans.fr/allies-evaluation/
*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*

CHALLENGE TASKs
______________________________________________________________________________________
The ALLIES evaluation focuses on  freeing speaker diarization
systems from the need of machine learning expert interventions upon two tasks:

        * Diarization Across time:
                automatic systems use the stream of incoming data
        to update their knowledge and adapt to new data in order
        to sustain performance across time

    * Lifelong learning speaker diarization
            systems are evaluated while processing a sequence
            of incoming audio documents with or without
            human-assisted learning including active learning (system initiative)
            or interactive learning (user initiative)
            Details are provided in the evaluation plan.

The ALLIES corpus consists of a new corpus of audio-visual
documents (news, debates, talk show.) including more
than 200 hours of TV shows from the French channel LCP.


BACKGROUND
______________________________________________________________________________________
Speech segmentation with speaker clustering, referred
to as speaker diarization, is a key pre-processing step
for several speech technologies including enriched automatic
speech recognition (ASR) or spoken document retrieval (SDR)
in very large multimedia repositories. The base accuracy of
such systems is of essential to allow applications to perform
adequately in real-world environments.
Performance of such systems usually degrades across time as
the distribution of incoming data moves away from the initial
training data (changes in accents, in recording conditions, etc).
Thus sustaining system performance across time requires frequent
interventions of machine learning experts which makes the maintenance
of such system very costly.


SCHEDULE
______________________________________________________________________________________
    From now until 31st of March: Registration for participants is open
    1st of March: Release of Development data
    1st of March: Beat platform is open for development
    1st of June: Evaluation data is released
    30th of June: Final submission
    End of July: Paper submission deadline
    November: Iberspeech conference in Valladolid


REGISTRATION
______________________________________________________________________________________
Send an email to :

        lifelong-speaker-evaluation@univ-lemans.fr

and specify
        * the task(s) you're willing to participate
        * the name of your team, that can be the name of your organization or any anonymous identity

More details on the evaluation plan.


EVALUATION PLAN
______________________________________________________________________________________
The evaluation plan and license agreement of the dataset can be downloaded here:

https://lium.univ-lemans.fr/allies-evaluation/


ORGANIZERS
Anthony Larcher, Le Mans Université, France
Olivier Galibert, LNE, France
Andre Anjos, IDIAP, Switzerland
Marta Ruiz Costa, UPC, Spain
Loïc Barrault, Sheffield University, UK