======================================================================
ALBAYZIN 2016 SEARCH ON SPEECH EVALUATION

The Spanish Thematic Network on Speech Technology (RTTH) and the ISCA Special Interest Group on Iberian Languages (SIG-IL) are pleased to announce the ALBAYZIN 2016 Search on Speech Evaluation, which will be carried out as part of Iberspeech 2016, a biennial event gathering the Spanish researchers on speech Technology. This year?s event will take place in Lisbon (Portugal), on November 23-25, 2016 (see http://iberspeech2016.inesc-id.pt/ for details).

Research groups worldwide are invited to participate in this evaluation.
Here we just provide the key points. The full evaluation plan can be found attached.

**TASKS**

The ALBAYZIN 2016 Search on Speech evaluation involves searching in audio content a list of terms/queries. This evaluation focuses on retrieving the appropriate audio files that contain any of those terms/queries. Two different tasks are defined:

1) SPOKEN TERM DETECTION (STD), where the input to the system is a list of terms,
but this is unknown when processing the audio. This is the same task as in NIST STD 2006
evaluation [2] and Open Keyword Search in 2013 [3], 2014 [4], 2015 [5], and 2016 [6].

2) QUERY-BY-EXAMPLE SPOKEN TERM DETECTION (QbE STD), where the input to the system is an acoustic example per query and hence a prior knowledge of the correct word/phone transcription corresponding to each query cannot be made. This task must generate a set of occurrences for each query detected in the audio files, along with their timestamps as output, as in the STD task.QbE STD is the same task as those proposed in MediaEval 2011, 2012 and 2013 [1].

**REGISTRATION**

Interested groups must register for the evaluation before July 15th 2016, by contacting the organizing team at:

javiertejedornoguerales@gmail.com

with CC to Iberspeech 2016 Evaluation organizers at:

luisjavier.rodriguez@ehu.eus
lapiz@die.upm.es
alberto.abad@l2f.inesc-id.pt
ortega@unizar.es
ajst@ua.pt

and providing the following information:

  Research group (name and acronym)
  Institution (university, research center, etc.)
  Contact person (name)
  Email

**SCHEDULE**

?      June 30, 2016: Release of training and development data

?      July 15, 2016: Registration deadline

?      September 15, 2016: Release of evaluation data

?      October 15, 2016: Deadline for the submission of system outputs and description papers

?      October 31, 2016: Results distributed to participants

?      November 23-25, 2016: Evaluation Workshop at Iberspeech 2016


**REFERENCES**

[1] Metze, F., Anguera, X., Barnard, E., Davel, M., Gravier, G.: Language independent search in
mediaeval?s spoken web search task. Computer Speech and Language (2014).

[2] NIST: The spoken term detection (STD) 2006 evaluation plan. National Institute of Standards
and Technology (NIST), Gaithersburg, MD, USA, 10 edn. (September 2006), http://www.nist.gov/speech/tests/std

[3] NIST: NIST Open Keyword Search 2013 Evaluation (OpenKWS13). National Institute of Standards and Technology (NIST), Washington DC, USA, 1 edn. (July 2013), http://www.nist.gov/itl/iad/mig/openkws13.cfm

[4] NIST: NIST Open Keyword Search 2014 Evaluation (OpenKWS14). National Institute of Standards and Technology (NIST), Washington DC, USA, 1 edn. (July 2014), http://www.nist.gov/itl/iad/mig/openkws14.cfm

[5] NIST: NIST Open Keyword Search 2015 Evaluation (OpenKWS15). National Institute of Standards and Technology (NIST), Washington DC, USA, 1 edn. (July 2015), http://www.nist.gov/itl/iad/mig/openkws15.cfm

[6] NIST: NIST Open Keyword Search 2016 Evaluation (OpenKWS16). National Institute of Standards and Technology (NIST), Washington DC, USA, 1 edn. (July 2016), http://www.nist.gov/itl/iad/mig/openkws16.cfm