| | Call for Contributions
Special track
SATASK: Task-based evaluation of speech/audio interfaces
Chair
Prof. Gerald Penn, University of Toronto, Canada
gpenn@cs.toronto.edu
along with
ACHI 2023, The Sixteenth International Conference on Advances in
Computer-Human Interactions
https://www.iaria.org/conferences2023/ACHI23.html
April 24 - 28, 2023
Venice, Italy
Long gone is the hegemony of the Word Error Rate (WER). After years
of painstaking research by the Human-Computer Interaction (HCI)
community documenting the enormous variance between WER and the actual
performance of speech recognition systems within the context of
applications other than transcription, speech and acoustic signal
processing engineers are ready to listen.
There have been a few proposals for benchmarks already, such as SLUE
and ASR-GLUE, both coined from the wildly popular General Language
Understanding Evaluation (GLUE) for text processing, but to date the
speech analogues of GLUE have been much humbler in both their breadth
and uptake. At the same time, the same causes for concern exist as
for GLUE: is sentiment analysis actually a task? What constitutes a
task in related HCI work would be something people want to perform and
could express (dis)satisfaction with, e.g., voice-mail interaction or
meeting summarization. But this kind of vertical definition could
easily result in a proliferation of datasets and disagreements over
approaches.
The purpose of this track is to encourage proposals and proofs of
concept by both engineering and HCI researchers to converge on an
appropriate benchmark.
Topics include, but not limited to:
' Individual task proposals
' General criteria/desiderata for individual task proposals
' Frameworks and codebases to support evaluation benchmarks
' Case studies with existing benchmarks or ad hoc groups of tasks
' Proposals of new intrinsic measures
' Systemic considerations governing balance across tasks/measures
' Automated proxies for human-subject evaluation
Contribution Types
- Regular papers [in the proceedings, digital library]
- Short papers (work in progress) [in the proceedings, digital library]
- Posters: two pages [in the proceedings, digital library]
- Posters: slide only [slide-deck posted on www.iaria.org]
- Presentations: slide only [slide-deck posted on www.iaria.org]
- Demos: two pages [posted on www.iaria.org]
Important Datelines
Inform the Chair: As soon as you decide to contribute
Submission: March 9 (earlier, better)
Notification: March 27
Registration: April 6
Camera-ready: April 6 Note: The submission deadline is somewhat flexible
Paper Format
- 6 pages on US-letter, plus up to 4 extra pages at additional cost
For further details, see: http://www.iaria.org/format.html
- Before submission, please check and comply with the editorial rules:
http://www.iaria.org/editorialrules.html
Publications
- Extended versions of selected papers will be published in IARIA
Journals: http://www.iariajournals.org
- Print proceedings will be available via Curran Associates, Inc.:
http://www.proceedings.com/9769.html
- Articles will be archived in the free access ThinkMind Digital Library:
http://www.thinkmind.org
Paper Submission
https://www.iariasubmit.org/conferences/submit/newcontribution.php?event=ACHI+2023+Special
Please select Track Preference as SATASK
Registration
- Each accepted paper needs at least one full registration, before the
camera-ready manuscript can be included in the proceedings.
- Registration fees are available at http://www.iaria.org/registration.html
Contact
Chair: Gerald Penn, gpenn@cs.toronto.edu
Logistics: steve@iaria.org
https://www.iariasubmit.org/conferences/submit/newcontribution.php?event=ACHI+2023+Special
Note: Onsite and Online Options
In order to accommodate a large number of situations, we are offering
the option for either physical presence or virtual participation. We
would be delighted if all authors manage to attend in person but are
aware that special circumstances are best handled by having flexible
options.
|
