![]() |
ISCApad Archive » 2022 » ISCApad #288 » Jobs » (2022-04-23) Doctoral position : Acoustic to Articulatory Inversion by using dynamic MRI images, INRIA, Nancy, France |
ISCApad #288 |
Friday, June 10, 2022 by Chris Wellekens |
Doctoral position : Acoustic to Articulatory Inversion by using dynamic MRI images Loria “Lorraine Research Laboratory in Computer Science and its Applications” is a research unit common to CNRS, the Université de Lorraine and INRIA. Loria gathers 450 scientists and its missions mainly deal with fundamental and applied research in computer sciences, especially the MultiSpeech Team which focuses automatic speech processing, audiovisual speech and speech production. IADI is a research unit common to Inserm the Université de Lorraine whose specialty is developing various techniques and methods to improve imaging of moving organs via the acquisition of MR images.
This PhD project founded by LUE (Lorraine Université d’Excellence) associates the Multispeech team and the IADI laboratory.
Start date is (expected to be) 1st October 2022 or as soon as possible thereafter.
Supervisors Yves Laprie, email yves.laprie@loria.fr Pierre-André Vuissoz, email pa.vuissoz@chru-nancy.fr
The project
Articulatory synthesis mimics the speech production process by first generating the shape of the vocal tract from the sequence of phonemes to be pronounced, then the acoustic signal by solving the aeroacoustic equations. Compared to other approaches to speech synthesis which offer a very high level of quality, the main interest is to control the whole production process, beyond the acoustic signal alone. The objective of this PhD is to succeed in the inverse transformation, called acoustic to articulatory inversion, in order to recover the geometric shape of the vocal tract from the acoustic signal. A simple voice recording will allow the dynamics of the different articulators to be followed during the production of the sentence. Beyond its interest in terms of scientific challenge, articulatory acoustic inversion has many potential applications. Alone, it can be used as a diagnostic tool to evaluate articulatory gestures in an educational or medical context.
Description of work
The objective is the inversion of the acoustic signal to recover the temporal evolution of the medio-sagittal slice. Indeed, dynamic MRI provides two-dimensional images in the medio-sagittal plane at 50Hz of very good quality and the speech signal acquired with an optical microphone can be very efficiently deconstructed with the algorithms developed in the MultiSpeech team (examples available on https://artspeech.loria.fr/resources/). We plan to use corpora already acquired or in the process of being acquired. These corpora represent a very large volume of data (several hundreds of thousands of images) and an approach for tracking the contours of articulators in MRI images which gives very good results was developed to process corpora. The automatically tracked contours can therefore be used to train the inversion. The goal is to perform the inversion using the LSTM approach on data from a small number of speakers for which sufficient data exists. This approach will have to be adapted to the nature of the data and to be able to identify the contribution of each articulator. In itself, successful inversion to recover the shape of the vocal tract in the medio-sagittal plane will be a remarkable success since the current results only cover a very small part of the vocal tract (a few points on the front part of the vocal tract). However, it is important to be able to transpose this result to any subject, which raises the question of speaker adaptation, which is the second objective of the PhD.
What we offer
Supervisors Yves Laprie, email yves.laprie@loria.fr Pierre-André Vuissoz, email pa.vuissoz@chru-nancy.fr
Application Your application including all attachments must be in English and submitted electronically by clicking APPLY NOW below. Please include:
log into Inria’s recruitment system (https://jobs.inria.fr/public/classic/en/offres/2022-04654) in order to apply to this position. |
![]() | Back | ![]() | Top |