![]() |
ISCApad Archive » 2013 » ISCApad #185 » Jobs » (2013-07-01) Postdoctoral position INRIA Nancy Grand-Est (Nancy, France) - Speech Group, LORIA |
ISCApad #185 |
Tuesday, November 12, 2013 by Chris Wellekens |
INRIA Nancy Grand-Est (Nancy, France) - Speech Group, LORIA Postdoctoral position Accurate 3D Lip modeling and control in the context of animating a 3D talking head Scientific Context The lips play a significant role in audiovisual human communication. Several studies showed the important contribution of the lips to the intelligibility of visual speech (Sumby & Pollack, 1954; Cohen & Massaro 1990). In fact, it has been shown that human lips alone carry more than half the visual information provided by the face (Benoît,1996). Since the beginning of the development of 3D virtual talking heads, researchers showed interest to model lips (Guiard-Marigny et al., 1996, Reveret & Benoît, 1998), as the lips increase intelligibility of the visual message. The existing models are still considered as pure parametric and numerical models and do not take into account the dynamic characteristic of speech. As audiovisual speech is highly dynamics, we consider that modeling this aspect is crucial to provide a lip model that is accurately animated, and reflects the real articulatory dynamics as observed in human vocal tract. In fact, the movement of the lips, even subtle, can communicate relevant information to the human receiver. This is even more crucial for some population such as hard-of-hearing people. Missions The goal of this work is to develop an accurate 3D lip model that can be integrated within a talking head. A control model will also be developed. The lip model should be as accurate dynamically as possible. When designing this model, the focus will be on the dynamics. For this reason, one can start from a static 3D lip mesh, using a generic 3D lip model, and then we will use MRI images or 3D scans to obtain more realistic shape of the lips. To take into account the dynamic aspect of the lip deformation, we will use an articulograph (EMA) and motion capture technique to track sensors or markers on the lips. The mesh will be adapted to this data. To control the lips, we will consider allowing a skeletal animation to be controlled by the EMA sensors or motion capture markers, using inverse kinematic technique, widely used in 3D modeling. In line with conventional skeletal animation, an articulated armature rigged inside the mesh is mapped to vertex groups on the lip mesh by a weight map that can be defined automatically from the envelope of the armature's shape and manually adjusted if required, where manipulating the armature's components deforms the surrounding mesh accordingly. The main challenge is to find the best topology of the sensors or markers on the lips, to be able to better capture accurately its dynamics. The main outcome is to accurately model and animate the lips based on articulatory data. It is very important to have readable lips in that can be lip-read by hard-of-hearing people. Bibliography C. Benoît (1996). On the Production and the Perception of Audio-Visual Speech by Man and Machine. Multimedia Communications and Video Coding, pp 277-284. M. M. Cohen & D. W. Massaro (1990), Synthesis of visible speech. Behavioral Research Methods and Instrumentation, 22, 260-263. T. Guiard-Marigny, N. Tsingos, A. Adjoudani, C. Benoit, M.-P. Cani (1996). 3D Models of the Lips for Realistic Speech Animation. Computer Animation 80-89 L. Reveret, C. Benoit (1998). A New 3D Lip Model for Analysis and Synthesis of Lip Motion in Speech Production. Proc. AVSP'98, Terrigal, Australia, Dec. 4-6, 1998. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of Acoustic Society of America, 26, 212-215. Q. Summerfield (1987), 'Some preliminaries to a comprehensive account of audio-visual speech perception', In: B. Dodd and R. Campbell, Editors, Hearing by Eye: The Psychology of Lip-Reading, Lawrence Erlbaum, Hillsdale, NJ. Competences Required qualification: PhD in computer science Appropriate candidate would have good knowledge in 3D modeling, speech processing and data analysis, as well as solid java programming skills. Additional Information Application deadline: 11 June 2013 Supervision and contact: Slim Ouni ( Slim.Ouni@loria.fr ) http://www.loria.fr/~slim Duration: 1 year (possibly extendable) Starting date: between Sept. 1st 2013 and Jan. 1st 2014 Salary: 2.620 euros gross monthly (about 2.135 euros net) medical insurance included. Application Procedure The required documents for an INRIA postdoc application are the following: - CV, including a description of your research activities (2 pages max) and a short description of what you consider to be your best contributions and why (1 page max and 3 contributions max); the contributions could be theoretical, implementation, or industry transfers. Include also a brief description of your scientific and career projects. - The report(s) from your PhD external reviewer(s), if applicable. - If you haven't defended yet, the list of expected members of your PhD committee (if known) and the expected date of defense (the defense, not the manuscript submission). - Your best publications, up to 3. - At least one recommendation letter from your PhD advisor, and possibly up to two other letters. The recommendation letter(s) should be sent directly by their author to the prospective postdoc advisor All these documents should be sent before June 11th About INRIA Established in 1967, Inria is the only public research body fully dedicated to computational sciences. Combining computer sciences with mathematics, Inria’s 3,400 researchers strive to invent the digital technologies of the future. Educated at leading international universities, they creatively integrate basic research with applied research and dedicate themselves to solving real problems, collaborating with the main players in public and private research in France and abroad and transferring the fruits of their work to innovative companies. The researchers at Inria published over 4,800 articles in 2010. They are behind over 270 active patents and 105 start-ups. The 171 project teams are distributed in eight research centers located throughout France. http://www.inria.fr/en/centre/nancy |
![]() | Back | ![]() | Top |