Speech Unit(e)s

The multisensory-motor unity of speech

Understanding how speech unites the sensory and motor streams,

And how speech units emerge from perceptuo-motor interactions

ERC Advanced Grant, Jean-Luc Schwartz, GIPSA-Lab, Grenoble, France

Proposal for a post-doc position beginning in September 2014: “Joint development of perception, action and phonology”

Context

The Speech Unit(e)s project is focused on the speech unification process associating the auditory, visual and motor streams in the human brain, in an interdisciplinary approach combining cognitive psychology, neurosciences, phonetics (both descriptive and developmental) and computational models. The framework is provided by the “Perception-for-Action-Control Theory (PACT)” developed by the PI (Schwartz et al., 2012).

PACT is a perceptuo-motor theory of speech communication, which connects in a principled way perceptual shaping and motor procedural knowledge in speech multisensory processing. The communication unit in PACT is neither a sound nor a gesture but a perceptually shaped gesture, that is a perceptuo-motor unit. It is characterized by both articulatory coherence – provided by its gestural nature – and perceptual value – necessary for being functional. PACT considers two roles for the perceptuo-motor link in speech perception: online unification of the sensory and motor streams through audio-visuo-motor binding, and offline joint emergence of the perceptual and motor repertoires in speech development.

Objectives of the post-doc position

The general objective here is to gather phonetic data on the joint development of perception, action and phonology, to assess how perceptuo-motor speech units emerge and evolve in the course of acquisition, reacquisition, evolution or learning of a given phonological system.

Among the considerable amount of studies about the development of speech production and speech perception in the first years of life, very few consider the link between perception and action. Apart from historical disciplinary reasons – perceptual development is mainly studied by psycholinguists and production development by phoneticians – there is also a theoretical cause to this puzzling separation. Since it is now well known that phonetic perceptual abilities in infants are well in advance of production skills, it has become implicitly obvious for most researchers that perception was largely independent from production. The advance of perceptual on production skills has actually been considered as a major weakness of motor theories, and generally a decisive argument in favor of auditory theories of speech perception.

The view in PACT is different. While it is known that the construction of perceptual prototypes begins earlier than the development of motor prototypes, we assume in PACT that the later development of speech production intervenes in modifying the perceptual repertoire. This will be studied in this post-doc position through two series of experiments, aiming at illustrating two possible mechanisms for motor-driven evolution of perceptual phonetic categories.

1. Reconfiguration of perceptual categories: coupled auditory and motor idiosyncrasies – The development of motor prototypes during acquisition should induce reconfiguration of perceptual categories. Therefore, motor idiosyncrasies – specific choices of individual ways to produce a target inside the space defined by native phonology – should result in perceptual idiosyncrasies: if a subject chooses to produce a specific phonological contrast in her language in a specific way, this should be mirrored in the way the subject perceives the same contrast. This hypothesis will be studied in various phonetic dimensions.

2. Creation of new categories by perceptuo-motor coupling – According to PACT, the acquisition of new motor knowledge could enable the infant to create a new category. A typical example concerns plosive place of articulation, for which acoustic characterisation remains controversial (e.g. Stevens & Blumstein, 1981; Sussman et al., 1998), while articulatory characterization is straightforward. The relationship between the emergence of such categories in perception and in production is seldom considered. We assume that once an infant knows how to produce syllables with different places of articulation, she can realise that they are associated with specific articulatory gestures and hence form a new perceptual category related to articulatory properties. We will test this assumption in infants before and after the onset of canonical babbling (3 to 12 months), to possibly relate the emergence of perceptual categories with the onset of articulatory abilities. The same kind of studies will be possibly undertaken on pre-lingual children after cochlear implantation.

The work will be realized within a speech team in GIPSA-Lab Grenoble (partners in the project: Jean-Luc Schwartz, Anne Vilain), in close collaboration with a developmental and cognitive psychology team in LPNC Grenoble (partners in the project: Hélène Loevenbruck, Olivier Pascalis, David Meary).

Practical information

The post-doc position is open for a two-year period, with a possible third-year prolongation. The position is open from September 2014, or slightly later if necessary.

Candidates should have an experience in phonetics, cognitive and neurocognitive psychology, and developmental psychology, or at least two of these three domains.

Candidates should send as soon as possible a short email to Jean-Luc Schwartz (Jean-Luc.Schwartz@gipsa-lab.grenoble-inp.fr) to declare their intention to submit a full proposal.

Then they must send a full application file in the next weeks. This application file will include an extended CV and a list of publications, together with a letter explaining why they are interested in the project, what could be their specific interests, possibly proposing other experiments related to the general question of the joint development of perception, action and phonology; and also how this position would fit into their future plans for the development of their own career. They should also provide two names (with email addresses) for recommendations about their applications. Preselected candidates will be interviewed.

Final selection should occur before mid-June.