Ph.D. position in Softbank robotics and Telecom-Paris
 
Subject:  Automatic multimodal recognition of users? social behaviors
in human-robot interactions (HRI)

*Places of work*  Softbank Robotics [SB] (Paris 15e) & Telecom Paris [TP] Palaiseau (Paris outskirt)

*Starting date* December 2019

*Funding* CIFRE http://www.anrt.asso.fr/fr/cifre-7843

*Context*
The research activity of the Ph.D. candidate will contribute to :
- Softbank Robotics robot?s software NAOqi, within the Expressivity team responsible for ensuring an expressive, natural and fun interaction with our robots.
- the Social Computing topic [SocComp.] of the S2a team [SSA] at Telecom-ParisTech, in close collaboration with other researchers and Ph.D. students of the team.

* Candidate profile*
As a minimum requirement, the successful candidate should have:
?    A master in one or more of the following areas: human-agent interaction, deep learning, computational linguistics, cognitive sciences, affective computing, reinforcement learning, natural language processing, speech processing
?    Excellent programming skills (preferably in Python)
?    Excellent command of English
?    Very good communication skills, commitment, independent working style as well as initiative and team spirit

Given the multidisciplinary aspect of the subject, priority will be given to multidisciplinary profiles. Ph.D. applicant?s interest in social robotics is required.

*Keywords* Human-Machine Interaction, Social Robotics, Deep Learning, Social Computing, Natural Language Processing, Speech Processing, Computer Vision, Multimodality

*Supervision* :  
Industrial: Marine Chamoux (Softbank robotics),
Academic: Chloé Clavel [Clavel],  Giovanna Varni [Varni] (Telecom-Paris)

*How to apply*
Applications should be sent as soon as possible (the first review of applications will be made in early September). The application should be formatted as **a single pdf file** and should include:
?    A complete and detailed curriculum vitae
?    A letter of motivation
?   The academic credentials and the transcript of grades
?    The contact of two referees

The pdf file should be sent to the three supervisors: mchamoux@softbankrobotics.com, chloe.clavel@telecom-paristech.fr, giovanna.varni@telecom-paristech.fr



*Description*
Social robotics, and more broadly human-agent interaction is a field of human-machine interaction for which the integration of social behaviors is expected to have great potential. 'Socio-emotional behaviors' (emotions, social stances) include thus the role and the reactions of the user towards the robot during an interaction. These behaviors could be expressed differently depending:
-on the user (age, emotional state, ...): some users may have a dominant behavior with the robot, considering it a tool to achieve a goal. Others are more cooperative with the robot, they can be more friendly with it. Still others try to trap or 'troll' the robot.
-on the interaction context  (users do not behave in the same way when interacting with a pepper selling toys, or with a pepper bank secretary). Besides, in each of these situations, the robot must be able to adapt its behavior, and to provide a coherent interaction between the user and the robot, avoiding confusion and frustration.

This Ph.D. will focus on multimodal modeling for the prediction of the user's socio-emotional behaviors during interactions with a robot and on building an engine that is robust to real-life scenarios and different contexts. In particular, the Ph.D. candidate will address the following points:
- the encoding of contextual multimodal representations relevant for the modeling of socio-emotional behavior. Thanks to the robot, we have access to a lot of information on context (market, robot intention, demographics, multi or mono user interaction, etc.) that could be combined to our multimodal representation.
- the development and evaluation of models that take advantage of the complementarity of modalities in order to monitor the evolution of the user's socio-emotional behaviors during the interaction (e. g. taking into account the inherent sequentially of the interaction structure)
The models will be based on sequential neural approaches (recurrent networks) that integrate attention models as a continuation of the work done in [Hemamou] and [BenYoussef19].

Selected references of the team:
[Hemamou] L. Hemamou, G. Felhi, V. Vandenbussche, J.-C. Martin, C. Clavel, HireNet: a Hierarchical Attention Model for the Automatic Analysis of Asynchronous Video Job Interviews.  in AAAI 2019
[Garcia] Alexandre Garcia, Chloé Clavel, Slim Essid, Florence d?Alche-Buc, Structured Output Learning with Abstention: Application to Accurate Opinion Prediction, ICML 2018
[Clavel&Callejas] Clavel, C.; Callejas, Z., Sentiment analysis: from opinion mining to human-agent interaction, Affective Computing, IEEE Transactions on, 7.1 (2016) 74-93.
[Langlet] C. Langlet and C. Clavel, Improving social relationships in face-to-face human-agent interactions: when the agent wants to know user?s likes and dislikes , in ACL 2015
[Maslowski]  Irina Maslowski, Delphine Lagarde, and Chloé Clavel.  In-the-wild chatbot corpus: from opinion analysis to interaction problem detection, ICNLSSP 2017.
[Ben-Youssef17]  Atef Ben-Youssef, Chloé Clavel, Slim Essid, Miriam Bilac, Marine Chamoux, and Angelica Lim.  Ue-hri: a new dataset for the study of user engagement in spontaneous human-robot interactions.  In  Proceedings of the 19th ACM International Conference on Multimodal Interaction, pages 464?472. ACM, 2017.
[Ben-Youssef19] Atef Ben Youssef; Chloe Clavel; Slim Essid Early Detection of User Engagement Breakdown in Spontaneous Human-Humanoid Interaction, IEEE Transactions on Affective Computing, 2019
[Varni] Varni G., Hupont, I., Clavel, C., Chetouani, M. Computational Study of Primitive Emotional Contagion in Dyadic Interactions. IEEE Transactions on Affective Computing, 2017.

[SB] https://www.softbankrobotics.com/emea/fr
[TP] https://www.telecom-paristech.fr/eng/  
[SocComp.] https://www.tsi.telecom-paristech.fr/recherche/themes-de-recherche/analyse-automatique-des-donnees-sociales-social-computing/
[SSA] http://www.tsi.telecom-paristech.fr/ssa/#
[Clavel] https://clavel.wp.imt.fr/publications/
[Varni] https://sites.google.com/site/gvarnisite/