research
Emotion expression in human communication causes modulations in speech prosody, facial and bodily gestures as well as in lexical usage. Furthermore, in interactions, these modulations also affect the behaviors of interaction partners, shaping the conversational discourse and altering the communicative environment and outcome. We view emotion as an ever-presenting, background component in the human cognitive system which interacts with perceived events from the environment and determine our responses to the outside world. Please click below subtopics for more details.
Multimodal Emotion Expression
Emotional expression is a complex interplay of complementary, supplementary or even conflicting multimodal cues, such as facial expressions, body language, speech prosody. Extracting informative features as well as understanding and effectively modeling the dynamics of these modalities is important for building emotion recognition systems. Our work focuses on encoding emotion using multimodal signals, as well as analyzing emotional effect on the interplay between multimodal cues. People involved: Carlos Busso(Alumni), Angeliki Metallinou(Alumni), Zhaojun Yang(PhD student) Corpus: IEMOCAP, CreativeIT