Current Research Directions
Speech Production and Articulation kNowledge (SPAN)
The SPAN Group bridges multiple interdisciplinary departments, labs and projects at USC. It brings together faculty and students from the Viterbi School of Electrical Engineering, the College’s Department of Linguistics, and the Department of Computer Science.An appropriate understanding of how language is produced in space and time directly informs, and is informed by, our knowledge of the phonological representation of speech, a representation that we take to be intrinsically articulatory and dynamic.
Speech to speech translation
US federal agencies are seeking new technologies to support multinational, multilingual operations. One significant challenge in such operations is cross-language communication between speakers of different native languages. DARPA and other DoD research groups envision enabling cross-language communication via computer-mediated translation of speech between the native languages of the speakers. The University of Southern California will develop SpeechLinks Solutions prototypes enabling human-human communication via automated two-way speech-to-speech language translation of English-Farsi and English-Dari.
Communicative channels, such as gestures, facial expressions and speech, are jointly invoked to convey and express an intended message, which includes not only the spoken content (verbal channel), but also implicit cues (non-verbal channel) that enrich human communication. Notable among these cues are the emotional states, which play a crucial role in inter-personal human interaction. Recent findings suggest that rational and intelligent communication between humans is closely related to emotions. Therefore, it is essential that emotions be integrated within human-machine interfaces that are more in tune with the users’ needs and preferences. Toward creating interfaces, it is first necessary to study how emotions modulate and interact with the verbal and non-verbal channels in human communication.
Center for Autism Research in Engineering (CARE)
Children diagnosed with autism have impairments related to social interaction and communication. Methods for addressing these impairments attempt to engage children in stimulating social activities that facilitate developing basic social behavior such as sharing interests with others, self-initiation of social behaviors, joint attention, and experience-sharing. It has often proved difficult, however, to identify activities that are interesting and engaging for the diverse spectrum of children with autism. These components are essential for the creation of advanced human behavioral informatics as well as natural human-machine communication systems.
Behavioral Signal Processing (BSP)
Behavioral signal processing focuses on gathering, analyzing and modeling multimodal behavior signals, both overtly and covertly expressed. This builds upon foundational efforts in speech/language/audio/visual/physiological signal processing. Critically, in addition to processing objectively-specified behavioral content in richer ways (e.g., what someone said and did), behavioral signal processing entails automating a host of subjectively-specified entities such as those related to socio-emotional states of people (e.g., how negative or frustrated a person is; politeness; engagement etc).
Media Informatics and Content Analysis (MICA)
MICA conducts research on a wide variety of media related tasks that focus on human-centric applications. Our emphasis is on speech, music, text and multimodal information processing. MICA’s research applications and systems development concentrate on domains with direct societal relevance ranging from automated movie content analysis to understand gender roles to gathering informatics on human interactions from speech and body gestures.
Behavior and Experience Analysis and Modeling (BEAM)
The BEAM group investigates human experience modeling and behavior analysis using tools from engineering, psychology, mathematics, and social sciences. The team's projects bring together faculty, post-docs, and students across disciplines and universities to discover new methods for understanding people and both their physiological and mental states.