Projects

Short Description:

Sponsored by: NSF

Short Description:

Sponsored by: NSF

Short Description: TBA

Sponsored by: DARPA

Short Description: TBA

Sponsored by: NSF Smart Connected Health

Short Description: Study of cognitive aging and dementia for a sub-sample of the Longitudinal Aging Study in India (LASI). The goals include the development of a protocol for the assessment of dementia and mild cognitive impairment (MCI) that can be used within community settings in the ongoing, nationally representative LASI, and an assessment of the prevalence of dementia and MCI in India

Sponsored by: TBA

Short Description: This project combines detailed survey and interview-based research into stakeholder perspectives and preferences on officer communication, a community-led process to refine research objectives in our analysis of communication during motor vehicle stops (Phase 1) with sophisticated human annotation and ML measurement of communication using audio, CPS-3 of 4 video, and personnel data that are rarely shared with outside researchers (Phase 2)

Sponsored by: NSF

Short Description:

Sponsored by: Simons Foundations

Short Description: The major goal of this project is to develop an evidence driven technology supported approach for mental health interventions.

Sponsored by: NIH through Colliga Apps Corp

Short Description:

Sponsored by: Apple

Short Description:

Sponsored by: Simons Foundation

Short Description: Develop methods for detecting emotions and other indicators of influence from social media data.

Sponsored by: DARPA INCAS

Short Description: The goal is to develop multimedia signal informatics for discerning gender representations in media.

Sponsored by: Google (With Geena Davis Institute on Gender in Media)

Short Description: Project to develop robust speech anonymization and voice conversion methods that account for both static and dynamic speaker attributes.

Sponsored by: IARPA

Short Description: Developing methods for for effective, explainable, and multi-lingual authorship attribution and privacy protection

Sponsored by: IARPA HIATUS

Short Description: Objectively illuminate the complex interplay between multimodal neural and biobehavioral signals of preconscious processing (e.g., about a certain intent in response to an external stimulus), mental states (e.g., emotions), and well-being or risk therein (e.g., suicidal ideation)

Sponsored by: DARPA NEAT

Short Description:

Sponsored by: Walt Disney Company

Short Description: Sensor analysis and fusion for detecting human states and affect is a challenging open problem in ubiquitous computing research, as well as a popular milestone for the automobile industry. The use of driver assistance systems has become increasingly popular due to advances in Artificial Intelligence, with the aim of improving road safety and reducing the number of accidents caused by human error. Despite their great potential, the deployment of such technologies is still at infant stage, especially when considering the driver’s affective state, which can greatly impact driving performance. This project aims to address this issue by developing systems and improving the performance of affective state detection in driving with the use of multimodal biometric sensor information, such as EDA, ECG, PPG, and respiration.

Sponsored by: Toyota Research Institute North America (TRINA)

Short Description:

Sponsored by: NSF

Short Description: The goal is to develop multimedia signal informatics for discerning gender representations in media.

Sponsored by: Google Giving through Geena Davis Institute

Short Description:

Sponsored by: Disney Research

Short Description: The major goal of this project is to develop real-time MRI for vocal tract imaging and the study of articulatory control mechanisms.

Sponsored by: NIH/NIDCD

Short Description:

Sponsored by: Amazon-USC Center

Short Description: The aim of the project is to detect adversarial attacks and develop defense algorithms to make neural systems more resilient against such attacks for audio-based speaker identification.

Sponsored by: DARPA

Short Description: The Tracking Individual Performance with Sensors (TILES) is a project holding multimodal data sets for the analysis of stress, task performance, behavior, and other factors pertaining to professionals engaged in a high-stress workplace environments. Biological, environmental, and contextual data was collected from hospital nurses, staff, and medical residents both in the workplace and at home over time. Labels of human experience were collected using a variety of psychologically validated questionnaires sampled on a daily basis at different times during the day. The data sets are publicly available and we encourage researchers to use it for data mining and testing their own human behavior models.

Sponsored by: IARPA (MOSAIC Program)

Short Description: Develop quantitative metrics based on signal processing and machine learning as markers of behavior change in ASD.

Sponsored by: Simons Foundation (With Cornell Weil Medical College)

Short Description: Develop multilingual emotion and sentiment analysis techniques for disaster assistance/humanitarian relief scenarios.

Sponsored by: Department of Defense (DARPA LORELEI program)

Short Description: The goal is to develop behavioral signal processing methods to support suicide research and interventions.

Sponsored by: Department of Defense (with University of Utah)

Short Description: The goal is to develop automated behavioral coding algorithms for a coach's behavior in obesity interventions.

Sponsored by: CDC (with Arizona State University, Northwestern, U Washington)

Short Description: The goal of this effort is to create the computational framework for analysis and modeling therapist fidelity in the Motivational Interviewing style of therapy that aims to change behavior of addiction patients.

Sponsored by: NIH/NIAAA (with University of Washington)

Short Description: The goal of this project is create novel technology-facilitated measures of socio-communication abilities.

Sponsored by: NIH (with UCSF)

Short Description: The goal of this effort is to investigate the articulatory basis of inter speaker variability.

Sponsored by: NSF (With MIT Lincoln Laboratory)

Short Description: The goal of this project is to develop computational techniques to study relationship dynamics using data scientific approaches.

Sponsored by: NSF

Short Description: The goal of this project is to develop and apply multimodal technologies for investigating aggressive behavior in dating couples.

Sponsored by: NSF

Short Description: The goal of this project is to develop signals and systems techniques to enable user adaptive environmental control.

Sponsored by: NSF

Short Description: The focus in this project to develop computational techniques to analyze and model expressive communication in voice and face.

Sponsored by: NIH NIDCD through Emerson College

Short Description: We examine the role of machine intelligence and adaptive classification systems in the context of wireless communications and cooperation to provide a coherent global framework for real-time evaluation and decision making for system deployments, navigation, and adaptive search.

Sponsored by: Office of Naval Research (ONR)

Short Description: The proposed research focuses on creating robust, widely-deployable and cost-effective technology solutions for enabling and enhancing cross-lingual spoken interaction between people who do not share a common language. Our target application focuses on communication between healthcare personnel who speak English only and patients with limited-English proficiency.

Sponsored by: National Science Foundation (NSF)

Short Description: We propose an Expedition to define and explore a new research area of Behavior Imaging— integrated technologies for multi-modal computational sensing and modeling to capture, measure, and understand human behaviors. Our motivating goal is to revolutionize the diagnosis and treatment of behavioral and developmental disorders.

Sponsored by: NSF: Expeditions in Computing

Short Description: BE A SCIENTIST! is a new model in informal science education (ISE) for bringing cutting-edge science to the public through service learning, with applicability to different science, technology, engineering and mathematics (STEM) fields. The goal is to create a scalable, technology-based, ISE infrastructure to recruit more underserved families, encourage deeper participant understanding of the scientific process and cost-effectively assess long term program impact on the participants.

Sponsored by: National Science Foundation (NSF)

Short Description: The goals of the Exploration and Exploitation in Actuated Communication Networks project are to design networking tools for mobile underwater networks, develop novel navigation mechanisms for communication-constrained autonomous underwater vehicles and to ultimately integrate sensing and classification to provide solutions for the exploration-exploitation tradeoff.

Sponsored by: National Science Foundation (NSF)

Short Description: Human vocal behavior–conveying both verbal and non verbal information–signifies a key vehicle for such computational modeling, and offers a promising venue for evidence driven behavioral assessment and diagnostics. Our team brings extensive experience in voice, speech, spoken language and associated nonverbal behavior analysis.

Sponsored by: Defense Advanced Research Projects Agency (DARPA)

Collaborators: Raytheon BBN Technologies

Short Description: Human spoken language–conveying both verbal and non verbal information–is a key vehicle for not only determining “who” is saying “what” but offers a promising venue for capturing higher-level socio-emotional behavioral assessment and modeling. Specific problems being studied include: Automatic Speech Recognition Robust to Age, Gender, and Linguistic background-Robust Voice activity detection and Speaker Segmentation-Novel Speech Features and others.

Sponsored by: Army Research Office (ARO)

Short Description: Speech processing in low signal-to-noise (SNR) condition is challenging depending on data recording and transmission conditions. As the spectral characteristics change from one type of sound to another, the way their acoustic properties change in low SNR depends on the type of sound and the spectral characteristics of the noise.

Sponsored by: Defense Advanced Research Projects Agency (DARPA)

Collaborators: IBM TJ Watson

Short Description: This interdisciplinary proposal identifies a specific set of HRI research questions in socially assistive robotics, the study of robotic systems capable of providing help through social rather than physical interaction. The research focus is on two key issues: (1) the role of the robot’s physical embodiment in the interaction; and 2) the use of expressive embodied communication and user modeling toward personalized time-extended assistive interaction.

Sponsored by: National Science Foundation & Nancy Laurie Marks Family Foundation & Clinical and Translational Sciences Institute

Short Description: Psychotherapy intervention studies collect audio and/or videotapes of therapy sessions to use in treatment adherence and process research. These tapes record the complex series of interactions between therapist and client, and as such, they encode the active ingredients of the therapy – when the therapy works, the tapes should tell us why.

Sponsored by: National Institute of Health (NIH): NIAAA

Collaborators: Univeristy of Washington & University of Callifornia Irvine

Short Description: Obstructive sleep apnea syndrome (OSAS) has been reported to occur in approximately 50% in morbidly obese children, compared to only ~2% in the general pediatric population. This marked difference in prevalence rates becomes even larger if one were to include other forms of sleep-related breathing disorders (SRBD) such as hypoventilation and hypercapnia or hypoxemia without frank obstructions.

Sponsored by: National Institute of Health (NIH): NHLBI

Short Description: The long-term goal of this project is to wed state-of-the-art technology for imaging the vocal tract with a linguistically informed analysis of the speech tasks or goals requisite in the production of spoken language. Importantly, this includes furthering our understanding of the relations between vocal tract shaping and speech acoustics.

Sponsored by: National Institute on Deafness and Other Communication Disorders (NIH NIDCD)

Short Description: The proposal aims to understand, as a function of vocal tract structure, (1) the ways speakers modulate the spatiotemporal organization of syllable articulation and (2) the ways such articulatory variations interact with speech acoustic properties.

Sponsored by: Department of Justice & MIT Lincoln Lab

Collaborators: MIT Lincoln Lab

Short Description: We propose to achieve the Babel program goals with a three-pronged approach: we will develop language-independent architectures and systems for Reliable Keyword Search; we will push the boundaries on Learning with Limited Labeled Training Data; and we will greatly enhance the Robustness to Noise, Channel, and Speaker Variability of the underlying speech recognition systems for keyword search.

Sponsored by: IBM Machine Learning

Short Description: In this new phase of the research program, we outline a research strategy that approaches the investigation of the control and coordination of speech initiation and cessation within a dynamical systems model of speech production.

Sponsored by: National Institute on Deafness and Other Communication Disorders (NIH NIDCD)

Short Description: Automatic Speech Recognition (ASR) can currently achieve very high performance given the right conditions: low noise, appropriately sized training data with the desired speaker population, and special hardware, such as wearable, directional microphones and push to talk buttons. Maintaining high performance while relaxing each of these conditions is ultimately necessary to achieve the full potential of this technology. Achieving robustness of performance in such systems hence is a major research goal to develop broadly usable speech technology systems especially in unconstrained interaction settings.

Sponsored by: Office of Naval Research (ONR)

Collaborators: BAE Systems

Short Description: Vocal expression of emotions is an integral part of human speech communication. Investigating how emotional information is encoded in speech is hence important from a variety of viewpoints both scientific (in advancing our understanding of human speech generation) and technological (in creating computational tools for the analysis, recognition and synthesis of emotional speech).

Sponsored by: National Institute of Health (NIH): NHLBI

Short Description: The goal of this proposal is to transform the current practice of observational behavior analysis methodologies by enabling a computational framework for the analysis and modeling of emotionally-rich human interactions through signal processing and machine learning technologies. Using two large existing audio visual corpora of real couple interactions with rich psychologist-specified annotations (from 134 couples and 130 families), we propose to design and test algorithms and tools for automated behavioral coding.

Sponsored by: National Institute of Health (NIH): NHLBI

Short Description: This research will develop multi-modal biometric fusion and supporting technologies to significantly increase the accuracy and usability of mobile handheld devices in a tactical environment. To be fully effective, mobile devices must be much faster, simpler to operate, and more consistent in acquiring quality biometrics relative to the current generation of systems.

Sponsored by: Army Research Lab

Collaborators: Cogent Systems

Short Description: The Transonics team will achieve the ground-breaking power of computer-mediated crosslanguage communication by applying our in depth understanding of multilingual speech recognition in acoustically challenging environments, speech-based machine translation, embedded systems technologies, and human dialogue and communication.

Sponsored by: Defense Advanced Research Projects Agency (DARPA) & BBN Technologies

Short Description: Robotics is poised to enter many arenas of human everyday use with the potential of positively impacting quality of life, especially for growing populations with special needs. However, research in the field of human-robot interaction (HRI) and the new field of socially assistive robotics (SAR), the study of robotic systems capable of providing help through social rather than physical interaction, is currently limited by a lack of richly interactive robotic testbeds suitable for human-subjects experimental evaluation.

Sponsored by: National Science Foundation (NSF)

Short Description: Robotics is poised to enter many arenas of human everyday use with the potential of positively impacting quality of life, especially for growing populations with special needs. However, research in the field of human-robot interaction (HRI) and the new field of socially assistive robotics (SAR), the study of robotic systems capable of providing help through social rather than physical interaction, is currently limited by a lack of richly interactive robotic testbeds suitable for human-subjects experimental evaluation.

Sponsored by: National Science Foundation (NSF)

Sponsored by: National Science Foundation (NSF – ITR)

Sponsored by: National Science Foundation (NSF)

Short Description: How do singers produce different sounds with different emotions? New technology tools, such as fast magnetic resonance imaging, combined with novel computational capabilities, such as statistical machine learning, offer ways for gaining insights into, and measuring and modeling, these processes in ways that were not possible before.

Sponsored by: National Science Foundation (NSF)

Short Description: We introduce a radically new, unified approach to speech-to-text which replaces the HMM at all levels yet is firmly grounded in a powerful statistical machine learning framework. It is motivated by insights into human speech perception, specifically recent studies of human word recognition in everyday conversations. These indicate that large improvements in speech-to-text under diverse conditions cannot be obtained by changing only isolated system components.

Sponsored by: Office of Naval Research (ONR) & University of Washington

Sponsored by: United States Army (ARO)

Sponsored by: United States Army (ARO)

Sponsored by: United States Army (ARO)

Short Description: The long-term goal of this project is to wed state-of-the-art technology for imaging the vocal tract with a linguistically informed analysis of the vocal tract constriction actions in order to understand the production and cognitive control of the compositional action units of spoken language. Our vision has been to develop and use real time MRI to illuminate the inherently dynamic speech production processes.

Sponsored by: National Institute on Deafness and Other Communication Disorders (NIH NIDCD)

Short Description: The overarching goal is to create, implement, and validate individualized engineering systems for enabling evidence driven health care and management. These systems will empower the health care experts with just-in-time information so they can plan and implement individualized intervention. We will target three distinct stakeholders: children, practitioners, and the community.

Sponsored by: Qualcomm & National Institutes of Health (NIH)

Short Description: The complex messages and emotions of spoken language must be communicated by the precise choreography of the jaw, tongue, lips, larynx, and respiratory system. The long term objective of the proposed research program is to understand how linguistic structure conditions the spatiotemporal realization of articulatory movement during speaking.

Sponsored by: National Institute on Deafness and Other Communication Disorders (NIH NIDCD)

Short Description: Conversational interfaces hold the promise of providing natural, easy and universal access to information. The research effort of this project targets two specific problems in conversational engineering under a unifying stochastic modeling framework: (1) dialog interaction modeling (2) user modeling in spoken dialog.

Sponsored by: National Science Foundation (NSF)

Short Description: The proposed research will study creativity in improvisation in both standard theatrical techniques where a script’s interpretation, including the physical performance, is improvised by an actor, and "improv theatre” where entire scenes are created by actors in real-time through improvisation. This research will increase the state of knowledge about improvisation, creativity, and intelligent agent design, as well as contribute meaningfully to theoretical and academic understanding of creative practice in theatre.

Sponsored by: National Science Foundation (NSF)

Short Description: The proposed study is expected to produce results that will enable future in-depth intervention studies of robot and computer-assisted therapies for communication and social skills for children with autism.

Sponsored by: Autism Speaks Foundation

Sponsored by: Department of Homeland Security

Sponsored by: Defense Advanced Research Projects Agency (DARPA) & BAE Systems

Sponsored by: United States Army (ARO)

Short Description: This project utilizes theories of health communication and relationship-oriented social marketing strategies to develop and field-test a prototype for distilling scientific articles on ASD into consumer-friendly audience-defined Science Briefs and to effectively disseminate these informational products to Latino families.

Sponsored by: National Institute on Mental Health (NIH NIMH) & American Recovery and Reinvestment Act (ARRA)

Short Description: The proposed research involves development of methods for content-based indexing of music databases using a combination of signal processing and knowledge-based methods, design of statistical algorithms for enabling queries using sung or hummed melodies, and design of robust search techniques for retrieving the queried information especially in the presence of uncertainty.

Sponsored by: National Science Foundation(NSF)-ITR

Short Description: The objectives of this project are (1) to develop computer-based training systems for rapid acquisition of mission-oriented communication skills, targeted at multiple languages and missions, and (2) to develop a toolset that permits the rapid construction of new training systems.

Sponsored by: Defense Advanced Research Projects Agency (DARPA) & Office of Naval Research (ONR)

Short Description: Our ultimate goal is to enable immersive conversations, where people can freely interact with one another and with the virtual agents without regard to the physical constraints of where they are. Through creation and implementation of robust speech recognition technology, we propose to advance the usabilityand usefulness of military training simulation environments.

Sponsored by: United States Army & STRICOM

Short Description: The goal is to establish a highly interdisciplinary research program focused on Communication, Technology, and Children to explore realistic child-machine interactions, with the aim of developing and improving spoken language and multimedia technologies for children. Specific initial goals aim to design, collect, and analyze pilot experimental data of young children interacting with machines using natural modalities of speech and gestures, as well as more traditional modalities (e.g., mouse, keyboard, & joystick).

Sponsored by: Zumberge Interdisciplinary Grant

Short Description: his project aims to make significant progress in effective voice synthesis for virtual humans, building on current TTS technology. Our goal is high-quality expressive speech, i.e., speech that is intended to covey or evoke a particular emotion and affect.

Sponsored by: United States Army & STRICOM