Research

The CARE group aims to create computational systems that augment clinican's analytical capabilities in diagnosis and personalized treament of neuro-cognitive social disorders, particularly autism spectrum disorder (ASD)-- the fastest growing developmental disability in the United States affecting 1 in 68 children (CDC, 2014). Autism research is a model for translational research of a psychiatric disorder; collaborating psychologists, engineers, and neurologists are translating findings about this complex, heterogeneous social-communicative disorder into mechanisms that will improve the lives of affected individuals. As such, we are proud to collaborate with many of the foremost autism researchers.



Vocal Behavior (speech, non-verbal, and language)

Deficits in social communication are the primary phenotypic markers of ASD. Children with ASD are almost universally delayed in acquiring spoken language, making language a key early indicator for diagnosis. Early communication issues can compound through adolescence and adulthood; however, early intervention is known to greatly improve the lives of individuals with autism. In our lab, we computationally assess the vocalizations and language use of individuals with autism at all stages of development, in order to advance understanding and translate findings for assessment, diagnosis, and intervention. We quantitatively analyze speech-prosody (the rhythm and timing of speech that communicate meaning and affect), non-verbal vocalizations (e.g., fillers, laughters, and sighs), and language (from affective word use to narrative formation) in ASD. Additionally, we propose the novel approach of examining the conversational cues of both child and psychologist, hypothesizing an interactive effect.

People involved: Daniel Bone, Rahul Gupta, Matthew Black, Chi-Chun Jeremy Lee (Alumni), Theodora Chaspari, Nikolaos Malandrakis

Related Funding

Simons Foundation Autism Research Initiative (SFARI) grant Developing Scalable Measures of Behavior Change for ASD Treatments, with Catherine Lord
  • Daniel Bone, Matthew Black, Chi-Chun Lee, Marian Williams, Pat Levitt, Sungbok Lee, Shrikanth S. Narayanan, "The Psychologist as an Interlocutor in Autism Spectrum Disorder Assessment: Insights from a Study of Spontaneous Prosody", Journal of Speech, Language, and Hearing Research, vol. 57, pp. 1162–1177, 2014.
  • Daniel Bone, Chi-Chun Lee, Theodora Chaspari, Matthew P. Black, Marian Williams, Sungbok Lee, Pat Levitt, Shrikanth S. Narayanan, "Acoustic-Prosodic, Turn-taking, and Language Cues in Child-Psychologist Interactions for Varying Social Demand", Proceedings of InterSpeech, 2013.
  • Daniel Bone, Theodora Chaspari, Kartik Audhkhasi, James Gibson, Andreas Tsiartas, Maarten Van Segbroeck, Ming Li, Sungbok Lee, Shrikanth S. Narayanan, "Classifying Language-Related Developmental Disorders from Speech Cues: the Promise and the Potential Confounds", Proceedings of InterSpeech, 2013.
  • Rahul Gupta, Kartik Audhkhasi, Sungbok Lee, Shrikanth S. Narayanan, "Speech paralinguistic event detection using probabilistic timeseries smoothing and masking", Proceedings of InterSpeech, 2013.
  • Theodora Chaspari, Emily Mower Provost, Shrikanth S. Narayanan, "Analyzing the structure of parent-moderated narratives from children with ASD using an entity-based approach", Proceedings of InterSpeech, 2013.
  • Daniel Bone, Matthew P. Black, Chi-Chun Lee, Marian E. Williams, Pat Levitt, Sungbok Lee, Shrikanth S. Narayanan, "Spontaneous-Speech Acoustic-Prosodic Features of Children with Autism and the Interacting Psychologist", Proceedings of InterSpeech, 2012.
  • Rahul Gupta, Chi-Chun Lee, Daniel Bone, Agata Rozga, Sungbok Lee, Shrikanth S. Narayanan, "Acoustical analysis of engagement behavior in children", Proceedings of Workshop on Child, Computer and Interaction (WOCCI 2012), 2012.
  • Theodora Chaspari, Emily Mower Provost, Athanasios Katsamanis, Shrikanth S. Narayanan, "AN ACOUSTIC ANALYSIS OF SHARED ENJOYMENT IN ECA INTERACTIONS OF CHILDREN WITH AUTISM", Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012.
  • Rahul Gupta, Chi-Chun Lee, Shrikanth S. Narayanan, "CLASSIFICATION OF EMOTIONAL CONTENT OF SIGHS IN DYADIC HUMAN INTERACTIONS", Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012.
  • Matthew P. Black, Daniel Bone, Marian E. Williams, Phillip Gorrindo, Pat Levitt, Shrikanth S. Narayanan, "The USC CARE Corpus: Child-Psychologist Interactions of Children with Autism Spectrum Disorders", Proceedings of Interspeech, Florence, Italy, 2011.

Physiological Signals

Despite the neurobiological roots of ASD, assessment and treatment are largely based upon behavioral criteria. Physiological indices, which give a window into a person's internal state, are rarely taken into account. We study the physiological dynamics within and between child, therapist, and parents to find therapeutic mechanisms of change and behavioral outcomes. Specifically, we develop knowledge-driven methods to represent electrodermal activity (EDA, a psychophysiological signal linked to stress, affect, and cognition), its co-evolution across individuals, and its relation to observed behavior. Our results suggest that quantitative measures of EDA can afford us new insights into the nature of symptomatology of individuals with ASD and their interaction with the environment.

People involved: Theodora Chaspari

  • Daniel Bone, Matthew Black, Chi-Chun Lee, Marian Williams, Pat Levitt, Sungbok Lee, Shrikanth S. Narayanan, "The Psychologist as an Interlocutor in Autism Spectrum Disorder Assessment: Insights from a Study of Spontaneous Prosody", Journal of Speech, Language, and Hearing Research, vol. 57, pp. 1162–1177, 2014.
  • Daniel Bone, Chi-Chun Lee, Theodora Chaspari, Matthew P. Black, Marian Williams, Sungbok Lee, Pat Levitt, Shrikanth S. Narayanan, "Acoustic-Prosodic, Turn-taking, and Language Cues in Child-Psychologist Interactions for Varying Social Demand", Proceedings of InterSpeech, 2013.
  • Daniel Bone, Theodora Chaspari, Kartik Audhkhasi, James Gibson, Andreas Tsiartas, Maarten Van Segbroeck, Ming Li, Sungbok Lee, Shrikanth S. Narayanan, "Classifying Language-Related Developmental Disorders from Speech Cues: the Promise and the Potential Confounds", Proceedings of InterSpeech, 2013.
  • Rahul Gupta, Kartik Audhkhasi, Sungbok Lee, Shrikanth S. Narayanan, "Speech paralinguistic event detection using probabilistic timeseries smoothing and masking", Proceedings of InterSpeech, 2013.
  • Theodora Chaspari, Emily Mower Provost, Shrikanth S. Narayanan, "Analyzing the structure of parent-moderated narratives from children with ASD using an entity-based approach", Proceedings of InterSpeech, 2013.
  • Daniel Bone, Matthew P. Black, Chi-Chun Lee, Marian E. Williams, Pat Levitt, Sungbok Lee, Shrikanth S. Narayanan, "Spontaneous-Speech Acoustic-Prosodic Features of Children with Autism and the Interacting Psychologist", Proceedings of InterSpeech, 2012.
  • Rahul Gupta, Chi-Chun Lee, Daniel Bone, Agata Rozga, Sungbok Lee, Shrikanth S. Narayanan, "Acoustical analysis of engagement behavior in children", Proceedings of Workshop on Child, Computer and Interaction (WOCCI 2012), 2012.
  • Theodora Chaspari, Emily Mower Provost, Athanasios Katsamanis, Shrikanth S. Narayanan, "AN ACOUSTIC ANALYSIS OF SHARED ENJOYMENT IN ECA INTERACTIONS OF CHILDREN WITH AUTISM", Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012.
  • Rahul Gupta, Chi-Chun Lee, Shrikanth S. Narayanan, "CLASSIFICATION OF EMOTIONAL CONTENT OF SIGHS IN DYADIC HUMAN INTERACTIONS", Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012.
  • Matthew P. Black, Daniel Bone, Marian E. Williams, Phillip Gorrindo, Pat Levitt, Shrikanth S. Narayanan, "The USC CARE Corpus: Child-Psychologist Interactions of Children with Autism Spectrum Disorders", Proceedings of Interspeech, Florence, Italy, 2011.

Facial Expressions and Gestures

Children with Autism Spectrum Disorder (ASD) are known to have difficulty in producing and perceiving emotional facial expressions. Their expressions are often perceived as atypical or awkward as compared to their typically developing (TD) peers by adult observers; this observation has been supported empiricallyin behavioral science research. Our research focuses on objective computational quantification and analysis of the atypicality in affective facial expressions of children with ASD. Using motion capture and speech data, we explore the subtle differences in affective facial expressions between children with and without ASD, findings that may be difficult to perceive visually. We use methodologies from statistics, signal processing, and dynamic modeling to answer these research questions.
People involved: Tanaya Guha, Anil Ramakrishna, Zhaojun Yang, Angeliki Metallinou (Alumni)

Related Funding

NIH/NIDCD R01 Verbal/non-verbal asynchrony in adolescents with high- functioning Autism, with Ruth Grossman
  • Angeliki Metallinou, Ruth Grossman, Shrikanth S. Narayanan, "Quantifying Atypicality In Affective Facial Expressions Of Children With Autism Spectrum Disorders", Proceedings of the IEEE International Conference on Multimedia & Expo (ICME), 2013.

Interaction Modeling & Affect

Human communication is an intricate activity in which signals are constantly exchanged through multiple modalities (vocal, visual, gestural), from speaker and listener alike (e.g., backchannelling). Thus, it is no surprise that the behavior of an adult interacting with a child will vary depending on the social-communicative skills of the child. We investigate the social-affective exchange between child and psychologist over the duration of an interaction. First, we have found that the prosodic, turn-taking, and language cues of the psychologist alter during ASD assessment depending on the level of social-communicative impairment the child displays. In other words, the psychologist's behavior is predictive of the child's level of impairment. This result provides the first empirical evidence of this intuitive behavior. Second, we have utilized an affective computing tool we previously created to study affective dynamics in these interactions. The tool is simple and robust, and does not require any training. Through our analysis in these data, we have shown, for instance, that children with higher ASD severity tend to lead the social-affective exchange more (likely because they are less responsive to the psychologist's affect, and the psychologist is hyper-responsive to their affect).

People involved: Daniel Bone, Chi-Chun Jeremy Lee (Alumni)
Software Tools: Vocal Arousal Toolkit

  • Emily Mower, Chi-Chun Lee, James Gibson, Theodora Chaspari, Marian Williams, Shrikanth S. Narayanan, "Analyzing the Nature of ECA Interactions in Children with Autism", Proceedings of Interspeech, Florence, Italy, 2011.
  • Emily Mower, Matthew Black, Elisa Flores, Marian Williams, Shrikanth S. Narayanan, "Design of an Emotionally Targeted Interactive Agent for Children with Autism", Proceedings of IEEE International Conference on Multimedia & Expo (ICME), pp. 1-6, 2011.

Collaborations

Due to a shared dedication to improving the lives of individuals with ASD, we have developed rewarding collaborations with top researchers in their individual fields, including: