Speech Recognition and Processing for Multimedia

Spring 2018

Time: Fridays, 4:00pm - 6:50pm
Location: OHE 100C

Instructors: Prof. Shri Narayanan
Office: EEB 430
Office Hours: Wednesday 10:30am - 12:00pm (and upon request)

Teaching Assistant: Manoj Kumar
Office: PHE 320
Office Hours: Wednesday 3:00pm - 5:00pm

Midterm: Friday, March 9, 4pm - 6pm (location TBA)
Final: Friday, May 4, 4:30pm - 6:30pm (location TBA)
Grading Policy: Homeworks: 25%, Midterm: 45%, Final: 30%

Course Materials: Lecture notes, additional reading materials, assignments and grades will be posted in USC DEN. Use your USC email address and password to log in.

Pre-requisites: Digital Signal Processing (EE483 or equivalent). Some knowledge of probability (EE503) desired. Familiarity with MATLAB will be useful for homework assignments.

Reference Texts: Lecture notes will be provided for most topics. In addition,
R&S: Rabiner and Schafer: Theory and Applications of Digital Speech Processing, Prentice Hall, 2010. (Required text).
HAH: Huang, Acero, and Hon: Spoken Language Processing, Prentice Hall, 2001. (Required text).
R&J: Rabiner and Juang: Fundamentals of Speech Recognition, Prentice Hall, 1993.
DHP: Deller, Hansen, and Proakis: Discrete-time Processing of Speech Signals, 2000.
TQ: T. Quatieri: Discrete-time Speech Signal Processing, Prentice Hall, 2001.

Course Timeline (approximate)(Reading from R&S):
Week 1 (Jan. 12): Course Introduction & Speech Production. (Chp 1, 2, 3)
Week 2 (Jan. 19): Acoustic Speech Models & Short-time Time Domain Processing.(Chp 5,6)
Week 3 (Jan. 26): Time Domain Processing & Short-time Fourier Transform (part 1). (Chp 6, 7)
Week 4 (Feb. 2): Short-time Fourier Transform (part 2) & Applications. (Chp 7)
Week 5 (Feb. 9): Linear Prediction (part 1) (Chp 9)
Week 6 (Feb. 16): Linear Prediction (part 2) (Chp 9)
Week 7 (Feb. 23): Homomorphic Signal Processing & Cepstral Analysis. (Chp 8)
Week 8 (Mar. 2: Auditory Models (Chp 4)
Week 9 (Mar. 9): Midterm. 9am - 11am (location TBA)
Week 10 (Mar. 16): Spring Break
Week 11 (Mar. 23): Speech Coding. (Chp 11, 12)
Week 12 (Mar. 30): Automatic Speech Recognition (ASR). (Chp 14) Classic, HMM/GMM systems
Week 13 (Apr. 6): Automatic Speech Recognition (ASR). (Chp 14) Contemporary, incl. end-to-end systems
Week 14 (Apr. 13): Speaker Recognition & Speech Enhancement
Week 15 (Apr. 20): Emotion Recognition, Clinical applications in speech processing & BSP applications
Week 16 (Apr. 27): Speech Synthesis; Course Wrap up. (Chp 13)
Week 17 (May 4): Final, 4:30pm-6:30pm (location TBA)

Homework Schedule:(approximate)
Out Due Topics
HW1 Jan 12 Jan 26 Week 1,2
HW2 Jan 26 Feb 9 Week 3,4
HW3 Feb 9 Feb 23 Week 5,6 (Midterm: Mar 9)
HW4 Mar 16 Mar 30 Week 7-11 (Spring break: Mar 11-18)
HW5 Mar 30 Apr 13 Week 12,13 (Finals: May 4)

Statement for Students with Disabilities
Any student requesting academic accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure the letter is delivered to me (or to TA) as early in the semester as possible. DSP is located in GFS 120. The phone number for DSP is (213) 740-0776.

Statement on Academic Integrity
USC seeks to maintain an optimal learning environment. General principles of academic honesty include the concept of respect for the intellectual property of others, the expectation that individual work will be submitted unless otherwise allowed by an instructor, and the obligations both to protect one's own academic work from misuse by others as well as to avoid using another's work as one's own. All students are expected to understand and abide by these principles. Scampus, the Student Guidebook, contains the Student Conduct Code:
Students will be referred to the Office of Student Judicial Affairs and Community Standards for further review, should there be any suspicion of academic dishonesty. The Review process can be found at:

Emergency Preparedness/Course Continuity in a Crisis
In case of a declared emergency if travel to campus is not feasible, USC executive leadership will announce an electronic way for instructors to teach students in their residence halls or homes using a combination of Blackboard, teleconferencing, and other technologies.