Music in Media

Investigating and analyzing music’s place in film, television, and advertisement

Project Summary

Music is an integral part of enjoyment and consumption of media. This project looks at how music is used in film, television, and advertisements. By analyzing how music is used in these contexts, we can better understand how music is used as directive, driving further inquiry into human affective experiences and music’s broad-reaching applications in multi-modal contexts.

Video demonstration of the project. Sample clip shown from Parasite (2020) movie.

Music’s Use in Film

Music plays a pivotal role in film and media. Music played during ads, movies, and television shows gives narrative cues and suggestions to the audience. Different styles of music can also convey different affect, and movies of a particular genre can contain “cross-genre” scenes

Music and Visual Features

By combining the visual information contained in media (such as brightness and contrast) with the audio information contained in the same media (like musical features and dialogue), we can better capture what this media is trying to convey and impart on its audience.

Recent News

06/28/2021 – Our paper on music genre classification was accepted for presentation at Content Based Multimedia Indexing (CBMI)! The paper’s title, “Loss Function Approaches for Multi-label Music Tagging” is available starting today.

03/31/2021 – Our paper on movie genre classification using music in movies was accepted for publication at PLOSOne! The paper’s title, “A computational lens into how music characterizes genre in film,” is now available online.

03/18/2021 – We await to hear word on our work involving movie genre classification using musical signatures. If accepted, our work will appear in PLOSOne journal.

02/01/2021 – We await to hear word on our work involving music genre and mood classification. If accepted, our work will appear in Content Based Multimedia Indexing (CBMI).

11/25/2020 – Our team won an international music tagging challenge called MediaEval, where we classified the musical genre and mood of 55,000 songs better than competitors from all over the world. It is a great honor to have our work be recognized! The code that we used to train and run our best performing model is here.

Selected Publications

Highlights from Studying Music in Media

Best-performing prediction model’s f1-score: .65

It is possible to use the musical cues of a movie to accurately predict a movie’s genre(s)

Tonal Features

Were helpful in predicting movies with higher accuracy, especially horror films

6/7

Film genres could be better predicted by using both musical features and visual cues