Yoonsoo Nam, Adam Lehavi, Daniel Yang, Digbalay Bose, Swabha Swayamdipta, and Shrikanth Narayanan. Does Video Summarization Require Videos? Quantifying the Effectiveness of Language in Video Summarization. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 8396–8400, , April 2024.
Video summarization remains a huge challenge in computer vision due to the size of the input videos to be summarized. We propose an efficient, language-only video summarizer that achieves competitive accuracy with high data efficiency. Using only textual captions obtained via a zero-shot approach, we train a language transformer model and forego image representations. This method allows us to perform filtration amongst the representative text vectors and condense the sequence. With our approach, we gain explainability with natural language that comes easily for human interpretation and textual summaries of the videos. An ablation study that focuses on modality and data compression shows that leveraging text modality only effectively reduces input data processing while retaining comparable results.
@INPROCEEDINGS{10445931,
author={Nam, Yoonsoo and Lehavi, Adam and Yang, Daniel and Bose, Digbalay and Swayamdipta, Swabha and Narayanan, Shrikanth},
booktitle={ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
title={Does Video Summarization Require Videos? Quantifying the Effectiveness of Language in Video Summarization},
year={2024},
volume={},
number={},
pages={8396-8400},
abstract={Video summarization remains a huge challenge in computer vision due to the size of the input videos to be summarized. We propose an efficient, language-only video summarizer that achieves competitive accuracy with high data efficiency. Using only textual captions obtained via a zero-shot approach, we train a language transformer model and forego image representations. This method allows us to perform filtration amongst the representative text vectors and condense the sequence. With our approach, we gain explainability with natural language that comes easily for human interpretation and textual summaries of the videos. An ablation study that focuses on modality and data compression shows that leveraging text modality only effectively reduces input data processing while retaining comparable results.},
keywords={Natural languages;Signal processing;Image representation;Transformers;Data models;Vectors;Speech processing;Video Summarization;Multimodal Transformers;Data Compression},
doi={10.1109/ICASSP48485.2024.10445931},
ISSN={2379-190X},
link = {https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10445931},
month={April},}
Generated by bib2html.pl (written by Patrick Riley ) on Tue Jun 10, 2025 07:11:57