Computer Vision for Automated Video Editing and Understanding. (Talk)

Video content creation has boomed in recent years. Every day hundreds of thousands of video hours are uploaded to the internet. Thus, video content editing has become more popular and accessible to amateur users. However, current Computer Vision (CV) techniques have not studied technologies to help video editing become a less tedious task. Currently, editors spend hours cutting and stitching videos to deliver final edited videos that convey stories. This cutting process is creative but is often repetitive. With the recent advances in CV, one would expect that a system could learn some cutting patterns and help the editors to speed up their jobs. Although there have been advances from the Human-Computer Interaction community to create better user experiences when performing cuts, there are no approaches from the CV side. We propose to study the automated video editing and understanding problem from the CV perspective. In this talk, I will present two works that study how to make cuts in video editing more automated. The first work presents a label-agnostic approach driven by data that enables learning editing patterns by leveraging edited content found on the web. The second one shows a taxonomy-based method in which we propose to semantically understand the cuts by creating a new dataset that follows the taxonomy editors follow. Finally, I will talk about recent works using creative content for other types of tasks in video analysis and research.