🙌Of course, nothing would have been possible without the lead of Weihan Xu (wx83.github.io) and the amazing collaboration with Paul Liang, Haven Kim, Julian McAuley and Taylor Berg-Kirkpatrick!
📜Paper: arxiv.org/pdf/2410.05586
🎦Demo: wx83.github.io/TeaserGen_Of...
📽And old movies!
🧑🏫And TED talks!
🎬We tested our models on documentaries of various topics, including environmental, societal and historical issues.
🛠️We proposed two methods for narration-video matching:
1️⃣a pretraining-based model using pretrained contrastive language-vision models and
2️⃣a deep sequential model that learns the mapping between the narrations and visuals.
🛠️We adopted a two-stage approach for generating a documentary teaser:
1️⃣first, we generate the teaser narration given the transcribed narration of the documentary;
2️⃣then, we select the relevant visual content to accompany the generated narration.
🔥We presented the DocumentaryNet, a collection of 1,200+ high-quality documentaries with their teasers.
🎉Super excited to share that our TeaserGen project led by Weihan Xu (wx83.github.io) has been accepted to #ICLR2025!
🔍We explored a new task of generating teasers for long documentaries.
🤩We presented a new dataset, new models, and new evaluation metrics for teaser generation.
