Ziel
Video is a rich source of information. It provides temporal and spatial information about scenes. However, this information is implicitly buried inside the raw video data, and is very inefficiently organized. While the standard sequential frame-based representation of video data is adequate for viewing in a "movie mode", it fails to support rapid access to information of interest that is required in many emerging applications. VIBES proposes new content-based representations of video data, which explicitly emphasize the geometric, photometric, and dynamic components of information. In particular, VIBES will develop methods for rapid video search, hyper-linking, re-animation and view synthesis, which will enhance the content and enrich the experience of video sequences. Video is a rich source of information. It provides temporal and spatial information about scenes. However, this information is implicitly buried inside the raw video data, and is very inefficiently organized. While the standard sequential frame-based representation of video data is adequate for viewing in a "movie mode", it fails to support rapid access to information of interest that is required in many emerging applications. VIBES proposes new content-based representations of video data, which explicitly emphasize the geometric, photometric, and dynamic components of information. In particular, VIBES will develop methods for rapid video search, hyper-linking, re-animation and view synthesis, which will enhance the content and enrich the experience of video sequences.
OBJECTIVES
Video provides continuous coverage of scenes over an extended region both in time and in space. That is what makes it more than a plain collection of images. In VIBES, our objective is to make video a first class data type, which can be searched on content, annotated, hyper-linked, and edited much as text can be now. Furthermore, video has many more modes of information than simple text. For example, it contains scene geometry and extended actions over multiple frames.
Our objectives are also to extract and use these "modes". With these aims, VIBES proposes new ways of exploring and using video that have the potential of leading to significant breakthroughs in video consumption and new industrial, commercial, and home entertainment applications. The tools we develop will enable cut detection, indexing, synthesis, and classification of non-static and non-rigid scenes.
DESCRIPTION OF WORK
The project contains eight interlinked work packages investigating two main themes:
1. Rapid browsing and retrieval: A video or a DVD will be automatically augmented with hyperlinks connecting shots containing a particular actor, type of action, or scene. E.g. all scenes inside the casino in "Casablanca". Such facilities will change the way in which video is addressed, significantly reducing the tedium and inefficiency of current serial video browsing.
2. 3D scene synthesis and human animation models: 3D scene geometry for virtual reality environments will be automatically generated for particular shots. E.g. the yellow brick road in the "Wizard of Oz" could be reconstructed, and a viewer could then walk down it using VR together with virtual actors. 3D dynamical models of actors from classic movies will be learned and used to generate new scenes involving the actors -synthetic thespians- or to replace one actor by another. For example, replace the "hero" in Home Alone or Toy Story by a texture mapped dynamical model of your son or daughter.
Wissenschaftliches Gebiet
Programm/Programme
Thema/Themen
Aufforderung zur Vorschlagseinreichung
Data not availableFinanzierungsplan
CSC - Cost-sharing contractsKoordinator
100 44 STOCKHOLM
Schweden