14 DAY TRIAL // Get ready for a new level of video special effects and interactive processing, courtesy of Microsoft Research and the University of Washington. Just watch the video fragment embedded at the bottom of this article, offered by Miel Van Opstal, Microsoft Product Evangelist, in order to get an idea of the Interactive Video Cutout technology developed by Jue Wang, Pravin Bhat, R. Alex Colburn, Maneesh Agrawala and Michael F. Cohen. Essentially, Interactive Video Cutout deals with offering the user an interactive system designed to allow the extraction of foreground objects from a video segment. And if you want a deeper insight on the project, Microsoft research is also delivering a whitepaper on Interactive Video Cutout for download as a .PDF file.
"We present an interactive system for efficiently extracting foreground objects from a video. We extend previous min-cut based image segmentation techniques to the domain of video with four new contributions. We provide a novel painting-based user interface that allows users to easily indicate the foreground object across space and time. We introduce a hierarchical mean-shift preprocess in order to minimize the number of nodes that min-cut must operate on. Within the min-cut we also define new local cost functions to augment the global costs defined in earlier work. Finally, we extend 2D alpha matting methods designed for images to work with 3D video volumes", reads a fragment of the abstract from the Interactive Video Cutout white paper.
Microsoft's Interactive Video Cutout is designed to span across both time and space in terms of matting, during the extracting of foreground objects. What you have to understand is that the technology does not involve operating on each frame separately. Such an approach would inevitably conduct to a lack of coherence between various segments of video. Interactive Video Cutout is set up to manage large volumes of pixels seamlessly and at the same time, deliver a comprehensive interface for the user. The Redmond company is doing it all with a volumetric painting interface, the hierarchical segmentation algorithm, local color and edge costs and spatio-temporal alpha matting.
"We present a novel interface that allows users to indicate foreground and background regions by painting on surfaces within the spatio-temporal video volume. We introduce a hierarchical mean-shift segmentation preprocess that allows us to solve a global min-cut optimization in interactive time - 10 to 15 seconds for 100 to 200 frame sequences at 640x480 or 720x480 resolution. We define new spatially local color and edge models within a min-cut framework to leverage the advantages video offers. We show how to combine the local models with the global models from previous work. We extend the 2D alpha matting presented in to work with 3D spatiotemporal video objects while preserving both spatial and temporal smoothness of the alpha matte," Microsoft added on Interactive Video Cutout.