Rotoscoping and Video Segmentation

Rotoscoping, the detailed delineation of scene elements through a video shot, is a painstaking task of tremendous importance in professional post-production pipelines. While pixel-wise segmentation techniques can help for this task, professional rotoscoping tools rely on parametric curves that offer the artists a much better interactive control on the definition, editing and manipulation of the segments of interest.
Sticking to this prevalent rotoscoping paradigm, we propose a novel framework to capture and track the visual aspect of an arbitrary object in a scene, given a first closed outline of this object. This model combines a collection of local foreground/background appearance models spread along the outline, a global appearance model of the enclosed object and a set of distinctive foreground landmarks. The structure of this rich appearance model allows simple initialization, efficient iterative optimization with exact minimization at each step, and on-line adaptation in videos. We demonstrate qualitatively and quantitatively the merit of this framework through comparisons with tools based on either dynamic segmentation with a closed curve or pixel-wise binary labelling.




Publications

ROAM: a Rich Object Appearance Model with Application to Rotoscoping
Miksik O,*, Perez-Rua J-M.*, Torr P.H.S and Perez P.
In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2017, Honolulu, USA
* Joint first authors
PDF | Code | Show BibTex | Show Details