Improved seam merging for spatial and temporal video retargeting

05 Apr 2015
Total votes: 0

Degree:

In the image resizing process, preserving structure over content is important to obtain good results. For this task, Mashiba and Ikehara proposed the seam merging algorithm, a content and structure-aware method based on the seam carving image resizing technique. We'd like to introduce a method to extend it for structure-aware video resizing.
In order to achieve this we replace the dynamic programming method used in the original seam merging method with graph cut algorithm; in this formulation, a seam is given as a solution of the mincut-maxflow problem. Starting from this, we improve results with videos, using a vector-tracking based energy function.
Furthermore we describe a method to adapt the algorithm we developed to resize video temporally rather than spatially, in order to remove empty scenes from it, based on the motion vector density. Rather than removing columns of pixels from any frame of the video, we merge parts of frame in which there is less motion.
We apply our technique to both images and videos, showing its effectiveness for both spatial and temporal resizing.
Thesis: 
References: 

[1] Kazu Mishiba and Masaaki Ikehara. Improved seam merging for content- aware image resizin; IEICE Trans. inf. & syst., 2013.
[2] Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. ACM Trans. Graph, 2007.
[3] Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min- cut/max-flow algorithms for energy minimization in vision; IEEE Transactions on PAMI, 2004.
[4] S. Avidan M. Rubinstein, A. Shamir. Improved seam carving for video retargeting. ACM SIGGRAPH conference proceedings, 2008.
[5] Kazu Mishiba and Masaaki Ikehara. Seam merging for image resizing with structure preservation, 2011
[6] G. Farnebäck. Two-frame motion estimation based on polynomial expansion, 2005.