• Save
Algorithmen zum Verkleinern von digitalen Videos
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
704
On Slideshare
553
From Embeds
151
Number of Embeds
3

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 151

http://www.foerderverein-technische-fakultaet.at 132
http://www.ftf.or.at 17
http://webcache.googleusercontent.com 2

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Algorithmen zum Verkleinern von digitalen Videos Wolfgang Effelsberg Praktische Informatik IV Universität Mannheim Germany
  • 2. Content1. Motivation2. Scaling und Cropping3. Seam Carving for Still Images4. An Improvement for Diagonal Lines5. Seam Carving for Video6. Conclusion and OutlookAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 2 von digitalen Videos 23. Juli 2012
  • 3. 1. Motivation• Mobile devices (smart phones, pad computers) are getting very popular for showing videos.• The representation of still images and videos on those devices requires an adaptation to the screen size and the aspect ratio.• Scaling and Cropping are easy to implement but they do not work very well.• Seam Carving is a promising technology for still images and videos. Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 3 von digitalen Videos 23. Juli 2012
  • 4. 2. Scaling and CroppingScalingScaling reduces the image linearly, without looking at its content.„Letterboxing“ is used if the format does not fit on the screen.Example: Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 4 von digitalen Videos 23. Juli 2012
  • 5. CroppingCropping cuts the image parallel to the edges until the final format is reached,without looking at the image content. Parts of the image can get lost.Example: Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 5 von digitalen Videos 23. Juli 2012
  • 6. 3. Seam Carving for Still ImagesRemoving of specific pixels from the image, based on the importance of thecontentBased on an „energy function“ describing the importance of different parts of thecontent Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 6 von digitalen Videos 23. Juli 2012
  • 7. ExampleReduce the width by 40 %Energy is maximal on edge pixels in the image. original image energy image Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 7 von digitalen Videos 23. Juli 2012
  • 8. Naive ApproachRemove from each line the pixels with minimal energy original image 200 pixels removed Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 8 von digitalen Videos 23. Juli 2012
  • 9. Remove Entire ColumnsRemove the 200 columns with the smallest energy original image 200 columns with minimal energy removed Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 9 von digitalen Videos 23. Juli 2012
  • 10. Definition of a Seam• A vertical seam is an 8-connected path of pixels, running from the top to the bottom of the image and containing exactly one pixel per row.• Horizontal seams are defined analogously. Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 10 von digitalen Videos 23. Juli 2012
  • 11. Advantage of SeamsCan be adapted better to the regions of interest (regions of high energy) Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 11 von digitalen Videos 23. Juli 2012
  • 12. Seams for our Example200 vertical seams with minimal energy removed original image 200 vertical seams removed Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 12 von digitalen Videos 23. Juli 2012
  • 13. Our Other ExampleIt os obvious that seam carving removes unimportant content first.Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 13 von digitalen Videos 23. Juli 2012
  • 14. Energy Functions (1)A simple energy function based on the difference in light intensity between two pixels δ δ e( I ( x, y )) = I ( x, y ) + I ( x, y ) δx δyBetter (but more complicated) energy functions are possible, for example, based onimportant objects found in the image. Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 14 von digitalen Videos 23. Juli 2012
  • 15. Energy Functions (2)We use saliency and face recognition to identify regions of interest δ δ esal + face ( x, y ) = ws ⋅ saliency ( x, y ) + w f ⋅ face( x, y ) + I ( x, y ) + I ( x, y ) δx δy original image saliency map face map seams with esal+face new as energy function image Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 15 von digitalen Videos 23. Juli 2012
  • 16. 4. An Improvement for Diagonal LinesA problem with traditional seam carving: diagonal lines original image width reduced to 40% by seam carvingAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 16 von digitalen Videos 23. Juli 2012
  • 17. The Problem with Diagonal Lines (1)When a seam crosses a diagonal line visible artefacts appear. a diagonal line seams cutting that line seams removed Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 17 von digitalen Videos 23. Juli 2012
  • 18. The Problem with Diagonal Lines (2)Especially critical if several seams cross the line at the same place.Idea: distribute the seams more evenly over the line. Neigboring seams line after removing distribute seams evenly line after removing cutting a line the seams the seamsAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 18 von digitalen Videos 23. Juli 2012
  • 19. Solution for Diagonal Lines (1)Solution: modify the energy function between the computation of the seams: increasethe energy at intersection points seam cutting a line modified energy energy function third seam function and new modified again seam Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 19 von digitalen Videos 23. Juli 2012
  • 20. Solution for Diagonal Lines (2)Result original image traditional seam carving improved seam carvingAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 20 von digitalen Videos 23. Juli 2012
  • 21. 5. Seam Carving for VideoFirst idea: apply seam carving for each frame separately the video gets shaky. original video seam carved for each frame separately Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 21 von digitalen Videos 23. Juli 2012
  • 22. Seam Carving for Video: First IdeaThe video frames define a 3D block over time.We remove 2D “seam surfaces”. The seam pixels of one surface are connected inthe third dimension.We use a graph algorithm (min-cut max-flow) to discover optimal “seam surfaces”.Problem: takes considerable time to compute. time Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 22 von digitalen Videos 23. Juli 2012
  • 23. Seam Carving for Videos: Second IdeaFast seam carving for video• Accumulate all energy values of all frames in the first frame.• Compute the optimal seams there.• Map those seams back to the block of frames. Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 23 von digitalen Videos 23. Juli 2012
  • 24. Example for the Second Idea seams carved separately fast seam carving for each frameAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 24 von digitalen Videos 23. Juli 2012
  • 25. Another Example scaling fast seam carvingAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 25 von digitalen Videos 23. Juli 2012
  • 26. 6. Conclusions and Outlook• Seam Carving is a useful technique to reduce image sizes without losing relevant content.• Finding a good energy function is a real challenge.• Improvements are possible for diagonal lines.• Flicker removal is of critical importance for videos. Fast Seam Carving is a useful algorithm for that problem.• Seam carving does not work well if most of the content is relevant for the viewer.• Seam Carving can be combined with other techniques to create pleasant smal-screen versions of video. Algorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 26 von digitalen Videos 23. Juli 2012
  • 27. Vielen Dank … … für Ihre Aufmerksamkeit! Meine Mailadresse ist effelsberg@informatik.uni-mannheim.deAlgorithmen zum Verkleinern Klagenfurt Wolfgang Effelsberg 27 von digitalen Videos 23. Juli 2012