Image resize by solving a sparse linear system

From AIRWiki
Jump to: navigation, search

Part 1: project profile

Project name

Content-aware Image Resize.

Project short description

This project presents a methodology for a two stages content aware image resizing. The first stage computes the saliency matrix which will be used in the second stage to relocate all pixels in a smaller image size.

Dates

Start date: 2008/01/01

End date: 2008/09/06

People involved

Project head(s)
Students currently working on the project
  • Alessio C. Bonsignore
  • Stefano Anelli

Part 2: project description

State of the art

This project is based on the article Non-homogeneous Content-driven Video-retargeting, Lior Wolf, Moshe Guttmann, Daniel Cohen-Or The School of Computer Science, Tel-Aviv University, 2007, in which the author describes content aware video resize using a linear system to compute the mapping function.

The problem

The normal way of resizing a picture doesn’t take in account it’s content and differs from the resizing of a web page where each object is modified on the basis of its content. So, there is the need for an image resizing that is content aware.

The Idea

The idea behind content aware resizing is to treat in a different way the important regions of the image respect to those regions with less information. To achieve this goal it’s possible to analyze the source image building a saliency matrix based on gradient magnitude, features detection and eventually face detection (not included in the project). After that a sparce linear system is set up using several different constraits which force pixels to keep a proper position in a smaller size image. The system solution gives the new position of each pixel that better fit the image keeping as much information as possible from the source.

The Project

All these algorithms were implemented in Matlab and tested on several pictures (source code is available by emailing the project head [Pierluigi Taddei] ).

The project comprehend two versions of the liquidResize() function. One for both width and height resize, and the other only for width. Moreover, the first modifies the saliency matrix to take into account the features detected. The second insted does the same thing using more constraits in the linear system.

Report

The report about this project is available here: Report

Conclusion

From results emerge that several image can be successfully resized as the one in the image below, but many situations are still troublesome. Straight lines for example will easily be bent. Moreover, if there's too much information in the image it will be resized uniformly just like a classic image resized techinque.

The main issue is the computational time. To get a 50% width resize of a small 300x225 image (as the kitty example) you need almost two minutes. And to resize both width and height you have to double that time.

Mathlab screenshot of the final result