Image resize by solving a sparse linear system
Part 1: project profile
Content-aware Image Resize.
Project short description
This project presents a methodology for a two stages content aware image resizing. The first stage computes the saliency matrix which will be used in the second stage to relocate all pixels in a smaller image size.
Start date: 2008/01/01
End date: 2008/09/06
- Pierluigi Taddei
- Vincenzo Caglioti
Students currently working on the project
- Alessio C. Bonsignore
- Stefano Anelli
Part 2: project description
State of the art
This project is based on the article Non-homogeneous Content-driven Video-retargeting, Lior Wolf, Moshe Guttmann, Daniel Cohen-Or The School of Computer Science, Tel-Aviv University, 2007, in which the author describes content aware video resize using a linear system to compute the mapping function.
The normal way of resizing a picture doesn’t take in account it’s content and differs from the resizing of a web page where each object is modified on the basis of its content. So, there is the need for an image resizing that is content aware.
The idea behind content aware resizing is to treat in a different way the important regions of the image respect to those regions with less information. To achieve this goal it’s possible to analyze the source image building a saliency matrix based on gradient magnitude, features detection and eventually face detection (not included in the project). After that a sparce linear system is set up using several different constraits which force pixels to keep a proper position in a smaller size image. The system solution gives the new position of each pixel that better fit the image keeping as much information as possible from the source.
All these algorithms were implemented in Matlab and tested on several pictures (source code is available here: liquidResize source code ).
The project comprehend two versions of the liquidResize() function. One for both width and height resize, and the other only for width. Moreover, the first modifies the saliency matrix to take into account the features detected. The second insted does the same thing using more constraits in the linear system.
The report about this project is available here: Report
From results emerge that several image can be successfully resized as the one in the image below, but many situations are still troublesome. Straight lines for example will easily be bent. Moreover, if there's too much information in the image it will be resized uniformly just like a classic image resized techinque.
The main issue is the computational time. To get a 50% width resize of a small 300x225 image (as the kitty example) you need almost two minutes. And to resize both width and height you have to double that time.