In general, the super resolution
method focuses on the problem of recovering the low resolution to high
resolution image. That is using input image and super-resolution algorithms to
predict high resolution image, and the output can be represent the downscale
version of the high-resolution image:
represents a downscale operator. Since super-resolution is an ill-posed
problem, we need some powerful prior knowledge to
constrain it. Because given a , for calculate the , it has infinite way.
Our algorithm focuses on the problem of recovering the low resolution facial image
to high resolution facial image, and rely on the external example based method
to solve it. Using the method of predicting high-resolution patches, and the
predicted image patches are finally integrated into .
3.1 Local structure prior
Super-resolution is a
typical ill-posed problem in image processing. Therefore, the problem need to be solved under some
constraints and the strength of the constraint will affect the quality of the
target high-resolution image. In solving this problem of front human face image
resolution, the similarity of the front human face image can be used to
generate the constraint.
Through a series of image
processing steps to normalize, equalize, eye alignment and crop the front face
images, we can ensure that for any , can find
and at the same position has same features for all
3.2 Data generation
First, for the generate the
input image . To get a more blurred input image, we select
an image (Ground truth) and using the Nearest Neighbor method to down scale 4
times, and then using the Nearest Neighbor method upscale 2 times (In general,
the super resolution methods are used BICUBIC downscale 2 times). The reason is that
the general downscale 2 times, the results of the prediction image and the
general interpolation algorithm, there is little difference in effect, so it is
difficult to show the performance of its algorithm. For generating high frequency image and high
frequency patch , where – and = – . A two-dimensional image can be decomposed into different frequency
components. The low-frequency components describe a wide range of information,
while the high – frequency components describe the specific details. It’s also
for this reason that we want to get target high-resolution image by predicting
high-frequency image patches .
Therefore, we can establish
the relationship through the and F to restore the , and each F will change according to the patch
For input image that are not pre-processed by the eye alignment, the
result will have artifacts, as shown in Fig 2. The reason is all used database
images are collated by eye alignment, so we need do the same preprocessing of
the input image. The Fig 3 shows the we used database which collated by eye
use Viola-Jones face detection algorithm 29,
which can help us find out the location of the eye. For the Viola-Jones face
detection algorithm, the basic principle is to learn a classifier through
training set, and then slide it in the test image with different scale windows.
Make a classification of each scan to determine whether the current window is
the target to be detected. We can according to the coordinates of the eyes part
to crop image size to get the input image. This part can be summarized in
Algorithm 1, and Fig. 4 shows the detail process.
For the F, we can
represent it by a liner mapping function in a linear
where A will also
change according to the patch location, in other words, each group of patches ( and its corresponded and ) has a different A?so we can express it by a locally weighted
regression as follows:
weight w, we introduce the SRLSP’ s
18 smooth weighting:
where i represents the position of
patch and ? is a smoothing parameter, is the squared Euclidean distance between
the and , when
the distance is smaller, w is bigger.
regularization term was added to complete the final equation, Eq. (4) can
be written as the following expression:
= + (5)
? is the regular
which can balance the regular term and , and the
regularization term is the Frobenius norm ().
We can get by
Eq. (5) and Eq. (2), but for getting , we need combine it with :
Finally, a common restoration
algorithm strategy is used to average the overlapping
prediction patches () to obtain .
We have the same regression equation structure with SRLSP. The difference
is that, for predict single patch, in the SRLSP, they extract the pixels of the
fixed position in the predicted high-resolution patch, then to combine it with
the pixels which in the corresponding input patch. Our approach is to get the
high-frequency patch from the regression equation and get the high-resolution
patch in combination with the input patch ().