We find that the effective image structure facilitates the kernel estimation. In previous researches, bilateral filtering and shock filtering are used to predict the image edges [2]. In this paper, we select an image smoothing algorithm based on norm [10] to extract the outstanding structure. -norm can smooth the region which gradient is small. The structure extraction model is defined as:

where is the structure image, is the blurred image, is the coordinate of the image pixel, which can be written as

is the -norm of. It denotes the number of non-zero pixels, and respectively denotes the absolute value of image gradient in the direction of horizontal and vertical coordinates.

In addition, structures which are smaller than the size of kernel will have a negative impact on the kernel estimation. While, inaccurate kernel fail to restore image. So we add ar(x) to the model which showed as model (1).

Formula (3) is first used in [9], is the gradient of the blurred image, is the absolute value, is a window centered at the pixel of. We assume that and respectively express Formula (3) in the direction of horizontal and vertical. Then, we solve model (1) by substituting two auxiliary variables of, therefore, model (1) is transformed to minimizing, shown as Formula (4) and we call it model (4) in the following paragraphs,

The solution of model (4) can be decomposed into two sub problems―and:

The sub problem about:

We can use the Fast Fourier Transforms (FFT) to compute. Based on Paseval theorem, the solving equation can be expressed as:

where and represent the Fast Fourier Transforms and its inverse operation respectively, and denote the gradient filter in direction and direction, is the element-wise multiplication operator, is the conjugate operator.

The sub problem about:

We add and to Formula (7) and use and to replace and respectively. As Formula (8) shows.

According to [10], we can get the result of Formula (8) as following

Proof details can be found in [10]. After obtaining the salient structure of image, we use the shock filter to further enhance the structural information. Figure 2 shows the different results between the model with filter and the model without filter. We can see that the kernel and latent image with used are more satisfying than without used.

In the last few years, a variety of regularizations were used to estimate the kernel, the most commonly used were norm [1] [2] [9] [11] and norm [4] [8]. Although norm can be solved fast, it always produce noise so that it cannot achieve an accurate kernel. Pan [7] employed and norm in estimating kernel, but it did not combine the two regularization in essence, the model of Pan is shown as Formula (10)

where the first submodel is data fidelity term, and is the gradient of the blurred image, is the gradient of the structure image, and are the weight value. Formula (10) is a non-convex function, and it is difficult to be minimized. Pan minimized this function by using iterative reweighed least square (IRLS). IRLS requires a lot of time because of the complexity of computation.

Now, we propose a new kernel estimation method based on norm. In fact, the blur kernel is the path of camera movement, so the kernel is sparse. For the sparsity of the kernel, we utilize norm ensure it. But norm can’t preserve the continuity in general. So norm is used to guarantee the continuity. And our model can be written as Formula (11) and we call it model (11) in the following paragraphs.

In order to solve model (11), we add two auxiliary variables into model (11), model (11) can be rewritten as:

We call it model (12) and the solving of it is same to Section 3.2.

To remove noise, we set the kernel elements with the value smaller than 0.075 of the biggest one to zero. Then, the remaining non-zero values are normalized so that their sum becomes one. The experimental results will be compared in Section 4.

The latent image can be restored via the deconvolution operation of the kernel and blurred image. In order to ensure the details and smoothness of the latent image, as well as suppress ringing artifacts produced in process of deconvolution, we use norm as regularization to restore the latent image, the model can be expressed as Formula (13).

Formula (13) is a nonconvex function, so IRLS can be used to minimize it.

From Section 3, we can see that our method mainly takes advantage of regularization. norm has the excellent effect on ensuring image sparsity on the blur kernel, because kernel is the displacement path generated by the camera shake. This path is sparse in the image. On the blurred image, the degeneration of image details is more serious than the image outline. So it is more effective to using the image structure to estimate the kernel. The image structure is also sparse.

For the image structure extraction and the kernel estimation, we both execute in the gradient image. Because of the pixel value of gradient image are all zero in addition to the outline section. It can improve execution efficiency.

There are several parameters in our algorithm. In model (1), we set, in model (12), we set = 0.1 or 0.5 and. We use MATLAB R2015b to write code. Figure 3 shows our results in comparison with Xu & Jia [9], Pan [7]. Because of Pan use the IRLS (iterative reweighed least square) to solve Formula (10) at the stage of kernel estimation, it is very time-consuming. We use the Fast Fourier Transforms to solve our models, so our algorithm is more efficient. Table 1 shows the time contrast results between Pan [7] and our algorithm. Figure 3 shows the pictures we used and comparison results among Xu & Jia [9], Pan [7] and our algorithm.

The complex building images are challenging for all methods, because the building images contain too many edges, the invalid edges will restrict the image restoration. Figure 4 shows that, our method can result the commendable latent images.

We evaluate our results on the dataset in [5] by comparing the Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) with Xu & Jia [9] and Pan [7]. In Figure 5, PSNR is employed to compare the reconstruction accuracy for the latent image. And Table 2 shows the SSIM results of Xu & Jia [9], Pan [7] and our algorithm. We can see that our accuracy is higher than other methods in most cases.