Haze is produced by the scattering of fine substances suspended in the atmosphere, which will reduce the quality of the image and affect subsequent analysis. In computer vision and computer graphics, the following equation is widely used to describe the generation of a hazy image [1] [2].

I ( x ) = J ( x ) t ( x ) + A ( 1 − t ( x ) ) (1)

where I is the hazy image, J is the scene radiance representing the haze-free image, A is the atmospheric light, t is the medium transmission, β is the scattering coefficient of the atmosphere and d is the depth of scene, t can be calculated with the following equation.

t ( x ) = e − β d ( x ) Math_5# (2)

Image dehazing can restore the lost details of the image and facilitate in-depth analysis. Therefore, this task has always been paid attention in computer vision and progressing rapidly. Traditional methods in single image dehazing based on physical priors [7] [8] do not require paired data to train, but often have certain requirements on the environment to perform well. Methods based learning [9] [10] require a lot of data for training，because it is difficult to obtain the paired images, the hazy images in the training set are usually artificially synthesized by the atmospheric model. There is a certain deviation between the synthesized hazy images and the real hazy images, which causes some models to perform poorly on the real hazy images.

Domain adaptation aims to reduce the differences between different domains, and it is mainly divided into feature level or pixel level. The feature-level adaptive method aims to align the feature distribution between the source domain and the target domain by minimizing the maximum mean difference (MMD) or applying adversarial learning strategies on the feature space [20]. Another study focused on pixel-level adaptation [21] [22]. These methods deal with the domain transfer problem by applying image-to-image translation.

Image-to-image translation [16] [23] refers to the translation of images from one domain to the other through learning. Image translation of supervised learning requires paired data sets, and the corresponding image translation algorithms include pix2pix [16] model, pix2pixHD [23] model, etc. Unsupervised learning image translation [17] [24] does not require paired datasets.

The overview model structure is shown in Figure 2. It is trained on paired data and real hazy images, paired data contains synthetic hazy images and clear images.

The synthetic hazy images domain and the real hazy image domain are respectively recorded as D X , D R . The clear image domain corresponding to the synthesized fog image domain is denoted as D Y . Synthetic hazy images, real hazy images, and clear images are recorded as x ∈ D X , r ∈ D R , y ∈ D Y . The latent space of x, y, r is recorded as Z x , Z y , Z r . VAE₁ which is trained on x, r includes encoder E D X , E D R and generator G D X , G D R . VAE₂ includes encoder E D Y and decoder G D Y . Inspired by [25], our model first encodes x and r into the latent space, and then adapts their domains; there is a big difference between the latent space Z x and Z y , we use mapping net (denote it as T Z ) to connect them.

The VAE1 losses for the real hazy images (r) are as follows:

L V A E 1 ( r ) = K L ( E D R , D X ( Z r | | N ( 0 , I ) ) ) + α E Z r ~ E D R , D X ( Z r | r ) [ | | G D R , D X ( r D R → D R | Z r ) − r | | 1 ] + L V A E 1 , G A N ( r ) (4)

z r ∈ Z D R is the Hidden code of real hazy images (r), r D R → D R is generated by G D R , KL stands for KL divergence. The second term is L1 loss, in order to reconstruct the image. The third term is the least-square loss [26], to make the reconstructed image realistic.

To adapt the domain from synthetic hazy images to realistic hazy images, we use adversarial loss as follows.

L V A E 1 , G A N l a t e n t ( r , x ) = E x ~ D X [ 1 − D D R , D X ( E D R , D X ( x ) ) 2 ] + E r ~ D R [ D D R , D X ( E D R , D X ( r ) ) 2 ] (5)

So the total loss of VAE1 is as follows.

min E D R , D X , G D R , D X max D D R , D X L V A E 1 ( r ) + L V A E 1 ( x ) + L V A E 1 , G A N l a t e n t ( r , x ) (6)

The mapping net learns the connection between the VAE1 and VAE2, it’s loss is as follows.

L T Z ( x , y ) = δ 1 L T Z + L T Z , G A N + δ 2 L p e r c e p t u a l (7)

where L T Z is stand for the L1 loss between T Z ( Z x ) mapped by T Z and Z y ,

L T Z = E | | T Z ( Z x ) − Z y | | 1 (8)

L T Z , G A N is the least-square loss(LSGAN), L p e r c e p t u a l is perceptual loss, as shown in Equation (9).

L p e r c e p t u a l = E [ ∑ i 1 n D T i | | ϕ D T i ( x D X → D Y ) − ϕ D T i ( y D Y → D Y ) | | 1 ] + ∑ i 1 n V G G i | | ϕ V G G i ( x D X → D Y ) − ϕ V G G i ( y D Y → D Y ) | | 1 ] (9)

where ϕ V G G i ( x ) is the i t h layer feature map of the discriminator (VGG network), and n V G G i indicates the number of activations in that layer. D T is the same as VGG.

The experiment environment uses the deep learning framework Pytorch1.6.0. The graphics card is NVIDIA RTX 3090Ti, the graphics memory is 32 GB.

The 8970 pairs of images selected from the RESIDE‑OTS [3] and 2864 real hazy images selected from RESIDE-beta [29] are used to train our model. We choose some images from HazeRD [4], SOTS (indoor and outdoor) to test the model’s ability on synthesizing hazy images.

We compare the model proposed qualitatively and quantitatively in this part. AuthESI [27], PSNR and SSIM is used to evaluate the quality of images. Density [28] of images is also measured.

As shown in Figure 3, some photos selected from the SOTS are used to synthesize hazy images. It can be seen from the second row of the figure that in some cases, the overall color shift will occur on the hazy images synthesized by Adobe Lightroom CC. And our model doesn’t have this kind of problem and generates hazy images successfully.

Table 1 is a quantitative test of the corresponding image of Figure 3. It can be

seen from the table that the quality of the hazy images synthesized by our model is quite high, and this is in the case of similar hazy images’ density.

As shown in Figure 4, these images are from HazeRD dataset, and some are synthesized by Lightroom CC and our method. It can be seen from the second column of the figure that there are some abnormalities around the plant, because the nearby depth has been wrongly measured, and our model performs well.

The quantitative results of Figure 3 are shown in Table 2. Similarly, our method performs well, indicating that our method also has a certain degree of generalization on various types of images.

To further improve the effective of our model, we use the DCP [7] to dehaze the iamges and measure the PSNR and SSIM. Since the DCP [7] is invalid for the sky, we choose the indoor pictures in Figure 3 for testing. First, test the PSNR and SSIM of the hazy image synthesized by our model, and then use DCP [7] to dehaze and then test PSNR and SSIM again. The results are as shown in Table 3. It can be seen that after DCP [7] processing our synthetic hazy images’ SSIM and PSNR are improved, which proves the effectiveness of our model.

Running our model repeatedly can obtain hazy images with different densities, as shown in Figure 5.