A Study of Pansharpened Images Based on the HSI Transformation Approach

doi:10.4236/jsea.2012.512B031

Paper Menu >>

Journal Menu >>

Journal of Software Engineering and Applications, 2012, 5, 163-166

doi:10.4236 /jsea.2 012.512b031 Published Online December 2012 (http://www.SciRP.org/journal/jsea)

163

A Study of Pansharpened Images Based on the HSI

Transformation App roac h

Yoshihiro Mitani1*, Yoshi hiko Hamamoto2

1Department of Intelligent System Engineering, Ube National College of Technology, Ube, Japan; 2Faculty of Engineering, Yama-

guchi University, Ub e, Japan.

Email: *[email protected]

Received 2012

ABSTRACT

A pan-sharpen technique artificially produces a high-resol ut i o n i mage b y i ma ge fu si on t echniq ue s us in g hig h -resolution

panchromatic and low-resolution multispectral images. Thus, the appearance of the color image can improve. In this

paper, the effectiveness of three pan-sharpening methods based on the HSI transform approach is investigated. Three

models are the hexcone, double hexcones, and Haydn’s approach. Furthermore, the e f fec t of s moo t hi ng t he low- resol u-

tion multispectral image is also investigated. The smoothing techniques are the Gaussian filter and the bilateral filter.

The experime ntal res ults sho w that Haydn’s mode l is super ior to o thers. The effective ness o f smoothi ng the lo w- reso-

lution multisp e c tral image is also shown.

Keywords: Multispectral Image; Pan-Sharpening; HSI Transformation; Smoothing Techniques

1. Introduction

Recently, there are many Earth observation satellites,

which usually provide both high-resol ution p a nchro mat ic

and lo w-resolution multispectral images. In remote sens-

ing, it is important to acquire high-resolution images. A

pan-sharpen technique artificially produces a high-reso-

lution image by fusing a high-resolution panchromatic

image and a low-resolution multispectral image [1]. Fig-

ure 1 ill ustrate s the outli ne of gener ating a pans harpened

image. By this pansharpening technique, the appearance

of the color image can improve. The pansharpening me-

thod is not limited to only a remote sensing image. If

there are both high-resolution panchromatic and low-

resolution multispectral images, the pansharpening ap-

proach is an adaptive technique for every image. There-

fore , t he fu ndament al stud y o f a p ans ha r pe nin g te c h nique

is believed to c ontribute to the develop ment of the image

processing techniques. J. Zhang reviews current tech-

niques of multi -source remote sensing data fusion [2]. In

[2], a considerable amount of effort has been devoted to

summarize the data fusion techniques of remote sensing

data. Moreover, the evaluation of pansharpening fusion

methods is reported [3]. However, there is little study of

a pan-sharpening method of HSI transformation models

[4,5]. Therefore, in this paper, the effectiveness of three

pan-sharpening methods based on the HSI transformation

approach is investigated. Three models are the hexcone,

double hexcones, and Haydn’s methods [6]. From the

experimental results, the HSI transformation approach

based on the Haydn's model works well. Furthermore,

the effectiveness of smoothing the low-resolution mul-

tispectral image before the HSI transformation is also

investigated. The smoothing methods are the use of the

Gaussian [4] and bilateral filters [7]. From the experi-

mental results, the pansharpened image is shown to be

improved by using the smoothing techniques of Gaussian

and bilatera l filters.

2. Pansharpened Images

A color is known to be expressed by various color fea-

ture models [4,6]. In this paper, we used the HSI family

of color model for making pansharpened images. The

HSI model is a color appearance system. It consis ts of hue,

saturation, and lightness (or intensity). The model based

on intensity, hue, and saturation is considered to be better

suited for huma n inter actio n. As t he a dva nta ge o f t he use

of the H SI mode l, ea ch o f hue , sa tur ati o n, a nd intensit y is

considered to be independent. In the studies of the pan-

sharpening method, the HSI transformation approach is

usually used. Figure 2 shows a flow of pan-sharpening

by the HSI transformation. We describe the pansharpened

images based on the HSI transformation approach as

follows: Suppose that there exist both the low-resolution

RGB color image and the high-resolution panchromatic

image. Firstly, each RGB element of the low-resolution

*Corresponding author.

A Study of Pansharpened Images Based on the HSI Transformation Approach

164

color image transforms into the HSI element, hue, saturation,

and intensity. Secondly, the gray value at the high-reso l u -

tion panchromatic image is replaced by the intensity

obtained from the HSI transformation at the low-resolution

color image. Finally, the pseudo high-resolution RGB

color image is generated by inverse transformation of the

HSI transformation. In this paper, we examine the

pansharpened images generated by three representative

HSI transformations. Three models are the hexcone,

double hexcones, and Haydn’s models [6]. Note that the

intensities (I) of the hexcone, double hexcones, and

Haydn’s models are defined by the brightness as I =

max{R,G,B}, I = (max{R,G,B}+min{R,G,B})/2, a nd I = R

+ G + B, respectively. The details of the HSI trans-

formation and inverse transformation are shown in the

image processing handbook [6].

Furthermore, in order to improve the pansharpened

images, we consider applying the smoothing of the low-

resolution multispectral image. The smoothing methods

are the use of the Gaussian and bilateral filters. The

smoothing technique is expected to use not only one at-

tention pixel but also the color information of its neigh-

bouring pixels. The Gaussia n filter is defined as follows:

( )( )

n wwm ww

1 mn

fim, jnexp()

2πσ2σ

gi, j1 mn

exp( )

2πσ2σ

=−=−

++ −

−

∑∑

(1)

where g(i,j) and f(i,j) denote the intensity of after and

before transformation at (i,j) coordinates, respectively.

The parameter

is the spatial variance. The effect of

smoothing is determined by a value of σ. The image gets

more blurred as the value of σ is getting larger. On the

other hand, if the value of σ is small, the image is con-

sidered to get sharpen.

The bilateral filter [7] is considered to smooth the im-

age, while preserving the edge information of the image.

In smoothing an image, the bilateral filter takes into ac-

count of not only pixel difference but also intensity dif-

ference. The bilateral filter [7] is defined as Equation (2).

Here, the parameters

and

are the spatial and

intensity variance. The larger the value 2

σ or

is,

the more an image gets smoothed. Otherwise, the image

gets sharpened. The role of

is the same as the 2

in the Gaussian filter. The effect of smoothing is deter-

mined by va lues of 1

σ and 2

σ.

( )

( )( )()

( )

( )()

( )

n wwm ww

fi, jfim,jn

fim, jnexp()exp()

2σ2σ

gi, jfi, jfim,jn

exp( )exp()

2σ2σ

=−=−

−++

++ −−

−++

−−

∑∑

(2)

Figure 1 . Outl ine of generati ng a pansha rpened image.

Figure 2. Flow of pan-sharpening by the HS I tra nsformation.

A Study of Pansharpened Images Based on the HSI Transformation Approach

165

3. Experi m ents

In the e xperi ments, bo th the low-resolution color and the

high -resolution panchromatic images are generated from

an original image. Then, we use 3 types of a ratio. Fig-

ure 3 shows a ratio of the image resolution between the

multispectral and panchromatic images. It indicates that

one pixel of the multispectral image corresponds to two

by two pixels of the panchromatic image. This shows a

ratio 1:2. In the exper iment, we change the ratio: 1:2, 1:3,

and 1:4. It is fundamentally important to investigate the

influence s of the ratio on the pansharpened images. For

these images, we obtain the pansharpened images using

three types of HSI transformation techniques. The effect-

tive ness o f pans harpe ned i mages i s exa mined in ter ms o f

the RMSE value. The RMSE value is one of the effective

measures of the image quality performance. The RMSE

value is defined by

() ()

j 0Y1

i0X 122

RMSE( RrGg(Bb))

3XY

= −

=−+ −+−

∑∑

(3)

where the RGB elements of the original image are R, G,

and B. On the other hand, the RGB elements of the pan-

sharpened image denote r, g, and b . The small er val ue of

the RMSE means that the pan-sharpening method works

better.

Table 1 is the RMSE values of three types of HSI

transformation approach for each of a ratio. From Table

1, the Haydn's model gives a smaller RMSE value at any

i mages and a ratio. The results also show that the double

hexcones type is usually superior to the hexcone one.

From the experimental result, we recommend to use the

Haydn’s model.

Figure 3 . A ratio of the imag e resoluti on between the multispectral and panchromatic images.

Table 1. RMSE values of th ree types of HSI t ransformation approach for ea ch of a ratio.

Ratio 1:2 1:3 1:4

Method Hexcone Double

Hexcone Haydn Hexcon e Double

Hexcone Haydn Hexcone Double

Hexcone H aydn

Aerial 16.68 7.49 6.87 16.87 8.72 8.47 17.02 9.28 9.26

Lenna 42.37 16. 80 8.75 42.52 1 7. 36 10.09 42.66 17.27 10.08

Mandrill 30.61 15.72 10.53 30.95 16.42 11.92 31.13 16.86 12.76

Table 2. RMSE values of the Gaussian, bilateral, and without filters using the Haydn's model for e ach of a r atio.

Ratio

1:2

1:3

1:4

Filter Gaussian Bilateral Without

filter

Gaussian Bilateral Without

filter

Gaussian Bilateral Without

filter

Aerial 6.64 6.64 6.87 7.90 7.86 8.47 8.78 8.78 9.26

Lenna 8.70 8.69 8.75 9.31 9.30 10.09 9.80 9.80 10.08

Mandrill 10.27 10.27 10. 53 11.30 11.29 11. 92 12.12 12.11 12.76

Table 3. Optimal parameter value which gives the minimum RMSE value.

Ratio

1:2

1:3

1:4

Filter Gaussian B i l at eral Gauss ian Bil at er al Gaus si an Bil ater al

Aerial 0.6 0.6, 80 0.8 0.8, 200 1.2 1.2, 200

Lenna 0.4 0.6, 100 0.8 0.8, 200 0.8 1.2, 300

Mandrill 0.6 0.6, 200 0.8 0.8, 300 1.2 1.8, 200

A Study of Pansharpened Images Based on the HSI Transformation Approach

166

From the results of Table 1, we focus on improving

the Ha ydn’s model. The effects of the parameter value σ

of the Gaussian filter a re investigated while changing the

values from 0.2 to 2.0. On the other hand, for the bilat-

eral filter, the value of 1

σ varies from 0.2 to 2.0, and

that of the

is from 10 to 400. The size of the Gaus-

sian and bilateral filter is 3 x 3. Table 2 is the RMSE

values of the Gaussian, bilateral, and witho ut filters using

the Ha ydn’s model for each of a ratio. The results of the

Gaussian and bilateral filters show the minimum RMSE

value s. T he n, Table 3 s ho ws t he op t imal parameter value

which gives the minimum RMSE value. From Table 2,

the results of the Gaussian and bilateral filters outper-

form that of without filter at any images and a ratio . Fur-

thermore, the RMSE value of the bilateral filter is equal

to or slightly superior to that of the Gaussian filter. By

using the Gaussian or bilateral filters, the pansharpened

image can improve. This shows the effectiveness of

smoothing the low-resolution multispectral image. From

Table 3, the optimal parameter values of σ for Gaussian

filter and 1

σ for bilateral filter are comparatively very

small. This means that the low-resolution multispectral

image may be getting sharpen. And, its image quality is

considered to improve for making pansharpened images

by appropriately adjusting the parameter values.

4. Conclusion

In this paper, we have examined the effectiveness of

pansharpened images using three types of HSI transfor-

mation approach. Experimental results show that the

Haydn's model is a promising method in the limited

study. And the use of the Gaussian or bilateral filters for

the low-resolution multispectral image is shown to be

effective. Therefore, the Haydn’s model with image

smoo th in g te ch niques s uch as Ga us sia n o r b ilater al filters

for the low-resolution multispectral image should be used.

In the future study, the effectiveness of other panshar-

pening methods [3 ] will be investigate d.

REFERENCES

[1] Y. Zhang and G. Hong, “An IHS and Wavelet Integrated

Approach to Improve Pan-Sharpening Visual Quality of

Natural Colour IKONOS and QuickBird Images,” Info r-

mation Fusion, 6, 20 05, pp. 22 5-234.

doi:10.1016/j.inffus.2004.06.009

[2] J. Zhang, “Multi-Source Remote Sensing Data Fusion:

Status and Trends,” International Journal of Image and

Data Fusion, Vol. 1, N o. 1, 2010, pp. 5-24.

doi:10.1080/19479830903561035

[3] M. Ehlers, S. Klonus, P. J. Astrand and P. Rosso, “Mul-

ti-Sensor Image Fusion for Pansharpening in Remote

Sensing,” International Journal of Image and Data Fu-

sion, Vol. 1, No. 1, 20 10, pp. 25-45.

doi:10.1080/19479830903561985

[4] K. N. Plataniotis and A. N. Venetsanopoulos, “Color

Image Processing and Applications,” Springer, 2000.

[5] Y. Mitani and Y. Hamamoto, “A Consideration of

Pan-Sharpen Images by HSI Transformation Approach,”

Proceedings of the Society of Instrument and Control En-

gineers Annual Conference, 2010, pp. 1283 -1284.

[6] M. Takagi and H. Shimoda, “Handbook of Image Analy-

sis,” Revised Edition, University of Tokyo Press, 2004.

[7] C. To masi and Y. M anduchi, “Bilateral F ilterin g for Gray

and Color Images,” Proceedings of the 1998 IEEE Inter-

national Conference on Computer Vision, 1998, pp.

839-846.