Contour Extraction of Skin Tumors Using Visual Attention and GVF-Snake Model

doi:10.4236/eng.2013.510B099

Paper Menu >>

Journal Menu >>

Engineering, 2013, 5, 482-486

http://dx.doi.org/10.4236/eng.2013.510B099 Published Online October 2013 (http://www.scirp.org/journal/eng)

Contour Extraction of Skin Tumors Using Visual

Attention and GVF-Snake Model*

Li Ma, Tianzhen Su

1College of Life Information & Instrument Engineering, Hangzhou Dianzi University, Hangzhou, China

2School of Automation, Hangzhou Dianzi University, Hangzhou, China

Email: [email protected], [email protected].cn

Received 2013

Abstract

Contour extraction of skin tumors accu rately is an important task for further feature generation of their borders and sur-

faces to early diagnose melanomas. An integrated approach, combining visual attention model and GVF-snake, is pro-

posed in the paper to provide a general framework for locating tumor boundaries in case of noise and boundaries with

large concavity. For any skin image, the visual attention model is implemented to locate the Region of Interests (ROIs)

based on saliency maps. Then an algorithm called GVF-snake is utilized to iteratively drive an initial contour, deriving

from the extracted ROIs, towards real boundary of skin tumors by minimizing an energy function. It is shown from ex-

periments that the proposed approach exceeds in two aspects compared with other contour-deforming methods: 1) ini-

tial contours generated from saliency maps are definitely located at neighboring regions of real boundaries of skin tu-

mors to speed up converges of contour deformation and achieve higher accuracy; 2) the method is not sensitive to nois-

es on skins and initial contours extracted.

Keywords: Visual A ttention; GVF-Snake; Contour Extraction; Skin Tumors

1. Introduction

Melanoma is known as one of the most malignant skin

tumors probably appearing on any parts of human bodies.

Clinical studies show that early diagnosis can effectively

improve the cure rate. In CAD systems, identifying and

extracting ski n tumors accurately is the important stage

as the further ABCD diagnosing rules [1], A(asymmetry),

B (border irregularity), C (color) and D (diameter), are

highly depended on the extracted tumor contours or ROIs.

Extractions of skin contours, especially for melanomas,

are considered as a challenging t ask as they appearing

variety in color surfaces and geometrical irregularity

along the tumor contours. There are many research works

in literatures on contour extractions for skin tumors.

Among them, a popular approach is the active contour

models called Snakes [2] based on contour evolutions

towards the optimal positions by minimizing its energy.

The disadvantages of the Snakes algorithm are: 1) the

initial contour should be provided manually; 2) defor-

mable contours might not reach the object boundaries in

case of weaker gradients in images. To overcome the

influences of weaker gradients, an algorithm of GVF-

snake [3] is presented by introducing a novel external

force, Gradient Vector F low, to generate a distributed

force field driven deformed contour towards object

boundaries with large concavity. Unfortunately the initial

curve is still needed by users for either GVF -snake or

ASM [4] and converges of deformed contours are sensi-

tive to initial curves. There are some modified versions

on initial contour generation but no general framework

for reliable format ion of initial contours [5,6].

In recent years, visual attention model becomes a hot

research field in video monitoring systems to identify

interested objects from visual information of environ-

ments. It is a process of selecting information based on

saliency in images (bottom-up) and on task-related prior

knowledge (up-down). The advantage of the model is

that it could drive attention to focus on candidate objects

even only in bottom-up module (data-driven). The moti-

vation of the pap er is to b ui ld a general model for

automatically extracting contours of skin tumors with ro-

bustness and accuracy. In this paper an integrated

approach of visual attention model and GVF-snake are

proposed to perform automatic boundary extraction of

skin tumors. The proposed algorithm is illustrated in

Figure 1 where dashed block indicates sub-task of the

system. It is shown in Figure 1 that t here are three sub-

blocks in the proposed algorithm for the contour extrac-

tion. The first one refers to visual attention based ROI

*This work is supported by National Scientific Foundation of China

(60775016).

L. MA, T. Z. SU

483

Input image

Gaussian pyramids

Feature contrast maps

Conspicuity maps

Saliency maps & ROI

Generation

of GVF

Normalization

of GVF

Edge

detection

Contour evolution

under GVF-Snake

Tumor boundary

ROI extraction using

visual attention

GVF-Snake force

field module

Initial Snake curve

Convergence process

Figure 1. The proposed method for contour extraction of

skin tumors.

extraction, and then GVF field is calculated to present

driven force map of the input image. Finally initial

contour from the extracted ROI is iteratively evolved

using GVF-snake model to locate the actual boundaries

of skin tumors based on the criteria of energy minimi-

zation.

2. Bottom-Up Saliency Map Formulation

2.1. Visual Attention Mechanism

The model of bottom-up salient region selectio n pre-

sented by Itti-Koch is based on center-surround mechan-

ism in bottom-up way [7]. Firstly, the input image is

processed progressively by Gaussian low-pass filter and

then sub-sampled to generate the pyramids of multi-fea-

ture maps with nine spatial scales, where Layer 0 is an

original image and the length and width in next layer is

reduced by half than the previous layer. Each feature,

such as intensity, color and orientation in general, is then

computed by a set of linear “center-surround” operations

akin to visual receptive fields. Center-surround is im-

plemented in the model as the difference between fine

and coarse scales: The center is a pixel at scale

{ }

2,3, 4c=

, and the surround is the correspond ing pixel

at scale

scd= +

, with

{ }

3, 4d∈

. So the feature con-

trast map, the region should be paid attention to, is gen-

erated by interpolation to the f iner scale and point-by-

point subtraction between center and surrounding layers.

2.2. Saliency Maps and ROI Generation

For images of skin tumors, color and intensity are key

visual features in human vision while orientations are

less-relevant to identifications of skin tumors. In particu-

lar, 6 for intensity features and 12 for color features from

both RG and BY channel of color differences [7]. After

the 18 feature contrast maps are formed, the normaliza-

tion is taken for each feature contrast maps. Then feature

maps in either color and intensity sub-group are com-

bined into two conspicuity maps in scale four:

for

intensity (Equation (1)) and

for color (Equation (2)).

( )

c sc

II cs

== +

=⊕⊕Ν

(1)

( )

c sc

CRGc sBYc s

== +



=⊕ ⊕Ν+Ν



(2)

where “

⊕

” is defined as across-scale addition. It is im-

plemented by reduction of each map to scale fou r and

point-by-point addition.

As for the final saliency map

, it is ineffective using

conventional method based on the linear combination of

the two conspicuity maps as malignant tumors with rough

surfaces in intensity space and variety in color space are

quite different from common objects with even surfaces.

So a nonlinear operator is proposed in the paper for the

final fusion of saliency map for objects with uneven sur-

faces. The saliency value at each pixel

(, )xy

is com-

puted as:

( )

(,)min(,,,,)SxyI xyCxy=ΝΝ

(3)

The generation of saliency map based on visual atten-

tion is presented in Figure 2, where Fig ur e 2(a). is an

original image of a melanoma with uneven color distri-

butions and irregular boundary. The conspicuity maps,

darker region referring to higher attention area, for both

intensity and color are represented in Figu re 2(b) and

Figure 2(c) respectively. Fig ur e 2(d) is the final saliency

map using Equation (3). It is noted from the figure that

color saliency map wo rks better than intensity one.

At last, ROIs would be further extracted from saliency

map. The procedure of ROI extraction is described as

following:

1) Threshold the saliency map to get a binary image.

(a) (b)

Figure 2. Formation of the saliency map of a melanoma. (a)

Original image; (b) Color channel; (c) Intensity channel; (d)

The saliency map.

L. MA, T. Z. SU

484

2) Take the opening operation to remove some noises.

3) If there are several ROIs, then the ROI candidates

are marked and the one with the largest area is chosen to

be the final ROI.

3. Contour Evolution by GVF-Snake

Snakes [2], also active contours, firstly proposed by Kass

are deformable curves within an image domain that can

move under the influences of internal forces, coming

from and within the curve itself, and external forces

computed from the image data. The curve evolution un-

der both internal and external forces is mathematically

described as a process of energy minimization in Equa-

tion (4).

( )

int

snake ext

ext

EE vsEvsds

= +

∫

(4)

where

snake

refers to a deformable curve energy,

int

represents internal forces and

ext

corresponds to ex-

ternal forces.

3.1. Gradient Vector Flow Snake

In the traditional Snakes,

ext

is computed by the nega-

tive gradient of the input image (

( )

ext

EI xy=−∇

Although it is simple and easy to be implemented, there

are two key drawbacks both on setting an initial curve

close to the true boundary and hardly converging to

boundary concavity. In order to solve these difficulties,

GVF [8] method is proposed by Xu Chenyang and Prince

in 1997. In this model, a new version of external forces

called gradient vector flow (GVF) fields is presented.

These fields are dense vector fields derived from images

by solving a pair of decoupled linear partial differential

equations which diffuses the gradient vectors of a binary

edge map of the image to minimize an energy func tion.

1) GVF formation. For any gray-scale image

( )

,I xy

its GVF is defined as

( )( )( )

,,, ,V xyuxyvxy=



where

( )

,xy

represen ts the coordinate. Then an energy func-

tion

is defined from the GVF field and the binary

edge map

( )

,f xy∇

( )

,I xy

( )

2222

xyxy

uuvvfVf dxdy

εµ

→

=++++ ∇−∇

∫∫

(5)

where

is a regularization parameter governing the

tradeoff between the first and the second term.

GVF field could be calculated by solving the Euler

equations of Equation (5) to minimize the energy func-

tion.

( )

2 22

xx y

yx y

uuu ufff

vvv vfff

=+∇ −−+



=+∇ −−+





(6)

where

222

dudu

udd

∇= +

and

∇

is the Laplacian Op-

erator,

should be set according to the amount of

noise present in the image (more noise, higher

2) GVF-snake. The active contour that us es the GVF

field as its external force is named as a GVF-Snake. It

effectively expands the scope and intensity of the exte r-

nal force. As for GVF -s nake considered, external force

term

ext

E−∇

in the energy functional

(,)

E st

of Snake

[2] (Equation (7)) is replaced with

( )

,V xy

( )( )( )

'' ''''

,,,

t ext

E stx stxstE

αβ

= −−∇

(7)

where

[, ]

x xy=

is the ith pixel on the current contour.

The contour evolution is iteratively performed using Eq-

uation (8) by minimizing the Equation (7).

( )

1 11

tt tt

xAIxuxy

yAIyvxy

γγ

−

− −−

−

− −−

=+−



=+−





(8)

where, the matrix

is a pentadiagonal banded

matrix related to parameter

αβ

in Equation (7) . As

mentioned above, the curve profile could move into the

concave boundary regions because there are stronger ex-

ternal forces but still needs setting initial curves.

3.2. Contour Extraction of Skin Tumor s Based

on Visual Attention and GVF-Snake

As mentioned above, GVF-snake provides the solution

for one of major defects in conventional snake with re-

spect to contour concavity, but its initial contour is still

given by users. In this way the final curves may not be

guaranteed to reach the actual boundaries of objects es-

pecially when initial contours are located at the weaker

regions of GVF fields. However, visual attention based

ROI extraction would make contribution to initial con-

tour generation automatically and ensure the contour

derived from saliency map to locate at neighboring re-

gions of real object boundary. The proposed algorith m of

combination of visual attention and GVF-Snake for skin

contour extractions is described below:

Step 1. Visua l a ttention is used for the input image to

obtain the ROI of a skin tumor from a saliency map (see

the Section 2).

Step 2. The GVF field of input image is generated using

the Equation (6) where edge gradient

( )

, [,]

f xyff∇=

can be calculated from the gray scale image

( )

,I xy

Step 3. The initial contour

( )

s xxx=

is ex-

tracted from the contour of ROI obtained in step1.

Step 4. The initial contour is iteratively evolved based

on Equation (8) whic h moves the deformable contour

towards the real contour of skin tumor in the image. Ite-

ration isn’t stopped until the most of points on the con-

tour are not change. Then the final result of contour ex-

L. MA, T. Z. SU

485

traction is obtained.

4. Experimental Results

To verify the effectiveness of our proposed method for

skin contour extraction, sample images were all down-

loaded from the Dermnet Skin Disease Image Atlas da-

tabase at website http://www.dermnet.com. Each image

was of size 300 × 200 pixels and working platform is

matlab 7.11 on Windows 7. The proposed method in this

paper is compared with other approaches (GVF-snake

and ASM).

4.1. Experiments of Contour Extraction for Skin

Tumors

Experiments are taken on three aspects: sensitivity on

initial contour, robustne ss from noises and accuracy

measure. Only melanomas are chosen from the database

as accurate extraction of their boundaries is more di ff i -

cult than moles.

In Figure 3 comparison a mong severa l methods is

presented for a given image of a melanoma wh ere curve

marked with dashed thin line (“x···x”) in white indicates

the initial contour and the one marked with “—” in red

represents the final bo und ary of the skin tumor. The

contour extracted on the proposed algorithm is given in

Figure 3(a) where the initial curve generated fro m visual

attention is located in the internal region of the tumor and

the final contour reaches the real boundary of the tumor.

The results of other methods, ASM and GVF-snake, are

shown in Figure 3(b), Figure 3(c) and 3(d) respectively.

As for ASM method, the initial contour in Figure 3(b)

derived from the averaged boundary in database of skin

tumors and it is noted that the final contour fails to reach

the real boundary as the range of contour deforming is

limited by its model parameters. The GVF-snake is

(a) (b)

Figure 3. Comparison results of contour extraction on dif-

ferenrt mothods, (a) proposed method; (b) ASM algorithm;

sensitive to the initial contour setting when given two

different initial contours (see Fig ures 3(c) and (d)).

The experiment in Figure 4 is for testing and compar-

ing methods above when noise exists. In Figure 4, there

are some hairs around a skin tumor. It is shown that the

final converged contour (Figure 4(a)) is located at actual

positions of the tumor boundary using proposed method

while either ASM (Figure 4(b)) method or GVF-snake

(Figures 4(c) and (d)) have poor results. It is the hairs

(noise) that interfere with moving direction of ASM and

GVF field fo r contour deformations. It is known from

experiments that the proposed approach (visual attention

& GVF-snake) performs better than the other two. It is

ensured that the initial contours from visual attention

model are located near boundaries of skin tumors as the

region of boundaries with strong feature contrast would

have larger values of saliency.

4.2. Accuracy Measurement

Accuracy of contour extractions is an important index for

algorithm evaluation. Apart from conventional accuracy

measure, another accuracy measure is given in the paper

using binary logi c s .

( )

1 100%

AM M

mAM

⊕

=−×

(9)

where

is the accuracy index,

( )

A⋅

means calculat-

ing the area of a region.

is the region surrounded

by an actual contour marked by users and

repre-

sents the region surrounded by the final contour extracted.

⊕

compl e t e the operatio n of XOR.

The experimental results are given in Table 1 where

accuracy and executing time are compared. Ten sample

images are selected randomly from sub-database of me-

lanomas. There are three methods, VA& GVF-snake,

GVF-snake and ASM used in the experiments. For the

(a) (b)

Figure 4. Comparison results for contour extraction on

different mothods. (a) Proposed method; (b) ASM algo-

rithm; (c) GVF-snake; (d) GVF-snake.

L. MA, T. Z. SU

486

Table 1. Performance comparisons.

Method

Accuracy VA & GVF-Snake GVF-Snake ASM

Average 91.145% 74.322% 55.441%

Minimum 86.197% 20.615% 36.35%

Executing time 3.523s 4.095s 34.472s

accuracy measure in Equation(9), two iterms, both aver-

aged value of

and its minimum in the testing group

are presented.

It is shown in Table 1 that: 1) the best performances

on extraction accuracy are achieved by the proposed

method in this paper; 2) the executing time of the VA &

GVF-Snake is the least among the three methods. And it

is worst for ASM on both accuracy and executing time.

5. Conclusion

Model based contour extraction of skin tumor faces dif-

ficulties in initial contour setting as it may be located at

weaker gradient fields and failed to reach real boundary

of a tumor. The proposed method combines visual atten-

tion model with GVF-snake to provide a solution for the

problems above. The visual attention, using feature con-

trast based saliency maps, generate an in itial contour n ear

the ideal bounda ry. Then it is further moved by GVF-

snake to secure converges to real border of a tumor. It is

indicated from the experiments that the proposed method

in this paper has three advantages over ASM model and

GVF-snake (both are sensitive to initial contours): 1)

initial contours are guaranteed to be near real boundaries

to make the process of curve evolution faster with higher

accuracy; 2) experimental results are not sensitive to

noises; 3) In itial c ontour is generated automatically. The

method is suitable for contour extractions of objects

which have large concavity or noises exist in the back-

ground of images.

References

[1] I. Maglogiannis, S. Pavlopoulos and D. Koutsouris, “An

Integrated Computer Supported Acquisition, Handling

and Characterization System for Pigmented Skin Lesions

in Dermatological Images,” Information Technology in

Biomedicine, Vol. 9, No. 1, 2005, pp. 86-98.

[2] M. Kass, A. Witkin and D. Terzopoulos, “Snakes: Active

Contour Models,” International Journal of Computer Vi-

sion, Vol. 1, 1998, pp. 321-331.

http://dx.doi.org/10.1007/BF00133570

[3] C. Xu and J. L. Prince, “Snakes, Shapes and Gradient

Vector Flow,” IEEE Transactions on Image Processing,

Vol. 7, 1998, pp. 359-369.

http://dx.doi.org/10.1109/83.661186

[4] T. Cootes, “An Introduction to Active Shape Models,”

Image Processing and Analysis, 2000, pp. 223-248.

[5] C.-C. Liu, C.-Y. Tsai, T.-S. Tsui and S.-S. Yu, “An Im-

proved GVF Snake Based Breast Region Extrapolation

Scheme for Digital Mammograms,” Expert Systems with

Applications, 2011, pp. 1-6.

[6] B. R. Wu, M. Xie, G. Li and J. J. Gao, “Medical Image

Segmentation Based on GVF Snake Model,” IEEE Con-

ference on Second International Intelligent Computation

Technology and Automation, Vol. 1, 2009, pp. 637-640.

[7] L. Itti, C. Koch and E. Niebur, “A Model for Saliency-

Based Visual Attention for Rapid Scene Analysis,” IEEE

Transactions on Pattern Analysis and Machine Intelli-

gence, Vol. 20, No. 11, 1998, pp. 1254-1259.

[8] J. L. Prince and C. Xu, “A New External Force Model for

Snakes,” Proceedings of the IEEE Computer Society Con-

ference on Computer Vision and Pattern Recognition,

1997, pp. 66-71.