Given a simple polygon with vertices ordered in counterclockwise order, for any three consecutive vertices, , define a turn: as: . We say that is a left turn if is positive, a right turn if is negative, and not a turn otherwise.

Proposition 1 Let a chain be a sequence of connected, nonintersecting line segments. Given two points and, such that the line segment does not intersect any line segment of, then and are said to be mutually visible with respect to chain.

Proposition 2 Let be a simple chain such that and are visible with respect to. If a point is exterior to the region enclosed by and collinear with and, then the region

enclosed by is not visible from. The proof appears in [1] .

Figure 1(b) shows an example of a point exterior to a chain. The point is collinear with and, and therefore, cannot see the region enclosed by. A line segment is

said to be a window of with respect to the region enclosed by. This region is said to be a hidden region with respect to. Note, that if is visible from then so is. We denote the window, of, as a left hand window (LHW) if is a left turn, and as a right hand window (RHW) if is a right turn. Looking back at Figure 1, we can see that is a left turn. Let be a viewpoint inside and a reflex vertex on the boundary of. We classify reflex vertices as active or inactive according to the possibility of each creating an area occluded from. In order to determine whether a reflex vertex is active or inactive with respect to a given viewpoint, we extend the adjacent edges of into the interior of until they hit the boundary to obtain back diagonals and. If and are mutually visible and lies outside the angle, then is said to be an active reflex point, otherwise is inactive. Given a viewpoint and an active reflex point, define a ray from in the direction of and passing through. Let be the set of intersection points of the ray with the boundary of, ordered according to the distance from. Denote the intersection point closest to as a reflection point. Figure 3(a) shows the reflection point (the first intersection point) with respect to a viewpoint and an active reflex vertex.

Consider a pair of vertices and on the boundary of, such that is a reflex vertex and is visible from. Define a diagonal as a line segment directed from vertex to vertex. A back diagonal with respect to is a line segment joining the vertex and the point, where is the nearest point on the edge of visible in the reverse direction of. The point is said to be a reflection point on the boundary of visible from along a ray directed from through. Figure 4 shows a back diagonal created by a reflex vertex and a vertex visible from.

Figure 5 shows an example of all back diagonals (dotted lines) induced by the reflex vertices. The introduction of all back diagonals induces a back diagonal partition (BDP) of. Such a partition is a planar graph defining a set of internal regions. Note that five back diagonals are induced by vertex -one for each of the vertices in visible from.

Proposition 3 A BDP of partitions into a collection of regions, each a convex sub polygon of.

Proposition 4 A back diagonal is an edge in a BDP such that viewpoints in the neighborhood on either side of it have different visible and area polygonal functional expressions. Thus, each region in a BDP has a unique visible and area polygonal functional expression.

The above propositions will be useful in construction an algorithm for the maximal VAP problem.

Note that the gray areas in Figure 3 are hidden regions with respect to the viewpoints. We define back edges for two cases. Case 1: If is an interior point of an edge of, then is said to be a reflection point of on through the active reflex vertex, and is called a back edge of through. The edge in Figure 3 is the back edge of the viewpoint through the reflex vertex. Case 2: If coincides with a vertex of, then the back edge is taken as. In this case, and induce a back diagonal on which lies. For ease of exposition, all of the following will be in terms of case 1. Note, that case 1 corresponds to a point that does not lie on a back diagonal with respect to. A left hand hidden region with respect to, a left hand active reflex vertex, a reflection point and its back edge, is a region bounded by the polygon defined by the circular vertex list:

where, and are the original vertices of in CCW order. A right hand hidden region is similarly defined for a right hand reflex vertex as. Figure 3(b) provides an illustration of and for a given observation point.

We can now define a visible polygon in terms of the definitions given above. Given a polygon and an observation point, define the visible polygon as the polygon comprised of subsequences of the original vertices of and all windows with respect to. The visible polygon contains all visible vertices and the reflection points of all active reflex vertices with respect to. From proposition 2 it should be clear that the regions enclosed by and are not visible from.

Define a rectangular coordinate system in such that each point in is represented by a set of ordered real valued coordinates. Given the coordinates of an observation point, the end points of back edge, and an active reflex point, the reflection point can be calculated as a function of and has the general form:

Here, are constants in terms of the points and. The hidden area as a function of (the reflection point) has the following form:

Here, are constants comprised of the coordinates of the original boundary vertices of. This is a direct result of applying Equation (2) to the boundary vertices,

of the left and right hidden regions, respectively. Given a point in, associate with it a set of active reflex vertices. Let and represent the number of left and right reflex vertices with respect to. Each active reflex vertex has an associated hidden region. Let represent a left/ right hidden regions of associated with an active reflex vertex, respectively.

The collection of left and right hand hidden regions associated with are denoted as; and, respectively. Since intersections of hidden regions are empty (except for possible vertex points), the area of the visible polygon may be represented as follows:

This area function will be derived in terms of the rectangular coordinates of point, in order to determine the directional move of, such as to increase the visible area. For example, Figure 6 shows a polygon, with a, and a visible polygon .

Let represent the area of the visible polygon associated with the view point and a left hand hidden region. Using and isolating the coordinates we get the area in terms of:

The only relevant vertices are the reflex vertex (which created the hidden region) and which is the visible vertex of the edge, the back edge of through. Note that is the intersection point between the line collinear with and. Repeating the process for a right hand side region, where is the visible vertex of the back edge we obtain:

The individual coordinates of can be convex or concave, increasing or decreasing or constant in and depending on the orientation of the ray and the back edge.

Since and depend on the view point, we must rephrase the area’s formulation using its coordinates. To do this two lines are constructed: (connecting and) and (collinear with the back edge) (see Figure 5). The intersection of and is the point. The right side of (9) contains functions of.

The total gradient is the sum of the partials for all active reflex vertices with respect to.

where, and are all active left and right reflex vertices with respect to, respectively.

Although this algorithm is based on movements between sub regions of a BDP, there is no need to actually construct it. It is only necessary to construct the back diagonals (line equations). This is because the algorithm searches for a critical event when the observer crosses a back diagonal. At his point, based on proposition 4, a recalculation of the area and new gradient are necessary.

The Steps of the Procedure

Given a simple polygon, and a point in. Set.

Step 1 Compute the area of as.

Step 2 Determine the visible polygon from and its area. Identify all “active” reflex vertices and the corresponding set of windows:, where is the active reflex vertex, and its reflection point which lies on the interior or endpoint its associated back edge. If, go to Step 7 (Stop).

Step 3 Find the total gradient that maximizes the increase in, using (12) and Proposition 5. If no such gradient exists, stop and go to Step 6.

Step 4 If lies on an original edge of, calculate a new gradient direction so as to slide along the edge in a direction such that the angle between the original gradient direction and the edge is minimal.

Step 5 Move in the gradient direction until one of the following critical events occurs:

a) hits a vertex or edge of.

b) One of the reflection points slides to the endpoint of its back edge (i.e. one of the vertices of).

c) If one of the windows becomes collinear with one of the edges that forms a reflex vertex, i.e.; or for a or window, respectively. This reflex vertex becomes inactive; drop its associated window from.

d) A new reflex vertex becomes active. Add the associated window to.

Step 6 If, compute the new point, and return to Step 2; otherwise, , and the entire polygon is viewable from.

Step 7 Stop. No local improvement is possible. Let be the best point. Compute, the total visible area from.

Note that the events in Step 5 correspond to the intersection of the gradient direction vector and all back diagonals.

Proposition 5 The sequence is increasing in.

Proposition 6 The Gradient Search Algorithm converges in a finite number of steps. This is true since the sequence in Proposition 1 is bounded from above by the area of.

Proposition 7 The gradient search algorithm has a time complexity.

Proof. In Step 2, the initial polygon can be calculated in time. This also identifies all active reflex vertices (bounded by). Finding will requires at most calculations. In Step 3, the movement in the direction of is performed until hitting a boundary edge or back diagonal. This requires calculations to find all intersections between the gradient directional line and all back diagonal lines. Steps 2 and 3 are repeated at most times in moving from back diagonal to back diagonal (resulting in at most points between and as back diagonals are not revisited. Thus, the total time complexity of the gradient search algo-

rithm is.

The example in Figure 7 provides a demonstration of the algorithm for the case of a simple polygon with three reflex vertices, one of them, remaining inactive throughout. The sequence of points and visible areas are, and.

Given a simple polygon, let be a collection of polygons.

1) Initially set, remove from and set it to the current polygon.

2) Start with a random point in. Call the Max VAP gradient search algorithm to find the point, and.

3) Let be the set of simple polygons obtained by subtracting the visibility polygon from. (i.e. the parts of the polygon which are not visible from). Add to the collection of polygons.

4) If stop, otherwise;

5) Select a polygon in and remove it, and return to Step 2.

The greedy heuristic described above terminates after a finite number of steps. The complexity of the greedy heuristic for solving the guardhouse problem is. There are at most visits to Step 2 each requiring a call to the max VAP algorithm. Thus, the time complexity of the greedy heuristic is.

Figure 8(a) shows a sample polygon containing 16 vertices. We start with point shown in (b) and improve it until reaching a local maximum. The visible area is then removed from the initial polygon, and two new sub polygons appear in (c). In (d) and (e), this process is repeated with the two parts left, resulting in the final solution (f) in which the entire polygon is covered by 4 observation points.