1) Basic idea

YOLO divides the input image into S × S grids. If the center coordinate of the GT (Ground Truth) of an object falls into a grid, the grid is responsible for detecting the object. The innovation of YOLO is that it reforms the Region Proposal detection framework: RCNN series need to generate Region Proposal in which to complete classification and regression. But there is overlap between Region Proposal, which will bring a lot of repetition work. However, YOLO predicts the bbox (bounding box) of the object contained in all grids, the location reliability, as well as the probability vectors of all classes at one time, thus it solves problem one-shot.

2) Network structure

YOLO network borrows Google Net while the difference is that YOLO uses the 1 × 1 convolutional layer (for cross-channel information integration) + 3 × 3 convolutional layer instead of the Inception module simply. YOLO v1 network structure consists of 24 convolution layers and 2 full connection layers, as shown in Figure 1.

Compared with the region proposal based method such as Fast R-CNN, YOLO v1 has a larger positioning error and a lower recall rate. Therefore, the main improvements of YOLO v2 are to enhance the recall rate and positioning ability, and include:

1) BN (Batch Normalization)

BN is a popular training technique since 2015. By adding BN layer after each layer, the entire batch data can be normalized to a space with a mean of 0 and variance of 1, which can prevent the gradient from disappearing as well as gradient explosion, and make network convergence faster.

2) Anchor boxes

In YOLO v1, the full connection layer is used to predict the coordinates of bbox directly after the convolutional layer. YOLO v2 removes the full connection layer by using the idea of Faster R-CNN, and adds Anchor Boxes, which effectively improves the recall rate.

3) Multi-scale training

The input image size for the YOLO v1 training network is fixed, where YOLO v2 adjusts the input image size randomly every 10 epoch during training, so that the model has a good detection effect on the multi-scale input images during the test.

YOLO v3 model is much more complex than YOLO v2, and its detection on small objects, as well as compact dense or highly overlapping objects is very excellent. The main improvements include:

1) Loss

YOLO v3 replaces the Softmax Loss of YOLO v2 with Logistic Loss. When the predicted objects classes are complex, especially when there are many overlapping labels in the dataset, it is more efficient to use Logistic Regression.

2) Anchor

YOLO V3 uses nine anchors instead of the five anchors of YOLO v2, which improves the IoU.

3) Detection

YOLO v2 only uses one detection while YOLO v3 uses three, which greatly improves the detection effect on small objects.

4) Backbone

YOLO v3 replaces darknet-19 network of YOLO v2 with darknet-53 network, which improves the accuracy of object detection by deepening the network.

This paper uses the latest YOLO v3 model to achieve the vehicle detection for aerial image.

The VEDAI (Vehicle Detection in Aerial Imagery) dataset is made by Sebastien Razakarivony and Frederic Jurie of University of Caen [9], whose original material is from the public Utah AGRC database. The raw images have 4 uncompressed color channels (three visible color channels and one near infrared channel). The authors firstly split the original large-field satellite image into 1024 × 1024 pixels JPEG format images, and then create the visible color channels dataset and the near infrared channel dataset, and finally down sample the above two datasets into 512 × 512 pixels, so VEDAI contains 4 subsets. In this paper, only the first subset of VEDAI (1024 × 1024, RGB 3 channels) is used. The shooting heights of all images in VEDAI are the same, and the GSD (Ground Sampling Distance) of 1024 × 1024 image is 12.5 cm pp (cm per pixel). VEDAI contains a total of 1250 images, and is manually annotated nine classes of vehicle (“plane”, “boat”, “camping car”, “car”, “pick-up”, “tractor”, “truck”, “van”, and “other”), a total of 2950 samples. The annotation of each sample includes: sample class, GT’s center point coordinates, direction, and the coordinates of GT’s 4 corners.

COWC (Cars Overhead with Context) dataset is made by T. Nathan Mundhenk and others of Lawrence Livermore National Laboratory [10], whose original materials are from six public websites. The COWC contains a total of 53 pictures in TIFF format, and the image size is between 2000 × 2000 to 19,000 × 19,000 pixels. COWC images have covered six geographic locations, namely Toronto (Canada), Selwyn (New Zealand), Potsdam and Vaihingen (Germany), Columbus and Utah (United States), in which the images of Vaihingen and Columbus are grayscale, while the others are in RGB color. The GSD of the image is 15 cmpp, so the size of vehicle is basically between 24 to 48 pixels. CWOC is manually annotated one class of positive samples (“car”) with a number of 32,716, as well as four classes of negative samples (“boats”, “trailers”, “bushes” and “A/C units”) that are easily confused with the vehicle with a number of 58,247. The annotation of each sample includes: sample class, and GT’s center point coordinates.

DOTA (Dataset for Object detection in Aerial images) is an aerial image dataset made by Xia Guisong of Wuhan University, Bai Xiang of Huazhong University of Science and Technology, and others [11]. In order to eliminate the deviation caused by different sensors, the original material comes from multiple platforms (such as Google Earth). DOTA is characterized by multi-sensor and multi-resolution, namely that the GSDs of the images are diversified. DOTA contains a total of 2806 images about 4000 × 4000 pixels, and is manually annotated 15 classes of sample (“plane”, “ship”, “storage tank”, “baseball diamond”, “tennis court”, “swimming pool”, “ground track field”, “harbor”, “bridge”, “large vehicle”, “small vehicle”, “helicopter”, “roundabout”, “soccer ball field” and “basketball court”) with a number of 188,282.The annotation of each sample includes: sample class, and the coordinates of GT’s 4 corners (where the top left corner is the starting point, arranged in a clockwise order).

The standard dataset for YOLO training mainly consists of two parts: images and labels, where images are JPEG format and labels are txt format documents. Labels and images are in one-to-one correspondence. Each label records annotations of the samples in the corresponding image. The annotation format is:

class GT’s center point coordinates ( x , y ) GT’s width and height ( w , h )

where ( x , y , w , h ) are normalized values, wrap the line to distinguish when there are multiple samples in one image. Since the input dimension of YOLO v3 training network is 416 × 416 × 3, the size of image used for training should not be too large, otherwise the characteristics of the sample after resize may be lost seriously. The basic information of the three public aerial image datasets described in Chapter 4 is shown in Table 1.

We process the above three datasets separately.

1) VEDAI

a) Image size is suitable and do not need to be processed;

b) Delete the annotation of “plane”, “boat”, and “other” three classes in labels;

c) Delete the “direction” in annotations;

d) According to the coordinates of GT’s 4 corners, calculate width and height:

w = x max − x min , h = y max − y min (1)

2) COWC

a) Delete the grayscale images;

b) Delete the annotation of negative samples, leaving only the positive sample “car”;

c) Split the images of COWC into 416 × 416 size and convert to JPEG format. When splitting, the coordinate of the sample center point is converted accordingly to ensure its position in the new image is correct. The remaining images less than 416 × 416 are padded with black.

d) According to the GSD of COWC, it is assumed that the size of vehicle in the image is unified to 48 * 48 pixels, therefore,

w = h = 48 / 416 = 0.115384615384615... (2)

3) DOTA

a) Except for “large vehicle” and “small vehicle”, delete all the annotations of other 13 classes in labels, “large vehicle” and “small vehicle” are unified to “car”;

b) Split the images of DOTA into 1024 × 1024 size. When splitting, the coordinates of GT’s 4 corners are converted accordingly to ensure their positions in the new images are correct. Abandon the remaining imagesless than 1024 × 1024.

c) Center point coordinate:

x = ( x max + x min ) / 2 , y = ( y max + y min ) / 2 (3)

d) Width and height:

w = x max − x min , h = y max − y min (4)

After processing, the information of the new datasets are shown in Table 2.

1) Batch size

Use YOLO v3 default parameter b a t c h _ s i z e = 64 .

2) Number of iterations

The dataset contains a total of 20,542 images, so one epoch needs to iterate: 20542 / 64 ≈ 320 times.

The YOLO training defaults to iterate 160 epochs, so the number of iterations is: 160 × 320 = 51200 times.

3) Learning rate

The initial learning rate is 0.001, after 60 epoch divided by 10, after 90 epoch divided by 10 once again.

4) Number of filters in the last layer of the network

f i l t e r s = ( c l a s s + 5 ) × 3 = ( 6 + 5 ) × 3 = 33