Most satellite images come with a special data format (such as GEOTIFF and HDF [8] [9]), and require special knowledge as well as a geospatial library (such as GDAL [10]) to process it. In order to help accomplish this task without extenal software dependecy, we first converted the original data from GEOTIFF into the common numpy format with the channel first option. The resulting dataset contains 16 images with a core area of 625 × 625 pixels, as well as an additional 92 padding pixels outside of the core box, making the total dimension a 809 × 809 pixel image. Each image contains 4 channels: the cyan, magenta, yellow, and key (black) color maps. Furthermore, binary masks for these images (where 1 represents a building pixel, and 0 represents anything else) were manually created. These images and masks were randomly split into three parts: 12 images/masks grouped as the training dataset, 2 images/masks as the validation dataset, and the final 2 images/masks as the testing dataset. An example of the input image and its corresponding masks are illustrated in Figure 1.

This study adopted a special network, called U-Net, to detect buildings. In this section, the U-Net architecture was briefly introduced and was followed by an illustration of the general workflow for training, validation, and testing. Several technical components, such loss functions, accuracy measurements, and optimization algorithms were also explained in this section.

Our indicator for loss during this project was Binary Cross Entropy [13], or BCE for short. It is a loss function used to classify binary (yes/no, A/B, 0/1) tasks. As such, it is represented by Loss Equation (1):

− 1 N ∑ i = 1 N     y i log y ^ i + ( 1 − y i ) log ( 1 − y ^ i ) (1)

where y ^ i and y i are the output scalar value and the target value of the i’th term, respectively, and N is the number of scalars within the output data for the model. Our purpose for using BCE is to allow quick processing and classification of our training examples, as it is equivalent to maximum likelihood estimation fitting, guaranteeing consistency and statistical efficiency.

For accuracy prediction, the Intersection over Union (IoU) metric, a 0 (least overlap) to 1 (most overlap) scale metric, was used to determine the amount of similarity between the predicted masks and the ground truth masks. Figure 3 illustrates an IoU as the ratio of the two bounds’ overlap over the total areas of the two bounds.

Our method adopted Adaptive Moment Estimation, or ADAM, for stochastic gradient descent optimization. ADAM optimization relies on the first and second moment of gradient to update its learning rates. It has an increased cost, due to

requiring the calculation of the second derivative, but with the added benefit of converging in circumstances that standard gradient descent may not, as it is invariant to gradient rescaling.

Mathematically, by defining the estimates of m t and v t as the mean and the uncentered variance of the gradients of current mini-batch g t , respectively, as well as the decay rates as β 1 and β 2 , ADAM can be represented as following Equations (2):

m t = β 1 m t − 1 + ( 1 − β 1 ) g t v t = β 2 v t − 1 + ( 1 − β 2 ) g t 2 (2)

This gives a decaying average for both that allows the gradient descent to proceed and eliminate more jagged routes to the intended destination.

Python 3.8 is the main program environment. Several packages were added, including PyTorch, Click, and NumPy for code development, as well as matplotlib for visualizations. PyCharm and Jupiter Notebook were crucial for debugging and arduously testing the code. All the code development and experiments were conducted on a 2020 16’ MacBook Pro with an 8-core i9 Processor and 16GB DDR4 Memory.

After running through 20 epochs (30 seconds per image/2.5 hours for the entire training) with the training data, a few insights were discovered as to the loss and accuracy of the masks as a whole.

The average of the 12 BCE values at each epoch was used to determine the training loss over time. There was a significant decreasing negative rate over time, as shown in the first graph of Figure 4.

As to the Validation Accuracy over time, the average of the 2 IoU values at each epoch were use to measure our prediction. Due to the high variance of the

data, a logarithmic trend line was plotted to show the general increase of the accuracy over time (Figure 4, second graph).

After training and validation, the final model was saved to a separate folder, called saved models.

Through the previous training and validation, we were able to get a well trained model to run on the test images. Figure 5 shows the result of the model on a test image. The general result seems to accurately find the buildings, leading us to conclude that our trained model was a good fit.