Convolutional neural networks (CNNs) are a deep learning approach, and developments have increasingly widened their range of practical application [1]. CNNs are widely used for medical imaging applications, such as detecting lung tumors in computed tomography images, detecting breast cancer in mammograms, and predicting the risk for cardiovascular disease based on retinal fundus photographs [2] [3] [4]. Their adoption in the clinical field makes ensuring their accuracy imperative. For a CNN to yield correct diagnoses, they require image data with sufficient spatial resolution to extract morphological features [5]. Because of practical limitations, medical images often do not provide sufficient spatial resolution [6]. Certain examination processes such as magnetic resonance imaging may require a patient to hold their breath, which limits the imaging duration. Target lesions often occupy a localized or limited part of an organ or structure in the human body, so the region of interest is much smaller than the overall image, and the cropped image does not have sufficient spatial resolution [7]. When images with insufficient spatial resolution are used as training data, the CNN does not achieve satisfactory diagnostic accuracy. Some studies have shown that using pixel interpolation to upsample images is a useful solution for improving the diagnostic accuracy of CNNs when the training data have insufficient spatial resolution [8] [9].

In contrast, for some medical imaging applications such as pathological imaging and digital mammography, much higher resolutions compared to general images are sometimes used. The performance of a CNN generally improves when the batch size of the training data is increased [10]. However, researchers are often forced to limit the batch size because of the memory limitations of the hardware. Downsampling of high-resolution images is a possible solution to maintaining a larger batch size when the processing hardware has limited memory [11]. Sabottke et al. reported that training a CNN with chest radiographic images downsampled from 1024 × 1024 pixels to 256 × 256 pixels does not compromise the diagnostic accuracy [11]. Although various pixel interpolation algorithms are applicable to image downsampling such as nearest neighbor (NN) [12], bilinear (BL) [12], Hamming window (HM) [13], bicubic (BC) [14], and Lanczos (LC) [15] interpolation, the effect of the interpolation method on the diagnostic accuracy of a CNN has not been fully investigated.

In this study, five pixel interpolation algorithms (NN, BL, HM, BC, and LC) were applied to downsampling images, which were then used as training data for a CNN. The objective was to compare their effects on the diagnostic accuracy.

The medical images used to train the CNN were taken from the public dataset of NIH chest X-rays registered in Kaggle (Chest14) [16]. This dataset contains 112,120 images of 30,805 patients. Each chest X-ray is a grayscale image with a size of 1024 × 1024 pixels. The chest X-rays were classified according to 15 diagnostic labels: normal (no disease), atelectasis, consolidation, infiltration, pneumothorax, pneumothorax, edema, emphysema, fibrosis, effusion, pneumonia, pleural thickening, cardiomegaly, nodule mass, and hernia. Over 90% of the data contained an abnormal diagnosis [16]. The dataset used is open data and has been anonymized.

PyTorch was used to build the CNN, and Pillow was used for the interpolation method with Python version 3.6.9. The hardware environment was a DGX Station (CPU, Intel Xeon E5-2698 v4 2.2 GHz; system memory of 256 GB, GPUs, Tesla V100 × 4; GPU memory, 32 GB per GPU).

The image data for the CNN were interpolated by using five pixel interpolation methods in the image-processing library of Python Pillow: NN [12], BL [12], HM [13], BC [14], and LC [15]. NN is a pixel interpolation algorithm that refers to the brightness value of the pixel nearest to the reference position [12]. BL is a linear interpolation algorithm that resamples the luminance value of a position (x, y) by referring to luminance values for 2 × 2 pixels (4 pixels) in the surrounding vicinity [12]. HN and LC are commonly used window functions. HM is a modified version of the Hanning window, where the values at both ends of the window are zero, so the signal components are not reflected in the spectrum [13]. BC is a cubic interpolation algorithm that resamples the luminance value of a position by referring to luminance values for a bicubic array of 4 × 4 pixels (16 pixels) [14]. HM and LC offer a better frequency resolution and narrower dynamic range than the Hanning window. LC is characterized by discontinuities at both ends of the interval and is one of many finite support approximations of the sinc filter. Each interpolated value is the weighted sum of two consecutive input samples [15].

The five interpolation methods were applied to downsampling the chest X-rays from an original size of 1024 × 1024 pixels to seven different sizes: 320 × 320, 256 × 256, 224 × 224, 192 × 192, 160 × 160, 64 × 64, and 32 × 32 pixels. In total, 112,120 images were generated and were separated into 86,524 training images and 25,596 testing images. Verification was not performed with the original data because the batch size could not be normalized with the other batch sizes owing to the amount of spatial computation. The sizes of the images used as input for the CNN were 320 × 320, 256 × 256, 224 × 224, 192 × 192, 160 × 160, 64 × 64, and 32 × 32 pixels. As shown in Figure 1, the CNN was a simple

network consisting of three 3 × 3 convolutional layers. It used zero padding, instance normalization, and a rectified linear unit activation function. To accommodate the above image sizes of the downsampled input data, the CNN was adjusted to have 360,000, 222,784, 166,464, 118,336, 78,400, 7744, or 576 fully connected layers respectively. For all resolutions, the batch size was fixed at 512. The CNN was trained on each image for 100 epochs in total, and the accuracy was evaluated after training and testing. The results were compared according to the maximum classification accuracy over 100 epochs.

Table 1 and Figure 2 present the classification accuracy of the test data with each model. Reducing the image size increased the accuracy. The best accuracy was obtained at a size of 64 × 64 pixels and was slightly worse at a size of 32 × 32 pixels. At 320 × 320 pixels, BC performed the best with a maximum accuracy of

0.6638. At 256 × 256 pixels, NN performed the best with a maximum accuracy of 0.6656. At 224 × 224 pixels, NN again performed the best with a maximum accuracy of 0.6712. At 192 × 192 pixels, BC performed the best with a maximum accuracy of 0.6650. At 160 × 160 pixels, HM performed the best with a maximum accuracy of 0.6739. At 64 × 64 pixels, BC performed the best with a maximum accuracy of 0.6787. At 32 × 32 pixels, BC performed the best with a maximum accuracy of 0.6724. Thus, the highest accuracy was obtained by BC at an image size of 64 × 64 pixels (0.6787), followed by HM at 64 × 64 pixels (0.6773).

The results of this study suggest that the diagnostic performance of the CNN is affected by the level of downsampling. The maximum accuracy was achieved at an image size of 64 × 64 pixels regardless of the interpolation method. This supports the work of Tan et al. [17], who reported an optimal input dimension depending on the number of parameters for the deep learning model. The present study suggests that the optimal downsampling level is related to the CNN structure. The CNN used here had a relatively simple structure with three hidden layers, so it is potentially able to extract structural features with relatively low resolution. Because more complicated models are commonly used in clinical applications of artificial intelligence, the optimal downsampling process may need to be reassessed according to the CNN that is being used. The classification accuracy increased with a decreasing image size and reached its maximum at 64 × 64 pixels. Because the CNN had a simple structure with few hidden layers and little ability to extract features, downsampling to a small image size such as 64 × 64 or 32 × 32 pixels may serve the same function as feature extraction by the CNN. In other words, a small image size may avoid the curse of dimensionality.

The results also suggested that the interpolation method affects the diagnostic performance of the CNN differently according to the image size. Figure 2 partially reveals the relationship between the image size and interpolation method. For small image sizes such as 64 × 64 and 32 × 32 pixels, BC tended to produce good results. Linear interpolation methods such as NN and BL were not suitable for strong downsampling, while BC performed better, perhaps because it is nonlinear. Linear interpolation (NN, BL) may offer better performance when images are downsampled to a certain size, and nonlinear interpolation (BC) may be more suitable when images are downsampled to extremely small sizes. Various interpolations are available for upsampling and downsampling images to different sizes. In the OpenCV and Pillow libraries, BL and BC are set as the default methods for image interpolation [https://pillow.readthedocs.io/en/stable/reference/Image.html]. BL was previously reported to result in the highest classification accuracy when images with a small size were upsampled [8]. In contrast, the present study suggests that BC results in the highest classification accuracy when images with a large size are downsampled. The two sets of results suggest that a suitable image interpolation method needs to be selected according to the image size and task (i.e., upsampling of downsampling). The limitations of this study were that the effects of the model size and dataset were not considered. Future work will involve investigating larger network architectures and multiple types of medical image datasets. BC also performed better in large downsamplings, but the difference in classification accuracy between BC and BL at 64 × 64 pixels was only 0.41%. The difference between BC and NN is even smaller at 0.34%. More studies with different hyperparameters, such as changing the batch size for each resolution, are needed to clarify why this result is significant.

In this study, five different interpolation methods were applied to the Chest14 dataset, and the effect of image downsampling on the classification accuracy of the CNN was investigated. The experimental results showed that there is an optimal image size for downsampling, and that a suitable interpolation method should be chosen. The results of this study suggest the importance of considering the interpolation method when creating a deep learning model. It was shown that choosing hamming or bicubic for downsampling may give better accuracy than other interpolation methods.

The authors declare no conflicts of interest regarding the publication of this paper.

Hirahara, D., Takaya, E., Kadowaki, M., Kobayashi, Y. and Ueda, T. (2021) Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy. Journal of Computer and Communications, 9, 150-156. https://doi.org/10.4236/jcc.2021.911010