Image classification is a challenging task and numerous efforts have been made to solve this problem. In recent studies, Deep Learning (DL) techniques presented better predictive performance than traditional machine learning algorithms in many domains, including image classification. Convolutional Neural Networks (CNNs) [1] is a deep learning technique that has gained global recognition in computer vision systems in recent years.

CNN has reached state-of-art performance in a number of applications including image classification [2] - [4] , detection [5] - [7] and segmentation [8] [9] . Compared to standard machine learning algorithms, deep learning methods learn from experience and through that, form a hierarchy of concepts to get a better understanding of the world. Learning through experience avoids the need for humans to manually engineer hand-crafted features which is a main advantage of deep models over traditional methods. Additionally, certain deep learning algorithms make explicit assumptions about the input data that allow them to perform remarkably well in specific tasks. For example, CNNs make assumptions about the properties inherent in images such as stationary of statistics and locality of pixel dependencies which allow for fewer connections and parameters, leading to less computational demands and easier training. Furthermore, because the design of CNNs was inspired by the human visual system, they can be trained to recognize complex visual patterns and rich features that have been shown to even outshine human performance.

Convolutional neural networks (CNNs) are similar to traditional artificial neural networks (ANNs). Both perform a feedforward and backward pass which includes computing a linear transformation between the input and weights, followed by a non-linear activation function, calculating the error and then backpropagating the error back to the weights. Both systems are also made up of a series of neurons but the connections between these neurons are how they differ.

In neural networks, each neuron in one layer operates independently and doesn’t share any connections but instead was connected to each neuron in the next layer. This is problematic when working with images because a 300 × 300 × 3 image would result in neurons with 270,000 weights which is computationally costly. In addition, the network will also need to be a lot larger to deal with this scale of input. As a result, it is impossible to have unlimited computational power and time to train huge ANNs.

Another reason is that huge ANNs would lead to overfitting. Overfitting occurs when a trained model fits the noise of the data thus hampering its ability to generalize well to new examples. Reducing the complexity of ANNs can stop or reduce the effects of overfitting. The less parameters require to train, the less likely the network will overfit to improve the predictive performance of the model.

Luckily, CNNs specialize in dealing with images by utilizing convolution along with local receptive fields to help control the amount of parameters. Imagine taking a filter of size 5 × 5, and sliding it across an entire image to find features that match the filter. This is exactly how convolution is performed but the filter is initialized with weights and dot products are computed between the weights and input image exactly as in neural networks. With convolution, instead of neurons being connected in a fully connected manner, neurons in one layer are only connected to a small region of the layer before it which is controlled by the size of the filter or receptive field.

Despite these remarkable qualities, a major drawback with deep learning models is that they require large amounts of data in order to perform well. Only through the rise of large annotated datasets such as ImageNet [4] and the growth of graphics processor units (GPUs) have CNNs been able to achieve the recent success that they’ve had. Additionally, with the rise of big data and increased complexity of tasks, training CNNs incur a high computational cost which is infeasible on a single machine without large computational resources. Moreover, CNN architectures have become increasingly deeper which on one hand, allows for better feature representations but also increases the complexity of the network. ResNet [10] is about twenty times deeper than AlexNet [4] and eight times deeper than VGGNet [3] . State-of-art CNNs require a massive number of parameters that need to be tuned during training which leads to extensive training times and models that are highly difficult to optimize.

To cope with increasingly complex tasks and volumes of data, the evolution of CNNs has severely impacted their efficiency. CNNs are innately both data and computationally intensive which makes speed and storage capacity a large limiting factor in reaching performance and scalability requirements. To overcome the imposed time and space obstacles, this work implements a parallelized CNN algorithm based on MapReduce [11] (MCNN) on a cloud computing cluster. The developed algorithm takes advantage of the computational structures inherent in CNNs that lends them to parallelization to achieve increased processing speed. Additionally, the use of cloud computing provides an economical means to facilitate data intensive applications by providing workload balancing and resource scheduling.

The rest of the paper is organzied as follows. Section 2 reviews some related CNN work in literature. Section 3 briefly introduces CNN algorithm. Section 4 describes in detail the design and implementation of the distributed MCNN algorithm. Section 5 evaluates the performance of the proposed MCNN in a MapReduce environment. Section 6 concludes the paper and points out some future work.

Due to the shortcomings of the traditional process, deep learning algorithms, more specifically CNN, are widely used for image classification. Commonly image processing tasks successfully addressed by CNN are handwritten and image recognition problems [12] .

Besides the handwritten problem, [13] uses a variation of convolutional networks, namely Neocognitron (NEO), for face recognition task. The positive rates for the NEO decline significantly when the classifier is tested under more unconstrainted conditions. In [14] , a visual automated system using CNN was adopted for visual tunnel inspection. A deep convolutional networks (ConvNets) is adopted to localize wooden knots in images of oak board. A significant improvement has been found by comparing to support vector machine. As in other studies, CNN outperformed the traditional machine learning techniques in [15] . This study investigated CNN applied to defect detection in different materials. By comparing to other studies, CNN does not need a feature extraction process due to its embedded module.

A data mining approach is proposed in [16] to improve the local binary patterns in texture analysis. Three different descriptors with three spatial resolutions are used to evaluate the proposed approach in texture images. In [17] , a J48 classifier is built to evaluate colonoscopy exam images using texture descriptors. As shown in the results, the proposed classifier has achieved a relatively high sensitivity. In [18] , a CNN with combination of texture-based feature extraction techniques are used for biological image classification. The proposed algorithm is compared with traditional techniques such as decision tree, neural networks, nearest neighbors and support vector machine. The proposed CNN algorithm has achieved predictive performance superior to traditional classification techniques.

For image classification through MapReduce, the literature presents the improvement of statellite images in [19] . A system based on Hadoop that implements the MapReduce programming model is used to improve the classification of large-scale remote sensing images. In [20] , a MapReduce-based distributed SVM is used for image classification annotation. SVM with bagging have shown better performance in classification than a single SVM. The proposed algorithm re-samples the training dataset based on bootstrapping. The training time is reduced significantly with a high level of accuracy in classification. A parallel design and realization method for particle swarm optimization with back-propagation neural network is proposed in [21] to improve the classification accuracy and runtime efficiency of the back-propagation neural network. The results demonstrate both higher accuracy and improved time efficiency from applying parallel processing on big data.

To summarize, research on CNN algorithms has been carried out from various dimensions, but mainly focuses on improving the classification accuracy. Improving the runtime efficiency of a CNN still remains an open challenge. This motivates the design of a Mapreduce-based CNN, which is an efficient distributed CNN algorithm building on a highly scalable MapReduce implementation for image classification.

Convolutional neural networks are comprised of three types of layers: convolutional layers, pooling layers and fully-connected layers.

Convolutional Neural Networks is one deep learning algorithm that can benefit from the parallelized computation offered by the MapReduce programming model. CNN iteratively adjusts weights in the network by computing their partial gradients after each set of the training data is propagated through the network. Thus parallelization during the training phase can be accomplished by distributing the data into a number of chunks. Each data chunk can then be fed to several CNNs and each CNN can be trained independently in parallel. The outputs can then be aggregated to produce the final results which are then used to update the weights for the next iteration. Figure 4 below shows a high level overview of the procedure.

The mappers take as input with the training data (x,y) and a set of randomly initialized weights $\left[w_1,w_2,\cdots ,w_n\right]$. The set of weights contain n weights to represent the number of hidden layers in the network so $w_i$ corresponds to the weights for hidden layer i. The training data is represented as a set of tuples (x,y) where x is a training instance and y is the ground truth label for that instance. Each mapper initializes the network with the given set of weights and the network is trained using the input samples. The output is a set of newly trained updated Weights $\left[\text{Δ}{w}_1,\text{Δ}{w}_2,\cdots ,\text{Δ}{w}_n\right]$ which is then fed to the reducer. After each iteration, the mappers receive a set of updated weights which are processed through the network until the max number of iterations is reached. The pseudo code for the map function is shown in Algorithm 1.

With the rise of big data and increased complexity of tasks, the efficiency of deep learning algorithms have been severely impacted by the nature of long training times and high computational cost. To be able to solve even greater problems of the future, learning algorithms must maintain high speed and accuracy through economical means. MapReduce [11] is one of the most efficient big data solutions, which enables the processing of a massive volume of data in parallel with many low-end computing nodes. This programming paradigm is a scalable and fault-tolerant data processing tool that was developed to provide significant improvements in large-scale data-intensive applications in clusters. To that end, this paper takes advantage of the MapReduce framework to develop a parallel CNN algorithm (MCNN). The proposed MCNN achieves high classification accuracy in image classification and close to linear speedup, scaleup and sizeup. The results demonstrate that the MapReduce framework is an effective tool to improve the speed and scalability of CNNs.

In terms of directions for future work, several ideas come to mind. Experiments conducted here utilize images from CIFAR-10. A much larger dataset such as ImageNet with additional compute nodes could be used to analyze how this affects performance in speedup, scaleup and sizeup. Additionally, instead of image classification, other computer vision tasks such as image segmentation or object detection could be applied. Trying other CNN architectures such as ResNet or GoogLeNet [26] and examining their performance in accuracy and speed of convergence is another interesting path. Lastly, ensemble methods similar to the work done in [27] can also be explored.