Before the rise of researches in pattern recognition field, images of objects and other structures were traditionally represented by vector spaces [1]. This was easily presented by the use of front, left, right and rear elevation views [2]. Generally, the traditional ways to represent objects usually oversimplify the nature, shape and size of the objects. Substantially, objects are made by patterns that can be simply observed physically and or hardly observed by applying different mathematical algorithms. In this context, we define pattern recognition as the process of data classification based on the statistical information extracted from the objects’ representation [3]. As per its essence in solving classification and clustering problems, pattern recognition is widely applied in various fields including natural language processing, medical diagnosis, image and video processing, document recognition, a list is too long to mention as referenced in [4] and [5].

Within all the fields of application mentioned thereof, a pattern recognition problem uses raw data which are processed to a format that a machine can comprehend. Pattern recognition problems are practically solved by different approaches including discriminant analysis, statistical decision theory, structural pattern recognition, combinatorial approaches, support vector machines and syntactic pattern recognition [6]. As its essence has been outlined in the previous paragraphs of this work, we introduce a pattern recognition problem that involves detection of wrong and right patterns and classification of the detected patterns with faults using Convolutional Neural Networks.

Several works have employed Laplacian, Sobel and Canny edge detectors [7] as preliminary steps for pattern recognition. In this work, we apply Laplacian edge detector because it uses kernel or filter and calculates the second order derivatives in a single pass of an image intensity. The Laplacian of an image is given by the following equation.

L ( x , y ) = ∂ 2 I ∂ x 2 + ∂ 2 I ∂ y 2 (1)

where I ( x , y ) is the intensity of an image in reference.

We employed Sobel operator that performs a 2-D operator spatial gradient measurement on an Image that emphasizes the regions of high spatial frequency that correspond to the edges.

| G | = G x 2 + G y 2 (2)

where G x and G y is the rotation and magnitude of the gradient respectively.

We lastly employed Canny edge detection which is a multi-step algorithm that can detect edges with noise suppressed at the same time. This process involves smoothing, gradient computing, thresholding M, suppressing non-maxima pixels and linking the edge segments to form continuous edges. The following are the procedures for canny edge detection.

• Define the Gaussian filter G σ = 1 2 π σ 2 exp ( − m 2 + n 2 2 σ 2 ) to reduce noise and unwanted details and textures.

• Smoothen the image with the Gaussian filter g ( m , n ) = G σ ( m , n ) ∗ f ( m , n ) .

• Compute gradient of g ( m , n ) using any of the gradient operators (Roberts, Sobel, Prewitt, etc) to get M ( m , n ) = g m 2 ( m , n ) + g m 2 ( m , n ) and θ ( m , n ) = tan − 1 [ g n ( m , n ) / g m ( m , n ) ] .

• Determine the minimum threshold M, mathematically defined as M T ( m , n ) = { M ( m , n )   i f   M ( m , n ) > T 0   o t h e r w i s e } .

If the above simple techniques do not perform better, some machine learning methods can be introduced to serve the purpose. Machine learning metric measures are used to validate whether the images are suitable for pattern recognition and segmentation [8] and [9]. The precise measures for thresholding are F₁-Score and Mathew Correlation Coefficient (MCC) that uses the original and predicted edges of an image.

We define a confusion matrix with two elements as Foreground and Background for each edge cases of the image to be recognized and segmented from the raw image. The following are the edge cases that considered during the calculation of the thresholding measurements.

From Table 1 we define the accuracy, F₁-score and MCC measure of the classification as Equation (3), (4) and (5) respectively.

Accuracy = T P + T N T P + T N + F P + F N (3)

F 1 -score = 2 ∗ T P 2 ∗ T P + F P + F N (4)

MCC = T P ∗ T N − F P ∗ F N ( T P + F T ) ( T P + F N ) ( T N + F P ) ( T N + F N ) (5)

F₁-score ranges from 0 to 1 showing that the higher the score close to 1 the better the prediction for the edges, while MCC lies between −1 and +1. MCC has double meaning whereby −1 states the negative correlation between the original edges and the detected edges (mismatching) while +1 shows the relationship between the predicted and original edges also known as absolute matching [9].

The figure below represents workflow that indicates the procedures on how to obtain the expected results using the proposed algorithms. The figure describes the process from raw image data to the measure of performance of the Convolutional Neural Networks model.

Figure 1 indicates that the colored image will be converted to gray scale by either open CV or matplot python libraries and then edge detection methods will be applied. Depending on the nature of the image, if the edge detection methods will not perform better then machine learning metrics will be applied for thesholding process. When the accuracy achieved will be smaller than 75% as described in [9] and [12], then the process goes back to the initial step for cropping different patterns for binary classification processing in CNN. Otherwise, the algorithm produces clear output for the raw image with faults.

Input data was obtained as an image with two different patterns and with non-uniform intensity. The image used was 383 by 676 in pixels and 324.7 KB where traditional methods were applied to detect the similar patterns in the colored image. We applied the pattern detection algorithm based on the patterns and the color intensity of the image. We also used template matching method to determine the patterns on the image where we were able to detect different patterns on the colored image. The methods used for edge detections are Laplacian, Sobel and Canny edge detectors.

We used the cropped template in Figure 2 and OpenCV library for image manipulation. Based on that template we were able to detect the matched patterns as shown in the following figure.

Figure 3 shows matched templates when the maximum matching threshold was set to 0.7 to avoid overwriting of matched templates on the colored image. From the colored image in Figure 3, we cropped different patterns of which some have wrong and right patterns. We had 700 images whose sizes were different because of cropping process and some of the images were rotated at an angle of 10˚. The images were resized to 40 by 40 pixels in order to be compatible with CNN. The data set was divided into training set and test sets whereby the former had 450 images and the later had 250 images respectively. We employed Convolutional Neural Network to classify the binary classification problem and determine the efficiency of our model. The CNN architecture accomplished both feature extraction and classification activities for the images.

The CNN architecture shown in Figure 4 was implemented in Python using Keras and Tensorflow libraries with GPU having two convolution layers with 32 and 64 nodes respectively. Two maximum pooling layers were put after every convolution layer to reduce the number of parameters in order to control overfitting and computational costs. The activation function for convolution layers was ReLU, applied for the sake of converting all negative pixel values to zero and for the last fully connected layer was Softmax that normalizes the probabilities to the interval [0, 1]. We used “adam” as the optimization function because it computes individual learning rates for different parameters [13].

The results are presented as obtained by edge detectors’ methods, thresholding metric measures and Convolutional Neural Networks. With these three algorithms, the results were displayed in a figure where all the algorithms are applied. To obtain precise and accurate results, images must of high quality, detectable background and foreground, and uniform intensity. As shown in Figure 2 the original image was colored however, its intensity was not uniform. Figure 5 shows all the methods used to determine patterns in the original image as explained in the methodology and experiment section.

Even though the edge detection methods were not able to detect the patterns accurately because of low quality and poor intensity of the raw image but CNN was able to classify the images that have right and wrong with faulty patterns.

The model used revealed the truth that the more increase in the input mages, the more accurately the classification problem is handled. We conclude that, the expected results could be precise and accurate for both edge detection methods and CNN if the quality of the raw image is maintained.

This research was supported by the Ministry of Trade, Industry and Energy for its financial support of the project titled “the establishment of advanced marine industry open laboratory and development of realistic convergence content”, PKNU LINC+ Project, and SMTECH Innovation Startup Project.

The authors declare no conflicts of interest regarding the publication of this paper.

Mlyahilu, J., Kim, Y. and Kim, J. (2019) Classification of 3D Film Patterns with Deep Learning. Journal of Computer and Communications, 7, 158-165. https://doi.org/10.4236/jcc.2019.712015