Using convolutional neural networks, two detection algorithms can be chosen for image object detection and classification: one stage detection with fast speed and low accuracy, and two stage detection with slow speed but high accuracy. This time, a two-stage Retina Net algorithm is used to recognize the target image. This algorithm adopts a fusion algorithm of ResNet, FPN, and FCN, and uses Focal loss as the loss function. Focal loss mainly solves the problem of target detection loss being affected by negative samples due to the imbalance of positive and negative sample areas during the object detection process. Firstly, classify the image (Figure 1(a)) to identify it as a picture of a sheep. Then, perform object detection (Figure 1(b)) to recognize it as a photo of a sheep and mark the position of each sheep in the image.

Figure 1. Image classification (a) and object detection (b).

In convolutional neural networks, the preferred region needs to be determined, and we use exhaustive search to generate candidate regions. The A pixel on the image and the B pixel in the lower right corner of A can determine a rectangular box, denoted as AB. A is located in the upper left corner of the image, and B traverses all positions except A to generate rectangular boxes A1 B1, ..., A1 Bn (Figure 2(a)). After obtaining a certain position of A in the middle of the image, B traverses all positions in the lower right corner of A and generates rectangular boxes Ak B1, ..., Ak Bn (Figure 2(b)). When A traverses all the pixels on the image, B traverses all the pixels in the lower right corner, and finally generates a set of rectangular boxes {AiBj}, which includes all the cocoa selection areas on the image.

Figure 2. Candidate area confirmation.

As long as the classification of each candidate region is accurate enough, it is certain that a region that is close enough to the actual object can be found. The exhaustive search method may yield accurate prediction results, but its computational complexity is also enormous. Assuming H = W = 100, the total number will reach 2.5 × 107 points. Assuming the classification is fine enough, exhaustive search can theoretically complete the detection task, but it requires designing bounding boxes, anchor boxes, and intersection to union ratios for object detection to accurately provide candidate regions.

Boundary box: Boundary boxes are often used to represent the specific position of objects and can be inserted into rectangular boxes of objects. It can be inferred that boundary boxes are a detection task that simultaneously predicts and detects the position and category of objects. There are usually two forms of bounding boxes: xyxy (x₁, y₁, x₂, y₂) and xywh (x, y, w, h).

In detection tasks, bounding boxes are called real boxes because the labels on the training dataset provide the coordinates of the real bounding box of the target object, which are (x₁, y₁, x₂, y₂). The model accurately predicts the possible positions of the target object, and the bounding boxes predicted by the model are called prediction boxes [16].

Anchor box: Anchor box is a type of target box imagined by people. Draw an anchor box of a specific size, find a center point, and draw a rectangular box.

In the detection task, an anchor box is formed on the image, and it is determined whether the target object is included before proceeding to the next step of object detection. Due to the mismatch between the object and the anchor box, it is necessary to make fine adjustments to the original anchor box to form an anchor box that can accurately obtain the position of the object. The model needs to predict the magnitude of the fine adjustment, and different models have different methods for generating anchor boxes and different micro adjustment amplitudes, because the anchor box position is fixed at the beginning.

Intersection to union ratio: Intersection to union ratio is a measurement indicator in object detection. In object detection tasks, the intersection to union ratio is used as a measurement indicator in object detection, and the specific calculation formula is as follows:

The green area in the “Intersection” section of the figure represents the overlapping area of two boxes, while the green area in the “Merge” section represents the merged area of two boxes. Dividing these two areas yields the intersection to union ratio between them, which is also known as the intersection to union ratio (Figure 3).

Figure 3. Comparison of Intersection and Intersection Ratio Operations.

If the positions of these two rectangular boxes A and B are A (x_a1’y_a1, x_a2’y_a2) and B (x_b1’y_b1, x_b2’y_b2), respectively, as shown in Figure 3(d): If there is an intersection between the two, the coordinates of the upper left corner of the intersection are:

The coordinates of the lower right corner of the intersection are:

Calculate the area of the first part to be delivered:

The area of rectangular boxes A and B is:

Calculate the combined area:

Calculate the intersection to union ratio:

In order to clearly demonstrate the relationship between the size of the intersection ratio and the degree of overlap, Figure 4 shows the corresponding positional relationship between two target boxes under different intersection ratios, ranging from I_oU = 0.95 to I_oU = 0.00.

R-CNN generates candidate regions through selective search, segments the image, and merges similar regions. The candidate boxes are uniformly adjusted to 227 × 227 and input into a CNN (such as AlexNet) to extract 4096 dimensional features. SVM is used for classification and non-maximum suppression is used to remove overlapping detection boxes, but the process is complex and time-consuming. YOLO divides the image into N × N grids, and directly predicts the bounding box position and category probability for each grid, achieving end-to-end detection and greatly improving speed. SSD is improved based on VGG16 by adding multi-scale convolutional layers, using default boxes for detection on different feature layers, and combining multiple aspect ratio presets to achieve adaptability to targets of different sizes. The three represent the technological evolution paths from regional proposal, single-stage detection to multi-scale prediction.

Figure 4. Relative position diagram between two boxes under different intersection and merger ratios.

In order to further evaluate the performance of the sheep detection model and select evaluation indicators for the experimental results, including recall rate R, accuracy rate P, mean average precision mAP, and average precision AP. The recall rate R is an evaluation criterion that reflects the incompleteness of the target detected by a model; Accuracy P is an evaluation criterion that reflects the accuracy of a model’s predictions; AP is the area under the Precision call curve, and generally speaking, the better the classifier, the higher the AP value. MAP is the average of multiple categories of AP. Mean is the average of the AP for each class, resulting in mAP. The size of mAP must be in the range of [0, 1], the larger the better. The calculation formulas for P, R, AP, and mAP are shown in equations (9), (10), (11), and (12), respectively.

Data preprocessing is necessary before the YOLOv3 algorithm, and it is a crucial and primary step in training convolutional neural network structures. Firstly, selecting appropriate data preprocessing methods can prevent overfitting, and then implementing data reading and preprocessing can help accelerate processing. The biometric technology based on YOLOv3 first normalizes the input biometric image with a size of 416 × 416, and then extracts multi-scale features through the Darknet-53 backbone network, generating feature maps at three scales: 13 × 13, 26 × 26, and 52 × 52. Using Feature Pyramid Network (FPN) to fuse deep and shallow features, three prior boxes are pre-set at each grid point to predict boundary coordinates and category probabilities. Finally, the non maximum suppression (NMS) algorithm is used to optimize the detection results, eliminate redundant boxes, and retain the optimal prediction, forming an end-to-end recognition pipeline from feature extraction to target localization, providing reliable technical support for individual identity recognition.

Data retrieval is the process of storing all descriptive information of an image in records, where each element contains a description of the image. The subsequent program demonstrates how to retrieve the image and annotate it based on the description in the records. Data preprocessing is the random processing of images, but with minimal changes. The main purpose is to suppress overfitting and improve the generalization ability of the model by increasing the training dataset. Common methods include randomly changing brightness, contrast, and color, random filling, random cropping, random scaling, random flipping, random scrambling of the real box arrangement order, and using numpy to implement these data augmentation methods.

This article standardizes the animal and plant dataset based on the Pascal VOC 2007 set format and uses the LableImg tool for animal and plant image annotation. Use the VOC_ label. py script to navigate this path to your own dataset, with the class selected as “face”. Finally, running the script will generate files such as train.txt, val.txt, test.txt, etc., and generate a labels file in devkit. Change the model parameters of YOLOv3 and adjust the batch. When batch equals 64, it’s enough, then adjust other parameters to end. The parameter settings are shown in Table 1. If the average loss of the result is less than 0.0607, the operation will be stopped at this time.

Table 1. Parameter settings for YOLOv3.

Adopting the Triplet loss proposed in FaceNet as the loss function. The Triplet Loss function consists of three images: the reference image (anchor), the positive example image (positive), and the negative example image (negative). Through continuous training, there is a small gap between the positive example image and the negative example image, but a large gap between them. The selected VGGFace16 and resNet50 represent two types of convolutional network structures with different network layers (16 layers and 50 layers), respectively, combined with two embeddings (2096 and 128) to construct a sheep face recognition model.

3.1.1. Individual Labeling of Livestock Herds

The dataset of this article mainly comes from photos of livestock herds in Yangchang Village, Zhaojun Town, Dalate Banner, Ordos. LabelImg software was used to label the above data as sheep, including pictures of sheep in various scenarios, totaling 400 images. Divide the annotated dataset into training, testing, and validation sets in an 8:1:1 ratio, resulting in 320 training sets, 40 testing sets, and 40 validation sets, respectively. Finally, the dataset was organized to obtain its label situation as shown in Figure 5:(a) The y-axis of the figure represents the number of labels, and the x-axis represents the type of labels. (b) In the figure, the horizontal axis represents the ratio of label width to image width, and the vertical axis represents the ratio of label height to image height.

Figure 5. Distribution of sheep face labeling labels (a) and aspect ratio (b).

3.1.2. Individual Tracking of Livestock Herds

The Deepsort object tracking algorithm is based on detectors, such as the YOLO object detector. Firstly, the object detector detects sheep in the image and passes the detection coordinates of the sheep to the Deepsort algorithm. The Deepsort algorithm predicts the target position of the sheep at the next moment based on the Kalman filter, and finally completes the target tracking based on the Hungarian algorithm (Figure 6), and assigns a unique ID number to each target.

Figure 6. Individual ID tracking of livestock herd.

3.2.1. Target Detection Model Training

The experiment used a self-built sheep dataset image, with sheep as the sample. The YOLOv3 network was trained using end-to-end stochastic gradient descent method, and the input image size was 640 × 640. The parameter values are set as follows: batch size is 64, momentum value is 0.937, decay value is 0.0005, total rounds are 100, and initial learning rate is 0.01. When one round of training is completed, the model training situation is validated using the validation set. The loss curves of the training set and validation set (Figure 7) show that from the 0th to the 50th batch on the training set, the loss of the training set decreases during training, and from the 50th to the 100th batch, the loss of the training set tends to stabilize, and the model gradually converges. Around the 25th batch on the validation set, the model gradually converges, and around the 50th batch, the Object loss shows an upward trend, indicating that the model may be overfitting.

Figure 7. Loss curves of training and validation sets.

The performance curve of the validation set shows that mAP is the area enclosed by the precision Pr and recall Re plotted as two axes, and mAP and mAP are the average detection accuracy under general threshold and high threshold, respectively (Figure 8). When the values corresponding to the curve gradually stabilize, the optimal training model can be determined. In Figure 8, the top and bottom are the loss curves of the training set and validation set, respectively. It can be seen that the three types of loss functions have a significant decrease in the model training from 0 to 50 times, and remain stable at 50 to 100 times, indicating that training improves the detection performance of the model.

The mAP curve fluctuates significantly from 0 to 50 times at the beginning of training, indicating that the convergence speed of the model in the early stage of training is fast and meets the requirements of model training. After 50 iterations, the model remained stable with minimal changes, indicating good training and no overfitting. The curve around 90 to 100 tends to stabilize, indicating that the training of the sheep detection model is basically completed at this time (Figure 8). The selection of the optimal model is calculated based on mAP-U accounting for 10% and mAP-H accounting for 90%. The accuracy and recall were 94.1% and 81%, respectively, and the mAP reached 63.3%.

Figure 8. Verification set performance curve.

3.2.2. Accuracy Evaluation of Sheep Face Recognition

After training the sheep face detection model with YOLO v3 object detection, computer vision technology is used to read the video stream, perform sheep face recognition and detection, and save the sheep face information in the detection box. After a series of tests, the accuracy is 94.1% and the recall rate is 81%. Real time detection of sheep face data has been achieved. The training data for sheep face detection includes sheep with different postures (Figure 9), different lighting conditions, and camera shake, indicating that the model can accurately detect sheep under different movements and angles, meeting the needs of sheep face detection.

Figure 9. Sheep face detection results ((a) front view; (b) side view).

Divide the sheep image dataset of different scenes into training set, testing set, and validation set in an 8:1:1 ratio, and conduct model training based on this division. When the performance test results of the model using VGGFace neural network and embedding dimension 2096 are obtained, the performance of this model is relatively high. Using a specific model to recognize sheep faces in different situations within the same sheepfold. The use of transfer method to train models with a small number of samples can achieve high accuracy and recall, and can better complete the task of recognizing frontal sheep faces in a small dataset.