Application of SSD network algorithm in panoramic video image vehicle detection system

3.1 Status of panoramic video image vehicle detection system

With the rapid development of world economy, the number of cars is increasing, and the road traffic problems in cities such as traffic jams, frequent traffic accidents, and serious environmental pollution are gradually expanding [10]. Therefore, how to further improve the traffic capacity of pedestrians and vehicles has become a major premise. At present, the application of intelligent transportation system (ITS) in the transportation system is an important means to solve these traffic problems.

ITS is an efficient integrated traffic management system implemented under the increasingly urgent situation of traffic congestion, environmental pollution, energy waste, low traffic efficiency, and so on [11]. This system works in a wide range and direction [12]. The working principle of ITS is shown in Figure 1.

Figure 1

Working principle of ITS.

As shown in Figure 1, as an important part of ITS, traffic video monitoring system is an efficient and real-time video monitoring system based on computer data processing, digital image processing, pattern recognition, and machine learning technology. This intelligent system can not only save manpower, but also help staff deal with emergencies more effectively to a certain extent, so as to improve the monitoring efficiency. A key problem in intelligent video surveillance is how to extract useful information from a large number of video data independently and effectively. Among them, the relatively important information is pedestrian and vehicle information [13].

To sum up, in urban road traffic, the vehicle detection problem contained in video surveillance system has become a research hotspot. People can control traffic by detecting traffic flow parameters, such as speed, traffic flow, and traffic density, so as to reduce traffic congestion, reduce the incidence of traffic accidents, and improve the traffic environment. The core content of vehicle detection is detection algorithm. Therefore, the research on moving vehicle detection algorithm under video image has important economic and social significance [14].

3.2 Vehicle detection algorithm based on CNN

At present, the detection algorithm based on video image is involved in many applications related to computer vision. For example, video surveillance needs to detect and track moving objects by analyzing video images, including intelligent environment and video retrieval [15]. In these applications, a moving target detection algorithm with low false detection rate is only used as a preprocessing step, which is quite important [16]. From the late 1800s to the present day, scholars have conducted a lot of research on the related challenges involved in motion target recognition and proposed many detection techniques, which have great guiding value in the field of motion target recognition. Among them, optical flow technology and frame difference method are the core methods of motion object detection after separating these existing algorithms.

The optical flow approach may detect moving targets without prior knowledge of the scene’s information. However, in complicated settings, interference variables such as shadow, occlusion, and noise will alter the optical flow field calculation findings. Furthermore, the optical flow method’s processing step is relatively difficult, so it does not have good real-time performance [17].

The disadvantage of the inter frame difference method is that it is as vulnerable to noise interference as optical flow method, and it is difficult to distinguish foreground motion and dynamic background. In addition, this method cannot effectively detect moving targets with uniform color or large outline, and the detection results will appear as “holes” and “ghosts” [18].

In the domains of computer vision, pattern classification and identification, and image processing, vehicle detection is one of the research subjects. CNNs have achieved significant progress in vehicle detection and diverse target detection tasks in recent years, putting them at the forefront of target detection technology and promoting its rapid development [19,20]. Figure 2 depicts the whole network structure of a CNN.

Figure 2

CNN complete network structure.

As shown in Figure 2, the CNN has a large proportion of convolution layers. Convolution kernel is a machine learning algorithm designed to process image data such as video, photo, or text. The convolution kernel can consist of any number of neurons, each using a series of task-specific operations to collect the input images and phrases, and then classify the images and transform them into an easy-to-understand format and describe the resulting output model. Moreover, the parametric matrix of the convolution kernel can be viewed as a filter with specific functions to perform the convolution operations. The parameter matrix of the convolution kernel is also initialized to the corresponding multi-channel form. The calculation formula is as follows:

where A i l − 1 represents the L – 1 feature map of the layer, and when l = 2, A i l − 1 represents the ith channel of the original input image.

The gradient descent method and the loss function are two important aspects of the training process. Gradient descent is an effective machine learning algorithm that can be used to optimize neural networks with the same input and output features, thereby reducing the impact of parameter settings on model performance while maintaining low computational complexity. Loss function refers to the direct relationship between the parameter value and the value of a function. The larger the parameter value, the smaller the function value. The loss function describes a function corresponding to the respective error size on the training set under a given input, thus indicating the deviation between the model error on the training set and the model error on the test set. The goal of training is to get the CNN’s output as close to the true value as possible. The loss function is used to quantify the difference between the model’s predicted and actual values in this research. The data and labels that make up the input to a generic neural network are labels, which are the real values that correlate with the data [21].

We take Softmax as an example of how to quantify the difference between the model’s projected and actual values. Assuming that the number of categories is N, and that the labels are 1 − N, the formula according to the Softmax formula is as follows:

Among them, the output of S is a matrix of 1 × N , and the corresponding CNN output is the probability of the first class to the Nth class. Softmax is calculated as follows:

where b j indicates whether this real value is the jth class or not. For example, if the number of categories is 5 and the true value of these data is the third category, b 3 = 1 , the others are equal to 0.

With the popularity of ITS, the demand for computer vision-based vehicle detection technology is increasing, and the requirements for accuracy and speed are also increasing [22]. Therefore, an SSD detection algorithm is presented in this article.

3.3 Detection ideas for target detection network SSD

SSD CNN is a deep CNN model based on regression method, which is mainly used to solve the target detection problem [23]. It can perform end-to-end training and optimization with real-time detection speed and high precision. It has a broad application field. At present, it has SSD shadow in pedestrian detection, face detection, object recognition, and other applications.

SSD CNN is a supervised deep-learning model. It uses the label information of the input data to guide the training and learning of the model, which is mainly composed of convolution layer, activation function layer, and pooling layer. It combines target recognition tasks with border regression tasks through a multitask learning strategy. It is integrated into an end-to-end deep CNN, which makes the optimization and training of the whole model very easy. Its structure diagram is shown in Figure 3.

Figure 3

SSD network structure diagram.

As shown in Figure 3, as the network gets deeper, the resolution of the additional characteristic response map decreases and the sensory field of a single neuron increases. Combined with multiscale feature response maps for joint detection, SSD can generate shallow feature vectors for small objects and deep feature vectors for large objects. This ensures that the amount of information is saved [24].

SSD has been improved on the basis of you only look once. By utilizing the limited perception domain of the convolution kernel, the feature extraction area is limited from the full image to the local one, that is, the prediction is made using only the characteristics of the surrounding area of a location [25]. Moreover, it uses the different resolution of the feature response map to extract the region characteristics at different scales. It greatly improves the detection accuracy.

It also uses the anchor frame mechanism of Faster-RCNN to establish the corresponding relationship between the position and the characteristics in the graph. However, unlike Faster-RCNN, Faster-SRCNN only generates different scaling scales in the last layer of the feature response map, and Faster-SRCNN generates candidate boxes with different scales and aspect ratios from the feature response maps of various resolutions. Region candidate boxes with different aspect ratios do not extract multiscale features. The candidate box for SSD is shown in Figure 4.

Figure 4

Candidate box for SSD.

SSD discretizes the output space using regional candidate boxes of varying scales and aspect ratios, as shown in Figure 4. It merely predicts the likelihood that each regional candidate box will contain a variety of targets, as well as the offset of each regional candidate box’s location. It also forecasts scores based on the dependability of each regional candidate box, leaves a box with the most likely target after non-maximum suppression, and adjusts the box’s position and form based on the offset. The detection result is the end outcome.

A special convolution operation called void convolution has been proposed by some researchers. It expands the perception field of the network, but does not change the size of the feature map, thereby reducing the loss of information in the image. It allows each convolution output to contain a larger range of information. In the underlying network of SSD, the convolution layer Conv6 uses a void convolution operation. An important parameter in void convolution is the delation parameter. An example of two-dimensional void convolution is shown in Figure 5.

Figure 5

Example of two-dimensional void convolution. (a) Hole convolution with delation value of 1. (b) Hole convolution with delation value of 2.

As shown in Figure 5, there are usually four main adjustable parameters in a convolution layer. They are the number of convolution cores, the size of convolution cores, the sliding step, and the edge filling parameters. Edge filling refers to filling the input image or the boundary of the feature map with 0 elements to increase the height and width of the image. Assuming the size of the convolution core is k × k and the edge filling parameter is p, the size of the output signature graph is w 1 × w 1 with the following formula:

When the void convolution is used, the value of the delation parameter is set to d, and the size of the output signature is w 2 × w 2 , where the value of w ₂ is as follows:

3.3.1 Activating the function layer

In the SSD CNN, the activation function mainly performs nonlinear transformation on the results obtained by the convolution layer, which enhances the nonlinear ability of the model. If the neural network consists of only layers that are fully connected to the convolution layer, the weights and bias only accept linear transformations, and the output of each layer is a linear combination of the inputs from the previous layer. As networks increase, only linear models cannot handle complex tasks such as language translation and image classification. Nonlinear activation functions commonly used in neural networks include Sigmoid, tanh, and ReLU. Among them, ReLU is the activation function used in SSD CNN. The three activation functions are defined as follows:

Sigmoid activation function:

(6) f ( a ) = 1 1 + e − a .

Tanh activation function:

(7) f ( a ) = 1 − e − 2 a 1 + e − 2 a .

ReLU activation function:

(8) f ( a ) = max ( 0 , a ) .

Diagrams of the three activation functions are shown in Figure 6.

Figure 6

Diagram of three activation functions. (a) Sigmoid activation function. (b) tan h activation function and (c) ReLU activation function.

As shown in Figure 6, the Sigmoid activation function is a monotonically increasing continuous function, and the output value of the function is always between 0 and 1. It is easy to derive, but its output is not zero-centric. The Sigmoid function curve tends to be smooth on both sides, indicating that its derivative tends to be 0 on both sides, which is soft saturated and easy to cause the disappearance of gradients, leading to training problems. ReLU can effectively reduce the time of model training.

3.3.2 Pooling layer

In the SSD convolution network, the pooled layer is often behind the convolution layer, which is not easily over-adapted. There are usually three ways to pool operations: average pooling, maximum pooling, and random pooling.

The size of the output feature map is h 1 × h 1 :

(9) h 1 = h − k + 2 × p s + 1 .

The L layer is shown by the following formula:

(10) a j l = f ( β l × Pool ( a i l − 1 ) + B l ) ,

where a i l − 1 represents the output of Layer L – 1, that is, the output of the convolution layer and the input of the pooling layer. f denotes the activation function, which can be either a Sigmoid function or a tanh function. If Pool is pooled to the maximum, then we have the following formula:

(11) Pool ( a i l − 1 ) = max a i l − 1 .

If Pool is pooled averagely, then we have the following formula:

(12) Pool ( a i l − 1 ) = mean a i l − 1 .

Random pooling refers to the random output of a value from a selected pooled area in terms of probability size. These three pooling methods will still produce the same pooling characteristics with local invariance when there is a slight change in the input data. The maximum and average pooling operation diagrams are shown in Figure 7.

Figure 7

Diagram of two pooling modes.

As shown in Figure 7, the essence of pooling is actually sampling. By aggregating data information from adjacent rectangular areas, the obtained statistics can be used to replace the characteristics of this area, which is pooling.

3.4 Vehicle detection in video image based on SSD network algorithm

During the training phase of classifying tasks, the input of the network is a single picture and its corresponding label, which the network learns through. This minimizes the error between the eigenvector and the label vector of the forward propagation output of the picture, as shown in the following formula:

Among them, a represents the input signal, w represents the network weight, and N represents the number of samples. One common feature of these input pictures is that the target in the picture is only one and occupies the majority of the image, and the background information is less, for example, ImageNet datasets, because redundant background noise can affect the learning of discriminatory features from the network. In the detection task, there are multiple targets in the picture. The label of the training set is no longer a picture corresponding to a category label, but a picture corresponding to multiple category labels and each category label corresponding to a location label.

The training phase of SSD network is actually learning the parameters w f of network feature extraction and the prediction parameter corresponding to each region candidate box. The network training process solves the following optimization functions, such as formula (14):

where Θ L represents the location prediction parameter, and Θ C represents the category probability prediction parameter.

For vehicle target detection tasks in panoramic video images, Precision, Recall, and FL-scope are used to evaluate the detection performance of different methods, which are defined as follows:

TP refers to the number of real vehicle targets detected, FP refers to the number of background errors detected as vehicle targets, and NP refers to the total number of real vehicle targets in the graph. Precision refers to the detection accuracy, Recall refers to the recall rate, FL-score is a balance factor between Precision and Recall, and it is the main reference index to evaluate the detection performance. The higher the Precision and Recall values, the better the detection performance.

4.1 Experiment of vehicle detection algorithm based on SD video image in different scenes

To verify the effectiveness of the proposed algorithm for vehicle detection in video images based on secure digital card (SD) three types of scenes are used as experimental datasets in this article. It compares optical flow, inter-frame difference, and SSD network algorithms on video, and then selects a frame in the output mask image. Finally, the calculated data and corresponding polyline diagrams are compared and analyzed.

Scenario 1: It is an ideal environment without too much external disturbance. In this article, these algorithms are simulated on the highway video under Baseline, and the result of the 800th frame image in the output frame sequence image is shown in Figure 8.

Figure 8

Comparison of recall and accuracy of three methods in Scenario 1.

As shown in Figure 8, we can see that the SSD network detection algorithm and other algorithms proposed in this article have better overall performance and better evaluation index under ideal scenarios. The recall rate and accuracy of SSD network algorithm are the highest compared with other algorithms.

Scenario 2: The moving target has dynamic background interference such as being obscured by the trunk and swaying leaves. In this article, these algorithms are simulated on fall video under Dynamic Background, and the result of the 800th frame image in the output frame sequence image is shown in Figure 9.

Figure 9

Comparison of recall and accuracy of three methods in Scenario 2.

In Figure 9, the graph shows that the evaluation indexes of other detection algorithms are not as good as those of SSD network algorithms. This also indicates to some extent that the better detection algorithms for all kinds of scenes are not always obtained by combining them. The detection performance of SSD network is much better than other algorithms. It greatly reduces the interference of dynamic background on the detection of moving vehicles. It is undeniable that the recall rate and accuracy of the proposed SSD network algorithm are the highest, which indicate that the algorithm is relatively robust.

Scenario 3: It has a night highway, low visibility, and other disturbing factors such as car light halo. In this article, these algorithms are simulated on the fluid highway under Night Videos, and the comparison results in the output frame sequence images are shown in Table 1.

As shown in Table 1, it can be seen that the optical flow method and the inter-frame difference method do not work very well in night scenes because the night light is weak. Therefore, the invisible areas of moving target vehicles increase, which greatly reduces the recall and accuracy of various detection algorithms. Also, in the detection process, there will be interference factors such as dizziness of headlight and road reflections, so the false detection rate is much higher than other scenes. Although the detection accuracy of the SSD network algorithm is not as high as that of the previous scenarios, it is still higher than the other two algorithms.

4.2 Comparison of CNN and SSD network algorithms

The Caffe deep learning framework from the Ubuntu 16.04 operating system is used in this article. The following are the specific settings of Caffe’s network training. The loss function is reduced and the network weights are updated using the stochastic gradient descent optimization method. In the video, the total number of training iterations is 600, and the batch size is 20 automobiles.

The automotive training set is used to train the CNN model and the SSD model, and the detection performance of the two models is compared, as shown in Figure 10.

Figure 10

The detection performance of two models with various training iterations is compared. (a) The first 300 iterations of two models’ detection performance are compared. (b) A comparison of two models’ detection performance over the last 300 iterations.

As shown in Figure 10, the distribution of a particular dataset is studied first when assessing the data distribution of cars. The region candidate box is then reset so that the output space can be discretized properly. The ideal matching threshold is then investigated for the reset area candidate box, which increases the SSD network identification accuracy.

The proposed method is compared to the classic single CNN target detection algorithm in order to validate its superiority, as shown in Figure 11.

Figure 11

Two test images. (a) Test diagram A and (b) Test diagram B.

As shown in Figure 11, for CNNs, the bottom features usually represent the edge contour information of the target, while the top features usually represent the abstract semantic information of the target. It is very helpful for target detection of specific absorption rate image to combine the low-level and high-level features.

Comparison of detection performance between CNN and SSD network algorithms for two test images is shown in Tables 2 and 3.

As shown in Tables 2 and 3, the detection accuracy of a single CNN algorithm is the lowest, which is only 0.7554. Its detection performance is also the worst: it cannot locate the target well, and the detected target frame is fixed and cannot be adjusted automatically. The main reason is that the CNN algorithm of one only considers the intensity information of the target. It does not make use of the size, structure, and other information of the target, and the use of image information is insufficient. SSD makes use of multiscale convolution feature maps at the bottom and at the top to make joint predictions and make full use of the features.