Traffic sign recognition (TSR), as a subsystem of advanced driving assistance system (ADAS), has attracted wide attention. TSR mainly includes two parts: traffic sign detection and traffic sign recognition. Traffic signs detection is to find signs from the image, recognition is to accurately classify the detected signs and determine their categories [1, 2]. Traffic signs contain important road information, which is an important guarantee for drivers and pedestrians to drive and travel safely. An important reason for traffic accidents is that drivers fail to see some important traffic signs in time and take corresponding measures. Domestic research on traffic sign detection and recognition lags behind Europe and America, but with the development of intelligent vehicles, many well-known universities and research institutes have joined the research ranks of traffic sign detection and recognition [3, 4]. In 1987, Japanese scholar Akatsuka et al. used color segmentation method to detect traffic signs and used template matching method to identify them . By the 1990s, some developed countries, such as in Europe and the United States, were unwilling to show their weakness, and began to study the traffic sign recognition system one after another. In view of the challenges in the process of detecting and identifying the information quantity of the static traffic sign of highway, it is necessary to explore and study constantly in order to make the field develop better and faster.
Liu et al. proposed a traffic sign recognition method based on graph model and convolution neural network (CNN), and established an application-oriented traffic sign recognition system of convolution neural network (R-CNN) based on area. A graph model based on super-pixel region of UCM was constructed to effectively utilize the multi-level information from bottom to top. A hierarchical saliency detection method based on graph model was proposed to extract the region of interest (ROI) of traffic signs, and convolution neural network (CNN)was used to extract and classify the features of ROIs. The test results show that the method is relatively simple, but the overall recognition rate is low . Zhang et al. proposed a circular sign image recognition method based on invariant moments and support vector machines. Firstly, according to the color and shape information of traffic signs, the original image was processed by color segmentation, morphological denoising and shape detection, and the region containing traffic signs in the image was obtained. Then the Hu moment and Zernike moment eigenvalues were extracted from the sign images, respectively. The eigenvalues were input into SVM for training and the grid search method was used to optimize the parameters of SVM. Finally, the optimized SVM method is used to recognize the traffic signs. Experiments showed that this method had a high recognition rate, but it did not normalize the size of the image, and the recognition efficiency was low . Wang et al. put forward a traffic sign recognition algorithm based on optimized CNN structure. Among them, BN method could be used to change the data distribution in the middle layer, normalize the output data of convolution layer to mean value of 0 and variance of 1, so as to improve the training convergence speed and reduce the training time. In GLP method, the first layer convolution network was trained, the parameters were retained, the second layer was trained, and the parameters were retained until all the convolution layers were trained. The SVM classifier focused on the wrong samples only and no longer processed the correct samples, thus improving the training speed. Experimental results showed that the proposed method was stable, but the image was not processed during the process, resulting in poor anti-interference performance of the algorithm . Liu et al. proposed a traffic sign classification method based on fusion space tower operator and histogram cross-kernel support vector machine. In this method, the appearance, color and contour information of traffic signs were described by extracting Gray-PHOW feature, Color-PHOW feature and PHOG feature. By extracting the characteristics of spatial histogram, the spatial distribution of various features of the image was described. After extracting the spatial distribution information of the appearance, color, contour and feature of the image, the fused spatial pyramid feature was obtained. This method had a certain recognition efficiency, but under a certain number of traffic signs there was a missing detection .
Aiming at the existing problems in the previous research results, a fast recognition algorithm of static traffic sign information based on SURF is proposed. The detailed process is as follows:
The static traffic sign image of highway is processed by mathematical morphology method to improve the recognition rate and efficiency of traffic sign information and enhance the interference resistance performance in the process of recognition.
Traffic sign image matching is realized by SURF feature matching method, and traffic sign information recognition is realized to further improve the recognition rate.
The validity of the algorithm based on SURF is verified by experiments.
2.1 2.1 Static traffic signs processing of highway
There are some noises in the collected static traffic sign images of highways, which need to be removed by digital morphological processing. In addition, because traffic signs are not only limited to border and color information, their internal information (such as speed limit signs will have limited speed) is also an important component of traffic signs [10, 11]. In order to obtain all the information of traffic signs, it is necessary to extract the region which conforms to the characteristics of traffic signs on the basis of noise removal. These tasks can also be completed by morphological processing.
Expansion: Expansion is the basic operation of morphological image processing, which can be used to connect the discontinuous cracks in traffic sign images. Assuming that A and B are two sets, A being expanded by B means that the origin of structural element B and A have the overlapping part, denoted as A ⊕ B, where the image mapping of B is denoted as B̄, and Eq. (1) is defined as A being expanded by B.(1)
where z represents the shift element, and B̄B is the reflection of set B on the symmetry of its origin. As shown, in Figure 1(a) is set A, and Figure 1(b) is a point-centered square element B, B expands A into a set of all shifted elements, which is the result shown in Figure 1(c).
According to the above expansion process, the discontinuous cracks in the traffic sign image are connected, which lays a foundation for fast identification of traffic sign information.
Corrosion is one of the most basic operations in morphological processing. Corrosion treatment of traffic sign images can shrink or refine the connected areas. Supposing there is a set A, where pixel coordinates are (x, y). The translation of point z = (z1, z2) to set A is (A)z, defined as Eq. (2):(2)
where B corrosion of A defines the movement of structural element B above A and the part that A is contained in structural element B is retained, denoted as 5, and defined as Eq. (3):(3)
Corrosion is used to shrink or refine the connected area of traffic sign image and improve the recognition rate of sign information.
Area filling can start from one point in the region, and expands the fill color from inside to outside to the whole area. For a pixel q in the image, there may be two adjacent connections: the 4-adjacency point and the 8-adjacency point. The 4-adjacent point of q refers to the four adjacent points of upper, lower, left and right, while the 8-adjacent point refers to the eight adjacent points of upper, lower, left, right, upper left, lower left, upper right and lower right, as shown in Figure 2.
Based on the definition of 4-adjacency point and 8-adjacency point, connected regions are also divided into 4-connected region and 8-connected region. The main difference between 4-connected region and 8-connected region is that their boundary conditions are different. In Figure 3(a), a 4-connected region is shown, but it does not satisfy the condition of a 8-connected region, because accessing 8-adjacent point in the region will cross the region when traversing the points in the region; and in Figure 3(b), a 4-connected region is also a 8-connected region.
Based on the above analysis, it is assumed that the boundary points of all pixel regions in Set A are 8-adjacent, if B is an element with symmetrical structure, A is an input static traffic sign image of the highway, and Ac is a complement of A, this process can be used to represent by Eq. (4):(4)
where X0 = q, in the iteration process, when Xk = Xk−1 is satisfied, the algorithm terminates. In the result, Set Xk represents the interior of the filled traffic sign. When Eq. (5) is satisfied, traffic signs, including the boundary and the entire internal area, have been filled.(5)
The result of Eq. (5) is the result of filling the traffic sign area. After morphological detection such as swelling, corrosion and region filling, the region of static traffic sign noise can be removed and the region of interest can be detected [12, 13], in order to improve the recognition rate and efficiency of traffic sign information, enhance the interference resistance in the process of identification and reduce the rate of missed detection.
2.2 Static traffic signs information recognition of highway based on SURF
According to the traffic sign image processing in Section 2.1, the SURF method is used to quickly identify the static traffic sign information of highway by feature matching. SURF feature is an efficient variant of SIFT algorithm. SURF inherits the advantages of SIFT. It is several times faster than SIFT and has better stability in multiple images. This is needed in the process of traffic sign recognition [14, 15, 16, 17, 18, 19, 20, 21].
In order to further improve the efficiency and accuracy of traffic sign recognition, the icon image is normalized. In the dimension normalization of the icon image, assuming the size of the original traffic sign image is m × n and the target image is m' × n', then the ratio of edge lengths of the two images is m/m' and n/n'. The (i, j)th pixel 5 (line i and line j) of the target image can be obtained by using the ratio of the length of a side to the original image, and its corresponding coordinate is (i × m/m', j × n/n'). Obviously, the coordinate value is not an integer, but only an integer can be used as the pixel value of the image. Bilinear interpolation algorithm role is to find the nearest four points to the corresponding coordinates, and calculate the value of the point. Its principle is shown in Figure 4.
In Figure 4, E11, E12, E21 and E22 denote four pixels. Linear interpolation is performed in the transverse direction, blue dot T1 is inserted between E11 and E12, and blue dot T2 is inserted between E21 and E22. Point P is obtained by interpolating in the longitudinal axis by T1 and T2.
Assuming that the values of function f at E11 = (x'1, y'1 ), E12 = (x'1, y'2), E21 = (x'2, y'1), and E22 = x'2, y'2) are known, linear interpolation is performed in the transverse direction as shown in Eq. (6) and Eq. (7). The difference in the longitudinal direction is shown in Eq. (8), and the value of f at P = (x', y') can be calculated, as shown in Eq. (9).(6)(7)(8)(9)
The result of Eq. (9) is the normalized result of icon size. Using the above results, the SURF feature points can be extracted. The core of SURF algorithm is to construct Hessian matrix. Hessian matrix is a square matrix composed of the second-order partial derivatives of multivariate functions in mathematics. Assuming that a pixel in the traffic sign image is represented by f (x', y'), its Hessian matrix is shown in Eq. (10):(10)
According to Eq. (10), the Hessian matrix of each pixel can be obtained, and then the positive and negative results of Hessian matrix discriminant Eq. (11) can be used as a basis to determine whether the point is an extremum, where H 0 is a Hessian matrix and det (H') is its eigenvalue.(11)
In order to obtain the features of acceleration robustness with size invariance, Gaussian filtering as shown in Eq. (12) is usually used before constructing Hessian matrix.(12)
where L' (x', t) represents the traffic sign image with different resolution, and can be convoluted by image function I (x', t) and Gaussian function G' (t) at point x'.
Among them, the expression of G' (t) is:(13)
Where, g (t) represents the Gauss function, and t represents the Gauss variance.
By analyzing the pixels processed by Hessian algorithm, these pixels are compared with the size of the points in its three-dimensional domain, and the points with the maximum or minimum value are retained as the initial feature points. Then the sub-pixel feature points are obtained by three-dimensional linear interpolation method, the threshold is set to remove the weaker feature extreme points, and the strongest feature points retained are used as the traffic sign feature points. The representation of the final feature set is:(14)
In the above equation, δ represents the weaker extreme point after removing threshold, which is controlled in the range of [4.5, 4.6], and the best removal effect is obtained, that is, the traffic sign image recognition effect is the best.
After extracting the SURF features, the FLANN algorithm is used to realize the feature point matching, and the static traffic sign information is recognized.
Eq. (15) is used to calculate the Euclidean distance between the matching feature points of two traffic sign images, and the minimum Euclidean distance of min (dist) is obtained. The threshold S is set, so that when S ≤ N1 × min (dist), it is determined as the best matching point.(15)
where (x'1, x'2, · · · , x'r) and (x''1, x''2, · · · , x''r) represent two matching SURF feature vectors in the feature set, respectively. The number of the best matching points in the two images is N 2, and the threshold value is S'. When N 2 ≥ S', the matching is successful, the traffic signs are judged to be the successful matching classes, so as to complete the information recognition of the static sign of the highway.
Experimental data sources: during the experiment, the standard image set is composed of several traffic signs,
and the experimental samples are shown in Figure 5. The experimental platform is Matlab. In the process of the experiment, the current research methods and the SURF-based algorithm for identifying the static traffic sign information are compared and implemented in the following aspects:
Recognition rate of traffic sign
Recognition efficiency of traffic sign
Interference resistance in traffic sign recognition process Missing recognition rate of traffic sign Recognition
The results are as follows:
As shown in Figure 6, the traffic sign recognition based on graph model and convolution neural network (CNN) and circular sign recognition based on invariant moments and support vector machine (SVM) are in a state of constant fluctuation. The recognition rate of static traffic sign based on SURF is the highest at the initial stage of operation, although the recognition rate is a slight fluctuation in the late running stage, the overall method is more feasible than the current method.
From the Figure 7, it can be seen that the method based on SURF has little time-consuming and high efficient in identifying the static traffic sign information of highway. Before identifying the traffic sign information, the algorithm processes the icon image and implements the size normalization, which effectively improves the efficiency of identifying the traffic sign information.
Figure 8 shows that the method based on SURF has better interference resistance ability than the current method in the process of identifying the static traffic sign information of highway. It not only improves the recognition rate and efficiency, but also enhances the interference resistance ability and robustness of the recognition process.
As can be seen from Figure 9, the missing recognition rate of SURF-based static traffic sign information recognition algorithm can be effectively controlled below 3.5%.
The proposed algorithm uses traffic sign region filling to detect the region of interest, and reduce the missing recognition rate of traffic sign information as much as possible.
The weaker feature extremum point, used to remove threshold δ, is taken as the object of discussion, and the most representative feature points in the traffic sign image can be obtained by observing whether the threshold is controlled in interval [4.5, 4.6], and then the best effect of traffic sign image recognition can be obtained.
The threshold δ is distributed in interval [4.3, 4.4], [4.5, 4.6] and [4.7, 4.8], and the recognition rate of traffic sign information is as follows.
As can be seen from Figure 10, when δ is subordinate to [4.3, 4.4] and [4.7, 4.8], the recognition rate of static traffic sign information is low and fluctuates greatly. When δ belongs to [4.5, 4.6], the traffic sign recognition rate is higher. Thus, it is feasible to control the δ in [4.5, 4.6].
With the development of transportation, traffic sign recognition has gradually become a hot topic both at home and abroad. As a typical problem in the field of pattern recognition, it will also promote the further development of the theory and technology of pattern recognition. At present, the performance of traffic sign recognition needs to be improved. A new algorithm based on SURF is proposed to identify the static traffic sign information of highway. Traffic sign image preprocessing and feature extraction and matching are used to detect and recognize the traffic sign information. Experimental results show that the algorithm is reliable. The following recommendations are proposed for further research:
A vehicle decision system is constructed to provide support for traffic sign information recognition.
There are many recognition algorithms, but in this paper, only SURF matching algorithm is used to identify the traffic sign information. Thus, the next step is to realize the traffic sign recognition by other methods.
National Natural Science Foundation of China - Trust network formation and its application research in intelligent recommendation (No. 61572418); Shandong Science and Technology Development Plan (No. 2016GGX109004).
Wang L., Bai J., Application of Multi-sensor Target Identification Based on Improved Evidence Theory, Bulletin Sci. Technol., 2016, 2(7), 134-137. Google Scholar
Kong D., Sun L., Wang J.Q. et al., Lane Markings Identification Algorithm for Laser Radar, Sci. Technol. Eng., 2017, 17(16), 87-92. Google Scholar
Wang Q., Cheng R.L., Zhu X.J., Design of Gesture Recognition Simulation System Based on Web-Based Carrier, Automat. Instrument., 2017, (1), 170-173. Google Scholar
Liu Y.C., Xu F., Gesture Recognition Based on Radar Technology., J. China Acad. Electr. Inform. Technol., 2016, 11(6), 609-613. Google Scholar
Huang Z., Yu Y., Gu J., et al., An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine, IEEE Trans. Cybern., 2016, 47(4), 920-933. Google Scholar
Liu Z.W., Zhao X.M., Li Q. et al., Traffic Sign Recognition Method based on Graphical Model and Convolutional Neural Network, J. Traffic Transport. Eng., 2016, 16(5), 122-131. Google Scholar
Zhang Z., Cheng W.W., Wu L., et al., Study on Circular Traffic Signs Recognition Method based on Invariant Moments and SVM, J. Electr. Meas. Instrument., 2017, 31(5), 773-779. Google Scholar
Wang X.B., Huang J.J., LiuW.J., Traffic Sign Recognition based on Optimized Convolutional Neural Network Architecture, J. Comp. Appl., 2017, 37(2), 530-534. Google Scholar
Liu Y.C., Chen Y.P., Zhang S.S. et al., Traffic Sign Recognition Based on Pyramid Histogram Fusion Descriptor and HIK-SVM, J. Transport. Syst. Eng. Inform. Technol., 2017, 17(1), 220-226. Google Scholar
Wang Y.F., Feng Q., Deng X.J., A New Target Recognition Algorithm Based on Geometric Difference, Comp. Meas. Contr., 2016, 24(7), 156-158. Google Scholar
Zhao D., Traffic Road Sign Recognition based on Computer Vision Outline Level, Electr. Design Eng., 2017, 25(14), 123-126. Google Scholar
Tan M., Wang B., Wu Z. et al., Weakly Supervised Metric Learning for Traffic Sign Recognition in a LIDAR-Equipped Vehicle, IEEE Trans. Intel. Transport. Syst., 2016, 17(5), 1415-1427. CrossrefGoogle Scholar
Lu X.Y., Du L.J., Fuzzy Biological Image Feature Extraction Simulation Research, Comp. Simul., 2017, 34(5), 397-400. Google Scholar
Luo H., Yang Y., Tong B. et al., Traffic Sign Recognition Using a Multi-Task Convolutional Neural Network, IEEE Trans. Intel. Transport. Syst., 2018, (99), 1-12. Google Scholar
Awadalla N.S., Hanna M.A., Ismail M.N., Hassan I.A., Elkhamisy M.A., Period Variation Study and Light Curve Analysis of the Eclipsing Binary Gsc 02013- 00288, Appl. Math. Nonlin. Sci., 2016, 1(2), 321-334. Google Scholar
Rodeiro-Guerra I., Luz Hernandez-Ojeda S., Alfredo Herrera-Isidron J., Hernandez-Balmaseda I., Padron-Yaquis S., Del Rosario Olguin-Reyes S.et al., Study of the Interaction of an Extract Obtained From the Marine Plant Thalassia Testudinumwith Phase I Metabolism in Rats, Rev. Int. Contamin. Ambient., 2017, 33(4), 547-557. CrossrefGoogle Scholar
Park H.N., Choi H.A., Won S.W., Fibrous Polyethylenimine/Polyvinyl Chloride Crosslinked Adsorbent for the Recovery of Pt (Iv) From Acidic Solution: Adsorption, Desorption and Reuse Performances, J. Clean. Prod., 2018, 176, 360-369. CrossrefGoogle Scholar
Caraballo T., Herrera-Cobos M., Marín-Rubio P. An Iterative Method for Non-Autonomous Nonlocal Reaction-Diffusion Equations, Appl. Math. Nonlin. Sci., 2017, 2(1), 73-82. Google Scholar
Peng W., Ge S., Ebadi A.G., Hisoriev H., Esfahani M.J., Syngas Production by Catalytic Co-Gasification of Coal-Biomass Blends in a Circulating Fluidized Bed Gasifier, J. Clean. Prod., 2017, 168, 1513-1517. Web of ScienceCrossrefGoogle Scholar
About the article
Published Online: 2018-12-31
Citation Information: Open Physics, Volume 16, Issue 1, Pages 1149–1156, ISSN (Online) 2391-5471, DOI: https://doi.org/10.1515/phys-2018-0135.
© 2018 Shaocui Guo and Xu Yang, published by De Gruyter. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0