Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access February 7, 2023

Zebra-crossing detection based on cascaded Hough transform principle and vanishing point characteristics

  • Chen Zhu , Dong-Yuan Ge EMAIL logo , Xi-Fan Yao , Wen-Jiang Xiang , Jian Li and Yong-Xiang Li
From the journal Open Computer Science

Abstract

In this study, a zebra-crossing detection method based on cascaded Hough transform (CHT) and vanishing point (VP) characteristics is proposed. In this method, the principle of detecting straight lines in the parallel coordinate system is applied to zebra-crossing detection. Each edge point of the image obtained by edge detection is represented in the parallel coordinate system to find the VP. Using the VP coordinate as the judgment condition, those straight lines that do not pass through the VP but meet the straight-line condition are excluded to obtain the straight lines passing through both sides of the zebra crossing, and finally fit the edge points on the straight line, and get the zebra-crossing fitting line segment. Experiments show that CHT has obvious advantages in detection accuracy and speed compared with the Hough transform. At the same time, VPs can be used to eliminate interference segments, which provide support for the accuracy of zebra-crossing detection. This method can get zebra-crossing location information without using region of interest extraction, which also provides a reference method for road detection in some specific cases.

1 Introduction

Intelligent transportation is the trend of future transportation development, and automatic driving has also set off an upsurge recently. Automatic driving is often inseparable from machine vision. Image recognition using machine vision can quickly collect road information and feed it back to neural network for road automatic driving. The function of zebra crossing is to guide pedestrians to cross the road safely. When the car approaches the zebra crossing, it should slow down or stop so that pedestrians can pass safely. Therefore, zebra-crossing recognition and detection is an essential part of intelligent transportation. Zebra crossing is composed of many straight lines, so we can study it from the idea of straight-line detection. The general fitting models mainly include the following: line model, B-spline model, hyperbolic model, double model, etc., as well as the method of learning a large number of samples for line detection according to neural network [1,2,3]. At the same time, using the important characteristics of vanishing point (VP) can also help us quickly find straight-line segments. The following is a discussion of some kinds of relevant literature. Ge et al. [4] proposed a method to calibrate the intrinsic parameters of the camera by using the VP. According to the corresponding equations obtained in light of camera’s model, an orthogonal learning neural network with lateral connection is designed according to the solution requirements. Finally, the intrinsic parameters of the camera can be obtained from the equilibrium system. Tuytelaars et al. [5] proposed an aid interpretation of aerial images based on cascaded Hough transform (CHT). It presents the iterated application of the Hough transform (HT) as a means to exploit regularities of structures in aerial images of urban and suburban areas. Shi et al. [6] proposed a fast lane detection algorithm based on feature point voting statistics, preprocessed the collected image, and screened the effective edge points of lane line based on the lane line width in the region of interest (ROI). The corresponding projection calculation of the screened edge points is carried out, and the relevant voting space is established. The position of the VP is detected on the vanishing line, and the lane line is quickly detected by using the VP. Dubská [7] proposed a method to detect VPs in natural images using the CHT method in the parallel coordinate system. This method changes an infinite space into a finite space through a clever and simple linear transformation, which allows us to rasterize the space and select VP by voting similar to the voting method in the HT, and it has a good detection effect for VP in natural images. Dubská [8] also proposed a parametric method called PClines, which combines the HT with the parallel coordinate system. As a result, PClines have better accuracy and speed in line detection than traditional HT. Lin et al. [9] proposed a method to detect VP with deep learning. This method uses prior knowledge to replace deep learning, and this prior knowledge no longer needs to be learned from data, avoiding the expensive training of hardware. Jia et al. [10] proposed a method to detect the lane line. The modified method first detects the edge of the road, extracts the edge points, estimates the path parameters, determines the model function using back propagation neural network, and finally realizes the reconstruction of the lane line using the least square method. For road model detection and neural network algorithm, multiple parameters are usually required for curve fitting, which has complex operation and large amount of calculation. Although the processing effect of a single image is very good, it is difficult to meet the requirements of real-time in the moving environment. Sun and Liu [11] proposed a fast HT algorithm. This algorithm improves the detection speed by reducing the amount of computation. At the same time, the algorithm structure is modified to improve the accuracy. Zhang et al. [12] proposed a multi-camera calibration algorithm based on the mutual constraint among VPs. This method uses the geometric characteristics of the European spatial position constraints to calculate the distortion parameters and eliminate the interference of miscellaneous points through target feature point constraint and camera projective invariance. Then, the camera parameters are solved by using the consistency of the included angle between the VPs in space and the parallelism of the straight line formed by the target feature points and the reverse ray of the VPs. The calibration results by this method have a certain degree of noise immunity and stability with the increase in noise. Wang et al. [13] proposed a visualization method used for chemical process optimal design. This method uses the parallel coordinate system for multi-dimensional vector visualization, and on this basis, a scan line algorithm for multi-dimensional vector linear regression in n-dimensional space is proposed. Wang et al. [14] aimed at the problem that the speed of lane detection is slow due to the dependence of the HT and proposed a lane detection method based on the parallel coordinate system. This method uses the parallel coordinate system to visualize lines and improves the speed of line detection.

Aiming at zebra-crossing detection in road detection [15,16,17,18,19,20], this study proposes a zebra-crossing detection method based on CHT and VP characteristics. In this method, the coordinates of VP [21,22,23] are obtained by referring to the CHT. Using the condition that the zebra crossing has VP in the perspective environment, after detecting the straight line through the CHT, the interference straight line is eliminated based on whether it passes through the VP, and finally, the zebra crossing is fitted. Compared with traditional algorithms, this method has better recognition speed and accuracy. At the same time, this method can get zebra-crossing location information without using ROI extraction, which also provides a reference method for road detection in some specific cases.

2 Image processing

Image processing [24] is an important research field of computer vision. The main purpose of image processing is to retain useful image information and increase the accuracy and speed of detection. The zebra-crossing detection method in this study mainly needs to obtain information of zebra-crossing edge points, which can be realized by Gaussian filtering combined with the Canny edge detection [25]. Of course, the premise is that the image has undergone gray processing and threshold segmentation.

2.1 Gaussian filter

Gaussian filter is a filter based on Gaussian normal distribution. Its basic idea is to get the value of each pixel in the image after weighted average from the values of itself and other pixels in the field. It can be expressed by the following equation:

(1) G ( x , y ) = 1 2 π σ 2 e x 2 + y 2 2 σ 2 ,

where x and y are coordinate values, G(x, y) represents the weight on the matrix, and the size is mainly determined by σ. The larger the σ, the smoother the image, and the smaller the σ, the sharper the image. Gaussian filtering is mainly used to remove the noise in the image.

2.2 Edge detection

Image edge detection generally needs to meet two prerequisites: one is to effectively suppress the noise in the image, and the other is to make the edge information of the image as complete as possible. Therefore, it is necessary to perform threshold segmentation and filtering before edge detection. Among some edge detection algorithms, the Canny edge detection algorithm has better detection effect. It mainly consists of the following three steps: 1. use the finite difference of the first-order partial derivative to calculate the amplitude and direction of the gradient; 2. non-maximum suppression [26] is used to retain the point with the maximum local gradient; and 3. use double threshold algorithm [27] to detect edge points.

The process of edge detection is shown in Figure 1, the threshold segmentation is shown in Figure 1(a), and the image after edge detection is shown in Figure 1(b).

Figure 1 
                  Process of edge detection. (a) Threshold segmentation image and (b) edge image.
Figure 1

Process of edge detection. (a) Threshold segmentation image and (b) edge image.

Figure 1(a) is obtained from the gray image through threshold segmentation. In the experiment, the threshold is set to (200, 255). The purpose of threshold segmentation is to separate zebra-crossing pixels from the surrounding image pixels. Figure 1(b) is obtained from the threshold segmentation image through the Canny algorithm. In this image, we get many edge points of the zebra crossing. This provides the premise for the later line detection.

3 Zebra-crossing detection

3.1 VP

The VP does not exist in real space. In the camera calibration, for the parallel line in Cartesian space, when the optical axis of the camera is not perpendicular to it, its image is no longer a parallel line, but will intersect at a point in the distance, which is the VP. As shown in Figure 2, the nine edges of the three visible surfaces of the cuboid form three VPs such as H, I, and K in the image space. VP is an important feature in most real images. It can be used in camera calibration, road detection, scene reconstruction, and other tasks. At the same time, because the VP is supported by scene components, it is often very stable, so their detection is robust to various distortions. Reliable and effective detection of VPs is helpful to other computer vision tasks.

Figure 2 
                  Diagram of VP.
Figure 2

Diagram of VP.

VP detection based on HT is common in VP detection. It represents the relationship between lines and points through coordinate transformation. The initial form of HT is Gaussian sphere projection and then expressed in various forms. In this study, CHT is used to detect the VP. The algorithm can be taken as twice HT. The original infinite space is transformed into a finite diamond space through simple linear transform. In this way, we can quantify the whole space and find the VP by voting.

3.2 Principle of Hough line detection algorithm

The idea of HT is the transformation between coordinates. Taking the straight-line equation y = kx + b as an example, it is a straight line in the y–x coordinate system. Taking several points on the straight line in the k–b coordinate system represents several straight lines, and the aggregation points in the k–b coordinate system correspond to the straight lines in the x–y coordinate system. Therefore, HT can detect discrete points in the image. If they can converge into one point in the k–b coordinate system, we keep these points, set a threshold, and use the voting mechanism to filter them, so as to obtain the parameter coordinates of the straight line in the original coordinate system.

3.3 CHT

The CHT is essentially twice HT. Here, we need to introduce the concept of the parallel coordinate system. The so-called parallel coordinate system is a group of points in n-dimensional space, which can be divided into n vertically equidistant parallel lines. At this time, the points in n-dimensional space are represented as broken lines between two parallel lines, and the coordinate information has a corresponding relationship, so that information of points can be intuitively represented by lines. Next, apply this concept to CHT. First, the first S-transform transforms the points and lines in the original x–y coordinate system to the parallel coordinate system. In order to better describe the parallel line in the real environment, the homogeneous coordinate system is used here, so that [x, y, and w] represents the point in the parallel coordinate system, and (a, b, c) represents a specified line ax + by + c = 0. For the convenience of quantification, the Cartesian coordinate system u–v is established in the parallel coordinate system, and the mapping equation of points and lines is described as the following equation.

(2) S d p ( [ x , y , w ] ) = ( x + y , d w , d x ) ,

(3) S d l ( ( a , b , c ) ) = [ d b , c , a + b ] ,

where homogeneous coordinates (x, y, w) are used, and p represents point-to-line, l represents line-to-point, [] represents a point, () represents a straight line, and d is the distance between parallel coordinate axes (d > 0), as shown in Figure 3. And w is the homogeneous coefficient, thus completing the conversion from point to line. Then, according to the principle of CHT, the second HT is introduced. At this time, points are mapped to lines, and lines are mapped to points. Let D be the distance between two coordinate axes in the second parallel coordinate system, as shown in Figure 3. The mapping relationship between points and lines is described as the following equations:

(4) S S d D p ( [ x , y , w ] ) = ( S D l S d p ) ( [ x , y , w ] ) = [ d D w , d x , x + y d w ] ,

(5) S S d D l ( ( a , b , c ) ) = ( S D p S d l ) ( ( a , b , c ) ) = ( d b + c , D a + D b , d D b ) ,

where the small circle in the middle of S D l S d p is called the multiplication of the mapping. Actually, it refers to a composite mapping of two maps. For example, S d l is a line-to-point mapping, and S D p is a point-to-line mapping. Then, S D p S d l is a line-to-line mapping.

Figure 3 
                  Diagram of CHT. (a) The Cartesian coordinate, (b) the first mapping, and (c) the second mapping.
Figure 3

Diagram of CHT. (a) The Cartesian coordinate, (b) the first mapping, and (c) the second mapping.

The CHT is shown in Figure 3, the coordinates of x c and y c in Figure 3(a) correspond to x p ( v c ) and y p coordinates in Figure 3(b). The coordinates of u c and v c ( x p ) in Figure 3(b) correspond to v p ( p c ) and u p coordinates in Figure 3(c).

After the transformation of S S d D , points in the Cartesian frame map to parallel coordinates. The transformation type is similar to the Hough coordinate transformation, as shown in Figure 4(a). Three points (A, B, and C) on a line l 2 under the Cartesian frame are mapped to three lines ( A x A y , B x B y , and C x C y ) in the new frame, where the intersection ( l ¯ 2 ) is the representation point of the original line ( l 2 ). However, lines with a slope greater than zero cannot be represented in S space. As shown in Figure 4(b), two points (A and B) of line L with a slope greater than 0 in the Cartesian frame and through one or four quadrants correspond to lines ( A x A y and B x B y ) in S space, but there is no intersection between two straight lines in S space, so you cannot represent a straight line L 1, and line L 2 with a slope greater than zero also cannot fall in S space. So, you need to create a space with a distance of d and in the opposite direction to y, where A x A y and B x B y are the two points on the straight line of L 2 in the Cartesian coordinate. With this space, you can represent a line with a slope greater than zero.

Figure 4 
                  Diagram of coordinate transformation. (a) Hough coordinate transformation and (b) TS space.
Figure 4

Diagram of coordinate transformation. (a) Hough coordinate transformation and (b) TS space.

S and T space can define all the lines in the x–y coordinate system. The transformation formula of T space is described as the following equations:

(6) T d p ( [ x , y , w ] ) = ( x + y , d w , d x ) ,

(7) T d l ( ( a , b , c ) ) = [ d b , c , a b ] .

The four equations for the point-to-point formulation of the CHT are described as the following equations:

(8) S S d D P ( [ x , y , w ] ) = ( S D l S d p ) ( [ x , y , w ] ) = [ d D w , d x , x + y d w ] ,

(9) S T d D P ( [ x , y , w ] ) = ( S D l T d p ) ( [ x , y , w ] ) = [ d D w , d x , x + y + d w ] ,

(10) T S d D P ( [ x , y , w ] ) = ( T D l S d p ) ( [ x , y , w ] ) = [ d D w , d x , x + y d w ] ,

(11) T T d D P ( [ x , y , w ] ) = ( T D l T d p ) ( [ x , y , w ] ) = [ d D w , d x , x + y + d w ] .

The x–y coordinate system (an infinite space) is transformed to a finite diamond according to CHT, which is shown in Figure 5, and there are corresponding relative between the two coordinates systems, for example the first quadrant (orange) of the Cartesian coordinate system in Figure 5(a) is corresponding to the third quadrant (orange) of the parallel coordinate system in Figure 5(b), i.e. the four quadrants between the two coordinate systems are corresponding respectively. Thus the line passing through three quadrants in the x–y coordinate system can be represented by the three broken lines after the CHT, which is uniquely determined by the four endpoints. Because the VP is the intersection of all lines in the same direction and has the maximum weight, it can be selected by voting. Finally, equations (12) and (13) are used to obtain the coordinates in its x–y coordinates.

(12) [ x , y , w ] o [ d D w , d x , sgn ( x y ) x + y + sgn ( y ) d w ] d ,

(13) [ x , y , w ] d [ D y , sgn ( x ) d x + sgn( y ) d y d D w , x ] o ,

where sgn(x) represents the step function, when x > 0, sgn(x) = 1, when x < 0, sgn(x) = −1, when x = 0, sgn(x) = 0. Note that when a or b equals 0, the expression equals 0, then requires sgn(x) = 1. The subscript o represents the original plane, and the subscript d represents the diamond space.

Figure 5 
                  Diagram of diamond space. (a) Cartesian coordinate system and (b) parallel coordinate system.
Figure 5

Diagram of diamond space. (a) Cartesian coordinate system and (b) parallel coordinate system.

3.4 Principle of parallel coordinate system

In the parallel coordinate system, the T space is the area between the −y axis and x axis, and the S space is the area between the x axis and y axis. We set the Cartesian coordinate system for the line l: ax + by + c = 0 (except for any line parallel to the Y axis), any point on it can be expressed as x , a b x c b , and we set three points A x 1 , a b x 1 c b , B x 2 , a b x 2 c b , and C x 3 , a b x 3 c b on the straight line. We add a parallel coordinate system to the Cartesian coordinate system, so that the x axis in the parallel coordinate system is located at the y-axis position of the Cartesian coordinate system, representing d as the parallel coordinate system axis spacing as shown in Figure 3. When the slope is less than 0, the transformed lines corresponding to three points A, B, and C will intersect in S space to obtain the line equation. The line equation is described as the following equation:

(14) y = a b 1 x i c b d x + x i .

The intersection coordinate is now d b a + b , c a + b .

Similarly, when the slope is greater than 0, the transformation lines corresponding to the three points of A, B, and C will intersect in T space, and the line equation of transformation is described as the following equation:

(15) y = 1 + a b x i + c b d x + x i .

The intersection coordinate is now db a b , c a b .

The principle of the parallel coordinate system for straight-line detection is applied to the zebra-crossing detection, the edge detection images are represented in the parallel coordinate system to find the VP and use the VP coordinates as the judgment condition, the lines that do not pass through the disappearance point but meet the condition of forming a straight line are eliminated, and the lines on both sides of the zebra crossing are obtained. Finally, the edge points on the straight line are fitted to obtain the zebra-crossing fitting line section.

3.5 Error correction of zebra line image

In general, we will get most image lines of zebra crossings in one direction on the premise of obtaining correct VP coordinates. However, in the experiment, some image lines of zebra crossings may fail to pass the VP due to errors. To solve this problem, we designed a correction method for some error image lines of zebra crossings. First, we will judge whether it meets the threshold setting of image lines. Second, we will judge whether these point sets are located below the Y coordinate of the VP. Third, we set a circular area centered on the VP. (The radius of area circle set in this study is 7.1 pixel.) Only the image lines passing through this area will be confirmed. Finally, the VP coordinate and the mean value coordinate of the point sets of image line are formed into a new line, which is regarded as the image line of the zebra crossing. However, the original point set is still used to fitting the zebra crossing. After completing these judgments, other interference lines will be eliminated. Flow chart of zebra-crossing error correction is shown in Figure 6.

Figure 6 
                  Flow chart of zebra-crossing error correction.
Figure 6

Flow chart of zebra-crossing error correction.

4 Experimental results and analysis

4.1 Comparison between HT and CHT algorithms

The experiment is simulated in MATLAB. The experiment uses a group of zebra-crossing pictures taken in real life to carry out the experiment. (The size of the picture is 3,024 × 4,032 × 3.) The primary goal of zebra-crossing detection is to accurately detect zebra crossings and achieve real-time performance on the premise of ensuring high accuracy. Through the experiment and analysis of traditional HT and CHT, it is concluded that both algorithms can detect zebra crossings. The experiment uses four scenes with different frames for testing. The results of experiment are shown in Table 1. Table 1 shows that the CHT algorithm has higher correct frames than the HT algorithm.

Table 1

Frame number comparison of zebra-crossing detection results

Scene Frame (f) HT (f) CHT (f)
1 600 510 560
2 1,000 900 906
3 1,250 1,125 1,200
4 1,500 1,290 1,356

By comparing the accuracy and speed of the two algorithms in Tables 1 and 2 (the results in Table 2 are obtained from Table 1), it is concluded that the accuracy and speed of the CHT algorithm are improved compared with the traditional HT.

Table 2

Accuracy and recognition speed of zebra-crossing detection results

Item The HT algorithm The CHT algorithm
Accuracy (%) 87.8 92.6
Speed (f s−1) 21 40

4.2 VP detection results

VP is the intersection of two or more parallel lines at infinity under the perspective of three-dimensional space. The zebra crossing is composed of many parallel line segments, so under the perspective condition, the line corresponding to each line segment will intersect at a point (VP) in the distance. Through the experiment, the VPs in the four images are obtained, as shown in Figure 6, and the position of the VPs is represented by red dots in the image. (VP in the Figure is the abbreviation of VP.)

The coordinate positions of the VPs of the four scenes (a)–(d) in Figure 7 are shown in Table 3. The third column in Table 3 represents the distance between the optical center and VPs.

Figure 7 
                  Results of VP detection. Zebra crossing and VP in scene (a), zebra crossing and VP in scene (b), zebra crossing and VP in scene (c), and zebra crossing and VP in scene (d).
Figure 7

Results of VP detection. Zebra crossing and VP in scene (a), zebra crossing and VP in scene (b), zebra crossing and VP in scene (c), and zebra crossing and VP in scene (d).

Table 3

VP coordinates in four scenes in Figure 6

Item VP coordinates/pixel Distance/pixel
Figure 6(a) (2. 0357 × 103, 0. 9980 × 103) 514.3774
Figure 6(b) (2. 0691 × 103, 1. 3054 × 103) 213.3147
Figure 6(c) (1. 7529 × 103, 0. 8948 × 103) 670.9377
Figure 6(d) (2. 1019 × 103, 1. 1783 × 103) 344.5787

Experiments show that light and pattern integrity have some influence on the accuracy of VP detection. The detection effect of strong light and dark road surface is obviously not as good as that of normal light road surface, and the image detection accuracy with large incomplete pattern or position offset pattern will also be greatly reduced.

4.3 Zebra-crossing detection result

We take Figure 7(a) as an example.

The result of zebra-crossing detection is shown in Figure 8. Figure 8(a) shows the zebra-crossing image after detecting the VP coordinates, eliminates the straight lines that do not pass through the VP, and sets the straight-line threshold to obtain each straight line passing through the zebra crossing. Figure 8(b) shows that we fit the edge points on the zebra-crossing straight line and finally obtain the fitting line segment of the zebra crossing. (Both sides of the zebra crossing are represented by green lines.)

Figure 8 
                  Result of zebra-crossing detection. (a) Retained green lines and (b) zebra-crossing detection result.
Figure 8

Result of zebra-crossing detection. (a) Retained green lines and (b) zebra-crossing detection result.

In Figure 8(b), the straight line equations of zebra crossings are extracted after zebra-crossing fitting, as shown in Table 4 (zebra crossings are ordered from left to right).

Table 4

Line equation of zebra-crossing line segment

Number VP coordinates Equation of the straight lines
1 (2.0357 × 103,0.9980 × 103) 0.4590 × 103 x + 1.7077 × 103 y − 2.6387 × 106 = 0
2 0.4710 × 103 x + 1.6097 × 103 y − 2.5653 × 106 = 0
3 0.4710 × 103 x + 1.4457 × 103 y − 2.4016 × 106 = 0
4 0.4890 × 103 x + 1.3317 × 103 y − 2.3245 × 106 = 0
5 0.5040 × 103 x + 1.1747 × 103 y − 2.1984 × 106 = 0
6 0.5130 × 103 x + 1.0337 × 103 y − 2.0760 × 106 = 0
7 0.5150 × 103 x + 0.8027 × 103 y − 1.8495 × 106 = 0
8 0.5210 × 103 x + 0.6497 × 103 y − 1.7090 × 106 = 0
9 0.5570 × 103 x + 0.4457 × 103 y − 1.5787 × 106 = 0
10 0.5840 × 103 x + 0.2967 × 103 y − 1.4850 × 106 = 0
11 0.6230 × 103 x + 0.0427 × 103 y − 1.3109 × 106 = 0
12 0.6310 × 103 x − 0.1403 × 103 y − 1.1446 × 106 = 0
13 0.6110 × 103 x − 0.3963 × 103 y − 8.4836 × 105 = 0
14 0.6170 × 103 x − 0.5783 × 103 y − 6.7895 × 105 = 0
15 0.6050 × 103 x − 0.8263 × 103 y − 4.0702 × 105 = 0
16 0.6020 × 103 x − 0.9923 × 103 y − 2.3525 × 105 = 0
17 0.5780 × 103 x − 0.1202 × 103 y + 2.3183 × 105 = 0
18 0.5750 × 103 x − 1.3663 × 103 y + 1.9296 × 105 = 0
19 0.5360 × 103 x − 1.5023 × 103 y + 4.0808 × 105 = 0
20 0.5130 × 103 x − 1.5953 × 103 y + 5.4770 × 105 = 0

5 Conclusion

In terms of image processing, a series of image processing algorithms are adopted, by setting the threshold that can automatically process the input image, with small manual participation. In this study, the principle of VP detection by CHT is used to detect the zebra crossing. Experiments show that using CHT algorithm to detect straight lines and VP has obvious advantages over HT algorithm in terms of detection accuracy and detection speed. At the same time, the VP can be used to eliminate the interference line segment to provide support for the accuracy of zebra-crossing detection. This method can get zebra-crossing location information without ROI extraction, which brings a reference method for road detection in some specific cases. Of course, if ROI is used to extract a detection target including VPs and the whole area of the zebra crossing, the detection efficiency will be improved. After that, we will also conduct further research in eliminating image noise and improving the overall detection accuracy and speed of the algorithm.

Acknowledgements

This work was supported by Innovation Project of Guangxi Graduate Education under Grant No. YCSW2022438, the National Natural Science Foundation of China under Grant Nos. 51765007 and 81960332, and Hangzhou Science and Technology Bureau Project under Grant No. 20191203B27.

  1. Author contributions: Chen Zhu, Dong-Yuan Ge, Xi-Fan Yao wrote the manuscript, and Wen-Jiang Xiang, Jian Li, and Yong-Xiang Li revised the manuscript.

  2. Conflict of interest: Authors state no conflict of interest.

  3. Data availability statement: This article does not cover data research. No data were used to support this study.

References

[1] M. X. Tan, R. M. Pang, and Q. V. Le, EfficientDet: Scalable and efficient object detection, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, 10778–10787.10.1109/CVPR42600.2020.01079Search in Google Scholar

[2] W. Y. Li, X. Y. Liu, and Y. X. Yuan, “SIGMA: Semantic-complete graph matching for domain adaptive object detection,” CVPR2022 (ORAL presentation), vol. 31, 2022, arXiv:2203.06398v3 [cs.CV].10.1109/CVPR52688.2022.00522Search in Google Scholar

[3] S. Li, C. H. He, R. H. Li, and L. Zhang, “A dual weighting label assignment scheme for object detection,” CVPR2022, vol. 18, 2022, arXiv:2203.09730 [cs.CV].10.1109/CVPR52688.2022.00917Search in Google Scholar

[4] D.Y. Ge, X.F. Yao, W.J. Xiang, E. C. Liu and Y. Z. Wu, “Calibration on camera’s intrinsic parameters based on orthogonal learning neural network and vanishing points,” IEEE Sens. J., vol. 20, no. 20. pp. 11856–11863, 2020.10.1109/JSEN.2019.2957134Search in Google Scholar

[5] T. Tuytelaars, L.V. Gool, M. Proesmans and T. Moons, The cascaded Hough transform as an aid in aerial image interpretation, Sixth International Conference on Computer Vision, 1998, pp. 67–72.10.1109/ICCV.1998.710702Search in Google Scholar

[6] L. Shi, D. K. Tan, N. H. Wang, N. Yuan and Y. Y. Zhang, “A fast lane detection algorithm based on feature point voting statistics,” Chin. J. Automot. Eng., vol. 8, no. 3, pp. 189–195, 2018.Search in Google Scholar

[7] M. Dubská, “Real projective plane mapping for detection of orthogonal vanishing points,” BMVC, 2013.10.5244/C.27.90Search in Google Scholar

[8] M. Dubská, “PClines – Line detection using parallel coordinates,” BMVC, 2011.10.1109/CVPR.2011.5995501Search in Google Scholar

[9] Y. C. Lin, R. Wiersma, S. L. Pintea, K. Hildebrandt, E. Eisemann and J. C. Gemert, “Deep vanishing point detection: Geometriors make dataset variations vanish,” CVPR2022, vol. 16, 2022, arXiv:2203.08586 [cs.CV].10.1109/CVPR52688.2022.00601Search in Google Scholar

[10] H. Q. Jia, Z. H. Wei, X. He and M. Y. Li, “A research on lane marking detection algorithm based on neural network and least squares method,” Automot. Eng., vol. 40, no. 3, pp. 363–368, 2018.Search in Google Scholar

[11] F. R. Sun and J. R. Liu, “Fast Hough transform algorithm,” Chin. J. Comput., vol. 10, no. 24. pp. 1103–1109, 2001.Search in Google Scholar

[12] C. Zhang, H. M. Yang, C. Han and F. Yang, “Multi-Camera calibration based on vanishing point constraints,” Acta Photonica Sin., vol. 45, no. 5, p. 0512004, 2016. (pp. 1–7).10.3788/gzxb20164505.0512004Search in Google Scholar

[13] S. M. Wang, X. J. Sun, K. F. Wang and P. J. Yao, “Optimal design by visualization using parallel coordinate system,” Comput. Appl. Chem., vol. 21, no. 1, pp. 11–15, 2004.Search in Google Scholar

[14] X. C. Wang, X. C. Lu, H. S. Zhang, Y. M. Xiao and M. Xie, “A lane detection method based on parallel coordinate system,” J. Univ. Electron. Sci. Technol. China, vol. 47, no. 3, pp. 362–367, 2018.Search in Google Scholar

[15] W. Xu and Y. J. Zhai, “Cross-modal biometric fusion intelligent traffic recognition system combined with real-time data operation,” Open. Comput. Sci., vol. 12, no. 1, pp. 332–344, 2022, 10.1515/comp-2022-0252 Search in Google Scholar

[16] S. Q. Yang, Y. H. Ma, M. Y. Fang, W. X. Qian, J. X. Cai and T. Liu, “Lane detection method in complex environments based on instance segmentation,” J. Zhejiang Univ. (Eng. Sci.), vol. 56, pp. 809–832, 2022.Search in Google Scholar

[17] X. H. Shi, J. Q. Wu, C. G. Wu, S. Cheng, X. H. Weng and Z. Y. Chang, “Residual network based curve enhanced lane detection method,” J. Jilin Univ. (Eng. Technol. Ed.), pp. 1–9, 2022, 10.13229/j.cnki.jdxbgxb20210618 Search in Google Scholar

[18] N. J. Zakaria, M. I. Shapiai, H. Fauzi, H. M. A. Elhawary, W. J. Yahya, M. A. A. Rahman, et al., “Gradient-based edge effects on lane marking detection using a deep learning-based approach,” Arab. J. Sci. Eng., vol. 12, pp. 10989–11006, 2020.10.1007/s13369-020-04918-4Search in Google Scholar

[19] N. Kanagaraj, D. Hicks, A. Goyal, S. Tiwari and G. Singh, “Deep learning using computer vision in self driving cars for lane and traffic sign detection,” Int. J. Syst. Assur. Eng. Manag., vol. 6, pp. 1011–1025, 2021.10.1007/s13198-021-01127-6Search in Google Scholar

[20] Y. N. Chen, H. Dai and Y. Ding, “Pseudo-stereo for monocular 3D object detection in autonomous driving,” CVPR2022, vol. 4, 2022, arXiv:2203.02112 [cs.CV].10.1109/CVPR52688.2022.00096Search in Google Scholar

[21] J. N. Zhang, Q. X. Su, P. Y. Liu, Z. J. Wang and H. Q. Gu, “Adaptive monocular 3D object detection algorithm based on spatial constraints,” J. Jilin Univ. (Eng. Sci.), vol. 54, no. 6, pp. 1138–1146, 2020.Search in Google Scholar

[22] G. F. Alvaro, F. M. Fernando, K. Ricardo, S. S. Ruben and D. D. M. Fernando, “A multi-threshold approach and a realistic error measure for vanishing point detection in natural landscapes,” Eng. Appl. Artif. Intell., pp. 713–726, 2019.10.1016/j.engappai.2019.08.001Search in Google Scholar

[23] B. Sana, H. Faycal and M. Abdellatif, “Vehicles distance estimation using detection of vanishing point,” Eng. Comput., vol. 9, pp. 3020–3093, 2019.Search in Google Scholar

[24] D. Y. Ge, X. F. Yao and W. J. Xiang, “Application of BP neural network and genetic algorithm in testing of micro-drill’s rounded corner,” Sens. Lett., vol. 9, no. 5, pp. 1943–1947, 2011.10.1166/sl.2011.1556Search in Google Scholar

[25] Q. Chen, L. X. Zhu and D. S. Xia, “Image binarization based on Canny’s operator,” J. Comput. Des. Comput. Graph., vol. 17, no. 6, pp. 1302–1306, 2005.Search in Google Scholar

[26] J. Zhang, Q. Y. Qi, H. L. Zhang, Q. Du, Z. Guo and Y. Y. Tian, “Detection of bird’s nest on transmission lines from aerial images based on deep learning model,” Int. J. Innov. Comput Inf. Control., vol. 18, no. 6, pp. 1755–1768, 2022.Search in Google Scholar

[27] D. A. Tyas, I. Candradewi, Baskara, N. P. Indarto, H. Abdurrahman, Y. Argha et al. “Horizon detection for UAV attitude based on image processing approach,” ICIC Express Letters, vol. 16, no. 12, pp. 1249–1258, 2022.Search in Google Scholar

Received: 2022-08-03
Revised: 2022-09-29
Accepted: 2022-10-15
Published Online: 2023-02-07

© 2023 the author(s), published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 29.3.2023 from https://www.degruyter.com/document/doi/10.1515/comp-2022-0260/html
Scroll Up Arrow