Research on QR image code recognition system based on artificial intelligence algorithm

1 Introduction

With the advent of the digital information age, Internet technology promotes the universal use of QR codes in real life, which greatly facilitates people’s daily life. With the rapid development of two-dimensional codes, the problem of image recognition of two-dimensional codes has also received extensive attention [1]. Due to its low cost, large amount of stored information, and the ability to scan without attaching to a database, the two-dimensional code quickly occupied the Chinese consumer market. Since the twentieth century, two-dimensional barcodes have been favored by people for their low cost, high error correction rate, and fast recognition speed. In the past 100 years, the development of the Internet has pushed the use of QR codes to a climax. It has penetrated into people’s daily lives. The application of 2D barcodes is not limited by equipment and time. It only needs to combine Internet technology with terminal technology, which promotes the widespread use of 2D barcodes and contributes to technology and modernization [2,3]. The basic building blocks of QR code recognition consisting of various indicators and markers are shown in Figure 1.

Figure 1

Basic building blocks of QR code recognition.

QR codes can be categorized as a 2D matrix code, which is a typical square-shaped code that can be determined by its dimensions and variations. As shown in Figure 1, the QR code can be structured into different modules like position markers, timing patterns, version number, format identifier, alignment marker, and data indicator [4,5].

1. Position markers: This is the position detection indicator of a QR code, which is represented by a small square combined of a lighter and a darker square. It determines the position and orientation of the QR code.

2. Timing pattern: They are the interconnected patterns formed by the alteration sequence of dark and light elements. They are responsible for determining the size, the number of rows and columns, and identification of distortion present in the QR code.

3. Version number: This indicator in the QR code helps to identify the version number of the code.

4. Format identifier: This indicator contains the information regarding the mask pattern number and the error correction level, which are needed to decode the QR code for identifying the type of content like URL, text, image, etc.

5. Alignment marker: The alignment markers determine the possibility of distortion in the QR code by identifying the point of alignment in the code.

6. Data indicator: The data encoded inside the QR code is decoded by the usage of this indicator. If the QR code is damaged, still it can be restored and read by using the error correction method [6].

QR code recognition technology is a hot topic in the field of digital image processing [7,8]. With the continuous development of the Internet of things, QR code has been gradually applied in various industries due to its strong information storage capacity, convenient and quick reading advantages, and safe and reliable coding technology [9,10]. At the same time, the two-dimensional code reading equipment is also developing toward the trend of intelligence, miniaturization, and networking. Therefore, the research on the QR image code recognition system based on artificial intelligence algorithm has great and far-reaching significance [11].

This study helps in combatting the various challenges faced in QR code recognition by proposing an improved adaptive median filter algorithm and a QR code distortion correction method based on backpropagation (BP) neural networks. The ensemble of artificial intelligence algorithms with image processing provides an efficient approach, which is capable of fitting the distorted QR image into the geometric deformation pattern. This study addresses an important research issue of two-dimensional code distortion, which was a serious research problem in the existing software systems. This article mainly emphasizes the preprocessing stage of the image retrieval, and significant improvement has been observed for the reading rate of the QR image code. The artificial intelligence algorithm improves the recognition rate of the two-dimensional code image.

This article has been organized as follows: Section 2 comprises the literature survey of state-of-the-art approaches in the QR image recognition domain. The research methods are elaborated in Section 3 of this article followed by the results and discussion in Section 4. Section 5 provides the conclusion and future perspective of this study.

2 Literature review

Foreign research on two-dimensional code technology began in the 1980s. Because it overcomes many limitations of one-dimensional bar codes, it has received attention and research from many countries in the world. The United States, Japan, South Korea, and other countries have applied two-dimensional barcodes to various documents, bills, embedded integrated circuits, warehousing and logistics, and pharmaceutical industries. Bar code equipment manufacturers such as SYMBOL and Zebra in the United States and Option in Japan have developed and produced a variety of multifunctional portable QR code reading equipment [12] with innovative technology.

Due to the late start of the research on 2D barcodes in China and many other developed countries, most of the 2D codes are developed by foreign countries. Therefore, we need to continue to improve the key technologies for the recognition of various QR codes to make the encoding of the domestic QR codes. The rules and recognition of technology have real independent intellectual property rights, and the improvement of the existing QR code image recognition technology is also very important.

The research of QR code recognition algorithms has always been a hot research topic in the field of image processing. QR codes are read based on digital image-processing algorithms. However, due to various reasons during the image acquisition process, it is easy to cause QR code image backgrounds to often exist and identify barcode-independent noise, and the acquired QR code image may have geometric distortion, or the background of the QR code image is very complicated, which makes it difficult to read the QR code with general purpose equipment, and the accuracy of decoding is greatly reduced. Therefore, how to properly process and correct the collected QR code images is a key technical issue for QR code reading [13].

The choice of gray threshold is the key to the success of binarization. Luiz F. F. proposed an improved correction algorithm based on background gray estimation for QR images with uneven illumination, and the experimental effect was significantly improved. Jiang and Wu conducted in-depth research on the two-dimensional code binarization algorithm and positioning recognition system and proposed improvements [14]. Xiaodan, Kaz, Hanaizumi, and others have improved the QR code encoding and decoding algorithms. Ohbuchi and others designed a two-dimensional QR code recognizer using a mobile phone camera.

In China, Wu Jiapeng et al. used the maximum between-class variance method to binarize the QR code image, and then directly applied gradient sharpening to the binary image to extract the edge of the QR code and then reduced the resolution. This method achieves the QR code image processing better, but it is not applicable to the complex environment and uneven illumination. Yan et al. used Hough transform to rotate the QR code image to a horizontal level and then used the edge detection algorithm and projection algorithm to calculate the boundary of the QR code module. This method is less effective when there is long line interference near the QR code. Liu Zhi et al. proposed a method based on convex hull to locate the boundary of a two-dimensional QR code. But it is only suitable for the recognition of QR code images without background interference and is not suitable for large-scale detection [15]. The authors in ref. [16] analyzed the binary images by scanning them horizontally and vertically to find all the combinations and angles of position marker finder patterns. The connected component analysis was done by Li et al. [17] followed by run-length encoding to identify the row-by-row ratio of position marker finder patterns. The classifier cascading concept was utilized by Belussi and Hirata [18] to identify the patterns of QR code, which are further analyzed for the identification of subgroups among them. Bodnár and Nyúl [19] utilize the combination of local binary patterns, Haar transform, and histogram-oriented gradients to find the patterns indicative of QR codes. However, the training process becomes time consuming in this case due to the collection of so many complex algorithms. Tribak and Zaz [20] scans the horizontal and vertical patterns of the QR code trailed by the principal component analysis (PCA) for the removal of false positives in the algorithm. Furthermore, these authors in another article utilized the invariant moment features for feature attribute description instead of using the PCA method [21]. This provides improved performance as the feature attributes are obtained by using the Euclidean distance method. Ciążyński and Fabijańska [22] utilized the window histogram approach and histogram-oriented gradients (HOG) for the identification of local features. This finds the angles between the two main gradient directions of the QR code; however, Szentandrási et al. [23] used the probability score estimation of chessboard-like structure to find the edge pixels in the QR code images. A canny edge detector-based approach was proposed by Gaur and Tiwari [24] along with the morphological operations for QR code detection. Suran [25] proposed a method, which collectively ensembles the Harris corner detection and convex hull algorithm to estimate the boundaries of the quadrilateral of a QR code. A novel algorithm has been proposed by Sun et al. [26] combining the Canny edge detection with the contour filling method to find the QR Code area by detection of four corners of the 2D QR barcode. Hansen et al. [27] proposed a deep learning-based methodology for finding the barcode. Zharkov and Zagaynov [28] proposed a fast and robust deep learning based method for the detection of 1D and 2D barcodes on the basis of sematic segmentation. Similarly, convolutional neural network-based architectures were also utilized in ref. [29,30] for the detection of QR codes and the pattern analysis.

The innovation of this article is based on the artificial intelligence algorithm. First, combined with the characteristics of the BP neural network, the neural network is used to learn the sample image, then the distorted image is fitted to the geometric deformation pattern, and finally the image is restored [31].

3 Research methods

3.1 Analysis of QR code recognition process

The information of the QR code is stored in a matrix, so it belongs to a matrix two-dimensional barcode, and a matrix two-dimensional barcode is generally composed of regular analytical graphics modules such as circles, squares, and hexagons. Graphics is the most convenient and effective way to read the information by camera. However, the use of camera-based reading methods to obtain the information of the QR code image will also cause uneven illumination and image distortion, which directly leads to the failure of the QR code image to be correctly interpreted or the loss of important information. Therefore, it is necessary to carry out various preprocessing works before QR code decoding to ensure the smooth progress of the subsequent decoding, thereby improving the accuracy of QR code reading.

The QR image recognition process starts with image acquisition, and then, a series of operations are performed, such as image preprocessing, barcode detection, information sampling, information error correction, information decoding, and finally output results. The common operation flowchart of QR code recognition is shown in Figure 2.

Figure 2

Common operating procedures for QR code identification.

In the entire QR code recognition process, image preprocessing is the core part of the whole process. There are fixed rules for information sampling, information error correction, and information decoding. After the image preprocessing is completed, the decoding can be performed according to the rules [32]. Therefore, this article focuses on the implementation of the image preprocessing method.

3.3 QR code distortion correction method based on BP neural network

3.3.1 BP algorithm flow

1. Initialization: a series of parameters in the neural network are randomly generated during the network initialization process.

2. Determine the learning sample of the neural network, the expected output of the sample, and the selected excitation function.

3. Learn to calculate the actual output through the neural network.

4. According to the global error and the local error of each layer, the connection weight and the threshold between each node are corrected.

5. Continue to input samples and repeat the aforementioned operations for learning. Once the error meets the requirements, the learning ends.

The specific algorithm flow chart is shown in Figure 3.

Figure 3

BP algorithm flow.

3.3.2 QR code distortion correction based on BP neural network

In this article, the basic idea of using the BP neural network to process QR code images is to find the polynomial relationship between these pixel coordinates through the self-learning ability of the BP neural network [34]. First, consider the point coordinate set on the distorted QR code image as the input layer node of the neural network, and the corresponding output layer node is the point coordinate set on the standard image corresponding to the distorted QR code image. The same operation is taken for multiple images, thus forming a set of learning sample data sets. Through the learning of the learning sample set, the neural network forms a distortion mode from point coordinates to point coordinates. This distortion mode is the source image. The polynomial relationship is investigated between the corresponding pixel point coordinates in the distorted image. After the learning mode is formed, the neural network can be used for distortion processing while processing images with similar QR code distortion in the future. Figure 4 illustrates with pictures as an example.

Figure 4

Sample picture collection indicated by capital (A–D) and the corresponding QR codes indicated by smalll (a–d).

As shown in the QR code image, Figure 4((A and a), (B and b), (C and c), and (D and d)) is called sample pair images, and the BP neural network is used to analyze the distorted images in each pair of images. The standard image learning forms a learning mode, and then, the similar distortion image can be processed for distortion correction. This is just for supplementary explanation, so only a few image pairs are listed. A lot of sample images are needed as learning samples while BP neural network learning. In the learning process, paired images are used for learning. The specific distortion correction process will be introduced in detail later.

In the learning process of the BP network in this article, the sample target pairs are input one by one and calculated by formulas (1) and (2) ω j i with Δ ω j i . It is iterated repeatedly to minimize the mean square error between the output value of the system and the target value. The BP network in this article can use the fastest gradient descent method in the weight adjustment process, that is, the adjustment of the network weight is carried out along the opposite direction of the gradient.

(1) w j i ( k + 1 ) = w j i ( k ) + Δ w j i ( k ) = w j i ( k ) + η δ j ( k ) y i ( k ) ,

(2) δ j ( k ) = f ′ j ( v j ( k ) ) ⋅ ( d j ( k ) − y j ( k ) ) , j is the output node f ′ j ( v j ( k ) ) ⋅ ∑ n δ p ( k ) w p j ( k ) , j is the hidden layer node,

where f ′ j ( ⋅ ) is the transfer function of the neuron, f ( ⋅ ) is the derivative, d j ,   y j is the target output and actual output of the neuron, v j ( k ) are neurons, and j is the input, namely, v j ( k ) =   ∑ i w j i ( k ) y i ( k ) .

The flow chart of QR image processing with the BP neural network is shown in Figure 5.

Figure 5

Flow chart of QR image processing with the BP neural network.

4 Results and discussion

The primary aim of this chapter is to verify the effectiveness of the algorithm proposed through the preprocessing of QR code images. The preprocessing process includes grayscale, filtering, binarization, distortion correction, and perspective projection inverse transformation. The filtering uses an improved adaptive median filter algorithm to filter the image, and the distortion correction uses a distortion correction method based on the BP neural network.

4.2 Experimental data collection

In the process of data collection in this article, first, the QR code image is taken by the mobile phone camera function and the normal QR code image is subjected to distortion simulation to obtain the QR sample image set. The process of the experiment is shown in Figure 6.

Figure 6

Test flow chart.

The pretreatment process is explained step by step in Figure 7.

Figure 7

QR image of experimental sample.

The collected color image is shown in Figure 8(a), and first, the gray-scale processing is performed to obtain the gray-scale image (Figure 8(b)).

Figure 8

The comparison of image (a) before grayscale conversion and (b) after the grayscale result.

From the comparison of Figure 8(a) and (b) in Figure 8, it can be clearly seen that the image after gray-scale processing has removed the color. However, there is the salt-and-pepper noise on the image that affects the quality of the image. The improved adaptive median filter algorithm is used to filter the image in Figure 9(a), and the processing result is shown in Figure 9(b).

Figure 9

Comparison of effect diagrams processed by the improved filtering algorithm: (a) Before filtering and (b) After applying improved filtering algorithm.

As shown in Figure 9(b), the filtered image almost completely eliminates the influence of salt-and-pepper noise, but the real QR code image only contains pixels with pixel values “1” and “0.” Therefore, the binarization process is required. Here, the OSTU (Maximum Inter-Class Variance) algorithm is used to binarize the image. The processing result is shown in Figure 10(b).

Figure 10

Binarization effect diagram: (a) Before binarization and (b) After binarization.

After the binarization operation, only the points with pixel values “1” and “0” remain in the image. The image shown in Figure 10(b) has serious distortion, and then the image is deformed by the distortion correction method, and the resulting image is processed through the ecological closing operation. The result is shown in Figure 11(b).

Figure 11

The result of distortion correction processing: (a) Before distortion correction and (b) After applying distortion correction processing.

As shown in Figure 11(b), the image processed by the algorithm proposed in this article has been greatly restored in terms of surface distortion correction, but there has been a serious perspective phenomenon, and there is a perspective phenomenon. The QR image of this will bring great difficulties or even cannot be read when reading information. Therefore, before reading the information, the image should be processed by the perspective inverse transformation. When processing the perspective transformation, the perspective projection inverse transformation process is used for processing. The result obtained through this process is shown in Figure 12(b).

Figure 12

The image after perspective correction.

Compared with the original image Figure 12(a), the image shown in Figure 12(b), after the inverse perspective transformation, has been greatly improved from an intuitive point of view. However, the aforementioned improvement cannot explain the effectiveness of the algorithm proposed in this article.

5 Conclusion

This article is mainly focused at the identification of QR codes that are more severely distorted, especially those printed on objects that are prone to wrinkles. In this case, the QR code recognition rate is very low or cannot be recognized at all. As an excellent two-dimensional code, the application of QR code in trademarks is bound to be a major development trend. As a result, it is of great significance to improve the recognition rate of the QR code in special circumstances, and the improvement of the recognition rate can also greatly promote the application of QR codes. Based on standard median filtering and classical adaptive median filtering, this article proposes an improved adaptive median filtering algorithm. The experimental results show that the algorithm has a good filtering and denoising effect even when the image has a high noise density. In addition, the image after filtering and denoising by this algorithm is also slightly prominent in terms of protecting details. A detailed analysis of the image preprocessing work of QR code symbols before the QR code decoding work is done followed by a detailed analysis and research on the current QR code image preprocessing methods. The application of all the algorithms are compared for correcting the image distortion, especially in the identification of QR codes printed on the surface of the items that are prone to wrinkles. The algorithm for processing the distorted QR code image is proposed in this article, and finally, the effectiveness of the algorithm is verified.

Although the research of QR code recognition rate in this article has a certain effect, it proves the effectiveness of the proposed method and provides a significant improvement of 14%. In future, the algorithm will be further optimized to improve the recognition rate for the QR code recognition application. The future perspective of this study will contribute to the QR code application and industry development in our country.