Real - time image defect detection system of cloth digital printing machine

: In order to solve the surface defects such as white silk, spots and wrinkles in the process of digital printing, a surface defect detection system for printed fabrics based on accelerated robust feature algorithm was proposed. Image registration is mainly carried out through accelerated robust feature ( SURF ) ; bidirectional unique matching method is adopted to reduce mismatch points, achieve accurate image registration, and extract defect information through di ﬀ erential algorithm. The performance of the improved sur ﬁ ng algorithm is veri ﬁ ed by using multiple images. The experimental results show that compared with the traditional template matching method, the detection accuracy of the system detection algorithm is 12% higher, and the average time is 42.81 ms shorter than the traditional template matching method. Experiments show that the improved sur ﬁ ng algorithm has short time and high precision. The system can meet the actual production needs. The new system can detect surface defects on printed fabrics with an accuracy of 98%. Conclusion: The algorithm has higher detection rate and faster detection speed, which can meet the needs of practical industrial applications.


Introduction
Digital ink-jet printing of fabrics began in the 1990s.It is a high-tech product of textile industry integrating ink-jet printing technology, computer data processing and precision instruments [1].The digital printing pattern is input into the computer, edited and processed by the printing control software, and then the special dye is sprayed directly onto the fabric by the digital nozzle to print the pattern of specific design and color.Digital printing first appeared in Europe.Compared with traditional printing technology, it not only simplifies the workflow, shortens the production cycle and reduces the cost of investment, but also solves the problem of single design and color of traditional printing, especially in the printing of high-definition patterns with rich colors [2].In recent years, with the comprehensive transformation and upgrading of industry, domestic enterprises began to vigorously develop this technology, played an excellent role in the domestic and foreign markets, and made remarkable achievements.However, digital printing technology also has some disadvantages.In the production process, there will be various types of defects in the printing pattern due to nozzle blockage, nozzle ink leakage, cloth wrinkles, motor step deviation and other faults.In the process of batch printing, if the faults are not found and eliminated in time, a large number of defective products will be produced, resulting in unnecessary waste of resources [3].At present, although each machine is assigned with corresponding testing personnel, the labor cost is high, the testing standards are not unified, and the human eyes will be tired due to long-time work, so the actual testing effect is not good.Therefore, it is very necessary to develop fast, accurate and modern industrial testing technology.
In recent years, many scholars have proposed using image processing technology to detect printing defects.Xin and Kim proposed a fabric defect detection method based on Improved Gaussian mixture model, which has high detection accuracy, but does not classify the defect results [4]; Silva and others proposed a fabric surface defect classification method based on convolutional neural network.This method can classify fabric defects, but its application is limited by application conditions [5]; Zheng and others combined Fourier analysis and wavelet shrinkage, proposed an unsupervised method for detecting fabric defects with periodic patterns.This method simplifies the detection process of periodic fabric defects, but the detection performance is poor [6]; Sachan and others proposed a detection method to extract the features of original local binary pattern with different resolutions by constructing the Gaussian pyramid model of printing image.For patterns with complex background texture, this method failed to effectively extract printing defects [7].Chl and others proposed a motion texture detection algorithm for Inkjet printing based on mixed Markov random field model.The algorithm extracts the motion texture features of the image, introduces the mixed Markov model, and constructs a motion texture feature map containing both background state and motion state.After the model is established, the moving texture detection process is transformed into the minimization of feature energy, so as to effectively improve the representation ability of the model for complex textures and improve the accuracy of texture detection [8].Pennec and others introduced a textile printing accuracy detection method based on machine vision.Based on the model of circular screen printing machine, JSEG algorithm is used to segment the image, find out the edge of the image, match the segmented edge information, and get the deviation region of the image.After experiments, the system has very good detection accuracy [9].Khan and others proposed a dynamic updating method of reference image based on image fusion.Through multi-channel differential operation on the image, the defect information is extracted.This method can continuously update and correct the reference image with the industrial production process, but this method is greatly affected by light and noise in the defect detection of printed fabric [10].Steel and Hartinger proposed a fabric printing color displacement detection method based on normalized cross-correlation.This method mainly matches the image to be measured by moving the standard template, and analyzes its differences to determine the color displacement, but this method has high requirements for parameters [11].According to the requirements of the printing inspection system, the inspection system can not only accurately detect the location of defects, but also carry out feature analysis on the detected defects to screen out those unqualified prints that affect product quality, and those that do not have much impact on product quality.Affecting defects can be treated as qualified products.This requires the defect detection algorithm to have the function of human eye intelligent recognition.At present, there are few researches on defect detection algorithms for this function.In order to meet the requirements of printing enterprises, it is necessary for us to carry out research work in this direction.
Taking printed fabrics as the research object, the research group designed a surface defect detection system of printed fabrics based on improved surf algorithm.
The experimental results show that the system can meet the needs of digital printed fabric defect detection.
2 Research methods 2.1 Defect analysis and hardware design of printed fabric

Common printing fabric defects
In the process of digital printing production, if the jet printing ink is not filtered thoroughly and contains small particles or motor operation deviation, the nozzle will be blocked; Ink leakage and uneven ink output of the nozzle due to mechanical failure of the nozzle; due to the uneven cloth pressing by the roller, there are problems such as cloth wrinkles.The above problems can lead to the surface defects of printing products, such as pass channel, uneven ink output, ink leakage of nozzle, cloth wrinkles and so on.Affect the value of the final product [12,13].Due to the blockage of the nozzle, uneven ink output of the nozzle and the deviation of the stepping motor, there are defects such as white silk, spots and folds in the fabric.See Table 1 for common defect types.

System hardware design
The hardware part is mainly composed of CCD camera, light source and image processing equipment.Among them, the CCD camera adopts MVGED501C-T color area array camera of a company, the camera resolution is 1,920 dpi × 1,080 dpi, and the pixel size is 3.45 μm × 3.45 μm; the lighting system adopts LED light source; the lens adopts MV-LD-25-5M-K lens [14].The schematic diagram of visual platform control is shown in Figure 1.
The working principle of the detection system is: when the roller turns a certain angle, the encoder collects the image from the industrial camera; then the camera converts the collected image into digital image signal and sends it to the upper computer to analyze and process the image to obtain the defect information; finally, the detection system sends instructions to the controller response module according to the defect information to control the roller and inkjet printer [15].

Surface defect detection algorithm of printed fabric
The defect detection function is the core function of the system.In order to realize accurate detection, the defect detection algorithm needs to preprocess the image and extract the characteristic parameters, and determine the defect by analyzing the characteristic parameters of the trapped image to be tested and the standard contour image [16].The software algorithm flow of defect detection system can be divided into five stages: installation initialization stage, parameter configuration stage, template image processing stage, test image processing stage and defect judgment stage.According to the functional requirements, overall architecture and data flow direction of the defect detection system, the digital printing defect detection system can be divided into square main processing module, data conversion module, clock module and power module.Each module cooperates with each other to realize the automatic defect detection function of the system [17].
The surface defect detection algorithm flow of printed fabric is shown in Figure 2. Carry out the same preprocessing operation for the template image and the image to be detected collected by the camera, use the improved surf algorithm to match the feature points of the preprocessed image, and affine transform to register the image; After registration, the difference information between the two images is obtained through image difference, that is, defect information.

Image preprocessing
As the collected printed fabric image is affected by equipment operation jitter and insufficient illumination in the factory, the image brightness is low, including noise and blurred details, which affects the subsequent operation.Therefore, the collected images need to be preprocessed.The main processing methods are image graying, image contrast enhancement and image sharpening.
The SURF algorithm is an improved algorithm based on SIFT.When extracting image feature points for feature  matching, it has a certain degree of stability for images with rotation, scale transformation, and brightness interference.The core of the algorithm is to use the Hessian matrix to obtain the local maximum value of the image and refer to the concept of integral graph, which reduces the complexity of the calculation process and is one of the most commonly used methods for image feature matching.
Let I(x, y) be the position of a pixel in the image, σ is the scale of the Hessian matrix, and the Hessian matrix at I(x, y) is defined: (1) x y L xx , L xy , L yy are the second-order partial derivatives at the pixel point I(x, y) after convolution by the secondorder Gaussian filter function, so that the discriminant of each pixel point Hessian can be obtained.
According to the sign of det(H), each pixel of the image can be classified to determine whether the point is a candidate feature point.
Since the color area array camera used cannot directly obtain the gray image, and the color image is not suitable for direct and rapid detection, it is necessary to gray the color image.The steps are as follows [18]: 1) Calculate the RGB component of each pixel of the image; 2) Calculate weighted gray value 0.3 × B + 0.59 × G + 0.11 × R; 3) Assign the calculated gray value to each corresponding pixel.
Because the collected image has the problem of low brightness, it is necessary to adjust the image contrast to enhance the image.Adjusting image contrast belongs to pixel transformation point operation, and the formula is the following formula (5): where g(i, j) is the adjusted pixel; f(i, j) is the pixel before adjustment; α is the gain coefficient, α > 0; β is the gain variable.
After the image is grayed and the contrast is adjusted, the image has been significantly improved, but there are some problems such as blurred details, so the image sharpening operation is needed.Image sharpening operations include gradient sharpening and Laplace sharpening.The gradient sharpening effect is good, but the noise resistance is poor.The Laplace method has simple calculation and good real-time performance.Therefore, the research group adopts Laplace operator to realize image sharpening.The main idea of the sharpening operator is to determine the value of the pixels in the image after traversing the Laplace 4.

Image registration
Feature point matching.Due to the placement of the camera and the vibration of the printing equipment, the image to be tested needs to be affine transformed to register the template image for the next defect detection operation.The author uses the improved surf algorithm to match the feature points, and combines the two-way uniqueness matching method with the single matching to eliminate the mismatched points; Then the least square method is used to fit the parameters of affine transformation for image registration.Surf algorithm uses the approximate image of Hessian matrix to construct the Hessian matrix of each pixel in the image.When the scale is σ, the Hessian matrix of pixel X = (x, y) is as follows (6): xx xy xy yy (6) where L xx is the second derivative of filtered image g(σ) in x direction; L xy is the second derivative of filtered image g (σ) in x and y directions; L yy is the second derivative of filtered image g(σ) in y direction.Compared with SIFT algorithm, surf algorithm introduces the concept of integral image and uses box filtering with changing size to replace image pyramid [19].The response image of Hessian matrix is obtained by using box filtering and integral image of each size.In order to select the feature points in the image, the original image needs to be transformed, that is, the approximate value of Hessian matrix of each pixel in the original image is used to form the transformation image.The weight is introduced to reduce the deviation caused by approximation, and the discriminant is the following formula (7): xx yy xy approx 2 (7) Judge whether the point is a point of interest according to the positive and negative results of Eq. (7).The size of the interest point is compared with the 26 points in its threedimensional field in the neighborhood of different scale space.If the interest point is the point with the largest eigenvalue in the neighborhood, it is recognized as the feature point in the region.
Take the feature point determined in the previous step as the center, select the area with radius of 6 S (s is the scale value of the feature point), count the Haar wavelet responses of all points in the 60°sector in the area, and give weights according to their responses.The direction of the feature point takes the direction of the longest vector in the area.Take the square box of 20S * 20S near the feature point.The box direction is the main direction of the feature point.Divide the box into 4 * 4 sub regions.Each sub region counts the harr wavelet features of 5 * 5 pixels.In this way, each sub region has a vector with four-dimensional components.The formula is as follows (8): where d x , d y , |d x |, |d y | represent the sum of horizontal direction and vertical direction and the absolute sum of horizontal direction and vertical direction respectively.So far, each feature point has a 16 * 4 = 64 -dimensional description operator.The Euclidean distance between feature points is calculated to judge whether the matching is correct.The smaller the distance, the higher the similarity.When the Euclidean distance is less than the set threshold, it can be determined that the matching is successful [20].
Affine transformation.Although the feature points in the template image and defect image are matched, in practice, the two collected images cannot be fully registered.The transformation parameters need to be determined by matching point pairs to obtain two fully registered images.The schematic diagram of affine transformation is shown in Figure 3.
The affine formula is as follows (9): where (m, n) and (m′, n′) are the coordinates of pixels before and after image transformation; d m and d n are the translation amounts; a, b, c, d are the rotation and stretching parameters.The matched feature point pair is obtained by SURF algorithm, and the least square method is selected to fit the matching points to obtain the above parameters.After determining the parameters, transform according to Eq. ( 9), and finally make the size, position and angle of template image and defect image completely consistent.

Defect detection
After two fully registered images are obtained, the image difference method is selected to obtain the defect location.The basic idea is to differentiate the two images, weaken the similar part of the image and highlight the changing part [21].The absolute value of the difference between the gray values of pixels in the same coordinate is taken as the new gray value of the difference image.After the differential operation of the image, the gray value of the defect position will be relatively high, so it will appear white; for the defect free position, because the pixel values of the two images are the same, the gray value is zero and appears as black.Because the acquired image is often accompanied by noise and affected by affine parameters during image registration, the difference image often has point or linear "false defects."After obtaining the above differential image with "false defect," carry out image binarization operation and morphological processing to eliminate "false defect" and prominent defect.Morphological processing is used to operate the image information that needs to be highlighted or removed in binary images, including corrosion, expansion, open operation and close operation.Corrosion reduces the image; expansion causes the image to expand outward.The open operation is often used to eliminate the impurity and burr in the binary image.Closed operation is usually used to supplement the blank points in the image area.

Result analysis
Experimental environment: the CPU is Intel(R) Core(TM) i5 4200U, the main frequency is 1.6 GHz, the memory is 8 G, and the software development tools are 64 bit Real-time image defect detection system of cloth digital printing machine  5 windows7, VS2015 and OpenCV3.1.0groups of different pictures in the common digital printing defect picture library are selected as samples, and each group includes a standard image and a defect image.Select the printed fabric with ink leakage defect as the display.Due to the influence of the actual shooting environment, the collected defect map cannot be directly differential with the template map.The two images need to be registered after preprocessing [22].
After the matched point coordinate information is obtained, the affine transformation parameters can be obtained by the least square method.Carry out differential operation between the registered defect map and the template map.Due to the different gray values of the template image and the defect image at the defect position, there is an obvious high brightness area at the defect position after the difference, and the template image cannot completely coincide due to the influence of affine transformation error.Therefore, there is a pattern ghosting in the difference image.The image is binarized and morphologically operated to eliminate the influence of ghosting and highlight the defect location.The threshold for image binarization is 42, and the structure matrix with unit structure of 6 × 6 is used for open operation processing.The connected region marking algorithm is used to mark the above point defects, and a rectangular frame with a width of 4 pixels is used to select them.In order to further verify the effectiveness of the detection algorithm of the system, 100 groups of different pictures in the above defect picture library are selected as samples.This algorithm is tested with the traditional template matching method, and the detection accuracy and average value are counted.The results are shown in Table 2.
It can be seen from Figures 4 and 5 that compared with the traditional template matching method, the detection accuracy of the system detection algorithm is 12% higher, and the average time is 42.81 ms shorter than the traditional template matching method.Experiments show that the improved surf algorithm has short time and high precision.The system can meet the actual production needs.

Conclusion
The defect detection algorithm is the core part of the printing inspection system.After analyzing and comparing the shortcomings of the commonly used defect detection algorithms, this paper innovatively proposes a defect detection method based on the surf algorithm.The surf algorithm combined with the two-way matching significantly improves the affine The accuracy of the transformation; the difference algorithm and the connected domain labeling algorithm can be used to directly and quickly extract and mark the defect position; the improved surf algorithm can ensure that the system can detect defects quickly and efficiently.The rationality and practicability of the algorithm are proved by the experimental results.The defect detection algorithm studied in  The average time/ms The average time this subject is mainly designed for the printed images with rich features, and does not consider the problem that the processing object is text.For the detection of the defect is text, it will definitely make mistakes simply using the image to detect the defect.Therefore, in order to better realize the integrity of the algorithm, it is necessary to add text detection algorithms such as OCR in the future.
Funding information: This research received no specifcgrant from any funding agency in the public, commercial, or not-for-profit sectors.
Author contributions: The author has accepted responsibility for the entire content of this manuscript and approved its submission.

Conflict of interest:
The author declares that they have no competing interest.
Informed consent: Informed consent was obtained from all individuals included in this study.
Ethical approval: The conducted research is not related to either human or animals use.

Figure 2 :
Figure 2: Flow chart of surface defect detection.

Figure 5 :
Figure 5: Average detection time results of different algorithms.

Table 1 :
Types of common defects

Table 2 :
Image information table AlgorithmAccuracyFigure 4: Detection accuracy results of different algorithms.