Smear character recognition method of side - end power meter based on PCA image enhancement

: Since it is di ﬃ cult for manual recording to track the rapid change of indication of the power meter, the power meter images are collected by the camera and auto - matically recognized and recorded to e ﬀ ectively overcome the disadvantages of manual recording. However, the complex scene lighting environment and smearing char - acter shadows make it di ﬃ cult to transfer captured images directly to convolutional neural networks for character recognition. A smear character recognition method of side - end power meter under complex lighting conditions is proposed in this article. First, the uneven illumination image enhancement algorithm is studied. Through the estimation of the illumination component of the image, the fusion weight is calculated by the principal component analysis for multiscale fusion, and the up - sampling and down - sampling are adopted to reduce the calculation of the algorithm and achieve the rapid image enhancement. A convolution neural network framework based on deep learning is proposed to realize the segmentation of smear characters, and the ﬁ nal segmented individual characters are fed into a network to identify the meter readings. The experimental results show that the proposed smear char - acter recognition method has fast recognition speed, and the recognition rate of samples with the smear character and complex illumination reaches 99.8%, which meets the requirements of power meter character recognition and is better than other algorithms.


Introduction
With the expansion of power consumption and the increase in the number of substations, more and more measuring instruments need to be monitored [1]. Accurate meter reading of measuring instruments is crucial to the operation of the substation [2]. In addition, the electric power inspection basically adopts the inspection robot to shoot and extract the data of the meter through the camera. However, there is a certain deviation in the recognition of the electric power meter under the complex illumination of the camera [3]. The character recognition accuracy of electric power meters needs to be improved under the complex illumination environment of the substations [4,5].
Currently, most of the instruments at the stations use LCD screens, but because the LCD screen has reflective characteristics, it is easy to cause uneven illumination of the collected images [6]. Moreover, the long black-andwhite response time of some LCD screens causes serious shadows when the characters are quickly refreshed, that is, when one character does not completely disappear, another character has appeared, resulting in the superposition of two characters with different brightness in the collected image. In addition, the scene lighting environment is different, and sunlight is different indoor and outdoor, which makes the collected images vary greatly. Therefore, how to accurately identify instrument characters with serious shadow under the conditions of uneven illumination is a challenging problem [7].
Character recognition using neural networks has been a research focus in recent years. The algorithms [8,9] detect single characters before using the DCNN model to identify them. This approach often requires the implementation of an efficient character detector capable of precisely detecting and cutting each character from the original word picture. Other methods [10] treat scene character recognition as an image classification issue, requiring each English word to be assigned to a category label (90k words). Traditional character recognition approaches that are not based on neural networks have also contributed to new concepts and representations. Almazán et al. [11] and Rodriguez-Serrano et al. [12] proposed embedding word images and text strings in public-oriented quantum space and turning word recognition into a retrieval issue. Lee et al. [13] suggested a method for identifying feature pools that can learn the character information subregion. For character recognition, Bai et al. [14] and Gordo [15] used mid-level features. By exchanging features across two tasks, Yao et al. [16] unified character recognition and detection.
This article aims to design a fast image recognition method for smear characters under different illumination conditions, including image enhancement and trailing shadow character image segmentation to improve the accuracy of automatic recognition of power meter reading.
This article is divided into four sections. Section 1 presents the introduction. The status and development of power instrument character recognition are comprehensively expounded. In Section 2, a method for smear character recognition of edge power meter under complex illumination interference is proposed. First, the overall process of the algorithm is introduced. Then, the image enhancement algorithm for uneven illumination is studied to realize the rapid enhancement of instrument images. Finally, a convolutional neural network framework based on deep learning is designed to realize the perfect segmentation of instrument shadow characters. Section 3 presents the experimental results and analysis, including data collection and calibration, binary segmentation experiments, and character recognition experiments. Section 4 presents the conclusion, which summarizes the research content.

Overall process
To achieve accurate and stable recognition of smear characters of power meters under complex lighting, improvements are mainly made in three aspects: image enhancement, binarization and character segmentation, and character recognition [17]. The overall process is shown in Figure 1. Among them, the fully convolutional network (FCN) [18] and the VGG [19] instrument character segmentation model were trained in advance, which are not marked in the flowchart. The major steps are as follows: (1) Camera images acquisition; (2) The fast adaptive image enhancement method based on PCA [20] fusion is used to correct uneven illumination, and then the binarization is carried out based on FCN and VGG instrument character segmentation method; (3) Remove small connected domains and noise by opening and closing operations; (4) Tilt correction; (5) Segmentation of single character; (6) Single characters are normalized and input to smear character recognition model for recognition.

Fast adaptive image enhancement based on PCA fusion
The original RGB image is converted into HSV color space, and the V component is extracted and downsampled. The illumination component of the image is extracted by multiscale Gaussian filtering and up-sampled to the size before down-sampling. Then, the illumination component is refined by guided filtering, and the illumination component is adaptively adjusted by different correction coefficients to obtain two images. The weights of the two images are calculated by PCA and fused to enhance the V component. Finally, the image is converted from HSV space to RGB space to obtain the final enhanced image. The specific process is shown in Figure 2.
The calculation amount of multiscale Gaussian filtering increases with the increase of the input image size [21], which affects the real-time performance of the algorithm. Down-sampling and up-sampling can well reduce the calculation amount, and guided filtering can refine the up-sampling results and retain the local characteristics of the illumination component. To avoid the small down-sampling size and excessive loss of image details, the linear interpolation method is used for down-sampling, and the down-sampling size is a quarter of the input size.
To balance the global and local characteristics of the extracted illumination components, the multiscale twodimensional discrete Gaussian function is used to extract the illumination components in the scene, and each component is weighted. Finally, the estimated illumination components are obtained. The calculation is as follows: denotes the illumination component extracted by the ith scale Gaussian function, θ i represents the corresponding weight coefficient, and N represents the number of scales. To balance the accuracy and calculation amount of illumination component extraction, N is set to 3, θ i is set to 1/3, and the parameters c of the three scales of Gaussian filtering are 4, 20, and 63, respectively.
To reduce the amount of fusion computation, a simple source image weighted fusion is adopted without multiscale decomposition to avoid the bias of fusion results to a certain source image. PCA is used to determine the weight coefficient of each source image as the key to image fusion. The specific process and result are shown in Figure 3, and it can be seen that the image process by fast adaptive image enhancement algorithm is significantly improved in terms of color, brightness, and detail.

FCN-and VGG-based smear character segmentation
Combining FCN and VGG16 network, the input is the original RGB image and the output is a binary map without the smear. The network consists of two parts: feature extraction and image recombination. The feature extraction is a down-sampling composed of five convolutional blocks, each convolutional block includes a convolutional layer and a maximum pooling layer, and the Relu activation function is chosen behind the convolutional layer to alleviate overfitting. The number of convolutional kernels in the convolutional layer is 256, the size is 3 × 3, the step size is 1, and the pad is 1. The pooling layer has a pooling window size of 2 × 2 and a step size of 2. The image reconstruction is performed by five deconvolution blocks to reduce the extracted information to images. The deconvolution block includes a deconvolution layer and a connection layer, where the convolution kernel of the deconvolution layer is 4 × 4 with a step size of 2. The overall structure of the network is shown in Figure 4, and the mean square error (MSE) is used as the loss function, which is defined as follows: where n is the number of samples, y i is the label, and ′ y i is the network prediction result.

Collection and calibration of data sets
The samples were collected in the instrument working environment. The light sources were placed in the left, right, and median positions of the instrument and combined with a 30 degree angle deflection between the camera and the instrument. A total of 7,600 color images were collected. Label production is divided into two stages: preparation and production. In the preparation stage, the accurate position of the characters in the instrument image is mainly counted. The image with good illumination is selected, and the approximate region of the characters is manually calibrated. The minimum circumscribed rectangle is determined, and the angle θ between the long edge of the minimum circumscribed rectangle and the horizontal x-axis is calculated. The binary image is rotated and corrected to make all the characters on the same horizontal line. The center P1(x 1 , y 1 ), P2(x 2 , y 2 ), and P3(x 3 , y 3 ) of the three calibration circles are determined by circle detection as the three points corresponding to the affine transformation in the label production stage. The position of each character is recorded by the vertical projection method, and the characters with complete segmentation   and no shadow are selected as the standard single character, as shown in Figure 5.
In the label production stage, Wolf algorithm is used to binarize the instrument image and detect the center of three calibration circles P1( 2 2 , and P3( ) ′ ′ x y , 3 3 , combined with the center P1(x 1 , y 1 ), P2(x 2 , y 2 ), and P3(x 3 , y 3 ) detected in the preparation stage, the affine transformation matrix A is calculated as shown in Eq. (3). Affine transformation of instrument image is obtained by affine transformation matrix, let P1, P2, and P3 coincide with P1, P2, and P3, respectively. On each character position recorded in the preparation phase, the shadow character is replaced with its corresponding standard single character to complete the label production, as shown in Figure 6. (3) To reduce the training time, the sample only represents the number of parts by the interceptor, and each sample is randomly rotated and shifted to expand three samples. Through this method, the total number of samples is 30,400, and 80% of the samples are randomly selected for training, and the remaining samples are used as testing samples.
The running time, IoU, and SNR mean of each algorithm are presented in Table 1. Some of the operation results are shown in Figure 7. The uneven illumination has a great influence on the binarization effect, and the binarization effect of the traditional methods on the smear is very poor. Niblack algorithm generally divides the target area, but with a lot of noise, it cannot remove the smear character. Because most of the local region of the instrument image is the background, uneven illumination causes Bernsen, Sauvola and Bradley algorithm to divide part of the background into target pixels and do not eliminate the smear character. Wellner algorithm  cannot segment strong light region and smear character. Wolf algorithm overcomes the influence of uneven illumination to some extent, but cannot smear character. The network designed in this article has a clear and no smear character on the binarization effect of uneven illumination and shadow images. The IoU and SNR values are higher than those of other methods. It is suitable for images with different light intensities and rotations. It has a strong generalization ability and fast processing speed, only slightly higher than Wellner.

Character recognition experiment
Smear characters are segmented for character recognition [27]. Character segmentation includes tilt correction, affine transformation, vertical projection, and horizontal projection. The specific effect is shown in Figure 8. Because the horizontal projection intervals of numbers, decimals and "VAC" characters are different, and "VAC" characters remain unchanged, decimals can be judged directly by the horizontal projection interval, so only numbers are reserved in segmentation. The single character is segmented by this algorithm, as shown in Figure 9, where the smear character is basically eliminated.
The individual characters after sample segmentation are categorized to make numerical labels, which are input into the CRNN model for training and testing, respectively.
To verify the effectiveness of the algorithms in this article, HOG + SVM [28], KNN [29], and CRNN [30] algorithms are used for character recognition. The CRNN algorithm parameters are as follows: input image size is 32 × 32, batch size is 64, training cycle is 3000, and learning rate is 0.0001. SVM uses linear kernel function, and HOG feature unit size is 3 × 3.
The experimental results are presented in Table 2. Compared with the direct recognition of characters, the recognition rate of all algorithms is higher after the processing of the proposed character segmentation algorithm, and the recognition rate of CRNN algorithm is improved by 21.71%, indicating that the proposed binarization algorithm provides a better basis for character recognition.

Conclusions
In this article, we design a fast recognition method for power meter characters by combining adaptive image enhancement, smear character segmentation based on FCN and VGG for the problem of difficult recognition of meter smear characters under complex illumination. The experimental results show that the proposed smear character segmentation network can achieve adaptive binary segmentation of meter images, effectively remove the effects of digital trailing, offset or rotation under severe illumination uneven conditions, with high recognition rate and fast recognition speed of the whole method, meeting the requirements of fast meter character recognition. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.