Chlortetracycline is a tetra ring spectrum antibiotic, widely used in medical treatment, agriculture and animal husbandry. At present, the industrial production of chlortetracycline mainly uses biological fermentation technology to ferment and culture Streptomyces aureus, and uses the metabolism of mycelium to obtain the metabolite of chlortetracycline . The modern biological fermentation industry is produced by a series of complex biochemical reactions using microbes. In the process of production, a large number of parameters measurement are needed to ensure that the fermentation process is suitable for the metabolic state of the mycelium, which is of great significance for improving the production efficiency of the industrial process. Most of the parameters of the process of chlortetracycline fermentation can be directly detected by industrial instruments, but there are still some parameters such as total sugar content, biological potency of chlortetracycline, amino nitrogen content and other parameters, which can only be detected by off-line analysis by artificial sampling . The total sugar content has great influence on the growth and fermentation of microorganism. Due to the large viscosity of fermentation broth, lack of total sugar content online monitoring instrument, and the current detection method is offline analysis of artificial site sampling. It has large labor intensity, large time lag and low measurement efficiency. It is difficult to meet the needs of modern industrial production process .
Bai Jianyun  used artificial neural network to conduct soft sensor modeling, and achieved NO_x mass concentration on-line detection. Huang Yonghong  used fuzzy neural network to study the soft sensor of the key parameters of lysine fermentation process.
Zhang Haiying  used the least squares support vector machine learning method for soft sensor of cutting force. Qiao Zongliang  proposed an improved support vector machine for soft sensing. Zhong Huaibing  proposed an on-line soft sensor method based on GPR machine learning principle. The above soft sensor methods have their own characteristics and can be used for soft sensor of relevant parameters in different industrial processes.
Because there are about 25 fermentation tanks in the whole process, samples of each fermentation tank need to be analyzed 3-5 parameters. Considering production and labor costs, at present, the factory determines that each fermentation tank is sampled every 4-8 hours.
In this paper, based on the fermentation process of chlortetracycline, artificial intelligence method was used to establish an online soft sensor model of total sugar content. The measurable data and total sugar content analysis data of chlortetracycline fermentation process was used for the training of soft sensor model, and untrained data were reserved for model verification. The experimental results show that the soft sensor model has higher prediction accuracy of total sugar content and it can meet the prediction requirement of difficult parameters in industrial fermentation process. There are more than 20 fermentation tanks in the production site of chlortetracycline. Samples are taken from each fermentation tank and several parameters are required to be analyzed after the samples are filtered first. In this way, a lot of time will be spent and the labor intensity of the analysts is also very high. Therefore, the sampling interval of chlortetracycline industry production site is set as 4-8 hours/time, but the prolonged sampling interval will lead to the blind feeding operation because the operators cannot timely understand the total sugar content in the fermentation tank, which will cause the fluctuation of the total sugar content in the fermentation tank and affect the output and quality of the product. Soft sensor of total sugar content is an online prediction method, which can reduce labor intensity and save production cost.
2 Several soft sensor modeling methods
2.1 Output recursive wavelet neural network
WNN is a feed-forward neural network with one or more hidden layer structures. It is an extension of the radial basis neural network, and the radial wavelet is used as an activation function in the hidden layer. The wavelet function is obtained by the shift of the parent wavelet through the translation and the scale expansion. The wavelet analysis is to decompose the related original signal into a series of wavelet functions to superpose . The wavelet transform is to transform the φ(t) of a radial wavelet function into the inner product of different signals at different scales, as shown in Eq. (1).
Where a > 0 scale is factor and τ is displacement factor.
Figure 1 shows the structure of ORWNN neural network. There are four layers, namely, input layer, wavelet layer, accumulation layer and output layer.
L1: Input Layer
The layer consists of two parts, namely, input real time vector data and delayed feedback values,
where the input vector is x = [x1 , x2..., xn ]T , The input and output feedback vector is o = [o1,o2,...,on]T , the weight vector of the feedback input is
Where q = [q1,q2,...,qn]T , qn ∈ q is the output vector of the input layer, n is the number of input layer nodes, youtput (t −1) is the output value of an interval unit that is delayed.
L2: Wavelet Layer
In this layer, φi (⋅) represents the wavelet function.
The Gauss wavelet function is used in this paper, it is
Where bi and ai is two factors that need to be constantly revised.
L3: Summing Layer
In the summation layer, the generalized T- norm is used to calculate the fuzzy neural network, and the output of each node in this layer is
L4: Output Layer
Each node in the output layer is used to calculate the linear combination of input quantities and get the output. The output of the model is
Where ωi is the weight value of each node.
2.2 Gauss regression model
Gaussian Process (GP) is a ubiquitous and important stochastic process in nature, the sample is a set of joint Gauss distribution [12,13]. Suppose the input and output sample set is
Where x = [x1,x2,...,xd] is a 1×d input vector, f(*) is unknown function claimed, ε is a Gauss white noise with a mean of 0 and a variance of
Where covariance matrix C is a n×n symmetrical positive determined matrix, it is written as
The common covariance functions are Constant, Linearity, Squared Exponential, Periodic, Mateŕn covariance and Rational Quadratic et al. . In this paper, the Mateŕn covariance function with noise term is used, and its calculation formula is
The maximum likelihood method is used to obtain the set of hyper parameters, that is,
Where tr (*) is the operation of finding the trace of a matrix.
For a new test sample x* , according to the analysis of the nature of the Gauss process, the test sample and the training sample should belong to the same distribution, and the joint distribution is
Where K*= [C(x*,x1), C(x*,x2),..., C(x*,xn)] T is the n ×1 order covariance matrix between test samples x* and training samples, C(x*,x*) is the covariance of the test sample x* itself, all the elements are obtained by covariance Eq. (9) either. Therefore, the distribution of the predicted the output of the GPR model y* obeys Eq. (13) and Eq. (14).
Where E (*) is the operation of taking the mean, Var (*) is the operation for variance.
The final predicted output of the GPR model takes the predicted mean
2.3 The method of model training and evaluation
A soft sensor model is built based on artificial neural network and machine learning theory, and the cumulative update learning method is used to train the soft sensor model. The experimental data of the process parameters of chlortetracycline normal fermentation tank in a factory were used to form the original data set. The process parameters are shown in Table 1. Fermentation time, temperature, pH, DO, air flow rate, air cumulative flow rate, feeding rate, feed accumulation and ammonia accumulation are easy to measure parameters at the scene, which is the input of the soft sensing model and the total sugar content as the output of the model, the training data sets for the input and output are constructed from the 15 batches of data in the original data sets according to the timing of each fermentation tank, which contains all the fermentation data in the process of production.
Input and output variables for soft sensor modeling of CTC.
|x1||Fermentation time||x6||air cumulative flow rate|
|x5||air flow rate||y||total sugar content|
Pucheng Zhengda Fujian Biochemical Co. Ltd. In China has nine 120 m3 fermentation tanks, and there are also more than 10 seed tanks, primary fermentation tanks and secondary fermentation tanks. Several batches of data sets were selected from each fermentation tank to train the soft sensor models, and several batches of data not used for training were left as verification data.
The method of cumulative update training is to use the training data set of historical tank batch to train the model, and the model is tested by the forecast data set. The new input and output data are updated to the fermentation history data set to form a new training data set to achieve the cumulative training of the soft sensor model. The cumulative update training algorithm flow is shown in Figure 2.
The data of the fermentation tank used in this paper are based on the production site of chlortetracycline. The input variable of the soft sensor model, that is, the data of the measurable parameters of the fermentation tank, is the industrial instrument testing data of the fermentation field. The total sugar content in the fermentation tank is the artificial sampling analysis data. The prediction model was trained by the data of multi batch fermentation tank, and some untrained fermentation tank data were used as the test data of the prediction model. The prediction value of the total sugar content was compared with the artificial analysis value, and the prediction accuracy of the soft sensor method was analyzed.
In order to analyze the prediction error of the soft sensing model, the calculation methods of mean relative error(MRE) and root mean square error(RMSE) are introduced. They are an effective method to test whether the soft sensing models meet the requirements of the total sugar content for measurement standard.
Where the N is the number of samples of the model, yi is the predicted value of the i sample, ŷi is the real value of the i sample.
Ethical approval: The conducted research is not related to either human or animal use.
3 Analysis of experimental results
The fermentation broth of chlortetracycline fermentation process is turbid, its composition is complex and its viscosity is very high. The existing total sugar content detection instrument cannot directly contact the fermentation liquid for detection. Therefore, only laboratory analysts can go to the site to sample the fermentation liquid. The total sugar content can be measured by special instrument analysis after filtration and other operations (here, it is called “manual measurement value”). There are more than 20 fermentation tanks (including primary seed tanks and secondary seed tanks) at the chlortetracycline fermentation site. Laboratory analysts need to sample, filter and analyze one by one. Besides the total sugar content, they also need to detect a number of other parameters, and this process is very time consuming. Therefore, we used the manual measurement value as the real value (benchmark value) of total sugar content and compared it with the predicted value of total sugar content online.
After the training of the soft sensor model, the field process data of two batches of the untrained factory T01 and T02 fermentation tanks were used as the input of the model. The total sugar content was predicted and compared with the total sugar content (set this to real value) measured by the off-line manual experiment, as shown in Figure 3 and Figure 4.
In Figure 3, based on the field data of the fermentation process of two batches (No.1 and No.2) of the T01 chlortetracycline fermentation tank, the prediction results of ORWNN-GPR integrated model, ORWNN model and GPR model are compared with the real values(manual measurement values). The experimental results show that the deviation between the predicted value and the true value of total sugar content in ORWNN-GPR integrated model is smaller than that in ORWNN and GPR models. The results show that the ORWNN-GPR integrated model has better prediction accuracy than the single model and higher online prediction accuracy of total sugar content.
In Figure 4, to illustrate that the prediction method proposed in this paper can be applied to different fermentation tanks, based on the field data of the fermentation process of two batches (No.1 and No.2) of the T02 chlortetracycline fermentation tank, The prediction results of ORWNN-GPR integrated model, ORWNN model and GPR model are compared with the real values (manual measurement values). The experimental results show that the deviation between the predicted value and the true value of total sugar content in ORWNN-GPR integrated model is smaller than that in ORWNN and GPR models. It is shown that ORWNN-GPR integrated model has higher accuracy and better generalization ability for online prediction of total sugar content.
The training data set of the soft sensor model is reduced to 50% of the original data set. After the model training, the total sugar concentration is predicted by using the parameter data of 1 batches of T01 fermentation tank to verify and compare the generalization ability of the soft sensor model.
The total sugar content in the fermentation of chlortetracycline is also a complex dynamic change, and the prediction accuracy of the single soft measurement method cannot maintain a high prediction accuracy throughout the fermentation cycle. The ORWNN-GPR combination method can maintain high prediction accuracy for online prediction of total sugar content in the chlortetracycline fermentation process.
In this paper, ORWNN-GPR model and two other ORWNN and GPR models were used to predict total sugar content. The mean square root error RMSE and mean relative error (MRE) were used as indices for statistical analysis, and the results are shown in Table 2.
Prediction and analysis of ORWNN-GPR model and two other ORWNN and GPR models.
|Soft sensing method||Quantity of training data||Mean relative error||RMSE|
|GPR||50% training data|
The prediction accuracy of the total sugar concentration in the ORWNN soft sensor model and the GPR soft sensor model can control the average error within 10%. In the environment with a large number of sample training data, the prediction error of the total sugar concentration in the ORWNN soft sensor model is small.
Under a small amount of training data, the prediction error of total sugar concentration in GPR soft sensing model is small. With the accumulation of training sample data, the accuracy of ORWNN soft sensor model is improved compared with that of GPR model. The ORWNN-GPR combined soft sensing method can ensure higher prediction accuracy in the early stage of model training and the prediction accuracy of the model increases with the cumulative update of training samples.
In the working cycle of the chlortetracycline fermentation tank, the soft sensing method based on ORWNN-GPR model can maintain the high precision of the total sugar content prediction value, effectively solve the problem of long time and serious lag in artificial sampling analysis, and provide rapid and reliable data support for the optimization control of the rate of sugar supplement, which can effectively reduce the cost of production and improve the production efficiency of the chlortetracycline fermentation tank.
The research object and data in this paper are from the actual production site, rather than from simulation and laboratory. Therefore, this manuscript written by our research group is different from the relevant articles published by other research groups.
The method of cumulative update training updates the new input and output data to the fermentation history data set to form a new training data set every time a prediction is performed, and can implement self-renewal of the soft measurement model. The chlortetracycline fermentation production process in this paper is a continuous industrial production process. The cumulative update training method can continuously use the new detection data to train to update the model parameters and maintain the prediction accuracy of the model. The main innovation works of this paper are as follows:
- A soft measurement model was established between the parameters (input) and the total sugar content (output) of the chlorotetracycline fermentation tank in accordance with the on-line undetectable parameter of the total sugar content in the process of chlorotetracycline fermentation.
- Based on the neural network structure and the basic principle of machine learning, this paper adopts ORWNN-GPR combination method to realize online prediction of total sugar content in the chlorotetracycline fermentation process.
- The experimental results show that the soft measurement method based on the ORWNN-GPR combination has higher prediction accuracy, effectively reduces the labor intensity of analysts, reduces production costs and stabilizes the production process, so it has better practical application value.
The total sugar content is an important parameter for the on-line automatic measurement of the fermentation process of the chlortetracycline. This paper combines the recursive wavelet neural network and the Gauss regression process to establish the online soft sensor model of the total sugar content of the fermentation tank. The prediction results of ORWNN method, GPR method and ORWNN-GPR method are compared with field data, The experimental results show that the ORWNN-GPR combined soft sensing model is more accurate than the single ORWNN soft sensor model and the GPR soft sensor model, and can meet the online prediction requirements of the total sugar content of the fermenting tank in the process of the production of chlortetracycline. The combined soft sensor method has practical application value.
This work is financially supported by Yantai “Double Hundred Plan” Talent Project (YT201803) in 2018, Natural Science Foundation (No. ZR2016FM28) of Shandong Province in 2016. The research work was supported by Pucheng Zhengda Fujian Biochemical Co. Ltd. We also thank Charoen Pokphand Group for providing the industrial datasets offed-batch CTC fermentation process.
Zhang HY, Li YJ, Gu JG, et al. Residual chlortetracycline measurement by high per formanc liquid chromatography. Chinese Journal of Analysis Laboratory. 2014;33(10):1130-4.
Xu SL, Gao Y, Hu GL, et al. Rapid Determination of Total Sugar Content of Goji Berries (Lycium barbarum) by Near Infrared Spectroscopy with Effective Wave number Selection. Food Science. 2016;37(12):105-109.
Yang JW, Chen XG, Jin HP, et al. Study on the industrial chlortetracycline fermentor glucose feed rate adjustment method based on soft sensor. Chinese Journal of Scientific Instrument. 2014;35(02):468-74.
Bai JY, Zhu ZJ, Zhang PH. Online soft measurement of NO_x mass concentration for circulating fluidized bed boiler based on BP neural network. Thermal Power Generation. 2016;45(12):78-83.
Huang YH, Sun YK, Wang B, et al. Research of soft sensor based on fuzzy neural network inverse system for lysine fermentation process. Chinese Journal of Scientific Instrument. 2010;31(04):862-7.
Zhang HY, Liao JY. A soft-sensing model on vibration cutting force based on least squares support vector machine and its application. Journal of Central South University (Science and Technology). 2010;41(03):982-7.
Qiao ZL, Zhang L, Zhou JX, et al. Soft sensor modeling method based on improved CPSO-LSSVM and its applications. Chinese Journal of Scientific Instrument. 2014;35(01):234-40.
Zhong H B, Xiong WL. Online soft sensor method based on GPR with test and compensation for singular point. Journal of Nanjing University of Science and Technology. 2017;41(04):503-10.
Ching-Hung Lee, Hua-Hsiang Chang. Output recurrent wavelet neural network-based adaptive backstepping controller for a class of MIMO nonlinear non-affine uncertain systems. Neural Computing and Applications. 2014;24(5):1035-45
Guoqiang C, Limin J, Jianwei Y, et al. Improved wavelet neural network based on hybrid genetic algorithm applicationin on fault diagnosis of railway rolling bearing. JDCTA: International Journal of Digital Content Technology and its Applications. 2010;4(2):13-41.
Inoussa G, Peng H, Wu J. Nonlinear time series modeling and prediction using functional weights wavelet neural network-based state-dependent AR model. Neurocomputing. 2012;86:59-74.
Wang L, Jin H, Chen X, et al. Soft sensor development based on the hierarchical ensemble of Gaussian process regression models for nonlinear and non-Gaussian chemical processes. Industrial & Engineering Chemistry Research. 2016;55(28):7704-19.
Chen T, Ren J. Bagging for Gaussian process regression. Neurocomputing. 2009;72(7):1605-10.
Roberts S, Osborne M, Ebden M, et al. Gaussian processes for time-series modelling. Philosophical Transactions. 2013;371(1984):1-27.