Skip to content
BY-NC-ND 3.0 license Open Access Published by De Gruyter Open Access April 26, 2016

Nonlinear Markov processes in big networks

Quan-Lin Li
From the journal Special Matrices

Abstract

Big networks express multiple classes of large-scale networks in many practical areas such as computer networks, internet of things, cloud computation, manufacturing systems, transportation networks, and healthcare systems. This paper analyzes such big networks, and applies the mean-field theory and the nonlinear Markov processes to constructing a broad class of nonlinear continuous-time block-structured Markov processes, which can be used to deal with many practical stochastic systems. Firstly, a nonlinear Markov process is derived from a large number of big networks with weak interactions, where each big network is described as a continuous-time block-structured Markov process. Secondly, some effective algorithms are given for computing the fixed points of the nonlinear Markov process by means of the UL-type RG-factorization. Finally, the Birkhoff center, the locally stable fixed points, the Lyapunov functions and the relative entropy are developed to analyze stability or metastability of the system of weakly interacting big networks, and several interesting open problems are proposed with detailed interpretation. We believe that the methodology and results given in this paper can be useful and effective in the study of big networks.

References

[1] N. Antunes, C. Fricker, P. Robert, D. Tibi, Metastability of CDMA cellular systems. In: Proceedings of the 12th Annual International ACM Conference on Mobile Computing and Networking, (2006), pp 206–214. 10.1145/1161089.1161113Search in Google Scholar

[2] N. Antunes, C. Fricker, P. Robert, D. Tibi, Analysis of Loss Networks with Routing, The Annals of Applied Probability, 16(4) (2006) 2007–2026. 10.1214/105051606000000466Search in Google Scholar

[3] N. Antunes, C. Fricker, P. Robert, D. Tibi, Stochastic networks with multiple stable points, The Annals of Probability, 36(1) (2008) 255–278. 10.1214/009117907000000105Search in Google Scholar

[4] F. Baccelli, F.I. Karpelevich, M.Y. Kelbert, A.A. Puhalskii, A.N. Rybko, Y.M. Suhov, A mean-field limit for a class of queueing networks, Journal of Statistical Physics, 66(3–4) (1992) 803–825. 10.1007/BF01055703Search in Google Scholar

[5] F. Baccelli, A.N. Rybko, S. Shlosman, Queuing networks with varying topology–a mean-field approach, arXiv preprint: arXiv:1311.3898, (2013). Search in Google Scholar

[6] J. Beltran, C. Landim, Tunneling and metastability of continuous time Markov chains, Journal of Statistical Physics, 140(6) (2010) 1065–1114. 10.1007/s10955-010-0030-9Search in Google Scholar

[7] J. Beltran, C. Landim, Tunneling and metastability of continuous time Markov chains II, the nonreversible case, Journal of Statistical Physics, 149(4) (2012) 598–618. 10.1007/s10955-012-0617-4Search in Google Scholar

[8] M. Benaim, On gradient like properties of population games, learning models and self reinforced processes, arXiv preprint: arXiv:1409.4091, (2014). 10.1007/978-3-319-16118-1_8Search in Google Scholar

[9] M. Benaim, J.Y. Le Boudec, A class of mean-field interaction models for computer and communication systems, Performence Evalution, 65(11–12) (2008) 823–838. 10.1016/j.peva.2008.03.005Search in Google Scholar

[10] M. Benaim, J.Y. Le Boudec, On mean field convergence and stationary regime, arXiv preprint: arXiv:1111.5710, (2011). Search in Google Scholar

[11] A. Bobbio, M. Gribaudo, M. Telek, Analysis of large scale interacting systems by mean field method. In: The Fifth International IEEE Conference on Quantitative Evaluation of Systems, (2008), pp 215–224. 10.1109/QEST.2008.47Search in Google Scholar

[12] C. Bordenave, D. McDonald, A. Proutiere, A particle system in interaction with a rapidly varying environment: Mean-field limits and applications, Networks and Heterogeneous Media, 5(1) (2010) 31–62. Search in Google Scholar

[13] K.A. Borovkov, Propagation of chaos for queueing networks, Theory of Probability & Its Applications, 42(3) (1998) 385–394. 10.1137/S0040585X97976222Search in Google Scholar

[14] A. Bovier, Markov processes and metastability, Lecture Notes TUB, (2003), pp 1–75. 10.1007/978-3-540-92796-9_4Search in Google Scholar

[15] A. Bovier, Metastability: A potential theoretic approach. In: Proceedings oh the International Congress of Mathematicians: Invited Lectures, (2006), pp 499–518. 10.4171/022-3/26Search in Google Scholar

[16] A. Bovier, M. Eckhoff, V. Gayrard, M. Klein, Metastability in stochastic dynamics of disorderedmean-field models, Probability Theory and Related Fields, 119(1) (2001) 99–161. 10.1007/PL00012740Search in Google Scholar

[17] A. Bovier, M. Eckhoff, V. Gayrard, M. Klein, Metastability and low lying spectral in reversibleMarkov chains, Communications in Mathematical Physics, 228(2) (2002) 219–255. 10.1007/s002200200609Search in Google Scholar

[18] A. Budhiraja, P. Dupuis, M. Fischer, K. Ramanan, Local stability of Kolmogorov forward equations for finite state nonlinear Markov processes, arXiv preprint: arXiv:1412.5555, (2014). 10.1214/EJP.v20-4004Search in Google Scholar

[19] A. Budhiraja, P. Dupuis, M. Fischer, K. Ramanan, Limits of relative entropies associated with weakly interacting particle systems, arXiv preprint: arXiv:1412.5553, (2014). 10.1214/EJP.v20-4003Search in Google Scholar

[20] A. Budhiraja, A.P. Majumder, Long time results for a weakly interacting particle system in discrete time, arXiv preprint: arXiv:1401.3423, (2014). 10.1080/07362994.2014.1003434Search in Google Scholar

[21] M.F. Chen, From Markov Chains to Non-equilibrium Particle Systems. World Scientific, (2004). 10.1142/5513Search in Google Scholar

[22] R.W.R. Darling, J.R. Norris, Differential equation approximations for Markov chains, Probability surveys, 5(1) (2008) 37–79. 10.1214/07-PS121Search in Google Scholar

[23] D.A. Dawson, Critical dynamics and fluctuations for a mean-field model of cooperative behavior, Journal of Statistical Physics, 31(1) (1983) 29–85. 10.1007/BF01010922Search in Google Scholar

[24] D.A. Dawson, X. Zheng, Law of large numbers and central limit theorem for unbounded jump mean-field models, Advances in Applied Mathematics, 12(3) (1991) 293–326. 10.1016/0196-8858(91)90015-BSearch in Google Scholar

[25] F. Delcoigne, G. Fayolle, Thermodynamical limit and propagation of chaos in polling systems,Markov Processes and Related Fields, 5(1) (1999) 89–124. Search in Google Scholar

[26] F. den Hollander, Metastability under stochastic dynamics, Stochastic Processes and their Applications, 114(1) (2004) 1–26. 10.1016/j.spa.2004.07.007Search in Google Scholar

[27] R.L. Dobrushin, Queuing networks - without explicit solutions and without computer simulation. In: Conference on Applied Probability in Engineering, Computer and Communication Sciences: Keynote Lecture, Paris, (1993). Search in Google Scholar

[28] N.G. Duffield, Local mean-field Markov processes: An application to message-switching networks, Probability Theory and Related Fields, 93(4) (1992) 485–505. Search in Google Scholar

[29] N.G. Duffield, R.F. Werner, Local dynamics of mean-field quantum systems, Helvetica Physica Acta, 65(8) (1991) 1016–1054. Search in Google Scholar

[30] P. Dupuis, M. Fischer, On the construction of Lyapunov functions for nonlinear Markov processes via relative entropy. Submitting for publication, (2011). Search in Google Scholar

[31] S.N. Ethier, T.G. Kurtz, Markov Processes: Characterization and Convergence. John Wiley & Sons, (1986). 10.1002/9780470316658Search in Google Scholar

[32] T.D. Frank, Nonlinear Markov processes, Physics Letters A, 372(25) (2008) 4553–4555. 10.1016/j.physleta.2008.04.027Search in Google Scholar

[33] M.I. Freidlin, A.D. Wentzell, Random Perturbations of Dynamic Systems. Springer, (1984). 10.1007/978-1-4684-0176-9Search in Google Scholar

[34] C. Fricker, N. Gast, Incentives and redistribution in homogeneous bike-sharing systemswith stations of finite capacity, EURO Journal on Transportation and Logistics, Published Online: June 7, (2014), pp 1–31. Search in Google Scholar

[35] C. Fricker, N. Gast, H. Mohamed. Mean field analysis for inhomogeneous bike sharing systems. In: DMTCS Proceedings, 1 (2012), pp 365–376. 10.46298/dmtcs.3006Search in Google Scholar

[36] A. Galves, E. Olivieri, M.E. Vares, Metastability for a class of dynamical systems subject to small random perturbations, The Annals of Probability, 15(4) (1987) 1288–1305. 10.1214/aop/1176991977Search in Google Scholar

[37] N. Gast, B. Gaujal, A mean field model of work stealing in large-scale systems, ACM SIGMETRICS Performance Evaluation Review, 38(1) (2010) 13–24. 10.1145/1811099.1811042Search in Google Scholar

[38] N. Gast, B. Gaujal, A mean field approach for optimization in discrete time, Discrete Event Dynamic Systems, 21(1) (2011) 63–101. 10.1007/s10626-010-0094-3Search in Google Scholar

[39] N. Gast, B. Gaujal, J.Y. Le Boudec, Mean field for Markov decision processes: from discrete to continuous optimization, IEEE Transactions on Automatic Control, 57(9) (2012) 2266–2280. Search in Google Scholar

[40] R.J. Gibbens, P.J. Hunt, F.P. Kelly, Bistability in communication networks. In: Disorder in Physical Systems: A Volume in Honour of John M. Hammersley, (Eds G. Grimmett and D. Welsh), Oxford University Press, (1990), pp 113–127. Search in Google Scholar

[41] C. Graham, Chaoticity on path space for a queueing network with selection of the shortest queue among several, Journal of Applied Probability, 37(1) (2000) 198–201. 10.1239/jap/1014842277Search in Google Scholar

[42] C. Graham, Functional central limit theorems for a large network in which customers join the shortest of several queues, Probability Theory and Related Fields, 131(1) (2004) 97–120. 10.1007/s00440-004-0372-9Search in Google Scholar

[43] R.A. Hayden, A. Stefanek, J.T. Bradley, Fluid computation of passage-time distributions in large Markov models, Theoretical Computer Science, 413(1) (2012) 106–141. 10.1016/j.tcs.2011.07.017Search in Google Scholar

[44] P.J. Hunt, T.G. Kurtz, Large loss networks, Stochastic Processes and their Applications, 53(2) (1994) 363–378. 10.1016/0304-4149(94)90071-XSearch in Google Scholar

[45] J. Jacod, A. Shiryaev, Limit Theorems for Stochastic Processes. Springer, (2003). 10.1007/978-3-662-05265-5Search in Google Scholar

[46] F.I. Karpelevich, A.N. Rybko, Thermodynamical limit for symmetric closed queuing networks, Translations of the American Mathematical Society, 198(2) (2000) 133–156. 10.1090/trans2/198/10Search in Google Scholar

[47] F.P. Kelly, Loss networks, The Annals of Applied Probability, 1(3) (1991) 319–378. 10.1214/aoap/1177005872Search in Google Scholar

[48] C. Kipnis, C. Landim, Scaling Limits of Interacting Particle Systems. Springer, (1999). 10.1007/978-3-662-03752-2Search in Google Scholar

[49] V.N. Kolokoltsov, Nonlinear Markov Processes and Kinetic Equations. Cambridge University Press, (2010). 10.1017/CBO9780511760303Search in Google Scholar

[50] V.N. Kolokoltsov, Nonlinear Lévy and nonlinear Feller processes: An analytic introduction, arXiv preprint: arXiv:1103.5591, (2011). Search in Google Scholar

[51] V.N. Kolokoltsov, J.J. Li,W. Yang, Mean field games and nonlinearMarkov processes, arXiv preprint: arXiv: 1112.3744, (2011). Search in Google Scholar

[52] T.G. Kurtz, Solution of ordinary differential equations as limits of pure jumpMarkov processes, Journal of Applied Probability, 7(1) (1970) 49–58. 10.2307/3212147Search in Google Scholar

[53] T.G. Kurtz, Limit theorems for sequences of jump Markov processes approximating ordinary differential processes, Journal of Applied Probability, 8(2) (1971) 344–356. 10.2307/3211904Search in Google Scholar

[54] T.G. Kurtz, Strong approximation theorems for density dependent Markov chains, Stochastic Processes and Their Applications, 6(3) (1978) 223–240. 10.1016/0304-4149(78)90020-0Search in Google Scholar

[55] J.Y. Le Boudec, D. McDonald, J. Mundinger, A generic mean-field convergence result for systems of interacting objects. In: Proc. Conf. IEEE on the Quantitative Evaluation of Systems, (2007), pp 3–18. 10.1109/QEST.2007.8Search in Google Scholar

[56] Q.L. Li, Constructive Computation in Stochastic Models with Applications: The RG-Factorizations. Springer, (2010). Search in Google Scholar

[57] Q.L. Li, Tail probabilities in queueing processes, Asia-Pacific Journal of Operational Research, 31(2) (2014) 1–31. 10.1142/S0217595914400077Search in Google Scholar

[58] Q.L. Li, G.R. Dai, J.C.S. Lui, Y. Wang, The mean-field computation in a supermarket model with server multiple vacations, Discrete Event Dynamic Systems, 24(4) (2014) 473–522. 10.1007/s10626-013-0171-5Search in Google Scholar

[59] Q.L. Li, Y. Du, G.R. Dai, M. Wang, On a doubly dynamically controlled supermarket model with impatient customers, Computers & Operations Research, 55 (2014) 76–87. Search in Google Scholar

[60] Q.L. Li, J.C.S. Lui, Block-structured supermarket models, Discrete Event Dynamic Systems, Published Online: June 29, (2014), pp 1–36. Search in Google Scholar

[61] Q.L. Li, F.F. Yang, Mean-field analysis for heterogeneous work stealing models. In: Information Technologies andMathematical Modelling: Queueing Theory and Applications, Springer, (2015), pp 28–40. 10.1007/978-3-319-25861-4_3Search in Google Scholar

[62] T.M. Liggett, Interacting Particle Systems. Springer, (2005). 10.1007/b138374Search in Google Scholar

[63] M.J. Luczak, C. McDiarmid, On themaximumqueue length in the supermarket model, The Annals of Probability, 34(2) (2006) 493–527. 10.1214/00911790500000710Search in Google Scholar

[64] M.J. Luczak, C. McDiarmid, Asymptotic distributions and chaos for the supermarket model, Electronic Journal of Probability, 12(1) (2007) 75–99. 10.1214/EJP.v12-391Search in Google Scholar

[65] J.B. Martin, Point processes in fast Jackson networks, The Annals of Applied Probability, 11(3) (2001) 650–663 10.1214/aoap/1015345344Search in Google Scholar

[66] J.B. Martin, Y.M. Suhov, Fast Jackson networks, The Annals of Applied Probability, 9(3) (1999) 854–870. 10.1214/aoap/1029962816Search in Google Scholar

[67] V.V. Marbukh, Investigation of a fully connected channel-switching network with many nodes and alternative routes, Automation & Remote Control, 44(12) (1984) 1601–1608. Search in Google Scholar

[68] M.D. Mitzenmacher, The Power of Two Choices in Randomized Load Balancing, Ph.D. thesis, Department of Computer Science, University of California at Berkeley, USA, (1996). Search in Google Scholar

[69] S.A. Muzychka, K.L. Vaninsky, A class of nonlinear random walks related to the Ornstein-Uhlenbeck process, Markov Processes and Related Fields, 17(2) (2011) 277–304. Search in Google Scholar

[70] E. Olivieri, M.E. Vares, Large Deviations and Metastability. Cambridge University Press, (2005). 10.1017/CBO9780511543272Search in Google Scholar

[71] V.I. Oseledets, D.V. Khmelev, Stochastic transportation networks and stability of dynamical systems, Theory of Probability & Its Applications, 46(1) (2002) 154–161. 10.1137/S0040585X97978786Search in Google Scholar

[72] S.G. Peng, Nonlinear expectations and nonlinear Markov chains, Chinese Annals of Mathematics, 26(2) (2005) 159–184. 10.1142/S0252959905000154Search in Google Scholar

[73] L.C.G. Rogers, D. Williams, Diffusions, Markov Processes, and Martingales, Vol. 1: Foundations. John Wiley & Sons, (1994). Search in Google Scholar

[74] A.N. Rybko, S. Shlosman, Poisson Hypothesis for information networks (a study in non-linear Markov processes), arXiv preprint: arXiv:0303010, (2003). Search in Google Scholar

[75] T. Shiga, H. Tanaka, Central limit theorem for a system ofMarkovian particleswithmean field interactions, Probability Theory and Related Fields, 69(3) (1985) 439–459. 10.1007/BF00532743Search in Google Scholar

[76] F. Spitzer, Interaction of Markov processes, Advances in Mathematics, 5(2) (1970) 246–290. 10.1016/0001-8708(70)90034-4Search in Google Scholar

[77] Y.M. Suhov, N.D. Vvedenskaya, Fast Jackson networks with dynamic routing, Problems of Information Transmission, 38(2) (2002) 136–153. 10.1023/A:1020010710507Search in Google Scholar

[78] A. Sznitman, Topics in Propagation of Chaos. Springer-Verlag lecture notes in mathematics 1464, (1989), pp 165–251. 10.1007/BFb0085169Search in Google Scholar

[79] D. Tibi, Metastability in communication networks, arXiv preprint: arXiv:1002.0796, (2010). Search in Google Scholar

[80] S.R.E. Turner, Resource Pooling in Stochastic Networks, Ph.D. Thesis, Statistical Laboratory, Christ’s College, University of Cambridge, (1996). Search in Google Scholar

[81] A.G. Turner, Convergence of Markov processes near saddle fixed points, The Annals of Probability, 35(3) (2007) 1141–1171. 10.1214/009117906000000836Search in Google Scholar

[82] K. Vaninsky, S. Myzuchka, A. Lukov, A multi-agent nonlinear Markov model of the order book, arXiv preprint: arXiv:1208.3083, (2012). Search in Google Scholar

[83] N.D. Vvedenskaya, R.L. Dobrushin, F.I. Karpelevich, Queueing system with selection of the shortest of two queues: An asymptotic approach, Problems of Information Transmission, 32(1) (1996) 20–34. Search in Google Scholar

[84] N.D. Vvedenskaya, Y.M. Suhov, Dobrushin’s mean-field limit for a queue with dynamic routing, Markov Processes and Related Fields, 3(3) (1997) 493–526. Search in Google Scholar

[85] W. Whitt, Stochastic-Process Limits: An Introduction to Stochastic-Process Limits and Their Application to Queues. Springer, (2002). 10.1007/b97479Search in Google Scholar

[86] S. Zachary, I. Ziedins, A refinement of the Hunt-Kurtz theory of large loss networks,with an application to virtual partitioning, The Annals of Applied Probability, 12(1) (2002) 1–22. 10.1214/aoap/1015961154Search in Google Scholar

Received: 2015-7-30
Accepted: 2016-3-29
Published Online: 2016-4-26

©2016 Quan-Lin Li

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Scroll Up Arrow