Document Type : Research Article
Authors
Department of Mathematics, Tabriz Branch, Islamic Azad University, Tabriz, Iran.
Abstract
This paper presents a capable recurrent neural network, the so-called µRNN for solving a class of non-convex quadratic programming problems. Based on the optimality conditions we construct a new recurrent neural network (µRNN), which has a simple structure and its capability is preserved. The proposed neural network model is stable in the sense of Lyapunov and converges to the exact optimal solution of the original problem. In a particular case, the optimality conditions of the problem become necessary and sufficient. Numerical experiments and comparisons with some existing algorithms are presented to illustrate the theoretical results and show the efficiency of the proposed network.
Keywords
[1] Bacciotti, A. (1992). “Local stabilizability of nonlinear control systems”, Advances in Mathematics for Applied Sciences.
[2] Bazaraa, M.S., Shetty, C.M. (1990). “Nonlinear programming theory and algorithms”, Wiley and Sons, New York.
[3] Bertsekas, D.P. (1989). “Parallel and distributed numerical methods”, Prentice-Hall, Englewood, Cliffs, NJ.
[4] Best Michael, J. (2017). “Quadratic programming with computer programs”, Advances in Applied Mathematics, University of Waterloo, Canada, CRC Press.
[5] Beyer, D., Ogier, R. (2000). “Tabu learning: A neural networks search method for solving non-convex optimization problems”, IEEE International Joint Conference Neural Networks 2, 953-961.
[6] Bian, W., Chen, X. (2013). “Worst-case complexity of smoothing quadratic regularization methods for non-Lipschitz optimization”, SIAM, Journal of Optimization, 23 (3), 1718-
1741.
[7] Bian, W., Chen, X., Ye, Y. (2014). “Complexity analysis of interior point algorithms for non-Lipschitz and non-convex minimization”, Mathematical Programming, 149(1-2), 301-327.
[8] Boob, D.P. (2020). “Convex and structured non-convex optimization for modern machine learning: Complexity and algorithms”, Georgia Institute of Technology.
[9] Carmon, Y., Duchi, J.C. (2020). “First-order methods for non-convex quadratic minimization”, SIAM Review, 62(2), 395-436.
[10] Chen, X., Womersley, R., Ye, J. (2011). “Minimizing the condition number of a Gram matrix”, SIAM Journal on Optimization, 21, 127-148.
[11] Chicone, C. (2006). “Ordinary differential equations with applications”; Second edition, Springer-Verlag, New York.
[12] Cui, Y., Chang, T. H., Hong, M., Pang, J.S. (2020). “A study of piecewise linear-quadratic programs”, Journal of Optimization Theory and Applications, 1-31.
[13] Effati, S., Mansoori, A., Eshaghnezhad, M. (2015). “A projection neural network for solving bilinear programming problems”, Neurocomputing, 1-20.
[14] Effati, S., Ranjbar, M. (2011). “A novel recurrent nonlinear neural network for solving quadratic programming problems”, Applied Mathematical Modelling, 33, 1688-1695.
[15] Eshaghnezhad, M., Effati, S., Mansoori, A. (2016). “A neurodynamic model to solve non-linear pseudo-monotone projection equation and its applications”, IEEE Transactions on Cybernetics, 1-13.
[16] Gao, X.B., Liao, L.Z., Xue, W. (2004). “A neural network for a class of convex quadratic minimax problems with constraints”, IEEE Transactions on Neural Networks, 15(3), 622-628.
[17] Hopfield, J.J., Tank, D. (1985). “Neural computation of decisions in optimization problem”, Biod. Cybern., 52, 141-152.
[18] Horn, R.A., Johnson, C.R. (1990). “Matrix Analysis”, Cambridge University Press.
[19] Huan, L., Fang, C., Zhouchen, L. (2020). “Accelerated first-order optimization algorithms for machine learning”, Proceedings of the IEEE, doi: 10.1109/JPROC.2020.3007634.
[20] Huan, L., Lin, Z. (2019). “Provable accelerated gradient method for non-convex low-rank optimization”, Machin Learning.
[21] Huang, F., Gu, B., Huo, Z., Xhen, S., Huang, H. (2019). “Faster gradient-free proximal stochastic methods for non-convex nonsmooth optimization”, The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), 1503-1510.
[22] Jeyakumar, V., Lee, G.M., Li, G.Y. (2009). “Alternative theorems for quadratic inequality systems and global quadratic optimization”, SIAM Journal on Optimization, 20(2), 983-1001.
[23] Jeyakumar, V., Rubinov, A.M., Wu, Z.Y. (2007). “Non-convex quadratic minimization problems with quadratic constraints: Global optimality conditions”, Mathematical Programming, series, A 110, 521-541.
[24] Jeyakumar V., Srisatkunarajah S. (2009). “Lagrange multiplier necessary condition for global optimality for non-convex minimization over a quadratic constraint via S-lemma”, Optimization Letters, 3, 23-33.
[25] Khalil, H.K. (2002). “Nonlinear systems”, Prentice Hall, Third edition.
[26] Kong, W., Melo, J.G., Monteiro, R.D.C. (2019). “An efficient adaptive accelerated inexact proximal point method for solving linearly constrained non-convex”, Composite Problems, 1-28.
[27] Leung, M.F., Wang, J. (2019). “Minimax and bi-objective portfolio selection based on collaborative neurodynamic optimization”, IEEE Transactions on Neural Networks and Learning Systems, 1-12.
[28] Liu, S., Jiang, H., Zhang, L., Mei, X. (2020). “A neurodynamic optimization approach for complex-variables programming problem”, Neural Networks, 129, 280-287.
[29] Lu, S. (2018). “First-Order methods of solving non-convex optimization problems: Algorithms, convergence, and optimality”, Electrical and Electronics Commons.
[30] Luenberger, D.G. (1948). “Introduction to Linear and Nonlinear Programming”, Reading MA: Addison-Wesley.
[31] Malek, A., Hosseinipour-Mahani, N. (2015). “Solving a class of non-convex quadratic problems based on generalized KKT conditions and neurodynamic optimization technique”,
Kybernetika, 51, 890-908.
[32] Mansoori, A., Effati, S. (2019). “An efficient neurodynamic model to solve nonlinear programming problems with fuzzy parameters”, Neurocomputing, 334, 125-133.
[33] Mansoori, A., Effati, S. (2019). “Parametric NCP-based recurrent neural network model: A new strategy to solve fuzzy non-convex optimization problems”, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 1-10.
[34] Modarres, F., Hassan, M.A., Leong, W.J. (2011). “A symmetric rank-one method based on extra updating techniques for unconstrained optimization”, Computers and Mathematics with Applications, 62, 392-400.
[35] Nasiri, J., Modarres Khiyabani, F. (2018). “A whale optimization algorithm (WOA) approach for clustering”, Cogent Mathematics and Statistics, doi.org/10.1080/25742558.2018/1483656.
[36] Nazemi, A.R. (2012). “A dynamic system model for solving convex nonlinear optimization problems”, Communications in Nonlinear Science and Numerical Simulation, 17, 1696-1705.
[37] Nazemi, A.R. (2014). “A neural network model for solving convex quadratic programming problems with some applications problems”, Engineering Applications of Artificial intelligence, 32, 54-62.
[38] Pant, H., Soman Jayadeva, S., Bhaya, A. (2020). “Neurodynamical classifiers with low model complexity”, Neural Networks, 132, 405-415.
[39] Park, S., Jung, S.H., Pardalos, P.M. (2020). “Combining stochastic adaptive cubic regularization with negative curvature for non-convex”, Journal of Optimization Theory and applications, 184 (3), 953-071.
[40] Rudnick-Cohen, E., Herrmann, J.W., Azarm, S. (2020). “Non-convex feasibility robust optimization via scenario generation and local refinement”, Journal of Mechanical Design, 142 (5), 1-10.
[41] Slotine, J.J.E., Li, W. (1990). “Applied Nonlinear Control”, Wiley and Sons, New York.
[42] Strekalovsky, A.S. (2018). “On non-convex optimization problems with D. C. equality and inequality constraints”, IFAC papers online, 51-32, 895-900.
[43] Strekalovsky, A. (2019). “Nonconvex optimization: From global optimality conditions to numerical methods”, AIP Conference Proceedings 2070, 020015, 1-4.
[44] Tank, D.W., Hopfield, J.J. (1986). “Simple neural optimization networks: On A/D converter, signal decision circuit and a linear programming circuit”, IEEE Transactions on Circuits and Systems, 33, 533-541.
[45] Tian, Y., Lu, C. (2011). “Nonconvex quadratic formulations and solvable conditions for mixed integer quadratic programming problems”, Journal of Industrial and Management Optimization, 7 (4), 1027-1039.
[46] Valizadeh Oghani, A., Khiabani, F. M., Farahmand, F.H. (2020). “Data envelopment analysis technique to measure the management ability: Evidence from Iran cement industry”, Cogent Business and Management, doi.org/10.1080/23311975.2020.1801960.
[47] Xu, C., Chai, Y., Qin, S., Wang, Z., Feng, J. (2020). “A neurodynamic approach to nonsmooth constrained pseudo convex optimization problem”, Neural Networks, 124, 180-192.
[48] Xue, X., Bian, W. (2007). “A project neural network for solving degenerate convex quadratic program”, Neurocomputing, 70, 2449-2459.
[49] Yan, Y. (2014). “A new nonlinear neural network for solving quadratic programming problems”, Springer International Publishing Switzerland, 347-357.
[50] Yang, Y., Cao, J., Xu, X., Liu, J. (2012). “A generalized neural network for solving a class of minimax optimization problems with linear constraints”, Applied Mathematics and Computation, 218, 7528-7537.
)