Control and Optimization in Applied Mathematics

In collaboration with Payame Noor University and the Iranian Society of Instrumentation and Control Engineers

Current Issue

By Issue

By Author

By Subject

Author Index

Keyword Index

About Journal

Aims and Scope

Editorial Board

Publication Ethics

Indexing and Abstracting

Related Links

FAQ

Peer Review Process

Journal Metrics

News

Special Issues

Novel Hybrid Conjugate Gradient Algorithms via Newton-Direction Alignment: Convergence Analysis and Image Restoration Applications

Articles in Press

Document Type : Research Article

Authors

1 8th May 1945 University, Guelma, 24000, Algeria

2 College of Computer Science and Mathematics, University of Mosul, Iraq

Abstract

This paper introduces two hybrid nonlinear conjugate gradient algorithms, NRB1 and NRB2, for solving unconstrained optimization problems. Both methods are constructed as a convex combination of the AlBayati–AlAssady (BA) and Conjugate Descent methods (CD). The first variant, denoted NRB1, employs an adaptive combination parameter designed to align its search direction more closely with Newton’s method, improving curvature approximation and acceleration of convergence. The second variant, NRB2, independently satisfies the conjugacy condition without relying on line search mechanisms, enhancing numerical stability. Both methods guarantee sufficient descent and global convergence properties under the strong Wolfe line search criteria. Extensive numerical experiments, using the performance profile of Dolan and Moré, demonstrate that NRB1 and NRB2 consistently outperform some classical conjugate gradient methods, including BA, CD, and Dai–Yuan, particularly for large-scale problems. Furthermore, NRB1 is applied to image restoration under salt-and-pepper noise, achieving competitive or superior peak signal-to-noise ratios (PSNR) compared to the Fletcher–Reeves (FR) algorithm with significantly fewer iterations, especially at high noise intensities.

Highlights

  • Two novel hybrid conjugate gradient algorithms (NRB1 and NRB2) are proposed as convex combinations of the AlBayati–AlAssady (BA) and Conjugate Descent (CD) methods.
  • NRB1 aligns the search direction with Newton's direction via an adaptive combination parameter, improving curvature approximation and convergence speed.
  • NRB2 satisfies the conjugacy condition independently of the line search, enhancing numerical stability.
  • Both algorithms provably satisfy the sufficient descent condition and achieve global convergence under strong Wolfe line search conditions.
  • Extensive numerical experiments using Dolan–Moré performance profiles confirm that NRB1 outperforms classical methods (BA, CD, DY) in terms of iterations, CPU time, and function evaluations, especially for large-scale problems.
  • The NRB1 method demonstrates remarkable efficiency in image restoration problems: it achieves competitive or superior PSNR on four benchmark images (Lena, House, Elaine, Cameraman) corrupted by salt-and-pepper noise at 50%, 70%, and 90% levels, with fewer iterations than Fletcher–Reeves (FR).

Keywords

Main Subjects

References
[1] Andrei, N. (2008). “Another hybrid conjugate gradient algorithm for unconstrained optimization”. Numerical Algorithms, 47(1), 143–156. https://doi.org/10.1007/ s11075-007-9152-9.
[2] Andrei, N. (2009). Another nonlinear conjugate gradient algorithm for unconstrained optimization. Optimization Methods and Software, 24(1), 89–104. https://doi.org/10. 1080/10556780802393326
[3] Andrei, N. (2020). Nonlinear conjugate gradient methods for unconstrained optimization. Springer International Publishing, Cham. (Springer Optimization and Its Applications, Vol. 158). https://doi.org/10.1007/978-3-030-42950-8.
[4] Al-Bayati, A. Y., Al-Assady, N. H. (1986). “Conjugate gradient method”. Technical Report, School of Computer Studies, University of Leeds, UK.
[5] Babaie–Kafaki, S., Mirhoseini, N., Aminifard, Z. (2023). “A descent extension of a modified Polak–Ribiere–Polyak method with application in image restoration problem”. Optimization Letters, 17(2), 351–367. https://doi.org/10.1007/ s11590-022-01878-6.
[6] Bongartz, I., Conn, A. R., Gould, N. I. M., & Toint, P. L. (1995). “CUTE: constrained and unconstrained testing environments”. ACM Transactions on Mathematical Software, 21, 123–160. https://doi.org/10.1145/200979.201043.
[7] Djordjević, S. S. (2017). New hybrid conjugate gradient method as a convex combination of LS and CD methods. Filomat, 31(6), 1813–1825. https://doi.org/10.2298/ FIL1706813D
[8] Dolan, E. D., & Moré, J. J. (2002). Benchmarking optimization software with performance profiles. Mathematical Programming, 91(2), 201–213. https://doi.org/10. 1007/s101070100263
[9] Fletcher, R. (1987). Practical methods of optimization (2nd ed.). John Wiley & Sons. https://doi.org/10.1002/9781118723203
[10] Fletcher, R., Reeves, C. M. (1964). Function minimization by conjugate gradients. The Computer Journal, 7(2), 149–154. https://doi.org/10.1093/comjnl/7.2.149
[11] Hamdi, A., Sellami, B., Belloufi, M. (2020). “A new hybrid conjugate gradient method as a convex combination of BA and DY methods”. Communications in Optimization Theory, 2020 (Article ID 5), 1–9. https://doi.org/10.23952/cot.2020.5.
[12] Hassan, B. A., Sadiq, H. M. (2022). Efficient new conjugate gradient methods for removing impulse noise images. European Journal of Pure and Applied Mathematics, 15(4), 2011–2021. https://doi.org/10.29020/nybg.ejpam.v15i4.4568
[13] Hestenes, M. R., Stiefel, E. (1952). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6), 409–436. https://doi.org/10.6028/jres.049.044
[14] Liu, J. K., Li, S. J. (2014). “New hybrid conjugate gradient method for unconstrained optimization”. Applied Mathematics and Computation, 245, 36–43. https://doi.org/ 10.1016/j.amc.2014.07.096
[15] Mellal, R., Sellami, N. (2025). A novel hybrid conjugate gradient algorithm for solving unconstrained optimization problems. Statistics, Optimization and Information Computing. https://doi.org/10.19139/soic-2310-5070-2807
[16] Mellal, R., Sellami, N., Hassan, B. A. (2025). Novel hybrid conjugate gradient technique based on the Newton direction applied to image restoration problem. Statistics, Optimization and Information Computing, 15(3), 1758–1775. https://doi.org/10. 19139/soic-2310-5070-2897
[17] Nocedal, J., Wright, S. J. (2006). Numerical optimization (2nd ed.). Springer. https:// doi.org/10.1007/978-0-387-40065-5
[18] Powell, M. J. D. (1977). “Restart procedures of the conjugate gradient method”. Mathematical Programming, 2, 241–254. https://doi.org/10.1007/BF01593790
[19] Wolfe, P. (1969). “Convergence conditions for ascent methods”. SIAM Review, 11, 226– 235.
[20] Wolfe, P. (1971). “Convergence conditions for ascent methods. II: Some corrections”. SIAM Review, 13, 185–188
Control and Optimization in Applied Mathematics

Articles in Press, Accepted Manuscript
Available Online from 25 May 2026
Files
Share
How to cite
Statistics