In collaboration with Payame Noor University and the Iranian Society of Instrumentation and Control Engineers

Document Type : Research Article

Authors

1 Department of Mathematics, University of Thi-Qar, Thi-Qar, Iraq

2 Department of Mathematics, College of Education for Pure Sciences, University of Thi-Qar, Nasiriyah, Iraq

Abstract

Estimating the differential entropy of a normal distribution reduces to estimating the log-variance, but plug-in estimation is biased and target-based shrinkage can be sensitive to misspecification. We propose AROS, a smooth finite-sample shrinkage framework built on the exact log-Gamma law of the sample variance. A digamma correction provides an unbiased entropy baseline, and AROS-S chooses its tuning parameter by minimizing risk over a declared target-uncertainty set subject to a worst-case risk bound. Exact risks are evaluated under quadratic and asymmetric exponential loss, with independent quadrature and Monte Carlo validation. Under quadratic loss and exact target specification (ξ = 0), AROS-0 reduced risk by 51–53% for n = 5–100; this gain was local rather than uniform. For b = 1, AROS-S satisfied the safety bound and had lower worst-case quadratic risk than the reconstructed Huntsberger comparator across the examined sample sizes and significance levels. A Michelson speed-of-light illustration shows how an estimated historical variance can provide a provisional calibration of the target-uncertainty radius; bootstrap intervals describe variability under the reported resampling scheme, conditional on the historical target and tuning choices, and not estimation accuracy. The results support smooth target borrowing when a scientifically defensible uncertainty set is available, with automatic reversion to the baseline when safe shrinkage is not justified.

Highlights

  • Exact log-Gamma theory gives an unbiased entropy baseline and finite-sample risks for normal data.
  • A smooth, bounded adaptive weight borrows an external variance target without any pre-test step.
  • At the exact target under quadratic loss, AROS-0 cuts risk by 51–53% for n = 5–100; the gain is local.
  • AROS-S tunes shrinkage under a worst-case risk bound and reverts to the baseline when unsafe.
  • For b = 1, AROS-S has lower worst-case quadratic risk than the Huntsberger-type pre-test estimator.

Keywords

Main Subjects

[1] ALoraini, N. M., Hamdaoui, A., Benkhaled, A. (2026). “A new approach for shrinkage estimators of the multivariate normal mean vector under the balanced loss criterion”. Frontiers in Applied Mathematics and Statistics, 11, 1709968. https://doi.org/10.3389/ fams.2025.1709968
[2] Al-Qaragholi, O. J. A., Jiheel, A. K. (2026). “Optimization-oriented double pre-test shrinkage estimators for Pareto reliability under progressive Type-II censoring and precautionary loss”. Control and Optimization in Applied Mathematics, 11(2), 83–103. https: //doi.org/10.30473/coam.2026.77066.1388
[3] Armstrong, T. B., Kline, P., Sun, L. (2025). “Adapting to misspecification”. Econometrica, 93(6), 1981–2005. https://doi.org/10.3982/ECTA21991
[4] Armstrong, T. B., Kolesár, M. (2020). “Simple and honest confidence intervals in nonparametric regression”. Quantitative Economics, 11(1), 1–39. https://doi.org/10.3982/ QE1199
[5] Cai, T. T., Liang, T., Zhou, H. H. (2015). “Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional Gaussian distributions”. Journal of Multivariate Analysis, 137, 161–172. https://doi.org/10.1016/ j.jmva.2015.02.003
[6] Casella, G., Strawderman, W. E. (1981). “Estimating a bounded normal mean”. The Annals of Statistics, 9(4), 870–878. https://doi.org/10.1214/aos/1176345527
[7] Edelmann, D. (2026). “The optimal standardization factor for the variance of the normal distribution: A historical perspective and recommendations for practice”. Journal of Statistical Theory and Practice, 20, Article 49. https://doi.org/10.1007/ s42519-026-00567-2
[8] Efron, B., Morris, C. (1973). “Stein’s estimation rule and its competitors—An empirical Bayes approach”. Journal of the American Statistical Association, 68(341), 117–130. https://doi.org/10.1080/01621459.1973.10481350
[9] Gray, H., Leday, G. G. R., Vallejos, C. A., Richardson, S. (2018). “Shrinkage estimation of large covariance matrices using multiple shrinkage targets”. arXiv preprint arXiv:1809.08024. https://doi.org/10.48550/arXiv.1809.08024
[10] Gupta, M., Srivastava, S. (2010). “Parametric Bayesian estimation of differential entropy and relative entropy”. Entropy, 12(4), 818–843. https://doi.org/10.3390/ e12040818
[11] Ikeda, Y., Nakada, R., Kubokawa, T., Srivastava, M. S. (2021). “Linear shrinkage estimation of the variance of a distribution with unknown mean”. Communications in Statistics – Theory and Methods, 50(9), 2039–2047. https://doi.org/10.1080/03610926. 2019.1657457
[12] Kozachenko, L. F., Leonenko, N. N. (1987). “Sample estimate of the entropy of a random vector”. Problems of Information Transmission, 23(2), 95–101.
[13] Kubokawa, T. (1999). “Shrinkage and modification techniques in estimation of variance and the related problems: A review”. Communications in Statistics – Theory and Methods, 28(3–4), 613–650. https://doi.org/10.1080/03610929908832317
[14] Kubokawa, T. (2024). “Stein’s identities and the related topics: An instructive explanation on shrinkage, characterization, normal approximation and goodness-of-fit”. Japanese Journal of Statistics and Data Science, 7, 267–311. https://doi.org/10. 1007/s42081-023-00239-6
[15] Maatta, J. M., Casella, G. (1990). “Developments in decision-theoretic variance estimation”. Statistical Science, 5(1), 90–101. https://doi.org/10.1214/ss/1177012263
[16] Misra, N., Singh, H., Demchuk, E. (2005). “Estimation of the entropy of a multivariate normal distribution”. Journal of Multivariate Analysis, 92(2), 324–342. https://doi.org/10.1016/j.jmva.2003.10.003
[17] Ohtani, K. (1999). “Risk performance of a pre-test estimator for normal variance with the Stein-variance estimator under the LINEX loss function”. Statistical Papers, 40(1), 75–87. https://doi.org/10.1007/BF02927111
[18] Sahni, P., Kumar, R. (2023). “Huntsberger type shrinkage entropy estimator for variance of normal distribution under LINEX loss function”. Reliability: Theory & Applications, 18(1), 491–499. https://doi.org/10.24412/1932-2321-2023-172-491-499
[19] Stein, C. M. (1964). “Inadmissibility of the usual estimator for the variance of a normal distribution with unknown mean”. Annals of the Institute of Statistical Mathematics, 16, 155–160. https://doi.org/10.1007/BF02868569
[20] Stigler, S. M. (1977). “Do robust estimators work with real data?” The Annals of Statistics, 5(6), 1055–1098. https://doi.org/10.1214/aos/1176343997
[21] Tong, T., Jang, H., Wang, Y. (2012). “James–Stein type estimators of variances”. Journal of Multivariate Analysis, 107, 232–243. https://doi.org/10.1016/j.jmva.2012. 01.019
[22] Tong, T., Wang, Y. (2007). “Optimal shrinkage estimation of variances with applications to microarray data analysis”. Journal of the American Statistical Association, 102(477), 113–122. https://doi.org/10.1198/016214506000001266
[23] Vasicek, O. (1976). “A test for normality based on sample entropy”. Journal of the Royal Statistical Society: Series B (Methodological), 38(1), 54–59. https://doi.org/10.1111/j.2517-6161.1976.tb01566.x
[24] Zellner, A. (1986). “Bayesian estimation and prediction using asymmetric loss functions”. Journal of the American Statistical Association, 81(394), 446–451. https://doi.org/ 10.1080/01621459.1986.10478289
[25] Zhang, Y.-Y. (2026). “The empirical Bayes estimators of the variance parameter of the normal distribution with a normal-inverse-gamma prior under Stein’s loss function”. Axioms, 15(2), 127. https://doi.org/10.3390/axioms15020127