A study of minimax shrinkage estimators dominating the James-Stein estimator under the balanced loss function

1 Introduction

Estimating the mean parameters is one of the most often encountered difficulties in multivariate statistical analysis. Various studies have dealt with this issue in the context of MVN distribution. When the dimensionality of the parameter space is greater than three, the efficiency of the MLE approach is not fulfilled. There are certain limitations to this approach, which have been shown by Stein [1] and James and Stein [2].

A common strategy for enhancing the MLE is the shrinkage estimation approach, which reduces the components of the MLE to zero. The shrinkage estimation approach has been used for enhancing different estimators, such as ordinary least squares estimator [3], and preliminary test and Stein-type shrinkage ridge estimators in robust regression [4]. In the context of enhancing the mean of the MVN distribution, Khursheed [5] studied the domination and admissibility properties of the MLE of a family of shrinkage estimators. Baranchik [6] and Shinozaki [7] also studied the minimaxity of some shrinkage estimators. In addition, several studies have examined the minimaxity and domination properties for various shrinkage estimators under the Bayesian framework, including Efron and Morris [8,9], Berger and Strawderman [10], Benkhaled and Hamdaoui [11], Hamdaoui et al. [12,13], and Zinodiny et al. [14]. Most of these studies have used the quadratic loss function to compute the risk function.

This paper introduces a new class of shrinkage estimators that dominate the James-Stein estimator and the MLE. In order to get a competitive estimator, the estimator has to be unbiased and have a good fit. This can be done by implementing the balanced loss function in the estimation procedure of the competitive estimator. The balanced loss function has been suggested by Zellner [15], and its performance and applications to estimators have been discussed by Sanjari Farsipour and Asgharzadeh [16], JafariJozani et al. [17], and Selahattin and Issam [18].

Therefore, we consider the random vector Z to be normally distributed with an unknown mean vector θ and covariance matrix σ 2 I q , where q is the dimension of parameter space and I q is the q × q identity matrix. As the main object of this paper is to propose a new estimator of θ , we estimated the unknown parameter σ 2 by S 2 ( S 2 ∼ σ 2 χ n 2 ). Then, we construct a new class of shrinkage estimators of θ derived from the MLE. Specifically, the new class of shrinkage estimators is proposed by modifying the James-Stein estimator. We consider adding a term of the form γ ( S 2 / ‖ Z ‖ 2 ) 2 Z to the James-Stein estimator T α ( 1 ) ( Z , S 2 ) = ( 1 − α S 2 / ‖ Z ‖ 2 ) Z , where α and γ are real constant parameters that both depend on the integer parameters n and q . We show that these estimators are minimax and dominating the James-Stein estimator for any values of n and q . The balanced loss function is implemented in the computation of the risk function to compare the efficiency of the proposed estimators over the James-Stein estimator.

The rest of this paper is composed of the following sections: In Section 2, we establish the minimaxity of the estimators defined by T α ( 1 ) ( Z , S 2 ) = ( 1 − α S 2 / ‖ Z ‖ 2 ) Z . Section 3 introduces the new shrinkage estimator class and its domination criterion over the James-Stein estimator. The efficiency of the new estimator classes is explored through simulation studies in Section 4. Then, we conclude our work in Section 5.

2 A class of minimax shrinkage estimators

We assume here the random variable Z is following an MVN distribution with mean vector θ and a covariance matrix σ 2 I q , where the parameters θ and σ 2 are unknown. Thus, the term ‖ Z ‖ 2 σ 2 follows a non-central chi-square distribution with q degrees of freedom and non-centrality parameter λ = ∥ θ ∥ 2 σ 2 . As the aim of this paper is to establish an effective estimator for the mean parameter θ , we consider the statistic S 2 ( S 2 ∼ σ 2 χ n 2 ) as an estimate of the unknown parameter σ 2 . Thus, for any estimator T of θ , the balanced squared error loss function is defined as follows:

where T 0 is the target estimator of θ , ω is the weight given to the closeness between the estimators T and T 0 , and 1 − ω is the relative weight attributed to the accuracy of the estimator T . The associated risk function to the L ω ( T , θ ) function is defined as follows:

Benkhaled et al. [19] demonstrated that the MLE of θ is Z ≔ T 0 . Then, its risk function becomes ( 1 − ω ) q σ 2 . This finding shows the minimaxity and inadmissibility property of T 0 for q ≥ 3 . Consequently, the minimaxity property is also achieved for any estimator that dominates the estimator T 0 .

Now, let consider the estimator

where α is a real constant parameter that can be related to the values of the parameters n and q .

From Proposition (2.1), the minimaxity and domination criterion of the estimator T α ( 1 ) ( Z , S 2 ) to the MLE is achieved under the following condition:

Thus, the risk function R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) is minimized at the optimal α value ( α ^ ) as follows:

Then, by considering α = α ^ , we get the James-Stein estimator

From Proposition 2.1, the risk function of T J S ( Z , S 2 ) is expressed as follows:

Based on equation (5), the positive part of James-Stein estimator can be defined as follows:

where 1 − α ^ S 2 ‖ Z ‖ 2 + = max 0 , 1 − α ^ S 2 ‖ Z ‖ 2 , and its risk function associated with L ω is shown in the following formula:

where I α ^ S 2 ‖ Z ‖ 2 ≥ 1 represents the indicating function of the set α ^ S 2 ‖ Z ‖ 2 ≥ 1 . Both equations (6) and (8) show that R ω ( T J S ( Z , S 2 ) , θ ) and R ω ( T J S + ( Z , S 2 ) , θ ) are less than ( 1 − ω ) q σ 2 = R ω ( Z , θ ) , which proves the domination and minimaxity of both estimators T J S and T J S + over the MLE.

3 The improved shrinkage estimators of the James-Stein estimator

In this section, we construct a class of shrinkage estimators that has the domination property over the James-Stein estimator T J S ( Z , S 2 ) . This class of estimators is a modified version of T J S ( Z , S 2 ) . Specifically, we extend T J S ( Z , S 2 ) given in equation (5) by adding the term γ ( S 2 / ‖ Z ‖ 2 ) 2 Z , where γ behaves like α in equation (2). These new estimators are then investigated regarding their superiority to the James-Stein estimator T J S ( Z , S 2 ) . The modified version of the James-Stein estimator is shown in the following formula:

4 Simulation results

We conduct here a simulation study for comparing the efficiency of the proposed estimators T α ( 1 ) ( Z , S 2 ) and T γ ^ , J S ( 2 ) ( Z , S 2 ) to the estimators T J S ( Z , S 2 ) , T J S + ( Z , S 2 ) and the MLE. We consider here α = ( 1 − ω ) ( q − 2 ) 2 ( n + 2 ) in the estimator T α ( 1 ) ( Z , S 2 ) . This comparison is done based on the risk ratio of these estimators to the MLE. Thus, the risk ratios of these estimators are denoted as follows: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) . We consider here all estimators to be functions of λ = ‖ θ ‖ 2 σ 2 .

Figures 1, 2, 3, 4, 5, show the curve of the risk ratios for simulated values of λ in the interval ( 1 , 30 ) and for relatively low and high values of n , q , and ω . The risk ratio of the MLE is represented by the horizontal line at the value of one. The gap between the curves of estimators indicates the gain magnitude of the estimator. We observed that the curves of all risk ratios for the different sets of n , q , and ω values are entirely located below 1, which indicate the domination of these estimators to the MLE Z . Consequently, these estimators are considered minimax.

Figure 1

Curves of the risk ratios: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) for n = 6 , q = 8 , and ω = 0.1 .

Figure 2

Curves of the risk ratios: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) for n = 20 , q = 8 , and ω = 0.1 .

Figure 3

Curves of the risk ratios: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) for n = 6 , q = 8 , and ω = 0.5 .

Figure 4

Curves of the risk ratios: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) for n = 20 , q = 12 , and ω = 0.1 .

Figure 5

Curves of the risk ratios: R ω ( T α ( 1 ) ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T J S ( Z , S 2 ) , θ ) R ω ( Z , θ ) , R ω ( T γ ^ , J S ( 2 ) ( Z , S 2 ) ) R ω ( Z , θ ) , and R ω ( T J S + ( Z , S 2 ) , θ ) R ω ( Z , θ ) for n = 20 , q = 12 , and ω = 0.5 .

Among these estimators, the positive-part James-Stein estimator ( T J S + ) was the more efficient estimator for values of λ less than approximately 10. It means that T J S + dominates all the considered estimators. Also, we note that the estimator T γ ^ , J S ( 2 ) ( Z , S 2 ) dominates the James-Stein estimator T J S for the various values of n , q , and ω . We also observe a larger gain of the estimator T γ ^ , J S ( 2 ) ( Z , S 2 ) for low values of ω . The gain of the estimators T γ ^ , J S ( 2 ) ( Z , S 2 ) , T J S ( Z , S 2 ) , and T J S + ( Z , S 2 ) was very similar in a specific period of λ values, which depend on the combination of the values of the n and q . To study this similarity, we conduct simulation studies for all combinations of the selected values of n and q for different sets of values of λ and ω .

Tables 1, 2, 3, 4 show the results of the risk ratios of the estimators T α ( 1 ) ( Z , S 2 ) , T γ ^ , J S ( 2 ) ( Z , S 2 ) , and T J S ( Z , S 2 ) . Each cell of the tables represents the risk ratio of these estimators in order. We observe a strong relationship between the gain of the risk ratios and the values of ω and λ . The gain of all risk ratios was large with small values of λ and ω and tended to vanish with the increase of λ and ω values. Also, the difference in the gain of risk ratios was observed in small values of λ . This difference indicated the domination of an estimator to another. Thus, the estimator T γ ^ , J S ( 2 ) ( Z , S 2 ) dominated both estimators T α ( 1 ) ( Z , S 2 ) and T J S ( Z , S 2 ) for small values of λ . However, as the values of λ and ω increased, the difference in the gain of these estimators became negligible (i.e., no improvement of the proposed estimators over the James-Stein estimator). The other parameters n and q have also an influence on the gain of the estimators. The gain of the estimators was large for large values of n and q under fixed values of ω . Specifically, the increase of q had significant influence on the gain than the increase of n values. This means that having large values of n , q , and λ with value of ω close to zero leads to a larger gain of the estimators, which leads to a significant improvement. Thus, we conclude that the improvement of the considered estimators is clearly affected by the values of the parameters n , q , ω , and λ .