Skip to content
Publicly Available Published by De Gruyter June 15, 2017

Generalized Confidence Intervals for Intra- and Inter-subject Coefficients of Variation in Linear Mixed-effects Models

  • Johannes Forkman ORCID logo EMAIL logo

Abstract

Linear mixed-effects models are linear models with several variance components. Models with a single random-effects factor have two variance components: the random-effects variance, i. e., the inter-subject variance, and the residual error variance, i. e., the intra-subject variance. In many applications, it is practice to report variance components as coefficients of variation. The intra- and inter-subject coefficients of variation are the square roots of the corresponding variances divided by the mean. This article proposes methods for computing confidence intervals for intra- and inter-subject coefficients of variation using generalized pivotal quantities. The methods are illustrated through two examples. In the first example, precision is assessed within and between runs in a bioanalytical method validation. In the second example, variation is estimated within and between main plots in an agricultural split-plot experiment. Coverage of generalized confidence intervals is investigated through simulation and shown to be close to the nominal value.

1 Introduction

The coefficient of variation is the sample standard deviation divided by the sample mean. This measure of dispersion is used in a wide range of applications where the standard deviation tends to increase linearly with the level of the observations. In these applications, observations cannot take negative values. Specifically, measurements are made on a ratio scale. The coefficient of variation can be used as an estimator of the population coefficient of variation, defined as the square root of the population variance, i. e., the square root of the second central moment, divided by the expected value.

Much research has been made on inference for a single coefficient of variation. When the sample consists of independent observations from a normal distribution, an exact confidence interval for the coefficient of variation can be computed using the noncentral t-distribution [1]. Approximate confidence intervals have been developed using asymptotic normality [2] and Taylor series expansion [3]. Vangel [4] and Forkman [5] advocated the use of McKay’s approximation [6, 7] for the coefficient of variation. The approximate confidence interval method proposed by Forkman [5] can be used when there are samples from populations with different expected values but with a common coefficient of variation, which is a situation that was also considered by Verrill and Johnson [8]. Hayter [9] constructed a bounded confidence interval for a coefficient of variation when observations are normally distributed. Wong and Wu [10] proposed a confidence interval using a modified signed log likelihood ratio method, which can also be used for gamma distributed observations. They showed that for small samples of normally distributed observations, this method performs as well as the Vangel [4] method.

The present article considers linear random-effects and mixed-effects models for balanced and unbalanced sets of normally distributed observations. In balanced datasets, each sub-class contains the same number of observations (the exact mathematical definition of balance is complex but has been formalized by a number of authors; for further references, see [11, p.4]). Linear random-effects and mixed-effects models include several variance components. It is often practical to express the estimates of the variances as coefficients of variation, i. e., as standard deviations divided by the mean.

The intra-subject coefficient of variation assesses the reproducibility or reliability of a measurement [12]. In bioanalytical method validation, coefficients of variation specify within- and between-run precision [13]. In crop variety testing, expressing variability as coefficients of variation facilitates comparison of variability between crops [14].

Shoukri et al. [15] derived confidence intervals, based on a variance-stabilizing transformation, for the intra-subject coefficient of variation in the balanced one-way random-effects model. The present article derives generalized confidence intervals for intra- and inter-subject coefficients of variation in balanced and unbalanced random-effects and mixed-effects models with two variance components. Generalized confidence intervals are constructed using generalized pivotal quantities [16]. This type of confidence intervals has already been suggested for some other inferential problems on the coefficient of variation: Tian [17], Behboodian and Jafari [18] and Jafari [19] proposed generalized confidence intervals for a single coefficient of variation shared by many populations, and Krishnamoorthy and Lee [20] introduced generalized confidence intervals for differences and ratios of two coefficients of variation. The generalized pivotal quantities approach may also be used for the problem of testing equality of coefficients of variation [18, 19].

In the balanced one-way random-effects model, no exact bounds for the inter-subject variance are available that are functions of complete sufficient statistics [21]. A method for construction of generalized pivotal quantities and generalized confidence intervals for variance components was developed by Chiang [22], Iyer and Mathew [23], Iyer and Patterson [24] and Hannig et al. [25]. Their method is here extended to the problem of computing generalized confidence intervals for coefficients of variation. Burdick et al. [26] presented generalized confidence intervals for functions of variance components in mixed-effects models with balanced datasets and random-effects models with balanced or unbalanced datasets. For mixed-effects models and unbalanced datasets, the approach of the present article is similar to the method that was proposed for ‘a more general model’ by Gamage et al. [27], who considered the problem of computing confidence intervals for best linear unbiased predictions. However, their method uses transformation of data, but the method proposed in the present article does not.

The objective of this article is to propose methods for computing generalized confidence intervals for intra- and inter-subject coefficients of variation in mixed-effects models with two variance components. Section 2 provides theory and proposes methods. Section 3 studies, through simulation, the performance of the proposed methods and compares with the method proposed by Shoukri et al. [15]. Section 4 gives two examples: a bioanalytical method validation exemplifies the use of a one-way random-effects model, and an agricultural field experiment exemplifies a mixed-effects split-plot model. R scripts for these examples are published online as supplementary material on the Journal’s web page.

2 Theory and methods

2.1 Generalized confidence intervals

Let Y denote a random vector with a distribution that is dependent on a vector ξ of parameters, and let y be a realization, i. e. an observation, of Y. Assume that we are interested in a function θ of the parameters: θ=fξ. A pivotal quantity for θ is a function g of Y and θ if the distribution of gY,θ does not depend on ξ [28]. A generalized pivotal quantity for θ is a function h of y, Y and ξ if i) the distribution of hy,Y,ξ, where y is fixed, does not depend on ξ, and ii)h(y,Y,ξ|Y=y), i. e., hy,y,ξ, is a function of y and θ only [16]. If specifically hy,y,ξ=θ, then the function h is a fiducial generalized pivotal quantity for θ [25]. A 1α equal-tailed generalized confidence interval for θ can be obtained as the interval between the α/2th and the (1α/2)th quantiles of the distribution of hy,Y,ξ, which can be estimated through simulation.

2.2 Statistical model

Consider the linear mixed-effects model

(1)y=Bβ+Cu+e

where y is an N-vector of observations, B is a design matrix for fixed effects, β is a vector of fixed effects parameters, C is an N×a design matrix for random effects, u is an a-vector of realized random effects, and e is an N-vector of residual error effects. The vectors of random effects u and e are realizations of the random vectors UN0,σA2Ia and EN0,σE2IN, which are independent. Thus, the vector of observations y is a realization of the random vector Y=Bβ+CU+E, and YNBβ,V, where V=CCTσA2+INσE2. Let μ denote the expected value of the mean of the elements of EY when the dataset is balanced. The intra- and inter-subject coefficients of variation are defined asγE=σE/μ and γA=σA/μ, respectively.

In a more general setting, the random variables of U need not be normally distributed. When these variables are independent and identically distributed, but not necessarily normal, model (1) is the semiparametric mixed-effects model for clustered data proposed by Tao et al. [29] When the distributions of the random effects and errors are not known, or when the linear form of the conditional expectation might be questioned, model (1), including the assumptions of normality, may be considered as a working model. Generally, σA2 and σE2 can be defined as the variance of the distribution of the random variables of U and E.

2.3 Balanced datasets

Let n be the number of observations at each level of the random-effects factor, let XA and XE denote the random sum of squares associated with the random-effects factor and the residual errors, respectively, and let Yˉ denote the random overall mean, i. e., Yˉ=1NTY/N. Then

(2)UA=XAnσA2+σE2,UE=XEσE2

are central chi-square distributed with numbers of degrees of freedom depending on the design, and

(3)Z=YˉμNnσA2+σE2,

where μ=EYˉ, is standard normal distributed. Let xA and xE denote the observed sum of squares associated with the random-effects factor and the residual errors, respectively, and let yˉ denote the observed overall mean, i. e., yˉ=1NTy/N. If, in (2) and (3), xA, xE and yˉ are substituted for XA, XE and Yˉ, respectively, the following quantities are obtained:

(4)σˆE2=xEUE,σˆA2=xAnUAσˆE2n,
(5)μˆ=yˉZnσˆA2+σˆE2N.

The expressions in (4) and (5) are fiducial generalized pivotal quantities for σE2, σA2 and μ. Indeed, since conditioned on y, the statistics xE, xA and yˉ are constants, the quantities in (4) and (5) do not depend on ξ=σA2,σE2,μ, and when Y=y, the random variables XA, XE and Yˉ equals xE, xA and yˉ, respectively, which by (2) and (3) implies σˆE2|Y=y=σE2, σˆA2|Y=y=σA2 and μˆ|Y=y=μ. Fiducial generalized pivotal quantities γˆE and γˆA, for the intra- and inter-subject coefficients of variation, respectively, are easily obtained as

(6)γˆE=σˆE2μˆ,γˆA=max0,σˆA2μˆ.

Generalized 1α confidence intervals are obtained through simulation of generalized pivotal quantities. The confidence interval limits are the α/2th and the (1α/2)th quantiles of the simulated values. Technically, negative confidence interval limits can be obtained for μ, σA2, γE and γA, although these parameters are known to be positive. Negative confidence interval limits can be replaced with zeros.

2.4 Unbalanced datasets

Let A denote the Moore-Penrose generalized inverse of A, where A is any matrix. Let D=BC and XE=YTINDDTDDTY. Since INDDTDDTV/σE2 is idempotent (see Appendix) and INDDTDDTB=0, the quadratic form UE=XE/σE2 is central chi-square distributed with d=rank(INDDTDDT) degrees of freedom [11, p. 467]. If the observed error sum of squares xE=yTINDDTDDTy is substituted for XE in the expression for UE, a fiducial generalized pivotal quantity σˆE2 for σE2 is obtained as σˆE2=xE/UE.

Similarly, let P=V1V1BBTV1BBTV1 and XT=YTPY. Since PV is idempotent (see Appendix) and PB=0, the quadratic form XT is central chi-square distributed with t=rank(P) degrees of freedom. Let σˆA2 be the v such that yTPvyYTPY=0, where Pv=Vv1Vv1BBTVv1BBTVv1 with Vv=CCTv+INσˆE2. Then σˆA2 is a fiducial generalized pivotal quantity for σA2. This is so, because σˆA2 is a function of y, Y and ξ=σA2,σE2,μ, which equals σA2when Y=y, and the distribution of σˆA2, conditioned on y, does not depend on ξ. Indeed, the distribution of yTPvyYTPY is the same as the distribution of gv=yTPvyxE/σˆE2UA, where UAχtd2. Since the distribution of gv, conditional on y, does not depend on ξ, neither does the distribution of σˆA2, conditional on y. Given random realizations of UE and UA, the fiducial generalized pivotal quantity σˆA2 can be computed through solving the equation gv=0. This equation can be solved using the Newton-Raphson procedure, utilizing that the derivative of gv is gv=yTPvCCTPvy [30, p. 325].

The unbalanced dataset can be regarded as a subset of a corresponding balanced dataset with M observations, where M>N. Let B0 denote the fixed-effects design matrix for the balanced dataset. The population mean can then be defined as μ=1MTB0β/M. Consider the random variable Yˉ=1MTB0βˆ/M, where βˆ=BTV1BBTV1Y and V is known. Since Yˉ is Nμ,1MTB0BTV1BB0T1M/M2, we can write

μˆ=yˉZ1MTB0BTVˆ1BB0T1M/M2
=dyˉYˉμ1MTB0BTVˆ1BB0T1M/M21MTB0BTV1BB0T1M/M2,

where Vˆ=CCTσˆA2+INσˆE2, ZN0,1. Since Vˆ=V when y=Y, the quantity μˆ takes the value μ when y=Y. The distribution of μˆ, conditioned on yˉ, is not dependent on ξ=σA2,σE2,μ. Thus, μˆ is a fiducial generalized pivotal quantity for μ. Given σˆE2, σˆA2 and μˆ, as specified for unbalanced datasets, fiducial generalized pivotal quantities for γE and γA can be obtained through the same eq. (6) as used for balanced datasets. In repeated sampling, Vv and Vˆ can become computationally singular. For this reason, Vv and Vˆ may be used instead of Vv1 and Vˆ1.

3 Simulation studies

3.1 The one-way random-effects model

Consider the one-way random-effects model yij=μ+ai+eij, where yij is the jth observation at the ith level, μ is an intercept, ai is an effect of the ith level, and eij is a residual error of the jth observation at the ith level, i= 1, 2, …, a; j= 1, 2, …, n. It is assumed that yij, ai and eij are realized values of the random variables Yij, Ai and Eij, respectively, where AiN0,σA2, EijN0,σE2, and Yij=μ+Ai+Eij.

Following Shoukri et al. [15], observations were simulated with intercept μ=10 and γE{0.01, 0.02, 0.04}, which corresponds to intra-subject variances σE2{0.01, 0.04, 0.16}. Shoukri et al. [15] studied n{2, 3, 5} and ρ{0.3, 0.4, 0.6, 0.7, 0.8}, where ρ is the intra-class correlation coefficient: ρ=σA2/σA2+σE2. The present evaluation was confined to n{2, 5} and ρ{0.3, 0.6, 0.8}. Finally, Shoukri et al. [15] investigated numbers of subjects a{12, 25, 50, 75}, but the present study was extended to include cases a{4, 8, 12, 25, 50, 75}. In order to evaluate the performance of the method for unbalanced data, the same cases were explored, but in each simulated dataset 0.125N randomly selected observations were removed before analysis. The symbol denotes the floor function, so that 0.125N is the largest integer not greater than 0.125N, where N=an. For each of the 54 cases, 10,000 balanced and 1000 unbalanced datasets were randomly generated. Since analyses of unbalanced datasets were more time consuming, less unbalanced than balanced datasets were simulated. For the 10,000 balanced datasets, 95 % confidence intervals for γE were computed using the method proposed by Shoukri et al. [15]. In addition, generalized 95 % confidence intervals were computed for γA and γE, each using 1000 random samples of the fiducial generalized pivotal quantities γˆE and γˆA. Generalized 95 % confidence intervals for γE and γA were also computed for the 1000 unbalanced datasets, each based on 1000 random samples of the fiducial generalized pivotal quantities γˆE and γˆA.

The Newton-Raphson procedure employed in the method for unbalanced datasets used v=0 as starting value. The procedure was stopped when gv<1010, v>1010, or the maximum number of iterations, 100, was reached. Results obtained at the maximum number of iterations were discarded.

Tables 1 and 2 present observed frequencies of confidence intervals covering the true values of γE and γA, for cases with n=2 and n=5, respectively. In the simulations of balanced datasets, the generalized 95 % confidence intervals showed coverage close to the nominal level 0.95, although it was noted that coverage of generalized 95 % confidence intervals for γA was slightly too large for small values of a when n=2 (Table 1). Also in simulations of unbalanced datasets, the generalized 95 % confidence intervals covered the true values in approximately 95 % of the cases. However, confidence intervals tended to be somewhat too narrow, especially for γE when n=2. Shoukri et al. [15] showed that their 95 % confidence intervals cover the true γE with approximately probability 0.95 when a=12 and n=5. When a=12 and n=2, the probability is lower than 0.95. The present study confirms their results, but reveals that coverage of the Shoukri et al. [15] 95 % confidence interval is too small when a=4 and n=2 (Table 1). In this situation, the generalized 95 % confidence interval introduced here performs better. Tables A1 and A2, published online as supplementary material on the Journal’s web page, present the results for a{25, 50, 75}. In these cases, observed coverage was close to 0.95.

Table 1:

Estimated coverage of 95 % confidence intervals (C.I.) for the intra- and inter-subject coefficients of variation γE and γA in a one-way random-effects model when the number of observations per subject is n=2. Random effects are normally distributed. Coverage depends on the number of subjects, a, the intra-subject coefficient of variation, γE, the intra-class correlation coefficient ρ, and whether the dataset is balanced or not.

Shoukri 0.95 C.I.Generalized 0.95 C.I.
BalancedBalancedUnbalanced
aγEργEγEγAγEγA
40.010.30.9060.9480.9610.9490.947
0.60.9070.9460.9540.9340.959
0.80.9090.9540.9570.9400.950
0.020.30.9100.9490.9530.9460.950
0.60.9060.9490.9550.9400.958
0.80.9040.9480.9550.9380.948
0.040.30.9080.9480.9550.9440.950
0.60.9050.9500.9570.9400.958
0.80.9060.9490.9520.9360.954
80.010.30.9330.9490.9530.9490.945
0.60.9330.9520.9520.9450.953
0.80.9280.9530.9490.9560.944
0.020.30.9260.9460.9540.9520.949
0.60.9270.9460.9510.9450.964
0.80.9320.9510.9490.9460.953
0.040.30.9290.9490.9520.9460.943
0.60.9310.9500.9510.9590.954
0.80.9290.9490.9510.9400.957
120.010.30.9370.9480.9490.9380.936
0.60.9380.9520.9480.9470.943
0.80.9340.9490.9470.9280.938
0.020.30.9320.9470.9490.9390.953
0.60.9340.9480.9520.9340.939
0.80.9350.9480.9480.9350.948
0.040.30.9350.9480.9480.9440.951
0.60.9330.9480.9510.9460.947
0.80.9370.9460.9520.9430.944
Mean0.9240.9490.9520.9430.949
Table 2:

Estimated coverage of 95 % confidence intervals (C.I.) for the intra- and inter-subject coefficients of variation γE and γA in a one-way random-effects model when the number of observations per subject is n=5. Random effects are normally distributed. Coverage depends on the number of subjects, a, the intra-subject coefficient of variation, γE, the intra-class correlation coefficient ρ, and whether the dataset is balanced or not.

Shoukri 0.95 C.I.Generalized 0.95 C.I.
BalancedBalancedUnbalanced
aγEργEγEγAγEγA
40.010.30.9400.9490.9470.9540.952
0.60.9420.9530.9480.9400.945
0.80.9360.9450.9520.9520.953
0.020.30.9410.9490.9510.9560.945
0.60.9430.9520.9450.9450.950
0.80.9400.9480.9500.9400.941
0.040.30.9440.9570.9500.9570.938
0.60.9340.9470.9510.9410.955
0.80.9400.9580.9530.9540.946
80.010.30.9440.9480.9510.9650.951
0.60.9470.9490.9520.9490.943
0.80.9440.9500.9500.9380.933
0.020.30.9440.9450.9510.9380.947
0.60.9440.9470.9480.9450.952
0.80.9440.9470.9500.9310.949
0.040.30.9400.9460.9500.9480.950
0.60.9460.9530.9480.9440.940
0.80.9470.9520.9470.9440.950
120.010.30.9460.9470.9490.9480.937
0.60.9450.9470.9500.9420.939
0.80.9460.9480.9490.9490.945
0.020.30.9500.9530.9460.9410.962
0.60.9460.9480.9510.9520.956
0.80.9470.9470.9470.9570.964
0.040.30.9510.9510.9490.9420.951
0.60.9450.9480.9490.9460.946
0.80.9450.9460.9470.9470.955
Mean0.9440.9490.9490.9470.948

The semiparametric mixed-effects model for clustered data (Section 2.2) allows for other random-effects distributions than the normal. In a study of robustness, three other distributions were investigated: a logistic distribution, a gamma distribution with shape parameter 4, and a gamma distribution with shape parameter 16. These distributions differ with respect to skewness and kurtosis. For a random variable Y with expected value μY and standard deviation σY, skewness and kurtosis may be defined as EYμY3/σY3 and EYμY4/σY43, respectively. Using these definitions, skewness and kurtosis are 0 for the normal distribution. The shape of the density function of the logistic distribution is similar to the normal, because skewness is 0, but kurtosis is 1.2. Skewness and kurtosis of a gamma distribution with shape parameter α is 2/α and 6/α, respectively. A gamma distribution with shape parameter 4 is more skewed and has heavier tails than a gamma distribution with shape parameter 16.

Tables 3 and 4 shows estimated coverage when the random-effects distribution is non-normal, for n=2 and n=5, respectively. Each case was simulated 10,000 times. Computations of confidence intervals were carried out using the same methods as previously specified, based on the now incorrect assumption of normally distributed random effects of subjects. Generalized confidence intervals were computed using 1000 random samples of fiducial generalized pivotal quantities.

Table 3:

Estimated coverage of 95 % confidence intervals (C.I.) for the intra- and inter-subject coefficients of variation γE and γA in a one-way random-effects model when the number of observations per subject is n=2. Between-subjects effects are logistic distributed, gamma distributed with shape parameter 16, and gamma distributed with shape parameter 4. Error effects are normally distributed. Coverage depends on the number of subjects, a, the intra-subject coefficient of variation, γE, and the intra-class correlation coefficient ρ.

Shoukri 0.95 C.I.Generalized 0.95 C.I.
LogisticΓ(16)Γ(4)LogisticΓ(16)Γ(4)
aγEργEγEγEγEAγEγAγEγA
40.010.30.9100.9090.9030.9490.9540.9500.9520.9440.955
0.60.9070.9080.9110.9490.9490.9490.9540.9510.946
0.80.9070.9090.9070.9440.9410.9430.9520.9470.934
0.020.30.9060.9050.9030.9490.9500.9480.9480.9480.949
0.60.9090.9100.9120.9470.9470.9530.9550.9500.946
0.80.9110.9030.9070.9490.9370.9490.9500.9530.936
0.040.30.9110.9060.9090.9520.9540.9490.9540.9560.951
0.60.9060.9070.9080.9480.9440.9490.9570.9500.950
0.80.9100.9080.9070.9560.9400.9500.9520.9510.945
80.010.30.9260.9250.9260.9490.9480.9460.9460.9470.939
0.60.9270.9300.9300.9470.9370.9480.9500.9480.930
0.80.9300.9210.9290.9480.9230.9450.9390.9470.921
0.020.30.9300.9300.9250.9500.9480.9500.9520.9480.945
0.60.9270.9290.9280.9490.9310.9490.9510.9510.934
0.80.9260.9280.9270.9480.9220.9470.9430.9470.924
0.040.30.9250.9300.9270.9480.9430.9500.9510.9460.949
0.60.9280.9310.9260.9470.9330.9490.9470.9500.933
0.80.9290.9310.9290.9480.9210.9490.9450.9490.927
120.010.30.9340.9360.9360.9480.9460.9470.9500.9490.936
0.60.9320.9340.9360.9460.9220.9450.9480.9520.926
0.80.9340.9350.9400.9470.9130.9510.9400.9520.907
0.020.30.9340.9350.9400.9530.9460.9490.9440.9540.939
0.60.9330.9350.9340.9480.9280.9500.9460.9470.927
0.80.9380.9340.9340.9490.9130.9490.9470.9480.916
0.040.30.9330.9370.9360.9460.9430.9500.9480.9470.941
0.60.9350.9320.9360.9500.9310.9470.9450.9480.926
0.80.9360.9370.9380.9490.9120.9510.9400.9510.919
Mean0.9230.9230.9430.9490.9360.9490.9480.9490.932
Table 4:

Estimated coverage of 95 % confidence intervals (C.I.) for the intra- and inter-subject coefficients of variation γE and γA in a one-way random-effects model when the number of observations per subject is n=5. Between-subjects effects are logistic distributed, gamma distributed with shape parameter 16, and gamma distributed with shape parameter 4. Error effects are normally distributed. Coverage depends on the number of subjects, a, the intra-subject coefficient of variation, γE, and the intra-class correlation coefficient ρ.

Shoukri 0.95 C.I.Generalized 0.95 C.I.
LogisticΓ(16)Γ(4)LogisticΓ(16)Γ(4)
aγEργEγEγEγEγAγEγAγEγA
40.010.30.9400.9360.9340.9470.9400.9490.9500.9440.937
0.60.9420.9370.9390.9460.9320.9500.9480.9470.927
0.80.9400.9440.9410.9490.9260.9510.9410.9490.930
0.020.30.9400.9430.9380.9510.9400.9490.9460.9470.935
0.60.9460.9380.9410.9530.9310.9500.9410.9490.935
0.80.9340.9410.9370.9500.9260.9510.9420.9490.931
0.040.30.9420.9380.9390.9530.9370.9490.9490.9490.939
0.60.9380.9400.9400.9500.9300.9520.9470.9540.934
0.80.9380.9410.9430.9580.9310.9560.9460.9560.936
80.010.30.9450.9430.9430.9500.9290.9480.9450.9470.927
0.60.9450.9410.9430.9490.9180.9430.9410.9470.917
0.80.9450.9420.9420.9480.9150.9470.9380.9470.916
0.020.30.9430.9420.9480.9470.9310.9450.9420.9540.929
0.60.9450.9430.9450.9500.9210.9470.9370.9490.917
0.80.9470.9460.9420.9500.9150.9500.9400.9490.912
0.040.30.9450.9430.9450.9500.9300.9460.9430.9500.934
0.60.9400.9460.9460.9470.9180.9510.9450.9510.922
0.80.9440.9420.9440.9490.9110.9460.9460.9520.922
120.010.30.9450.9430.9450.9480.9320.9460.9410.9510.918
0.60.9440.9440.9490.9460.9090.9460.9380.9510.905
0.80.9510.9500.9510.9510.9030.9490.9360.9500.900
0.020.30.9440.9510.9480.9450.9240.9520.9450.9510.924
0.60.9450.9490.9490.9470.9130.9520.9390.9520.908
0.80.9490.9500.9440.9490.9040.9500.9410.9460.907
0.040.30.9450.9430.9490.9450.9270.9470.9420.9530.927
0.60.9500.9470.9490.9530.9160.9480.9430.9490.912
0.80.9410.9470.9500.9470.9100.9510.9430.9540.915
Mean0.9430.9430.9440.9490.9230.9490.9430.9500.923

Comparisons of Tables 3 and 4 with Tables 1 and 2, respectively, shows that generalized confidence intervals for γE were robust against non-normally distributed effects of subjects. Also the Shoukri et al. [15] 95 % confidence intervals for γE were robust. The generalized 95 % confidence intervals for γE still outperformed the Shoukri et al. [15] 95 % confidence intervals for γE.

Coverage of 95 % generalized confidence intervals for γA decreased as a consequence of non-normality. However, estimated coverage was never below 0.90 in these cases. Specifically, coverage of 95 % generalized confidence intervals for γA increased with γE and decreased with the intra-class correlation coefficient ρ. Moreover, coverage of 95 % generalized confidence intervals for γA decreased with the number of subjects. Cases with larger number of subjects (a=25, 50, 75) were also investigated through simulation. Tables A3 and A4, published online as supplementary material on the Journal’s web page, presents the results. Coverage did not continue to decrease dramatically with larger numbers of subjects, but stabilized at levels lower than 0.95.

In the semiparametric mixed-effects model for clustered data proposed by Tao et al. [29], between-subjects effects might not be normally distributed, but error effects are. Robustness was also investigated for the reverse situation, i. e., with normally distributed between-subject effects and non-normally distributed error effects. In the supplementary material, Tables A5 and A6 show the results of this investigation, which was carried out in the same way as the other simulation studies. Coverage of 95 % generalized confidence intervals for γA was close to 0.95, but coverage of 95 % generalized confidence intervals for γE become too small, often approximately 0.90. The Shoukri et al. [15] 95 % confidence intervals for γE performed in a similar way. Generally, the 95 % generalized confidence intervals for γE performed slightly better than the Shoukri et al. [15] 95 % confidence intervals.

3.2 The mixed-effects split-plot model

A balanced split-plot experiment with main-plot factor A and subplot factor B can be analyzed [31] using the model yijk=ϕ+κi+αj+βk+αβjk+aij+eijk, where yijk is the observation from the subplot of the ith replicate that was treated with the jth level of A and the kth level of B, ϕ is an intercept, κi is an effect of the ith replicate, αj is an effect of the jth level of A, βk is an effect of the kth level of B,αβjk is an interaction effect of the jth level of A with the kth level of B,aij is an effect of the jth main plot in the ith replicate, and eijk is a residual error, i= 1, 2, …, r; j= 1, 2, …, a; k= 1, 2, …, n. It will be assumed that yijk, aij and eijk are realized values of the random variables Yijk, Aij and Eijk, respectively, where AijN0,σA2, EijkN0,σE2, and Yijk=ϕ+κi+αj+βk+αβjk+Aij+Eijk.

A simulation study was performed based on the agricultural split-plot example of Section 4.2. Based on estimates obtained using the lme function of the R package nlme, (γA,γE) was set to (0.006, 0.062), ϕ was set to 77.044, ρ2,ρ3,ρ4 was set to (0.344, −3.075, −3.944), α2,α3,α4 was set to (−3.075, −8.050, −4.825), β2,β3,β4 was set to (−23.275, −3.150, −1.900, 2.325), and (αβ22,αβ32,αβ42,αβ23,αβ33,αβ43,αβ24,αβ34,αβ44) was set to (2.325, 6.400, 7.975, −1.150, 7.075, 3.950, −5.800, 4.925, 3.700). All other fixed-effects parameters were set to 0. These parameter setting was used for random generation of 10,000 balanced datasets. In addition, 10,000 unbalanced datasets were randomly generated. This was achieved by random selection of five observations that were removed before analysis. A new random selection of five missing values was made for each of the 10,000 datasets. In the Newton-Raphson procedure, the same starting value and stopping rules were used as for the simulation study of the one-way random-effects model (Section 3.1). For each generated dataset, generalized 95 % confidence intervals were computed using 1000 random samples of the pivotal quantities γˆE and γˆA.

For the balanced datasets, estimated coverage (i. e., the observed frequency of 95 % confidence intervals covering the true parameter value) was 0.943 and 0.949 for γE and γA, respectively. For unbalanced datasets, coverage was estimated to 0.950 and 0.942 for γE and γA, respectively.

4 Examples

4.1 A bioanalytical method validation using a one-way random-effects model

Following the European Medicines Agency’s (EMA) guideline on bioanalytical method validation [13], ligand-binding assays or immunoassays should be studied using quality control (QC) samples assayed in at least six runs over several days. Table 5 includes measurements of concentration of a QC sample in a precision study.

Table 5:

Measurements of a QC sample.

RunConcentration (mg/l)RunConcentration (mg/l)
11196109
1113*6103
21037103
291*7100
31188108
3106*8101
493998
493*994
510910105
5110*10106
  1. Removed for the unbalanced example.

Generalized 95 % confidence intervals were computed based on 10,000 random samples, each comprising one observation from each of the random variables UA, UE and Z. For each sample, the fiducial generalized pivotal quantities σˆE2, σˆA2, μˆ, γˆE and γˆA were computed using (4)–(6). The lower 2.5th and the upper 97.5th percentiles of the obtained empirical distributions of these fiducial generalized pivotal quantities are the lower and the upper limits of the generalized 95 % confidence intervals, respectively. The first row of Table 6 reports the obtained limits, i. e., the resulting generalized 95 % confidence intervals for σE2, σA2, μ, γE and γA. Figure 1 is an R script (www.r-project.org) for this analysis.

Figure 1: An R script for the balanced bioanalytical validation example.
Figure 1:

An R script for the balanced bioanalytical validation example.

Table 6:

Generalized 95 % confidence intervals. Each example used 10,000 random samples.

σE2σA2μγEγA
One-way
Balanced(10.6, 66.1)(7.70, 166)(98.8, 109)(0.031, 0.078)(0.027, 0.124)
Unbalanced(4.37, 65.9)(3.40, 222)(99.5, 112)(0.020, 0.078)(0.018, 0.142)
Split-plot
Balanced(11.3, 28.8)(−3.51, 10.4)(64.9, 67.3)(0.051, 0.081)(0.000, 0.049)
Unbalanced(8.53, 23.7)(−1.88, 17.2)(64.6, 67.8)(0.044, 0.074)(0.000, 0.063)

Five observations were removed (Table 5). In this case, the procedure described in Section 2.4 yielded generalized 95 % confidence intervals as presented on the second row of Table 6.

4.2 An agricultural split-plot experiment analyzed using a mixed-effects model

Federer and King [32] presented an example of a split-plot agricultural field experiment with r=4 replicates investigating effects of a=4 seedbed preparations and n=4 planting methods on maize yield (Table 7).

Table 7:

Observed yields (bushels per acre) of maizea [32].

Planting method
Replicate1234
Seedbed preparation 1
181.1b46.2*78.677.7
272.251.670.973.6
372.953.6*69.870.3
474.657.0*69.672.3
Seedbed preparation 2
174.149.172.066.1
276.253.871.865.5
371.143.767.666.2
467.8*58.860.660.6
Seedbed preparation 3
168.454.572.070.6
268.247.676.775.4
367.146.470.766.2
465.653.365.6*69.2
Seedbed preparation 4
171.550.976.475.1
270.465.075.875.8
372.554.967.675.2
467.850.265.663.3
  1. Copyright 2007 by John Wiley & Sons, Inc. All rights reserved.

  2. This number differs from the source, since it was noted that [32] actually used this number in their analysis.

  3. Removed for the unbalanced example.

The proposed method for computing generalized 95 % confidence intervals was applied. The fiducial generalized pivotal quantities σˆE2, σˆA2, μˆ, γˆE and γˆA, as specified in (4)–(6), were generated 10,000 times. The 2.5th percentiles of the generated distributions were used as lower generalized 95 % confidence limits, and the 97.5th percentiles were used as upper generalized 95 % confidence limits. The third line of Table 6 presents the obtained generalized 95 % confidence intervals.

For an example of an unbalanced dataset, five randomly selected observations were removed from the dataset (Table 7). Based on generation of 10,000 pivotal quantities γˆA and γˆE, the method for unbalanced datasets gave generalized 95 % confidence intervals as presented on the fourth row of Table 6.

5 Discussion

This article contributed the idea of using generalized inference methodology for computation of confidence intervals for intra- and inter-subject coefficients of variation in mixed-effects models. A method was developed for unbalanced datasets in mixed-effects models. Burdick et al. [26] studied other functions of variance components than the intra- and inter-subject coefficients of variation. Their methods for unbalanced datasets apply to random-effects models only.

Simulation studies indicated good performance in terms of coverage at the 95 % confidence level, although these studies also verified that the proposed methods are approximate. Since deviations from the nominal confidence level 0.95 were generally not large, the proposed methods can be recommended for practical purposes. However, the proposed methods are sensitive to the assumptions of normal distributions. The simulation studies revealed that the 95 % confidence interval proposed by Shoukri et al. [15] for the intra-subject coefficient of variation in the balanced one-way model tends to be too narrow in small datasets. For such datasets, the generalized 95 % confidence intervals proposed in the present article should be preferred.

The postulated linear mixed-effects model (1) is a fully parametric model, which assumes normal distributions. However in practice, random effects and errors are rarely exactly normally distributed. As noted in Section 2.2, model (1) can be viewed as a special case of the more realistic semiparametric mixed-effects model that was proposed by Tao et al. [29]. For this case, when the distribution of the random effects is unknown, model (1), which assumes the normal distribution, was proposed as a working model. This approach is not unproblematic, since the performance of the method depends on how well the assumption of normality is met. In practice, the proposed generalized confidence intervals may, in the non-normal case, have coverage somewhat other than the nominal. Still, the simulation studies indicate that the proposed generalized confidence intervals for γE give better coverage than the Shoukri et al. [15] confidence intervals for γE, also when the assumption of normality is not fulfilled. Furthermore, the present article introduced a method for computing confidence intervals not only for the intra-subject coefficient of variation γE, but also for the inter-subject coefficient of variation γA.

The generalization to more than two variance components is straightforward for balanced datasets. Every variance component has a unique unbiased estimator based on sums of squares [33]. Thus, a generalized pivotal quantity can be specified for any function of the parameters of a balanced mixed-effects model [24]. The generalization to mixed-effects models with several variance components is more difficult for unbalanced datasets. Although extensions have been made to the special case of unequal cell frequencies in the last stage [27, 33, 34, 35], unbalanced cases are generally less explored in the literature on generalized inference. It is possible that the method presented here can be generalized to more complex mixed-effects models, but since the proposed method involves numerical computations, it might be a challenge to provide an efficient general algorithm.

As pointed out by Hannig et al. [25], when h is a fiducial generalized pivotal quantity and θ is a scalar, then Ty,Y,ξ=θhy,Y,ξ is a generalized test variable as defined by Tsui and Weerahandi [36] for testing the null hypothesis H0:θθ0 against the alternative H1:θ>θ0. The methodology of the present article can for this reason also be used for hypothesis testing.

Acknowledgement

This research was conducted using the resources of High Performance Computing Center North (HPC2N).

Appendix

INDDTDDTV/σE22=INDDTDDTCCTρ+ININDDTDDTCCTρ+IN=INDDTDDTCCTρCCTDDTDDTρ+INDDTDDTCCTρ+IN=INDDTDDTINDDTDDTCCTρ+IN=INDDTDDTV/σE2

where ρ=σA2/σE2.

PV2=V1INBBTV1BBTV1INBBTV1BBTV1V=V1IN2BBTV1BBTV1+BBTV1BBTV1BBTV1BBTV1V=V1IN2BBTV1BBTV1+BBTV1BBTV1V=PV

References

1. Owen DB. A survey of properties and applications of the noncentral t-distribution. Technometrics 1968;10:445–478.10.2307/1267101Search in Google Scholar

2. Miller EG, Feltz CJ. Asymptotic inference for coefficients of variation. Commun Stat Theory Methods 1997;26:715–726.10.1080/03610929708831944Search in Google Scholar

3. Mahmoudvand R, Hassani H. Two new confidence intervals for the coefficient of variation in a normal distribution. J Appl Stat 2009;36:429–442.10.1080/02664760802474249Search in Google Scholar

4. Vangel MG. Confidence intervals for a normal coefficient of variation. Am Stat 1996;50:21–26.Search in Google Scholar

5. Forkman J. Estimator and tests for common coefficients of variation in normal distributions. Commun Stat Theory Methods 2009;38:233–251.10.1080/03610920802187448Search in Google Scholar

6. McKay AT. Distribution of the coefficient of variation and the extended “t” distribution. J R Stat Soc 1932;95:695–698.10.2307/2342041Search in Google Scholar

7. Forkman J, Verrill S. The distribution of McKay’s approximation for the coefficient of variation. Stat Probab Lett 2008;78:10–14.10.1016/j.spl.2007.04.018Search in Google Scholar

8. Verrill S, Johnson RA. Confidence bounds and hypothesis tests for normal distribution coefficients of variation. Commun Stat Theory Methods 2007;36:2187–2206.10.1080/03610920701215126Search in Google Scholar

9. Hayter AJ. Confidence bounds on the coefficient of variation of a normal distribution with applications to win-probabilities. J Stat Comput Simul 2015;85:3778–3791.10.1080/00949655.2015.1035654Search in Google Scholar

10. Wong AC, Wu J. Small sample asymptotic inference for the coefficient of variation: normal and nonnormal models. J Stat Plan Inference 2002;104:73–82.10.1016/S0378-3758(01)00241-5Search in Google Scholar

11. Searle SR, Casella G, McCulloch CE. Variance components. Hoboken: Wiley, 2006.Search in Google Scholar

12. Quan H, Shih WJ. Assessing reproducibility by the within-subject coefficient of variation with random effects models. Biometrics 1996;52:1195–1203.10.2307/2532835Search in Google Scholar PubMed

13. European Medicines Agency. Guideline on bioanalytical method validation. London: The European Medicines Agency, 2011. Available at: http://www.ema.europa.eu/ema/.Search in Google Scholar

14. Laidig F, Drobek T, Meyer U. Genotypic and environmental variability of yield for cultivars from 30 different crops in German official variety trials. Plant Breed 2008;127:541–547.10.1111/j.1439-0523.2008.01564.xSearch in Google Scholar

15. Shoukri MM, Elkum N, Walter SD. Interval estimation and optimal design for the within-subject coefficient of variation for continuous and binary variables. BMC Med Res Method 2006;6:1–10.10.1186/1471-2288-6-24Search in Google Scholar PubMed PubMed Central

16. Weerahandi S. Generalized confidence intervals. J Am Stat Assoc 1993;88:899–905.10.1080/01621459.1993.10476355Search in Google Scholar

17. Tian LL. Inferences on the common coefficient of variation. Stat Med 2005;24:2213–2220.10.1002/sim.2088Search in Google Scholar PubMed

18. Behboodian J, Jafari AA. Generalized confidence interval for the common coefficient of variation. J Stat Theory Appl 2008;7:349–363.Search in Google Scholar

19. Jafari AA. Inferences on the coefficients of variation in a multivariate normal population. Commun Stat Theory Methods 2015;44:2630–2643.10.1080/03610926.2013.788711Search in Google Scholar

20. Krishnamoorthy K, Lee M. Improved tests for the equality of normal coefficients of variation. Comput Stat 2014;29:215–232.10.1007/s00180-013-0445-2Search in Google Scholar

21. Burdick RK, Quiroz J, Iyer HK. The present status of confidence interval estimation for one-factor random models. J Stat Plan Inference 2006;136:4307–4325.10.1016/j.jspi.2005.07.004Search in Google Scholar

22. Chiang AK. A simple general method for constructing confidence intervals for functions of variance components. Technometrics 2001;43:356–367.10.1198/004017001316975943Search in Google Scholar

23. Iyer H, Mathew T. Comments on Chiang (2001). Technometrics 2002;44:284–285.10.1198/004017002188618473Search in Google Scholar

24. Iyer HK, Patterson PL. A receipe for constructing generalized pivotal quantities and generalized confidence intervals. Colorado State University, Dept. of Statistics, 2002. Report No.: 2010/10.Search in Google Scholar

25. Hannig J, Iyer H, Patterson P. Fiducial generalized confidence intervals. J Am Stat Assoc 2006;101:254–269.10.1198/016214505000000736Search in Google Scholar

26. Burdick RK, Borror CM, Montgomery DC. Design and analysis of gauge R&R studies : making decisions with confidence intervals in random and mixed ANOVA models. Philadelphia, PA: Society for Industrial and Applied Mathematics, 2005.10.1137/1.9780898718379Search in Google Scholar

27. Gamage J, Mathew T, Weerahandi S. Generalized prediction intervals for BLUPs in mixed models. J Multivar Anal 2013;120:226–233.10.1016/j.jmva.2013.05.011Search in Google Scholar

28. Shao J. Mathematical statistics, 2nd ed. New York: Springer, 2003.10.1007/b97553Search in Google Scholar

29. Tao NG, Palta M, Yandell BS, Newton MA. An estimation method for the semiparametric mixed effects model. Biometrics 1999;55:102–110.10.1111/j.0006-341X.1999.00102.xSearch in Google Scholar PubMed

30. McCulloch CE, Searle SR, Neuhaus JM. Generalized, linear, and mixed models [Internet], 2nd ed. Hoboken, NJ: Wiley, 2008. Available at: http://www.loc.gov/catdir/enhancements/fy0827/2008002724-d.html. Cited: 19 Aug 2015.Search in Google Scholar

31. Piepho HP, Büchse A, Emrich K. A hitchhiker’s guide to mixed models for randomized experiments. J Agron Crop Sci 2003;189:310–322.10.1046/j.1439-037X.2003.00049.xSearch in Google Scholar

32. Federer WT, King F. Variations on split plot and split block experiment designs. Hoboken: Wiley, 2007.10.1002/0470108584Search in Google Scholar

33. Khuri AI, Mathew T, Sinha BK. Statistical tests for mixed linear models. New York: Wiley, 1998.10.1002/9781118164860Search in Google Scholar

34. Zhou L, Mathew T. Some tests for variance components using generalized p values. Technometrics 1994;36:394–402.10.2307/1269954Search in Google Scholar

35. Li XM, Wang J. A simple method for testing variance components in unbalanced nested model. Commun Stat Simul Comput 2011;40:1310–1323.10.1080/03610918.2011.569861Search in Google Scholar

36. Tsui KW, Weerahandi S. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J Am Stat Assoc 1989;84:602–607.10.2307/2289949Search in Google Scholar


Supplemental Material

The online version of this article (DOI: https://doi.org/10.1515/ijb-2016-0093) offers supplementary material, available to authorized users.


Published Online: 2017-6-15

© 2017 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 29.3.2024 from https://www.degruyter.com/document/doi/10.1515/ijb-2016-0093/html
Scroll to top button