Show Summary Details
More options …

# The International Journal of Biostatistics

Ed. by Chambaz, Antoine / Hubbard, Alan E. / van der Laan, Mark J.

IMPACT FACTOR 2018: 1.309

CiteScore 2018: 1.11

SCImago Journal Rank (SJR) 2018: 1.325
Source Normalized Impact per Paper (SNIP) 2018: 0.715

Mathematical Citation Quotient (MCQ) 2018: 0.03

Online
ISSN
1557-4679
See all formats and pricing
More options …
Volume 12, Issue 2

# Sample Size for Assessing Agreement between Two Methods of Measurement by Bland−Altman Method

Meng-Jie Lu
• Department of Medical Statistics, Jinling Hospital, Southern Medical University, 305 East Zhongshan Road, Nanjing 210002, China
• Email
• Other articles by this author:
/ Wei-Hua Zhong
• Department of Medical Statistics, Jinling Hospital, Southern Medical University, 305 East Zhongshan Road, Nanjing 210002, China
• Email
• Other articles by this author:
/ Yu-Xiu Liu
• Corresponding author
• Department of Medical Statistics, Jinling Hospital, Southern Medical University, 305 East Zhongshan Road, Nanjing 210002, China
• Email
• Other articles by this author:
/ Hua-Zhang Miao
• Department of Medical Statistics, Jinling Hospital, Southern Medical University, 305 East Zhongshan Road, Nanjing 210002, China
• Email
• Other articles by this author:
/ Yong-Chang Li
• Department of Medical Statistics, Jinling Hospital, Southern Medical University, 305 East Zhongshan Road, Nanjing 210002, China
• Email
• Other articles by this author:
/ Mu-Huo Ji
Published Online: 2016-11-12 | DOI: https://doi.org/10.1515/ijb-2015-0039

## Abstract:

The Bland–Altman method has been widely used for assessing agreement between two methods of measurement. However, it remains unsolved about sample size estimation. We propose a new method of sample size estimation for Bland–Altman agreement assessment. According to the Bland–Altman method, the conclusion on agreement is made based on the width of the confidence interval for LOAs (limits of agreement) in comparison to predefined clinical agreement limit. Under the theory of statistical inference, the formulae of sample size estimation are derived, which depended on the pre-determined level of α, β, the mean and the standard deviation of differences between two measurements, and the predefined limits. With this new method, the sample sizes are calculated under different parameter settings which occur frequently in method comparison studies, and Monte-Carlo simulation is used to obtain the corresponding powers. The results of Monte-Carlo simulation showed that the achieved powers could coincide with the pre-determined level of powers, thus validating the correctness of the method. The method of sample size estimation can be applied in the Bland–Altman method to assess agreement between two methods of measurement.

## 1 Introduction

The original article by Bland and Altman [1] which proposed the method of agreement analysis has received more than 30,000 citations in the biomedical literature and has increased in usage in recent years. Nature asked Thomson Reuters, which now owns the SCI, to list the 100 most highly cited papers of all time. The third most frequently cited statistics paper (number 29) is a 1986 publication by British statisticians Martin Bland and Douglas Altman who proposed a technique – now known as the Bland–Altman plot [2]. It has been stressed that estimates of limits of agreement (LOAs) should be accompanied with confidence intervals. However, they found that “confidence intervals are seldom quoted” in reports of method comparison studies [35]. A reporting standards for Bland–Altman agreement analysis in laboratory research found that the CI limits of LOAs were reported in only 6 % of 50 studies published later than 2012 [6]. Many researchers forget that Bland and Altman presented the limits of agreement as a reference interval only. The LOAs do not guarantee coverage on the range of potential differences between the two measurements and cannot be used directly for statistical inference. Just as Hamilton and Stamey pointed out, the Bland–Altman limits of agreement by themselves provide only a reference interval and should never be used as the determining factor to conclude agreement between two methods [7]. They informed that future researchers should take this variability into account and always provide confidence intervals when using the LOAs approach. Like any other clinical trials, it is essential to determine the sample size for method comparison studies [8]. Although Bland has given the sample size for a study of agreement between two methods of measurement which were available from his website [9], the sample size was determined without considering the power of the statistical procedure and could not guarantee the power of test. Sample size calculations were performed in only 30 % of publications reviewed [6]. Bland and Altman hoped to find time to publish some of these, for example, on sample size estimation for measurement method agreement studies, but up to now it is still not solved satisfactorily [3].

In this paper we propose a method to calculate sample size for Bland–Altman method, and Monte-Carlo simulations are used to validate the correctness of the method. In Section 2, we introduce the assumptions and theory of Bland–Altman method. Section 3 focuses on the derivation of the new method of sample size estimation for Bland–Altman method. In Section 4, Monte-Carlo simulation is used to obtain the corresponding powers. The results of Monte-Carlo simulation showed that the achieved powers could coincide with the pre-determined level of powers, thus validating the correctness of the method. In Section 5, we show a clinical worked example from a set of measured data of free prostate specific antigen (FPSA), which is often used to evaluate the presence of prostate cancer and other prostate disorders. We give concluding remarks in Section 6.

## 2.1 Assumptions

Suppose that the measurements of two methods are made on each of n subjects drawn from some population of interest. Suppose further that the two measurements, ${x}_{i}$ and ${y}_{i}$ respectively and the difference, ${d}_{i}$ for subject i (i=1,2, …, n)

${d}_{i}={x}_{i}-{y}_{i}$

The important first step of the Bland–Altman method is to plot the data and to check its pattern and distribution. The differences for the two methods are plotted against their means and if the data are well-behaved, then construction of the various limits and interpretation of the data is simple and straightforward. The assumptions of the limits of the agreement method are that the differences values resulting from two measurements should have an approximately normal distribution, constant variance of the differences, and no proportional bias [10]. Proportional bias is present when the differences increase or decrease in proportion to the average values [11].

## 2 LOAs and confidence interval estimation

Suppose difference variable $D$ is a random variable which follows a normal distribution with mean μ and variance σ2. It is well-known that 100(1−γ) % of the population lies between $\mathrm{\mu }±{z}_{1-\mathrm{\gamma }/2}\mathrm{\sigma }$. In practice, both μ and σ are unknown and have to be estimated. We take $\stackrel{ˉ}{D}$ and ${S}_{D}^{2}$ as estimators of μ and σ2 respectively.

The $100\left(1-\mathrm{\gamma }\right)\mathrm{%}$ LOAs can be calculated as

$\stackrel{ˉ}{D}±{z}_{1-\mathrm{\gamma }/2}{S}_{D}$

where ${z}_{1-\mathrm{\gamma }/2}$ is the cumulative 100(1−γ/2)% percentile of a standard normal distribution, ${S}_{D}$ is the standard deviation of the differences, $\stackrel{ˉ}{D}+{z}_{1-\mathrm{\gamma }/2}{S}_{D}$ is the upper limit value of LOA, and $\stackrel{ˉ}{D}-{z}_{1-\mathrm{\gamma }/2}{S}_{D}$ is the lower limit value. 95 % LOAs are the most common, which are mean minus 1.96 standard deviations and mean plus 1.96 standard deviations respectively. These limits are expected to contain 95 % of paired differences between measurements by the two methods.

It is important to consider confidence interval of LOAs [12]. The $100\left(1-\mathrm{\alpha }\right)\mathrm{%}$ confidence interval estimation of $100\left(1-\mathrm{\gamma }\right)\mathrm{%}$ LOAs derived by Bland and Altman can be calculated as [1]

$Lower=\stackrel{ˉ}{D}-{z}_{1-\mathrm{\gamma }/2}{S}_{D}-{t}_{1-\mathrm{\alpha }/2,n-1}{S}_{D}\sqrt{\frac{1}{n}+\frac{{z}_{1-\mathrm{\gamma }/2}^{2}}{2\left(n-1\right)}}$(1)

$Upper=\stackrel{ˉ}{D}+{z}_{1-\mathrm{\gamma }/2}{S}_{D}+{t}_{1-\mathrm{\alpha }/2,n-1}{S}_{D}\sqrt{\frac{1}{n}+\frac{{z}_{1-\mathrm{\gamma }/2}^{2}}{2\left(n-1\right)}}$(2)

We can obtain the upper confidence limit of upper limit of the LOAs and the lower confidence limit of lower limit of the LOAs, where n is the sample size. Generally, we set $\mathrm{\gamma }$ and $\mathrm{\alpha }$ as 0.05. If the 95 % confidence interval for the 95 % LOAs comes within the pre-defined agreement limits which are clinically acceptable, the two methods agree sufficiently to fulfil the agreement requirements. In fact, the correspondence between confidence intervals for LOAs and hypotheses tests here is identical. Providing A is the lower limit and B is the upper limit of LOAs of population differences, we can construct the following simultaneous hypotheses: H01 is A < –δ, H11 is A ≥ –δ and H02 is B > δ, H12 is B ≤ δ. When the two null hypotheses are rejected simultaneously, the two measurements would be inferred to agree. The hypotheses of Bland–Altman method are quite similar to the equivalence [13].

## 3 Sample size formulae

Considering the confidence interval estimation of LOAs has symmetry of μ and –μ (μ ≥ 0) and the sample size estimations of these two situations should be the same, we just discuss the situation when μ ≥ 0. According to the statistical inference principle of Bland–Altman limits of agreement, we can separate total type I error $\left(\mathrm{\alpha }\right)$ into two parts which are both $\mathrm{\alpha }/2$. Similarly, we can separate total type II error $\left(\mathrm{\beta }\right)$ into two parts. One is the first type II error $\left({\mathrm{\beta }}_{1}\right)$ of the upper limit value of LOAs and the other is the second type II error $\left({\mathrm{\beta }}_{2}\right)$ of the lower limit value of LOAs (Figure 1.) [13].

Figure 1: Decomposition diagram of type I error (α) and type II error (β) for Bland and Altman method. Total type I error (α) consisted of two parts which are both α/2. Similarly, total type II error (β) consisted of two parts. One is the first type II error (β1) of the upper limit value of LOAs and the other is the second type II error (β2) of the lower limit value of LOAs.

We can get a direct estimate of ${\mathrm{\beta }}_{1}$ and ${\mathrm{\beta }}_{2}$:

$\begin{array}{rl}& \begin{array}{ccc}\stackrel{‾}{D}& +& {z}_{{}_{1-\mathrm{\gamma }/2}}{S}_{D}-{z}_{{\mathrm{\beta }}_{1}}s{e}_{LOAs}=\mathrm{\delta }+{z}_{\mathrm{\alpha }/2}s{e}_{LOAs}\\ \stackrel{‾}{D}& -& {z}_{{}_{1-\mathrm{\gamma }/2}}{S}_{D}+{z}_{{\mathrm{\beta }}_{2}}s{e}_{LOAs}=-\mathrm{\delta }-{z}_{\mathrm{\alpha }/2}s{e}_{LOAs}\\ {\mathrm{\beta }}_{1}& =& \mathrm{\Phi }\left(\frac{\stackrel{‾}{D}+{z}_{1-\mathrm{\gamma }/2}{S}_{D}-\mathrm{\delta }}{s{e}_{LOAs}}-{z}_{\mathrm{\alpha }/2}\right)\\ {\mathrm{\beta }}_{2}& =& \mathrm{\Phi }\left(\frac{-\stackrel{‾}{D}+{z}_{1-\mathrm{\gamma }/2}{S}_{D}-\mathrm{\delta }}{s{e}_{LOAs}}-{z}_{\mathrm{\alpha }/2}\right)\end{array}\\ & \end{array}$

where $\mathrm{\Phi }\left(\bullet \right)$ is defined as the cumulative density function of standard normal distribution $N\left(0,1\right)$, $s{e}_{LOA\mathrm{s}}$ is the standard error of the lower limit or upper limit of LOAs, $s{e}_{LOAs}={S}_{D}\sqrt{\frac{1}{n}+\frac{{z}_{1-\mathrm{\gamma }/2}^{2}}{2\left(n-1\right)}}$, $\mathrm{\delta }$ is the maximum allowable difference that can be accepted clinically, it needs to be defined in advance.

According to the statistical distribution theory, it is best to calculate the type II error $\left(\mathrm{\beta }\right)$ under the assumption of a non-central t-distribution [14], that is:

${\mathrm{\beta }}_{1}=1-probt\left[{t}_{1-\mathrm{\alpha }/2,n-1},n-1,{\mathrm{\tau }}_{1}\right]$(3)

${\mathrm{\beta }}_{2}=1-probt\left[{t}_{1-\mathrm{\alpha }/2,n-1},n-1,{\mathrm{\tau }}_{2}\right]$(4)

where $probt\left[\bullet ,n-1,{\mathrm{\tau }}_{1}\right]$ denotes the cumulative distribution function of a Student’s non-central t-distribution with $n-1$ degrees of freedom and non-centrality parameter ${\mathrm{\tau }}_{1}$.

The non-centrality parameters ${\mathrm{\tau }}_{1}$ and ${\mathrm{\tau }}_{2}$ are non-central parameters defined as

${\mathrm{\tau }}_{1}=\frac{\mathrm{\delta }-\stackrel{ˉ}{D}-{z}_{1-\mathrm{\gamma }/2}{S}_{D}}{{S}_{D}\sqrt{\frac{1}{n}+\frac{{z}_{1-\mathrm{\gamma }/2}^{2}}{2\left(n-1\right)}}}.$

${\mathrm{\tau }}_{2}=\frac{\mathrm{\delta }+\stackrel{ˉ}{D}-{z}_{1-\mathrm{\gamma }/2}{S}_{D}}{{S}_{D}\sqrt{\frac{1}{n}+\frac{{z}_{1-\mathrm{\gamma }/2}^{2}}{2\left(n-1\right)}}}.$

We can get an estimate of the power:

$\begin{array}{ccc}power& =& 1-\mathrm{\beta }=1-\left({\mathrm{\beta }}_{1}+{\mathrm{\beta }}_{2}\right)\\ & =& probt\left({t}_{1-\mathrm{\alpha }/2,n-1},n-1,{\mathrm{\tau }}_{1}\right)+probt\left({t}_{1-\mathrm{\alpha }/2,n-1},n-1,{\mathrm{\tau }}_{2}\right)\end{array}$(5)

When $\mathrm{\mu }=0$, the sample size calculation can be written as follows:

$n=\frac{\left(2+{z}_{1-\mathrm{\gamma }/2}^{2}\right){\left[tinv\left(1-\mathrm{\beta }/2,n-1,{t}_{1-\mathrm{\alpha }/2,n-1}\right)\right]}^{2}{S}_{D}^{2}}{2{\left({z}_{1-\mathrm{\gamma }/2}{S}_{D}-\mathrm{\delta }\right)}^{2}}$(6)

where $tinv\left(1-\mathrm{\beta }/2,n-1,{t}_{1-\mathrm{\alpha }/2,n-1}\right)$ is defined as the inverse of a Student’s non-central t-distribution.

In eq. (6), $tinv\left(1-\mathrm{\beta }/2,n-1,{t}_{1-\mathrm{\alpha }/2,n-1}\right)$ is related to sample size $\left(n\right)$, we need to use iterative method to calculate sample size. Firstly we replace non-central t-distribution quantile with standard normal distribution quantile to obtain the initial value $\left({n}_{0}\right)$, and then iterate step-by-step until $n$ reaches a stable value.

When $\mathrm{\mu }>0$, we firstly calculate by eq. (6) to obtain an initial value $\left({n}_{1}\right)$, then calculate by eq. (5) to achieve the power. If the estimated power is close enough to the pre-specified power then $n$ is the sample size that we want to estimate. Otherwise we make ${n}_{1}$ equal to ${n}_{1}+1$ and judge whether the estimated power is close enough the pre-specified power. Repeat the procedure above until be closest to but greater than the pre-specified power. Table 1 summaries reasonable estimates of the sample size using eqs (5) and (6) for various standardized difference limits $\left(\mathrm{\mu }/\mathrm{\sigma }\right)$, different standardized agreement limits $\left(\mathrm{\delta }/\mathrm{\sigma }\right)$, and different type II error $\left(\mathrm{\beta }\right)$ assuming that the data are well-behaved.

Table 1 can be a reference for clinical researchers to estimate the sample size in the agreement assessment trial between two methods of measurement.

Table 1:

Sample size $\left(n\right)$ for Bland–Altman method with non-central t-distribution for different standardized difference limits $\left(\mathrm{\mu }/\mathrm{\sigma }\right)$, different standardized agreement limits $\left(\mathrm{\delta }/\mathrm{\sigma }\right)$, and different type II error $\left(\mathrm{\beta }\right)$. $\left(\mathrm{\alpha }=0.05\right)$.

## 4 Simulation

Monte-Carlo simulation studies with 10,000 replicates were performed to examine the validity and correctness of the proposed formulae for estimating sample size by calculating empirical powers. Simulation data were generated on the basis of normal distribution by considering typical situations under different parameter settings. If the achieved power is very close to the pre-specified power, then it could be proved that our formulae can estimate the reasonable sample size.

As claimed above, simulating corresponding powers is easy: Firstly, we define the mean of differences $\left(\mathrm{\mu }\right)$, the standard deviation of differences $\left(\mathrm{\sigma }\right)$ and the pre-defined clinical agreement limit $\left(\mathrm{\delta }\right)$. And then we estimate the sample size with eqs (5) and (6), and calculate the 95 % confidence interval for the 95 % LOAs. If they lie within the pre-defined clinical agreement limits $\left(-\mathrm{\delta },\mathrm{\delta }\right)$, then we draw a conclusion that the two methods agree, otherwise disagree. We repeat the procedure above 10,000 times and compute the times $\left(t\right)$ that draw agreement conclusion. The value of $\frac{t}{10000}×100\mathrm{%}$ is the achieved power. Table 2 presents the achieved power for different parameter settings corresponding to the Table 1.

Table 2 indicates that the achieved powers are generally close to the pre-specified power of 80 % or 90 %. It shows that the formulae give reasonable estimates of the sample size using eqs (5) and (6) for various parameter settings.

Table 2:

Power for Bland–Altman methods with non-central t-distribution for different standardized different limits $\left(\mathrm{\mu }/\mathrm{\sigma }\right)$, different standardized agreement limits $\left(\mathrm{\delta }/\mathrm{\sigma }\right)$, and different type II error $\left(\mathrm{\beta }\right)$. $\left(\mathrm{\alpha }=0.05\right)$.

Bland has given the sample size for a study of agreement between two methods of measurement which were available from his website [9]. In the 1986 Lancet paper they gave a formula for the confidence interval for the 95 % limits of agreement. The standard error of the 95 % limit of agreement is approximately root $\left(\sqrt{3{s}^{2}/n}\right)$, where s is the standard deviation of the differences between measurements by the two methods and $n$ is the sample size. The confidence interval is the estimate of the limit, $\stackrel{ˉ}{d}$ plus or minus $1.96s$, plus or minus 1.96 standard errors, then the sample size can be worked out.

We set $\mathrm{\alpha }=0.05$, $\mathrm{\beta }=0.20$, $\mathrm{\mu }=-0.4\text{\hspace{0.17em}}0.4$, ${\mathrm{\sigma }}^{2}=1$, $\mathrm{\delta }=2.7$, and pre-specified power = 80 %. Figure 2 shows the sample sizes and powers of B-A method and new method under different parameter settings. With the Bland–Altman method, the sample size is calculated without considering the power of the statistical procedure, and so the probability of obtaining the required width is only 0.50. With the new method, the achieved power is generally close to the pre-specified power of 80 %.

Figure 2: Comparisons of sample size and power (%) between Bland–Altman method and New method under different parameter settings. The parameter settings are α = 0.05, β = 0.20, μ = −0.4 ~ 0.4, σ = 1, δ = 2.7.

The number of subjects required in the Bland–Altman method proposed by Bland is determined on the basis of the expected width of a confidence interval. It fails to explicitly consider the probability of achieving the desired interval width and may thus provide sample sizes that are too small to have enough power. However, the new method is more appropriate, because it can ensure an adequate probability of achieving the desired precision.

## 5 Example

We show a clinical worked example from a set of measured data of free prostate specific antigen (FPSA), which is often used to evaluate the presence of prostate cancer and other prostate disorders. AIA-1800 and I2000 methods were used to measure the FPSA. In the process of measurement, a same random sequence of sample was used in the two instruments [15]. Through a pre-experiment we get the mean and standard deviation of differences between AIA-1800 and I2000 methods are 0.001167 mmol/l and 0.001129 mmol/l respectively. Defining $\mathrm{\alpha }=0.05$, $\mathrm{\beta }=0.20$, $\left(-\mathrm{\delta },\mathrm{\delta }\right)=\left(-0.004,0.004\right)\mathrm{m}\mathrm{m}\mathrm{o}\mathrm{l}/\mathrm{l}$, we can calculate that a sample size of 83 would be needed to provide 80 % power to assess agreement between two methods of measurement. Monte-Carlo simulation is used to obtain the corresponding power which is equal to 80.51 %, closely to the pre-defined power (80 %).

## 6 Conclusion

Based on the statistical inference principle and mathematical distribution theory, we have derived the calculating formula of sample size for Bland–Altman method under different parameter settings. For the sake of convenience, we have given a set of table which can be easily find out the sample size for different standardized difference limits $\left(\mathrm{\mu }/\mathrm{\sigma }\right)$, standardized agreement limits $\left(\mathrm{\delta }/\mathrm{\sigma }\right)$, and type error $\left(\mathrm{\beta }\right)$ under two-sided $\mathrm{\alpha }=0.05$. Both α and β should be considered to have sample size large enough to ensure that the half width of a 100(1−α)% confidence interval is no larger than a pre-specified width with a pre-specified assurance probability 100(1−β)%. We carried out Monte-Caro simulation studies to validate the correctness of the proposed method. The simulation results reveal that the achieved powers could coincide with the pre-determined level of powers, thus validating the correctness of the formulae.

It is important to be aware of the pre-specified clinically acceptable agreement limits. As with equivalence or non-inferiority clinical trials, the clinical agreement limits need to be determined in advance by clinical researchers and biostatistician. Defining these agreement limits may be a difficult aspect in designing the measurement comparison studies, because they depend upon not only the clinical scenario but also other variables. Nevertheless, an attempt must be made to define them; a Delphi survey (opinion from experts) may be used to design the study. This survey is a group facilitation technique, which is an iterative multistage process designed to transform an opinion into group consensus [16].

Lin et al. [17] had discussed some issues about sample size using the tolerance interval; however, there are some deficiencies. First of all, in Lin’s study, the hypothesis of the sample size calculating method is provided just under $\mathrm{\mu }=0$, which is not considered for $\mathrm{\mu }\ne 0$. In fact, the two measurements may not be perfectly consistent $\left(\mathrm{\mu }\ne 0\right)$, but we still believe their consistency as the population difference within a certain acceptable range (δ). In Lin’s paper, it can be seen that the simulated power is consistently less than the pre-specified power for all design specifications, so the sample sizes are much under-estimated.

Hahn and Meeker [18] defined a tolerance interval that is an interval that one can claim to contain at least a specified proportion, p, of the population with a specified degree of confidence, 100(1–α)% and provided the sample size estimation about the tolerance interval. We can see that their sample size estimation is based on the desired precision without considering type II error (β) or power, and just addresses the frequently asked question “How large a sample do I need to obtain a confidence interval?” Although our confidence interval of LOAs is similar to their tolerance interval, the theories and the procedures of sample size estimation are totally different. Our method of sample size estimation is derived not only on the pre-determined level of α but also on the β.

Recently, studies of the agreement between two instruments or clinical tests have proliferated in ophthalmic literature. McAlinden et al. used a method of sample size calculation for agreement studies on the basis of method proposed by Bland [19]. The sample size was calculated without considering the power of the statistical procedure, so the probability of obtaining the required width was only 0.50 [20]. During the study design stage, considering the power in sample size calculations could lead to expected conclusions under the predetermined power level. Cesana et al. provided another sample size estimation required for demonstrating a Pearson correlation coefficient between the differences and the means of the measures [20], and we think this method is unreasonable. Actually, the correlation coefficient given by Cesana reflected the proportional bias. As we know, one of the assumptions about the application of the Bland–Altman method is no proportional bias. Without fulfillment of the assumption, this method would not be applicable.

There are some limitations to this study. Our sample size formulae are just appropriate for the data which are well-behaved. If the data behavior is not very well, such as non-normality or non-constant variance of the differences (heteroscedasticity) and proportional bias, the formulae are not suitable to solve the problem of estimating sample size.

## Acknowledgments

We are grateful for the constructive comments of Dr. Jian-Jun Yang. We also thank the editors and the anonymous reviewers for valuable comments that have helped us significantly improved our manuscript.

## Funding

This study was funded by a grant from the National Natural Science Foundation of China (No.81473066).

## References

• 1. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;i:307–310. Google Scholar

• 2. Noorden RV, Mahen B, Nuzzo R. The top 100 papers. Nature 2014;514:550–553.

• 3. Bland JM, Altman DG. Agreed statistics measurement method comparison. Anesthesiology 2012;116:182–185.

• 4. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999;8:135–160.

• 5. Bland JM, Altman DG. Appling the right statistics: analyses of measurement studies. Ultrasound Obst Gyn 2003;22:85–93.

• 6. Chhapola V, Kanwal SK, Brar R. Reporting standards for Bland–Altman agreement analysis in laboratory research: a cross-sectional survey of current practice. Ann Clin Biochem 2015;52:382–386.

• 7. Hamilton C, Stamey J. Using Bland−Altman to assess agreement between two medical devices-don’t forget the confidence intervals! J Clin Monit Comput 2007;21:331–333.

• 8. Bella ML, Teixeira-Pintoc A, McKenzied JE, Oliviere J. A myriad of methods: calculated sample size for two proportions was dependent on the choice of sample size formula and software. J Clin Epidemiol 2014;67:601–605.

• 9. Bland JM. How can I decide the sample size for a study of agreement between two methods of measurement? Available at: http://www-users.york.ac.uk/~mb55/meas/sizemeth.htmAccessed: 15 Aug 2015.

• 10. Woodman RJ. Bland−Altman beyond the basics: creating confidence with badly behaved data. Clin Exp Pharmacol Physiol 2010;37:141–142.

• 11. Ludbrook J. Confidence in Altman-Bland plots: a critical review of the method of differences. Clin Exp Pharmacol Physiol 2010;37:143–149.

• 12. Stockl D, Cabaleiro DR, Uytfanghe KV, Thienpont LM. Interpreting method comparison studies by use of the Bland−Altman plot: reflecting the importance of sample size by incorporating confidence limits and predefined error limits in the graphic. Clin Chem 2004;50:2216–2218.

• 13. Julious SA. Sample size for clinical trials with Normal data. Stat Med 2004;23:1921–1986.

• 14. Forbes C, Evans M, Hastings N, Peacock B. Statistical distributions, 4th ed . Hoboken: John Wiley & Sons, 2011:187–188. Google Scholar

• 15. Zhou YH, Zang JJ, Wu MJ, Xu JF, He J. Allowable total error and limits for erroneous results (ATE/LER) zones for agreement measurement. J Clin Lab Anal 2011;25:83–89.

• 16. Hasson F, Keeney S, Mckennna H. Research guidelines for the Delphi survey technique. J Adv Nurs 2000;32:1008–1015.

• 17. Lin SC, Whipple DM, Ho CS. Evaluation of statistical equivalence using limits of agreement and associated sample size calculation. Commun Stat Theor Methods 1998;27:1419–1432.

• 18. Hahn GJ, Meeker WQ. Statistical intervals – a guide for practitioners. New York: John Wiley & Sons, 1991:150–167. Google Scholar

• 19. McAlinden C, Khadka J, Pesudovs K. Statistical methods for conducting agreement (comparison of clinical tests) and precision (repeatability or reproducibility) studies in optometry and ophthalmology. Ophthalmic Physiol Opt 2011;31:330–338.

• 20. Cesana BM, Antonelli P. Agreement analysis: further statistical insights. Ophthalmic Physiol Opt 2012;32:436–440.

Published Online: 2016-11-12

Published in Print: 2016-11-01

Citation Information: The International Journal of Biostatistics, Volume 12, Issue 2, 20150039, ISSN (Online) 1557-4679, ISSN (Print) 2194-573X,

Export Citation

© 2016 Walter de Gruyter GmbH, Berlin/Boston.

## Citing Articles

[1]
Cecília Czakó, Lilla István, Mónika Ecsedy, Zsuzsa Récsán, Gábor Sándor, Fruzsina Benyó, Hajnalka Horváth, András Papp, Miklós Resch, Ágnes Borbándy, Zoltán Zsolt Nagy, and Illés Kovács
International Journal of Retina and Vitreous, 2019, Volume 5, Number 1
[3]
ShagunBhatia Shah, AjayKumar Bhargava, Uma Hariharan, Gayatri Vishvakarma, ChamoundRai Jain, and Anamica Kansal
Indian Journal of Anaesthesia, 2018, Volume 62, Number 8, Page 584
[4]
Nik Nazihah Nik Azis, Fiona Lee Min Chew, Siti Famira Rosland, Azura Ramlee, and Jemaima Che-Hamzah
Journal of American Association for Pediatric Ophthalmology and Strabismus, 2019
[5]
Pietro Picerno, Pietro Caliandro, Chiara Iacovelli, Chiara Simbolotti, Michele Crabolu, Danilo Pani, Giuseppe Vannozzi, Giuseppe Reale, Paolo Maria Rossini, Luca Padua, and Andrea Cereatti
Scientific Reports, 2019, Volume 9, Number 1
[6]
Xuebin Wang, Yuan gao, Yunyun Yang, Zhengyue Liu, Lihong Gao, Fengbo Wu, Xianmi Yang, Xiaojian Xu, Zhuo Wang, and Shusen Sun
Journal of Pharmaceutical and Biomedical Analysis, 2019, Page 112890
[7]
David Jeffrey Canty, Martin Kim, Ranjan Guha, Tuan Pham, Alistair G. Royse, Sandy Errey-Clarke, Julian A. Smith, and Colin F. Royse
Journal of Cardiothoracic and Vascular Anesthesia, 2019
[8]
Maxence Compagnat, Jean Christophe Daviet, Charles S Batcho, Romain David, Jean Yves Salle, and Stephane Mandigout
Brain Injury, 2019, Volume 33, Number 10, Page 1341
[9]
Maxence Compagnat, Charles Sebiyo Batcho, Romain David, Nicolas Vuillerme, Jean Yves Salle, Jean Christophe Daviet, and Stéphane Mandigout
Sensors, 2019, Volume 19, Number 11, Page 2497
[10]
Lifei Liu, Shangyingying Li, Mao Ye, Yonggang Li, Jie Tian, and Yanzhe Tan
Echocardiography, 2019, Volume 36, Number 5, Page 948
[11]
Minh D. Pham, Berhan A. Haile, Iskandar Azwa, Adeeba Kamarulzaman, Nishaan Raman, Alireza Saeidi, Maria Kahar Bador, Margaret Tan, Jiawei Zhu, Yi Feng, Julian H. Elliott, Mary L. Garcia, Fan Li, Suzanne M. Crowe, Stanley Luchters, David A. Anderson, and Yi-Wei Tang
Journal of Clinical Microbiology, 2019, Volume 57, Number 4
[12]
Roslyn J. Simms, Trushali Doshi, Peter Metherall, Desmond Ryan, Peter Wright, Nicolas Gruel, Maatje D. A. van Gastel, Ron T. Gansevoort, Wendy Tindale, and Albert C. M. Ong
[13]
Francisco Morales-Acuna, Brycen Ratcliffe, Caleb Harrison, Sierra Crowe, Evan Bockover, Robert Pawlak, and Alvaro N. Gurovich
European Journal of Applied Physiology, 2019, Volume 119, Number 4, Page 901
[14]
Momoko Iwamoto, Aurelie Calzia, Amelie Dublineau, François Rouet, Janin Nouhin, Sokchea Yann, Sophorn Pin, Chhorvy Sun, Kimchamroeun Sann, Chhit Dimanche, Celine Lastrucci, Rebecca M. Coulborn, David Maman, Jean-Philippe Dousset, and Anne Loarec
Journal of Viral Hepatitis, 2018
[15]
Sebastian Jaramillo, Mar Montane-Muntane, David Capitan, Ferran Aguilar, Antoni Vilaseca, Annabel Blasi, and Ricard Navarro-Ripoll
Transfusion, 2018
[16]
Aysun Çetin, Ahmet Şen, Ihsan Çetin, Behzat Çimen, Leyla Cimen, Göktug Savas, Ahmet Öztürk, and Mustafa Yavuz Koker
Turkish Journal of Biochemistry, 2018, Volume 0, Number 0
[17]
A. Szederjesi, L. Baronciani, U. Budde, G. Castaman, A. S. Lawrie, Y. Liu, R. Montgomery, F. Peyvandi, R. Schneppenheim, A. Várkonyi, J. Patzke, and I. Bodó
Journal of Thrombosis and Haemostasis, 2018
[18]
Cecília Czakó, Gábor Sándor, Mónika Ecsedy, Zsuzsa Récsán, Hajnalka Horváth, Zsuzsanna Szepessy, Zoltán Zsolt Nagy, and Illés Kovács
Scientific Reports, 2018, Volume 8, Number 1
[19]
Guy-Marino Hinnouho, Maxwell A Barffour, K Ryan Wessells, Kenneth H Brown, Sengchanh Kounnavong, Bigphone Chanhthavong, Kethmany Ratsavong, Chidchamai Kewcharoenwong, and Sonja Y Hess
Journal of Clinical Pathology, 2018, Volume 71, Number 6, Page 532
[20]
A Verkhovsky, M Smit, A Levin, and JF Coetzee
Southern African Journal of Anaesthesia and Analgesia, 2018, Page 1
[21]
Mohammed H. Rashid, Mark A. Stevenson, Shea Waenga, Greg Mirams, Angus J. D. Campbell, Jane L. Vaughan, and Abdul Jabbar
Parasites & Vectors, 2018, Volume 11, Number 1
[22]
Maksym Misyura, Mahadeo A Sukhai, Vathany Kulasignam, Tong Zhang, Suzanne Kamel-Reid, and Tracy L Stockley
Journal of Clinical Pathology, 2018, Volume 71, Number 2, Page 117
[23]
M. Compagnat, S. Mandigout, D. Chaporro, J.Y. Salle, and J.C. Daviet
Annals of Physical and Rehabilitation Medicine, 2018
[24]
Daniel P. Credeur, Leryn J. Reynolds, Seth W. Holwerda, Jennifer R. Vranish, Benjamin E. Young, Jing Wang, John P. Thyfault, and Paul J. Fadel
Experimental Physiology, 2018
[25]
Ji-Hyun Lee, Hyo-Jin Byon, Yoon-Hyeong Choi, In-Kyung Song, Jin-Tae Kim, Hee-Soo Kim, and Chandra Ramamoorthy
Pediatric Anesthesia, 2017