Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Clinical Chemistry and Laboratory Medicine (CCLM)

Published in Association with the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)

Editor-in-Chief: Plebani, Mario

Ed. by Gillery, Philippe / Greaves, Ronda / Lackner, Karl J. / Lippi, Giuseppe / Melichar, Bohuslav / Payne, Deborah A. / Schlattmann, Peter


IMPACT FACTOR 2018: 3.638

CiteScore 2018: 2.44

SCImago Journal Rank (SJR) 2018: 1.191
Source Normalized Impact per Paper (SNIP) 2018: 1.205

Online
ISSN
1437-4331
See all formats and pricing
More options …
Volume 53, Issue 6

Issues

Optimizing the use of the “state-of-the-art” performance criteria

Rainer Haeckel / Werner Wosniok / Thomas Streichert
Published Online: 2015-03-20 | DOI: https://doi.org/10.1515/cclm-2014-1201

Abstract

The organizers of the first EFLM Strategic Conference “Defining analytical performance goals” identified three models for defining analytical performance goals in laboratory medicine. Whereas the highest level of model 1 (outcome studies) is difficult to implement, the other levels are more or less based on subjective opinions of experts, with models 2 (based on biological variation) and 3 (defined by the state-of-the-art) being more objective. A working group of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) proposes a combination of models 2 and 3 to overcome some disadvantages inherent to both models. In the new model, the permissible imprecision is not defined as a constant proportion of biological variation but by a non-linear relationship between permissible analytical and biological variation. Furthermore, the permissible imprecision is referred to the target quantity value. The biological variation is derived from the reference interval, if appropriate, after logarithmic transformation of the reference limits.

Keywords: biological variation; permissible uncertainty; reference intervals

Introduction

The organizers of the first EFLM Strategic Conference “Defining analytical performance goals” identified three models for defining analytical performance goals in laboratory medicine. Whereas the highest level of model 1 (outcome studies) is difficult to implement, the other levels are more or less based on subjective opinions of experts, with models 2 (based on biological variation) and 3 (defined by the state-of-the art) being more objective. The benefits and disadvantages of models 2 and 3 have already been discussed elsewhere [1, 2].

Problems with the biological variation model are the large variability between studies (e.g., 2.3%–31.9% for triglycerides [3]), data are often generated from relatively young and healthy subjects, dependence on the time span studied (hours to years), and possible effects of measurand concentrations. Owing to the great diversity of literature reports, many authors consider biological variation not suited to set metrological requirements. Problems with the state-of-the-art model are lack of scientific reasoning, often based on “old” data, which may be outdated, lack of transparency, lack of neutrality (dependency on industry), and the lack of relationship between what is achievable and what is needed clinically.

A working group of the German Society of Clinical Chemistry and Laboratory Medicine (DGKL) has developed a combination of models 2 and 3 to overcome some disadvantages inherent to both models [2].

New concept for defining permissible performance criteria

Three types of biological variation (CVB) have been described: (1) intra-individual variation (CVI), (2) inter-individual variation (CVG), and (3) combined CVB (combined CVI and CVG). As a surrogate for the biological variation [2], the empirical (biological) coefficient of variation (CVE) has been proposed [1, 2]. According to Simundic et al. [4], CVE could be termed CVA+I+G.

The empirical biological variation CVE derived from the reference interval (RI) was proposed because laboratories are obliged to provide RI for all measurands, must check their transferability (if taken from external sources), and should review the RI periodically as required by ISO 15189 [5]. Furthermore, reference limits (RL) usually vary less than data on biological variation in the literature and often are from consensus groups.

RLs reflect the biological variations, including the analytical variation. If values are normally distributed, the empirical (biological) standard deviation, sE, can be estimated by (upper RL–lower RL)/3.92 [2]. However, a “true” empirical normal distribution does not exist in laboratory medicine. For small reference ranges and relatively high mean values (e.g., sodium and chloride concentrations in plasma or hematological quantities), the distribution usually looks close to “normal”, but a log-normal distribution has the same shape under these conditions. For relatively large reference ranges (e.g., thyreotropin, triglycerides, enzymes in plasma), it is obvious that the data are not normally distributed. Generally, it may be assumed that laboratory data follow a power normal distribution, i.e., (xλ–1)/λ has a normal distribution, where the coefficient λ controls the shape of the distribution. Important special cases are λ=1, indicating a normal distribution, and λ=0, the lognormal distribution. The λ value for a specific data set can be estimated by numerical methods.

If λ is unknown and cannot be estimated from data, e.g., because only RLs are given, but there are no individual values, the assumption of a logarithmic distribution was recommended [6]. Assuming a lognormal distribution, the empirical standard deviation (sE) is calculated from the lower reference limit (RL1) and the upper reference limit (RL2) by Eq. (1), as recently explained [2]:

sE,ln=(lnRL2lnRL1)/3.92. (1)(1)

Then, the CVE of a logarithmic normal distribution expressed on the linear scale (CVE*) can be calculated from Eq. (1), as explained in the Appendix:

CVE=(exp(sE,ln2)1)0.5100. (2)(2)

CVE* can be considered as a surrogate for the conventional biological variation (CVB), although it also contains the analytical variation. The CVE* values were compared (Figure 1) with the combined CVB values [CVB=(CVI2+CVG2)0.5] for 64 quantities (in blood, serum, and plasma) listed in the RiliBÄK [8]. The CVB data were taken from Ricos et al. [7]. Although this list is already very comprehensive, CVB data are available for only 80% of the RiliBÄK measurands. As shown in Figure 1, CVB and CVE* correlate quite well. The greatest divergences (above the regression line=black line in Figure 1) are obtained with C-reactive protein, CA 19-9, and prostate-specific antigen (PSA). In the Ricos et al. [7] list, a relatively high CVG value of 130% is presented for CA 19-9. Erden et al. [9] more recently published a CVG=64.2%, a value that appears more realistic. In Table 1, the intra-individual biological variation (CVI) of PSA is compared from several reports. The most comprehensive survey was published by Söletormos et al. [10] on behalf of the European Group on Tumor Markers. The CVI values from 13 studies varied between 2.1% and 22.9%. These large differences between studies make a choice of the correct CVB value difficult. Ricos et al. [7] selected a CVI=18.1, whereas a CVI of about 13% (close to the mean of the reported span) would correlate with the CVE* quite well.

Comparison of CVE* values with combined CVB values taken from Ricos et al. [7]. PSA, prostate-specific antigen; CRP, C-reactive protein.
Figure 1:

Comparison of CVE* values with combined CVB values taken from Ricos et al. [7].

PSA, prostate-specific antigen; CRP, C-reactive protein.

Table 1:

The intra-individual variability of plasma PSA reported by several researchers.

The empirical biological variation CVE* derived from the reference range can be applied to derive permissible analytical uncertainty and to determine quantity quotients standardizing reporting laboratory results.

Proposal for determining permissible analytical uncertainty

Various approaches have been presented to define permissible analytical uncertainty as a proportion of biological variation. Cotlove et al. [16] proposed to multiply CVB with a constant factor of 0.5. Fraser [11] suggested a three-class model with factors 0.25, 0.5, and 0.75, and Haeckel and Wosniok [1] suggested a five-class model. All models use more or less arbitrary factors. If permissible limits derived of the present state-of-the-art [root-mean-square of measurement deviation (RMSD) in Figure 2] are compared with biological variation data (CVE*), a curved relationship may be observed (analog to the curved line in Figure 2). With the Cotlove et al. [16] model (analog to the straight line in Figure 2), too stringent requirements are postulated, which can hardly be reached by the present technology for measurands with relatively small CVE* values. For measurands with larger biological variation, the requirements may be too permissive. The curved relationship for the present technology can be simply described by Eq. (3):

Comparison of empirical biological variation (CVE*) with permissible analytical variation derived of the present state of the art (technical feasibility). RMSD (circles), root-mean-square measurement deviation (column 3 of Table 2a of the German Guideline 2008); straight line, 0.5·CVE* line; diamonds, present proposal.
Figure 2:

Comparison of empirical biological variation (CVE*) with permissible analytical variation derived of the present state of the art (technical feasibility). RMSD (circles), root-mean-square measurement deviation (column 3 of Table 2a of the German Guideline 2008); straight line, 0.5·CVE* line; diamonds, present proposal.

pCVA=(CVE0.25)0.5. (3)(3)

This equation approximately describes the empirical relation between CVE* and the major part of the presently used permissible imprecision values. Eq. (3) describes a curved relation between permissible analytical CV (pCVA) and CVE*. It can be easily adapted to future technical improvements by modifying its parameters. The exponent in Eq. (3) may be reduced from 0.5 to, e.g., 0.45 if one wishes more stringent permissible limits.

An algorithm to estimate pCVA values for a single target or measured values has been recently proposed [2].

Permissible limits for combined and expanded measurement uncertainty

Three types of measurement uncertainty supported by many international organizations (BIPM, IEC, IFCC, ISO, IUPAC, IUPAP, and OIML) are described in the Guide to the Expression of Uncertainty in Measurement [17]: standard uncertainty uS (imprecision, standard deviation), combined uncertainty uC (u12+u22+u32)0.5, and expanded uncertainty U=kuC (if coverage factor k=1.96, the level of confidence is 95%). In laboratory medicine, at least two components have to be combined, imprecision and bias.

The permissible uncertainty (of measurement) can be given by Eq. (3), e.g., u1 in uC. For permissible bias (pB) (e.g., u2 in uC), 0.5·pCVA was previously suggested [2]. According to the GUM concept [17], an uncertainty component should be added for which also 0.5·pCVA was proposed [2]:

pB=(0.5pCVA2+0.5pCVA2)0.5=0.7pCVA. (4)(4)

Then, the permissible limit of uC is (pCVA2+pB2)0.5:

uC=1.22pCVA.

Considering a 95% probability of uC, the expanded uncertainty can be calculated by Eq. (5):

pU%=1.96puC=2.39pCVA, (5)(5)

where pU% corresponds to the RMSD value of the RiliBÄK 2008 [8, column 3 in Table B1a]. The pU% in relation to CVE* values were compared with the RMSD for several measurands (Figure 2). Mean limits for 64 plasma quantities were about 5% lower than those required by the new RiliBÄK [8]. The straight line represents the 0.5 multiplication factor for the biological variation as proposed by Fraser [11]. The problem with this more or less arbitrary classification is the difficulty to which a single measurand may be attributed. The expanded uncertainty also leads to a curved relation with CVE* (like pCVA vs. CVE*).

Permissible limits for ring trials (EQAS)

Recently, we have proposed that permissible limits in external quality assessment schemes (EQAS) may be derived of pCVA, respectively, of pU% values [18]. Analogous to this approach, the 90% interval of the permissible limits for EQAS may be

pUEQAS%=1.64pU%=3.92pCVA (6)(6)

and the 95% interval may be

pUEQAS%=1.96pU%=4.68pCVA. (7)(7)

Examples for the various permissible uncertainties were recently published [2]. Mean limits for 64 plasma quantities were similar to those required by the new RiliBÄK [8].

Discussion

The empirical (biological) variation (CVE*) derived from the reference range is suggested as a surrogate for the biological variation. RLs are available to all measurands, and most probably, the laboratories have more experience with these data because they have to validate them before their introduction in the diagnostic service and then to verify them periodically according to good laboratory practice [5]. CVE* values can be used to derive permissible uncertainty by algorithms that may reconcile the presently competing biological variation model and the state-of-the-art model.

Although CVE values correlate quite well with biological variation data (Figure 1), they combine intra- and inter-individual variations (CVC). The relation between both is not constant. The better choice would be to use only CVI data. Presently, however, these data vary considerably in the literature (see example presented in Table 1). Therefore, several authors refused to use CVI to derive permissible performance criteria [1, 2, 19]. The application of CVE data may be an acceptable compromise until more reliable data on intra-individual variations are available. Furthermore, CVE data vary between laboratories.

The working group “Guide limits” of the DGKL developed easily to handle Excel tools for the estimation of RIs from intra-laboratory data pools and the estimation of the permissible uncertainty. These tools are distributed gratuitously, e.g., via website of the DGKL [20] or from the authors and could be implemented by software companies in their information systems.

Appendix

Estimation of CVE*

  1. Assuming a logarithmic normal distribution for the measurand, the biological standard deviation derived of the RLs is on the logarithmic scale

    sE,ln=(lnRL2lnRL1)/3.92.

  2. The mean on the logarithmic scale is

    meanln=(lnRL2+lnRL1)/2.

  3. The corresponding mean on the linear scale is [21]

    meanlin=exp(meanln+0.5sE,ln2).

    However, this mean is not needed to calculate CVE*, as is shown below.

  4. sE,ln can be transformed to the linear scale according to Aitchison and Brown [21]:

    sE,lin=meanlin(exp(sE,ln2)1)0.5.

  5. CVE*=sE,lin·100/meanlin.

  6. CVE*=meanlin·(exp sE,ln2–1)0.5·100/meanlin.

  7. CVE*=(exp(sln2)–1)0.5·100.(2)

Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.

Financial support: None declared.

Employment or leadership: None declared.

Honorarium: None declared.

Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

References

  • 1.

    Haeckel R, Wosniok W. A new concept to determine permissible imprecision and bias on the basis of biological variation and state-of-the-art. Clin Chem Lab Med 2011;49:623–35.CrossrefGoogle Scholar

  • 2.

    Haeckel R, Wosniok W, Gurr E, Peil B. Permissible limits for uncertainty of measurement in laboratory medicine. Clin Chem Lab Med 2015. doi: 10.1515/cclm-2014-0874 [Epub ahead of print 23 Jan 2015].PubMedWeb of ScienceCrossrefGoogle Scholar

  • 3.

    Smith SJ, Cooper GR, Myers GL, Sampson EJ. Biological variability in concentrations of serum-lipids: sources of variation among results from published studies and composite predicted values. Clin Chem 1993;39:1012–22.PubMedGoogle Scholar

  • 4.

    Simundic AM, Kackov S, Miler M, Fraser CG, Hylthoft Petersen P. Terms and symbols used in studies on biological variation: the need for harmonization. Clin Chem 2015;61:1–2.CrossrefGoogle Scholar

  • 5.

    International Standard Medical laboratories – particular requirements for quality and competence, ISO 15189-2012.Google Scholar

  • 6.

    Haeckel R, Wosniok W. Observed, unknown distributions of clinical chemical quantities should be considered to be log-normal: a proposal. Clin Chem Lab Med 2010;48:1393–6.CrossrefWeb of ScienceGoogle Scholar

  • 7.

    Ricos C, et al. Available from: www.westgard.com. Accessed 2014.

  • 8.

    Richtlinie der Bundesaerztekammer zur Qualitätssicherung laboratoriumsmedizinischer Untersuchungen. Dt Aerzteblatt 2008;105:C301–13. Available from: www.aerzteblatt.de/plus1308.

  • 9.

    Erden G, Barazi AO, Tezcan G, Yildirimkaya MM. Biological variation and reference change values of CA 19-9, CEA and AFP in serum of healthy individuals. Scand J Clin Lab Invest 2008;68:212–8.CrossrefGoogle Scholar

  • 10.

    Söletormos G, Semjonow A, Sibley PE, Lamerz R, Hylthoft Petersen P, Albrecht W, et al. Biological variation of total prostate-specific antigen. A survey of published estimates and consequences for clinical practice. Clin Chem 2005;51:1342–51.CrossrefPubMedGoogle Scholar

  • 11.

    Fraser CG. Biological variation: from principles to practice. Washington, DC: AACC Press, 2001:1–151.Google Scholar

  • 12.

    Deijter SW, Martin JS, McPherson RA, Lynch JH. Daily variability in human serum prostate-specific antigen and prostatic acid phosphatase: a comparative evaluation. Urology 1988;32:288–92.CrossrefGoogle Scholar

  • 13.

    Erden G, Özden T, Soydas A, Yildirimkaya MM. Biological variation and reference change value (RCV) of prostate specific antigen (PSA) levels in the serum of healthy young individuals. Gazi Med J 2009;20:152–6.Google Scholar

  • 14.

    Schifman RB, Ahmann FR, Elvick A, Ahmann M, Coulis K, Brawer MK. Analytical and physiological characteristics of prostate specific antigen and prostatic phosphatase in serum compared. Clin Chem 1987;33:2086–8.PubMedGoogle Scholar

  • 15.

    Gurr E, Krönig F, Golbeck A, Arzideh F. Limits for the determination of guiding values from intralaboratory data basis, demonstrated by prostate specific antigen (PSA). J Lab Med 2009;33:67–70.Google Scholar

  • 16.

    Cotlove E, Harris EK, Williams GZ. Biological and analytic components of variation in long-term studies of serum constituents in normal subjects. Clin Chem 1970;16:1028–32.PubMedGoogle Scholar

  • 17.

    ISO (International Organisation for Standardisation). ISO/IEC Guide 98-3:2008 Uncertainty of measurement. Part 3: guide to the expression of uncertainty in measurement (GUM:1995). Genf: ISO, ISBN 92-67-10188-9.Google Scholar

  • 18.

    Haeckel R, Wosniok W, Kratochvila J, Carobene A. A pragmatic approach for permissible limits in external assessment schemes with a compromise between biological variation and the state of the art. Clin Chem Lab Med 2012;50:833–9.PubMedWeb of ScienceCrossrefGoogle Scholar

  • 19.

    Fuentes-Arderiu X. Variability of the biological variation. Scand J Clin Lab Invest 2002;62:561–4.CrossrefGoogle Scholar

  • 20.

    Permissible imprecision (pCVA) and combined uncertainty (pU%) for a particular measurand (xi). Available from: www.dgkl.de. Accessed on 10 January, 2014.

  • 21.

    Aitchison J, Brown JA. The lognormal distribution. Cambridge: Cambridge University Press 1969:1–176.Google Scholar

About the article

Corresponding author: Rainer Haeckel, Bremer Zentrum für Laboratoriumsmedizin, Klinikum Bremen Mitte, 28305 Bremen, Germany, Phone: +49 412 273448, E-mail:


Received: 2014-12-05

Accepted: 2015-02-09

Published Online: 2015-03-20

Published in Print: 2015-05-01


Citation Information: Clinical Chemistry and Laboratory Medicine (CCLM), Volume 53, Issue 6, Pages 887–891, ISSN (Online) 1437-4331, ISSN (Print) 1434-6621, DOI: https://doi.org/10.1515/cclm-2014-1201.

Export Citation

©2015 by De Gruyter.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Arnaud Josiane, Marina Patriarca, Béatrice Fofou-Caillierez Ma’atem, Montserrat González-estecha, MªCarmen González gómez, Irene De Graaf, Patriarca Valeria, Martine Ropert-Bouchet, Liesbeth Schröer-Janssen, Carla Siebelder, Te Winkel Marieke, Ventura Alemany Montserrat, and Cas Weykamp
Journal of Trace Elements in Medicine and Biology, 2019, Page 126414
[2]
F. Ceriotti, P. Fernandez-Calle, G. G. Klee, G. Nordin, S. Sandberg, T. Streichert, J.-L. Vives-Corrons, and M. Panteghini
Laboratornaya sluzhba, 2019, Volume 8, Number 1, Page 93
[3]
Rainer Haeckel, Eberhard Gurr, Torsten Hoff, and on behalf of the working group Guid
LaboratoriumsMedizin, 2016, Volume 40, Number 4

Comments (0)

Please log in or register to comment.
Log in