Abstract

Method comparison studies are usually evaluated by several statistical tests (e.g., regression analysis) which sufficiently describe the analytical (dis)agreement between the results of two procedures. However, they do not provide any information how differences, if observed, influence diagnostic decision making.

A novel statistical approach is described to test the clinical relevance of differences between two analytical procedures. The new procedure requires a population-based probability which describes the distribution of values within the population under study and an analytical probability quantifying the risk of errors due to replacing one method by the other. The population probability was derived from 171 subjects from two outpatient departments (internal medicine and dermatology) who were subjected to an oral glucose tolerance test because type 2 diabetes mellitus was suspected. The analytical probability was determined from duplicate glucose measurements in venous and capillary blood, and venous plasma in the fasting and 2 h post-challenge state by the routine method used in a central laboratory (Ebio analyzer) and a (POCT) glucometer (Elite). The two probabilities were combined into one “error rate” (discordance rate).

The new concept was applied to three examples. In the first example, a comparison between two analytical systems led to discordance rates above 15%. After transforming the Elite analyzer results by a regression function, the discordance rate decreased below 5%, which was considered to be acceptable for the diagnostic purpose studied. In the second example, discordance rates were estimated by comparing different sample systems with each other. The use of whole blood in comparison with venous plasma led to discordance rates of 5–7% for venous blood and 7–10% for capillary blood. The same data set was also used in a third example to derive decision limits for capillary and venous blood from the established plasma values.

The proposed procedure estimates the diagnostic error rate based on analytic performance characteristics and population probabilities. It extends the concept of (un)efficiency by including the effect of variability about a decision limit and the distribution of the measurement values in the patient population.