This paper reflects the opinion of the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group Accreditation and ISO/CEN standards (WG-A/ISO). It aims to provide guidance for drawing up local/national documents about validation and verification of laboratory methods. We demonstrate how risk evaluation can be used to optimize laboratory policies to meet intended use requirements as well as requirements of standards. This is translated in a number of recommendations on how to introduce risk evaluation in various stages of the implementation of new methods ultimately covering the whole process cycle.
The scope of this paper is to provide guidance for drawing up local/national documents about validation and verification. In essence, this is of course described in ISO 15189:2012 , which was developed as a standard to serve as a fundamental tenet for the quality management systems in medical laboratories. The standard is mandatory on which procedures have to be in place and what aspects have to be covered by these procedures, but deliberately not on the specific way that these aspects have to be covered. The standard relies on risk management by laboratory professionals to ensure that the local demands are met under local circumstances. For validation and verification of examination procedures, this means that laboratory professionals use their professional assessment to judge which performance characteristics need what evaluation to make sure that the intended quality can be delivered. This opinion paper provides a structured approach for such assessment. It guides how to meet the requirements of the standard without becoming too prescriptive, which would limit the professional judgment. Although the differences between validation and verification were clarified in the revised version of 2012 (184.108.40.206-220.127.116.11), questions remain and differences in their interpretation between countries are quite large .
In some countries, for example France, an explanation of ISO 15189:2012 is issued by the National Accreditation Body, accompanied by recommended templates on how to perform a validation or verification , . Often, supplementary demands are made on top of the requirements from ISO 15189:2012, thereby restricting the freedom that was originally offered. Additionally, some laboratories have published their own practice to fulfil the requirements as a useful example to others .
We propose to apply risk consideration (in other words, professional evaluation) to verification/validation procedures to find the right balance between regulatory requirements on one side and professional autonomy on the other side, both within the intentions and requirements of the standard. Practically, this means that we will not define numbers of samples required for verification and validation experiments or prescribe the performance of certain experiments at all. We recommend careful consideration in light of the intended use of the examination procedure, taking into account all available information. By documenting the line of thought, choices become traceable. This proposal has its roots in the Dutch guideline on verification and validation, as adopted by the national scientific society . We describe in a point-by-point approach the subsequent steps needed to fulfil the requirements of the standard with respect to verification and validation. This sequence starts with the selection of an examination procedure and ends with its approval for implementation when fit for purpose. Closely related subjects like intended use, metrological traceability (ISO 15189:2012 18.104.22.168), and internal (ISO 15189:2012 22.214.171.124) and external quality control (ISO 15189:2012 126.96.36.199) are addressed within the framework of validation and verification.
This paper is primarily intended for examination procedures used for human diagnosis or follow-up in clinical chemistry laboratories. Measuring equipment such as analyzers are not taken into consideration, at least not where installation and technical-operational aspects of such measuring equipment are concerned. The focus essentially is on quantitative measuring of a measurand in a biological matrix; however, it may also (partly) apply to qualitative analysis. Furthermore, this paper may also be useful and relevant for other disciplines in laboratory medicine.
The introduction of a new examination procedure
In compliance with ISO 15189:2012, examination procedures shall be subjected to independent validation or verification by the medical laboratory prior to being introduced . We define validation as the demonstration via objective evidence that a new or modified examination procedure from one’s own working environment (or laboratory) is appropriate for a specific intended use in medical diagnostics, and that it complies with the relevant acceptance criteria as described by the medical laboratory. A verification, by our definition, always concerns an already validated examination procedure, and its appropriateness should be confirmed. Which performance characteristics and acceptance criteria are relevant for validation or verification shall, depending on the intended application, be established in the validation or verification plan in order to objectively assess if the examination procedure is appropriate for the intended use. The “intended use” of a medical examination procedure comprises the clinical condition and/or the indication prompting the examination procedure, as well as the manner in which the examination procedure, including preparation, is to be carried out. The acceptance criteria and the intended use are connected. Changing the intended use for specific situations will possibly alter the acceptance criteria.
The laboratory shall use only examination procedures that have been validated or verified by the laboratory for the intended use.
Validation or verification
In practice, there is frequent confusion about the choice between validation and verification. This depends on the availability of reliable and valid data on the performance characteristics of the envisioned examination procedure (Figure 1). The data can, for example, be supplied by the diagnostics supplier (CE- or FDA-approved) or be taken from peer-reviewed texts/journals or from validation data from other accredited laboratories.
The laboratory collects reliable and valid data on the performance characteristics and examines the data to ascertain if the acceptance criteria have been met. If so, verification of relevant performance characteristics is sufficient. If the performance characteristics are either not available or do not meet the acceptance criteria, the laboratory shall collect its own data (validation). Therefore, for example, it may occur that for procedures validated elsewhere, verification can be sufficient for precision, trueness and decision limit procedures, but that supplemental validation of the sample’s stability will be necessary.
Validation of examination procedures
ISO 15189:2012  emphatically states a number of categories of examination procedures to be validated. The term “methods” mentioned in ISO 15189:2012  corresponds to the new term, “examination procedures”.
The laboratory shall validate examination procedures derived from the following sources:
non-standard examination procedures
laboratory-designed or developed examination procedures
validated examination procedures used outside their intended use
validated examination procedures subsequently modified
Recommendation II further explained: The non-standard procedure named under (a) is named as such in ISO 15189:2012  but in practice always refers to the procedure here listed under (b) or (c). In (c), a “validated examination procedure” is considered equivalent to the terminology “standard method” named in ISO 15189:2012 .
Part of the validation process is the objective establishment (through measurement) of the relevant performance characteristics. In the aforementioned situations, either the performance characteristics are not established according to ISO 15189:2012, or the examination procedure has been modified in such a way that these performance characteristics are, according to ISO 15189:2012, not automatically valid for the modified examination procedure . If an examination procedure is modified after validation or verification, the effect of such a modification shall be taken into consideration and evaluated (documented motivation). A professional evaluation, for example of decreasing the sample volume in a test to make it suitable for pediatric samples, would take into account whether this decrease could change the reaction mixture conditions significantly and as a consequence which performance characteristics might be affected by such a change.
If an examination procedure is modified, it shall be taken into consideration and documented if there are potential/relevant consequences for the performance characteristics and, if so, what these consequences are.
When an examination procedure is modified with respect to the documented motivation, a (supplemental) validation or verification shall be performed.
Verification of examination procedures
The performance characteristics of the examination procedure evaluated during the verification process shall be relevant for the intended use of the results of the examination. If all the relevant performance characteristics are both available and valid for one’s own laboratory, the examination procedure can be subject to verification instead of validation. Here, “valid” means that the data have been obtained under documented similar conditions as those for the intended use. The laboratory shall have information available from the manufacturer/developer of the examination procedure or examination results from reliable independent studies in order to be able to confirm the known performance characteristics of the examination procedure. When more than one device is used to measure the measurand, the correct operation of each individual device shall be verified.
Examination procedures that, on the basis of available documentation, do not necessarily have to be validated shall at least be verified for the relevant performance characteristics.
If evidence from a validation performed elsewhere is incomplete, verification is insufficient and a supplemental validation in one’s own laboratory shall be necessary.
When using more than one analyzer for the same measurand, an appropriate verification of each individual device shall be performed applying the appropriate acceptance criteria.
Establishing a validation or a verification plan
For both validation and verification, performance characteristics shall be assessed against the acceptance criteria. These acceptance criteria are pre-established and justified in the validation and verification plans, respectively. Besides the acceptance criteria, the measurand, intended use and the examination procedure shall have to be established; furthermore, the investigators and the staff member with the appropriate authority, based on professional education and responsibility, shall be identified.
Particularly for validation of an (self-developed) examination procedure, it is essential to clearly define the measurand. Describing the measurand requires knowledge of the component or chemical entity (analyte), the matrix and the condition of the analyte, as well as the characteristics of the examination procedure used. This plays a crucial role, for example, in immunochemical examination procedures.
Validation/verification shall take place according to a pre-established validation/verification plan, which is authorized by staff with the appropriate authority, comprising at least the following elements:
intended use of the examination procedure
documentation of the measurand
selection of the relevant performance characteristics
acceptance criteria valid for the intended use
identity of investigators and competent authorizer(s)
The laboratory shall document the procedure used for the validation and record the results obtained. A staff member with the appropriate authority shall assess validation results and compile the assessment report.
The following possible relevant characteristics for validation are proposed in ISO 15189:2012, section 188.8.131.52 (under NOTE): measurement trueness; measurement accuracy; measurement precision, including measurement repeatability and measurement intermediate precision; measurement uncertainty; analytical specificity, including interfering substances; analytical sensitivity, detection limit and quantitation limit; and measuring interval, diagnostic specificity and diagnostic sensitivity of the measurement . Whether or not these performance characteristics have to be examined is left to the judgment of the staff member with appropriate authority; in addition, examination should be statistically sound. Professional evaluation with regard to its intended use is an important element in the selection of the performance characteristics to be considered. Results of preliminary studies of the intended method or specifications claimed by the manufacturer as well as medical views or guidelines concerning acceptance criteria could be used as the starting point and should be documented as such.
In validation and verification, the laboratory shall take into consideration all these performance characteristics. As accuracy is the combination of bias and imprecision, it can be omitted as an individual performance characteristic. As the lack of accuracy is defined as measurement uncertainty, satisfactory bias and imprecision will automatically lead to acceptable measurement uncertainty, which then satisfies ISO 15189:2012 section 184.108.40.206 that requires laboratories to define performance criteria for measurement uncertainty . At the moment, measurement uncertainty is the subject of a separate ISO standard under development  and therefore will not be discussed further here.
At least for some performance characteristics, local analytical confirmation is inevitable. Imprecision is subject to local conditions and therefore must be assessed by analysis. Also, comparison to the previous method if applicable must be assessed under local conditions, as such information is essential to inform test requesters about changes to be expected.
In verification, the staff member with appropriate authority shall take the following performance characteristics into consideration: measurement precision, measurement trueness, detection limits, stability, reference interval, comparison to previous method if present, medical decision values and interferences. Such consideration should be based on professional evaluation in the light of the intended use. Imprecision and comparison to previous method must be assessed by local experiments.
In validation, the staff member with appropriate authority shall take into consideration all the performance characteristics listed in ISO 15189:2012 220.127.116.11.
Whenever specific performance characteristics are neither applicable nor feasible in view of the nature of the examination procedure or the prevalence of pathology, this should be documented and motivated.
Common models for establishing acceptance criteria for precision and bias can be found in the Milan consensus statement on this topic .
The documentation associated with a validation or verification consists of a validation/verification plan, the results, including the raw data and a validation/verification report, in which, besides the examination of the acceptance criteria, the implementation is also described.
The results obtained shall be documented and stored in a validation/verification report, at least for the period in which the examination procedure is in use.
The results obtained shall be demonstrably evaluated using the acceptance criteria established in the validation/verification plan. The conclusion of the evaluation and whether or not it is appropriate for the intended use shall be established and archived in a validation/verification report. The validation/verification report shall be assessed and authorized by a staff member with the appropriate authority.
Note: the documentation is stored for the duration of use, extended with the (self-established) storage period of all other registrations pertaining to results from the laboratory.
Release, implementation and performance assurance
A positive conclusion in the validation/verification report forms the basis for releasing the method for the intended use. The examination procedure to be released shall be described in a standard operating procedure (SOP). An internal and external quality control system, or a suitable alternative as mentioned in ISO 15189:2012 5.6.2 and 5.6.3, shall be established before the examination procedure may be released.
The release of an examination procedure on the basis of compliance with the acceptance criteria shall be carried out by a staff member with the appropriate authority, with a starting date stated in the documentation.
The examination procedure investigated shall be documented in a SOP in compliance with ISO 15189.
In implementing the method, an appropriate quality assurance procedure shall be established with acceptance limits for internal and external quality controls, taking into consideration decision limits of medical importance.
Implementation of a new examination method requires communication to requesters and all the other parties on the relevant modifications arising from the results of the validation/verification. It will be necessary to establish an implementation procedure and/or checklist.
The process of implementation of a new or modified examination procedure shall be set out in an instruction or checklist.
The requester should be informed of modifications in reported results or examination procedures used that might have consequences for the interpretation by the requester.
When modifications in examination procedures are judged to be of importance to the requester, it shall be documented how requesters are to be informed.
To provide assurance in the long term that the examination procedure performs in the same manner as during the validation/verification, also after release, a procedure should be established for the use and assessment of internal and external quality samples, including decision limits of medical importance.
There shall be a procedure by which it can be periodically ascertained if the original acceptance criteria for measurement precision and trueness, as determined during the validation/verification, are still met. If, upon reflection, more flexible criteria are justifiable, this should be substantiated.
Discussion and conclusions
With this paper, we aim to offer guidance on validation and verification of examination procedures complying with ISO 15189. The described approach valorizes the competences of the specialist in laboratory medicine to assess the requirements and specifications needed for a new examination procedure in the medical laboratory and to carry out an appropriate verification or validation. The structured approach in professional evaluation and documentation provides a traceable line of thought in which all requirements of this part of the standard are addressed. This approach relies on the skills of laboratory professionals trained and experienced in the issues at hand. It is meant to balance efforts and outcomes, securing accurate results, and preserving resources to be used elsewhere in favor of effective healthcare.
When regulators (national or supranational) impose precise restrictive guidelines, these shall be applied. We would then hope that in the spirit of ISO 15189:2012, such guidelines come into being with the public participation of the scientific organizations and are subject to a continuous PDCA cycle, balancing efforts and outcomes ultimately to adjust regulations and to promote effective healthcare. Obviously, the outcome of vigilance and surveillance efforts, of audits by accrediting bodies, and of self-assessment by professional organizations can be used as input for this cycle.
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: None declared.
Employment or leadership: None declared.
Honorarium: None declared.
Competing interests: The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.
1. ISO 15189:2012 Medical laboratories – Requirements for quality and competence.Search in Google Scholar
3. COFRAC SH REF 02. Exigences pour l’accréditation selon la norme NF EN ISO 15189. 2016. Available from: https://www.cofrac.fr/documentation/SH-REF-02. Accessed 23 August 2018.Search in Google Scholar
4. COFRAC SH GTA 04. Guide technique d’accréditation de vérification (portée A)/validation (portée B) des méthodes en biologie médicale. Available from: https://www.cofrac.fr/documentation/SH-GTA-04 Accessed 23 August 2018.Search in Google Scholar
5. Antonelli G, Padoan A, Aita A, Sciacovelli L, Plebani M. Verification of examination procedures in clinical laboratory for imprecision, trueness and diagnostic accuracy according to ISO 15189:2012: a pragmatic approach. Clin Chem Lab Med 2017;55:1501–8.10.1515/cclm-2016-0894Search in Google Scholar
6. Validation and verification of examination procedures in medical laboratories: a practical proposal for dealing with the ISO15189:2012 demands, 2017. DOI: 10.13140/RG.2.2.18137.72809, 2013. Available from: https://www.eflm.eu/upload/docs/Verification%20and%20Validation%20-%20Dutch%20NS.pdf. Accessed 13 August 2018.Search in Google Scholar
7. ISO/DTS 20914 Medical laboratories – Practical guide for estimation of measurement uncertainty (standard under development).Search in Google Scholar
8. Sandberg S, Fraser C, Horvath AR, Jansen R, Jones G, Oosterhuis W, et al. Defining analytical performance specifications: consensus statement from the 1st Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine. Clin Chem Lab Med 2015;53:833–5. (Milan conference 2014).10.1515/cclm-2015-0067Search in Google Scholar PubMed
©2021 Marc Thelen et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.