Skip to content
BY 4.0 license Open Access Published by De Gruyter March 15, 2023

Clinical evidence requirements according to the IVDR 2017/746: practical tools and references for underpinning clinical evidence of IVD-MDs

  • Karine Charrière ORCID logo EMAIL logo and Lionel Pazart

Abstract

In May 2022, the European Regulation 2017/746 (IVDR) came into force. It changes the approach of in vitro medical devices (IVD-MDs) for industry and institutions. It reinforces the clinical evidence requirements to improve performance, safety and transparency. Despite extended transition periods and existing guides, IVDR remains difficult to interpret and bringing devices into compliance requires efforts. The generation of clinical evidence is essential to demonstrate compliance with IVDR, and encompasses scientific validity, analytical performance and clinical performance. It is required to demonstrate, per intended use in the target population and clinical care pathway, IVD-MDs clinical performance (compared to a predefined clinical performance). Thus, there is a need for IVD-manufacturers and end-users in health care institutions, to obtain guidance on how to generate this clinical evidence. This article aims industrials and clinicians to identify key steps imposed by the IVDR for bringing IVD-MDs to the EU-market. We propose a general view of performance evaluation requirements for IVD-MDs and provide key references, including how to establish study design that will enable to document clinical performance of existing, refined or emerging medical tests. Finally, we propose a roadmap to address the relevant questions and studies in relation to the documents requested in the IVDR.

Introduction

The new European regulation for in vitro diagnostic medical devices 2017/746 [1] (IVDR) came into force on May 2022 and replaces the directive 98/79/EC (IVDD) [2]. It reinforces the requirements for manufacturers with a view to improving the performance and safety of devices and increasing transparency throughout the industry sector.

Extensions to transition periods have been granted. They depend on the class of the concerned device and the status of its CE marking under IVDD [3]. They are mainly justified by the lack of certified notified bodies (NB) able to deliver the CE marking. Under the IVDD, 8% of the in vitro diagnostic medical devices (IVD-MDs) on the market involved CE marking with control by NB. With the new regulation, the proportion should increase up to 78% [4]. There are questions about the degree of preparation of producers/users of IVD-MDs to meet these stricter obligations, in particular on the point of demonstrating performances. Furthermore, only eight NB are currently certified, compared to about 20 previously. The European Commission wants to be reassuring [5], but the number of NB remains largely insufficient and there is a risk that companies will not be able to market new IVD-MDs [6]. The Medical Device Coordination Group (MDCG) proposed solutions in the MDCG 2022-14 [7] and the European Commission published an amendment in January 2023 providing a phased roll out [8]. IVDR 2017/746 still leads to more threats than opportunities, especially for home-made test, high risk IVD-MDs [9] or for IVD-MDs use outside their approved indications. Therefore, the authors highlighted the effort required to ensure that all these tests can be performed in compliance with the IVDR.

Others point out difficulties of understanding and implementing articles from IVDR, even if the MDCG has published support documents to help with interpretation and despite the existence of international standards [7, 10]. For example, van Deutekom and Haitjema raised the issue that software development is not integrated into the ISO-15189 standard and note a lack of knowledge in medical laboratories on how to deal with software developed in-house, using ISO-15189 in synergy with IEC-62304 [10, 11]. They published recommendations to help the scientific community to facilitate the compliance with IVDR and described three use cases to illustrate their proposed methodology [12].

All IVD-MDs must comply with the new IVDR. This includes up-classification of devices already on the market, CE-marked devices under IVDD, using IVD-MDs outside their approved indications and marking of new devices. This paper aims at helping developers of IVD-MDs (researcher, manufacturer, labs staff) to better understand the key requirements in the IVDR, such as the evaluation of clinical evidence. We also provide references describing how to meet them.

Overview of IVDR content

The IVDR contains 10 chapters and 15 annexes (Supplementary Table 1). It establishes the rules for placing on the market, making available on the market or putting into service IVD-MDs for human use and their accessories in the European Union, and applies to performance studies concerning these devices and their accessories carried out in the Union. A number of new concepts compared to the IVDD are included (near-patient testing devices, companion diagnosis devices, falsified device, kit, common specifications, genetic testing and single-use testing). All manufacturers have to prove that their IVD-MDs comply with the general safety and performance requirements, even for in-house IVD-MDs.

Classification of IVD-MDs is now based on the risk level. Class A corresponds to the least risky devices and includes sample containers or buffer solutions (among others). Self-testing devices with no risk like pregnancy or fertility tests belong to class B. Companion devices, such as those used to screen for cancer, perform prenatal screenings or blood grouping devices are C classified. D is the riskiest class with devices for the detection or diagnosis of life-threatening or transmissible diseases. The definition of the IVD-MDs class conditions the whole CE marking process. For example, class A non-sterile devices can be self-certified. For other classes, conformity assessment involves a NB. For class B and above, a quality system is required. For classes C and D, the device conformity can be proved in two ways: establishing a complete quality management system or a type examination and a “production quality assurance”. Class D devices also involve batch control, but an additional opinion of a reference laboratory is required.

Performance evaluations and interventional studies are reported as an addition to the initial technical or post-marketing surveillance documentation.

Note that the IVDR covers IVD-MDs throughout their life cycle. Consequently, follow-up to ensure post-marketing performance monitoring, vigilance and market surveillance must be performed and the technical documentation regularly updated.

Key steps for bringing IVD-MDs to the EU-market

Throughout the lifecycle of an IVD-MD (see [13] for a detailed flow chart of the certification cycle), performance evaluation is not the first step (Supplementary Figure SF1). Indeed, it is necessary to ensure that the device is covered by the IVD-MD definition. To summarize, an IVD-MD is a MD intended to be used in vitro for the examination of samples from the human body, the purpose being to provide information on the patient’s health status. Specimen containers are also considered as IVD-MDs.

Once the IVD-MD status is identified, the intended use of the IVD-MD must be precisely and carefully defined. The compliance with the essential requirements must then be demonstrated, including performance evaluation for the expected intended use of the IVD-MD. CE mark can be obtained only if all conditions of performance and safety are met. It must be maintained by carrying out post-market surveillance and vigilance.

About clinical evidence

Clinical evidence must be provided during demonstration of performance and more precisely during the demonstration of clinical performance. According to the IVDR, “clinical evidence means clinical data and performance evaluation results, pertaining to a device of a sufficient amount and quality to allow a qualified assessment of whether the device is safe and achieves the intended clinical benefit(s), when used as intended by the manufacturer”.

In other words, clinical evidence demonstrates that the intended clinical benefit(s) and safety is achieved in accordance with the state of the art in the concerned medical field. This clinical evidence is based on three main pillars (Supplementary Figure SF2).

  1. Scientific validity proves that the analyte, marker or molecule being studied is relevant to the physiological or pathological condition of interest.

  2. Analytical performance is the ability of the device to measure this analyte, marker or molecule accurately and reproducibly. It is strictly a technical performance as no correlation with the targeted pathology is required. It is possible to demonstrate the analytical performance of an IVD-MD with artificial samples. This must demonstrate the accuracy, the measurements reproducibility, the limit of the blank, the analytical specificity and all other technical parameters ensuring the reliability of the measurements.

  3. Clinical performance is the ability of an IVD-MD to produce results which correlate with the actual clinical condition, depending on the target population and user. There exists an underlying intention of use that can be highlighted by a performance indicator. In order to demonstrate clinical benefit, dedicated guides depict the general principles of clinical evidence [7, 14].

Two other concepts are important to stakeholders but are addressed differently by the IVDR: clinical benefit and clinical utility. The notion of clinical benefit often appears in the IVDR but no mention of clinical utility is found.

Clinical benefit “means the positive impact of a device related to its function such as that of screening, monitoring, diagnosis or aid to diagnosis of patients, or a positive impact on patient management or public health”. Important, clinical benefit is not identically addressed by MDR and IVDR. While for MD, clinical benefit is measured directly by a positive impact on the patient’s health, for IVD-MDs, the objective is to obtain precise medical information on the patient. The positive impact on the patient is therefore indirect, since the final clinical result depends on other therapeutic options.

Clinical utility is “the likelihood that a test will, by prompting an intervention, result in an improved health outcome” (National Institute of Health). In their article, Horvath et al. defined clinical utility as “Ability of a test to improve health outcomes that are relevant to the individual patient” [15]. The clinical utility clearly targets the patient, whereas the clinical benefit is a broader concept extended to public health but potentially with no direct benefit for the tested patient. For example, screening for COVID 19 allows protecting populations by implementing an adapted public health policy more than improving a patient health outcome. Furthermore, clinical benefit is clearly defined and will not be in the “eyes of the beholder” like clinical utility could be, as explain by Lesko et al. in their interesting issue [16].

Performance evaluation process

Collection of clinical evidence is a part of a performance evaluation process. This process is maintained throughout the lifecycle of the IVD-MD. In addition to the IVDR and other documents [7, 13, 17], ISO 20916 [18] can be used to ensure that the clinical performance study will result in reliable and robust results.

Writing the performance evaluation plan (PEP)

The starting point of the process is the performance evaluation plan (PEP). The PEP identifies the approach, the method to be used and the steps required to generate clinical evidence. Although it remains very global, it describes how to prove that the three above-mentioned pillars are robust. It thus integrates basic elements concerning the IVD-MD (intended use, characteristic, analyte, type of patients, general requirements), more precise elements concerning the evaluation itself (methods, statistical analysis plan, state of the art, benefit/risk ratio) and steps planned for post-marketing performance monitoring (Figure 1A).

Figure 1: 
Documentation to be produced for the performance evaluation. The performance evaluation plan is used to plan the entire process (A), while the clinical performance evaluation plan should be thought of as a clinical investigation protocol (B). After the completion of one of the three types of clinical performance studies (C), a clinical performance evaluation report (D) is written. It will be added to the performance evaluation report which also includes the scientific validity and analytical performance reports (E).
Figure 1:

Documentation to be produced for the performance evaluation. The performance evaluation plan is used to plan the entire process (A), while the clinical performance evaluation plan should be thought of as a clinical investigation protocol (B). After the completion of one of the three types of clinical performance studies (C), a clinical performance evaluation report (D) is written. It will be added to the performance evaluation report which also includes the scientific validity and analytical performance reports (E).

To write an appropriate PEP, it is important to define minimum clinical performance levels a new test must reach (Figure 2, 1). Lord et al. propose a 5 steps approach to help achieving this goal [19]. The first step is to define the intended benefits: improving disease outcomes, reducing iatrogenic harm or other benefits (reducing cost for example). The second step is to map current practice. The third step is to propose test role by redrawing the clinical pathway with the new test positioned in: add-on, triage, replacement test. It can be a new pathway (fulfilled an unmet need) if there is no existing test. The fourth step is to link clinical performance requirements to intended benefits such as improving detection, improving rule out or improving test process. The final step is to set minimum acceptable clinical performance levels.

Figure 2: 
Roadmap for the collection of clinical evidence for relevant questions, performance assessments and mandatory reporting to comply with IVDR.
Figure 2:

Roadmap for the collection of clinical evidence for relevant questions, performance assessments and mandatory reporting to comply with IVDR.

Collecting existing data

With these elements, data can be collected and analysed (Figure 2, 2 and 3). As described by Leeflang and Allerberger the first point is to check the scientific validity i.e. to discriminate patients with the disease of interest from those without using the studied analyte. The second is to validate the analytical performance with evaluation of the metrological traceability of test results and verification whether the results are within allowable limits of measurement according to ISO 17511:2020 [20]. If the results are robust clinical performances with the targeted population can be validated [21].

For each of these aspects, data could already exist in scientific literature and the preferred method to collect existing data is systematic reviews. The PICO method is frequently used for formulating research questions: define the Population of interest (Patient/Problem), the intervention (mean treatment) performed on the Population, the Control treatment and the Outcome [22]. To check the quality of included articles, PRISMA is a recommended method (Preferred Reporting Item for Systematic Review and Meta-Analysis) [23, 24].

Quadas-2 (QUality Assessment of Diagnostic Accuracy Studies) is the recommended method to assess the quality of diagnostic accuracy studies. It summarizes the review questions, tailors the tool and produces review-specific guidance, constructs a flow diagram for the primary study, and judges bias and applicability [25].

Quadas-C is an adaptation of Quadas-2 for comparative diagnostic test accuracy studies [26] and investigates the same domains as Quadas-2 (patient selection, index test, reference standard and flow and timing).

It is also possible to use QUIPS (Quality In Pronosis Studies) and PROBAST (Prediction model Risk Of Bias ASsessment Tool) for assessing the risk of bias and applicability of diagnostic and prognostic prediction model studies. QUAPAS has been recently proposed [27]. It is based on Quadas-2, QUIPS and PROBAST.

If studies address the question of test-treatment strategies on outcomes (such as morbidity or mortality) other tools could be more appropriate (like RoB 2 for assessing risk of bias in randomized trials and ROBINS-I for assessing risk of bias in non-randomized studies of interventions) [28, 29].

Generating new data

In the majority of innovative devices, existing data will not be sufficient. Manufacturers or researchers have to create their own datasets and perform evaluation tests (Figures 2, 4). Even if analytical tests can start with laboratory calibrated artificial samples, it is essential to confront the device to a real situation in which samples come from patients or volunteers.

The IVDR identifies three different types of clinical performance studies with increasing stringency: retrospective studies with leftover specimens, prospective sample studies, and interventional studies (Figure 1C).

Studies with leftover specimens do not require authorization from the competent authorities but do require sponsoring (an ethical and data protection framework for individuals). Studies with prospective samples (without risk for the subjects and non-interventional) add the constraint of obtaining authorization from the competent authorities. Finally, interventional studies or studies with potential risks for subjects have extra obligations for sponsors and considerations to ensure the safety of individuals undergoing research. Only this category requires reporting to the European database Eudamed. They all start by writing a Performance Study Plan (Figure 1B) and end by writing the performance study report (Figure 1D).

Sometimes, guidance exists. For example, an effort has been made by the European Union (commission implementing regulation (EU) 2022/1107) to define test standards for class D IVD-MDs [30] like those intended for detection of blood group antigens in blood group systems for example. Other documents are available on a disease-by-disease basis. For example, MDCG 2021-21 defines performance evaluation of SARS-CoV-2 in vitro diagnostic medical devices [7] or instructions given by the WHO [31].

The GHTF/SG5/N8:2012 is a good tool with a decision tree to help manufacturers or researchers to choose a study design [32]. It highlights that most of the studies for IVD-MDs are observational and describes different possible designs: cross-sectional studies (single time-point design), longitudinal studies (the same analyte measured over a period), prospective or retrospective studies. The choice between different designs depends on various considerations such as study objectives, intended use, targeted analytical performance (precision, accuracy and metrological traceability, linearity …), clinical characteristics (diagnosis sensitivity/specificity) and test results. For example, if the test results are continuous like a concentration, the concordance between new test and gold standard method can be investigated through the Bland-Altman method [33]. If results are categorical or dichotomous, the κ statistics can be used [34].

One other important point is the number of samples. Indeed, an oversized sample will lead to a loss of time and money for a potentially very limited precision gain. An undersized sample will not lead to a relevant conclusion. Stergiou et al. illustrate these effects in blood pressure measurements. They showed that a study with sample size of n=35 is adequate for a high accuracy device (<14% chance of failure) but inadequate for a moderate accuracy device (28% failure probability). A sample size of n=80 would be adequate for a moderate accuracy device (18%). Increasing the number of patients would not be relevant since the gain would be only 1% for 10 additional patients (n=90, 17%) [35].

Sammut-Poxell et al. made simulation study to examine the effect of sample size on specificity and sensitivity of COVID-19 diagnostic tests [36]. They showed that the probability of failure by testing only 30 negative samples was 38.9%. By increasing the number of negative cases to 250, the probability of failure is reduced to 1.9%. For sensitivity, using 30 positive cases leads to a 10.9% probability of failure compared to 1.7% when testing 150 positive cases. They also studied impact of prevalence to calculate sample size.

The main studied parameters of clinical performance are diagnostic sensitivity and specificity, areas under the curve (ROC curve), positive or negative predictive values, or positive and negative likelihood ratio. Some formulae to calculate the needed sample size based on the precision around these parameters exist [37].

Evaluation performance report and follow-up

The final performance evaluation report compiles the scientific validity, analytical performance and clinical performance reports of the device (Figures 1E and 2, 5). It allows the reviewer judging the safety and effectiveness of the IVD-MD. However, the performance evaluation process does not stop here. Post-marketing surveillance and post-market performance follow-up must allow a long-term control of the safety of the IVD-MD and its performance in real life. Monitoring and updating should also allow identifying any misuse or off-label use of the IVD-MD. For example, cross-reactivity with macroprolactin is a cause of positive interference in prolactin immunoassays but the instructions for use are not sufficient to deal with. This problem would be solved if manufacturers complied with IVDR, developing a specific test for the 23 kDa monomeric form of prolactin or at least properly informing users and explain how to detect or mitigate this interference [38].

It is therefore essential that IVD manufacturers remedy medical tests that are not sufficiently appropriate for clinical use, such as prolactin immunoassays, and seek partnerships with end-user representative organizations, such as the Scientific Division of the International Federation of Clinical Chemistry.

Conclusions

The IVDR reinforces the notions of traceability in terms of safety and performance of in vitro devices. This leads to an increase of the documentation to be produced and induces additional administrative constraints. Since the time required to obtain the necessary authorizations can be long (without any guarantee of a favorable outcome), this difficult and time-consuming work should be anticipated.

For existing devices, peer-reviewed literature or published experience with routine diagnostic tests can be an important source of clinical performance data. This is, for existing tests, sufficient in many cases. Review of scientific literature should follow established guidance, like PRISMA for systematic review [24, 25]. Specific tools could be used for assessment of diagnostic accuracy studies [26] or comparative diagnostic test accuracy studies [27]. For others, analytical performance tests should be performed in the same way as before (sensitivity, specificity, accuracy, linearity, etc.). For almost all, NBs also expect data from clinical performance studies with leftover specimens, prospective samples and/or from interventional studies. Study design depends, among others, on objectives, expected performances and intended use. GHTF/SG5/N8:2012 provides different designs and examples [33]. It is also crucial to define the sample size based on predefined clinical criteria such as area under the curve, positive or negative predictive value, or positive and negative likelihood ratio and prevalence [36, 37].

To summarize, a single approach does not exist. The roadmap proposed in this paper, inspired by articles from Lord et al. [19] and from Leeflang and Allerberger [21] aims at helping researchers, manufacturers and laboratories staff asking the relevant questions related to the documents requested in the IVDR (scientific validity report, analytical performance report, clinical performance report). A large number of documents and guidelines describing the IVDR roll out and how to conduct performance studies exist. In this paper, we provide key references which, we believe, will help concerned developers to easily find their way in this complex regulatory environment which is still under construction.


Corresponding author: Mme Karine Charrière, PhD, Centre d’Investigation Clinique, Centre Hospitalier Universitaire de Besançon, INSERM CIC 1431, BioInnovation – 4 Rue Charles Bried, 25030, F-25000 Besançon, France, E-mail:

  1. Research funding: None declared.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Not applicable.

  5. Ethical approval: The local Institutional Review Board deemed the study exempt from review.

References

1. The European Parliament. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU (Text with EEA relevance) [Internet]. OJ L; 2017. Available from: http://data.europa.eu/eli/reg/2017/746/oj/eng.Search in Google Scholar

2. The European Parliament. Directive 98/79/EC of the european parliament and of the council of 27 October 1998 on in vitro diagnostic medical devices [Internet]. OJ L; 1998. Available from: http://data.europa.eu/eli/dir/1998/79/oj/eng.Search in Google Scholar

3. The European Parliament. Regulation (EU) 2022/112 of the european parliament and of the council of 25 January 2022 amending regulation (EU) 2017/746 as regards transitional provisions for certain in vitro diagnostic medical devices and the deferred application of conditions for in-house devices (text with EEA relevance) [Internet]. OJ L; 2022. Available from: http://data.europa.eu/eli/reg/2022/112/oj/eng.Search in Google Scholar

4. MedTech, Europe. Analysing the availability of IVDs in May 2022 [Internet - cited 2022 Aug 24]; 2022. Available from: https://www.medtecheurope.org/resource-library/medtech-europe-survey-report-analysing-the-availability-of-in-vitro-diagnostic-medical-devices-ivds-in-may-2022-when-the-new-eu-ivd-regulation-applies/.Search in Google Scholar

5. European Commission. New Approach Notified and Designated Organisations. [Internet - cited 2022 Aug 18]; 2022. Available from: https://ec.europa.eu/growth/tools-databases/nando/index.cfm?fuseaction=directive.notifiedbody&dir_id=35.Search in Google Scholar

6. Ries, F. Question parlementaire | Manque d’organismes notifiés à l’aube de la mise en application du règlement relatif aux dispositifs médicaux de diagnostic in vitro | P-001914/2022 | Parlement européen [Internet]; 2022. Available from: https://www.europarl.europa.eu/doceo/document/P-9-2022-001914_FR.html [Accessed 4 Jan 2023].Search in Google Scholar

7. The Medical Device Coordination Group. Guidance - MDCG endorsed documents and other guidance [Internet - cited 2022 Aug 22]; 2022. Available from: https://health.ec.europa.eu/medical-devices-sector/new-regulations/guidance-mdcg-endorsed-documents-and-other-guidance_en.Search in Google Scholar

8. The European Parliament. EUR-Lex - 32022R0112 - EN - EUR-Lex [Internet - cited 2023 Feb 4]; 2022. Available from: https://eur-lex.europa.eu/eli/reg/2022/112/oj/eng.Search in Google Scholar

9. Cobbaert, C, Capoluongo, ED, Vanstapel, FJLA, Bossuyt, PMM, Bhattoa, HP, Nissen, PH, et al.. Implementation of the new EU IVD regulation – urgent initiatives are needed to avert impending crisis. Clin Chem Lab Med 2022;60:33–43. https://doi.org/10.1515/cclm-2021-0975.Search in Google Scholar PubMed

10. International Organization for Standardization. ISO 15189:2022 [Internet - cited 2023 Feb 4]. Available from: https://www.iso.org/fr/standard/76677.html.Search in Google Scholar

11. The International Electrotechnical Commission. IEC 62304:2006+AMD1:2015 CSV | IEC Webstore | cyber security, smart city [Internet - cited 2023 Feb 4]. Available from: https://webstore.iec.ch/publication/22794.Search in Google Scholar

12. van Deutekom, HWM, Haitjema, S. Recommendations for IVDR compliant in-house software development in clinical practice: a how-to paper with three use cases. Clin Chem Lab Med 2022;60:982–8. https://doi.org/10.1515/cclm-2022-0278.Search in Google Scholar PubMed

13. MedTech, Europe. IVD Regulation - Flowchart - Fact Sheets [Internet - cited 2022 Aug 22]; 2017. Available from: https://www.medtecheurope.org/resource-library/ivd-regulation-flowchart/.Search in Google Scholar

14. MedTech, Europe. Clinical Evidence Requirements for CE certification [Internet - cited 2022 Aug 22]; 2022. Available from: https://www.medtecheurope.org/resource-library/clinical-evidence-requirements-for-ce-certification-under-the-in-vitro-diagnostic-regulation-in-the-european-union/.Search in Google Scholar

15. Horvath, AR, Lord, SJ, StJohn, A, Sandberg, S, Cobbaert, CM, Lorenz, S, et al.. From biomarkers to medical tests: the changing landscape of test evaluation. Clin Chim Acta 2014;427:49–57. https://doi.org/10.1016/j.cca.2013.09.018.Search in Google Scholar PubMed

16. Lesko, LJ, Zineh, I, Huang, SM. What is clinical utility and why should we care? Clin Pharmacol Ther 2010;88:729–33. https://doi.org/10.1038/clpt.2010.229.Search in Google Scholar PubMed

17. European commission. Public health: commission proposes a progressive roll-out of the new In Vitro Diagnostic Medical Devices Regulation [Internet - cited 2022 Aug 18]; 2021. Available from: https://ec.europa.eu/commission/presscorner/detail/en/ip_21_5209.Search in Google Scholar

18. International Organization for Standardization. ISO 20916:2019 [Internet - cited 2023 Feb 4]. Available from: https://www.iso.org/standard/69455.html.Search in Google Scholar

19. Lord, SJ, St John, A, Bossuyt, PM, Sandberg, S, Monaghan, PJ, O’Kane, M, et al.. Setting clinical performance specifications to develop and evaluate biomarkers for clinical use. Ann Clin Biochem 2019;56:527–35. https://doi.org/10.1177/0004563219842265.Search in Google Scholar PubMed

20. International Organization for Standardization. ISO 17511:2020 [Internet - cited 2023 Feb 15]. Available from: https://www.iso.org/fr/standard/69984.html Search in Google Scholar

21. Leeflang, MMG, Allerberger, F. How to: evaluate a diagnostic test. Clin Microbiol Infect 2019;25:54–9. https://doi.org/10.1016/j.cmi.2018.06.011.Search in Google Scholar PubMed

22. Nishikawa-Pacher, A. Research questions with PICO: a universal mnemonic. Publications 2022;10:21. https://doi.org/10.3390/publications10030021.Search in Google Scholar

23. Page, MJ, Moher, D, Bossuyt, PM, Boutron, I, Hoffmann, TC, Mulrow, CD, et al.. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021;372:n160. https://doi.org/10.1136/bmj.n160.Search in Google Scholar PubMed PubMed Central

24. Page, MJ, McKenzie, JE, Bossuyt, PM, Boutron, I, Hoffmann, TC, Mulrow, CD, et al.. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71.Search in Google Scholar PubMed PubMed Central

25. Whiting, PF, Rutjes, AWS, Westwood, ME, Mallett, S, Deeks, JJ, Reitsma, JB, et al.. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529–36. https://doi.org/10.7326/0003-4819-155-8-201110180-00009.Search in Google Scholar PubMed

26. Yang, B, Mallett, S, Takwoingi, Y, Davenport, CF, Hyde, CJ, Whiting, PF, et al.. QUADAS-C: a tool for assessing risk of bias in comparative diagnostic accuracy studies. Ann Intern Med 2021;174:1592–9. https://doi.org/10.7326/m21-2234.Search in Google Scholar PubMed

27. Lee, J, Mulder, F, Leeflang, M, Wolff, R, Whiting, P, Bossuyt, PM. QUAPAS: an adaptation of the QUADAS-2 tool to assess prognostic accuracy studies. Ann Intern Med 2022;175:1010–8. https://doi.org/10.7326/m22-0276.Search in Google Scholar PubMed

28. Sterne, JAC, Savović, J, Page, MJ, Elbers, RG, Blencowe, NS, Boutron, I, et al.. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;366:l4898. https://doi.org/10.1136/bmj.l4898.Search in Google Scholar PubMed

29. Sterne, JA, Hernán, MA, Reeves, BC, Savović, J, Berkman, ND, Viswanathan, M, et al.. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919. https://doi.org/10.1136/bmj.i4919.Search in Google Scholar PubMed PubMed Central

30. European commissions. Commission implementing regulation (EU) 2022/1107 of 4 July 2022 laying down common specifications for certain class D in vitro diagnostic medical devices in accordance with regulation (EU) 2017/746 of the european parliament and of the council (text with EEA relevance) [Internet]. OJ L; 2022. Available from: http://data.europa.eu/eli/reg_impl/2022/1107/oj/eng.Search in Google Scholar

31. World Health organization. Performance Evaluation. Prequalification of Medical Products (IVDs, Medicines, Vaccines and Immunization Devices, Vector Control). 2020 [Internet - cited 2023 Jan 17]. Available from: https://extranet.who.int/pqweb/vitro-diagnostics/performance-evaluation.Search in Google Scholar

32. Global Harmonization Task Force Study Group 5. International Medical Device Regulators Forum. Clinical Safety/Performance [Internet - cited 2023 Jan 19]; 2012. Available from: https://www.imdrf.org/documents/ghtf-final-documents/ghtf-study-group-5-clinical-safetyperformance.Search in Google Scholar

33. Martin Bland, J, Altman, DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;327:307–10. https://doi.org/10.1016/s0140-6736(86)90837-8.Search in Google Scholar

34. Cohen, J. A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46. https://doi.org/10.1177/001316446002000104.Search in Google Scholar

35. Stergiou, GS, Alpert, B, Mieke, S, Asmar, R, Atkins, N, Eckert, S, et al.. A universal standard for the validation of blood pressure measuring devices: association for the advancement of medical instrumentation/European society of hypertension/international organization for standardization (AAMI/ESH/ISO) collaboration statement. Hypertens Dallas Tex. 2018;71:368–74, https://doi.org/10.1161/hypertensionaha.117.10237.Search in Google Scholar PubMed

36. Sammut-Powell, C, Reynard, C, Allen, J, McDermott, J, Braybrook, J, Parisi, R, et al.. Examining the effect of evaluation sample size on the sensitivity and specificity of COVID-19 diagnostic tests in practice: a simulation study. Diagn Prognostic Res 2022;6:12. https://doi.org/10.1186/s41512-021-00116-4.Search in Google Scholar PubMed PubMed Central

37. Hajian-Tilaki, K. Sample size estimation in diagnostic test studies of biomedical informatics. J Biomed Inf 2014;48:193–204. https://doi.org/10.1016/j.jbi.2014.02.013.Search in Google Scholar PubMed

38. Fahie-Wilson, MN, Cobbaert, CM, Horvath, AR, Smith, TP. Interference by macroprolactin in assays for prolactin: will the in Vitro Diagnostics Regulation lead to a solution at last? Clin Chem Lab Med 2022;60:1350–5. https://doi.org/10.1515/cclm-2022-0460.Search in Google Scholar PubMed


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/cclm-2022-1252).


Received: 2022-12-09
Accepted: 2023-02-27
Published Online: 2023-03-15
Published in Print: 2023-06-27

© 2023 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 5.3.2024 from https://www.degruyter.com/document/doi/10.1515/cclm-2022-1252/html
Scroll to top button