Evaluating and using innovative technologies: a lesson from Theranos?

Rigorous standards have been set for the evaluation and use of new drugs, unlike the use of diagnostic tests in a clinical setting. A large body of evidence demonstrates the current limitations and the lack of harmonized requirements for evaluating and adopting new diagnostic assays and innovative technologies [1]. Moreover, regulations in this sphere differ from country to country. The European Directive, CE marking (European In Vitro Diagnostic Medical Devices Directive 98/79/EC, Conformité Européenne) simply certifies that a product has met European Union health, safety, and environmental standards, whereas the US procedure – which is more complex and involves the US Food and Drug Administration (FDA) among other institutions – is governed by the Clinical Laboratory Improvement Amendments (CLIA) of 1988 [2]. Different regulatory procedures are implemented in the UK, Canada, Australia and elsewhere [3]. However, the main issue is not the lack of internationally consolidated harmonized requirements, but the dire lack of any assurance that any of the innovative technologies and procedures for diagnostic purposes guarantee benefit – rather than risk – for patient care. As stressed by Joshua Sharfstein in a recent paper on the issuing of an FDA draft guidance document containing a framework for the application of agency standards for quality, safety, and validity to laboratory-developed tests [4], “innovation is not just novelty; it is novelty that works” [5]. In the last few decades, the extraordinary advances made in our understanding of the molecular and biochemical bases of human diseases have paved the way for the introduction into the clinical practice of a new generation of laboratory tests for earlier and more accurate diagnoses, as well as for the identification of relevant risk factors, which should ultimately allow the provision of “personalized medicine” and better clinical outcomes. Even more promising is the introduction of “omics” (e.g., genomics, proteomics, transcriptomics, and metabolomics), and the translation of “omics” into clinical practice, which should help laboratory professionals make their role and value more visible [6]. On the basis of this prediction, V. Roy emphasized the crucial role of laboratory tests in modern medicine, stating that “in the era of molecular medicine, more and more diagnoses will be made by laboratory tests in asymptomatic patients. The goal is to make the diagnosis before clinical signs or symptoms resulting from organ damage become evident” [7]. The medical community and policy makers, who have underestimated the importance of the above issue, have been slow to grasp the paradigmatic change occurring in laboratory tests. Only recently has this aspect received the attention it deserves. In commenting on an article concerning genetic testing, Tom Walley underlined the evidence that “recent technological developments have created a new generation of laboratory diagnostics, which promise to provide better ways of detecting diseases and monitoring response to treatment” [8], and stressed the need for a careful evaluation of the tests themselves, starting from the purpose and the clinical context in which they are to be used, and the adoption of the ACCE framework, based on 1) analytical validity, 2) clinical validity, 3) clinical usefulness, and 4) any ethical, social or legal implications [8]. Given the importance of laboratory information in modern medicine, and the need to assure quality and safety in patient care, the evaluation of old and new technologies in laboratory medicine continues to be widely debated – and increasingly controversial.

This issue of the journal contains a paper by the well-known scientist, Eleftherios Diamandis, that is the first scientific article exploring a new and intriguing diagnostic technology provided by a commercial organization, Theranos (https://www.theranos.com/) [9], as an example of how the discovery of disruptive technologies may change the scenario of laboratory medicine in the near future. Elizabeth Holmes, founder and CEO of Theranos, has been included in Forbes’ best “30 Under 30” group, and has been listed as one of the youngest female billionaires in the US. Diamandis raises serious concerns regarding the Theranos technology, maintaining that the system has not been independently evaluated, and as none of its results have been appeared in the literature, it cannot be compared with conventional technologies. Nor has it provided evidence on the trueness, reproducibility, specificity, and long-term robustness of the innovative technology used; the finger prick process presents challenges as its commutability and correlation with traditional veni-puncture has not been verified. In addition, Theranos supports self-testing and self-interpretation of laboratory results. This tricky approach could lead to diagnostic errors and patient harm [9]. Laboratory tests are only a piece in a complete diagnostic workup, starting from requesting an appropriate test (the right test at the right time for the right patient) and leading through to the right result accompanied by the right laboratory report to enable the right interpretation and the right utilization for a sound diagnosis and effective therapeutic intervention. The list of Theranos available tests includes some obsolete examinations (e.g., total T3 and total T4), as well as wrong terminologies (e.g., microalbumin), and the emphasis on running more than 30 tests from a drop of blood does not support the current search for appropriateness of test request. Soon after the paper by Diamandis had been accepted by the journal for publication, a report appeared on Yahoo. Written by the journalist Kevin Loria, it raised similar concerns (http://finance.yahoo.com/news/scientists-skeptical-secret-blood-test-100000161.html). Loria emphasizes the skepticism of many scientists, particularly as “there is nothing to really look at, to read, to react to” as stated by the President of the American Association for Clinical Chemistry (AACC). Theranos has bypassed the traditional process of peer review or publishing in peer-reviewed journals or having peer laboratories evaluate their product, as mentioned by Jerry Yeo and other scientists. In an interview for Fortune in 2014, Elizabeth Holmes described the Theranos approach as “lab-on-a-chip technology” based on microfluidics and, on searching the US Patent Office for “Theranos”, this term appears in nine of the 31 patents. Doubtless microfluidics is cutting-edge technology but it is not exclusive to Theranos, and other potential roadblocks are described in the Kevin Loria report. The Theranos experience should thus highlight the ever pressing need to improve upon the evaluation and validation process of old and new diagnostic technologies in order to really assure quality and patient safety – and to closely link the request for a laboratory test to the appropriate clinical question and setting. We trust that the paper by Diamandis will heighten awareness in the scientific community of this increasingly relevant issue, and we will welcome further contributions to better understand and promote improvements in evaluating and using innovative technologies in laboratory medicine.