Collaborative Trial Tests
Collaborative Trial Tests for Method Validation: Lessons to be Learned
This article describes the lessons that were learned in the course of a project entitled “Collaborative trial study for the determination of aflatoxin B1 in ground products of peanuts and corn by immuno-affinity clean-up and thin-layer chromatography,” which was co-funded by IUPAC’s Division on Chemistry and the Environment (#1999-010-1-600). The project, which was carried out between 2000 and 2003, aimed to obtain a robust and simple validated method capable of quantifying aflatoxin B1 in corn and peanuts at levels of 1-10 ng/g using thin-layer chromatography (TLC). The project did not achieve its objective, but it did provide some valuable lessons that the authors felt were important to share with the IUPAC community. The article describes the background and results of the collaborative study, as well as lessons learned and conclusions.
by E. Anklam and J. Stroka
Mycotoxins are food contaminants that are frequently found in a variety of different food products. Among these toxins, the aflatoxins are the most well known. Aflatoxins were identified in food and feed contaminants in the 1960s and can occur in many valuable products derived from plant origin, such as peanuts, pistachios, figs, paprika, and corn, to name the most relevant. One aspect that all these products have in common is that they are produced mainly in regions with warmer climates. Very often, countries in these regions (e.g., Africa, South America and Asia) are also economically disadvantaged and a large fraction of the produce is grown for export to industrial states.
Importing countries, among which are the Member States of the European Union, have legislative limits for some of these contaminants in order to protect the health of consumers and animals. In EU countries, these limits follow the ALARA principle (As Low As Reasonably Achievable) in order to ensure high-quality imports. Currently, the most effective approach to keeping mycotoxins from entering the food market is the regular monitoring of them at all stages of production. As a result, reliable and validated analytical methods must be available to ensure that the producer can trace any contamination prior to shipping or processing. In the worst cases, such contamination can result in the rejection of goods at the point of entry into another market.
In general, the analysis of mycotoxins, and in particular the analysis of aflatoxins, is a challenging task, as a suitable method should be easy to perform, robust in its application, and reliable in its results. Furthermore, all parties should agree on the analytical results obtained from such a method. This has been recognized by the European Commission, which has set performance criteria in EU Directives for analytical methods, rather than so called “reference methods.”
The validation of a method for the analysis of aflatoxin B1 at a level around 2 ng/g has recently been successfully carried out using high-performance liquid chromatography (HPLC) in combination with a very powerful clean-up and concentration step involving immuno-affinity.1 This well-established clean-up procedure was expanded in order to encompass successfully the determination of aflatoxin M1 in milk by thin layer chromatography (TLC).2 TLC is a relatively simple but powerful chromatographic method that is still widely used in control laboratories, but has been replaced in the food sector to a certain extent by HPLC. However, HPLC requires more expensive, high-quality solvents and requires advanced instrument support and reliable electrical power because short power shutdowns or voltage fluctuations will result in analysis failure. With TLC, standard solvents can be used in most cases and electricity or maintenance problems play only a marginal role. As a result, and because this technique is much cheaper, TLC is still commonly used in economically disadvantaged areas.
Robust and reliable analytical methods are necessary to control compliance with legislative limits. For aflatoxins, the limits have been set at relatively low levels in Europe. As many food commodities at risk of contamination with aflatoxins are produced in developing countries, the limits have to control the food prior to export. Due to the fact that TLC is widely applied in many countries, this IUPAC project aimed to validate a TLC method involving mainly developing countries. The method and in-house validation results have been successfully tested with simple and economic alternatives to commercial densitometers developed in our laboratory, and as described in the literature.3-4
Collaborative Study Design
A total of 27 laboratories from 16 countries participated in the collaborative trial study for the determination of aflatoxin B1 in peanut and corn by TLC. The countries of origin of the participating laboratories were Argentina, Australia, Brazil, Bulgaria, Cuba, Egypt, France, Hungary, Indonesia, Iran, Latvia, Malaysia, Philippines, Switzerland, Thailand, and Uruguay. The final collaborative trial was performed in 2002.
A multi-media show, to be used in combination with the method description, was produced on CD-ROM and sent to study participants to show them the important and critical steps. Homogenous test materials of corn and peanuts were produced by the coordinator in 50 g units. Each matrix consisted of blind duplicate samples for blank and naturally contaminated materials of aflatoxin B1 ranging from 1 ng/g to 10 ng/g. The number of samples and the conduct of the trial were in compliance with the IUPAC Harmonized Protocol.5 Each participating laboratory received all coded samples and the necessary consumables (e.g., immuno-affinity columns, standards) and some specific glassware.
Results of the Collaborative Trial
The collaborative trial study was finalized at the beginning of 2003. A total of 17 laboratories, out of the 27 that received the materials, submitted results. One laboratory was not able to detect any aflatoxin in the samples and those results were therefore excluded from further data evaluation. This resulted in 16 laboratories submitting analytical data. The analysts were asked to comment on several questions concerning the procedure.
Based on the submitted results it can be concluded that the participants could identify different levels of aflatoxin B1 in the samples. However, the statistical analysis of the results from the collaborative trial and the comments made by the participants led to the conclusion that the method may not be “fit for purpose” at this stage when compared with the requirements given by European legislation. However, it must be stressed that most of the results were obtained by individual visual detection (not using densitometric scanners). Figures of repeatability and reproducibility are listed in the following table, showing clearly that there are problems with the analysis of aflatoxin B1 at these low levels.
|RSD r for corn||127 %|
|RSD r for peanut||76 %||33 %|
|139 %||125 %||80 %|
|RSD R for peanut||145 %||91 %||81 %|
|Relative Standard Deviation (RSD) of repeatability (r) or reproducibility (R) obtained from the collections of results.|
Although communication was frequent between participants and the coordinator of the collaborative trial, and considerable effort has been put into this project from all sides, it appears that at this stage the proposed TLC approach is not applicable in all laboratories.
Several critical issues have been identified during this trial, which are mainly due to logistics. For example, problems occurred due to the late arrival of parcels in some cases, but also due to communication problems, as some participants did not respond after parcels arrived (as asked by the coordinator). This also concerns those laboratories of the 27 that did not submit results at all. In fact, it appeared that the majority of participants who had been involved previously in international projects and for whom personal contacts (face to face) existed were the first ones to supply results and to successfully perform the analysis.
Most problems were caused by the participants’ lack of familiarity with the method and the calculations used. Since most of the participants analyzed the samples by visual inspection, there was, unfortunately, too little data available to evaluate the method by densitometry, as was originally envisaged. This explains to a certain extent why the method proved unsatisfactory in the lower contamination ranges. To conclude, visual inspection at these low levels is not to be recommended, although it is thought that instrumental quantification might be the solution for the determination of aflatoxin B1 by TLC at these levels.
Because commercially available densitometers may not always be affordable for laboratories in economically deprived areas, the coordinators also aimed to supply a solution for this by designing cheap alternative instruments that show similar performance.4 However, these instruments are still at the prototype stage at this time, but are being further developed and will be used in a future validation study, it is envisaged, using the same TLC method.
Lessons Learned—Recommendations for Future
Collaborative Trial Studies
Some of the comments from and results of communication with selected laboratories led to recommendations for future collaborative studies of this kind:
- An appropriate selection of laboratories must be made. This is more significant the greater the geographical distance of all involved parties. On the one hand, it is helpful to collaborate with known partners, while it is also critical to exclude laboratories that are new to the field.
- A training workshop involving “hands-on” training—held either at the beginning of the project or after a pre-trial, so that results could be discussed—would certainly have facilitated communication regarding problems. However, it must be considered that, although this is the most appropriate strategy in such cases, the additional costs are high. Unfortunately “long-distance communication” by email and phone alone do not allow any “hands-on” problem-solving solutions (e.g., training on use of the method).
- After such a workshop the participants should implement the method at their own laboratories. Furthermore, it is crucial that participants have a clear understanding of the “philosophy” behind each step of the method right from the beginning. This holds especially true when the method includes relatively new principles such as immuno-affinity clean up or other new techniques. In addition, the pit-falls and critical steps could be discussed and demonstrated. This can be accomplished by drafting an “extended” method description, as is usually done for a collaborative trial.
- A questionnaire on problems faced by the laboratories should be sent out for the trial and pre-trial in order to learn about the laboratories’ experiences with a particular method. This is very important, as participants must be familiar with the method in order to participate in a collaborative trial.
- A formal agreement with the official heads of laboratories of unknown partners might help to stress the importance of such method validation projects.
- In order to keep the workload for participants low, only one matrix should be handled at one time. Unlike previous collaborative trials that were successfully carried out using HPLC, the effort of the TLC method must not be underestimated, as laboratories using HPLC often have sophisticated laboratory infrastructure that might not be available in all cases. Even though this would mean a greater logistical and organizational workload for the coordinator, it would likely optimize the outcome.
1. J. Stroka et al., JAOAC Int.83, 320 (2000); J. Stroka et al., JAOAC Int. 84, 1116 (2001); J. Stroka et al., JAOAC Int.86, 1179 (2003)
2. S. Dragacci et al., IDF/IUPAC/IAEA Joint project report on the validation of an analytical method to determine the content of aflatoxin M1 in milk and milk powder by thin layer chromatography after an immunoaffinity clean-up (2001)
3. J. Stroka et al., J. Chromatogr. A, 904, 251 (2000)
4. J. Stroka et al., J. Chromatogr. A, 904, 263 (2000); Stroka et al., J. Planar Chromatogr.14, 109 (2001)
5. IUPAC Technical Report: “Protocol for the design, conduct and interpretation of method-performance studies”, prepared for publication by W. Horwitz, Pure Appl. Chem. 67, 331 (1995)
E. Anklam <firstname.lastname@example.org> and J. Stroka are both from the European Commission's Institute for Reference Materials and Measurements in Belgium. E.A. is a member of the IUPAC Chemistry and the Environment Division.
Page last modified 18 July 2004.
Copyright © 2003-2004 International Union of Pure and Applied Chemistry.
Questions regarding the website, please contact email@example.com