Evaluation of the performance of sysmex XN-3100 automated hematology analyzer regarding the sysmex XE-2100 and microscopic examination

Objectives: We performed a verification study of the Sysmex XN-3100 hematology analyzer in comparison with the XE-2100 according to the guidelines of the Clinical and Laboratory Standards Institute (CLSI) and the International Council for Standardization in Hematology (ICSH). Materials and methods: Blood samples and quality control materials were used for precision. For comparison, we used the current XE-2100 as the comparative method and analyzed 540 blood samples. The Passing-Bablok and Bland-Altman tests were performed according to the CLSI EP09-A3 and a carryover study was performed according to the CLSI H26-A2 guidelines. The flagging performance of the two analyzers was compared, using two experienced laboratory technicians as the reference method. Results: The Sysmex XN-3100 demonstrated high levels of precision for most parameters. For the comparison analysis, all parameters, except for MCHC, monocytes and basophils were within the systematic error limits of desirable biological variability criterion (SeDBV). The carryover was less than 0.4% for all parameters. The flagging performance of the XN-3100 was satisfactory and the overall efficiency was high. Conclusions: The XN-3100 not only showed a strong correlation and agreement with the XE-2100 but also displayed a comparable analytical sensitivity, and increased specificity, which may result in an improved turnaround time and throughpu. (CellaVision, Lund, Sweden) digital microscope. The inter-rater reliability values (Cohen ’ s kappa coefficient) for blasts, abnormal lymphocytes, atypical lymphocytes, and immature granulocytes were 0.820, 0.827, 0.930, and 0.895, respectively. The sensitivity and specificity of the flagging performance were also determined in the verification process.

The WBC differential assessment includes two-or three-part differential count, including mononuclear cells or agranulocytes (monocytes and lymphocytes) and granulocytes (neutrophil, eosinophil, and basophil count). Occasionally, an additional WBC count is added for immature granulocytes. Some AHAs also assess nucleated red blood cells (NRBCs) and reticulocytes as a standard parameter of the CBC [2,3].
Before setting up a new AHA in a clinical diagnostic laboratory, the Clinical and Laboratory Standards Institute (CLSI) [4] and the International Council for Standardization in Hematology (ICSH) [5] stipulate that a validation should be carried out by the manufacturer under optimum circumstances, followed by verification by the clinical laboratory to ensure its analytical performance. Moreover, the ISO 15189 [6] recommends a verification process for clinical laboratories prior to installation. However, these guidelines define neither the limits of acceptability for diverse parameters nor a minimal range interval for the reliability of parameters. Thus, the sample size for the verification study varies based on the laboratory data available [1].
The verification involves a performance evaluation of the AHA on the accuracy, precision, carryover, reportable range of test results and reference intervals (normal ranges), the limit of blank, detection and quantitation, clinically reportable, and analytic measuring intervals [6]. However, if the same manufacturer has provided the new AHA with the analyzer already in use, a less comprehensive verification can be performed [7].
The Sysmex XN-3100 (Sysmex Corporation, Kobe, Japan) is a hematology analyzer that has been improved to ameliorate the accuracy of samples with very low blood cell counts and is equipped with a more sophisticated flagging function. The measurements of red cell count and impedance of platelets are similar to those of the Sysmex XE-2100 hematology analyzers. However, the XN series offers new channels that enable enhanced flagging performancethese include a white-cell differential channel (WDF) for the determination of leukocyte subtypes (lymphocytes, monocytes, neutrophils, basophils, eosinophils, and immature granulocytes), a white-cellnucleated channel (WNR) for nucleated red blood cells (NRBCs, erythroblasts), a white cell precursor channel (WPC) for reflex testing when a blast or abnormal lymphocyte flag is detected, and a fluorescent platelet channel (PLT-F) for the optic fluorescence platelet count [7,8]. As all samples are tested for the NRBC count using the WNR channel, there is no need to repeat the NRBC count for suspicious samples on the NRBC channel as performed on the Sysmex XE-2100. The WNR channel is assigned to NRBC counts, which utilizes the light scattering and fluorescence methods instead of the impedance method. The XN series is not equipped with a separate basophil channel as on the XE series because it is integrated into the new WDF channel, which also undertakes an optimized differentiation of lymphocytes and monocytes. If a sample displays a blast/abnormal lymphocyte flag, then the WPC channel is activated for reflex testing. An additional low-WBC mode is available on the XN series for the detection of WBC counts of 0.5 × 10 9 /L or less. At this level, a reflex analysis is performed for the leucocyte differentiation [9,10].
As the XN series have newer channels than the XE series, we performed a basic verification study to check the performance of these channels before implementing it in our laboratory. First, we aimed to assess certain analytical characteristics of the Sysmex XN-3100 (precision, accuracy, and carryover) and then determine the systematic error between analyzers. Subsequently, we compared the systematic error of each parameter with the upper limits according to the Desirable Biological Variation Database specifications (SeDBV%). Our eventual aim was to check the sensitivity and specificity values of the flags in blood smears through microscopic examination.

Blood samples
A total of 720 fasting blood samples from our daily routine workload were collected into K 2 EDTA-anticoagulated tubes (Vacutainer® 4.0 mL; Becton Dickinson, Plymouth, UK) and processed within 2 h between April and November 2018. Analysis was performed on 0.7 mL of the initial 4 mL blood sample. The samples were drawn from patients with a broad range of diseases to analyze the whole analytic range from low to high counts of blood cells, along with abnormal cells (such as blasts and myeloblasts). We used 10 patients to calculate the within-run precision, five hundred and 40 for the comparison, 10 for the carryover study, and one hundred and 60 to determine the flagging performance.
This study protocol has been reviewed and approved by the Koç University ethics committee (approval No. 2018.316.IRB1.039).

Precision
In the present study, the within-run and between-batch precision analyses were conducted according to the ICSH guidelines [5] to observe the agreement between repeated measurements of a sample.
For the within-run precision (repeatability/reproducibility), samples of 10 patients with high, normal, and low blood cell counts were selected. 10 consecutive measurements were performed for each sample, with all reported parameters in a single run. Considering the between-batch precision analysis, XN-CHECK quality control materials (Sysmex Europe GmbH, Norderstedt, Germany) with high, normal, and low concentrations were studied in duplicate over 20 days.

Comparison study
In our comparison process, we used the current AHA (Sysmex XE-2100) as the comparative method to assess the inaccuracy and the systematic error. We performed the Passing-Bablok and Bland-Altman analyses according to CLSI guidelines [11] to detect the differences between analyzers. The mean % of differences (MD%) obtained by the Bland-Altman plots can be estimated as the systematic error. The upper limits for the systematic error recommended by the SeDBV are displayed in Table 2. We compared the MD% values in the parameters with the upper limit of systematic error according to the SeDBV% to see if this difference is acceptable.
If the new AHA has previously been validated by the manufacturer, the ICSH guidelines recommend including fewer blood samples for the evaluation. In the present study, a total of 540 blood samples were collected in three weeks and analyzed within 2 h after collection, according to ICSH guidelines [5,12]. The blood samples were collected from different sources, including outpatient clinics, emergency services, intensive care units, oncology, and hematology departments, involving normal and abnormal samples in almost equal numbers.

Carryover
We performed our carryover study following the CLSI H26-A2 guideline [4], which covers the assessment of high (H1, H2, H3) and low (L1, L2, L3) samples three times consecutively. First, the samples with high blood count parameters were analyzed, followed by the samples with low counts to observe the influence of the first measured sample.
We assessed the percentage of carryover by the formula: [13,14].

Flagging performance
While the WNR channel of the Sysmex XN-3100 generates flags for NRBC, WBC, and platelet clumps, the WDF channel generates flags for atypical lymphocyte and blast/abnormal lymphocytes. In contrast with the XE series, the Sysmex XN-3100 has an additional WPC channel for reflex testing to differentiate blasts, abnormal/atypical lymphocytes, and immature granulocytes. Q-flag settings were generated, as reported by the specifications of the manufacturer. The default settings of the manufacturer were implemented for reflex testing. The flags created by the Sysmex XN-3100 were "Blasts?"(for blasts), "IG% present"(for metamyelocytes, myelocytes, and promyelocytes), "Left shift?"(for bands), "Abn Lymph?"(for abnormal lymphocytes), "Atyp Lymph?(for atypical lymphocytes) and "NRBC present"(for NRBC) [15].
We aimed to assess the overall efficiency of the flagging performance of the Sysmex XN-3100. We specially selected 160 samples from patients with hematologic malignancies to increase the possibility of detecting flags. Two laboratory technicians, whom we considered to be the reference, evaluated the presence/absence of blasts, abnormal and atypical lymphocytes of blood smears using a CellaVision DM1200 (CellaVision, Lund, Sweden) digital microscope. The inter-rater reliability values (Cohen's kappa coefficient) for blasts, abnormal lymphocytes, atypical lymphocytes, and immature granulocytes were 0.820, 0.827, 0.930, and 0.895, respectively. The sensitivity and specificity of the flagging performance were also determined in the verification process.

Statistical analysis
For the within-run and between-batch precision measurements, we calculated the mean, standard deviation (SD) and coefficient of variation (CV%) for low, medium and high levels of each parameter to compare with the CV% cut-off values of the manufacturer and desirable biological variation specification (DBVs) [7,16].
We performed the Passing-Bablok regression analysis and Bland-Altman tests according to the CLSI document [11] to determine the inaccuracy and the mean difference between the two analyzers. For acceptable performance, the 95% CI of the intercept and slope should include 0.0 and 1.0, respectively [11,17]. Data were processed using the MEDCALC® v10.2.0 software (MedCalc, Ostende, Belgium).

Results
We used the DBVs values published by Perez et al. [7] for both within-run and between-batch comparison.

Comparison with the manufacturer's specifications
All parameters were within the ranges specified by the manufacturer, except for low levels of eosinophils (37.27 vs. <25%).

Comparison with the manufacturer's specifications
Only the CV% of low Hgb exceeded the recommended manufacturer's specifications (1.2 vs. <1%).

Comparison (correlation analyses)
The IC 95% intercept values obtained by the Passing-Bablok regression analysis of RBC, HGB, HCT, MCH, MCHC, RDW-SD, NEUT, and LYPM did not include "0". The results obtained with the Sysmex XN-3100 correlated well with those obtained with the Sysmex XE-2100 ( Table 2). The upper limits for the SeDBV are displayed in Table 2 [18]. The MD% values obtained with Bland-Altman analysis for all parameters, except for monocytes, basophils, and MCHC, were within the systematic error limits.

Carryover
The carryover effect was little or non-existent for all parameters and was ≤0.5% for each parameter, meeting the manufacturer's specifications (The results are provided in the Supplementary data).

Flagging performance
The flagging performance of the Sysmex XN-3100 and XE-2100 for 160 samples with flags was evaluated based on the recommendations of the CLSI protocol H20-A2 [12]. Overall, the Sysmex XN-3100 showed a similar sensitivity with a higher specificity compared to the corresponding values for the Sysmex XE-2100.

Discussion
In the present study, the within-run precision of MCHC, eosinophils, and basophils showed higher CV% values than those of the DBVs. Nevertheless, apart from eosinophils, they showed satisfactory CV% values consistent with the manufacturer's specifications in line with. Perez et al. [7]. In particular, eosinophils, basophils, and immature granulocytes may display high discrepancies due to their very low percentages, leading to inaccuracies even in small variations [7,13].
MD%, mean difference obtained from Bland-Altman plots; SeDBV%, the limit of systematic error according to the desirable biological variability criterion; WBC, white blood cells; RBC, red blood cells; HGB, hemoglobin; HCT, hematocrit; MCV, mean cell volume; MCH, mean corpuscular hemoglobin; MCHC, mean corpuscular hemoglobin concentration; RDW-SD, red cell distribution width; PLT, platelets; NEUT, neutrophils; LYMPH, lymphocytes; MONO, monocytes; EO, eosinophils; BASO, basophils; IG, immature granulocytes; NRBC, nucleated red blood cells; -, unspecified. A ; Bias higher than the limit of systematic error according to the desirable biological variability criterion. In the between-batch precision study, the CV% value for low HGB levels exceeded the manufacturer's specifications, but it was acceptable for the DBVs. MCH and MCHC had higher CV% values for all low, medium, and highquality control samples for the DBVs while having an acceptable CV% value for the manufacturer's specifications. Overall, the Sysmex XN-3100 showed good precision, and the results obtained were concordant with the specifications of the manufacturer and DBVs.
In the Bland-Altman analysis, the difference exceeded the SeDBV only for monocytes, basophils, and MCHC. The systematic error was frequently calculated as positive, showing higher cell counts with the Sysmex XN-3100 compared with the Sysmex XE-2100.
The mean difference (MD%) value for basophils exceeded the SeDBV in line with Inaba et al. (Table 2) [19]. This inconsistency for basophils between the two analyzers may have arisen from either the different channels in which basophils were assessed or their low numbers in peripheral blood. There is no additional channel on the XN series for basophils, and the basophil count is performed on the optimized WDF channel [9].
As for monocytes, the MD% obtained using Bland-Altman analysis was higher than that of the SeDBV, which may be due to the difficulty in the morphologic classification of monocytes. Meintker et al. [20] also reported that the absolute monocyte count might be underestimated in some analyzers.
Concerning IG and NRBC, the high MD% values obtained with the Bland-Altman plots displayed a low agreement, although the pre-specified values of SeDBV specification for these two parameters are lacking.
The criteria that the 95% CI of intercept and slope should include 0.0 and 1.0, respectively, were not met in most parameters (except for MCV, PLT, and EO). If the methods are precise and the 95% CIs for the slope and intercept are very narrow, the y=x equation may not be obtained, but the comparison may be acceptable [16]. In this case, Bland-Altman analysis can be performed as an alternative test. In contrast to regression analysis, the outlier data are displayed on Bland-Altman plots.
In a comparison study of AHAs [16] seven modules of the Sysmex XN series were distributed over three laboratories to compare with the Sysmex XE-2100 AHA. Correlation coefficients analysis and Bland-Altman difference plots were performed to see the limits of agreement. Given that at least 90% of the calculations per parameter and per module must comply with those limits of agreement, and aside from RBC and HCT, all the parameters on each module met these criteria. In addition, all parameters were within the manufacturer's specifications and the biological variability with respect to reproducibility.
We have decided to show the Passing-Bablok and Bland-Altman charts of the most interesting parameters, including HGB, RBC, RDW-SD, EO, and MONO in Figure 1.
The HGB values are randomly scattered above or below 0, showing no systematic difference. The systematic difference between the two analyzers is almost equally positive or negative ( Figure 1A).
As for the RBCs, the distribution tends to be below and above 0 on the left and right sides of the chart respectively, which means that the Sysmex XN-3100 measurements may be smaller than those of Sysmex XE-2100 at low levels or vice versa, which is a negligible error ( Figure 1B). Regarding  RDW-SD, an excellent agreement is noted between the two analyzers for low levels, while measurements with the Sysmex XN-3100 for high levels are greater, which is also an inevitable positive bias ( Figure 1C). The distribution for EO above or below 0 is similar for the two analyzers, being closer to 0 for high levels and further away for low levels, indicating an increased variability at low levels ( Figure 1D). Monocytes showed a low agreement with a high MD% value exceeding the SeDBV ( Figure 1E).
In the current study, we observed little or no carryover for WBC, RBC, HGB, RET, and PLT, all having a range of 0.0-0.36% (Table 3).
Seo et al. [8] evaluated the performance of new channels on the XN series by comparing it with the Sysmex XE-2100 and manual method. In addition to analytical sensitivity rates comparable to the Sysmex XE-2100, the number of abnormal flags was decreased concerning blast, abnormal lymphocyte, and atypical lymphocyte, indicating increased specificity rates. In the present study, we found similar results with Seo et al. (Table 4). The higher specificity rates for all the blast, abnormal lymphocyte, atypical lymphocyte, and immature granulocyte flags lead to increased efficiency on the Sysmex XN-3100, which may help to reduce the workload and cost-effectiveness in our laboratory. Hence the laboratory could improve its' workflow by reducing the turnaround time (TAT).
The merits of this study are that a detailed statistical analysis was performed for all processes, and, for the comparison study in particular, a large number of patient samples having normal and abnormal blood cells in almost equal numbers were included.
There are three main limitations. First, a comparison analysis was not performed based on different levels. Hence, the results cannot be generalized for different levels separately. Leukopenic, anemic, or thrombocytopenic conditions could have been evaluated separately. This is of particular importance for low levels where accuracy plays a crucial role [21]. Second, we could not compare our results with reference to morphological or flow cytometric assessments. Third, three examiners are required to evaluate the flagging performance of blood smears. Although the number of samples was almost zero for the disagreement between two examiners and we gave Cohen's kappa coefficient, only two reviewers were included.
In conclusion, the Sysmex XN-3100 displayed a satisfactory performance with respect to precision, comparison, and carryover. The new channels on the Sysmex XN-3100 displayed an improved sensitivity and specificity compared with the Sysmex XE-2100. According to our data, the Sysmex XN-3100 may improve workflow in the hematology laboratory and enables us to report reliable test results to clinicians.