Comparison of the BD Veritor System for Flu A+B with the Alere BinaxNOW Influenza A&B Card for Detection of Influenza A and B Viruses in Respiratory Specimens from Pediatric Patients
The performance characteristics of two commercially available rapid tests for influenza, the BD Veritor System for Flu A+B (BD) and the Alere BinaxNOW influenza A&B card (BN), were evaluated using 200 frozen clinical specimens collected from January 2011 to June 2012 from pediatric patients. Real-time reverse transcriptase PCR (RT-PCR) was used as the gold standard to evaluate the results obtained by the two different assays. Of the 200 specimens tested, real-time RT-PCR assay detected influenza A or B virus in 116 samples, while BD detected 104 samples and BN detected 84 samples as positive. The overall sensitivity and specificity for detection of both influenza A and B virus in comparison to those of real-time RT-PCR were 89.6% (95% confidence interval [CI], 82.2 to 94.3) and 98.8% (95% CI, 92.6 to 99.9) for BD Veritor and 72.4% (95% CI, 63.2 to 80.0) and 100% (95% CI, 94.5 to 100.0) for BinaxNOW. Workflow analysis indicated that overall processing times for a batch size of 10 specimens were virtually identical between both systems. Overall, these results indicate that the BD Veritor assay was more sensitive than the BinaxNOW assay in detection of influenza A and B viruses in respiratory specimens from pediatric patients.
Influenza, commonly known as “flu,” is a contagious respiratory illness caused by influenza viruses A and B, with seasonal circulation during the winter months. In the United States, seasonal influenza causes substantial morbidity and mortality, with approximately 350,000 hospitalizations and 50,000 deaths annually. Accurate and rapid diagnosis of influenza is necessary for appropriate patient management and anti-influenza therapy. There are several methods available for routine clinical diagnosis of influenza. These methods include traditional microbiological techniques, such as viral cell culture, rapid antigen-based tests, direct fluorescent antibody (DFA) tests, and serologic testing, and molecular techniques, such as reverse transcriptase PCR (RTPCR). Although viral cell culture provides acceptable sensitivities for influenza A/B, major limitations are that culture is labor-intensive and time-consuming (2 to 7 days); hence, results may be obtained too late to effectively influence patient care. Although DFA test results may be available sooner, it is a technically demanding test, and its performance is highly variable based on the expertise of the technologist. Molecular tests, such as real-time RT-PCR, have become the gold standard to detect influenza virus in clinical laboratories due to their high level of sensitivity and specificity and relatively short turnaround time (1 to 6 h). In recent years, rapid antigen detection tests (RADT) have been adopted in point-of-care clinical settings because they are simple to use, cost less, and provide results within 15 to 30 min. However, recent studies have highlighted the poor sensitivity of many of these tests, thus limiting their usefulness.
The BD Veritor System for rapid detection of influenza A and B is a chromatographic immunoassay that was recently FDA cleared for qualitative detection of influenza A and B in respiratory specimens. It represents a new category of influenza detection tests that utilize proprietary chemistries and interpretive algorithms to increase test performance and incorporation of an instrument-based objective digital readout of the test result.
The aim of this study was to compare the performance characteristics of the BD Veritor System for Flu AB (laboratory kit) with the Alere BinaxNOW A&B card, an immunochromatographic assay for detection of influenza A and B antigens that requires a subjective, visual read by the test operator. Frozen clinical samples were tested by both assays, and the results were compared with a validated laboratory-developed real-time RT-PCR assay to determine the performance characteristics per CLSI guidelines. Additionally, a Lean workflow analysis was undertaken to compare the dwell time, hands-on time, and overall processing time between the two systems.
MATERIALS AND METHODS
Clinical specimens. A total of 200 frozen clinical respiratory specimens collected from pediatric patients between >14 days and <18 years old from January 2011 to June 2012 and stored in universal transport medium (UTM) (Diagnostic Hybrids, Inc., Athens, OH) were used in this study. The study was reviewed and approved by the Institutional Review Board (IRB) of Children’s Mercy Hospitals and Clinics. All specimens were stored at 80°C until used for this study.
Influenza detection. Batches of 10 to 15 samples were analyzed each day. Each sample was thawed, aliquoted in three different tubes, stored at 4°C, and tested within 6 h. Quality control was performed for each assay according to standard procedures followed in the clinical microbiology laboratory. To prevent any operator bias, test operators were rotated approximately every 50 samples and were blinded to results of the other method. All samples were de-identified prior to use in this study. Both assays were completed and compared as follows.
(i) BD Veritor System for Flu A&B. Testing by the BD Veritor System for Flu AB (BD Diagnostics, Sparks, MD) was carried out according to the manufacturer’s instructions. In brief, 300 l of each sample was transferred to an RV reagent C tube and mixed thoroughly, and three drops of the processed sample was carefully dispensed into the sample well of the BD Veritor System for Flu AB test device. After 10 min, the test device was inserted into the BD Veritor System reader, and test results were digitally displayed by the instrument. The instrument read time was 10 s.
(ii) Alere BinaxNOW influenza A&B card. Rapid antigen testing using the BinaxNOW influenza A&B card (Alere Inc., Scarborough, ME) was carried out according to the manufacturer’s instructions. Using a transfer pipette, samples were added drop by drop in the middle of a white sample pad of the influenza A&B card and then sealed according to the protocol. After 15 min of incubation, results were read visually by the test operator. A positive influenza A or influenza B result was determined when a pink-to-purple sample line appeared in the middle third or the top third of the window, respectively. In addition to this, the presence of a pink-to-purple “control line” at the bottom of the window was also determined.
(iii) RNA extraction and real-time RT-PCR. Two hundred microliters of total sample containing 180 l of aliquoted specimen and 20 µl of MS2 (internal control) was used for RNA extraction, and a portion of the extracted nucleic acid was tested using a laboratory-developed influenza A/B-pH1N1 RT-PCR assay as described previously. Briefly, total RNA was extracted with the NucliSENS easyMAG automated extraction system (bioMérieux Inc., Durham, NC) according to the manufacturer’s instruction, and the nucleic acid was eluted in 55 µl of elution buffer. The sample eluate was aliquoted into two separate Eppendorf tubes and stored at -20°C until further testing by the laboratory-developed influenza A/BpH1N1 assay. Detailed real-time RT-PCR conditions have been described earlier.
A second RT-PCR was done on influenza A-positive samples to further subtype as seasonal H1N1 or seasonal H3N2 according to methods published earlier. Briefly, real-time RT-PCR for seasonal A/H1N1 and seasonal A/H3N2 was performed on the Applied Biosystems 7500 Fast real-time PCR system (Life Technologies Corporation, Foster City, CA) in a total reaction volume of 25 µl containing 12.5 µl of 2X reaction mixture with ROX, 0.5 µl of respective forward and reverse primers (40 µM stock) and probe (10 µM stock) specific for seasonal HIN1 and H3N2, 0.5 µl of SuperScript III reverse transcriptase-PlatinumTaq mix, 4.0 µl of nuclease-free water, and 5 µl of template RNA. The following thermal cycling protocol was used to detect seasonal H1N1 and H3N2: 50°C for 30 min (reverse transcription), 95°C for 2 min (reverse transcriptase enzyme inactivation), and 45 cycles of 95°C for 15 s (denaturation) and 55°C for 30 s (annealing and signal acquisition). All primers and probes used in this study were purchased from Integrated DNA Technologies (IDT Inc., Coralville, IA).
Workflow analysis. A Six Sigma and Lean expert performed two time and motion studies focused on the collection of the following metrics: total processing time, total processing steps, hands-on time, and potential walk-away time. Two separate batches of 10 specimens were analyzed utilizing the two different analytical methods, and a regression analysis was performed to ensure consistency in the measurements collected and processes documented. Time and process measurements were collected and analyzed from the time the specimens were acquired from the bins on the processing counter to the time that actionable results were generated. Times collected were then compared to the manufacturer’s package insert to ensure processing adhered to manufacturer guidelines. The “dwell time” was defined as any time that the operator was not actively engaged in hands-on testing during the batch analysis.
Statistical analysis. Statistical analysis on the difference in diagnostic yield among the assays was analyzed by Fisher’s exact test. Whisker box plot was used to demonstrate different cycle threshold (CT) values between various samples. Reportable data were summarized in two-by-two data tables listing the number of specimens in each of the four result categories: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). A TP specimen was defined as one in which the reference method (real-time RT-PCR) was positive, while a TN was one in which the reference method yielded a negative result. An FP specimen was defined as one in which the test method gave a positive result and the reference method was negative. An FN specimen was one in which the test method was negative and the reference method was positive. The following calculations were used: sensitivity TP/(TP + FN), specificity = TN/(TN + FP).
The 200 samples included in this study were obtained from male and female children at almost equal ratios: 52.5% male and 47.5% female. The samples were either nasopharyngeal swabs (78.5%) or nasopharyngeal aspirates/wash specimens (21.5%). Of the 200 specimens tested, 116 (58%) were identified as positive for either influenza A or influenza B by RT-PCR; the BD Veritor System for Flu A+B assay detected 104 positive samples (52%), and BinaxNOW detected 84 positive samples (42%). All positive samples detected by BinaxNOW were also detected by the BD assay. Additionally, the BD Veritor System detected 20 true positive samples that were missed by BinaxNOW. None of the samples were identified as false positive by BinaxNOW. One sample (nasopharyngeal wash) originally obtained from a 14-month-old male patient was false positive for influenza A by the BD assay. The overall sensitivity and specificity for detection of both influenza viruses in comparison to real-time RT-PCR. Real-time RT-PCR identified a total of 92 specimens as influenza A positive and 24 specimens as influenza B positive. The sensitivity for influenza A detection by the BD Veritor System was significantly higher than for BinaxNOW (P < 0.0001). On the other hand, sensitivity for influenza B detection by BD Veritor was higher (87.5%) than that for BinaxNOW (70.8%), but this difference was not statistically significant (P < 0.125) due to the limited number of positive samples. The specificities for both influenza A and influenza B were high (> 99%) and comparable for both assays.
The maximum CT values for influenza A samples that were detected by BD Veritor and BinaxNOW were 32.77 and 28.99, respectively. The influenza A-positive specimens were further subtyped into seasonal H3N2, seasonal H1N1, and pandemic H1N1 by real-time RT-PCR. For the 92 samples that were identified as influenza A, 21 samples were found to be pandemic H1N1, 66 samples were seasonal H3N2, and five influenza A-positive samples were untypeable. The sensitivity and specificity of the BD Veritor System and BinaxNOW for detection of each subtype. The difference between the sensitivities of the BD Veritor System (90.9%) and BinaxNOW (68.1%) in detection of H3N2 subtype was statistically significant (P = 0.0021). Review of the real-time PCR CT values indicated that the samples missed by BinaxNOW but detected by the BD Veritor System had relatively higher CT values. This indicates that the BD Veritor System was more capable of detecting low viral loads than BinaxNOW.
Workflow analysis. Conducting a comprehensive workflow analysis is critical in the diagnostic laboratory. Although the BD Veritor System and BinaxNOW are both antigen-based influenza tests, the protocols differ. The BD Veritor requires an extra step for sample dilution and the incubation period is for 10 min, whereas the BinaxNOW protocol does not require sample dilution and the influenza A&B card is incubated for 15 min. We wanted to compare the overall processing time, including hands-on time and dwell time required for each assay system in batch testing mode. A workflow analysis demonstrated that the overall processing times for a batch size of 10 specimens were virtually identical between the two system.
Lateral-flow influenza antigen tests are widely used as a point-of-care test in a clinical setting due to their short turnaround times, ease of use, and low costs. Results can be obtained within 15 to 30 min and therefore have the potential to significantly improve patient care. In this study, we compared the performance characteristics of two FDA-cleared influenza tests, the BD Veritor System for Flu A+B and the Alere BinaxNOW influenza A&B card, for detecting influenza A and B viruses in clinical samples obtained from pediatric patients and compared the results with real-time RT-PCR as the gold standard. We found that the overall sensitivity for influenza virus detection was higher for the BD Veritor System (89.6%) than for BinaxNOW (72.4%).
BD Veritor represents a next-generation assay that enables instrument-based objective test interpretation with precise accuracy. We speculate that unlike the instrument-based objective reads, the visual reads are subjective, with a potential for negative impact on the test performance due to variability in expertise of the test operator, especially with weak-positive specimens. In fact, a recent report by Toepfner et al. found that technical errors, along with lack of experience and expertise of test operators, had a significant negative impact on the rapid antigen test result for group A streptococcal tonsillopharyngitis.
Two recent studies showed that BinaxNOW has 71% to 80% sensitivity for detecting influenza A and only 37% to 47% sensitivity for detecting influenza B in patients, compared with viral culture, indirect immunofluorescence (IFA), and real-time RTPCR. In another retrospective study with frozen nasopharyngeal specimens from adult and pediatric patients, the authors reported sensitivities of 62.2% and 54.5% by BinaxNOW for influenza A and influenza B, respectively, compared with real-time RT-PCR. In our study, we found better sensitivities for BinaxNOW for detection of influenza A (72.8%) and influenza B (70.8%) than for real-time RT-PCR. The differences between earlier reports and our finding regarding higher sensitivity of BinaxNOW could be due to the patient population. Most of the earlier studies were conducted on either adult or mixed (both adult and pediatric) populations, whereas our samples were strictly obtained from a pediatric population which tends to shed high viral titers during infection.
The clinical sensitivity for rapid detection of pandemic H1N1 by available kits is reportedly low. Drexler et al. reported only 11% sensitivity for pandemic H1N1 by BinaxNOW compared to realtime RT-PCR. Our data suggest that both the BinaxNOW and BD Veritor have a similar, high sensitivity (90.9%) for detecting pandemic H1N1, although a significant difference was observed between the two assays for detecting the seasonal H3N2 strain. One H3N2 sample that was detected by BinaxNOW but invalid by the BD Veritor System had a CT value of 21.89. The median CT value for influenza A-positive samples that were detected by both BD Veritor and BinaxNOW (BN+/BN) was 22.86 (range, 15.27 to 28.99) and was lower than the median CT value for samples that were detected only by the BD Veritor System (BD+/BN-), which was 26.73 (range of 21.74 to 32.77). Similarly, the median CT value for influenza B samples that were detected by both the assays (BD+/BN+) was 23.77 (range, 18.76 to 25.73) and was lower than the median CT value for samples that were detected only by BD Veritor System m (BD+/BN-), which was 25.79 (range, 23.94 to 26.93). The performance differences were observed in samples with higher CT values, which correlate with lower viral loads. An earlier study reported that among the 131 subjects positive for influenza by RT-PCR, those with a false-negative BinaxNOW had a median RT-PCR CT value of 30.1 compared with a median CT value of 26.0 for those with positive results. In the current study, median CT values for BN true-positive and false-negative specimens were 23 (range, 15 to 29) and 26 (range, 23 to 37), respectively.
A recent comparative study performed by the CDC for detection of seven seasonal H3N2 control strains found that BD Veritor was able to detect all seven control strains, while BinaxNOW detected only five strains. The same study also found that the BD Veritor System has a 20to 40-fold-higher sensitivity in detecting H3N2 control strains than BinaxNOW. A comparative analysis for limit of detection (LOD) for influenza control strains by both the BD Veritor System and BinaxNOW has been published by Peters et al.. The authors noted that the BD Veritor System for Flu A+B was 8to 32-fold more sensitive than the BinaxNOW influenza A&B card. Recent reports found that both BinaxNOW and BD Veritor can detect evolving influenza strains like A(H3N2v) and A(H7N9) with different capacities. Since the performances of the antigen tests vary based on the circulating influenza strains, it is good laboratory practice to confirm antigen-negative specimens by reference methods. According to the package insert, BinaxNOW recommends that all negative samples should be confirmed by viral culture, while BD Veritor suggests testing all negative samples by viral culture or FDA-approved molecular assay for further confirmation.
Although the overall processing times for a batch size of 10 specimens were virtually identical for the BD Veritor System and BinaxNOW, there were important differences in how the hands-on times were spent. For BinaxNOW, the hands-on time was mostly focused on post-analytical processing, including a visual read and interpretation of the results by the test operator. In contrast, the hands-on time for the BD Veritor System was spent on the pre-analytical process, while the result interpretation process was automated by the BD Veritor System reader. Automating the result process should help remove variability in result interpretation and potentially reduce the likelihood of reporting error and incorrect influenza diagnosis. Although we did not perform direct comparison between single test and batch test mode, it is logical to conclude that the turnaround time for a single test may be faster by the BD Veritor System since it has a shorter incubation time (10 min) than BinaxNOW (15 min), despite the fact that the BD Veritor System has one more preanalytical processing step than BinaxNOW.
In summary, the BD Veritor System for Flu A+B was observed to be more sensitive than the Alere BinaxNOW influenza A&B card for detection of influenza A and B viruses from respiratory specimens. In addition, an objective read by the BD Veritor System reader potentially minimizes operator errors, particularly with interpretation of specimens with low viral titers, and this may represent a significant improvement over traditional lateral flow antigen tests used for influenza diagnosis.