Hindawi Canadian Journal of Infectious Diseases and Medical Microbiology Volume 2022, Article ID 2594564, 10 pages https://doi.org/10.1155/2022/2594564 Research Article The Comparative Analysis of Two RT-qPCR Kits for Detecting SARS-CoV-2 Reveals a Higher Risk of False-Negative Diagnosis in Samples with High Quantification Cycles for Viral and Internal Genes Roberto Luraschi ,1 Carlos Barrera-Avalos ,1 Eva Vallejos-Vidal ,1,2,3 Javiera Alarcón,1 Andrea Mella-Torres ,1 Felipe Hernández,1 Ailen Inostroza-Molina,1 Daniel Valdés,1,4 Mónica Imarai ,1,4 Claudio Acuña-Castillo ,1,4 Felipe E. Reyes-López ,1,5 and Ana Marı́a Sandino 1,4 1 Centro de Biotecnologı́a Acuı́cola, Facultad de Quı́mica y Biologı́a, Universidad de Santiago de Chile, Santiago, Chile 2 Centro de Nanociencia y Nanotecnologı́a CEDENNA, Universidad de Santiago de Chile, Santiago, Chile 3 Facultad de Medicina Veterinaria y Agronomı́a, Universidad de Las Américas, Santiago, Chile 4 Department of Biology, Faculty of Chemistry and Biology, University of Santiago de Chile, Santiago, Chile 5 Department of Cell Biology, Physiology and Immunology, Universitat Autònoma de Barcelona, Bellaterra, Spain Correspondence should be addressed to Felipe E. Reyes-López; felipe.reyes.l@usach.cl and Ana Marı́a Sandino; ana.sandino@usach.cl Received 26 August 2021; Accepted 2 June 2022; Published 5 July 2022 Academic Editor: Louis Detolla Copyright © 2022 Roberto Luraschi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. The early detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) using the real-time quantitative poly- merase chain reaction (RT-qPCR) as a gold-standard molecular tool has allowed to test and trace the viral spread and the isolation of COVID-19-infected patients. The detection capacity of viral and internal genes is an essential parameter to consider and analyze during the assay. In this study, we analyze the performance of the two commercial RT-qPCR kits used in Chile, TaqMan 2019- nCoV Control Kit v1 (Thermo Fisher) and MaxCov19 (TAAG Genetics), for the COVID-19 diagnosis from nasopharyngeal swab ™ samples (NPSs). Our results show a lower sensitivity of the TAAG kit compared to the Thermo Fisher kit, even in the detection of SARS-CoV-2 mutations associated with its variants. This study reinforces the relevance of evaluating the performance of RT- qPCR kits before being used massively since those with lower sensitivity can generate false negatives and produce outbreaks of local infections. 1. Introduction changes in the virus can undermine the protection provided by vaccines, increase infections, and generate sprouts locally The COVID-19 pandemic has profoundly collapsed health and globally. Thus, it is crucial to keep up a continuous systems in the world. Since it was declared a pandemic by the search and testing to detect new possible infections by SARS- World Health Organization (WHO) on March 11, 2020 [1], CoV-2. One of the main ways to control its spread is the more than 6.2 million people have died to date [2]. Although early and effective detection of positive cases. For this, the statistics currently indicate a decrease in infections and molecular technique of real-time polymerase chain reaction deaths in some countries with an increase in the vaccinated (RT-qPCR) has been widely used and recommended for the population [3], the appearance of new variants or genetic diagnosis of COVID-19 disease [4]. There are a series of 2 Canadian Journal of Infectious Diseases and Medical Microbiology commercial RT-qPCR kits available to carry out this tech- reactions followed the manufacturer’s instructions with nique in a massive way on the market. These allow the some modifications. Briefly, the reaction mix contained detection of different target genes of SARS-CoV-2, such as envelope (E), nucleocapsid (N), spike (S), open reading ™ 5 μl of TaqMan Fast Virus 1-Step Master Mix 4X, 1 μl of viral gene assay 20X, 1 μl of RNase P assay 20X, 11 μl of frame 1ab (ORF1ab), or RNA-dependent RNA polymerase nuclease-free water, and 2 μl of total RNA extracted from (RdRp), depending on the type of manufacturer [5]. the NPSs. The thermal amplification conditions include However, important differences in the detection efficiency the reverse transcription at 50°C for 5 minutes, pre- could compromise the kit performance and diagnosis. For denaturation at 95°C for 20 s, followed by 45 cycles at 95°C example, the Gerbion GmbH & Co. diagnostics kit showed a for 3 seconds and 60°C for 30 seconds. All the RT-qPCRs detection efficiency of 49%, while the kit from SolGent Co., were performed on the Agilent AriaMx Real-Time PCR Ltd. manufacturer showed a detection efficiency of 92% in System (Agilent Technologies, Reference code. G8830A). SARS-CoV-2 detection [6]. On the other hand, a study in Data were extracted using the Agilent AriaMx software. India showed differences of up to 21% between the RT-qPCR kits used for the COVID-19 diagnosis [7]. In this study, we evaluate the performance of the RT- 2.4. RT-qPCR Amplification Using TAAG Kit. The detection qPCR TaqMan 2019-nCoV Assay Kit v1 (Thermo Fisher) of SARS-CoV-2 was made following a one-step strategy and MaxCov19 kit (TAAG Genetics), widely used for the using the RT-qPCR MaxCov19 (TAAG Genetics, Reference detection and diagnosis of SARS-CoV-2 in NPSs of patients code. TAGK01052). This assay was carried out using the N1 from the population of Santiago, Chile. We compared the Cq and E gene probes (loading 5 μl of total RNA extracted from values for the detection of internal reference and viral genes. NPSs per reaction), with some modifications. In brief, 2 μl of We found that Cq values are directly related to the SARS- total RNA extracted from NPSs and 3 μl of nuclease-free CoV-2 detection efficiency and diagnosis. The results in- water were loaded in each reaction. Positive control probes dicate a lower sensitivity of the TAAG kit than the Thermo for viral genes and RNase P (internal reference gene) were Fisher kit in detecting SARS-CoV-2 and its impact on the included and assessed individually in the PCR plate. All the analysis and detection of SARS-CoV-2 variants. Our study RT-qPCRs were performed on the Agilent AriaMx Real- emphasizes that the lower SARS-CoV-2 detection of an RT- Time PCR System (Agilent Technologies, Reference code. qPCR kit could compromise the diagnostic and lead to G8830A). Data were extracted using the Agilent AriaMx uncontrolled spread. software. 2. Materials and Methods 2.5. PCR Efficiency and Limit of Detection (LoD). To establish 2.1. Patient Samples. Nasopharyngeal swab samples (NPSs) PCR efficiency and LoD for internal reference and viral gene were collected in the hospitals that belong to the Central probes assessed for TAAG and Thermo Fisher kits, we ran Metropolitan Health Service (SSMC) of Santiago, Chile. The RT-qPCRs using 10-fold serial dilutions. To get the maxi- swab samples were preserved and transported in a CIT- mum representation of values in the curve, we used for the 10-fold serial dilutions a reference pool made from ran- ® OSWAB transport kit (Reference code. 2118-0015; Citotest Labware Manufacturing Co., Ltd, Jiangsu, China). All the domized ten total RNA extracted from NPSs with a Cq samples were processed before the first 24 hours after the value close to 20. The RT-qPCRs were carried out in sampling collection. triplicate according to the specific conditions indicated by the manufacturer and described above. All the RT-qPCRs were performed on the Agilent AriaMx Real-Time PCR 2.2. RNA Extraction. According to the manufacturer’s in- System (Agilent Technologies, Reference code. G8830A). structions, total RNA extraction was done using the Pure We determined the slope by linear regression in RNA Extraction Kit (TAAG Genetics, Santiago, Chile. GraphPad Prism and defined the required levels for PCR Reference Code: TAGGE01003). The extracted RNA was efficiency (E%) > 95% and R-squared (R2) > 0.95. The used immediately for SARS-CoV-2 detection by RT-qPCR. primer efficiency was calculated according to the formula Efficiency (E%) � (10(-1/Slope))-1) ∗ 100 [8]. To determine the detection limit, we select ten samples with Cq values 2.3. RT-qPCR Amplification Using Thermo Fisher Kit. The close to 30. Thus, a standard curve determined the detection of SARS-CoV-2 was carried out using the ORF1ab minimum detection limit for the TAAG RT-qPCR kit for ™ gene, N gene, and S gene probe (TaqMan 2019nCoV Assay Kit v1 (Thermo Fisher Scientific, Reference code. A47532) each probe’s amplification. The experimental TAAG RT- qPCR kit LoD was established based on the last dilution of using a one-step strategy. Positive control probes for viral all the triplicates amplified. We also considered the R2 genes and the RNase P internal reference gene (TaqMan 2019-nCoV Control Kit v1; Thermo Fisher Scientific, ™ (intended as a goodness-of-fit measure for linear re- gression) and the probe efficiency (closer to 100%, Reference code. A47533) were included and assessed in intended 100% as the optimum probe efficiency value). ™ the PCR plate. The polymerase from TaqMan Fast Virus The experimental LoD was compared with the maximum ™ 1-Step Master Mix (Applied Biosystems , Reference code. 44-444-36) was included in each reaction. The Cq value suggested by the manufacturer and detailed in the kit brochure. Canadian Journal of Infectious Diseases and Medical Microbiology 3 2.6. SARS-CoV-2 Variants Detection. The detection of dif- The limit of detection (LoD) and the efficiency per- ferent punctual mutations (HV69/70del; N501Y; P681H; centage (E%) of the probes for the TAAG RT-qPCR kit were E484 K; K417 N/T) was made by the AccuPower SARS- CoV-2 Variants ID Real-Time RT-PCR kit (Bioneer Ref- ® determined, including the N1 (LoD: Cq value � 29.87, equivalent to 1152.2 copies/μl; E% � 115.4%) (Figure 1(e)) erence code: SMVR-2112) according to manufacturer’s and E viral gene probes (LoD: Cq value � 31.09, equivalent to recommendations with some modifications. In brief, 5 μl of 496.26 copies/μl; E% � 118.5%) (Figure 1(f )). The LoD for RNA extracted from the NPS samples were used for each the RNase P internal reference gene probe of the TAAG kit ™ reaction. The Bioneer Exicycler 96 Realtime Quantitative Thermal Block (Bioneer Co., Reference code: A-2060-1) was was not determined due to the lack of amplification when the samples were diluted (data not shown). In this way, the used for this assay. Data were extracted using the Bioneer manufacturer’s recommended criteria for considering a ™ Exicycler 96 software. sample as a suitable sample for analysis: the Cq value is ≤ 38 for the RNase P reference gene probe. The manufacturer recommends a Cq value ≤ 37 both for N1 and E viral genes to 2.7. Ethics Statement. All the experimental procedures in- consider the sample with SARS-CoV-2 positive diagnostic. cluded in this study were authorized by the Ethical Com- The same analysis was performed for the Thermo Fisher mittee of the University of Santiago of Chile (No. 226/2021) RT-qPCR kit. Thus, the LoD for the ORF1ab gene was 5.97 and the Scientific Ethical Committee of the Central Met- copies/μl (Cq � 37.15), 6.34 copies/μl for the N gene ropolitan Health Service, Ministry of Health, Government of (Cq � 36.63); 10.28 copies/μl for the S gene (Cq � 35.72), and Chile (No. 370/2021), and following the Chilean law in force. 3.51 copies/μl for the RNase P reference gene probe (Cq � 37.09) (Supplementary Figures 1 and 2). Table 1 2.8. Data Representation and Statistical Analysis. Paired two- summarizes the LoDs for each RT-qPCR kit evaluated. sided student’s t-test was used for the Cq and RFU analysis of the reference gene. A one-way paired ANOVA test was used for the viral gene to analyze the differences between RFU and 3.1. Amplification of Internal Reference and Viral Gene Probes Cq values. Statistical software GraphPad Prism 8 was used to of TAAG RT-qPCR Kit. Once the LoD for the TAAG kit was analyze and graph the data. A p < 0.05 was considered determined, 91 NPS random samples were analyzed and statistically significant. compared their amplification parameters (Cq and RFU values) between the Thermo Fisher and TAAG RT-qPCR 3. Results kits. The paired comparison of the Cq values of the RNase P internal reference gene probe (Figure 2(a)) showed a sig- We evaluated the amplification profile using 2 and 5 μl (the nificant difference between both RT-qPCR kits. The samples last volume recommended by the manufacturer) of total analyzed by the Thermo Fisher kit showed a mean Cq value RNA extracted from 91 random NPSs. Thirty-four samples of 22.30 ± 2.62, while the TAAG kit showed 37.71 ± 4.07. were previously diagnosed as SARS-CoV-2 positive by the Importantly, all these samples showed amplification for the Thermo Fisher kit. Importantly, for the RNase P internal RNase P internal reference gene probe, making them suit- reference gene probe (Figure 1(a)), no significant differences able for diagnostic when analyzed by the Thermo Fisher kit were observed in the mean Cq value of the samples when 2 μl (Figure 2(b)). However, when TAAG analyzed the same (35.28 ± 3.74) and 5 μl (35.14 ± 5.37) of total RNA were samples, only 74.7% (68/91) of them could be considered loaded. This implies that the 87.9% (80/91) and 79.1% (72/ suitable samples for diagnosis because their Cq values were 91) samples, respectively, showed amplification with a Cq ≤38 ( the maximum recommended Cq value for sample value ≤ 38 (considered as a suitable sample for analysis, diagnosis). The paired comparison of RFU values obtained according to manufacturer’s instructions) (Figure 1(d)). For for both kits also showed significant differences the N1 viral gene probe (Figure 1(b)), no significant dif- (Figure 2(c)), with mean values of 5426 ± 596.1 and ferences were observed between their mean Cq values when 921.8 ± 467.2 for the Thermo Fisher and the TAAG RT- 2 μl (28.14 ± 6.66) and 5 μl (28.82 ± 7.29) of total RNA were qPCR kits, respectively. loaded. However, 91.2% (31/34) and 85.3% (29/34) of Also, the comparison of the viral genes of both kits was samples evaluated showed amplification with a Cq value ≤ 37 assessed. The paired comparison of the mean Cq values of respectively, according to the manufacturer’s recommen- viral gene probes of both RT-qPCR kits (Figure 2(d)) showed dation for a positive SARS-CoV-2 sample. For the E viral a significant difference, except between the S gene probe gene probe (Figure 1(c)) significant differences were ob- (31.38 ± 8.65) with the N1 gene probe (34.45 ± 9.39) and the served between their mean Cq values when 2 μl E gene probe (35.77 ± 9.34) from the TAAG kit. The paired (30.35 ± 6.35) and 5 μl (43.06 ± 5.39) of total RNA were comparison of the RFU values showed significant differences loaded. This implied that 91.2% (31/34) and 17.6% (6/34) of between any of the viral gene probes between both RT-qPCR the samples registered amplification with a Cq value ≤ 37 (set kits (Figure 2(e)). No differences in the mean RFU value following the manufacturer’s recommendation for a positive were registered between ORF1ab (3838 ± 649.4) and S viral SARS-CoV-2 sample) (Figure 1(d)). Based on these results gene probes (3311 ± 1910) for the Thermo Fisher kit. The and previous antecedents for other RT-qPCR kits evaluated same case was also reported between N1 (1263 ± 1025) and E (Santibáñez et al., 2021), we defined 2 μl as the chosen viral gene probes (1722 ± 1706) for the TAAG kit loading volume for the TAAG RT-qPCR kit. (Figure 2(e)). 4 Canadian Journal of Infectious Diseases and Medical Microbiology RNase P gene N1 gene E gene 25 15 20 **** 20 25 30 25 30 35 30 Cq value Cq value Cq value 35 35 40 40 40 45 45 45 35.28 ± 3.74 35.14 ± 5.37 30.35 ± 6.35 43.06 ± 5.39 28.14 ± 6.66 28.82 ± 7.29 50 50 50 2 µl 5 µl 2 µl 5 µl 2 µl 5 µl (a) (b) (c) Suitable NPSs for diagnosis LoD N1 gene LoD E gene 35 35 87.9% 80 79.1% 70 30 30 60 # of total samples 50 Cq value Cq value 25 25 40 91.2% 85.3% 91.2% 30 20 20 y= -3.000 x + 20.84 20 y= -2.945 x + 22.08 R2= 0.8010 R2= 0.9144 10 17.6% E%= 115.4 E%= 118.5 0 15 15 RNase P N1 E gene -4 -3 -2 -1 0 -4 -3 -2 -1 0 gene gene log10 dilution log10 dilution 2 µl RNA 5 µl RNA (d) (e) (f ) Figure 1: Determination of total RNA extracted from nasopharyngeal swab samples (NPSs) loading volume and the limit of detection (LoD) of SARS-CoV-2 probes using TAAG RT-qPCR kit. The upper section of the figure represents the Cq value paired comparison for total RNA extracted from NPSs loading volume determination. The analysis was made from the same NPSs using the manu- facturer’s recommended volume of 5 μl and 2 μl of total RNA extracted. In the graphs, the comparison was made for (a) RNase P internal reference, (b) N1, and (c) E viral gene probes. Each spot is a different analyzed sample for each volume condition (2 μl; 5 μl). On (a-c), the mean ± standard deviation (mean ± SD) for all the samples evaluated is indicated for each total RNA volume condition (2 μl; 5 μl). The line linking the spots indicate the paired result obtained for the same sample assessed using the two different volume conditions. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, paired two-sided student’ t-test was applied (n � 80 random NPSs, n � 31 N1 gene, and n � 31 E gene random positive SARS-CoV-2 NPSs, ∗∗∗∗ p < 0.05). The lower section of the figure represents (d) the number of amplified samples using TAAG probes loading 2 and 5 μl of total RNA extracted. Limit of detection (LoD) for (e) N1 gene probe and (f ) E gene probe for SARS-CoV-2 detection. The analysis included 10-fold serial dilutions from a reference pool made from randomized ten total RNA NPS-extracted samples with a Cq value close to 20 (previously obtained using the Thermo Fisher kit). For the LoD determination, a linear regression was performed. When comparing the number of samples with a positive diagnosis for SARS-CoV-2 based on the viral gene ampli- Table 1: Summary of limit of detection (Lod, viral copies/µl) for fication, it was observed that the number of positive samples each probe of the analyzed RT-qPCR kits detected by each kit varies worryingly. The Thermo Fisher kit showed amplification for 100% (20/20) of positive samples TAAG RT-qPCR Kit LoD Thermo Fisher RT-qPCR Kit LoD for viral ORF1ab and N gene probes (Figure 2(f )), while for (copies/μl) (copies/μl) the viral S gene probe, it only detected 80% (16/20) of those RNase P N1 E RNase P ORF1ab N S positive samples. In the case of the TAAG kit, the viral gene n/d 1152.2 496.26 3.51 5.97 6.34 10.28 probes N1 and E were noticed in only 65% (13/20) and 60% Canadian Journal of Infectious Diseases and Medical Microbiology 5 RNase P gene Cq's for RNase P gene Suitable NPSs for diagnosis RFU's for RNase P gene 15 100 7000 **** **** 100% 90 20 6000 80 5000 25 70 74.7% RFU (reference gene) Cq (reference gene) # of total samples 60 4000 30 RNase P LoD for 5426 ± 596.1 22.30 ± 2.62 Thermo Fisher 50 3000 35 (Cq= 36.9) 40 2000 40 30 RNase P LoD for TAAG 1000 (Cq= 38.0) 20 45 10 0 37.71 ± 4.07 No Cq 921.8 ± 467.2 50 (Cq= 46) 0 Thermo Fisher TAAG Thermo Fisher TAAG Thermo Fisher TAAG (a) (b) (c) SARS-CoV-2 viral genes Cq's for SARS-CoV-2 genes RFU's for SARS-CoV-2 genes 10 8000 15 7000 a b ab 6000 a 20 a b 5000 25 RFU (viral gene) N1 gene LoD for TAAG Cq (viral gene) (Cq= 29.87) 4000 30 b E gene LoD for 3000 35 TAAG (Cq= 31.09) 2000 3838 ± 649.4 5327 ± 1494 40 27.92 ± 5.29 26.29 ± 5.52 31.38 ± 8.65 34.45 ± 9.39 35.77 ± 9.34 Viral LoD recommended 1000 by TAAG 45 (Cq= 37.0) 0 3311 ± 1910 1263 ± 1025 1722 ± 1706 No Cq 50 (Cq= 46) ORF1ab N S N1 E ORF1ab N S N1 E gene gene gene gene gene gene gene gene gene gene Thermo Fisher TAAG Thermo Fisher TAAG (d) (e) SARS-CoV-2 positive samples Positive samples diagnosed % of false negative samples by TAAG 100% 100% 20 15 100 19 14 18 65.0% 90 17 13 80.0% 60.0% 16 12 80 15 11 14 65.0% 50.0% 70 13 10 60.0% 45.0% 12 9 60 % of samples # of samples # of samples 11 40.0% 40.0% 8 10 50 9 7 40.0% 40.0% 8 6 40 7 25.0% 25.0% 5 6 30 5 4 4 3 20 3 2 2 10 1 1 0 0 0 ORF1ab N S N1 E N1 E (+) SARS-CoV-2 (+) SARS-CoV-2 N1 gene E gene gene gene gene gene gene gene gene (+) RNase P Thermo Fisher TAAG TAAG N1 gene (LoD) E gene (LoD) LoD brochure experimental N1 gene (Brochure) E gene (Brochure) Brochure E gene (LoD) E gene (Brochure) (f) (g) (h) Figure 2: Comparative analysis for the detection of SARS-CoV-2 from random-chosen nasopharyngeal swab samples (NPSs) using Thermo Fisher and TAAG RT-qPCR kits. The comparison was made from the same NPSs using the previously optimized volume of 2 μl of total RNA extracted. The upper section of the figure represents the analysis for the amplification of RNase P internal reference gene probes from 91 random NPSs. (a) Cq value for internal reference RNase P gene probe amplification using Thermo Fisher and TAAG RT-qPCR kits (n � 91 NPSs). In graph, the horizontal red-dotted line represents the LoD for RNase P reference gene probe (Cq � 36.9) by Thermo Fisher kit, and the horizontal blue- dotted line represents the maximum recommended Cq value (Cq � 38) by the manufacturer of suitable sample for diagnosis using TAAG RT- qPCR kit RNase P reference gene probe. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). (b) Summary of the percentage of suitable NPSs for diagnosis using the internal reference RNase P gene probe (detailed on (a)) by both RT-qPCR kits. (c) RFU value from the same samples analyzed on (a). Samples with a Cq value > 38 were considered as not suitable according to manufacturer’s recommendations. For statistical analysis, a paired two-sided student’s t-test was applied (n � 91 random NPSs, ∗∗∗∗ p < 0.05). The lower section of the figure represents the analysis for the amplification of SARS-CoV-2 viral gene probes from 91 random NPSs. Paired comparison for (d) Cq and (e) RFU values from viral gene probe amplification using Thermo Fisher (ORF1ab, N, and S gene probes) and TAAG (N1 and E gene probes) kits (n � 20). In graph (d), the horizontal colored-dotted lines blue and green, respectively, represent the experimental LoD for N1 gene (Cq � 29.87) and E gene (Cq � 31.09) probes of TAAG RT-qPCR kit. The horizontal black dotted line represents the maximum Cq value (Cq � 37) rec- ommended by the manufacturer for positive sample diagnosis using TAAG RT-qPCR viral probes. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, a paired one-way ANOVA-test with multiple comparison test analysis applied. Lowercase letters above each probe’s spot columns indicate which probes do not show significant differences between them. (n � 20 NPSs, ∗∗∗∗ p < 0.05). In graphs (a), (c), (d), and (e), the mean ± standard deviation (mean ± SD) is indicated for the viral gene probe amplification obtained for all the samples evaluated. The line linking the spots indicated the paired result obtained for the same sample assessed by both RT-qPCR kits. (f) Summary of the number of positive NPSs detected using Thermo Fisher (ORF1ab, N, and S probes) and TAAG RT-qPCR kit (N1 and E viral gene probes applying brochure or experimental LoD criteria ((n) � 20). (g) Summary of the number of positive NPSs diagnosed using TAAG RT-qPCR kit considering the RNase P internal control amplification. In the graph, “(+) SARS-CoV-2” represents a positive virus diagnosis, and “(+) RNase P” represents the amplification of the internal reference gene. (h) Summary of the percentage of false negative NPSs diagnosed by TAAG RT-qPCR kits applying brochure or experimental LoD criteria. 6 Canadian Journal of Infectious Diseases and Medical Microbiology (12/20) of those positive samples, respectively, when rec- qPCR kit on samples previously detected with a Cq value ommended manufacturer criteria were applied (Figure 2(f ), close to 30 by the Thermo Fisher kit (Figure 3(e)). TAAG brochure). However, when the experimentally de- termined LoD was used (Figure 2(f ), TAAG experimental), both viral gene probes detected only 40% (8/20) of those 3.2. Impact of the TAAG Diagnostic Kit on the Detection of positive samples. SARS-CoV-2 Variants. Twelve previously diagnosed SARS- When the criteria of Cq values recommended by the CoV-2 positive samples by Thermo Fisher kit were randomly manufacturer for viral genes (Cq ≤ 37) and the internal chosen and analyzed for the presence of SARS-CoV-2 reference gene (Cq ≤ 38) are applied to ensure the di- variants by RT-qPCR. The detection of single nucleotide agnostic of samples, the situation becomes more com- variations (SNV) was focused on the S gene. Six of these plex for the TAAG kit. For N1 and E probes, only 50% samples showed amplification only for the SARS-CoV-2 (10/20) and 45% (9/20), respectively, of positive SARS- probe but not for any SNV probes (SARS-CoV-2(+) SNV(-); CoV-2 samples were reported when the RNase P internal denoted with triangles (▲)) (Figure 4(a)). The TAAG N1 and reference gene showed amplification (Figure 2(g)). E gene probes identified only the 33.3% (2/6) of samples However, when the experimentally determined LoD was from these six samples (Figure 4(b)). The other six samples applied, only 25% (5/20) of the samples were reported as showed amplification for SARS-CoV-2 and for the N501Y, positive for SARS-CoV-2 for TAAG viral gene probes K417 N/T, and E484 K SNV probes (SARS-CoV-2(+) (Figure 2(g)). As a consequence, the analysis with the N1 SNV(+); denoted with squares (■)) (Figure 4(a)) suggesting viral gene probe (Figure 2(h)) showed a 35% (7/20) and that these samples correspond to the SARS-CoV-2 variant of 60% (12/20) false-negative diagnosis when the manu- concern (VOC) Gamma. From these six samples, the TAAG facturer’s recommendation (Cq ≤ 37) and experimental N1 gene probe identified only 50% (3/6), whereas the E gene LoD (Cq ≤ 29.87) were applied, respectively. Similar probe identified only 33.3% (2/6) (Figure 4(b)). results were obtained for the E viral gene probe (Figure 2(h)), 40% (8/20) and 60% (12/20) of false- 4. Discussion negative diagnosed samples applying the manufacturer’s recommendation (Cq ≤ 37) and experimental LoD Currently, multiple RT-qPCR kits are commercially avail- (Cq ≤ 31.09), respectively. able for SARS-CoV-2 detection. However, they lack ap- Because of the worrying percentage of false negatives propriate clinical analysis and validation for use in the observed for the TAAG kit (Figure 2(h)), we evaluated the COVID-19 pandemic. Various studies of comparative performance of the TAAG kit with ten previously diagnosed analysis and proofs of RT-qPCR kits have previously been SARS-CoV-2 positive samples with Cq values close to 30 reported. For example, comparisons of kits used for using the Thermo Fisher kit (Figure 3). The amplification of COVID-19 diagnosis in Korea showed differences of up to the RNase P internal reference gene probe (Figure 3(a)) 25% in the sensitivity of detection of SARS-CoV-2 [9]. In showed significant differences between the mean Cq value of China, some kits used have 83% and 94% sensitivity, both RT-qPCR kits (CqThermo Fisher � 24.19 ± 1.08; according to Sansure Biotech Inc. and BioGerm Medical, CqTAAG � 40.19 ± 5.79). The RNase P internal reference gene respectively, both with 100% specificity without cross-re- probe amplification showed all the samples suitable for action in the detection of other viruses [10]. On the other diagnosis by the Thermo Fisher kit (Figure 3(b)). However, hand, Eberle et al. 2021, analyzed nine commercial RT-qPCR when TAAG analyzed the same samples, only 60% (6/10) of kits used for the diagnosis of COVID-19 in Bavaria, Ger- these could be considered suitable samples for diagnosis many, with differences of up to 50% in detection sensitivity because their Cq value was ≤ 38, the maximum Cq value [6]. Also, a study using RNA from cell cultures identified recommended for diagnosis. When the amplification of the differences in the number of viral copies detected in eleven viral genes was analyzed, we registered differences between commercial RT-qPCR kits, which varied between 3.3 and the mean Cq value of ORF1ab (31.16 ± 0.87), N1 330 RNA copies, where one also jointly noticed another (37.34 ± 5.74), and E (42.20 ± 6.13) viral gene probes human coronavirus (MERS), revealing nonspecificity in the (Figure 3(c)). The percentage of these SARS-CoV-2 positive diagnosis for SARS-CoV-2 [11]. All these studies strongly samples diagnosed by the viral gene probes of the TAAG kit support the importance of an in-depth analysis of the (Figure 3(d)), following the manufacturer’s recommenda- performance of the RT-qPCR kits used to control the current tions, was 60% (6/10) and 30% (3/10) for N1 and E viral gene pandemic, which translates into better or worse control and probes, respectively. When the criteria of internal reference traceability of infected patients. However, to date, no study gene amplification (Cq ≤ 38) ((+) RNase P) were applied, has reported an analysis of the RT-qPCR kits widely used to both viral probes showed 30% (3/10) of samples as positive diagnose SARS-CoV-2 in the Chilean population. Fur- for SARS-CoV-2 (Figure 3(d)). This implies for the TAAG thermore, none of the previous studies has considered a deep kit a 40% (4/10) and 70% (7/10) false-negative samples were analysis of the RT-qPCR parameters of the cellular internal diagnosed for N1 and E gene probes, respectively reference gene (only viral target genes are mainly consid- (Figure 3(e)). However, no one positive sample was detected ered). The amplification of the internal reference gene is when the experimentally determined LoD was applied to essential but underestimated data in an RT-qPCR analysis to TAAG viral gene probes (Figure 3(d)). Thus, 100% (10/10) of ensure the accuracy of negative and positive results in false-negative diagnoses were recorded using the TAAG RT- clinical diagnosis [12]. Canadian Journal of Infectious Diseases and Medical Microbiology 7 RNase P gene Suitable NPSs for diagnosis Cq's for RNase P gene 100% 20 10 **** 9 25 8 24.19 ± 1.08 7 30 60.0% 6 # of samples RNase P LoD for Cq value Thermo Fisher 35 (Cq= 36.9) 5 4 40 RNase P LoD for TAAG 3 (Cq= 38.0) 2 45 40.19 ± 5.79 1 No Cq (Cq= 46) 50 0 Thermo Fisher TAAG Thermo Fisher TAAG (a) (b) SARS-CoV-2 viral genes Cq's for SARS-CoV-2 genes Positive samples diagnosed % of false negative samples by TAAG 25 10 100 *** N1 gene LoD for TAAG 9 90 * * (Cq= 29.87) 30 8 80 E gene LoD 7 70 for TAAG (Cq= 31.09) 60% 35 31.16 ± 0.87 6 60 % of samples # of samples Cq value 5 50 Viral LoD 40 recommended 4 40 by TAAG (Cq= 37.0) 30% 30% 30% 3 30 45 2 20 37.34 ± 5.74 42.20 ± 6.13 No Cq 1 10 (Cq= 46) 0% 0% 0% 0% 50 0 0 ORF1ab N1 E (+) SARS-CoV-2 (+) SARS-CoV-2 N1 gene E gene gene gene gene (+) RNase P Thermo Fisher TAAG N1 gene (LoD) E gene (LoD) LoD N1 gene (Brochure) E gene (Brochure) Brochure (c) (d) (e) Figure 3: Comparative analysis between Thermo Fisher and TAAG RT-qPCR kits with a known Cq value close to 30 (for ORF1ab gene probe, Thermo Fisher). The comparison was made for the same NPSs using the previously optimized volume of 2 μl of total RNA extracted. The upper section of the figure represents the analysis for the RNase P internal reference gene probe amplification from 10 NPSs previously diagnosed as SARS-CoV-2 positive by the Thermo Fisher RT-qPCR kit with a Cq value close to 30. (a) Paired comparison of Cq value of RNase P internal reference gene probes from Thermo Fisher and TAAG kits. In graph, the horizontal red-dotted line represents the LoD for RNase P reference gene probe (Cq � 36.9) by Thermo Fisher kit, and the horizontal blue-dotted line represents the maximum recommended Cq value (Cq � 38) by the manufacturer for suitable sample diagnosis using TAAG RT-qPCR kit RNase P reference gene probe. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, a paired two-sided student’s t- test was applied (n � 10 NPSs, ∗∗∗∗ p < 0.05). (b) Number of suitable samples diagnosed with a Cq value within the detection range of the internal reference gene probes. The lower section of the figure represents the analysis for the viral gene probe amplification from 10 NPSs previously diagnosed as SARS-CoV-2 positive by the Thermo Fisher RT-qPCR kit with a Cq value close to 30. (c) Paired comparison of Cq value of viral gene probes from Thermo Fisher (ORF1ab viral gene probe) and TAAG (N1 and E viral gene probes) kits. In graph, the horizontal colored-dotted lines blue, green, and red, respectively, represent the experimental LoD for N1 gene (Cq � 29.87), E gene (Cq � 31.09), and ORF1ab gene (Cq � 37.14) viral gene probes. Samples with Cq � 46 denote no amplification (indicated in the graph by a black broken line). For statistical analysis, paired one-way ANOVA-test with multiple comparison test analysis was applied. (n � 10 NPSs, ∗∗∗∗ p < 0.05). In graphs (a) and (c), the mean ± standard deviation (mean ± SD) is indicated for the viral gene probe amplification obtained for all the samples evaluated. The line linking the spots indicate the paired result obtained for the same sample assessed by both RT-qPCR kits. (d) Summary of the number of positive NPSs diagnosed using TAAG RT-qPCR kit considering the RNase P internal control amplification. In the graph, “(+) SARS-CoV-2” represents a positive virus diagnosis, and “(+) RNase P” represents the amplification of the internal reference gene. (e) Summary of the percentage of false negative NPSs diagnosed by TAAG RT-qPCR kits applying brochure or experimental LoD criteria. We found that the TAAG kit showed serious difficulties the Thermo kit showed a detection efficiency of 100% for N in amplifying the internal and viral reference genes, along and ORF1ab genes. In addition, the samples analyzed with with both high Cq and low RFU values. This was reflected in the Thermo Fisher kit showed RFU values up to 6 times a difference of up to 40% in SARS-CoV-2 detection where higher than those observed with the TAAG RT-qPCR kit, 8 Canadian Journal of Infectious Diseases and Medical Microbiology Variant detection SNV probes detection 100% 100% 15 6 20 5 25 4 Cq (viral gene) 30 # samples 50% 3 35 33.3% 33.3% 33.3% 2 40 1 45 50 0 SARS-COV-2 N501Y K417N E484K Hv69/70Del P681H ORF1ab N1 E gene gene gene SARS-CoV-2 Thermo Fisher TAAG Gamma (N501Y + K417N + E484K) SARS-CoV-2 Gamma variant (a) (b) Figure 4: Impact of the TAAG diagnostic kit on the detection of SARS-CoV-2 variants. Twelve previously diagnosed SARS-CoV-2 positive samples by Thermo Fisher kit were randomly chosen and analyzed for the presence of SARS-CoV-2 variants by RT-qPCR. (a) Cq values for detected SNV probes associated with SARS-CoV-2 variants (N501Y, K417 N/T, E484 K, Hv 69/70 del, and/or P681H). When only the original SARS-CoV-2 variant was detected, it was denoted with triangles (▲). When the Gamma variant sample (N501Y, K417 N/T and E484 K probes positive amplification) was detected, it was denoted with squares (■). The other six samples showed no amplification for any of the SNV probes tested. (b) Summary of the diagnostic impact of the TAAG (N1 and E genes) and Thermo Fisher (ORF1ab gene) on the detection of SARS-CoV-2 variants. indicating less inhibition in the RT-qPCR reaction. Based on competent sample [18]. In relative expression analyses, the these data, it is not surprising that the diagnosis of NPSs reference gene is essential to normalize the expression levels includes a high number of false-negative diagnoses by the of target genes [12]. In diagnosis, it is a solid basis to validate TAAG kit. This is because the TAAG kit has an LoD of over the proper process of total RNA extraction and subsequent 1000 viral copies/μl for N1 and about 500 viral copies/μl for amplification of the SARS-CoV-2 genes of interest used the E gene, which is equivalent to a sensitivity of up to 100 (ORF1ab, N, S, E, and RdRp). Due to the nonamplification of times less compared to the Thermo kit. Fisher. Conse- an internal gene, the results may be invalidated [19]. quently, the detection of SARS-CoV-2 by the TAAG kit was The comparative analysis between the Thermo Fisher seriously compromised. and TAAG RT-qPCR kits revealed differences in the RFU Our report was closely related to that previously de- values obtained for the same set of samples, indicating scribed [13] for the Thermo Fisher RT-qPCR kit, which has differences in kit performance and specifically in the degree shown a sensitivity of >96% in detecting SARS-CoV-2. This of inhibition for each reaction [20]. A low RFU value could evidence was also according to other reports showing the even be mistaken for a background signal [21]. In this way, optimal performance of the Thermo Fisher kit for the de- the incorporation of RFU values analysis supports our re- tection of low viral loads (Cq > 30) [13], and in the detection sults. For this reason, we suggest that it is also a parameter that of SARS-CoV-2 in samples without a previous total RNA should be carefully considered to decide if a sample is suitable extraction process (which decreases the amount of free RNA for issuing a diagnostic result. Therefore, these findings indicate in the sample to be analyzed) [14]. This background confirms limitations on the use of the TAAG kit for the diagnosis of the high sensitivity of this kit for the diagnosis of COVID-19. COVID-19. As a consequence, the misidentification of positive On the other hand, most of the comparative investigations of cases also compromises the identification of circulating SARS- different RT-qPCR kits for the diagnosis of COVID-19 focus CoV-2 variants in the population (like the gamma variant). RT- only on the amplification of the SARS-CoV-2 gene, omitting qPCR kits were developed to detect characteristic SNV asso- the information of the reference gene, which can invalidate a ciated with SARS-CoV-2 variants. However, samples must be result if it presents faulty parameters [15–17]. For example diagnosed as positive using an RT-qPCR kit before this other kits use, for example, exogenous internal controls (e.g., analysis. Therefore, it is essential to avoid using low-sensitivity EAV; equine arteritis virus, etc.). However, this is only an RT-qPCR kits since this could include false-negative diagnoses intrinsic control and is not related to quality and a and ambiguities in the clinical diagnosis. This is a matter of Canadian Journal of Infectious Diseases and Medical Microbiology 9 critical concern, especially at this epidemiological stage of the Resources for Research Projects on the Coronavirus pandemic with the presence of more contagious variants such (COVID-19) (project number COVID1038; ANID, Gov- as omicron [22]. ernment of Chile), Fondecyt regular project numbers Our results do not consider the specificity of the analyzed 1201664 (M.I.) and 1211841 (F.E.R.-L.) “Fondecyt iniciación kits. In this regard, some reports indicate cross-reaction with 11221308 (EV-V)” (ANID, Government of Chile), and other types of viruses [11], which is an aspect to be con- DICYT-USACH project number 021943AC (C.A-C.) grants. sidered by the manufacturers of commercial RT-qPCR kits. EV-V was partially funded by FONDEQUIP grant project Thus, we suggest an exhaustive analysis of all commercial number EQM200016 “and Basal Grant CEDENNA AFB- kits before they are used for the best control of this and other 180001” (ANID; Government of Chile) infectious diseases Supplementary Materials 5. Conclusions Supplementary Figure 1: probe efficiency and limit of de- Our results indicate that (1) the TAAG kit shows low Cq tection (LoD) of Thermo Fisher RT-qPCR. The efficiency values for internal and viral genes of SARS-CoV-2 due to low and LoD of Thermo Fisher probes were determined using detection of viral copies/μl . (2) The TAAG RT-qPCR kit 10-fold serial dilutions from ten positive samples. The LoD negatively impacts the detection for the gamma variant of for the (A) ORF1ab gene probe was 5.97 copies/μl SARS-CoV-2 compared to the Thermo Fisher kit. (3) The (Cq � 37.15); (B) 6.34 copies/μl for the N gene probe massive use of kits with a high risk of diagnosis of false (Cq � 36.63); (C) 10.28 copies/μl for the S gene probe negatives could jeopardize the application of public policy (Cq � 35.72); and (D) 3.51 copies/μl for the RNase P ref- measures to control the pandemic. erence gene probe (Cq � 37.09). Supplementary Figure 2: Data Availability standard curve for Thermo Fisher RT-qPCR probes. Stan- dard curves for (A) ORF1ab gene, (B) N gene, (C) S gene, The data used to support the findings of this study may be and (D) RNase P gene probes, developed from 10-fold serial released upon direct reasonable request to the corre- dilutions of the synthetic positive control of the Thermo sponding author, who can be contacted at e-mail felipe.r- Fisher RT-qPCR kit. In the graph, the dotted black line eyes.l@usach.cl and felipe.reyes@uab.cat. denotes the LoD for each probe. (Supplementary Materials) Consent References The study contemplated the use of an oral informed consent [1] N. Zhu, D. Zhang, W. Wang et al., “A novel coronavirus from delivered to the patients by the health centers responsible for patients with pneumonia in China,” New England Journal of the collection of the nasopharyngeal swab samples used for Medicine, vol. 382, no. 8, pp. 727–733, 2019. SARS-CoV-2 testing. [2] WHO, “WHO coronavirus (COVID-19) dashboard,” https:// covid19.who.int/. [3] Our World in Data, “Coronavirus (COVID-19) vaccinations,” Disclosure https://ig.ft.com/coronavirus-vaccine-tracker. The funders had no role in study design, data collection and [4] V. M. Corman, O. Landt, M. Kaiser et al., “Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR,” Euro analysis, the decision to publish, or the preparation of the Surveillance, vol. 25, no. 3, pp. 1–8, 2019. manuscript. [5] J. F.-W. Chan, K.-H. Kok, Z. Zhu et al., “Genomic charac- terization of the 2019 novel human-pathogenic coronavirus Conflicts of Interest isolated from a patient with atypical pneumonia after visiting Wuhan,” Emerging Microbes & Infections, vol. 9, no. 1, All authors declare that they have no known competing pp. 221–236, 2020. financial interests or personal relationships that could have [6] U. Eberle, C. Wimmer, C. Wimmer et al., “Comparison of appeared to influence the work reported in this paper. nine different commercially available molecular assays for detection of SARS-CoV-2 RNA,” European Journal of Clinical Authors’ Contributions Microbiology & Infectious Diseases, vol. 40, no. 6, pp. 1303– 1308, 2021. Roberto Luraschi and Carlos Barrera-Avalos contributed [7] J. Singh, A. K. Yadav, A. Pakhare et al., “Comparative analysis equally to this work. of the diagnostic performance of five commercial COVID-19 qRT PCR kits used in India,” Scientific Reports, vol. 11, no. 1, 2021. Acknowledgments [8] M. W. Pfaffl, “A new mathematical model for relative quantification in real-time RT–PCR,” Nucleic Acids Research, The Laboratory of Virology had the support from the vol. 29, no. 9, p. e45, 2001. COVID-19 diagnosis in the University laboratories network [9] K. H. Hur, K. Park, Y. Lim, Y. S. Jeong, H. Sung, and (Ministry of Sciences, Ministry of Health, Government of M. N. Kim, “Evaluation of four commercial kits for SARS- Chile) for diagnosis tasks. A.S was supported by National CoV-2 real-time reverse-transcription polymerase Chain Doctorate Scholarship no. 21180465 (ANID; Government of reaction approved by emergency-use-authorization in Korea,” Chile). The authors also thank the Rapid Assignment of Frontiers of Medicine, vol. 7, no. 521, pp. 521–610, 2020. 10 Canadian Journal of Infectious Diseases and Medical Microbiology [10] Y. Lu, L. Li, S. Ren et al., “Comparison of the diagnostic efficacy between two PCR test kits for SARS-CoV-2 nucleic acid detection,” Journal of Clinical Laboratory Analysis, vol. 34, no. 10, 2020. [11] Z. Iglói, M. leven, Z. Abdel-Karem Abou-Nouar et al., “Comparison of commercial realtime reverse transcription PCR assays for the detection of SARS-CoV-2,” Journal of Clinical Virology: The Official Publication of the Pan American Society for Clinical Virology, vol. 129, pp. 104510–104516, 2020. [12] B. Kozera and M. Rapacz, “Reference genes in real-time PCR,” Journal of Applied Genetics, vol. 54, no. 4, pp. 391–406, 2013. [13] A. Garg, U. Ghoshal, S. S. Patel et al., “Evaluation of seven commercial RT-PCR kits for COVID-19 testing in pooled clinical specimens,” Journal of Medical Virology, vol. 93, no. 4, pp. 2281–2286, 2021. [14] C. Beltrán-Pavez, C. Márquez, G. Muñoz et al., “SARS-CoV-2 detection from nasopharyngeal swab samples without RNA extraction,” Journal of Virological Methods, vol. 287, 2020. [15] P. B. van Kasteren, B. van der Veer, S. van den Brink et al., “Comparison of seven commercial RT-PCR diagnostic kits for COVID-19,” Journal of Clinical Virology, vol. 128, 2020. [16] L. Shen, S. Cui, D. Zhang, C. Lin, L. Chen, and Q. Wang, “Comparison of four commercial RT-PCR diagnostic kits for COVID-19 in China,” Journal of Clinical Laboratory Analysis, vol. 35, no. 1, pp. 1–7, 2021. [17] J. Zhao, Y. Zhang, and B. Cao, “Comparison of the perfor- mance of six SARS-CoV-2 nucleic acid detection kit in positive samples using RT-PCR,” Journal of clinical chemistry and laboratory medicine, vol. 4, 2021. [18] T. M. Id, O. Nickel, A. Aximu-petri et al., “A direct RT-qPCR approach to test large numbers of individuals for SARS-CoV- 2,” PLoS One, vol. 15, no. 12, pp. 1–10, Article ID e0244824, 2020. [19] R. Santibáñez, Á. Luraschi, C. Barrera-avalos et al., “The comparison of three real-time PCR kits for SARS-CoV-2 diagnosis reveals discrepancies on the identification of pos- itive COVID-19 cases and dispersion on the values obtained for the detection of SARS-CoV-2 variants,” 2021, https:// www.medrxiv.org/content/10.1101/2021.07.13.21260484v1. [20] M. Sidstedt, P. Rådström, and J. Hedman, “PCR inhibition in qPCR, dPCR and MPS-mechanisms and solutions,” Analyt- ical and Bioanalytical Chemistry, vol. 412, no. 9, pp. 2009– 2023, 2020. [21] A. E. Platts, G. D. Johnson, A. K. Linnemann, and A. Stephen, “Real-time PCR quantification using A variable reaction ef- ficiency model,” Analytical Biochemistry, vol. 380, no. 2, pp. 315–322, 2009. [22] Y. Araf, F. Akter, Y. D. Tang et al., “Omicron variant of SARS- CoV-2: genomics, transmissibility, and responses to current COVID-19 vaccines,” Journal of Medical Virology, vol. 94, no. 5, pp. 1825–1832, 2022.