Diagnostic accuracy of Panbio rapid antigen tests on oropharyngeal swabs for detection of SARS-CoV-2
Abstract
Background
Antigen-detecting rapid diagnostic tests (Ag-RDTs) for the detection of SARS-CoV-2 offer new opportunities for testing in the context of the COVID-19 pandemic. Nasopharyngeal swabs (NPS) are the reference sample type, but oropharyngeal swabs (OPS) may be a more acceptable sample type in some patients.
Methods
We conducted a prospective study in a single screening center to assess the diagnostic performance of the Panbit COVID-19 Ag Rapid Test (Abbott) on OPS compared with reverse-transcription quantitative PCR (RT-qPCR) using NPS during the second pandemic wave in Switzerland.
Results
402 outpatients were enrolled in a COVID-19 screening center, of whom 168 (41.8%) had a positive RT-qPCR test. The oropharyngeal Ag-RDT clinical sensitivity compared to nasopharyngeal RT-qPCR was 81% (95%CI: 74.2–86.6). Two false positives were noted out of the 234 RT-qPCR negative individuals, which resulted in clinical specificity of 99.1% (95%CI: 96.9–99.9) for the Ag-RDT. For cycle threshold values ≤ 26.7 (≥ 1E6 SARS-CoV-2 genomes copies/mL, a presumed cut-off for the infectious virus), 96.3% sensitivity (95%CI: 90.7–99.0%) was obtained with the Ag-RDT using OPS.
Interpretation
Based on our findings, the diagnostic performance of the Panbio Covid-19 RDT with OPS samples, if taken by a trained person and high requirements regarding quality of the specimen, meet the criteria required by the WHO for Ag-RDTs (sensitivity ≥80% and specificity ≥97%) in a high incidence setting in symptomatic individuals.
Introduction
The SARS-CoV-2 pandemic has killed millions of people worldwide [1]. Large scale testing allows for identification and isolation of infected individuals, and quarantining contacts, thus limiting community transmission. Currently, SARS-CoV-2 RT-qPCR performed on nasopharyngeal swabs (NPS) is the gold-standard diagnostic test.
While displaying excellent sensitivity and specificity, RT-qPCR is costly, subject to reagent and material shortages during pandemics, and requires experienced personnel and complex infrastructure. Antigen rapid diagnostic tests (Ag-RDTs) are easy to use, more affordable, decentralizable, and provide quick results; offering an attractive alternative to RT-qPCR during pandemics. Their drawbacks are mainly reduced sensitivity relative to RT-qPCR.
The World Health Organization (WHO) considers a sensitivity ≥80% and a specificity ≥97% as acceptable performance for SARS-CoV-2 Ag-RDTs [2]. Currently, only validations of Ag-RDTs performed with NPS have shown satisfactory results [3–12], and no studies have evaluated Ag-RDTs using oropharyngeal swabs (OPS). OPS sampling could be a useful alternative to NPS sampling, as seen with RT-qPCR tests [13, 14]. Here we describe a prospective study comparing the diagnostic performances of an Ag-RDT using OPS with RT-qPCR using NPS for detection of SARS-CoV-2.
Methods
Ethics
The study was approved by the cantonal ethics committee (Commission Cantonale d’Ethique de la Recherche, CCER, Geneva, Nr. 2020–02323). All enrolled patients provided written informed consent form.
Setting, study design and participants
The study took place from November 3 to 19, 2020, at an outpatient SARS-CoV-2 screening site at the Geneva University Hospitals during the second pandemic wave in Geneva, with very high incidence during the testing period of >2000/100.000 per 14 days at the start of the study. The majority of patients had symptoms compatible with SARS-CoV-2 infection, and a small proportion were asymptomatic contacts. All participants were ≥16 years old with suspected SARS-CoV-2 infection according to the local governmental testing criteria. This included suggestive symptoms for COVID-19 and/or recent exposure to a SARS-CoV-2 positive person. Asymptomatic individuals were included if they were notified by the Swiss COVID-19 app about a contact, offering the option to get tested on day 5 after contact, or if they received a notification from local health authorities (screening of people with high-risk exposure in a cluster).
Sampling procedure
Participants were swabbed twice: one NPS performed by a nurse at the screening site, for the reference RT-qPCR; and an OPS done by an experienced doctor, using a tongue depressor in a well-lit environment with an emphasis on consistent technique, for the Ag-RDT.
A pilot study tested 28 RT-qPCR-positive individuals without ensuring the back of the oropharynx was reached, yielding only 11 Ag-RDT positives (S1 Table). Therefore, patients were only included if the posterior wall of the oropharynx could be reached.
Data collection
The clinical data collected for each patient was: duration of any symptoms when samples were collected, potential close contact with a positive person within 14 days, symptoms (rhinorrhea, odynophagia, myalgia, chills, dry vs productive cough, hemoptysis, fever, anosmia, ageusia, gastrointestinal symptoms, asthenia, dyspnea, chest pain and headache), and comorbidities (hypertension, cardiovascular disease, chronic lung disease, diabetes, chronic renal failure, active cancer, severe immunosuppression, pregnancy and obesity (BMI> 40 kg/m2)).
Click Here – Panbio Nasopharyngeal Test
Ag-RDT procedure
Aside from the sample type, the Panbio (Abbott) Ag-RDT device was run and read by a biologist according to the manufacturer’s protocol on site in the testing centre. All samples were tested within the time frame given by the manufacturer. Equivocal results were read by a second healthcare worker. No invalid Ag-RDT results occurred.
SARS-CoV-2 detection by RT-qPCR
All NPS samples were analyzed using the Cobas SARS-CoV-2 RT-PCR assay on the 6800 system (Roche), targeting ORF1 and the E-gene. To convert Ct values into RNA copy numbers, we tested serial of dilutions of cultured SARS-CoV-2, which were quantified by using in vitro transcribed RNA obtained from the European Virus Archive [15] by using the Charité E gene assay [16]. Cycle-threshold (Ct) values for the E-gene were converted into viral load (VL) with the following formula: log10 SARS-CoV-2 copies/mL = (Ct-44.5)/-3.3372.
Statistical methods
Ag-RDT sensitivity and specificity was determined relative to RT-PCR. With a positivity rate of 37.5%, and an Ag-RDT sensitivity/specificity of 85%/95%, a sample size of 400 could determine sensitivity and specificity with confidence intervals (CI) of 79.3–90.7% and 92.3–97.7%, respectively. Fischer’s exact test was used to compare Ag-RDT sensitivity by Ct values (above/below 26.7). All analyses were performed using STATA version intercooled 16 (Stata Corp., College Station, TX, USA). Statistical significance was defined as p<0.05 (two-sided).
Results
During the study period, 402 participants were included. Eight patients were excluded either because the throat was insufficiently accessible or because consent was withdrawn. The participants’ socio-demographic characteristics are summarized in S2 Table. 168 participants (41.8%) were RT-qPCR-positive with a mean Ct value of 24.97 (SD ±5.63, 3.3E6 SARS-CoV-2 copies/mL equivalent) for 166 RT-PCR analyses. Two specimens, positive for the ORF1 target at a high CT values but negative for the E-gene, were interpreted as positive in the analysis for sensitivity and specificity but excluded from Fig 1; both were Ag-RDT negative.
Discussion
There are over 10 clinical studies (8 preprints, 2 published) evaluating the performance of the Panbio™ Ag-RDT [3–12] using only manufacturer recommended NPS. Those studies, with over 6000 subjects, have reported sensitivity and specificity ranges of 71.4%-91.7% and 94.9%-100%, respectively. Considering only Ct values <30 yielded test sensitivities from 87.7% to 97.8% [3, 6–9]. Similarly, samples from <5 DPOS yielded a sensitivity between 77.2 and 94.8% [3, 5, 6, 8, 9].
For some patients in whom NPS sampling is not feasible, OPS could be an attractive alternative, thus OPS sample validation is critical. This is the first publication investigating the diagnostic accuracy of the Panbio Ag-RDT using OPS. Our results meet show this off-label use to still meets the WHO targets of ≥80% sensitivity and ≥97% specificity. Interestingly, while ensuring high OPS sample quality, we obtained similar results to our previously published NPS evaluation, with no statistical difference in clinical sensitivity and specificity [3].
We previously demonstrated similar clinical and analytical sensitivities between NPS and OPS sampling for SARS-CoV-2 detection using RT-qPCR [14]. However, some studies showed reduced [13] sensitivity and lower rates of virus isolation in cell culture for OPS when compared to NPS, suggesting a risk of reduced Ag-RDT sensitivity when using OPS [17].
Our present study shows that despite the use of OPS, contrary to manufacturer recommendations, we obtained highly reliable results, in a scenario of high incidence and thus high positive-test rates (41.8% in our study population), and under the requirement that the sample was taken by a trained person with high requirements regarding the quality of the specimen. Similar to studies on NPS specimens, the highest sensitivity was seen in the early symptomatic period as well as for patients presenting with high nasopharyngeal VL.
Although a few positive samples with lower Ct values were missed, the majority of false-negative samples were from individuals with high Ct values (≥30), corresponding to a low VL below the presumed cut-off for infectious virus (Ct ≤ 26.7 in our hands or 1E6 SARS-CoV-2 RNA copies/mL). It was shown previously that a VL above 1E6 SARS-CoV-2 RNA copies/ml can serve as a correlate for contagiousness, and presence of culturable SARS-CoV-2 is unlikely to be found if VL are below this cut-off [17–20]. These results suggest that these individuals are not likely to be contagious and that these false-negative Ag-RDT results should not result in further transmission.
In conclusion, the use of Ag-RDTs with OPS might prove to be an acceptable alternative to NPS, and could increase test acceptance for selected groups such as children.