- Research note
- Open Access
Lessons from a comparison of immuno-chromatographic and chemiluminescent micro-particle immunoassay in the diagnosis of syphilis
BMC Research Notesvolume 11, Article number: 701 (2018)
To synthesize lessons from comparison of results obtained from the immuno-chromatographic SD Bioline testing method and the chemiluminescent micro-particle immunoassay Architect in the diagnosis of syphilis at Livingstone Central hospital laboratory.
The specificity and sensitivity of SD Bioline syphilis 3.0 against the chemiluminescent immunoassay using the Architect syphilis Treponema pallidum (TP) was 85.3% and 91.3% respectively with substantial agreement between the two test methods (88%, ĸ = 0.76; p < 0.0005). We recommend further comprehensive study with a larger sample size and clinical details to ascertain the validity of our findings. We also recommend using a non-treponemal test with the current treponemal tests being used to aid diagnosis.
The laboratory serologic diagnosis of syphilis can be complicated  especially with limited diagnostic tools owing to the natural course of the infection which is characterized by periods with and without clinical manifestations . Furthermore, the disease stages show variable sensitivity and specificity for some test methods in detecting Treponemal pallidum (TP) antibodies [1, 2]. It is therefore mandatory to interpret laboratory results on the basis of patient history and physical examination findings . The traditional screening of syphilis recommended by the Center for Disease Control and Prevention (CDC) is the use of a nontreponemal test, such as rapid plasma reagin (RPR) . However, the use of chromatographic and chemiluminescence immunoassay (CIA) treponemal test methods for screening syphilis is empirical and gaining acceptability . It is important and mandatory to evaluate the diagnostic performance of laboratory test methods in every laboratory so as to instill confidence in the clinician and patient regarding the reliability of laboratory results . Several reports have imaged that have compared the SD Bioline syphilis test with other treponemal methods, stressing the importance of conducting pre-clinical and -laboratory performance and applicability of all testing methods for syphilis diagnosis [6, 7]. At Livingstone Central hospital, which is based in the Southern Province of Zambia, we use the SD Bioline syphilis test only (recently introduced), as an index test to aid in the diagnosis of syphilis for patients while Architect syphilis TP is specifically used for screening blood donors. The disease burden of syphilis in our region is unknown owing to paucity of studies. We decided to perform an exploratory study whose objective was to compare the SD Bioline syphilis test performance against the Architect syphilis TP. This was prodded by the clinicians’ concern for the high TP positivity rate recorded for the months, March to May, 2017 using the SD Bioline syphilis test.
This was a cross-sectional study conducted at the Livingstone Central hospital laboratory research section.
Setting and sample size
Data collection was planned after the index test but before the reference standard was performed. We collected data for the SD Bioline syphilis 3.0 positivity rate for the months of March, April and May, from the laboratory information system as shown in Table 1.
We then randomly selected 100 samples from the syphilis suspected specimens that tested reactive and non-reactive (1:1) on SD Bioline which were compared with the reference standard, Architect syphilis TP. We used the available electronic laboratory information system to confirm the clinical suspicion for syphilis. Samples had been collected through venipuncture into plain glass vacutainers containing no clot activators, anticoagulants, preservatives or separator materials which and were stored at 2–8 °C for a minimum of 45 min before centrifugation to extract serum for testing.
The SD Bioline syphilis 3.0 (Standard diagnostics, INC. Korea) is a solid phase immunochromatographic assay for the qualitative detection of antibodies of all isotypes [Immunoglobulin G (IgG), Immunoglobulin M (IgM), Immunoglobulin A (IgA)] against TP. It is pre-coated with recombinant TP antigens that bind to the TP antibodies, if present in the patient sample to produce a visible line. In this study, SD Bioline syphilis 3.0 is the index test.
The Architect syphilis TP (Abbott GmbH & Co. KG, Germany) is a two-step chemiluminescent microparticle immunoassay (CIA or CMIA) for the qualitative detection of IgM and IgG antibodies to TP intended to aid in the diagnosis of syphilis infection and as a screening test in blood donors to prevent transmission of TP to recipients. The method and principle of analysis are described in detail elsewhere . Architect syphilis TP was used as our reference standard.
Since the sample size was not calculated based on any hypotheses but selected based on the availability of reagents, we most likely introduced selection bias by selecting 50 positives and 50 negative samples. However, the selection of these samples was random to minimize selection bias.
Descriptive statistics was used to summarize data while Chi square or fishers exact test compared results for the two diagnostic testing methods. A concordance test assessed the degree of agreement between the two tests and receiver operating curve (ROC) to evaluate the benefit of using the index test.
We used the Standards for Reporting of Diagnostic Accuracy Studies (STARD) .
Total number of syphilis suspected samples collected and tested between March and May were 781 (Table 1) of which 116 (14.9%) tested positive with SD Bioline syphilis rapid kit. There was a drop, in positivity rate for the month of May compared to the first 2 months.
50 samples that tested negative and positive respectively, on SD Bioline syphilis rapid kit were retested on Architect and the diagnostic tests then evaluated (Table 2).
As shown in Table 2, the false positive rate for SD Bioline was 14.8% with a sensitivity of 91.3% while the false negative rate was at 8.7% with a specificity of 85.2%. There was a significant difference between the two tests p < 0.0005. Positive and negative likelihood ratios of 6.2 and 0.1 were reported respectively (data not shown).
The positive and negative predictive values were 84% and 93% at prevalence of 54% (expected). However, using actual point prevalence of 15.3%, the positive and negative predictive values were 53.7% and 46.9%.
Roc curve for SD Bioline against the Architect test
We performed a receiver operator curve (ROC) analysis (graph not shown) yielding an area under the curve of 0.88, (standard error, 0.032; 95% confidence interval [0.81, 0.95] p < 0.001).
We also performed a concordance test (Table 3 below) to assess the degree of agreement between the two tests.
The specificity (85.3%) and sensitivity (91.3%) of SD Bioline against the Architect in our study were significantly low especially that SD Bioline and the Architect sensitivity and specificity compared to several non-treponemal tests can go as low as 95% and 98% respectively . However, in another study, the sensitivity of SD Bioline was comparable (92%) , moreover a study conducted in Tanzania found the test to be 79% sensitive and 96% specific .
The marginal distributions of test ratings did not indicate prevalence or bias problems, suggesting that Cohen’s kappa was an appropriate index of inter-rater reliability, so we performed an inter-rater reliability test using Cohen’s kappa (ĸ) to determine if there was agreement between the two diagnostic tests in our study. The results (88%, ĸ = 0.76; p < 0.0005) obtained indicate substantial agreement . An almost perfect agreement (ĸ = 0.81–1.00) is the best every lab would want to achieve. However, the difference could be explained by the technicality involved in the performance of the two tests, the architect being fully automated, hence not prone to a lot of errors and expert interpretation as compared to the immunochromatographic method used in the SD Bioline syphilis 3.0 which is prone to pre- and -analytical errors. Nonetheless, the scope of our study could not allow us to determine the real difference.
The clinical implications of our results in the diagnosis of syphilis can only be assessed with additional clinical information. It remains a standard that when interpreting laboratory results for syphilis, we must consider the clinical stage aided by patient history and physical examination. Without these, erroneous diagnosis and bad management decisions is the consequence [4, 14].
The laboratory implication from the results we obtained warrants a follow up study to investigate and determine the efficiency and application of the chemiluminescence immunoassay and immunochromatographic assay in syphilis diagnosis as it is difficult to prove that positive or negative results obtained using these platforms are necessarily false [1, 15] hence the need for patient medical history including previous treatment for syphilis, and history of sexual risk factors [1, 2]. Additionally, we recommend adding a non-treponemal test like RPR and possibly implement reverse algorithm in syphilis diagnosis. The use of both non-treponemal and treponemal tests for diagnosis is warrantable for laboratories . It also means that the laboratory needs to obtain complete clinical information for accurate interpretation of results to guide the clinician in managing the patients appropriately . The test methods involved in our study are treponemal tests. It is well known that these tests can remain positive almost for life in previously infected individuals regardless of treatment  while antibodies measured in non-treponemal tests do disappear after some years . It is therefore, vital, to use both treponemal and non-treponemal tests, to differentiate previous infection from current . However, we also know that a confirmed positive serological test result does not differentiate disease stage and we may not tell if the infection is past or current in some cases . Given the limitations of TP tests, we can still use them to make accurate diagnosis aided by clear patient history and evidence from physical examination .
For primary syphilis, the recommended tests to perform include direct examination, non-treponemal tests, and treponemal tests and since there is no single optimal test, a combination of these is required to aid in the diagnosis [1, 2]. Detection of TP in lesions is definitive evidence of syphilis but a negative result does not rule out syphilis . PCR-based tests have a high reliability [1, 18]. In the first 2–3 weeks, serology may not be positive in some cases, and in early primary syphilis, treponemal tests maybe recommended . When there is evidence of a genital ulcer at physical examination and a non-treponemal test is reactive, this may not necessarily indicate primary syphilis  and in such cases a repeat test over a two to 12 week period may be required to rule out syphilis . In secondary syphilis, the use of direct physical examinations, and use of both non-treponemal tests and treponemal tests are still recommended [17, 19]. In persons with a history of syphilis, a fourfold increase in titer when using quantitative methods provides presumptive diagnosis of secondary syphilis . In latent syphilis, as in all stages, it is recommended to use both treponemal and non-treponemal tests . However, non-treponemal tests are reactive in early latent syphilis though the sensitivity declines over time [1, 17]. Prevalence information of syphilis in a population is cardinal for interpretation, as in low prevalence populations, false-positive results may be common with both treponemal and non-treponemal tests .
As a general rule, a reactive treponemal test in the absence of a reactive non-treponemal test may require confirmation . For tertiary syphilis, non-treponemal and treponemal tests are recommended [15, 17, 19]. In this stage approximately 30% may not be reactive using non-treponemal tests, while treponemal tests are reactive almost always . Therefore, we think treponemal tests should always be considered .
The specificity and sensitivity of the immunochromatographic assay, SD Bioline syphilis 3.0 against the chemiluminescent immunoassay, Architect syphilis TP was 85.3% and 91.3% respectively with substantial agreement between the two test methods. However, we recommend further comprehensive study with a larger sample size and clinical details to ascertain the validity of our findings. We recommend using both treponemal and non-treponemal tests to aid diagnosis of syphilis.
Because we did not have access to clinical information, our study had limited variables needed for a comprehensive comparison. For example, we needed to compare our results to clinical findings and as well, compare with a non-treponemal test in order to provide an accurate conclusion on the implications of the false positive and false negative rates.
Our cohort was not well characterized to draw useful conclusions with clinical and laboratory implications. However, we want to use the lessons from our findings to propose for a well characterized follow-up study.
False-positive reactions can occur with treponemal tests but this is less common than with nontreponemal tests . Examples of conditions were false positive tests are reported include but not limited to advancing age, brucellosis, cirrhosis, hyperglobulinemia, malaria, pregnancy, systemic lupus erythematosus et cetera . Our study lacked information on these factors that have implications on the interpretation of TP laboratory results.
Centres for Disease Control and Prevention
- CIA OR CMIA:
receiver operating curve
rapid plasma reagin
Standards for Reporting of Diagnostic Accuracy Studies
Morshed MG, Singh AE. Recent trends in the serologic diagnosis of syphilis. Clin Vaccine Immunol. 2015;22:137–47.
Centers for Disease Control and Prevention (CDC). Discordant results from reverse sequence syphilis screening–five laboratories, United States, 2006–2010. MMWR Morb Mortal Wkly Rep. 2011;60:133–7.
Jonckheere S, Berth M, Esbroeck MV, Blomme S, Lagrou K, Padalko E. Evaluation of different confirmatory algorithms using seven treponemal tests on Architect Syphilis TP-positive/RPR-negative sera. Eur J Clin Microbiol Infect Dis. 2015;34:2041–8.
Workowski KA, Berman S, Centers for Disease Control and Prevention (CDC). Sexually transmitted diseases treatment guidelines, 2010. MMWR Recomm Rep. 2010;59:1–110.
Binnicker MJ. Which algorithm should be used to screen for syphilis? Curr Opin Infect Dis. 2012;25:79–85.
Lee J-H, Lim CS, Lee M-G, Kim H-S. Evaluation of a rapid immunochromatographic treponemal antibody test comparing the Treponema pallidum particle agglutination assay. J Clin Lab Anal. 2015;29:383–6.
Binnicker MJ, Jespersen DJ, Rollins LO. Treponema-specific tests for serodiagnosis of syphilis: comparative evaluation of seven assays. J Clin Microbiol. 2011;49:1313–7.
Park Y, Park Y, Joo SY, Park MH, Kim H-S. Evaluation of a fully automated treponemal test and comparison with conventional VDRL and FTA-ABS tests. Am J Clin Pathol. 2011;136:705–10.
Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6:e012799.
Seña AC, White BL, Sparling PF. Novel Treponema pallidum serologic tests: a paradigm shift in syphilis screening for the 21st century. Clin Infect Dis. 2010;51:700–8.
Mehra B, Bhattar S, Saxena S, Rawat D, Bhalla P. Evaluation of SD BIOLINE syphilis 3.0 for rapid diagnosis of syphilis: report from a regional sexually transmitted infection reference laboratory in North India. J Lab Physicians. 2016;8:36–40.
Aboud S, Lyamuya EF, Sufi J, Haule E. Comparison of teh performance of SD Bioline Syphilis 30 assay with the RPR test for the syphilis screening in Dar es Salaam. Tanzania. Tanzania Med J. 2006;21:6–7.
McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22:276–82.
Larsen SA, Steiner BM, Rudolph AH. Laboratory diagnosis and interpretation of tests for syphilis. Clin Microbiol Rev. 1995;8:1–21.
Mattei PL, Beachkofsky TM, Gilson RT, Wisco OJ. Syphilis: a reemerging infection. Am Fam Physician. 2012;86:433–40.
Soreng K, Levy R, Fakile Y. Serologic testing for syphilis: benefits and challenges of a reverse algorithm. Clin Microbiol Newslett. 2014;36:195–202.
Ratnam S. The laboratory diagnosis of syphilis. Can J Infect Dis Med Microbiol. 2005;16:45–51.
Tong M-L, Lin L-R, Liu G-L, Zhang H-L, Zeng Y-L, Zheng W-H, et al. Factors associated with serological cure and the serofast state of HIV-negative patients with primary, secondary, latent, and tertiary syphilis. PLoS ONE. 2013. https://doi.org/10.1371/journal.pone.0070102.
Brown DL, Frank JE. Diagnosis and management of syphilis. Am Fam Physician. 2003;68:283–90.
SKM and BMH conceived the study. SKM, MM, and BMH conducted the data collection. SKM, MM, AK, YC, PT, BMH contributed to data analysis and led the writing of the manuscript. SKM is the principal investigator and guarantor. BMH is the senior author. All authors read and approved the final manuscript.
We are very grateful to all Laboratory personnel and the office of the Senior Medical Superintendent at Livingstone Central hospital for their support and enthusiasm for research projects.
The authors declare that they have no competing interests.
Availability of data and materials
All data generated or analysed during this study are included in this published article. For other data, these may be requested through the corresponding author.
Consent to publish
Ethics approval and consent to participate
This study is part of an approved larger study being conducted at Livingstone Central hospital. Ethical approval was obtained from the University of Zambia Biomedical Research Ethics Committee (Assurance No. FWA00000338 IRB00001131 of IORG0000774) and permission to conduct the study at Livingstone Central hospital was granted by the Hospital Administration.
This protocol has not received any grant from any funding agency in the public, commercial or not-for-profit sectors.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.